This depends on what you mean by restrict access. You can (1) hide your course from students, (2) change the enrollment options to restrict access to everyone that you don't manually enroll, or (3) add a password so that only those that know the password can enroll.
Let's take a look at how you can go about accomplishing each of these.
You can hide a course in which you are the designated teacher by going to Edit settings from the main course page.
Under General on the Edit course settings page, toggle the Course visibility from Show to Hide. This will make your course invisible to anyone without a course creator role, in other words, it will hide it from all students, even the students that are currently enrolled.
You can restrict access to the course to anyone that isn't already enrolled in a couple of ways. You can add a password to the course. This will throw up an enter password screen to anyone that that attempted to enter the course that isn't already enrolled.
To add a password, click Participants and then on the Participants screen, click the small gear icon above the Enroll users button.
Select Self enrollment (Student) in the menu like is shown below.
On the Self enrollment screen, enter a password beside Enrollment key.
At the bottom of the page, click Save changes. This will force anyone that attempts to enter your course to enter a password. If they don't know they password, they won't have access to your course or its content.
Alternatively, you can also just turn off self enrollment altogether which means that no one will be given an option to enroll and it will be up to you to manually enroll any and all users that have access to the course.
To turn off self enrollment, go to Participants, click the small gear/settings icon, and select Enrollment methods in the dropdown menu.
To turn off self enrollment, click the small eye icon in the Edit column across from Self enrollment (Student). This hides/disables the option for anyone to self enroll / enroll themselves in your course.