Unless you want to restrict everyone or restrict no one, you will need to classify one or more groups as restricted. You may wish to create a group rather than rely upon the existing groups; for that, use your system's recommended program to edit /etc/group and create a group, such as "restrict". In this group, place all users in your /etc/passwd file that you want to be restricted.
For virtual users, you will most likely want to restrict them; in fact, it is the default, unless you change the u-always-restrict-virtual-users configuration option. The reason for that default behavior is more of a sanity check, since that is what most sites prefer and it is possible to forget to place the virtual user in a restricted group, leaving your server's entire directory tree available for perusal.