I'm not sure I understand what a backup catalog is. Does that mean it backs up all my photos along with all the information from PE (like tags, face recognition etc.)?
Yes, it does backup both the media files and the data stored in your catalog.
The resulting backup folder is not a simple copy and it can only be used to restore the media and the catalog where you want.
If I just copy my photo folders to another drive will the PE information (tags, face recognition, groups etc.) not go along with it?
The PE information (in the catalog) will be lost. However, part of that info (tags, captions, ratings...) can be stored in the media files themselves. If you import them in another catalog or another software, they can be retrieved. However you'll lose albums, stacks, version sets...
Alternatively, is it possible to save only the PE information, without having to copy the photos themselves (to save space)?
You can copy or move the catalog folder. You find it from the menu Help >> System information. But even if you have a copy or backup of the media files from another system, you won't be able to keep the links in the catalog to the new location. Only the backup and restore process of the organizer can do that.