I usually work the project on my laptop at home, but these day I hope I'm able to work on my office computer when I have free time during workday. So is there any way that I can save the project file in the creative cloud and then open the file up in another computer with creative cloud? And do I also need to backup all my video files I have used in the project?
Saving anything to your Creative Cloud folder is just like using Dropbox, OneDrive, or other type of cloud file storage service. In your user folder (both on Windows and macOS) there is a folder called "Creative Cloud Files." You can put your entire project folder in there and it will sync with another machine that you're logged into the Creative Cloud Desktop application with the same Adobe ID.
And yes, you need all assets you're using on one machine to be on the other. One way to keep things simple is to have a duplicate of your project structure on both machines, then just sync your .pproj file back and forth. This will only work if you're not adding any new assets to your project or changing the folder structure on your disk. If you have enough Creative Cloud storage for this project then you can leave the entire project folder in your Creative Cloud folder and work from there on both machines.
If I'd like to duplicate my asset file such as video on both machine, do I need to set the file path in the same place where it was in the first machine? Or I don't have to do this step which Premiere Pro could find the file itself in the new machine?
If you have all of your assets in the same folder structure and you move that entire folder, Premiere should detect the assets again on a new machine.
My project folder structure looks something like this. When I move the entire top-level folder, everything relinks with no issue, regardless of the machine. And if you do have to relink, you should only have to relink one clip and Premiere will find the rest since the location relative to the project file is the same.