Colourising a video frame by frame

Here's the problem. I colourise old photos using curves adjustment layers. Most images will have around 10 layers, each with a mask. Works great for photos. I wanna do it with some old black and white footage, see how that goes.

The problem is - extracting frames from a video to layers isn't really working for me, because I end up with a project with around 150 layers, each of which will have around 10 adjustment layers. It's becoming just a bit of a mess. Is there a way to extract frames from a clip but not have them treated as layers? Seems like there's a more elegant way to do it.