Prototype Links

Hi guys! I'm new to XD, and I'm designing a prototype that I had a question about...

Let's say I have a "back" button, BUT I want the button to link back to the page the user came from even though you can get to that page from other links. Ideally, I would want each interaction with that button to take them back to the specific page they came from.

example: I see a picture of a movie I want to see, I click it, and it takes me to the movie page. If I click the back button I want to go back to that page with the picture. BUT, I can also get to the movie page from a search, and if I got to the movie page from a search, if I click the back button I want to go back to the search page, not the menu with the image from earlier. 

Is there any way to do that, that's not making eleventy billion pages to link separately? am I making any sense lol 

Thanks for you help!