How exactly does Coldfusion serve files with Apache?

Hi @10xpg , interesting question.
What you’re seeing is normal, intentional ColdFusion behaviour. But it is rarely noticed, unless when things are set up exactly the way you did. Your conclusion ("ColdFusion is checking both webroots and prioritizing its own") is close..

Before explaining, let me start with a correction. One part of your conclusion is correct and one part is incorrect. 

• Incorrect: ColdFusion is checking both webroots. 
• Correct: ColdFusion is prioritizing its own web server.

When the page  http://250.2.36.996:8080/sampler/app/index.cfm is requested what actually happens is as follows:

1.  The request is first handled by the web server, in this case, Apache. Apache doesn't actually "execute" the file. Instead, it looks in /var/www/html/, confirms that the file exists, hands the URI (/sampler/app/index.cfm) to ColdFusion via the ColdFusion connector (mod_jk).
2.  ColdFusion (actually the Tomcat application server) receives the URI. By default, when ColdFusion receives a URI it first checks its own native "Document Root".  That is the prioritizing part.. This root is /opt/coldfusion/cfusion/wwwroot. There ColdFusion finds /sampler/app/index.cfm. Because this is its "native" home, it assumes this is the intended file and executes it. That is why you see "hi".
3.  When the file does not exist in its native home, ColdFusion then falls back to the physical path mapping provided by the connector. That mapping (provided by the Apache CGI environment) points to Apache’s docroot. So ColdFusion finds the file in /var/www/html/ and executes it. That is why you see "hello".
   
   There are several ways to get only Apache to serve the requests. I'll share with you what I consider to be best-practice. In any case, it is what I use. Besides, you have already experimented with it:
       -  Ensure that your application files never reside in the ColdFusion internal webroot. So delete (or move) the sampler folder from /opt/coldfusion/cfusion/wwwroot.

   As a result, ColdFusion will find no match in its internal root and will always fall back to the file Apache is pointing to.

10x, the behavior you describe is not new--though it does surprise most and can cause confusion. It's an essentially undocumented feature of the cf web server configuration (wsconfig) tool that was used to connect your apache to cf. And though you didn't ask if it's possible, I'm not aware of any way to disable that functionality (of how it looks in the cf wwwroot for a file first). 

Instead you're asking "how to actually get the file apache is serving using the <IP:PORT>". So please clarify: are you asking how to get apache to serve the content without that 8080 port? If so, that's just about apache configuration (having nothing at all to do with CF). You can find that easily online, or someone here may chime in with details. 

But to be clear, if you simply remove that duplicated content from the cf wwwroot folder (files with same names in same folders as /var/www/html), that will "solve" the problem. 

Let us know what you decide.