Skip to main content
AlexCraig
Inspiring
February 4, 2024
Answered

How to connect solr to MS Sql Server in CF 2023

  • February 4, 2024
  • 1 reply
  • 1383 views

I am creating a new thread for this question so someone else in the community might be able to easily find it.

I have searched far and wide and have discovered that CF 2023 supports a solr function that did not exist in CF 4.51.

I finally found this on Quora.

******************************

You can connect Solr with MS SQL Server by using the Solr Data Import Handler (DIH) and configuring it to fetch data from SQL Server. Here are the basic steps:

  1. Install and configure Solr on your server.
  2. Download the Microsoft JDBC driver for SQL Server.
  3. Copy the JDBC driver JAR file to the Solr server.
  4. Create a data source name (DSN) for the SQL Server database using the JDBC driver.
  5. Configure the Solr DIH to use the SQL Server DSN as a data source.
  6. Define the SQL queries that will fetch the data from the SQL Server database.
  7. Run a full import to fetch the data from SQL Server into Solr.

Once the data is imported, you can use Solr to index and search the data.

**********************************************

I have already accomplished steps 1 thru 4.  I have not been able to find any info on how to accomplish steps 5, 6 & 7.

I presume I must then need to learn how to code the parallel /sql request handler.

Any help would certainly be appreciated.  Thanks in advance.

This topic has been closed for replies.
Correct answer AlexCraig

The very phrase you quote--from the message in the application.log--does indeed show "additional info", at least in that you'd not shown previously the error message saying, "Could not find QUERY [jobs]". And the exception log indicates the same error. 

 

So help us out: had you seen that before and not told us? Or did you miss it being in what you've just posted? Either is OK. Just help us understand.

 

More to the point, if you may feel it "doesn't tell me anything meaningful ", then I'll share first that I see now how that example code (from the Adobe docs) happened to use a cfquery called "getJobData" but then referred in the cfindex to "jobs".  That seems a mistake, easily corrected by them or you: did you perhaps just use that EXACT code, changing only the datasource value and sql statement? If so, then just change either that cfquery name value or the cfoutput query value, so that they match. 

 

Second, if you may respond with more questions, it may help for you to clarify something: are you perhaps the owner of this web site (with this code from cf 4.51, from 25 years ago), who is not really a developer otherwise? And are you perhaps "just needing to get this web site running on a modem cf version", for whatever reasons?  If all that's true, knowing it may better help us in guiding you. We do tend to get questions here from developers rather than others, and we may leave out details or presume something's more apparent than maybe it woukd be for someone in that situation. 

 

But maybe the suggestions before that question will get you over this hump and you'll be all set. 


To answer your first question, I'd seen the "Could not find QUERY [jobs]" a number of times as I was trying to perfect the code as it was choking on earlier lines in the code as I worked my way through getting it correct and thought it was a generic message of little consequence.  I apologize for my ignorance.

 

The example in adobe docs was [query="getCourses"].  I had originally changed it to "getJobData" and then changed it to "jobs" because our current cfindex code scheme in 4.51 uses "jobs".  Of course, I am trying to keep as much of the nomenclature in 2023 the same as that used 4.51  in possible.

 

To answer your next question, yes I own the website.  As I believe I mentioned in a previous thread, the 4.51 development was done by a young man I placed as a CF developer who was touted by his boss as a "whiz kid" after he worked for him for a couple of years.  The "whiz kid" did the inital 4.51 coding for us as a side job and his work was exceptional.  Given the size of our system, we only found 3 bugs in it after pounding on it for a full year.

 As to myself, a long, long time ago, in a different universe I did a fair mount of COBOL programming on a Series 1 IBM mini-computer.  Bottom line, I learned enough about syntax & logic to be able to handle "maintenance programming" on our 4.51 system.  I could read the "whiz kids" code well enough to modify it as needed without issue.  Bottom, bottom line writing new CF code from scratch is a challenge.  I can learn it, but it requires a fair amount of time & effort.

 

With all that said, your last suggestion concerning the code did the trick.  Changing the input & output query names to match (jobs); commenting out the "custom1" field solved my problem. I now have a collection with 5500+ documents in it.  Thanks for your patience & help Charlie.  Apparently, "old dogs can learn new tricks.

For anyone who stumbles across this thread who needs a template from which to create a collection from a database table, this one worked.

**************************

<!---EXAMPLE #3: Index a QUERY (type = "custom") using custom1. ------------>
<!--- Retrieve data from the table. --->
<cfquery name="jobs" datasource="TEN">
select * from tblJobOrders
</cfquery>
<!--- Update the collection with the above query results. --->
<!--- key is JobOrderNum in the tblJobOrders table. ---->
<!--- body specifies the columns to be indexed for searching. --->
<cfindex
query="jobs"
collection="TPI_Jobs"
action="Update"
type="Custom"
key="JobOrderNum"
title="title"
body="JobOrderNum,JobTitle,JobReq1,JobReq2,JobReq3,JobReq4,JobPref1,JobPref2,JobPref3,JobPref4,JobReqNotes"
>
<h2>Indexing Complete</h2>

***************************************

1 reply

Charlie Arehart
Community Expert
February 4, 2024

Before going any further, the solr dih feature is only supported in cf enterprise and not cf standard.  Which are you using?

 

As for learning more on the dih feature, it was covered a lot back in 2012 when it's support was introduced with cf10, iirc. I'm on a phone as I write, so gathering up resources would be a bit challenging right now.

 

But in hearing that it's indeed a solr feature (not a cf one), that may help you broaden your own search for more info.

 

Finally, I'll note first that I don't think it requires the ms driver, so could work with the Adobe provided sql server driver. Second, people who don't use cf enterprise can of course still integrate db data into solr collections. Besides using cfindex and the like, many also switch to calling on the solr implemented in cf by way of making cfhttp calls that send rest commands to the solr instance. Nothing in the cf docs covers that, but again solr docs will discuss it, and cfhttp can make such calls from cf. 

/Charlie (troubleshooter, carehart. org)
AlexCraig
AlexCraigAuthor
Inspiring
February 4, 2024

Morning Charlie,  Aha!  Your reply is very helpful.  Thank you.

We are running cf standard.  So my next step is to learn how to create a solr collection from 6 columns in a given table in our sql/server database that are used in searches.

I presume that would require coding a .cfm script.

I have not been able to find an example of such a script thus far.  Hopefully, you additional info will help me find one.  Thanks again.

Alex Craig, General Manager&quot;Avid Saltwater Fly Fisherman&quot;
Charlie Arehart
Community Expert
February 4, 2024

Well, first, you could do that with cfindex...as one could even back in cf 4.51 with Verity, before cf9 introduced solr to replace that. And its indeed documented--including an example, yay--at the CF docs page for the cfindex tag. Note that importing a query (the result of a cfquery, where you name the fields you want) entails using the CFINDEX  attributes type="custom" and the query attribute naming that cfquery. See the examples for more, at the bottom of the page, especially example 3 and perhaps 4 and 5:

https://helpx.adobe.com/coldfusion/cfml-reference/coldfusion-tags/tags-i/cfindex.html

 

Finally, in case anyone else may seek to use that data import handler feature with CF (which was added in Cf10, but enterprise-only), I have found that somehow curiously the docs for it seem to have disappeared from the Adobe site. Can't explain it...and can't find it even via archive.org. But here (for folks on enterprise, not CF Standard) is a link to the page in the PDF of the docs for CF10:

https://help.adobe.com/en_US/ColdFusion/10.0/Developing/coldfusion_10_dev.pdf#page=508 

 

But, Alex, I think the first link above should get you going. Let us know how it goes.

/Charlie (troubleshooter, carehart. org)