Participant

Question

CF9 and SOLR indexing

Forum|Forum|16 years ago
March 3, 2010
3 replies
2340 views

We are using CF9 64-bit and setting up a SOLR collection for an HR application. The database contains several million records and includes resumes that we want to do full text searches on.

We started out by using cfindex to create the index but it would bomb out after just a few thousand records with an error about "warming threads" (I don't have the exact error handy but can get it later) and the indexing would have to be manually restarted. This wasn't a good solution for a multi-million record operation..

Next, we created a custom Data Import Handler (DIH) outside of CF using the instructions in the SOLR wiki. This index worked great and was very fast. However, the ColdFusion tags (cfsearch, etc.) would not work with this index. We even made sure to duplicate the required nodes (<custom1> <custom2>, etc.) that the cfindex tag would have created. Still cannot search that index.

We'd really rather not reinvent the wheel and have to write custom search code. Obviously, we like using CF and it would be great if we can use the built-in indexing and searching capability.

Any ideas on how we can either 1) make the <cfindex> work without stopping OR 2) go ahead and use the custom DIH and be able to make the <cfsearch> work properly?

Dana

Advanced techniques

This topic has been closed for replies.

F

Faisal Bawa

Participating Frequently

I am also having the same issue

P

PolicyDesigner

Participating Frequently

I only have just over 500 records that I am trying to index, which they do consist of some large documents, and I try to loop through using the cfindex and I also get this error:

Error_opening_new_searcher_exceeded_limit_of_maxWarmingSearchers4_try_again_later

I found that if I put this in my loop

thread = CreateObject("java", "java.lang.Thread");

thread.sleep(1000);

</cfscript>

then I no longer have the error, but it does take a long long time to index. I also would like a better solution.

The coldfusion debugger shows that it is erroring out on the custom4 field. I don't know if the custom fields are struggling more than the main body field. Anyway, I am continuing to research my options.

C

Cheerful_sharpness1549

Participant

I am having a very similar problem... Did you ever find a resolution for this, other than adding the sleep time?

P

PolicyDesigner

Participating Frequently

I never did find a solution, I am very surprised at this. I am just waiting around for an answer.

A

Adam Cameron.

Inspiring

Couldn't you just use the full-text search capabilities of whatever DB it is you're running? Do you even need to use CF & Solr for this?

--

Adam

D

DanaPayneAuthor

Participant

Thanks, Adam.

Just using SQL is an interesting suggestion but not really a good solution in our case. We'd lose all the built-in advantages of a search engine like SOLR, for example faceting, exposing "near duplicates", etc. Our users are already used to doing "concept searching" but the search engine we were using can't be upgraded to work on our new platform, so we have to replace it. We thought SOLR was just the ticket.

Maybe SOLR integration into CF9 is just too new and still has too many bugs to work out? Are there any insiders at Adobe monitoring this forum who might know?

I'm really hoping we can find a way to make SOLR work for us so we don't have to roll our own concept search from scratch.

Dana

Sign up

To post, reply, or follow discussions, please sign in with your Adobe ID.

Sign in to Adobe Community

To post, reply, or follow discussions, please sign in with your Adobe ID.

Scanning file for viruses.

This file cannot be downloaded