• Global community
    • Language:
      • Deutsch
      • English
      • Español
      • Français
      • Português
  • 日本語コミュニティ
    Dedicated community for Japanese speakers
  • 한국 커뮤니티
    Dedicated community for Korean speakers
Exit
0

CF9 and SOLR indexing

Community Beginner ,
Mar 03, 2010 Mar 03, 2010

Copy link to clipboard

Copied

We are using CF9 64-bit and setting up a SOLR collection for an HR application. The database contains several million records and includes resumes that we want to do full text searches on.

We started out by using cfindex to create the index but it would bomb out after just a few thousand records with an error about "warming threads" (I don't have the exact error handy but can get it later) and the indexing would have to be manually restarted. This wasn't a good solution for a multi-million record operation..

Next, we created a custom Data Import Handler (DIH) outside of CF using the instructions in the SOLR wiki. This index worked great and was very fast. However, the ColdFusion tags (cfsearch, etc.) would not work with this index. We even made sure to duplicate the required nodes (<custom1> <custom2>, etc.) that the cfindex tag would have created. Still cannot search that index.

We'd really rather not reinvent the wheel and have to write custom search code. Obviously, we like using CF and it would be great if we can use the built-in indexing and searching capability.

Any ideas on how we can either 1) make the <cfindex> work without stopping OR 2) go ahead and use the custom DIH and be able to make the <cfsearch> work properly?

Dana

TOPICS
Advanced techniques

Views

2.2K

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
LEGEND ,
Mar 03, 2010 Mar 03, 2010

Copy link to clipboard

Copied

Couldn't you just use the full-text search capabilities of whatever DB it is you're running?  Do you even need to use CF & Solr for this?

--

Adam

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Community Beginner ,
Mar 04, 2010 Mar 04, 2010

Copy link to clipboard

Copied

Thanks, Adam.

Just using SQL is an interesting suggestion but not really a good solution in our case. We'd lose all the built-in advantages of a search engine like SOLR, for example faceting, exposing "near duplicates", etc. Our users are already used to doing "concept searching" but the search engine we were using can't be upgraded to work on our new platform, so we have to replace it. We thought SOLR was just the ticket.

Maybe SOLR integration into CF9 is just too new and still has too many bugs to work out? Are there any insiders at Adobe monitoring this forum who might know?

I'm really hoping we can find a way to make SOLR work for us so we don't have to roll our own concept search from scratch.

Dana

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Jul 14, 2010 Jul 14, 2010

Copy link to clipboard

Copied

I only have just over 500 records that I am trying to index, which they do consist of some large documents, and I try to loop through using the cfindex and I also get this error:

Error_opening_new_searcher_exceeded_limit_of_maxWarmingSearchers4_try_again_later

I found that if I put this in my loop

<cfscript>

    thread = CreateObject("java", "java.lang.Thread");

    thread.sleep(1000);

</cfscript>

then I no longer have the error, but it does take a long long time to index.  I also would like a better solution.

The coldfusion debugger shows that it is erroring out on the custom4 field.  I don't know if the custom fields are struggling more than the main body field.  Anyway, I am continuing to research my options.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Aug 09, 2010 Aug 09, 2010

Copy link to clipboard

Copied

I am having a very similar problem... Did you ever find a resolution for this, other than adding the sleep time?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Aug 09, 2010 Aug 09, 2010

Copy link to clipboard

Copied

I never did find a solution, I am very surprised at this.  I am just waiting around for an answer.

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Jun 17, 2011 Jun 17, 2011

Copy link to clipboard

Copied

hmm...thats a shame. I too am having the same problem. Every time I find someone with a similar problem, the solution is never there. Indexing seems to run fast some days and slow the other days. The documentation for SOLR isnt the best either. It seems as though SOLR is nice for smaller searches, but once you start getting into larger amounts of documents it seems to fail miserably. 

I'm currently indexing 4,000 documents and its been running for 40 minutes.

Also, how come they released a built-in version of SOLR into coldfusion without any crawlers?

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
New Here ,
Apr 29, 2015 Apr 29, 2015

Copy link to clipboard

Copied

LATEST

I am also having the same issue

Votes

Translate

Translate

Report

Report
Community guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting. Learn more
community guidelines
Resources
Documentation