Efficient Indexing

[nikhil20101]

Hi,

I have written an application where users can search for keywords within a set of documents in a directory. Users can also upload documents. My current implementation is such that whenever a user uploads a document, I am reindexing (<cfindex action="update" ... >) and this is takes a lot of time and user has to wait a while before the action is completed.

Can I make this re-indexing happen at the background OR is there any other way to not make the user wait a long time before his form is submitted?

Thanks

Nikhil

[ilssac]

nikhil20101 wrote:

Can I make this re-indexing happen at the background OR is there any other way to not make the user wait a long time before his form is submitted?

Both!

You could use a schedule task to do the indexing on a 'ahem' schedule.  Once a day, Once an hour or whatever might work.  This, of course, means that new data will not be indexed until the next scheduled task is executed.

You could fire off some type of asynchronous process that does the indexing, while not involving the thread that is processing the form request.  Thus that request can be allowed to complete and the user can go on their way while the indexing is occurring.  This can be done in ColdFusion with either Gateways or the <cfthread...> tags, whichever meets your needs better.  With old school servers it can also be done with the <cfhttp...> tag utilizing a timeout parameter of zero.  But that use has largely been supplanted by the <cfthread...> tag.  Just becareful that multiple users firing off multiple tasks to index your data at the same time does not cause a problem with this solution.

[nikhil20101]

ilssac,

cfthread works but every time I upload/delete a new document I am reindexing the entire collection which I feel is not efficient. Is there something using which I can add/delete index upon upload/deletion of that file.

Thanks

Nikhil

[Adam Cameron.]

You don't have to reindex the entire collection each time there's an update. I'm @ the pub typing this on a phone @ present, so can't (/won't) look up the docs for you, but check the docs for : you can update individual entries.

--

Adam

[nikhil20101]

Adam,

I searched for updating index for individual files but didn't find suitable results. Can you please give me some pointer to find the answer.

I want something like, if a file is uploaded then a new index should be created and if a file is deleted then only that particular index has to be removed.

Thanks

Nikhil

[Adam Cameron.]

Did you look up the official online docs:

http://help.adobe.com/en_US/ColdFusion/9.0/CFMLRef/WSc3ff6d0ea77859461172e0811cbec22c24-7d04.html

does what you want, ne?

--

Adam