Unable to crawl site with Verity Spider
We are running ColdFusion 9 Enterprise (version 9,0,1,274733) on a SUSE Linux Enterprise server running Apache 2.2.3, and we would like to crawl our site to create a site-wide search engine. We tested Solr Server initially but recently switched to Verity K2 Server since Solr does not include a spider component. Based on the ColdFusion 9 documentation, it sounds like Verity includes a built-in web crawler (Verity Spider), but we have not been able to run it successfully.
We are trying to run this command from /opt/coldfusion9/verity/k2/_ilnx21/bin:
./vspider -cmdfile /opt/coldfusion9/bin/testcmd.txt
Contents of testcmd.txt:
-style /opt/coldfusion9/verity/Data/stylesets/ColdFusionVspider/
-collection /opt/coldfusion9/verity/collections/recycle
-start http://localhost/
-locale english
-cgiok
-abspath
-reparse
-indmimeinclude text/*
-indmimeexclude text/css
The command produces the following error message:
vspider - Verity, Inc. Version K5.5.0 (_ilnx21, Aug 22 2007)
I18nMsg Info: Error E0-1509 (Drvr): dlopen() returned: vzlib.so: cannot open shared object file: No such file or directory
I18nMsg Info: Error E0-1510 (Drvr): Error loading driver library 'vzlib.so'
2011/05/03 13:43:08 Info: [vspider] (ind006000) Message database loaded from [/opt/coldfusion9/verity/k2/common/ind.msg].
2011/05/03 13:43:08 ERROR: [vspider] (ind002006) VDK: Error E0-1509 (Drvr): dlopen() returned: vzlib.so: cannot open shared object file: No such file or directory.
2011/05/03 13:43:08 ERROR: [vspider] (ind002006) VDK: Error E0-1510 (Drvr): Error loading driver library 'vzlib.so'.
2011/05/03 13:43:08 Info: [vspider] (ind012207) Last indexing job was interrupted 1 time(s). Running in auto-restart mode.
2011/05/03 13:43:08 ERROR: [vspider] (ind002006) VDK: Error E0-1509 (Drvr): dlopen() returned: vzlib.so: cannot open shared object file: No such file or directory.
2011/05/03 13:43:08 ERROR: [vspider] (ind002006) VDK: Error E0-1510 (Drvr): Error loading driver library 'vzlib.so'.
2011/05/03 13:43:08 Info: [vspider] (ind006001) License loaded from [/opt/coldfusion9/verity/k2/common/runtime.lic].
2011/05/03 13:43:08 Info: [vspider] (ind005005) Licensed for local spidering.
2011/05/03 13:43:08 Info: [vspider] (ind005008) Not licensed for remote spidering.
IndExcept from indmtutl.cxx:177
sh: /opt/coldfusion9/verity/k2/_ilnx21/bin/vdiag.sh: /bin/sh^M: bad interpreter: No such file or directory
Segmentation fault
webserver:/opt/coldfusion9/verity/k2/_ilnx21/bin #
Any suggestions? Thanks!
Message was edited by: justoffthecoast
