Skip to main content
Participant
May 3, 2011
Question

Unable to crawl site with Verity Spider

  • May 3, 2011
  • 1 reply
  • 884 views

We are running ColdFusion 9 Enterprise (version 9,0,1,274733) on a SUSE Linux Enterprise server running Apache 2.2.3, and we would like to crawl our site to create a site-wide search engine. We tested Solr Server initially but recently switched to Verity K2 Server since Solr does not include a spider component. Based on the ColdFusion 9 documentation, it sounds like Verity includes a built-in web crawler (Verity Spider), but we have not been able to run it successfully.

We are trying to run this command from /opt/coldfusion9/verity/k2/_ilnx21/bin:

./vspider -cmdfile /opt/coldfusion9/bin/testcmd.txt

Contents of testcmd.txt:

-style /opt/coldfusion9/verity/Data/stylesets/ColdFusionVspider/

-collection /opt/coldfusion9/verity/collections/recycle

-start http://localhost/

-locale english

-cgiok

-abspath

-reparse

-indmimeinclude text/*

-indmimeexclude text/css

The command produces the following error message:

vspider - Verity, Inc. Version K5.5.0 (_ilnx21, Aug 22 2007)

I18nMsg Info: Error   E0-1509 (Drvr): dlopen() returned: vzlib.so: cannot open shared object file: No such file or directory

I18nMsg Info: Error   E0-1510 (Drvr): Error loading driver library 'vzlib.so'

2011/05/03 13:43:08 Info: [vspider] (ind006000) Message database loaded from [/opt/coldfusion9/verity/k2/common/ind.msg].

2011/05/03 13:43:08 ERROR: [vspider] (ind002006) VDK: Error   E0-1509 (Drvr): dlopen() returned: vzlib.so: cannot open shared object file: No such file or directory.

2011/05/03 13:43:08 ERROR: [vspider] (ind002006) VDK: Error   E0-1510 (Drvr): Error loading driver library 'vzlib.so'.

2011/05/03 13:43:08 Info: [vspider] (ind012207) Last indexing job was interrupted 1 time(s).  Running in auto-restart mode.

2011/05/03 13:43:08 ERROR: [vspider] (ind002006) VDK: Error   E0-1509 (Drvr): dlopen() returned: vzlib.so: cannot open shared object file: No such file or directory.

2011/05/03 13:43:08 ERROR: [vspider] (ind002006) VDK: Error   E0-1510 (Drvr): Error loading driver library 'vzlib.so'.

2011/05/03 13:43:08 Info: [vspider] (ind006001) License loaded from [/opt/coldfusion9/verity/k2/common/runtime.lic].

2011/05/03 13:43:08 Info: [vspider] (ind005005) Licensed for local spidering.

2011/05/03 13:43:08 Info: [vspider] (ind005008) Not licensed for remote spidering.

IndExcept from indmtutl.cxx:177

sh: /opt/coldfusion9/verity/k2/_ilnx21/bin/vdiag.sh: /bin/sh^M: bad interpreter: No such file or directory

Segmentation fault

webserver:/opt/coldfusion9/verity/k2/_ilnx21/bin #

Any suggestions? Thanks!

Message was edited by: justoffthecoast

    This topic has been closed for replies.

    1 reply

    Inspiring
    May 4, 2011

    We are running ColdFusion 9 Enterprise (version 9,0,1,274733) on a SUSE Linux Enterprise server running Apache 2.2.3, and we would like to crawl our site to create a site-wide search engine. We tested Solr Server initially but recently switched to Verity K2 Server

    I'm not sure that's a good move.  Verity is extinct technology and will be dropped from CF entirely in CF10.

    Solr does come with with a crawler, it's just not integrated with CF: http://nutch.apache.org/about.html

    --

    Adam

    Participant
    May 4, 2011

    Thanks for the heads up. We did manage to get the 'vzlib.so' errors sorted out, but the indexing job crashes at random points with a segmentation fault and hasn't yet finished successfully. With Verity being discontinued, we're going to check out Nutch + Solr instead.