Search facility on my site

[nigelh70638339]

I have been trying this for ages and nothing has really helped and I am just going round in circles.

My site contains thousands of individual pages that I want be found quickly and without using the Google Search which I currently have.

The site is basically split between country and then further split into brand and some are split by year.

I have installed MAMP (as I am using a Mac) and tried to create a database. I discovered that there were not enough rows (only 40 max) and looked into it. This led me to discovering that I need to create php pages which I did containing a search box and another for the results. Tried it and I was prompted to set up a test server which I did. The result when I tested the pages was that I received an 404 error. Even basic html pages showed the same.

Surely there has to be a simplified tutorial, step-by-step to help with this difficult task.

I am not afraid of working with it but at the same time I am not comfortable with long passages of text that waffle on about the reasoning of ab and c. I am looking for a simple explanation that will point me in the right direction.

Thanks in advance!

[osgood_]

nigelh70638339  wrote

I have been trying this for ages and nothing has really helped and I am just going round in circles.

My site contains thousands of individual pages that I want be found quickly and without using the Google Search which I currently have.

The site is basically split between country and then further split into brand and some are split by year.

I have installed MAMP (as I am using a Mac) and tried to create a database. I discovered that there were not enough rows (only 40 max) and looked into it. This led me to discovering that I need to create php pages which I did containing a search box and another for the results. Tried it and I was prompted to set up a test server which I did. The result when I tested the pages was that I received an 404 error. Even basic html pages showed the same.

Surely there has to be a simplified tutorial, step-by-step to help with this difficult task.

I am not afraid of working with it but at the same time I am not comfortable with long passages of text that waffle on about the reasoning of ab and c. I am looking for a simple explanation that will point me in the right direction.

Thanks in advance!

Using php like that won't work. Normally when you use php to search a site all the pages main text needs to be stored in a database field so you can write a mysql query to search through the field and return urls, based on the pages unique id, to a results page if any matching words are found or you can set up a keyword column and search through that. I dont think thats an option as if you have thousands of pages it would take a lifetimes work to set up a searchable database, so what you are really looking for is a way to search through your html pages and set up a page which returns any matches to a  results page when found.

These work similar to Google but search within your site. Google is poor because it indexes anything any everything, most of which is out of date and it takes forever to drop the indexing, so if you have a site that changes content regularly its always out of date, which is hugely frustrating for visitors if they keep clicking on information, index by Google, which is no longer available.

Personally I have never used a search facility like I describe because most of the sites I build don't contain thousands of pages and its easy if I do then need to add a search to build something  based on keywords and then return result based on the id of the 'product/page' stored within the database. What you need is a much more generic solution of searching your whole site I would assume.

I think some of the other locals in the forum will have used a database-less solution other than Google, so maybe they will be able to point you in the right direction

[nigelh70638339]

Thanks osgood.

The site is mainly graphical with images of packs of cards. These packs of cards are what I would like to be searchable. I am rebuilding the site so now would be a good time to try and put something together.

Google search is 'bleep' as it either does not find things or shows results that have nothing to do with the site.

I hope that someone can help as it would help visitors find specific packs. There are in total over 3,500 packs so the database will be very big, I understand that.

[osgood_]

nigelh70638339  wrote

Thanks osgood.

The site is mainly graphical with images of packs of cards. These packs of cards are what I would like to be searchable. I am rebuilding the site so now would be a good time to try and put something together.

Google search is 'bleep' as it either does not find things or shows results that have nothing to do with the site.

I hope that someone can help as it would help visitors find specific packs. There are in total over 3,500 packs so the database will be very big, I understand that.

If you are completely rebuilding the site then php search IS the way to go.

Lets assume you are going to have a 'details' page for each pack of cards. You would store those details in a database so you only have to build one master details page, rather than 3500 individual website details pages, which would be a nightmare to manage and maintain.

Part of the details, which are stored in your database, could be a keyword field where keywords are stored, (keywords need to be comma seperated) These keywords will be related to each specific pack of cards. When a use inserts those keywords into your search form the database keywords field would be searched for any matches. If a match is found you can built a 'results' page  and list the relevant packs of cards associated with the keywords the user has input.

The list of matched pages on the results page is dynamically linked to the 'details' page, so the details information for a particular pack of cards gets retrieved from your database when the link on the results page is clicked. This is achieved via a unique database id which is passed via a url link. A unique id is a dedicated field in the database used to distinguish between database entries. It is automatically generated each time you add data to the databse.

Maybe that gives you an idea of the workflow.

[nigelh70638339]

Each pack has its own page where each card is displayed as a png as well as some text information.

I only really want the search to go to the pages that when an image is clicked then goes yo he more detailed page. This way the user can see the entire series while targeting the pack searched for, if that makes any sense. Either I still have to add all 3,500+ packs in the database along with the links.

I have seen some tutorials that show that the site can be created purely via MySQL and then formatted with CSS but again I sort of follow along and then just get massive overload and get lost.

[Nancy OShea]

I can see why Google search isn't working for you.   Your site has not been indexed.

This appears on Google.com when I enter your URL

Ultimate Top Trumps

www.ultimate-top-trumps.co.uk/

A description for this result is not available because of this site's robots.txt

I looked at your robots.txt file and it is disallowing search bot access to EVERYTHING on your URL.  Basically, you've blocked search engine access from crawling your site.  That needs to be fixed ASAP.

In addition, you have no sitemap.xml for search engines to follow.  That also needs to be fixed ASAP.

Learn about robots.txt files - Search Console Help

Build and submit a sitemap - Search Console Help

When those 2 matters are fixed, you can tell Google to re-index your site by logging in to your Google Webmaster Tools console.  Be patient.  It will take up to 1 month for Google to index your entire site.  But after that, you should be able to use the following search form to find content on your domain alone.

Below is the HTML code for a site specific Google Search form.

<!doctype html>

<html lang="en">

<head>

<meta charset="utf-8">

<title>Google Site Search</title>

</head>

<body>

<script>

//GOOGLE SEARCH

//Enter domain of site to search

//Use the one that yields best results either www. or non-www.

var domainroot="www.ultimate-top-trumps.co.uk/"

function Gsitesearch(curobj){

curobj.q.value="site:"+domainroot+" "+curobj.qfront.value

}

</script>

<form class="search" action="http://www.google.com/search" method="get" onSubmit="Gsitesearch(this)">

<input name="q" type="hidden" />

<input name="qfront" type="search" required class="searchField" placeholder="Google Site Search" maxlength="50"/>

<input type="submit" class="search-button" value="SEARCH" />

</form>

</body>

</html>

[nigelh70638339]

I have checked out the Robots.txt file and corrected it.

I created this file a long time ago when I was greener around the gills than I am now!

Thanks again!

[Jon Fritz]

Don't forget to remove that robots.txt file.

Any search engine that follows the directives of the robots.txt file will never index your site as long as it's there in that format.

[nigelh70638339]

Remove it? I thought that you have to have one?

I have replaced the old one with the new.

[Jon Fritz]

The default action of a search engine crawler is to follow and index all pages and files, unless told otherwise by the robots.txt or robots <meta> tag.

You only need to include one if you plan to disallow access to the crawlers. The one you had disallowed access to all areas of your site via the * operator.

EDIT: Yep, doing too many things at once.

[EbaySeller]

I think Jon misinterpreted your recent robots.txt file to say disallow everything on your site, but since your file does not have the trailing slash at the end of disallow it's actually allowing all access to your site.

User-agent: *

Disallow: /

The "User-agent: *" means this section applies to all robots. The "Disallow: /" tells the robot that it should not visit any pages on the site.

To allow all robots complete access

User-agent: *

Disallow:

(or just create an empty "/robots.txt" file, or don't use one at all)

[nigelh70638339]

Ah right, OK.

Thanks again everyone, its really appreciated, thanks!

[Nancy OShea · Accepted solution]

If you want to avail yourself to other search engines besides Google, look at:

1)  Freefind.com -- an online search engine service.  The free version contains ads.  For approx. $9/mo you can have add-free search results.  After joining, Freefind will crawl your site and index it within approx 24 hours.   Try the free version first to see if you like it. 

Site Search Engine, Free and Pro Versions - FreeFind.com

2) Zoom from Wrensoft.  Zoom is standalone search engine software for indexing your site.   They have tiered pricing based on how many pages you need indexed.  If you make changes to your site, you will need to manually generate a new index each time.

Zoom Website Search Engine - Add search to a website, intranet or CD

[nigelh70638339]

I think that I will go with the Freefind search engine. I have got some code tried it out and it looks the biz!

All I need to do now is format it so everything is nice and central.

Thank you Nancy and osgood!