Looking at creating a custom search engine w/ Google

Date: Tue Dec 02 2008
Google offers a way to create a Custom Search Engine which means that you can define your own search engine and offer it for others to use. It's even possible to generate income by running your own search engine. The custom search engine has a strictly limited set of sites from which search results are drawn.

The advantage is to focus the search results.  With the normal google search search results come from the whole of the Internet and it's often difficult finding what you want. Kinda like searching for a needle in a haystack, eh? With a custom search engine there is a specific list sites being searched and search results come only from those sites. If the owner of a specific custom search engine has done a good job the targeting of the custom search engine will give better search results.

The basics to create a custom search engine is trivial. Simply go to http://www.google.com/coop/cse/ and follow the instructions, Google has made the process dead trivial. For simple custom search engines, that is.

Once you have the basic custom search engine created let's take a tour of the customizations available. On the Control Panel for a given custom search engine there is a series of tabs labeled

Basics: Name, description, keywords, whether the search engine can be cooperatively managed, etc. It is important to note that Google's thought is some search engines will be managed by a group of people.

Sites: This is where you list the sites available in the search engine. In this tab you add sites manually, one at a time. An 'exclude' site is one not used in search results. It is also possible to add a list of sites to the search engine using methods in the 'Advanced' tab.

Indexing: It's unclear exactly what this tab is for. A sitemap file is used by each specific site to notify Google (and other search engines) of the pages on that individual site. Clearly if you want to use a Google CSE to provide search for your own site it's important that your site export a sitemap file. But if your custom search engine is indexing other sites besides your own you will not have the ability to provide a sitemap file for those sites, and hopefully the owners of those sites will export their own sitemap file, but this page doesn't give you a clue as to whether that's happened or not.

Refinements: Appears to be keywords or key phrases you can attach to the search engine to help users with their searches. They don't explain this very well.

Look and feel: Customizes the colors and branding of the search results page.

Code: Provides a code snippet you can embed in other sites. Google provides a page on their site that displays your search engine, but you can also put the search engine on your own site. You can also publish the code and encourage others to embed your search engine in their sites.

Collaboration: Is where you manage the list of people who can manage this search engine.

Make money: Allows you to integrate the custom search engine with your adsense account. Google of course puts advertising on the search results pages, and if you link the search engine to your adsense account then you'll get a cut of the income.

Business account: Is where you convert the custom search engine into Google Site Search. With Site Search you're paying Google a fee and have more control. Google doesn't put their own advertising on the search results, and you have total control. Also with the Custom Search Engine you can only list 5,000 sites maximum, and with Site Search you can any number of sites.

Advanced: Offers a way to upload files that describe the search engine. For example you can keep a database of sites to index, dump the database to a file, and upload that file to Google. If you're indexing hundreds or thousands of sites it's simpler to maintain the search engine as a database that you upload to Google en masse rather than one-by-one adding sites using the manual method. You must however be ready to understand some badly written documentation.

Preview: Let's you try out your search engine e.g. to make sure it works.