Not all search engines are created equal. In fact most of the sites refered to as search engines are really directories.
- A 'Search Engine' must operate a program (usually called a 'spider') which travels the internet, examining each page that it comes across and storing an index of those pages in a database. It is often possible to suggest or request that the spider visit a particular site. It is estimated that only 15% of the available fixed pages are indexed at any given time and none of the information available in database driven web sites. Click here for more information on the workings of search engines.
- To get the most out of search engines it's necessary to move from the basic search into the advanced options. The advanced search on Google is a good example of the kinds of criteria that can be set in a search. It also helps to know which engines are better for the particular type of search that you want to undertake. The University at Albany has published the results of its research on search engines at http://library.albany.edu/internet/choose.html.
- The better known search engines include:
- Directories don't rely on a spider for results. They depend almost exclusively on human editors. They can be industry or cause specific, community, country, or global in scope. While Yahoo is the best known example of a directory the Open Directory Project http://www.dmoz.org with nearly 3 million sites, over 40,000 editors, and over 400,000 categories is the largest and best. The home page will probably look familiar to you since the format and data are used by many thousands of sites worldwide.
- There is an amazing compendium of search engines and regional directories sorted by country at http://www.twics.com/~takakuwa/search/search.html