WebJunction

In case you hadn't noticed, the Web is a pretty big place. Your best shot at finding a particular piece of information is by using a search engine. Search engines are online services that help you find specific Web pages by using key word searches or drilling down through lists of information categories.

Most search engines work by using automated agents called robots, or spiders, that crawl through the Web and find information from individual sites. They pick up words from the title, URL, or the full text of Web pages -- depending on the ranking alrgorithms of the specific search engine.

There are many engines out there, and the right choice will depend on what you're using it for.

Some search engines are metasearch engines, which means that they search through many different search engines' databases at the same time.

Subject directories group Web sites by topic. The information in a subject directory is usually selected and sorted by a human intermediary. Subject Directories are often a good place to start your research.

Many search engines function as search engines and subject directories. Most people tend to use their favorite engine, and sometimes they may try another just out of interest. We have found that with the dozens of search engines on the Net, there is no perfect search engine. WebJunction.org has listed the "big ten" as: AltaVista, Excite, Google, GoTo.com, Yahoo, HotBot, Northern Light, LookSmart, Lycos, and Open Directory. TechSoup has assembled its own list of favorite engines. Try a few and see which are appropriate to your needs.

TechSoup Favorites

• Ask Jeeves allows the user to ask questions of the search engine in natural language. It attempts to answer the questions that users pose in the search box. This can be helpful when you need an answer to basic technical questions. (For example, you could ask Jeeves, "What does it mean if my monitor is flickering?" You will be directed to a Q&A page for monitor problems.) While this can work well, sometimes the engine just produces a list of vaguely related search results. The bottom line is that this is a good option for the technophobic, and that's really its only benefit over any other search engine.
   
• Good old Excite.com is a highly personalizable portal and a comprehensive search engine that updates its entire database frequently. Because it attempts to index only the most popular pages on the Web, Excite is a good choice if you're looking for large, popular Web sites, but not so good if you want to find more obscure sites. It has a collection of searching tools and tips. Its most attractive features are its ability to search its directory, the entire Web, or a database of photos. Its yellow pages and the white pages are also helpful.
   
• Google is the big daddy of all the search engines, and is now most popular with computer geeks and casual users alike. Many Web-savvy surfers use the name as a verb synonymous with Web searching.
  
  Google is a metasearch engine that conducts the search across many different search engines at once. It delivers results that pay attention to the proximity of the search terms you enter. This eliminates unnecessary filtering. Google also ranks its results based on the amount of hits (and links from other sites) per URL. Google also has a separate subject directory, a news portal that lists news by the most recently updated articles, a toolbar you can use so you don't need to visit its site to search, a directory of listservs and groups, and an image gallery.
  
  The Google Directory derives its information from The Open Directory Project. The ODP is a volunteer-edited directory that aims to be the most comprehensive directory on the Web. Experts volunteer to be editors of certain content areas. Unlike Yahoo's alphabetical listings, the Google Directory has listings by order of importance.
   
• HotBot enables the researcher to perform power searches through its database which includes the databases of Google, Lycos, and Ask Jeeves, so the results are more comprehensive. Wired Digital, a subsidiary of Wired Magazine, originally launched HotBot in 1996. Lycos purchased Wired Digital in 1998 and continues to run HotBot as a separate search engine. Its benefits include a selection of "skins" to customize the look and feel of the user interface, advanced filtering options to hone and refine your search, and a toolbar like Google's, so you never need to go to a separate site to search.
   
• LookSmart is a useful tool, as it functions as a subject directory and a search engine. LookSmart's benefit is that it can search its directory, the entire Web, or its own database of articles. You can search for a topic and then pull down a menu to choose the type of magazine within which you would like the database to search. Its drawback is that it doesn't have any academic journals in its database.
   
• Mamma, based in Quebec, bills itself as a "smart" metasearch engine. Like an ordinary metasearch engine, it searches a variety of engines, directories, and content sites. What makes it smart is that it eliminates duplicates and ranks them according to relevance. It also will tell you which search engine was used to find a particular result.
   
• SurfWax is a metasearch engine that displays its results in an accessible interface. Surfwax's results are based on meaning, rather than keywords, making them more relevant. This engine allows you to manage your results in several different views. It has a "site snaps" feature to give you a snapshot of a site before clicking into it. It also provides a breakdown of the statistics for each search by displaying how many results came from each particular database.
   
• Teoma is a company that works with other search engines to refine their results. It does this by monitoring what users choose from the list of search results. Sites that get clicked on more than others rise higher in Teoma's rankings. In other words, it serves up search results that were acceptable to people with similar search queries. This way, its searches are based more on context and concept, and not just on the words themselves. The search engine also provides you with suggestions of other terms to refine your search.
   
• WiseNut is a LookSmart engine. The WiseGuide Feature automatically creates categories for you. It therefore can function simultaneously as a subject directory and a search engine. Wisenut consistently yields many relevant results and their "Sneak-a-peek" feature allows you to preview a Web site before taking the time to click in.
   
• Yahoo, best known as a search engine, is actually a hierarchical directory. Users can search the Web through Yahoo, but it works best for surfers who want to pick their way through organized lists of topic-specific resources. You can search the entire Yahoo site, or you can search within each category.
  
  When Search Engines Aren't Helping
  
  Although there are many ways to find information on the Web, sometimes you simply cannot find the information that you need in the amount of time that you have. When you find that it is too time-consuming to weed through all the hits that your search yields, try posting a question to a listserv or a message board. On sites like the TechSoup Community, you can ask a specific question like, "Does anyone know what is a good site for Bay Area Web Developers?" (See related article on listservs and visit the TechSoup Nonprofit Technology Online Communities resource list.)
  
  

Visit TechSoup for technology information, access to donated and discounted products, and support from nonprofit experts and your peers.

Editor's Note: This article was originally written in May 2000. It was updated in January 2004.