20120823

How to Search the Deep Web - the real one


How to Search the Deep Web

Search engines such as Google index over a trillion pages on the World Wide Web, but many of the results from queries can be useless or irrelevant. There is, however, another part of the web that common search engines don't reach. The "deep web" (also know as the "hidden web", "invisible web" or "Deepnet") is estimated to be many times larger than the "surface web".
The deep web contains information that isn't indexed by a search engine. For instance, databases generate results on the fly, and do not have searchable pages. The deep web also holds academic studies and papers, scientific research, government publications, electronic books, bulletin boards, mailing lists, online card catalogs, articles, directories, many subscription journals, archived videos, images and more. Searching the deep web isn't hard, and anyone can do it. This article will get you started.
  1. 1
    Determine the specific topic you need to find. It helps to write down the search terms you want to use. Look at the list and add synonyms for the keywords.
  2. 2
    Categorize your topic. For instance, is it related to health? Automobiles? The economy?
  3. 3
    Decide what type of source you'd like to search. For instance, do you want to search:
    • Library materials (books, journals, reference books)
    • Data or statistics
    • Research results (academic, scientific, engineering, medical, etc.)
    • Everything related to your topic

  4. Choose your starting point based on your objective. You can start from a database search engine, library sites, specialty sites, a directory of sites, or general search sites.


    • For database searches, you can begin with CompletePlanet or Geniusfind.
    • For reference material searches, try LibrarySpot.comINFOMINE or Internet Public Library.
    • There are many specialty sites. For instance, science.gov provides links to the U.S. government's scientific deep web site. FreeLunch.com contains economic data from across the world. NHS Direct has health information and advice within England. You can find a good starter list of specialty sites at Online Education Database, or use one of the search engines below.
    • You can use a search engine that helps you find the right search engine, such as Search Engine Guide orGeniusfind. Search Engine Guide works best if you choose a topic before you do a search, while Geniusfind requires you pick a topic first.
    • If you aren't sure what you need, there are some general search sites for the deep web such as Internet Archive and Clutsy that make a good starting point.


Tips

  • Familiarize yourself with the search conventions of that particular site. Most offer a simple search bar like google.com or yahoo.com. Some sites have an advanced search capability for additional search criteria or to narrow your initial search. Other sites don't offer a search bar until you've picked a topic.
  • Some deep web sites require subscriptions or payment to access materials. However, many excellent resources are free.
  • There are no hard-and-fast rules for searching the deep web. Experiment.


Warnings

  • Deep web is infamous for containing unethical content. As one would with surface web, please take caution before clicking on anything sketchy or unusual!