Sure, Google searches billions of web pages, but it doesn’t come close to indexing the entire internet. A vast repository of information exists on the web that search engines such as Google don’t have access to. This is commonly referred to as the “deep web” or “invisible web.” Most of the deep web is made up of the contents of specialized, searchable databases that contain high quality information such as journal articles, technical reports, and scientific data.

The best way to start your search of the deep web is with one of the many valuable subscription databases Dibner Library provides access to. You can browse databases by subject area or ask a librarian for help in selecting a database for your research.

You can also search the deep web using free search portals that dig beneath the surface web to locate subject-based information that may not be indexed by Google. For example, the Science Accelerator utilizes federated search technology which retrieves results from multiple research databases located in different departments within the U.S. Department of Energy. It includes full-text documents, bibliographic citations, ePrints, patents as well as project summaries.

Other useful search tools for digging into the deep web include:

• E-print Network
• Infomine
• Scitopia
• TechXtra

For easy access, you can find these sites and others listed in the research tools section in the right sidebar of this blog.