How does a search engine work ??

By | August 1, 2013

             Internet is a wide ocean of information. Its size doubles every year and so does the number of users. Under such circumstances the information retrieval can be a tedious task, though available. Hence there exists a need to simplify this process. This need is fulfilled by search engines. Search engines give any information regarding the required subject available on internet, based on the keywords. Almost everyone of us is using search engines but never bothered How does a search engine work ?. To tell the  working methodology of a search engine  simple words:

They send crawlers, which send the links related to the keywords as hits. Search Engines analyze these links and display results based on PageRank.


The World-Wide Web is moving rapidly from text-based towards multimedia content, and requires more personalized access. The amount of information on the web increases vigorously and so do the number of new users, inexperienced in the art of web search.

Search engines use automated software programs known as Spiders or Robots to survey the Web and build their databases. Web documents are retrieved by these programs and analyzed. Data collected from each web page are then added to the search engine index . When you enter a query at a search engine site, your input is checked against the search engine’s index of all the web pages it has analyzed . The best URLs are then returned to you as hits, ranked in order with the best results at the top.

Internet search engines are special tools on the Websites or a separate website that are designed to help people find information on the World Wide Web.

Difference between a Search Engine and a Directory

A directory (say Yahoo!) stores the name of the site, a relevant category and a short description of whats contained in the site. The information is stored as a hierarchy, with divisions represented by separate pages. When the site is

searched, the search is performed on the title and description of the site, not on the contents of the site.

The search engine (as Google) links all the URLs on the web. Then based on the keyword it sends its crawlers, which return the linked pages with the keywords as hits. It then ranks all the pages sent by them and displays results.

Different methods of searching used by a Search Engine::

There are differences in the ways various search engines work, but they all perform three basic tasks:

  • They search the Internet — or select pieces of the Web content – based on important words known as keywords.
  • They keep an index of the words they find, and where they find them.
  • They allow users to look for words or combinations of words found in that index.

Types of Search Engines::

There are three basic categories of search engines:

1) Spider or crawler-based search engines.
2) Directories powered by humans.
3) Combinations or hybrids of spider and directories.

Spider-based search engines create their listings by using digital spiders that crawl the Web. People sort the spiders’ findings and enter the information into the search engine’s database, which can then be searched by users.

There are also human-powered search sites, such as Yahoo!. Marketers submit a short Web site description to the directory, or the site’s editors may write one for sites they review. Users search match against the descriptions submitted, which means that changes to Web pages will not affect listings. Generally, today’s search engines present both types of results.

Anatomy & Working of Search Engines >>

Know these Words

  1. Spider: A spider is a robotic program that downloads Web Pages. It works just as the browser does when connected to a web site and download a page.
  2. Crawler: As a spider downloads pages, it can strip apart the page and look for links. It is the crawlers job to then decide where the spider should go to next based on the links, or based upon a pre-programmed list of URLs.
  3. Indexer: An indexer rips apart a page into it’s various components and analyze them.
  4. Database: The database is the storage medium for all the data a search engine downloads and analyzes.

Author: Ravi Bandakkanavar

A Techie, Blogger, Web Designer, Programmer by passion who aspires to learn new Technologies every day. It has been 6 years since I have been publishing articles and enjoying every bit of it. I want to share knowledge and build a great community with people like you.

Suggested read for you:

Did it help? Comment here..