What is Index?
The index, or the index database, holds the web pages that an engine has discovered and stored after processing, or crawling, the Internet. Using this database, an engine avoids scanning the Internet when juxtaposing user queries. Websites placed in the index will yield quick and relevant results, which saves time. To be included in an index, a web page’s site must be properly formatted, not cluttered by blocking commands, and devoid of blocking directives, among other things.
Taking Google as an example of a search engine, its index contains billions of URLs ready to be served in response to queries. However, it does differ from human-curated directories, which store summarized entries. Niche listing sites are human-curated, which makes them fundamentally different from Google.
Related terms: crawl, crawling, indexing, SERP, search engine database.