Indexing is the process by which a search engine analyses a fetched page and stores it in its index — the giant database of web content it searches against to answer queries.
Definition
Indexing is the process by which a search engine analyses a fetched page and stores it in its index — the giant database of web content it searches against to answer queries. After a crawler downloads a page, the indexing pipeline parses the HTML, renders JavaScript where needed, extracts the text, images, links and structured data, evaluates the page's topic and quality, checks directives like noindex and canonical, and decides whether and how to store the page.
Only indexed pages can appear in search results.
A page can be crawled but not indexed for many reasons: a noindex directive, a canonical pointing to another URL, content judged too thin or duplicative to be worth storing, a soft 404, a page blocked by a rendering failure, or simply a quality bar the page does not clear. "Crawled — currently not indexed" and "Discovered — currently not indexed" are standard states reported in search-engine consoles, and on large sites a meaningful share of URLs sit in them.
In context
For iGaming affiliate sites, indexing coverage is a core health metric. A site that has published 2,000 reviews and guides but only has 600 of them indexed is not competing for the other 1,400 queries, regardless of how good those pages are.
The usual causes are quality-related — thin, templated, or near-duplicate pages that the search engine chooses not to store — or technical: rendering problems, conflicting directives, a bloated URL space that dilutes crawl and index attention, or slow discovery because the pages are poorly internally linked.
Improving indexing is mostly about raising the average quality and reducing the noise. Consolidate or remove thin pages so the remaining set clears the bar.
Fix directive conflicts (a page that is noindexed, or canonicalised away, or blocked from rendering). Trim the low-value URL space with noindex and robots.txt so crawl budget concentrates.
Strengthen internal linking so important pages are discovered and re-crawled. Then monitor the index-coverage report over weeks, because indexing decisions are revisited and a page's status can change as the site's overall quality signal moves.
Worked example
An affiliate site has 2,100 published URLs but only 640 indexed; the console shows 900 as "crawled — currently not indexed", most of them thin templated operator pages. After merging those into 200 substantive reviews with original testing and removing the rest, indexed coverage rises to 1,050 over two months and organic traffic roughly doubles.
Related terms
Frequently asked questions
Browse the full iGaming & affiliate glossary — hundreds of EN/RU terms with examples.
← Back to glossary