Deindexing

Author: Courtney Radsch

See also deplatforming/de-platforming.

This term refers to the intentional removal and unintentional removal of results from search engines and/or indexing websites. A website can be deindexed using robot.txt, which can be implemented to prevent other sites from crawling pages or sites or manually delisted. Search engines can implement themselves to remove or reduce the visibility of unwanted or low-quality results, such as spam or ‘clickbait’. There are paid services that can be hired to remove unwanted results from search engines, such as reputation management companies. Governments have required the deindexing of specific categories of information. For example, the “right to be forgotten” allows individuals to request that their personal data be removed from search engine results, amounting to deindexing through the delisting of results.

A more technical understanding of ‘deindexing’ refers to the de-indicization of a page from the search engine crawlers, which removes the need for a presentation of a ‘sanitized version’ of the results in the first place. This can be obtained by websites voluntarily by using robot.txt files, which convey that specific message to search engines. However, it is more complex to accomplish when the information should only be removed from search engines in connection with a particular keyword, as that cannot be accomplished (at least for the time being) without human intervention.