How does the crawler handle 301 or 302 redirect?
Search.io Site Search handles HTTP redirects automatically when a page is reindexed. The redirected pages are removed from the Collection(index) when they have a 301 or 302 HTTP status code.
The destination page of the redirect gets added to the Collection and will be shown in the search results.
Note: Any page that doesn’t have any meta changes detected will be re-crawled after 3-6 days in any case. The redirected pages might still appear in the search results until the next re-crawl takes place.
How to remove redirected pages immediately from a Collection:
You can also manually trigger the crawler to remove the page if you don't want to wait by following these steps:
- Log in to your Search.io account
- Select the relevant Collection
- Navigate to 'Domains' and click on 'Diagnose'
- Enter the URL that you have removed/redirected and press "Diagnose"
- The result and details of the record would be returned. Press "Add to Index"
The page will be removed from the index in a few minutes, and the "State" would change to "Redirect" the next time it is diagnosed. See a screenshot below:
