How to review the crawler log

The crawler log offers insights into the activity of the crawler. It allows for further investigation into which pages were indexed and which were not and why.

1. In the navigation, select Configuration › Crawler Activity and click the name of the crawler you want to see the logs for.
   You can also go to Configuration › Crawlers and click the timestamp for the crawler you want to see the logs for.
2. You are now taken to the crawler logs.
3. Optional: Enter a search term in the Search in table field to search for a specific URL or message.
4. Optional: Select a domain from the Domain drop-down, to limit the log entries to a specific domain.
5. Optional: Select a log type in the Log Type drop-down, to limit the log entries to specific log types.