Follow or stop a website crawl
See how far a crawl has got and stop it if needed.
Where to find it
Sources › Website
Open this screen
Steps
- Open Sources › Website after starting a crawl.
- Find it under Crawls: the Page URL, Whole site or Chosen pages, how many of up to 200 pages were read, and its status.
- Press Stop on a running crawl to end it.
- When it finishes, the site appears in the list with the pages read; click it to see Pages read and the content.
Settings on this screen
| Setting | What it does | Default |
|---|---|---|
| Stop | Ends a running crawl. Managers only. | — |
Troubleshooting
"The crawl stopped when the server restarted. Start it again."
Add the site again with Add & crawl.
"Too many crawls are running right now. Try again in a few minutes."
Wait a few minutes and start it again.
Tips
- Start a new crawl only after the running one has finished or been stopped.
Good to know
- A crawl reads up to 200 pages in up to 180 seconds.