Best practice for async crawling

If your website has content that is rendered asynchronously, you may want to look into async crawling. However, before deciding if there is a need for this, it is recommended to consider the following:

• Which part of the page is loaded asynchronously?
  • Are these parts of the page important content for the search engine?
• Is it possible to add all relevant information as meta tags in the HTML, making it possible for the crawler to fetch the needed information this way?

As async crawling requires more resources, it is recommended to first assess if there are easy-to-implement workarounds that will allow the crawler to index the page normally.

Keep in mind that Cludo’s support will always be involved in configuring async crawling for your site. Please reach out to support if you would like to have async crawling enabled or understand your options better.