Once a crawler has crawled the defined domain(s), you may experience a specific file not being added to the search index. This will typically be due to one of the following reasons:
- The file is bigger than 15MB which is the Cludo limit. Such large files can be pushed directly to the Cludo search index, using the API.
- The file format is not support. Click here to see all supported file formats.
- The URL of the file is outside the defined website for the crawler.
- The file is an orphan file, meaning that there are no links to it, preventing the crawler from ever finding it.