When searching, you may experience the same content appearing more than once in the results. Since a crawler is unable to index the same URL twice, this will always be due to the same content existing on multiple URLs. That is, of course, unless you have two crawlers that index . . . Read more
Once a crawler has crawled the defined domain(s), you may experience a specific file not being added to the search index. This will typically be due to one of the following reasons:
Change default translations Default translations are the small pieces of text that appear in the template of the website, such as “did you mean..” or “your search for …”. These are automatically translated into the language that the search is set up for. However, it is possible to rephrase or . . . Read more
You may wish to implement Cludo on an intranet solution that is otherwise closed to the public. For this, you will want to consider how the crawler should access the site as well as how secure the implementation needs to be. Ways to allow crawling behind login In order to . . . Read more
The MyCludo interface supports both English and Danish. This guide will explain how to change the interface language for a user. Please note this language change does not affect any engines, visitors, or even other users in MyCludo. If the changed user is the currently logged-in user, the entire interface . . . Read more
When implementing search on a website, the CludoJS can be used to enable/disable and customize certain features. The options are defined as an object with multiple options. The table below describes each of the options available, what they do, and examples of values.All options are available for custom templates, however, . . . Read more
XPath (XML Path Language) is a path expression, used to refer to a specific subset of XML or HTML markup. In MyCludo, XPath can be used in the crawler configuration to define a specific part of the HTML of the page to use for a field. Using the HTML markup, different . . . Read more
It is possible to set a specific time schedule for the crawler to run at a specific time of day. Currently, it is not possible to set the time schedule for a crawler via MyCludo. To configure time-scheduled crawling, submit a support ticket, informing your timezone and at which time of . . . Read more
Async crawling is meant for websites where content is loaded asynchronously (AJAX-generated content). AJAX-generated content allows the web page and web browser to process data without having to reload the page. For example, if you hit a “Submit” button on the page, AJAX processes the information and updates the content . . . Read more
Currently, it’s not possible to enable smart crawling directly in MyCludo. To enable smart crawling, submit a support request, letting us know which crawler you would like to enable smart crawling for. A support agent will confirm that your site is eligible for smart crawling and inform you when it is . . . Read more