Our crawler will always attempt to make as many requests as possible, often requesting multiple pages per second, but the actual frequency of requests depends on the server response from the website. Some websites might also have a crawl delay set in their robots.txt, which can impact how many requests . . . Read more
Stop words are fill words that don’t provide any context and they will be ignored in the search query, thereby increasing relevancy in search. There is a list of stop words for every language that is supported by Cludo. Below you can find the stop words for a few languages: . . . Read more
The MyCludo interface is optimized for desktop use but is also responsive, meaning it will automatically adjust to fit various screen sizes, including mobile devices and tablets. Compatible MyCludo browsers Compatible Template browsers Cludo’s overlay, inline templates, and custom implementations are compatible across all device types—desktop, mobile, and tablet—on browsers . . . Read more
Lemmatization is the process of grouping together the inflected forms of a word so they can be analyzed as a single item, identified by the word’s lemma, or dictionary form. Unlike stemming, lemmatization depends on correctly identifying the intended part of speech and meaning of a word in a sentence, . . . Read more
The way Intelligent 404 referral URLs are registered is different from referred page URLs for searches (search origin pages). For searches, the referral information is added to the URL. When the visitor lands on the search results page, Cludo will grab the referralURL from the query parameters passed to the . . . Read more
The natural language processing at Cludo consists of multiple steps: Supported Languages Language ISO code Tokenization Elision Stop words Stemming Arabic ar ✅ ✅ ✅ ✅ Armenian hy ✅ ✅ ✅ ✅ Basque eu ✅ ✅ ✅ ✅ Brazilian pt-br ✅ ✅ ✅ ✅ Bulgarian bg ✅ ✅ ✅ ✅ . . . Read more
The number of languages available on a website typically influences how many crawlers and engines are required to deliver an effective site search experience. To ensure accurate and relevant indexing, it is best practice to assign a dedicated crawler to each language version of the website. This approach helps the . . . Read more
Cludo supports two operators when the search terms contains more than one word: Operators are set for the engine and can currently only be changed by Cludo. It is usually not recommended to switch to AND, since this is likely to result in more searches with 0 results. You can contact . . . Read more
Once a crawler has crawled the defined domain(s), you may experience a specific page not being added to the search index. This will typically be due to one of the following reasons: Feel free to contact support if you have further questions on why a page was not indexed as . . . Read more
When looking at the number of search requests to Cludo, those actually cover a few different types of requests: How to view search count statistics To get an overview of your current search request usage, log into MyCludo and go to the Account Overview from the top right menu. In . . . Read more