How Does CCM19 Handle Bot-Traffic?

CCM19’s pricing model is based not only on the number of domains (-) but also on page views. Therefore, we are often asked whether and how bot-visits to websites are counted as page views in CCM19.

As a general rule, page views that can be clearly attributed to bots are not counted as views. To achieve this, we have implemented the following measures:

1. For every visit, we check the user---agent, and if it identifies itself as a search engine (Google, Bing, Yandex, etc.), the visit is not counted.
2. We also use the robots.txt file to inform search engines that our tool should not be crawled. Search engines that comply with this directive are therefore also not counted.
3. We maintain a constantly growing filter list of IP-addresses belonging to bots that we have identified over the years. This list primarily includes crawlers from SEO-tools and hosting providers. Visits from bots on this filter list are also not counted.

This allows us to detect and filter out a large portion of bot-traffic, so that bot-visits are negligible for most customers.

How to Identify Bot-Traffic in Your Account

In rare cases, however, it is still possible that your CCM19-account may record bot-traffic. This is usually easy to spot in the statistics due to an extremely high number of visits with no recorded interaction with the banner. Example:

Occasionally, regular spikes may also appear in the statistics. These clearly indicate a crawler that scans the entire site at fixed intervals, thereby generating artificial traffic:

These bots currently slip through our filters because they do not identify themselves as bots, ignore the robots.txt file, or use proxy-servers—and thus do not have a fixed IP-address.

What You Can Do About Bot-Traffic

If you notice an increase in bot traffic that is noticeably straining your-quota in CCM19, you can analyze the log files on your server and try to identify the problematic bots. With this information, you can do two things:

1. Use the server configuration at-to prevent the bot from continuing to access your website.
2. If you do not want to block the bot’s access-entirely—for example, because you want your website to be crawled regularly by an SEO-tool—please provide us with the bot’s user--agent, or IP address. We will then check whether we can add this bot to our filter list.