If it seems that the Internet is mostly populated with bots, it’s because it is. A report by ANA and White Ops shows ad-fraud sites make up approximately 20 percent of ad-serving sites on the world wide web, not having a shred of human traffic attached to them. These websites make money almost exclusively from bot traffic, engineered to mimic human behavior in order to generate ad revenue for the site owners. It’s like The Thing, only if The Thing sat down behind a computer and started clicking.
Image credit: YouTube
Bot traffic is a problem that nobody is immune to, yourself included. How do you recognize and neutralize it? Take a few minutes out of your schedule to find out.
The first step is to raise your awareness to the highest level. There are different types of bots out and about on the web, so getting to know them might enable you to recognize them more easily. You don’t need to become a world-famous expert in bot traffic, just the right information will suffice.
More than half of traffic on the Internet comes from bots but they’re not all malicious in nature. You’ll often hear (somewhat surprisingly) the classification of good bots vs bad bots. The good bots are the web’s very own worker bees who crawl sites for a myriad of information and generally aim to make the Internet a better place. Good bots include:
Seeing as they form roughly 27% of online traffic, the rest (exactly 28.9% according to Incapsula’s Bot Traffic Report) falls on baddies such as:
Those would be the main offenders. As technology evolves, more and more bots from different categories are created. So you can also encounter scraper bots that act opposite to copyright bots or zombie bots (yes, there are zombie bots) who give impostor bots a run for their money by taking the whole security thing a step further and dig down deeper (send viruses, malware, and such).
Some of these sound terrifying, some don’t but they all have one thing in common - they’re up to no good. Luckily, there are ways to spot them and prevent them from doing serious damage.
Start by reviewing your analytics data. This is the place to get all insights regarding your traffic and do your due diligence. When you identify traffic you suspect as fake, check specific data parameters such as high bounce rate, low page or session count, and a high number of new sessions. Also, check to see if the sessions are consistent with the content. If your primary targets are English speaking audiences, receiving a good chunk of traffic from non-English speaking countries should raise a flag. If your analytical capabilities are limited, implement proper tracking and collect the data you need to more accurately determine fraudulent activities. No single metric will paint the whole picture, especially if both real and fake traffic are in play. That’s why you need to keep an eye on onsite factors such as slow loading times or slow website performance.
An example of a click bot
Image credit: ptcandadfly.wordpress.com
Identifying bot traffic largely depends on your ability to combine multiple factors and spot any suspicious data or lopsidedness. Then, there’s the matter of eliminating it, which should be the easier part. Bots regularly refresh cookies and you can use that to your advantage. Perform frequent updates to your exclusion list with known bot IP addresses, user and device IDs, as well as domain and subdomains of known fraudulent sites. Updates are important because new domains are registered almost every day. Once again, be aware of different tactics, publishers, ad networks, and else to identify strategies and trends in order to eliminate fraud. Understand various types of bot activities that both you and others encounter to deal with them more easily. Finally, nip the evil in the bud by engaging with trustworthy partners who are both transparent and active in mitigating bot traffic. Good partners will always be open to discuss the volume and type of their traffic and will be strict on any form of irregularities.
Your primary mission is to prevent bot attacks in the first place. Considering all the craftiness these malicious undesirables possess and exhibit, that’s easier said and done. And, as much as we hate to say this - that’s ok. Even the biggest cats in the game have trouble with it, and there’s only so much you can do. However, the fact remains that bots siphon advertisers’ budgets and manage to cover their tracks. According to Juniper Research, advertisers will lose an estimated $19 billion due to ad fraud in 2018, with bots being key drivers. That's the equivalent to $51 million per day, and what’s worse is that the figures are projected to rise on a yearly basis, reaching $44 billion by 2022.
Bot traffic isn’t going away anytime soon (if ever). The only way to protect yourself from bad bot traffic is to stay vigilant. It’s a process that demands constant analysis and learning from past experiences, so make sure your blacklists are up-to-date and learn more each day to have less trouble in the future. It’s always better to be safe than sorry, right?