How we protect client websites from bots and attacks

A client website slows down, every now and then it fails to open, and nobody knows why. Traffic is the same as last month. Then we look at the server logs and see who is taking it: foreign programs that walk through thousands of addresses in minutes, hunting for forgotten backups and passwords, testing whether they can get in somewhere. Real people wait in the meantime, because the server is serving them and those programs at the same time.

That is why we put the websites we look after behind Cloudflare. This article describes what it solves in practice, what we block, what we deliberately let through and why, and how you can check whether your own website has similar protection.

What we protect a website from

An attack on a small company website does not look like a scene from a film. It looks like this:

• Vulnerability scanners. A program walks the website and tries addresses where sensitive files tend to sit: database backups, configuration files, private certificate keys, the login page of a content management system. One forgotten backup is enough for an attacker to walk away with customer data.
• Password guessing. Repeated login attempts against the administration, often hundreds per minute.
• Bursts of requests. A robot starts downloading the whole website at once. There is no bad intent, but the server behaves the same way: everyone slows down.
• Form spam. Robots fill in contact and enquiry forms until the owner stops reading their own leads.

The interesting part is how those robots introduce themselves. We recently caught a scanner on our server that tried a dozen addresses in one second, things like a private certificate key or a configuration file, and claimed a different identity with every request: once Google's crawler, then OpenAI's search robot, then a robot from xAI. Anyone can write any name into a request. That is why protection must never rely on who a visitor claims to be.

What we actually do for our clients

Traffic goes through a filter first and reaches the server second

Cloudflare sits in front of the website like a gatehouse. A request passes through its network first and only then reaches the server, so obvious attacks and overload never touch the machine at all. We wrote about the platform in more detail in why we use Cloudflare.

We block the hunt for holes, not visitors

The rules target behaviour, not names. Requests for addresses that never existed on the site, typically a WordPress login on a site with no WordPress, or configuration files, stop at the gatehouse. A visitor browsing the pages never notices a rule exists.

We cap how many requests come from one address

When one place sends more requests in ten seconds than a person could possibly make, it gets a short timeout. The limit is set so that neither a fast reader nor an auditing tool working for the client runs into it.

We let artificial intelligence through, on purpose

Cloudflare can block robots that collect content for artificial intelligence with a single switch. We leave that switch off for our clients. People now ask about suppliers, services and prices in ChatGPT or Gemini, and those answers are built from websites those systems have read. Shutting them out means disappearing from the place where your customer is looking. We also keep the files those robots rely on open, meaning robots.txt and the sitemap.

We leave forms and payments alone

Protection must never show up on the part that earns the money. Form submissions, payment gateways, customer logins and accounting integrations stay unrestricted, and we click through them after every change.

We make sure the server sees the visitor's real address

When traffic goes through a gatehouse, by default the server sees the gatehouse as the sender, not the visitor. The application then counts every visitor as one person, and login or form protection starts limiting people who did nothing wrong. On our server we set this up so the application knows the visitor's real address. Without that, protection does more harm than good.

Email is not affected

Clients often worry that the change will cost them their email. It will not. The website and email travel separate routes. The records that route incoming mail stay untouched and keep pointing at the mail server. We cover that side separately in the article on why emails from a website land in spam.

How we make changes so nothing can break

With someone else's domain the rule is simple: back it up, then change it, and always keep a way back.

1. We export the complete state. Every record of the domain goes into a file, so we know what it looked like before we touched it.
2. We run the change as a dry run first. The output shows exactly what will change, and only when it matches expectations does it run for real.
3. We keep a record of what we changed. Every affected record has its original form stored, so rolling back is a single command.
4. We verify from the outside. The site has to answer, the administration login has to work, the form has to deliver a message and email has to arrive. We do not ask the settings, we ask the website.

What the owner gets out of it

• The site stays fast during peaks, because pointless traffic never reaches the server.
• Fewer chances of a disaster. A scanner that gets no answer finds no forgotten backup either.
• Calm under a surge. A sudden wave of requests ends at the gatehouse, not on a server that also runs other websites.
• Secure connections and modern protocols with no extra work, with certificates renewing themselves.
• You stay visible in answers from artificial intelligence, because we do not block the robots that build them.

For an ordinary company website and a smaller online shop, the free tier is enough. We only reach for paid plans where a site faces genuine attacks or needs rules the free tier does not offer.

How to tell whether your website has any of this

1. Try an address that does not exist on your site, for example the login page of a content management system you do not use. An error page is fine. A login form means there is something on your website you do not know about.
2. Find out who handles your domain. In any domain checking tool, look at the name servers. They tell you whether there is a protective layer in front of the website or whether visitors go straight to the server.
3. Ask your supplier three questions: what happens during a sudden surge, where the backup of the domain settings is, and whether login attempts on the website are limited.

If you have nobody to ask, take a look at our website management and maintenance, or at what taking a website over involves. We set the protection up as part of the takeover, not as a paid extra.