Over the past few days, I’ve seen an increasing number of websites that are hitting the old Your connection is not private error pages.

At first I thought it was a bit odd, and looked into the certificate a little bit and noticed that it was issued by Cloudflare Inc - Gateway ECC Certificate Authority. I thought it was a little strange, but thought, maybe they’re just using something on Cloudflare, and it’s a bit misconfigured. I let the owner know, and went on my merry way.

Today when I hit it one again, I looked at the certificate again, and found that, sure enough, it was issued by the same root cert. Something bigger must be going on here. Maybe there’s something new on the Cloudflare side that I’m hitting.

The Investigation

I happen to be running a number of applications, including the DNS for my house, from Cloudflare Zero Trust. It allows me to override the DNS for a number of internal applications to my home lab, and even use it with WARP to access them from my phone away from home.

Given that my home is running through Cloudflare Zero Trust for the DNS, I thought one of the first things to test was if it happened to be something related to the Cloudflare Zero Trust DNS.

While still using the Cloudflare Zero Trust DNS, this is

% dig www.noahpinion.blog +short
162.159.36.12

Switching over to 8.8.8.8 the DNS response was very different

% dig www.noahpinion.blog +short
target.substack-custom-domains.com.
104.18.40.87
172.64.147.169

Checking the webpage now, everything appeared to work as expected.

The next step was a bit roundabout. I dug into the IP address showing up when I was getting from the Cloudflare Zero Trust DNS, which led me to a Cloudflare forum post, talking about Cloudflare Zero Trust and domain blocking, but also linked out to Cloudflare Radar. I figured, why not, I’ll try it with the the last domain that failed.

And there it was, the answer was staring me right in the face.

Allow me to explain.

My weekend Experiment

On the weekend, I was browsing through some junk news sites, and I was astonished with how many ads they had these days. Like there was seriously as much vertical space spent on ads as content. It made me pine for my old Pi Hole days.

I used to use Pi Hole a lot on my home network. But it frequently came a point of failure, like if the Raspberry Pi that was running it went down, then “the internet stopped working”. That was a phrase I heard all to many times.

I eventually moved from a single instance of Pi Hole running on one Raspberry Pi, to a cluster with redundancy (more on that another day). But whenever I was doing maintenance on the cluster, if the wrong node when down at the wrong time, DNS resolution would stop, and then it was like the internet stopped working. So before I would do any changes on the network, I would turn off the local Pi Hole resolution on my local network, and my router configured to use Cloudflare Zero Trust DNS to take over.

Eventually once forgot to turn it back. Everything kept on chugging. The complaints about the internet stopped, and it just kind of stuck.

Until this weekend. Annoyed with the amount of adds this weekend, I started thinking, could I just configure Cloudflare Zero Trust DNS to work like Pi Hole? I did some digging around, and the idea showed some promise, at least on the small scale. I came across Tom Paulus’ post on Replacing PiHole with Cloudflare Gateway. The blog post explained the idea, and even included the Terraform he used to configure it.

I tried it on my local network, and it seemed to block a few Ads, but it sounded like the secret sauce would be in his Block Manual Trackers & Ads list. I spend a little time myself seeing what PiHole itself pulls from for it’s block list. Turns out, its a lot (as Tom mentioned, it’s over 500,000), and without going through a sales call to increase them, Cloudflare Zero Trust has a limit of 100 lists, and 1000 entries per list. So I parked the idea, and kind of forgot about it.

The Solution

Recall from the investigation how I had seen the newly seen domains category

During my experiment, I added 4 new block rules.

The Block Content Categories rule, was designed to block a number of Cloudflare categories domains, including Newly Seen Domains.

Also notice that note in the Additional policy settings that Block page is enabled. That there was the real cause of the problem. Turns out, the prerequisite for the Block page is to install the Cloudflare Gateway Certificate.

Adding that certificate is essentially adding a new root trust certificate to your devices. A Root Trust Certificate is what makes a certificate authority work. When you go to a website that your browser says is secure, it’s considered secure because it has a certificate that was issued by one of the Root Trust Certificates on your machine.

Why is this needed? Because given that most pages these days run over HTTPS, for Cloudflare to be able to display a Block page for the domain, your browser needs to trust the response it gets.

You might be asking then why Cloudflare doesn’t just use it’s root certificate authority that is trusted to generate these certificates. Well, the way that these block pages are effectively working is by being a man in the middle. If Cloudflare detects you’re going to a page that isn’t trusted, instead of giving you the DNS for the original site, my rule tells it to send the block page. In this case, the block page is hosted by Cloudflare, so it’s returning it’s own content in the place of the original content. They can’t do that with a generally trusted root certificate, because if they did, this would likely lead to their root certificate getting revoked from systems. That kind of behaviour is exectly what Symantec did to get their certificate revoked. So instead of using their General Root Certificate, they have one specifically for people using Zero Trust Gateway, where by installing that certificate, you are agreeing to let Cloudflare switch content like that.

For me, that was one bridge I wasn’t willing to cross. By trusting a root certificate, it means that everything that certificate signs will be trusted by your system. This includes anything purporting to be your email, banks websites, government tax filings, etc. Essentially, if it uses the certificate chain, Cloudflare could pretend to be it.

But for me, the solution was really simple. Once I turned off my Cloudflare Zero Trust Content based DNS Block rules, all of the the domains that previously threw up the Your connection is not private message started working just fine.