I am running this docker image: https://github.com/nextcloud/docker with a cloudflare tunnel, meaning the webserver would see all the traffic coming from a single ip in 172.16.0.0/12 .
The documentation says:
The apache image will replace the remote addr (IP address visible to Nextcloud) with the IP address from X-Real-IP if the request is coming from a proxy in 10.0.0.0/8, 172.16.0.0/12 or 192.168.0.0/16 by default
So I thought that this is a not a problem, as other docker images can also automagically figure out the real IP address from traffic coming from cloudflare tunnels.
In the beginning it worked fine, then it was SLOW. Like 2 full minutes to load new feeds on news, waiting ages to complete a sync, and so on. I rebooted the server on those instances, and then it worked fine for a day.
So because at the time i was running it on unraid, i blamed the lag on that OS + my weird array of HDDs with decades of usage on them. Migrated to debian on a nvme array and… same lag!
Wasted hours trying to use caddy+fpm instead of apache and it’s the same, worked fine for a day, then it was slow again.
Then I wondered: what if the program is “smart” and throttles it by itself without any warning to the admin if it thinks that an ip address is sending too many requests?
Modified the docker compose like this:
nextcloud:
image: nextcloud
became
nextcloud:
build: .
and I created a Dockerfile with
FROM nextcloud
RUN apt update -y && apt upgrade -y
RUN apt install -y libbz2-dev
RUN docker-php-ext-install bz2
RUN a2enmod rewrite remoteip
COPY remoteip.conf /etc/apache2/conf-enabled/remoteip.conf
with this as the content of remoteip.conf
RemoteIPHeader CF-Connecting-IP
RemoteIPTrustedProxy 10.0.0.0/8
RemoteIPTrustedProxy 172.16.0.0/12
RemoteIPTrustedProxy 192.168.0.0/16
RemoteIPTrustedProxy 173.245.48.0/20
RemoteIPTrustedProxy 103.21.244.0/22
RemoteIPTrustedProxy 103.22.200.0/22
RemoteIPTrustedProxy 103.31.4.0/22
RemoteIPTrustedProxy 141.101.64.0/18
RemoteIPTrustedProxy 108.162.192.0/18
RemoteIPTrustedProxy 190.93.240.0/20
RemoteIPTrustedProxy 188.114.96.0/20
RemoteIPTrustedProxy 197.234.240.0/22
RemoteIPTrustedProxy 198.41.128.0/17
RemoteIPTrustedProxy 162.158.0.0/15
RemoteIPTrustedProxy 104.16.0.0/12
RemoteIPTrustedProxy 172.64.0.0/13
RemoteIPTrustedProxy 131.0.72.0/22
RemoteIPTrustedProxy 2400:cb00::/32
RemoteIPTrustedProxy 2606:4700::/32
RemoteIPTrustedProxy 2803:f800::/32
RemoteIPTrustedProxy 2405:b500::/32
RemoteIPTrustedProxy 2405:8100::/32
RemoteIPTrustedProxy 2a06:98c0::/29
RemoteIPTrustedProxy 2c0f:f248::/32
and now because nextcloud is seeing all the different ip addresses it doesn’t throttle the connections anymore!
Thx for explaining. I’m not sure if I’m willing to do the same trade-offs. Supposedly their WAF is very good and quite some people use it. Probably for a good reason… It just comes at a hefty price. I’m doing selfhosting to emancipate myself, stay independent and in control. I’m not sure if becoming dependant on a single large company and terminating my encryption on their servers that do arbitrary magic and whatever with my packets is something that aligns with my goals. (Or ethics, since I think the internet is to connect people on a level playing field. And that’s no longer the case once many people transfer control to a single entity.) But I don’t see a way around that. Afaik you have to choose between one or the other. Are there competitors to cloudflare that handle things differently? Maybe provide people with the WAF and databases to run on their own hardware, let them stay in control and just offer to tunnel their encrypted data with a configurable firewall?
Edit: Just found modsecurity.org while looking that up. But I guess a good and quick database of bad actors’ IPs is another thing that would be needed for an alternative solution.
It’d be a challenge to keep up — 0 days aren’t going to be added to self hosted solution faster than they could be detected and deployed on a massively leveraged system. Economy of scales at full display.
I mean theoretically… I guess, if they do it right? It depends a bit. Some Linux distributions are crazy fast with patching stuff. And some stable channels have a really good track record of open vulnerabilities. Nowadays that’s not the only way of distributing software, vulnerability might depend on your docker container setup etc.
Are there actual numbers what Cloudflare adds on top? What 0-days they focus on? I mean do they have someone sitting there, reading Lemmy CVEs and then immediately getting to action to write a regex that filters out such requests?
And how much does it cost? They also list the same ModSecurity in their lower plans. I don’t think 0day protection would help people like me if it’s $200 a month.
The difference in my opinion is that doesn’t matter how fast upstream vendors patch issues, there’s a window between issue being detected, patch being implemented, release getting pushed, notification of release gets received, and then finally update getting deployed. Whereas at least on cloud WAF front, they are able to look at requests across all sites, run analysis, and deploy instantly.
There is a free tier with their basic “Free managed ruleset”, which they’ve deployed for everyone with orange cloud enabled when we saw the Log4J issue couple years back. This protection applies for all applications, not just the ones that were able to turn around quickly with a patch.
If you want more bells and whistles, there’s a fee associated with it, and I understand having fees is not for everyone, though the price point is much lower – you get some more WAF feature on the $25/mn ($20/mn amortized when paid annually) tier as well before having to fork out the full $250/mn ($200/mn when paid annually) tier. There’s a documentation page on all the price points and rulesets available.
I tried to look it up but I wasn’t very successful. What they do in their free tier keeps being a mystery to me. In the $20/month is the the core ruleset from ModSecurity. I don’t need to pay them $20 to deploy that for me, the dataset is free and publicly available. I’ve just installed it on my VPS… It’s only a few lines in Nginx to enable that.
And what you’re talking about is $200 a month. I seriously doubt anyone here uses that plan for their homeserver. I wouldn’t pay $2400 in a year for it.
I still don’t get how that would work. Sure you can filter spam that way. And migitate attacks while the worst wave washes through the net. Or do machine learning and find out if usage patterns change. But how would it extend to 0-days faster than the software gets patched? This sounds more like snake-oil to me. If someone finds a way to inject something into a Nextcloud plugin and change things in the database so they have access… And then they do it to 100 cloudflare customers… How would Cloudflare know? If it’s a 0-day, they -per definition- don’t know in advance. And they’re just WAF, they don’t know if a user is authorized by mistake or if they’re supposed to have access. And they don’t know anything about my database, since it runs on my machine. And they also don’t know about the endpoints of the software and which request is going to trigger a vulnerability unless this manifests in some obvious (to them) way. Like 100 machines immediately start blasting spam through their connection and there is one common request in the logfiles. Otherwise all they can do is protect against known exploits. Maybe race the software vendor and filter things before they got patched. I just don’t see any substantial 0-day protection that extends to more than “keep your server up to date and don’t use unmaintained software.” Especially not for the home-user.
The free tier rolled out was specifically to address upstream vendors patching Log4J too slowly. They’re able to monitor the requests and intercept malicious patterns before it hits the server running unpatched (due to upstream unavailable yet) applications. They are updating with more rules for the free tier set as far as they’ve stated. The extras from paid tiers are more extra rulesets and more analytics around what was blocked etc.
At the end of the day though, you do you; the benefit for me may not be benefit for you. I’m not selling their service, and have no benefit what so ever should anyone opt into their services.
Thx for explaining. I think I halfway know what this is about now. I don’t think I’m their target group. But I learned something about web application firewalls in the process and that is a good thing. I think I’m going to activate that for some of my private services since it’s so easy and look up if there are good ip ban lists. It’s a bummer that I don’t get to see proper documentation on this, since security is all about exact facts and scenarios. But I guess no answer is also an answer. If they just feed buzzwords to me, either my initial skepticism was warranted, or I’m just not their target audience and they only target enterprise users. Either way I’m better off with my current approach. I appreciate I got to learn something :-)
No problem! I appreciate the civil discussion! Thank you!