Self-Hosting Challenges Rise as Corporate AI Scrapers Saturate Home Networks

A self-hosting enthusiast is planning to move their photography work from the web to physical zines to escape the 'depressing' reality of corporate AI scrapers. The author reports that these scrapers, originating from large cloud providers like Azure and Alibaba, have saturated their home upload bandwidth to feed data into large language models (LLMs).

The Impact of LLM Scrapers

The author describes a shift in the self-hosting experience, moving from the joy of learning system management to the frustration of dealing with 'LLM Scrapers.' These automated systems hit the author's site to collect data for AI training, often saturating the home network's upload capacity.

While some scrapers use residential IP ranges via SDKs—which are harder to block without affecting real users—the majority of the problematic traffic originated from a small number of large cloud-allocated /16 IP ranges.

“I do not want my time spent on something that ends up feeding that machine.”

— Running web sites makes me not want to - Lowkey Party

Technical Mitigation Efforts

To defend against this traffic, the author utilized `nftables` and `nft-blackhole` to drop connections from known cloud provider IP ranges. These tools allow for granular filtering of network traffic at the firewall level.

Despite these efforts, the author notes that the sheer volume of 'legal' scraping makes it difficult to maintain a website without either high-level technical skills or paying a large corporation to provide a Content Delivery Network (CDN).

The Shift to Physical Media

Because of the 'corporate tech over-reach,' the author is considering removing their photography website entirely. The goal is to transition to selling physical zines, ensuring that the work remains in the physical world rather than feeding a machine that produces 'poor imitations of creativity.'

This move represents a trade-off: while it requires more manual work to produce and distribute physical goods, it removes the burden of defending against automated data harvesting. The author views this as a 'silver lining' to the current state of the web.