In the tech world, there are people who spent decades fighting for an open, accessible internet. Now those same people are starting to cheer when that same internet is locked down, walled off, and restricted.
What do they think? If it's hurting AI companies, it should be locked down.
On the other hand, there are those who consider blocking the internet a big mistake that threatens the principles that these supporters once supported.
There are many reasons to be concerned about LLM/Artificial Intelligence tools these days, for example, how they can be overrated, how they can be misused, and certainly about who has the power and control over these systems.
It is certainly worrying to see friends who once supported an open internet and the fundamental principles that govern it abandon those principles because they see that some AI companies can benefit from an open internet.
The problem is not only ideological, it is practical.
We are witnessing the construction of a fundamentally different internet, where access is controlled by gatekeepers and paywalls rather than governed by open protocols and user choice. And we are doing all this to stop AI companies, even though the real result will be to concentrate even more power in the hands of the same big tech companies, making the internet less useful for everyone else.
The collateral damage is already spreading beyond AI companies.
Reddit at one point began blocking the Internet Archive crawler from archiving Reddit streams because it was concerned that artificial intelligence companies would gain access to its content from the Wayback Machine.
What's the real reason behind a lot of this: Reddit discovered that user-generated content can be a source of revenue through licensing deals with artificial intelligence.
But instead of finding ways to capture that value while preserving access to the archives, it chooses to ban archiving altogether. We’re losing decades of human dialogue and cultural history because Reddit wants to ensure that AI companies pay for access to fresh content.
All of this suggests that we are moving far away from an open internet and towards one where it will not just be a “pay for scan” but a pay for everything if one wants to access anything on the internet.
George is still wondering what he is doing here….
There are ways to bypass all of this. Many companies are already doing it! See perplexity vs cloudflare, openai google search with SerpApi and other cases.
The winner will be the one who overcomes the limitations and trains his models despite them.