No more words

In a hurry? As always, here’s the summary:

AI models have eaten the internet!

Now that ChatGPT and company have swallowed everything from recipes to Reddit threads, they must either learn new tricks or collapse under the weight of their own AI-generated drivel. The solution may be fake websites where AI agents can practice booking Airbnb rentals and paying your bills.

Meanwhile, Google has overtaken OpenAI, prompting Sam Altman to declare "code red." And in China, children are managing social hierarchies with smartwatch likes.

This is Kludder of the week!

The language models that fascinate—and frighten—us have been trained on everything we've ever written. But now the well has run dry. ChatGPT, Claude, and Gemini have consumed all available digital text. But they're hungry for more.

The Scaling Law

In 2020, Jared Kaplan was part of Johns Hopkins University. There, he published a research paper showing that large language models like ChatGPT become more powerful and human-like as access to training data increases. This became known as the scaling law.

Today there's broad consensus that Kaplan was right. AI-experts have long been concerned about the day when all digital text has been chewed up and spit out by language models. Just as humans become more skilled and develop richer vocabularies, language models depend on being trained on texts. And now everything has been read—from recipes to Reddit threads, instruction manuals to job postings. Referring to this, Kaplan, now a Chief Scientist at Anthropic, has said we can't expect the same lightning-fast progress going forward, as all the training material has been exhausted.

But now that AI-generated content has flooded digital platforms, why can't we just feed artificial intelligence its own artificial intelligence? Steven Vaughan-Nichols, a journalist covering technology for decades, fears a model collapse. With language models succumbing to a form of cannibalism by training on content generated by themselves, they’ll be caught in a sloppified death spiral. Vaughan-Nicols, and many others believe that this kind of AI inbreeding will make the models incomprehensible, inaccurate, and chaotic or - to use AI-lingo - the inbreeding will lead to enshittification and platform decay.

A New Kind of Learning

No more human-made material means AI companies must think differently in training their models.

"We're going to keep investing more and more in AI until model collapse fully sets in and the AI responses are so bad that not even a brain-dead CEO can ignore it." - Steven Vaughan-Nichols writing for The Register.

One proposal is to mix AI generated content with human-generated content. That way, you dilute the AI material enough so it doesn't ruin the language models. That's what they’re hoping, anyway. But as things stand, we humans neither generate enough content nor generate it fast enough to sate the machines’ thirst.

"There have been extraordinary results over the last three or four years as the scaling laws kicked in, but we're no longer getting the same progress." - Demis Hassabis, CEO of Google DeepMind to the New York Times.

The AI companies’ ambition is for models to handle increasingly complex tasks. Anthropic seems to be specialising Claude towards coding work. Meta wants to be the social channel, driving spending and advertising using AI. OpenAI looks to be heading towards the enterprise market. A chat window is no longer enough, as they look to insert themselves in our lives. Now they want to order your groceries and work across various programs on your desktop. Yet, not unlike us humans, machines improve through trial and error—watching where a human clicks to achieve the desired result. Now this trial-and-error training method, called reinforcement learning, has spurred a new host of entrepreneurs.

The New York Times has an article about startups delivering AI-generated websites where language models can roam free. These websites are near-perfect replicas of familiar websites, a fact that hasn't been entirely unproblematic. The company AGI Inc replicated United Airlines' website and got the airline's lawyers on their case. Founder Div Garg had to modify the site to avoid copyright infringement. Similar websites replicas are popping up: The Airbnb copy, Staynb, was created to train AI models to navigate the lodging giant. Even Google's email service, Gmail, has been duplicated: Go Mail gives AI models access to experiment, test, and fail on their own. The hope is that models can train themselves through reinforcement learning, based on their own data.

"When you train the models, you need to use thousands of AI agents simultaneously so they can explore the websites and perform various actions. If you do that on a real website, the agents just get blocked." - Div Garg to the New York Times.

The Next Top Model or Sudden Collapse?

Google, OpenAI, and Anthropic all use reinforcement learningso that we can unleash autonomous agents that handles our Christmas shopping, sort our inboxes, and pays our bills.

Whether or not AI will revolutionise our lives depends on who you ask. Demis Hassabis, who leads Google's AI effort DeepMind, still believes in the technology. He just thinks it’ll move a bit slower. On the other end of the spectrum, you have people like Vaughan-Nichols. He belongs to the camp that believes models will break from having consumed their own regurgitation over and over. He may have a point:

Bloomberg conducted a study showing that language models with access to external information like documents and databases had a greater probability of providing misleading analyses or leaking sensitive information. Researchers use this as an example of how inaccurate information becomes even more inaccurate the more times a language model processes it. For us mere mortals who aren't AI researchers, the whole thing can be described as a game of Broken Operator. A small error or oversight at the very start ends up completely incomprehensible by the time it reaches the last ear.

And just like that, AI polarisation is in full bloom. It’s Heaven or hell and no in between. Even so, I find myself in limbo. Language models have made me work faster and more efficiently. At the same time, I spend less time on social media. What used to be friends and family has become an eternal screen of AI-generated black-belt karate cats, or deepfakes designed to provoke anger and outrage.

Maybe it’s not so bad, after all, if all these apps end up in their own death spiral. Which means we’ll have to find something else to do. Perhaps read a book. The good thing is, you can ask ChatGPT for reading recommendations.

It's read everything.

OpenAI's Holiday Stress

Sam Altman has declared "code red" for OpenAI. After Google launched Gemini 3 and leapfrogged competitor ChatGPT, Altman has asked employees to prioritize.

Moving ahead, all efforts will be focused on making ChatGPT even better. This means other initiatives, like AI agents and advertising opportunities, are being paused. It may be a smart decision giving ChatGPT some extra love: Recently, OpenAI has launched a social media platform (Sora), a browser (Atlas) with varying success. And Sam Altman is vocal about launching a physical AI product on the market. Meanwhile, Google which long seemed headed for the backwaters, is the one running the show these days.

A Little Dystopia

In China, smartwatches have become popular among children. The brand Little Genius has taken over and accounts for half of the global market for children's smartwatches.

But Little Genius—or Evil Genius, if you ask me—is a wolf in sheep’s clothing. The watch is used to buy goods, share videos with friends, and chat. But at the centre of Little Genius lies "likes." Children accumulate likes from their friends and level up, which in turn allows them to give out more likes. The children at the lowest levels only gets to distribute five likes. In order to rise through the ranks, other children need to gift you one of their thumbs-ups. Little Genius has created a massive popularity divide among Chinese children. If you're popular enough, you can be generous with your 150 likes. If you're not so lucky, think carefully about where you use the five reactions you have. If you don't get one in return, all hopes for popularity are dashed.