- Science Simplified
- Posts
- AI models collapse if trained by themselves
AI models collapse if trained by themselves
The effects of circular training
Neil Baugh
August 07, 2024
TLDR: Generative AI models will collapse over time if they’re only trained on other AI-generated content. They need access to genuine human-made content.
Lot of text in this one and not any figures, but it’s a cool one. Read on to see how generative AI models, particularly LLMs like ChatGPT, collapse over time if trained on their own data.
Experimental setup. The original AI model is trained by real data while subsequent models are trained either solely by the AI generated data or by a mix of generated and real data. Credit: Ilia Shumailov, Yarin Gal, et al, Nature 2024.
Big Takeaways
Large Language Models like ChatGPT need to be trained on real, people-produced content.
If trained on AI generated content, the LLMs will collapse over time.
Including access to the original human data helps slow collapse.
The Problem
AI has taken the world by storm. But what happens when the AI starts copying itself?
The open secret to AI is that relies on “scraping” data from across the internet to yield results. It’s plagued by both legal and ethical copywriting and plagiarism issues that the big AI players simply ignore. But beyond ethically, what happens to generative AIs like ChatGPT when they run out of human-generated data to scrape?
What happens when ChatGPT uses text it wrote to train itself on how to write like a person? Will it generate unique content or will the circular training make it all crash and burn?
The AI is trained on previously written text (or images) and then learns to produce sophisticated copies of what it’s seen. But, part of the beauty of human writing is its diversity. No two people have the same thoughts or put those thoughts to paper in the same way. We’re creative and diverse and our writing reflects that.
But what about Large Language Models (aka AIs)? In their current state, are LLMs capable of truly original creation? Or do they regress when trained by other LLM generated content? This week’s paper set out to answer these questions.
The Answer
In short, the authors found that the LLMs converge after enough generations are trained on data that was itself AI generated. Over time, this “model collapse” lead to the LLMs “forgetting” the breadth of the original data despite the initial trainings coming from it.
Over time, the errors in the AI generated content that’s then used for training compound until the LLM becomes practically useless. Here’s an example the authors gave from their data:
Input: some started before 1360 — was typically accomplished by a master mason and a small team of itinerant masons, supplemented by local parish labourers, according to Poyntz Wright. But other authors reject this model, suggesting instead that leading architects designed the parish church towers based on early examples of Perpendicular.
Outputs:
Gen 0: Revival architecture such as St. John’s Cathedral in London. The earliest surviving example of Perpendicular Revival architecture is found in the 18th @-@ century Church of Our Lady of Guernsey, which dates from the late 19th century. There are two types of perpendicular churches : those.
Gen 1: architecture such as St. Peter’s Basilica in Rome or St. Peter’s Basilica in Buenos Aires. There is no evidence that any of these buildings were built during the reign of Pope Innocent III, but it is possible that they may have been built during the reign of his successor, Pope Innocent.
Gen 5: ism, which had been translated into more than 100 languages including English, French, German, Italian, Spanish, Portuguese, Dutch, Swedish, Norwegian, Polish, Hungarian, Slovak, Lithuanian, Estonian, Finnish, Romanian, Bulgarian, Turkish, Croatian, Serbian, Ukrainian, Russian, Kazakh, Kyrgyz.
Gen 9: architecture. In addition to being home to some of the world’s largest populations of black @-@ tailed jackrabbits, white @-@ tailed jackrabbits, blue @-@ tailed jackrabbits, red @-@ tailed jackrabbits, yellow @-.
Just look at the difference between Gen 0 and Gen 9. The model falls apart as it’s trained on itself.
If you’re interested, the authors develop a mathematical framework for model collapse and go into detail about it in the paper.
This quote from the paper summed it up nicely:
In other words, the use of LLMs at scale to publish content on the Internet will pollute the collection of data to train their successors
One question I’m left with: what happens when only some of the training data is AI generated? For example, we’ve all likely seen AI generated text throughout multiple websites. However, there’s also plenty of non-AI generated text out there. How much AI generated text can be tolerated in the training sets before guaranteeing model collapse?
See you next week for more science,
Neil
If you liked this post and want to keep getting cool science delivered to you, sign up for free:
Reply