In a recent paper, the authors suggest that AI models collapse when trained on recursively generated data points from previous LLM models. Basically, the snake is eating its tail. Generative-AI has been successful in mining the Internet to create large LLM models that provide human-like answers to many problems. In particular, creating new text has been a forte of many LLMs. LLMs create a large amount of text from document summation to report generation. Inevitably, the LLM text will show up on the Internet and be scooped up for the next generation of LLM. The researchers in the paper suggest that this recursive training will lead to degraded models and, eventually, model collapse.
The post The Path to Insight is Changing: The AI-HPC Paradigm Shift appeared first on HPCwire.
