Researchers have designed a new type of large language model (LLM) that they propose could bridge the gap between artificial intelligence (AI) and more human-like cognition.
Called “Dragon Hatchling,” the model is designed to more accurately simulate how neurons in the brain connect and strengthen through learned experience, according to researchers from AI startup Pathway, which developed the model. They described it as the first model capable of “generalizing over time,” meaning it can automatically adjust its own neural wiring in response to new information.
“There’s a lot of ongoing discussion about specifically reasoning models, synthetic reasoning models today, whether they’re able to extend reasoning beyond patterns that they have seen in retaining data, whether they’re able to generalize reasoning to more complex reasoning patterns and longer reasoning patterns,” Adrian Kosowski, co-founder and chief scientific officer of Pathway, told the SuperDataScience podcast on Oct. 7.
“The evidence is largely inconclusive, with the general ‘no’ as the answer. Currently, machines don’t generalize reasoning as humans do, and this is the big challenge where we believe [the] architectures that we are proposing may make a real difference.”
A step towards AGI?
A key challenge is that human thinking is inherently messy. Our thoughts rarely come to us in neat, linear sequences of connected information. Instead, the human brain is more like a chaotic tangle of overlapping thoughts, sensations, emotions and impulses constantly vying for attention.
In recent years, LLMs have taken the AI industry much closer to simulating human-like reasoning. LLMs are typically driven by transformer models (transformers), a type of deep learning framework that enables AI models to make connections between words and ideas during a conversation. Transformers are the “brains” behind generative AI tools like ChatGPT, Gemini and Claude, enabling them to interact with, and respond to, users with a convincing level of “awareness” (at least, most of the time).
Although transformers are extremely sophisticated, they also mark the edge of existing generative AI capabilities. One reason for this is because they don’t learn continuously; once an LLM is trained, the parameters that govern it are locked, meaning any new knowledge needs to be added through retraining or fine-tuning. When an LLM does encounter something new, it simply generates a response based on what it already knows.
Imagine dragon
Dragon Hatchling, on the other hand, is designed to dynamically adapt its understanding beyond its training data. It does this by updating its internal connections in real time as it processes each new input, similar to how neurons strengthen or weaken over time. This could support ongoing learning, the researchers said.
Unlike typical transformer architectures, which process information sequentially through stacked layers of nodes, Dragon Hatchling’s architecture behaves more like a flexible web that reorganizes itself as new information comes to light. Tiny “neuron particles” continuously exchange information and adjust their connections, strengthening some and weakening others.
Over time, new pathways form that help the model retain what it’s learned and apply it to future situations, effectively giving it a kind of short-term memory that influences new inputs. Unlike traditional LLMs, however, Dragon Hatchling’s memory comes from continual adaptations in its architecture, rather than from stored context in its training data.
In tests, Dragon Hatchling performed similarly to GPT-2 on benchmark language modeling and translation tasks — an impressive feat for a brand-new, prototype architecture, the team noted in the study.
Although the paper has yet to be peer-reviewed, the team hopes the model could serve as a foundational step toward AI systems that learn and adapt autonomously. In theory, that could mean AI models that get smarter the longer they stay online — for better or worse.
