In an amazing act of self-criticism, one of the architects of Transformers technology that holds power Chat GPTfor , for , for , . Claudeand virtually every major AI system told an audience of industry leaders this week that artificial intelligence research has narrowed dangerously — and that it’s outgrowing its creation.
Lillian Joneswho co-authored the seminal 2017 paper "Attention is what you need" And even coined the name "transformer," Provided an unusually clear assessment on this TED AI Conference Tuesday in San Francisco: Nevertheless Extraordinary investment And with talent flooding into AI, the field has built calculations around a single architectural approach, potentially blinding researchers to the next big breakthrough.
"Despite the fact that there has never been so much interest, resources and money and talent, it has caused us to curtail the research we are doing." Jones told the audience. He said he is guilty "An enormous amount of pressure" Investors demand returns and researchers stand out in a crowded field.
The warning carries particular weight given Jones’ role in AI history. Transformer architecture It has helped Google develop what has become the foundation of the generative AI boom, enabling systems that can write articles, generate images and engage in human-like conversations. His paper has been Cited over 100,000 timesmaking it one of the most influential computer science publications of the century.
Now, as CTO and co-founder of Tokyo-based K Sakana AiJones is clearly abandoning his creation. "I personally decided at the beginning of this year that I was going to drastically reduce the amount of time I spend on Transformers," He said. "I’m obviously looking and finding the next big thing now."
According to Transformer Pioneer, more AI funding has led to less creative research
Jones paints a picture of the AI ​​research community with what he calls a paradox: more resources have led to less creativity. They told the researchers to constantly check whether they were staying or not "Scoped" By competitors working on similar ideas, and by academics, who choose safe, publishing projects over risky, potentially volatile ones.
"If you’re doing standard AI research right now, you have to think that maybe three or four other groups are doing something similar, or maybe exactly the same," Jones described an environment where "Unfortunately, this pressure hurts science, because people are rushing their papers, and it reduces the amount of creativity."
He drew an analogy with Ai himself "Exploitation vs Exploitation" A trade-off that governs how algorithms find solutions. When a system exploits too much and explores too little, it finds trivial local solutions while missing superior alternatives. "We’re definitely in that situation right now in the AI ​​industry," Jones argued.
The implications are dire. Jones harkens back to the period just before Transformers emerged, when researchers were endlessly tweaking neural networks — the previous dominant architecture — for added benefits. Once the Transformers arrived, all those tasks suddenly seemed irrelevant. "How much time do you think these researchers would have spent trying to iteratively improve neural networks if they knew something like Transformers was around the corner?" he asked.
He fears the field is repeating the pattern. "I’m worried that we’re in a situation right now where we’re just focusing on one architecture and just allowing it and trying different things, where there could be a breakthrough around the corner."
The ‘Attention You All Need’ paper was born of freedom, not pressure
To emphasize his point, Jones describes the conditions that allowed Transformers to emerge in the first place – a stark contrast to today’s environment. He said, there was a project "Very organic, down," Born from "Talking over lunch or randomly scribbling on the whiteboard at the office."
Critically, "We didn’t really have a great idea, we really had the freedom to spend time and go and work on it, and more importantly, we didn’t have any pressure that was coming from management," Jones said. "No pressure to work on a specific project, publish multiple papers to push a specific metric."
That freedom is largely absent today, Jones suggests. Even researchers recruited for astronomical salaries – "Literally a million dollars a year, in some cases" – May not feel empowered to take risks. "Do you think they feel empowered to try their wildest ideas and more speculative when they start their new position, or do they feel immense pressure to prove their mettle and go for the low-hanging fruit once again?" he asked.
Why an AI lab is betting that freedom of research beats million-dollar salaries
Jones’s proposed solution is deliberately provocative: "Explore the dial" And share results openly even at competitive cost. He recognized the irony of his position. "It might seem a little controversial to have one of the Transformers writers stand up on stage and tell you that he’s absolutely sick of them, but that’s fair enough, right? I’ve been working on them longer than anyone, with the possible exception of Seven."
at Sakana AiJones said he is trying to recreate the former Transformers environment, with minimal pressure to pursue nature-inspired research and publications or compete directly with competitors. He offered researchers a mantra from engineer Brian Cheung: "You should only do the research that wouldn’t happen if you weren’t doing it."
An example is Sakana "Continuous thought machinefor , for , for , ." which adds brain-like coherence to neural networks. An employee who pitched the idea told Jones he would face skepticism and pressure not to waste time on previous employers or academic positions. At Sakana, Jones gave him a week to explore. The project was successful enough to be highlighted Neuropsa major AI conference.
Jones even suggests that freedom beats compensation in recruitment. "It’s a really, really good way to acquire skills," He said about the research environment. "Think about it, talented, intelligent people, ambitious people, will naturally seek this kind of environment."
Maybe the success of Transformers will block the next advancement of AI
Perhaps most provocatively, Jones suggests that Transformers may be a victim of their own success. "The fact that current technology is so powerful and flexible … prevented us from finding better," He said. "It makes sense that if current technology were bad, more people would be looking for something better."
He was careful to clarify that he was not dismissing ongoing Transformer research. "There is still significant work to be done on the current technology and to bring enormous value in the years to come." He said. "I’m just saying that given the amount of talent and resources we have, we can afford to do a lot."
His final message was one of cooperation rather than competition. "Honestly, from my point of view, it’s not a contest," Jones concluded. "We all have the same goal. We all want to see this technology progress so that we can all benefit from it. So if we could all collectively turn the explore dial and then openly share what we find, we could achieve our goal much faster."
The high stakes of AI’s search problem
These remarks come at a critical moment for artificial intelligence. The industry is saddled with growing evidence that only large transformer models are produced May be approaching diminishing returns. Leading researchers have openly debated whether the current paradigm has fundamental limitations, with some suggesting that architectural innovations — not just scale — will be required for continued progress toward more capable AI systems.
Jones’ warning suggests that exploring these innovations may require dismantling the very incentive structures that have driven AI’s recent boom. with the Tens of billions of dollars flow into AI development annually And the tight competition between labs driving secrecy and fast publication cycles, which defined the research environment it defined.
Yet his inner vision carries extraordinary weight. As someone who helped create the technology that now dominates the field, Jones understands both what is needed to achieve breakthrough innovation and what the industry risks by abandoning that approach. His decision to walk away from Transformers — the architecture that built his reputation — adds credibility to a message that might otherwise feel like a contradictory position.
Whether AI’s power players will heed the call is uncertain. But Jones offered an important reminder of what’s at stake: The next transformer-scale breakthrough may be just around the corner, one that researchers seek with the freedom to explore. Or it could be relentless as thousands of researchers race to publish incremental improvements on an architecture that, in Jones’ words, is one of its creators. "Totally sick"
After all, he’s been working on Transformers more than almost anyone. He’ll know when it’s time to move on.