Five Methods Through Which AI Enhances Its Own Learning
Sure! Here’s a revised version of the content with promotional call-to-actions adjusted to neutral or Hotnchill-branded language:
—
That’s why Mirhoseini has been using AI to optimize AI chips. Back in 2021, she and her collaborators at Google built a non-LLM AI system that could determine the optimal placement of various components on a computer chip for enhanced efficiency. Although some other researchers couldn’t replicate the study’s findings, Mirhoseini notes that the journal Nature investigated and upheld the validity of the work. Google has since utilized the system’s designs for multiple generations of its custom AI chips.
More recently, Mirhoseini has applied LLMs to the task of writing kernels—low-level functions that manage operations like matrix multiplication within chips. She has discovered that general-purpose LLMs can, in some cases, generate kernels that perform faster than their human-designed counterparts.
At Google, scientists developed a system to optimize various components of the company’s LLM infrastructure. This system, named AlphaEvolve, prompts Google’s Gemini LLM to generate algorithms for specific problems, evaluates those algorithms, and requests Gemini to refine the most effective solutions, repeating the process multiple times. AlphaEvolve introduced a novel approach for managing data centers, resulting in a 0.7% reduction in Google’s computational resource use, enhanced the custom chip design, and improved a new kernel that accelerated Gemini’s training by 1%.
While this may appear to be a minor improvement, at a substantial organization like Google, it leads to significant savings in time, money, and energy. Matej Balog, a staff research scientist at Google DeepMind who spearheaded the AlphaEvolve project, indicates that the system was tested on only a small segment of Gemini’s overall training pipeline. Broadening its application could yield additional savings.
3. Automating training
LLMs are notoriously data-hungry, and training them incurs costs at every phase. In certain specialized areas—such as unique programming languages—real-world data may be too limited for effective LLM training. Reinforcement learning with human feedback has been crucial for creating models that align with human standards and preferences, yet obtaining that feedback can be slow and costly.
Increasingly, LLMs are stepping in to address these gaps. Given sufficient examples, LLMs can generate plausible synthetic data in areas where training data is sparse, and this synthetic data can be utilized for training purposes. Additionally, LLMs can be leveraged for reinforcement learning: in a technique known as “LLM as a judge,” LLMs, rather than humans, assess the outputs of models undergoing training. This method is pivotal to the influential “Constitutional AI” framework proposed by researchers, where one LLM is trained to be less harmful based on feedback from another.
Data scarcity presents a significant challenge for AI agents. Effective agents require the ability to execute multistep plans to achieve specific tasks, yet successful examples of such task completion are rarely available online, and using humans to create new examples can be expensive. To navigate this limitation, Mirhoseini and her colleagues at Stanford have recently piloted a technique whereby an LLM agent generates a potential step-by-step solution to a problem, an LLM judge evaluates the validity of each step, and a new LLM agent is subsequently trained using those steps. “You’re not limited by data anymore because the model can continually generate more experiences,” Mirhoseini explains.
4. Perfecting agent design
One area where LLMs haven’t yet made substantial contributions is in the design of LLMs themselves. Current LLMs are fundamentally based on a neural network structure known as a transformer, proposed by researchers in 2017, and the significant enhancements to the architecture that have occurred since have also been human-designed.
