🤖 AI & Beyond

Recent Research by Google DeepMind Presented at ICML 2023

Exploring AI Safety, Adaptability, and Efficiency for the Real World

Next week marks the start of the 40th International Conference on Machine Learning (ICML 2023), taking place 23-29 July in Honolulu, Hawai’i. ICML brings together the artificial intelligence (AI) community to share new ideas, tools, and datasets, and make connections to advance the field. From computer vision to robotics, researchers from around the world will be presenting their latest advances.

Our director for science, technology & society, Shakir Mohamed, will give a talk on machine learning with social purpose, tackling challenges from healthcare and climate by taking a sociotechnical view and strengthening global communities.

We are proud to support the conference as a Platinum Sponsor and to continue collaborating with our long-term partners LatinX in AI, Queer in AI, and Women in Machine Learning. At the conference, we’re also showcasing demos on AlphaFold, our advances in fusion science, and new models like PaLM-E for robotics and Phenaki for generating video from text.

Google DeepMind researchers are presenting more than 80 new papers at ICML this year. As many papers were submitted before Google Brain and DeepMind joined forces, papers initially submitted under a Google Brain affiliation will be included in a Google Research blog, while this blog features papers submitted under a DeepMind affiliation.

AI in the (Simulated) World

The success of AI that can read, write, and create is underpinned by foundation models – AI systems trained on vast datasets that can learn to perform many tasks. Our latest research explores how we can translate these efforts into the real world and lays the groundwork for more generally capable and embodied AI agents that can better understand the dynamics of the world, opening up new possibilities for more useful AI tools.

In an oral presentation, we introduce AdA, an AI agent that can adapt to solve new problems in a simulated environment, similar to human capabilities. In minutes, AdA can take on challenging tasks: combining objects in novel ways, navigating unseen terrains, and cooperating with other players. Likewise, we show how we could use vision-language models to help train embodied agents – for example, by instructing a robot on its actions.

The Future of Reinforcement Learning

To develop responsible and trustworthy AI, we must understand the goals at the heart of these systems. In reinforcement learning, one way this can be defined is through reward.

In an oral presentation, we aim to settle the reward hypothesis first posited by Richard Sutton, asserting that all goals can be viewed as maximizing expected cumulative reward. We clarify the precise conditions under which it holds and the kinds of objectives that can – and cannot – be captured by reward in a general form of the reinforcement learning problem.

When deploying AI systems, they need to be robust enough for the real world. We explore how to better train reinforcement learning algorithms within constraints, as AI tools often have to be limited for safety and efficiency. Our research, which received an ICML 2023 Outstanding Paper Award, investigates how we can teach models complex long-term strategy under uncertainty using imperfect information games. We demonstrate how models can play to win in two-player games, even without knowledge of the other player’s position and possible moves.

Challenges at the Frontier of AI

Humans can easily learn, adapt, and understand the world around them. Developing advanced AI systems that can generalize in human-like ways will help create AI tools we can use in our everyday lives and to tackle new challenges. One way that AI adapts is by quickly changing its predictions in response to new information. In an oral presentation, we examine plasticity in neural networks and how it can be lost over the course of training – along with ways to mitigate this loss.

We further present research that may help elucidate the type of in-context learning that emerges in large language models by analyzing neural networks meta-trained on data sources whose statistics change spontaneously, such as in natural language prediction. In an oral presentation, we introduce a new family of recurrent neural networks (RNNs) that outperform on long-term reasoning tasks, unlocking the potential of these models for the future.

Lastly, in ‘quantile credit assignment,’ we propose an approach to disentangle luck from skill. By establishing a clearer relationship between actions, outcomes, and external factors, AI can better comprehend complex, real-world environments.