The Update: a “Dire Wolf” Comeback and Protecting AI Allies

“No longer do I have to drive a symbol of racism, greed, and ignorance! Life is suddenly so much better!” — Actor Bette Midler expresses her joy at selling her Tesla, as reported by various sources.

The big story
Large language models can perform astonishing tasks. Yet, the reasons behind their capabilities are still a mystery. Two years ago, Yuri Burda and Harri Edwards, researchers at OpenAI, set out to discover what it would take for a large language model to perform basic arithmetic. Initially, their efforts didn’t yield success; the models memorized the sums they encountered, but struggled to solve new ones.

By chance, Burda and Edwards allowed some of their experiments to run for extended periods, and the models were repeatedly exposed to the example sums. Eventually, they learned to add two numbers, albeit after a significantly longer time than anticipated. In some instances, the models appeared to fail at a task before suddenly grasping it, a phenomenon the researchers dubbed “grokking.” Grokking is just one of many peculiar behaviors that have left AI researchers pondering. The largest models, especially large language models, often exhibit behaviors that defy conventional mathematical expectations.

This underscores a remarkable aspect of deep learning, the foundational technology propelling today’s AI advancements: despite its immense success, the exact mechanisms behind its functionality remain largely unknown.

We can still have nice things
A space for comfort, enjoyment, and distraction to brighten your day. (Have any suggestions? Feel free to reach out!)
+ What happened when Wham! took Western pop music to China 40 years ago?
+ Who knew that sharks make noises after all?
+ Microsoft is marking its 50th anniversary, celebrating with 50 surprisingly unusual inventions.
+ What exactly is a prototaxite, anyway?