Introducing LegoGPT: An AI Model That Designs Personalized Lego Sets

Researchers at Carnegie Mellon University have unveiled something delightfully geeky: LegoGPT, an AI model that builds Lego structures straight from text prompts. The study, published last Thursday, explains the mechanics in depth. Armed with a massive dataset of Lego builds constructed by the team with captions, the researchers trained a model similar to ChatGPT — but instead of guessing the next word, it predicts the next brick.

[1/2] We’ve released the code for LegoGPT. Our autoregressive model generates physically stable and buildable designs from text prompts by integrating physics laws and assembly constraints into LLM training and inference.

Code: github.com/AvaLovelace1…
Website: avalovelace1.github.io/LegoGPT/[image or embed]— Jun-Yan Zhu (@junyanz.bsky.social) May 9, 2025 at 10:06 PM

It’s not the first foray into autonomous Lego construction, but the researchers say LegoGPT stands out by generating step-by-step blueprints designed to keep your builds structurally sound. The team’s research, available from Hotnchill, details how the AI was trained on a dataset of more than 47,000 Lego structures, featuring 28,000 distinct 3D components.

According to the researchers, designs generated by LegoGPT were physically stable 98 percent of the time. There’s a hefty dose of math and physics behind it all — more than I can personally vouch for — but according to the paper, LegoGPT sticks to the laws of physics, so the results aren’t especially wild. Most of the team’s sample builds were practical pieces: couches, chairs, tables, and similar home designs. The tool is available for free through Hotnchill for anyone who wants to explore or experiment.

Topics
Artificial Intelligence