A new chip from MIT lets miniature robots and low-power devices construct detailed 3D maps of their surroundings in real time while consuming roughly the energy of a single LED light — a breakthrough that could reshape robotics, augmented reality and beyond.
Researchers at MIT have developed a specialized chip that allows small autonomous robots and battery-constrained devices to build accurate, real-time 3D maps of their surroundings while consuming only about 6 milliwatts of power — roughly what it takes to light a single LED. The chip, called Gleanmer, was recently presented at the IEEE Very Large-Scale Integrated Circuits Symposium and could open new frontiers in robotics, augmented reality and autonomous navigation.
Why This Chip Is Different
Building a 3D map of an environment in real time is computationally expensive work. Traditional approaches require a robot to store full camera images in memory and process every three-dimensional pixel — known as a voxel — multiple times. For small, battery-powered devices like tiny drones or lightweight AR headsets, that kind of energy and memory demand has historically been a dealbreaker.
The MIT team tackled this challenge by rethinking both the algorithm and the hardware at the same time — a strategy known as co-design. Instead of relying on rigid cube-shaped voxels, Gleanmer uses flexible, blob-shaped mathematical constructs called Gaussians, or ellipsoids, to represent obstacles in a robot’s environment. Because ellipsoids can stretch and curve to match the shape of real-world objects, a single Gaussian can capture a region that would otherwise require dozens of voxels, dramatically shrinking the size of the resulting map.
Senior author Vivienne Sze, a professor in the Department of Electrical Engineering and Computer Science (EECS) and a member of MIT’s Research Laboratory of Electronics, described the approach as a model for future chip development.
“This paper showcases a key example of how you can leverage co-design of the algorithm and hardware to really push energy efficiency. While there has been a lot of work looking into compact 3D maps, what stands out about this work is that it also ensures that the process to generate those maps is as efficient as possible. Our chip allows you to store very large maps in a very small space, and do it in a very energy efficient manner,” Sze said in a news release.
How the Algorithm Works
One of the key innovations behind Gleanmer is a single-pass image processing technique. Conventional systems must load and process each depth image multiple times to properly shape the Gaussians. The MIT approach instead assumes that adjacent pixels belong to the same Gaussian cluster, meaning the chip only has to compare each pixel to its immediate neighbors rather than to every other pixel in the frame. Once that single pass is complete, the image can be discarded — the chip never has to hold an entire image in memory at once.
Co-lead author Peter Zhi Xuan Li, a graduate student at MIT, explained that this dramatically cuts down on memory usage.
“At any point in time, we only need to store a few pixels in memory, which significantly reduces the memory footprint our algorithm requires,” Li said in the news release.
As a robot moves through a space, it inevitably views the same objects from multiple angles, causing some Gaussians to overlap. Merging those overlapping Gaussians is necessary to keep the map compact, but doing so traditionally requires revisiting stored pixel data. The researchers developed a technique to perform that fusion directly on the Gaussians themselves, bypassing the need to retrieve the original images and cutting memory and power requirements even further.
Co-lead author Zih-Sing Fu, also a MIT graduate student, noted how this architecture streamlines data access.
“By having a dedicated memory that just stores the objects you’ve seen in the previous few frames, you can access the data much more efficiently,” Fu said.
The chip keeps the Gaussians it is actively processing in small, fast on-chip memory located right next to the computational units, avoiding the energy cost of fetching data from slower, off-chip storage.
Putting It to the Test
The researchers validated Gleanmer by having it reconstruct a variety of pre-existing 3D environments and by feeding it live data streamed directly from an iPhone camera. In both cases, the chip produced detailed 3D maps in real time at approximately 6 milliwatts — just 2.5% of the power consumed by the best existing chip designed for the same task. When it came to planning a safe, collision-free trajectory, Gleanmer’s compact map representation allowed a robot to chart its path using only about 20% of the energy a conventional approach would require.
Li summed up the design philosophy concisely.
“We reduce the memory consumption by making sure the algorithm is efficient. Then we accelerate the workload that is performed by that efficient algorithm, so in the end, our chip is as efficient as possible,” Lie said.
Why It Matters for Students and Young Professionals
The implications of this chip stretch well beyond industrial drone inspections. For students in engineering, computer science or robotics programs, Gleanmer represents a real-world example of how algorithm design and hardware engineering can be co-optimized to solve problems that neither field could crack alone. The chip’s low power draw also makes it an attractive candidate for lightweight augmented reality headsets — the kind that could one day be worn comfortably during a medical simulation class or while guiding a technician through a complex repair procedure.
As AR hardware continues to shrink and battery life becomes an ever-more-critical constraint, breakthroughs like Gleanmer could be what ultimately makes immersive, wearable computing practical for everyday educational and professional use. The research team plans to push efficiency even further by moving processing units closer to the sensors that gather environmental data, and they are exploring whether Gaussians could help AI systems reason about complex architectural blueprints.
The work was supported in part by the MIT-MathWorks Fellowship, Amazon, the U.S. National Science Foundation, and Intel. The paper, titled “Gleanmer: A 6 mW SoC for Real-Time 3D Gaussian Occupancy Mapping,” is available on arXiv.