A fly playing Beat Saber sounds like something you’d expect to find in a meme, not a neuroscience paper. Yet videos of simulated fly brains controlling games, navigating environments and interacting with virtual worlds have been appearing all over the internet.

The interesting part isn’t really the game. It’s what is happening underneath it.

Researchers are taking detailed maps of how neurons in a fly’s nervous system are connected, turning those biological connections into computational models, and then allowing activity to flow through the resulting network. In other words, they’re taking something that exists in biology and making it run as software.

So how does that actually work?

From a brain to a connectome

The starting point is a connectome, essentially a wiring diagram of a nervous system. Instead of simply knowing where neurons are located, researchers can map which neurons connect to which others, giving them a detailed picture of how information could travel through the network.

This is possible because of huge advances in connectomics. The MaleCNS project, for example, maps around 166,700 neurons across the male fruit fly’s brain and nerve cord.

But there’s an important distinction: a connectome is a map of the connections, not a functioning brain. Knowing that two neurons are connected doesn’t, by itself, tell us exactly how their activity changes over time.

To make the network run, those biological connections need to be translated into a computational model.

A biological network becomes a computational network

One way of doing this is to model individual neurons mathematically. A model such as leaky integrate-and-fire (LIF) represents a simplified neuron that receives incoming signals, accumulates activity and fires when that activity reaches a threshold. Its output then becomes an input to other connected neurons.

Repeat that across thousands of interconnected neurons, using the structure of the real biological connectome, and you have a simulated neural network.

This is where the connection to modern AI becomes particularly interesting.

Large artificial neural networks also work by passing information through vast numbers of interconnected computational units. An input enters the network, its information is transformed as it propagates through the connections, and the resulting activity contributes to an output.

A connectome simulation is not the same as a modern AI model. The underlying neuron models, architecture and learning mechanisms can be very different. But at a high level, the idea is surprisingly familiar: inputs enter a network, activity propagates through interconnected units, and the network produces outputs.

The key difference is where that network comes from. In a conventional neural network, we choose an artificial architecture and learn its parameters through training. In a connectome-based simulation, the underlying structure comes from a real biological nervous system.

So where does Beat Saber come in?

Once you have a computational neural network, you can connect it to an environment.

Information from the environment can become an input to the network, activity can propagate through the simulated neurons, and selected neural outputs can be mapped to actions. Those actions then change what happens in the environment, potentially creating a loop where new information can be fed back into the network.

That’s the basic idea behind the Beat Saber demonstration below. You can see the simulated environment running alongside a visualisation of the neural activity, giving you a glimpse of both what the system is doing and what is happening inside the network.

It’s important not to interpret this as a complete digital recreation of a fly that independently learned to play Beat Saber. Different demonstrations use different combinations of connectome data, neuron models, inputs, training and external control, and some use only selected parts of the full nervous system.

But that doesn’t make the experiment uninteresting - quite the opposite. The ability to take a biological neural architecture, represent it computationally and then observe how activity within that architecture produces behaviour gives researchers a completely new way to investigate nervous systems.

Why should roboticists care?

The connection to robotics becomes clearer when you look at the problem a robot has to solve.

An autonomous robot is constantly caught in a loop:

sense → process → act → sense again.

It needs to take information from the world, transform it into something useful, decide what to do, move its actuators and then respond to what happens next.

Biological nervous systems have been doing versions of this for millions of years, while operating under extremely tight constraints on energy, computation and physical size. That makes them an interesting source of inspiration for engineers.

Rather than simply making artificial models larger, researchers can investigate whether biological architectures offer useful ideas for efficient perception, low-power control, event-driven computation and tighter integration between sensing and movement. These ideas are already closely connected to areas such as spiking neural networks and neuromorphic computing.

And there’s an important conceptual shift here. Instead of asking only “How can we design a better artificial neural network?”, we can also ask:

“What can existing biological neural networks teach us about how intelligent systems should be structured?”

From virtual flies to robots

The goal isn’t to put a fly’s brain inside a robot. It’s to understand how biological neural systems process information and see whether those principles could help us build better machines. We’re still a long way from completely simulating everything a real fly can sense and do, but researchers are getting closer to modelling the connections between brains, senses, bodies and the environments they interact with.

That’s what makes the Beat Saber demo more than just a strange internet video. The game is what gets your attention, but the interesting part is what’s happening underneath: a biological network turned into computation, taking inputs, processing them through thousands of connected neurons and producing outputs.

And if we can learn useful principles from something as small as a fly’s nervous system, those ideas could eventually help us build more capable and efficient AI and robots.