Learning when data is hard to acquire

Research into nuclear fusion is currently limited by researchers’ ability to run experiments. While there are dozens of active tokamaks around the world, they’re expensive machines and in high demand. For example, TCV can only sustain the plasma in a single experiment for up to three seconds, after which it needs 15 minutes to cool down and reset before the next attempt. Not only that, multiple research groups often share use of the tokamak, further limiting the time available for experiments.

Given the current obstacles to access a tokamak, researchers have turned to simulators to help advance research. For example, our partners at EPFL have built a powerful set of simulation tools that model the dynamics of tokamaks. We were able to use these to allow our RL system to learn to control TCV in simulation and then validate our results on the real TCV, showing we could successfully sculpt the plasma into the desired shapes. Whilst this is a cheaper and more convenient way to train our controllers; we still had to overcome many barriers. For example, plasma simulators are slow and require many hours of computer time to simulate one second of real time. In addition, the condition of TCV can change from day to day, requiring us to develop algorithmic improvements, both physical and simulated, and to adapt to the realities of the hardware.

Success by prioritising simplicity and flexibility

Existing plasma-control systems are complex, requiring separate controllers for each of TCV’s 19 magnetic coils. Each controller uses algorithms to estimate the properties of the plasma in real time and adjust the voltage of the magnets accordingly. In contrast, our architecture uses a single neural network to control all of the coils at once, automatically learning which voltages are the best to achieve a plasma configuration directly from sensors.

As a demonstration, we first showed that we could manipulate many aspects of the plasma with a single controller.