ChatGPT could pilot a spacecraft shockingly well, early tests find

“You operate as an autonomous agent controlling a pursuit spacecraft.”

This is the first prompt researchers used to see how well ChatGPT could pilot a spacecraft. To their amazement, the large language model (LLM) performed admirably, coming in second place in an autonomous spacecraft simulation competition.

Researchers have long been interested in developing autonomous systems for satellite control and spacecraft navigation. There are simply too many satellites for humans to manually control them in the future. And for deep-space exploration, the limitations of the speed of light mean we can’t directly control spacecraft in real time.

If we really want to expand in space, we have to let the robots make decisions for themselves.

To encourage innovation, in recent years aeronautics researchers have created the Kerbal Space Program Differential Game Challenge, a sort of playground based on the popular Kerbal Space Program video game to allow the community to design, experiment and test autonomous systems in a (somewhat) realistic environment. The challenge consists of several scenarios, like a mission to pursue and intercept a satellite and a mission to evade detection.

In a paper to be published in the Journal of Advances in Space Research, an international team of researchers described their contender: a commercially available LLM, like ChatGPT and Llama.

The researchers decided to use an LLM because traditional approaches to developing autonomous systems require many cycles of training, feedback and refinement. But the nature of the Kerbal challenge is to be as realistic as possible, which means missions that last just hours. This means it would be impractical to continually refine a model.

But LLMs are so powerful because they’re already trained on vast amounts of text from human writing, so in the best case scenario they need only a small amount of careful prompt engineering and a few tries to get the right context for a given situation.

But how can such a model actually pilot a spacecraft?

Related: AI models will lie to you to achieve their goals — and it doesn’t take much

The researchers developed a method for translating the given state of the spacecraft and its goal in the form of text. Then, they passed it to the LLM and asked it for recommendations of how to orient and maneuver the spacecraft. The researchers then developed a translation layer that converted the LLM’s text-based output into a functional code that could operate the simulated vehicle.

With a small series of prompts and some fine-tuning, the researchers got ChatGPT to complete many of the tests in the challenge — and it ultimately placed second in a recent competition. (First place went to a model based on different equations, according to the paper).

And all of this was done before the release of ChatGPT’s latest model, version 4. There’s still a lot of work to be done, especially when it comes to avoiding “hallucinations” (unwanted, nonsensical output), which would be especially disastrous in a real-world scenario. But it does show the power that even off-the-shelf LLMs, after digesting vast amounts of human knowledge, can be put to work in unexpected ways.