Imagine you are typing a sentence. But instead of a keyboard, joystick, or eye tracker, a device surrounding your head reads your intentions and generates that sentence on screen.
What’s new: Researchers presented Brain2Qwerty v2, an updated version of their previous system that can translate brain waves into text. The team included contributors at Meta, French National Centre for Scientific Research, Hospital Foundation Adolphe de Rothschild, Basque Center on Cognition, Brain and Language, Paris Cité University, and the French Institute for Research in Computer Science and Automation.
How it works: Brain2Qwerty v2 first (i) breaks brain activity into characters using an encoder, then (ii) converts character embeddings into word embeddings using a model the authors call an aligner, and finally (iii) corrects those words using a fine-tuned language model (Qwen3-4B). To train the system, the authors recorded the brain activity of 9 subjects typing sentences in English, totaling 90 hours or 22 thousand examples. They recorded it using a magnetoencephalography (MEG) machine, a non-invasive device that records a brain's magnetic activity.
- Given a recording of brain activity, an encoder composed of a convolutional neural network (CNN) followed by a CNN/transformer hybrid called a conformer generates character embeddings and classifies brain activity into characters. The encoder learned by minimizing the difference between generated and actual sequences of characters.
- Given the character embeddings, the aligner used a vanilla neural network to re-embed them. The aligner grouped these embeddings by word (as predicted by spaces in the generated sequences of characters) and averaged across words to produce word embeddings. The aligner learned to increase the similarity of the word embedding to the Qwen3 embedding of the ground truth word (if the generated word was correct) and decrease its similarity with the embeddings of other words.
- Given the character sequence and the word embeddings, a fine-tuned Qwen3-4B corrected the sequence. The authors fine-tuned Qwen3-4B for each subject with a LoRA adapter to generate the correct sentence, and averaged the adapters over all subjects.
Results: Brain2Qwerty v2 outperformed its predecessor. The authors also observed that increasing the training data increased the system’s performance on their dataset.
- Brain2Qwerty v2 achieved a 39 percent word error rate (percent of words guessed wrong), whereas v1 achieved a 43 percent word error rate.
- As the amount of data increased, the character error rate (percent of characters guessed wrong) of the system’s encoder decreased. For instance, at 20 hours of training data, their encoder achieved roughly 50 percent character error rate, while at 90 hours, it achieved roughly 25 percent character error rate. The authors did not observe a plateau in performance before running out of data.
- Comparing per-subject performance, training the system on each subject individually performed significantly worse than the authors’ method. For instance, for the median performance across the nine subjects, training on just that individual achieved a word error rate of 66.5 percent, while training with their combined method achieved a word error rate of 47.8 percent.
Behind the news: The first Brain2Qwerty research program compared MEG recordings with EEG (electroencephalography), finding that MEG readings enabled more accurate predictions of text. Brain2Qwerty v2 was MEG-only, updated the architecture, and used more training data than v1. The researchers also open sourced the training code for both versions, and released the data for v1.
Why it matters: It’s unsurprising that increasing data improves performance. But it may come as a surprise that training across multiple subjects’ brain activities improves performance, even compared to training on a single individual. After all, historically in brain-computer prostheses, models are often trained for a single user. Building a system that can interpret any individual's unique brain activity, learn common patterns, and continue to improve with data suggests that models recognizing text from brain waves should be able to improve performance with more data from more subjects, much like LLMs have improved with more data over the last few years.
We're thinking: Invasive procedures like surgically planting electrodes in the brain have enabled previous patients to communicate with error rates in the single digits. While the numbers the authors report don’t yet match those of these procedures, every percent down represents progress towards a future where patients don’t need to take the risk to get brain surgery.