loading page

Science AMA Series: We are Stanford neuroengineers who created a neural prosthesis for monkeys to type Shakespeare with their “minds”. AUA!
  • stanford_npsl ,
  • r/Science AMAs
stanford_npsl

Corresponding Author:[email protected]

Author Profile
r/Science AMAs
Author Profile

Abstract

We are Paul Nuyujukian MD PhD (Postdoc, soon to be Bioengineering faculty at Stanford) and Jonathan Kao PhD (Postdoc, soon to be Electrical Engineering faculty at UCLA), neuroengineers in the Stanford Neural Prosthetics Systems Laboratory, which is directed by Professor Krishna Shenoy PhD. We just published a paper in the Proceedings of IEEE in which we demonstrated a high-performance neural prosthesis where monkeys transmitted text one character at a time at a rate of up to 12 words per minute. Video of monkey typing Before we get ahead of ourselves, let us assure you that monkeys don’t understand English. In the video above, the monkeys only saw the green and yellow dots, and not the black letters (which were added afterwards in post-production as a visual aid). The game engine prompted the green targets in a specific sequence that if the monkeys got correct, would spell out words and sentences that we can all understand. The video above was a selection from Hamlet, but the primary data of the paper were articles from the New York Times that the monkeys were asked to “transcribe.” All they are doing though is navigating the white cursor to the green target at every trial, and earning a liquid reward for each success. The ability of the monkeys to control the cursor with their brain was accomplished via a brain-machine interface (BMI). BMIs are systems that record from the brain and translate these measurements to useful control signals, which could be used to control a robotic limb, wheelchair, or, as was in this case, a computer cursor. In this case, the BMI has similar functionality as a one-button computer mouse: it can move in two dimensions and click. The hardware interfaces used in the BMI, neural electrodes (the one we used was the Utah mulitelectrode array), are not new. They have been around for decades. What is new are the algorithms that translate (or, as we refer to them, “decode”) the brain signals into movement of the cursor. The machine learning decoding algorithms used in this study were ones that we developed recently (cursor movement and click decoders) that significantly improve the performance of communication BMIs, enabling our monkeys to achieve rates that are 2-3 times faster than rates achievable with prior algorithms. There is tons more we could write about (algorithm details, clinical trials that these findings have resulted in, what other medical conditions BMIs may help with, etc), but we’ll stop here and open it up to you all for questions. We look forward to answering as many of your questions as possible! 2PM PT - We are live! 6PM PT - We are done, thank you for the great questions! More videos: Dwell typing Click typing Media coverage: Stanford press release IEEE Spectrum NPR KQED - Future of You The Verge Wired UK