|
@sergeydoestweet | |||||
|
#tweeprint of our new preprint: we found speech-related neural activity obtained using multielectrode recordings in the dorsal (!) motor cortex of people with paralysis. biorxiv.org/content/early/…
1/
|
||||||
|
||||||
|
Sergey Stavisky
@sergeydoestweet
|
30. pro 2018. |
|
It’d be great to observe what motor cortical populations do during speaking with single-neuron resolution, but how? There’s no direct animal model, and intracortical recording requires surgery.
2/
|
||
|
|
||
|
Sergey Stavisky
@sergeydoestweet
|
30. pro 2018. |
|
Our in was that we already have multielectrode arrays chronically placed in people with tetraplegia as part of the @BrainGateTeam clinical trial to restore movement and communication.
3/
|
||
|
|
||
|
Sergey Stavisky
@sergeydoestweet
|
30. pro 2018. |
|
The “problem” (opportunity!) was that these arrays are in the dorsal ‘hand knob’ area involved in arm and hand movements — not speech. But as far as we know, no one had actually looked for speech-related activity there.
4/
|
||
|
|
||
|
Sergey Stavisky
@sergeydoestweet
|
30. pro 2018. |
|
We asked two of our participants, who are able to speak, to do a simple prompted speaking task.
5/ pic.twitter.com/luR8Ebl67M
|
||
|
|
||
|
Sergey Stavisky
@sergeydoestweet
|
30. pro 2018. |
|
The very cool, and surprising, result was that lo and behold, there was strong firing rate modulation on many electrodes when producing different speech sounds. Here’s an example neuron's spike rasters and trial-averaged firing rates:
6/ pic.twitter.com/ek0ScvJSJq
|
||
|
|
||
|
Sergey Stavisky
@sergeydoestweet
|
30. pro 2018. |
|
Most neurons also responded to simple (non-speaking) movements of the lips, jaw, and tongue, so we think this activity reflects motor control of the articulators, rather than higher-level processes like language.
7/ pic.twitter.com/0SAb4Ahd7L
|
||
|
|
||
|
Sergey Stavisky
@sergeydoestweet
|
30. pro 2018. |
|
Could these signals be useful for building speech BCIs to help people who cannot speak? We think so! We could decode amongst a set of syllables pretty well, despite having only 200 electrodes in a sub-optimal part of the brain.
8/ pic.twitter.com/dltUsK2Mvo
|
||
|
|
||
|
Sergey Stavisky
@sergeydoestweet
|
30. pro 2018. |
|
This makes me optimistic about intracortical approaches for speech BCIs, but there’s still a long road from this limited proof of feasibility to a real-time system synthesizing continuous speech. Recent work from the ECoG folks is paving the way in this domain.
9/
|
||
|
|
||
|
Sergey Stavisky
@sergeydoestweet
|
30. pro 2018. |
|
What about the neural population dynamics during speaking? There are a lot of interesting questions one could ask. As newcomers to speech from the arm control field, we chose to start by bringing what we know.
10/
|
||
|
|
||
|
Sergey Stavisky
@sergeydoestweet
|
30. pro 2018. |
|
We tested for whether two major motor cortical population dynamics features previously described by our group during arm movements were also present during speaking.
11/
|
||
|
|
||
|
Sergey Stavisky
@sergeydoestweet
|
30. pro 2018. |
|
Neural dynamics motif #1: Population activity is dominated by a large condition-invariant signal (CIS) when initiating speaking. Here we applied the methods of Kaufman et al 2016 (@MattAntimatt):
12/ pic.twitter.com/t2w2db72iP
|
||
|
|
||
|
Sergey Stavisky
@sergeydoestweet
|
30. pro 2018. |
|
Neural dynamics motif #2: Strong rotatory dynamics are present during speech production. Here we applied the methods of Churchland, Cunningham et al 2012: pic.twitter.com/cVh5CKQgx1
|
||
|
|
||
|
Sergey Stavisky
@sergeydoestweet
|
30. pro 2018. |
|
Why this is the case is a fascinating open question. It may be that if these are effective computational strategies for generating the right muscle commands for arm and hand movements, then they’re also well-suited for generating speech-producing muscle commands.
14/
|
||
|
|
||
|
Sergey Stavisky
@sergeydoestweet
|
30. pro 2018. |
|
Another open question is whether finding this activity in "arm" motor cortex is due to remapping because of the participants' tetraplegia. We don't think this is the case, but a definite answer is elusive because we'd need intracortical recording in able-bodied people. 15/15
|
||
|
|
||