☞ Chang Lab
Data analysis of human speech + neural data.
Python, R, Matlab, ggplot, and more.
Summer 2017, UCSF Dept of Neurosurgery.
In summer 2017, I worked at the UCSF Chang Lab, whose research involves locating the exact centers of different functions of speech in the human brain. I adapted and evaluated numerous state-of-the-art technologies in speech transcription and diarization for the research usecase; mapped the semantic meanderings of different categories of patient conversation with word2vec; and wrote tools to quickly assess, transcribe, and phoneme-align patient speech data to be fed into machine learning algorithms alongside neural data.

☞ Research at the Lab

The Chang Lab is headed by Edward Chang, a neurosurgeon at UCSF who specializes in the treatment of intractable epilepsy, trigeminal neuralgia, and brain tumors. In addition to performing these surgeries, he leads research involving data collected during the course of these procedures.

Before an excision surgery is performed, an array of electrodes is placed on the surface of the brain underneath the skull in a procedure called Electrocorticography (ECoG). This dense array is situated right above the auditory cortex and records neural electromagnetic activity to locate seizure activity, continuously collecting data for around a week before the surgery is performed.

The auditory cortex is also the main language processing center on the brain. The lab makes use of the peripheral data collected alongside seizure activity to study language and mood. The device offers a view directly into the brain, unparalleled in both spatial and temporal resolution to any other neural imaging technology available. By combining this neural data with recordings of speech that the patient produces or perceives, we are able to discover previously unfathomable insights about human cognition. Neural markers for basic linguistic features such as pitch, voicing, and aspiration can be located to extremely focused spots on the brain. More complex functions such as mood, focus, and grammar can be mapped to specific electrodes on the ECoG as well.

As a modern research lab, one of the key methods of analysis employed by the lab is searching for patterns between speech and neural data via machine learning algorithms, which is able to quickly and efficiently garner insights about electrode activity correlated with various properties of sound both percieved and produced.

The revolutionary work done by the lab has been published multiple times in Science and Nature, as well as numerous other journals influential in the neuroscience space.

I LOVE this campus. Very very much.

☞ Projects

Over the course of the summer, I worked on several major projects along with a partner intern, creating tools to analyze and annotate collected patient speech data before combining them with neural data.

My first project involved evaluating different audio transcription and speaker diarization algorithms against manually created ground truth transcptions of the speech data in different spaces and recorded on different microphones. Quantifying the varying results given by different algorithms available was a substantial task, given that the recorded audio had substantial irrelevant background hospital noise that needed to be filtered out.

My second project involved building a Flask app GUI and a suite of algorithms built to very quickly and accurately perform word- and phoneme-alignment on large volumes of audio data. This form of alignment allows timestamps for specific features to be efficiently extracted; for example, query all instances the patient produced the sound "ah". The patients were often asked to perform various basic speech tasks, producing or perceiving small units of language. Our tools provided a very quick way to align large datasets without transcription in a matter of minutes.

My third project involved revamping the team's command-line tools and utilities for file IO to and from the database server. The new commands for querying and stitching together exact specific segments of video and audio media collapsed many tedious steps into one, which was substantially safer and sped up the process by 100%.

My fourth project was an explorational dive into the semantic space of the audio data itself. Using word2vec, a machine learning method to create high-dimensional vector representations of word semantics, I was able to map out the meandering of conversation topics between patients and nurses in physical space on abstract axes.

Due to the sensitive nature of patient data and medical IP, I can't supply exact details of my work here. Feel free to ask my personally and I can continue the conversation with you further.

☞ Conclusion

I am incredibly thankful to the amazing souls at the Chang Lab for taking me under their wing and teaching me so much about the state of the art neuroscience, statistics, and machine learning that they use in their day-to-day. Everyone at the lab was remarkably humble, personable, and fun to work with, especially my co-intern, Ryon Sabouni!

Me with a collection of awesome colleagues!
SF, I'll see ya soon.