Walter Philip Dych / Speech · Gesture · Motor planning
How does a moving body change what we plan to say?
I am a research technician and incoming PhD student in Brain and Cognitive Sciences at the University of Rochester. I record speech, gesture, posture and breathing together to find out how far ahead speakers plan when their hands are part of the utterance.
The animation could not load. It shows a point-light figure, drawn from motion capture, describing a picture while gesturing.
Put your body into it: a whole-body approach to speech and gesture planning
Speakers rarely plan a whole sentence before they start talking. They plan a first chunk, begin, and plan the rest on the way. How large that chunk is, the planning scope, shifts with time pressure, sentence complexity and mental load.
Gesture has been left out of that picture, even though people gesture whenever they talk. A gesture’s stroke lands on the word it depicts, and the arm is slower than the mouth, so the hand has to start moving before that word is spoken.
My project asks whether that timing requirement forces speakers to plan further ahead, and whether the body gives the plan away before a word is said.
Design, models and pilot results-
Constraint My hypothesis
To land the stroke on time, a speaker has to plan at least as far as the word the gesture goes with.
Gesture → more whole-sentence planning
-
Competition
Preparing a precise movement draws on the same limited resources as speech, the way mental load does.
Gesture → shorter planning scope
-
Independence
Speech and gesture are separate systems that are only lined up during execution.
Gesture → no change in planning
One utterance, measured across the whole body
| Signal | Instrument | What it shows |
|---|---|---|
| Movement | 12-camera OptiTrack motion capture | When each phase of a gesture starts, where the stroke lands relative to the words, and how smooth the movement is. |
| Ground forces | AMTI force plates | The postural adjustment that comes before the arm moves, which can reveal a planned gesture before the hand does. |
| Breathing | Respiration belt | How deep a breath the speaker takes before starting, a marker of how much speech is planned. |
| Speech | Audio with forced alignment | Response time from the go-cue, the timing of every word, and filled pauses where planning falls behind. |
| Timing | Tucker-Davis Technologies hardware | Stimulus control and a shared clock, so all of the above line up to the millisecond. |
From gesture annotation to neural tracking to whole-body recording
Whole-body speech planning
My doctoral project. A deadline task that reveals how each sentence was planned, recorded with motion capture, force plates and respiration, then carried into natural speech.
MultiGest
An open toolkit that finds and annotates co-speech gestures from ordinary video, built so labs without motion capture can still do multimodal research.
Fast speech in noise
An EEG study of how time-compressed speech and background noise separately affect the brain’s tracking of speech and what listeners remember.
Timing precision online
Three experiments showing how uncontrolled differences in participants’ hardware limit timing precision in web-based auditory studies.
Gesture magnitude in Medʉmba
My first research project: mapping the size of co-speech gestures in a Grassfields Bantu language.
Built and piloted
- DoneRecording system built and synchronized: motion capture, force plates, respiration, audio.
- DoneDeadline task programmed and piloted, with an analysis pipeline from raw recordings to word-aligned trials.
- DonePlanning models and simulated experiments to set the sample size and check the analysis.
- NextAim 1 data collection, then natural speech with gestures free and restrained.
Manuscripts under review
-
Under review
Bottom-up noise enhancement versus top-down time compression deficits: Independent effects on neural speech tracking and memoryDych, W. P., Ikeda, M., Lee, M. Y., & Lim, S.-J.
-
Under review
Uncontrolled hardware variation introduces limits on timing precision in online auditory experimentsDych, W. P., Zi, L., & Lim, S.-J.
Linguist by training, instrument builder by habit
I studied linguistics at the University of Delaware, where annotating gesture by hand in Kathryn Franich’s PhonLab pushed me to write software that could do it. I then managed Sung-Joo Lim’s auditory cognitive neuroscience lab at Binghamton University, running EEG and brain stimulation studies of how listeners follow speech.
At Rochester I work with Gregg Castellucci, bringing the two together: how the hands move with speech, and how the brain schedules an utterance across the vocal tract and the limbs. I share tools and data openly so this kind of work is possible for labs without specialized equipment.
More about me, outreach and mentoringGet in touch
- Emailwdych@ur.rochester.edu
- LabCastellucci Lab, Brain and Cognitive Sciences, University of Rochester
- GitHubgithub.com/walterdych
- ScholarGoogle Scholar profile
- ORCID0009-0003-7882-3859
- LinkedInlinkedin.com/in/walter-dych