Walter Philip Dych  /  Speech · Gesture · Motor planning

How does a moving body change what we plan to say?

I am a research technician and incoming PhD student in Brain and Cognitive Sciences at the University of Rochester. I record speech, gesture, posture and breathing together to find out how far ahead speakers plan when their hands are part of the utterance.

Point-light talker

The animation could not load. It shows a point-light figure, drawn from motion capture, describing a picture while gesturing.

Jan 2027 PhD start, Castellucci Lab, University of Rochester
5 signals recorded on one clock: motion, force, breath, audio, video
2 first-author manuscripts under review
6 conference presentations in phonetics and neuroscience
01  Doctoral research

Put your body into it: a whole-body approach to speech and gesture planning

Speakers rarely plan a whole sentence before they start talking. They plan a first chunk, begin, and plan the rest on the way. How large that chunk is, the planning scope, shifts with time pressure, sentence complexity and mental load.

Gesture has been left out of that picture, even though people gesture whenever they talk. A gesture’s stroke lands on the word it depicts, and the arm is slower than the mouth, so the hand has to start moving before that word is spoken.

My project asks whether that timing requirement forces speakers to plan further ahead, and whether the body gives the plan away before a word is said.

Design, models and pilot results
  1. Constraint My hypothesis

    To land the stroke on time, a speaker has to plan at least as far as the word the gesture goes with.

    Gesture → more whole-sentence planning

  2. Competition

    Preparing a precise movement draws on the same limited resources as speech, the way mental load does.

    Gesture → shorter planning scope

  3. Independence

    Speech and gesture are separate systems that are only lined up during execution.

    Gesture → no change in planning

02  Method

One utterance, measured across the whole body

Tour the recording space
Signal Instrument What it shows
Movement 12-camera OptiTrack motion capture When each phase of a gesture starts, where the stroke lands relative to the words, and how smooth the movement is.
Ground forces AMTI force plates The postural adjustment that comes before the arm moves, which can reveal a planned gesture before the hand does.
Breathing Respiration belt How deep a breath the speaker takes before starting, a marker of how much speech is planned.
Speech Audio with forced alignment Response time from the go-cue, the timing of every word, and filled pauses where planning falls behind.
Timing Tucker-Davis Technologies hardware Stimulus control and a shared clock, so all of the above line up to the millisecond.
04  Where the project stands

Built and piloted

  • DoneRecording system built and synchronized: motion capture, force plates, respiration, audio.
  • DoneDeadline task programmed and piloted, with an analysis pipeline from raw recordings to word-aligned trials.
  • DonePlanning models and simulated experiments to set the sample size and check the analysis.
  • NextAim 1 data collection, then natural speech with gestures free and restrained.
05  Recent writing

Manuscripts under review

  • Under review
    Bottom-up noise enhancement versus top-down time compression deficits: Independent effects on neural speech tracking and memory
    Dych, W. P., Ikeda, M., Lee, M. Y., & Lim, S.-J.
  • Under review
    Uncontrolled hardware variation introduces limits on timing precision in online auditory experiments
    Dych, W. P., Zi, L., & Lim, S.-J.

All publications and talks

06  About

Linguist by training, instrument builder by habit

I studied linguistics at the University of Delaware, where annotating gesture by hand in Kathryn Franich’s PhonLab pushed me to write software that could do it. I then managed Sung-Joo Lim’s auditory cognitive neuroscience lab at Binghamton University, running EEG and brain stimulation studies of how listeners follow speech.

At Rochester I work with Gregg Castellucci, bringing the two together: how the hands move with speech, and how the brain schedules an utterance across the vocal tract and the limbs. I share tools and data openly so this kind of work is possible for labs without specialized equipment.

More about me, outreach and mentoring
Contact

Get in touch