Oral presentations are an essential yet challenging aspect of academic and professional life. To date, many commercial and research products have been developed to provide support for the authoring, rehearsal and delivery of presentations. however, little work has been conducted to provide real-time tracking of presentation content. Given the presentation slides with speaking notes, a presentation tracking system uses automatic speech recognition to track content coverage by the speaker. This can help speakers ensure that they cover their planned content while potentially reducing their speech anxiety and enabling various real-time presentation support technologies, such as automatic slide advance. Presentation tracking is, however, a complex task; due to the inaccuracy of current speech recognition systems and the fact that speakers rarely follow the exact presentation notes.
In this thesis, I present a novel framework for both on-line tracking of presentations at the sub-slide level, as well as global presentation tracking through a slide deck that allows for more speaker flexibility in choosing slides to present. Tracking is performed by semantic matching of the confusion network results from an automatic speech recognition system against the slide's content keywords. The keywords are selected and weighted based on word specificity and semantic similarity measures. My evaluation studies show that using confusion networks results in a more robust speech recognition system, while semantic matching reduces the reliance on the exact notes, and keyword weighting improves the accuracy of the tracking system. I will present my plans for improving tracking accuracy and addition of slide tracking to support more dynamic presentations. I plan to integrate this presentation tracking framework into two different applications to provide support for both presentation rehearsal and delivery, and conduct user studies to evaluate its effectiveness.
Harriet Fell (Advisor)
Timothy Bickmore (Co-Advisor)
Darren Edge (External Member, Microsoft Research)
Justification for committee composition, by Harriet Fell: