Abstract
This chapter summarizes the results of modeling filled pauses and prolongations to improve Slovak spontaneous speech recognition, by introducing them into the language model and speech recognition dictionary. We propose a hybrid method of robust statistical language modeling that combines hidden-event filled pause modeling with the random distribution of prolongations in a corpus of the Slovak written texts. We decided to use existing triphone context-dependent acoustic models designed for regular words for implicit acoustic modeling of selected types of hesitation fillers. Filled pauses are modeled and represented in a recognition dictionary by a small set of phonetic classes with similar acoustic-phonetic properties. We significantly improved the robustness for individual speakers in the task of transcription of the Slovak TEDx talks and speech recognition performance up to 4.56\(\%\) for prolongations and 7.90\(\%\) for filled pauses, relatively on average.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
References
Adell J, Bonafonte A, Escudero D (2010) Modelling filled pauses prosody to synthese disfluent speech. In: Proceedings of speech prosody, Chicago, IL, paper 624
Baranyi P, Csapó A (2012) Definition and synergies of cognitive infocommunications. Acta Polytech Hung 9(1):67–83
Baranyi P, Csapó A, Sallai GY (2015) Cognitive infocommunications. Springer International Publishing Switzerland
Barras C, Geoffrois E, Wu Z, Liberman M (2001) Transcriber: development and use of a tool for assisting speech corpora production. Speech Commun 33(1–2):5–22
Beňuš Š, Enos F, Hirschberg J, Shriberg E (2006) Pauses in deceptive speech. In Proceedings of speech prosody, Dresden, Germany, paper 212
Deme A, Markó A (2012) Lengthenings and filled pauses in the spontaneous speech of Hungarian adults and children. In Proceedings of workshop of fluent speech: combining cognitive and educational approaches, Ultrecht, Netherlands
Duchatea J, Laureys T, Wambacq P (2004) Adding robustness to language models for spontaneous speech recognition. In Proceedings of COST278 and ISCA tutorial and research workshop on robustness issues in conversational interaction, Norwich, UK, paper 11
Gavuvain JL, Adda G, Lamel L, Adda-Decker M (1997) Transcribing broadcast news: the LIMSI Nov96 Hub4 System. In Proceedings of 1997 darpa speech recognition workshop, Chantilly, Virginia
Hládek, D, Ondáš, S, Staš, J (2014) Online natural language processing of the Slovak language. In Proceedings of 5\(^{th}\) IEEE international conference on cognitive InfoCommunications, CogInfoCom 2014, Vietri sul Mare, Italy, pp 315–316
Kipyatkova I, Karpov A, Verkhodanova V, Železmý M (2013) Modeling of pronunciation, language and nonverbal units at conversational Russian speech recognition. Int J Comput Sci Appl 10(1):11–30
Lee A, Kawahara T (2009) Recent development of open-source speech recognition engine Julius. In: Proceedings of 2009 Asia-Pacific signal and information processing association: annual summit and conference, APSIPA ASC 2009, Sapporo, Japan, pp 131–137
Liu Y, Shriberg E, Stolcke A (2003) Automatic disfluency identification in conversational speech using multiple knowledge sources. In: Proceedings of EUROSPEECH, Geneva, Switzerland, pp 957–960
Moniz H, Mata AI, Viana MC (2007) On filled pauses and prolongations in European Portuguese. In: Proceedings of INTERSPEECH Antwerp, Belgium, pp 2645–2648
Moniz H, Trancoso I, Mata AI (2009) Classification of disfluent phenomena as fluent communicative devices in specific prosodic contexts. In: Proceedings of INTERSPEECH Brighton, UK, pp 1719–1722
Nunes R, Neves L (2008) Filled pauses modeling. L\(^2\)F—Spoken language system laboratory, INESC-ID Lisboa, Technical Report, Lisboa, Portugal, p 9
Ohta K, Kitaoka N, Nakagawa S (2014) Modeling filled pauses and silences for responses of a spoken dialogue system. Int J Comput 1998–4308(8):136–143
Ohtsuki K, Furui S, Sakurai N, Iwasaki A, Zhang Z-P (1999) Recent advances in Japanese broadcast news transcription. In: Proceedings of EUROSPEECH, Budapest, Hungary, pp 671–674
Oviatt S (1995) Predicting spoken disfluencies during human-computer interaction. Comput Speech Lang 9(1):19–35
Pakhomov SA, Savova G (2000) Filled pause distribution and modeling in quasi-spontaneous speech. In: Proceedings of the 14\(^{th}\) international congress of phonetic sciences, ICSP 1999, San Francisco, CA, pp 31–34
Peters J (2003) LM studies on filled pauses in spontaneous medical dictation. In: Proceedings of HLT-NAACL 2003, Edmonton, Canada, pp 82–84
Povey D, Ghoshal A, Boulianne G, Burget L, Glembek O, Goel N, Hannemann M, Motlicek P, Qian Y, Schwarz P, Silovsky J, Stemmer G, Vesely K (2011) The Kaldi speech recognition toolkit. In: Proceedings of IEEE 2011 Workshop on Automatic Speech Recognition and Understanding, ASRU 2011, Waikoloa, HI, US, pp 1–4
Prylipko D, Vasilenko B, Stolcke A, Wendemuth A (2012) Language modeling of nonverbal vocalizations in spontaneous speech. In: Sojka P et al (eds.), Text, Speech and Dialogue, LNAI 7499, Springer International Publishing Switzerland, pp 488–495
Rose RL (2013) Crosslingual corpus of hesitation phenomena: a corpus for investigating forst and second language speech performance. In: Proceedings of Interspeech, Lyon, France, pp 992–996
Rusko M, Juhár J, Trnka M, Staš J, Darjaa S, Hládek D, Sabo R, Pleva M, Ritomský M, Ondáš S (2016) Advances in the Slovak judicial domain dictation system. In: Vetulani, Z et al (Eds.), Human Language Technology Challenges for Computer Science and Linguistics, LNAI 9561, Springer International Publishing Switzerland, pp 16–27
Siu M-H, Ostendorf M (1996) Modeling disfluencies in conversational speech. In: Proceedings of ICSLP, Philadelphia, PA, pp 386–389
Sabo R (2008) Anotovaná rečová databáza parlamentných nahrávok (Annotated speech database of parliament proceedings. In: Rusko M et al (eds) Akustika a spracovanie reči. Slovakia, Bratislava, pp 131–135 (in Slovak)
Shriberg EE (1994) Preliminaries to a theory of speech disfluencies. Ph.D. thesis, University of California, Berkeley, p 406
Schramm H, Aubert XL, Meyer C, Peters J (2003) Filled-pause modeling for medical transcription. In: Proceedings of IEEE workshop an spontaneous speech processing and recognition, SSPR 2003, Tokyo, Japan, paper TM06
Somiya M, Kobayashi K, Nishiyaki H, Sekiguchi Y (2007) The effect of filled pauses in a lecture speech on impressive evaluation of listeners. In: Proceedings of INTERSPEECH 2007, Antwerp, Belgium, pp 2645–2648
Staš J, Juhár J (2015) Modeling of Slovak language for broadcast news transcription. J Electric Electron Eng 8(2):39–42
Staš J, Viszlay P, Lojka M, Koctúr T, Hládek D, Kiktová E, Pleva M, Juhár J (2015) Automatic subtitling system for transcription, archiving and indexing of Slovak audiovisual recordings. In: Proceedings of the 7\(^{th}\) Language & Technology Conference, LTC 2015, Poznań, Poland, pp 186–191
Staš J, Hládek D, Juhár J (2016) Adding filled pauses and disfluent events into language models for speech recognition. In Proc. of the 7th IEEE international conference on cognitive InfoCommunications, CogInfoCom 2016, Wroclaw, Poland, pp 133–137
Staš J, Koctúr T, Viszlay P (2016) Automatická anotácia a tvorba rečového korpusu prednášok TEDxSK a JumpSK (Automatic annotation and building of a speech corpus of TEDxSK and JumpSK talks). In: Proceedings of the 11\(^{th}\) workshop on intelligent and knowledge oriented technologies and 35th conference on data and knowledge, WIKT & DaZ 2016, Smolenice, Slovakia, pp 127–132 (in Slovak)
Stolcke A, Shriberg E (1996) Statistical language modeling for speech disfluencies. In: Proceedings of ICASSP, Atlanta, GA pp 405–408
Stolcke A, Shriberg E, Hakkani-Tur D, Tur G (1999) Modeling the prosody of hidden events for improved word recognition. In: Proceedings of EUROSPEECH, Budapest, Hungary, pp 311–314
Stolcke A (2002) SRILM—-An extensible language modeling toolkit. In Proc. of ICSLP, Denver, CO, pp 901–904
Viszlay P, Staš J, Koctúr T, Lojka M, Juhár J (2016) An extension of the Slovak broadcast news corpus based on semi-automatic annotation. In: Proceedings of LREC, Portorož, Slovenia, pp 4684–4687
Watanabe M (2009) Features and roles of filled pauses in speech communication: a corpus-bases study of spontaneous speech. Hituji Syobo Publishing, p 147
Žgank A, Rotovnik T, Maučec MS (2008) Slovenian spontaneous speech recognition and acoustic modeling of filled pauses and onomatopoeas. WSEAS Trans Signal Process 4(7):388–397
Žgank A, Maučec MS (2010) Modeling of filled pauses and onomatopoeas for spontaneous speech recognition. In: Shabtai NR (ed) Advances in speech recognition. Croatia, Sciyo, pp 67–82
Acknowledgements
The research in this chapter was partially supported by the Ministry of Education, Science, Research and Sport of the Slovak Republic under the research projects KEGA 055-TUKE-4/2016: “Transfer of Substantial Results of Research in Speech Technology into Education” and VEGA 1/0511/17: “Personalized Acoustic and Language Modeling”, and by the Slovak Research and Development Agency under the research project APVV-15-0517: “Automatic Subtitling of Audiovisual Content for Hearing Impaired”.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer International Publishing AG, part of Springer Nature
About this chapter
Cite this chapter
Staš, J., Hládek, D., Juhár, J. (2019). Modeling of Filled Pauses and Prolongations to Improve Slovak Spontaneous Speech Recognition. In: Klempous, R., Nikodem, J., Baranyi, P. (eds) Cognitive Infocommunications, Theory and Applications. Topics in Intelligent Engineering and Informatics, vol 13. Springer, Cham. https://doi.org/10.1007/978-3-319-95996-2_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-95996-2_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-95995-5
Online ISBN: 978-3-319-95996-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)