publications

ALS • AUTISM • DEMENTIA • DEPRESSION • HUNTINGTON'S DISEASE • MODALITY PLATFORM • MODALITY R&D • MULTIPLE SCLEROSIS •

PARKINSON'S DISEASE • SCHIZOPHRENIA

MCI / MODALITY PLATFORM | International Workshop on Spoken Dialog Systems 2023

Towards Integrating Eye-Gaze Tracking Into A Multimodal Dialog Agent For Remote Patient Assessment

D. Tisdale, J. Liscombe, D. Pautler and V. Ramanarayanan

R&D / MODALITY PLATFORM | International Workshop on Spoken Dialog Systems 2023

Pathology-Specific Settings For Voice Activity Detection in a Multimodal Dialog Agent For Digital Health Monitoring

J. Liscombe, H. Kothare, M. Neumann, D. Pautler, and V. Ramanarayanan

DEPRESSION / MODALITY PLATFORM | International Workshop on Spoken Dialog Systems 2023

Impact of Synthetic Voice and Avatar Animation on The Usability of a Dialogue Agent for Digital Health Monitoring

H. Kothare, D. Habberstad, M. Neumann, S. White, D. Pautler and V. Ramanarayanan

SCHIZOPHRENIA | International Society for CNS Clinical Trials and Methodology (ISCTM) Conference 2023

Using AI-driven platform to Detect Negative Symptoms of Schizophrenia Through Facial and Acoustic Analysis

A. Khan, V. Ramanarayanan, H. Kothare, D. Pautler, M. Parak, B. Parker, D. Suendermann-Oeft, C. Yavorsky, and JP. Lindenmayer

R&D | International Journal of Human–Computer Interaction 2022

Ethical Awareness in Paralinguistics: A Taxonomy of Applications

A. Batliner, M. Neumann, F. Burkhardt, A. Baird, S. Meyer, N. Thang Vu and B. W. Schuller

PARKINSON'S DISEASE | American Speech and Hearing Association (ASHA) Convention 2022

Use of a Telehealth Platform to Automatically Assess Prosodic Contours in Parkinson Disease

A. Exner, V. Ramanarayanan, D. Pautler, H. Kothare, J. Liscombe and J. Huber

SCHIZOPHRENIA | International Conference on Multimodal Interaction (ICMI) 2022

Towards Multimodal Dialog-Based Speech & Facial Biomarkers of Schizophrenia

V. Richter, M. Neumann, H. Kothare, O. Roesler, J. Liscombe, D. Suendermann-Oeft, S. Prokop, A. Khan, C. Yavorsky, J-P. Lindenmayer and V. Ramanarayanan

PARKINSON'S DISEASE | International Conference on Multimodal Interaction (ICMI) 2022

Exploring Facial Metric Normalization For Within- and Between-Subject Comparisons in a Multimodal Health Monitoring Agent

O. Roesler, H. Kothare, W. Burke, M. Neumann, J. Liscombe, A. Cornish, D. Habberstad, D. Pautler, D. Suendermann-Oeft and V. Ramanarayanan

PARKINSON'S DISEASE | 147th Meeting of the American Neurological Association (ANA) 2022

Human-in-the-Loop Machine Learning Curation of Problems That Bother Parkinson Disease Patients

C. Marras, L. Arbatti, A. Amara, K. Anderson, C. Bale, L. Chahine, S. Eberly, A. Hosamath, D. Kinel, S. Mantri, S. Mathur, D. Oakes, J. Purks, D. Weintraub and I. Shoulson

PARKINSON'S DISEASE | INTERSPEECH 2022

Statistical and Clinical Utility of Multimodal Dialogue-Based Speech and Facial Metrics for Parkinson's Disease Assessment

H. Kothare, M. Neumann, J. Liscombe, O. Roesler, W. Burke, A. Exner, S. Snyder, A. Cornish, D. Habberstad, D. Pautler, D. Suendermann-Oeft, J. Huber and V. Ramanarayanan

MULTIPLE SCLEROSIS | Speech Motor Control Conference (SMC) 2022

Assessment of Atypical Speech in Multiple Sclerosis Via A Multimodal Dialogue Platform: An Exploratory Study

H. Kothare, M. Neumann, J. Liscombe, O. Roesler, D. Habberstad, W. Burke, A. Cornish, L. Arbatti, A. Hosamath, D. Fox, D. Pautler, D. Suendermann-Oeft, I. Shoulson and V. Ramanarayanan

ALS | Speech Motor Control Conference (SMC) 2022

On Timing and Pronunciation Metrics for Intelligibility Assessment in Pathological ALS Speech

J. Liscombe, M. Neumann, H. Kothare, O. Roesler, D. Suendermann-Oeft and V. Ramanarayanan

AUTISM | Speech Motor Control Conference (SMC) 2022

Atypical Speech Acoustics and Jaw Kinematics During Affect Production In Children With Autism Spectrum Disorder Assessed by an Interactive Multimodal Conversational Platform

H. Kothare, V. Ramanarayanan, O. Roesler, M. Neumann, J. Liscombe, W. Burke, A. Cornish, D. Habberstad, B. Kopald, A. Bai, Y. Markiv, L. Cole, S. Markuson, Y. Bensidi-Slimane, A. Sakallah, K. Brogan, L. Lampinen, S. Skiba, D. Suendermann-Oeft, D. Pautler and C. Demopoulos

PARKINSON'S DISEASE | IEEE Conference on Engineering in Medicine & Biology (EMBC) 2022

Speech, Facial and Fine Motor Features for Conversation-Based Remote Assessment and Monitoring of Parkinson’s Disease

H. Kothare, O. Roesler, W. Burke, M. Neumann, J. Liscombe, A. Exner, S. Snyder, A. Cornish, D. Habberstad, D. Pautler, D. Suendermann-Oeft, J. Huber, and V. Ramanarayanan

SCHIZOPHRENIA | Annual Meeting of the Schizophrenia International Research Society (SIRS) 2022

Reliability, Validity and Internal Consistency of Multimodal AI Based Facial and Acoustic Biomarkers of Negative Symptoms in Schizophrenia

A. Khan, S. Prokop, S. Bashir, JP. Lindenmayer, B. Insel, D. Pautler, D. Suendermann-Oeft, V. Ramanarayanan, and C. Yavorsky

MCI | Annual Meeting of the Cognitive Neuroscience Society (CNS) 2022

On The Feasibility of Remotely Administered and Self Driven Cognitive Assessments Using a Multimodal Dialog Agent

W. Burke, D. Pautler, H. Kothare, D. Habberstad, O. Roesler, M. Neumann, J. Liscombe, A. Cornish, D. Suendermann-Oeft and V. Ramanarayanan

ALS | Annual Meeting of the American Academy of Neurology (AAN) 2022

Lessons Learned From A Large-Scale Audio-Visual Remote Data Collection For Amyotrophic Lateral Sclerosis Research

V. Ramanarayanan, M. Neumann, A. Anvar, D. Suendermann-Oeft, O. Roesler, J. Liscombe, H. Kothare, J. D. Berry, E. Fraenkel, R. Norel, A. V. Sherman, J. Kumm, and I. Navar

PARKINSON'S DISEASE | Motor Speech Conference 2022

Multimodal Dialog Based Remote Patient Monitoring of Motor Function in Parkinson’s Disease and Other Movement Disorders

O. Roesler, W. Burke, H. Kothare, J. Liscombe, M. Neumann, A. Cornish, D. Habberstad, D. Pautler, D. Suendermann-Oeft and V. Ramanarayanan

ALS | Motor Speech Conference 2022

On the Robust Automatic Computation of Speaking and Articulation Duration in ALS Patients Versus Healthy Controls

J. Liscombe, A. Ocampo, H. Kothare, O. Roesler, M. Neumann, D. Habberstad, A. Cornish, D. Pautler, D. Suendermann-Oeft and V. Ramanarayanan

PARKINSON'S DISEASE | Motor Speech Conference 2022

Collecting Remote Voice and Movement Data From People with Parkinson's Disease (PD) Using Multimodal Conversational AI: Lessons Learned From A National Study

A. Exner, V. Ramanarayanan, D. Pautler, S. Snyder, H. Kothare, J. Liscombe, S. Sridhar, O. Roesler, W. Burke, M. Neumann, D. Suendermann-Oeft and J. Huber

PARKINSON'S DISEASE | Journal of Parkinson’s Disease (JPD) 2022

Longitudinal Cohort Study of Verbatim-Reported Postural Instability Symptoms as Outcomes for Online Parkinson’s Disease Trials

I. Shoulson, L. Arbatti, A. Hosamath, S.W. Eberly, D. Oakes

R&D | Perspectives of the ASHA Special Interest Groups 2022

Speech as a Biomarker: Opportunities, Interpretability, and Challenges

V. Ramanarayanan, A. Lammert, H. Rowe, T. F. Quatieri and J. R. Green

ALS | The 32nd International Symposium on Amyotrophic Lateral Sclerosis and Motor Neuron Disease 2021

Multimodal Dialog Based Speech and Facial Biomarkers Capture Differential Disease Progression Rates For ALS Remote Patient Monitoring

M. Neumann, O. Roesler, J. Liscombe, H. Kothare, D. Suendermann-Oeft, J. D. Berry, E. Fraenkel, R. Norel, A. Anvar, I. Navar, A. V. Sherman, J. R. Green and V. Ramanarayanan

SCHIZOPHRENIA | Society for Neuroscience (SfN) Annual Meeting 2021

Multimodal Conversational Technology For Remote Assessment of Symptom Severity in People with Schizophrenia

H. Kothare, M. Neumann, O. Roesler, J. Liscombe, D. Pautler, D. Suendermann-Oeft, C. Yavorsky, A. Khan, JP Lindenmayer and V. Ramanarayanan

SYSTEM | International Workshop on Spoken Dialog Systems (IWSDS) 2021

Voice Activity Detection in Dialog Agents for Dysarthric Speakers

J. Liscombe, H. Kothare, M. Neumann, A. Ocampo, O. Roesler, D. Habberstad, A. Cornish, D. Pautler, D. Suendermann-Oeft and V. Ramanarayanan

PARKINSON'S DISEASE | ASHA Convention 2021

Development of a Telehealth Platform for the Assessment of Motor Speech Disorders

A. Exner, V. Ramanarayanan, D. Pautler, H. Kothare, S. Snyder, and J. Huber

SCHIZOPHRENIA | International Society for CNS Clinical Trials and Methodology (ISCTM) Conference 2021

Multimodal AI of Facial and Acoustic Biomarkers of Negative Symptoms in Schizophrenia

A. Khan, JP Lindenmayer, S. Bashir, S. Prokop, B. Insel, B. Fitapelli, K. Bodicherla, M. Parak, B. Parker, C. Yavorsky, H. Kothare, D. Pautler, D. Suendermann-Oeft and V. Ramanarayanan

PARKINSON'S DISEASE | 25th International Congress of Parkinson's Disease and Movement Disorders (MDS) 2021

Multimodal Conversational AI for Remote Patient Monitoring and Analysis of Parkinson's Disease

V. Ramanarayanan, A. Exner, D. Pautler, S. Snyder, W. Burke, H. Kothare, J. Liscombe, D. Suendermann-Oeft, A. Lambert, R. Kohlmeier, and J. Huber

PARKINSON'S DISEASE | Journal of Parkinson’s Disease (JPD) 2021

Predictive Value of Verbatim Parkinson’s Disease Patient-Reported Symptoms of Postural Instability and Falling

M. Javidnia, L. Arbatti, A. Hosamath, S.W. Eberly, D. Oakes, I. Shoulson

ALS | 22nd Annual Conference of the International Speech Communication Association (Interspeech) 2021

Investigating the Utility of Multimodal Conversational Technology and Audiovisual Analytic Measures for the Assessment and Monitoring of Amyotrophic Lateral Sclerosis at Scale

M. Neumann, O. Roesler, J. Liscombe, H. Kothare, D. Suendermann-Oeft, D. Pautler, I. Navar, A. Anvar, J. Kumm, R. Norel, E. Fraenkel, A. Sherman, J. Berry, G. Pattee, J. Wang, J. Green, V. Ramanarayanan

PARKINSON'S DISEASE | American Academy of Neurology (AAN) 2021

Cross-Sectional Profile of Most Bothersome Problems as Reported Directly by Individuals with Parkinson’s Disease

L. Vinikoor-Imler, L. Arbatti, A. Hosamath, I. Sapir, J. Shirvan, N. Maserejian, I. Shoulson

AUTISM | 22nd Annual Conference of the International Speech Communication Association (Interspeech) and National Autism Conference 2021

Investigating the Interplay Between Affective, Phonatory and Motoric Subsystems in Autism Spectrum Disorder Using a Multimodal Dialogue Agent

H. Kothare, V. Ramanarayanan, O. Roesler, M. Neumann, J. Liscombe, W. Burke, A. Cornish, D. Habberstad, A. Sakallah, S. Markuson, S. Kansara, A. Faerman, Y. Bensidi-Slimane, L. Fry, S. Portera, D. Suendermann-Oeft, D. Pautler, C. Demopoulos

PARKINSON'S DISEASE / HUNTINGTON DISEASE | Huntington Study Group (HSG) 2021

Topic Modeling of Verbatim Patient Problem Reports Shows Differing Priorities of Family Impact for Huntington Disease and Motor Symptoms for Parkinson Disease

J.L. Purks, K.E. Anderson, L. Arbatti, A. Hosamath, I. Shoulson

SYSTEM | Draft Report of the Voice Interaction Community Group of the World Wide Web Consortium 2021

Intelligent Personal Assistant Architecture, Version 1.2

D. Schnelle-Walka and D. Dahl

SCHIZOPHRENIA | 17th Annual Scientific Meeting of the International Society for CNS Clinical Trials and Methodology (ISCTM) 2021

Multimodal AI of Facial and Acoustic Biomarkers of Negative Symptoms

JP. Lindenmayer and A. Khan, in collaboration with D. Suendermann-Oeft, D. Pautler, V. Ramanarayanan, S. Bashir, M. Parak, S. Ahmad, S. Prokop, T. Abad, S. Sevy, and C. Yavorsky

PARKINSON'S DISEASE | American Neurological Association (ANA) 2021

Online Patient Reporting of Postural Instability Symptoms as Outcomes for Parkinson Disease Clinical Trials

I. Shoulson, L. Arbatti, A. Hosamath, S. Eberly, D. Oakes

ALS | 73th Annual Meeting of the American Academy of Neurology (AAN) 2021

Towards A Large-Scale Audio-Visual Corpus for Research on Amyotrophic Lateral Sclerosis

A. Anvar, D. Suendermann-Oeft, D. Pautler, V. Ramanarayanan, J. Kumm, J. Berry, R. Norel, E. Fraenkel, and I. Navar

R&D | 12th International Seminar on Speech Production (ISSP) 2020

Remote Monitoring of Respiratory Function Using a Cloud-Based Multimodal Dialogue System

H. Kothare and V. Ramanarayanan

R&D | 21st Annual Conference of the International Speech Communication Association (Interspeech) 2020

Toward Remote Patient Monitoring of Speech, Video, Cognitive and Respiratory Biomarkers Using Multimodal Dialog Technology

V. Ramanarayanan, O. Roesler, M. Neumann, D. Pautler, D. Habberstad, A. Cornish, H. Kothare, V. Murali, J. Liscombe, D. Schnelle-Walka, P. Lange, and D. Suendermann-Oeft

DEPRESSION | 58th Annual Meeting of the Association for Computational Linguistics (ACL) Workshop on NLP for Medical Conversations 2020

On the Utility of Audiovisual Dialog Technologies and Signal Analytics for Real-time Remote Monitoring of Depression Biomarkers

M. Neumann, O. Roesler, D. Suendermann-Oeft, and V. Ramanarayanan

PARKINSON'S DISEASE | Movement Disorders Society (MDS) 2020

Comparison of Categorical- and Verbatim-Reported Postural Instability Symptoms Among Fox Insight Parkinson Research Participants

M. Javidnia, L. Arbatti, A. Hosamath, S. Eberly, D. Oakes, A. Nguyen, I. Shoulson

SYSTEM | NSF Future Directions Workshop Report, National Science Foundation 2020

Toward User-Oriented Agents: Research Directions and Challenges

M. Eskenazi and T. Zhao, with contributions from P. Balakrishnan, J. Bigham, C. Busso, J. Callan, V. Chen, B. DiEugenio, M. Gasic, J. Gustafsson, J. Hirschberg, R. Hwa, A. Kerne, T. Korelsky, D. Litman, S. Mehri, R. Moore, L.P. Morency, M. Ostendorf, A. Raux, V. Reiser, W. Shen, A. Stent, D. Suendermann-Oeft, D. Traum, S. Ultes, C. Voss, and M. Walker

SYSTEM | Speech Technology Magazine 2020

The standardization of intelligent personal assistants

D. Schnella-Walka

PARKINSON'S DISEASE | American Society for Experimental Neurotherapeutics (ASENT) 2020

Informing Natural History and Clinical Trials from Verbatim Reports of >20,000 Parkinson Patients

L. Arbatti, C. Marras, D. Standaert, C.M. Tanner, L. Smolensky, C. Kopil, J. Hamilton, E. Flagg, M. Javidnia, C.A. Christopher, A. Hosamath, A. Nguyen, I. Shoulson

R&D | Motor Speech Conference 2020

Leveraging Multimodal Dialog Technologies & AI/ML for Patient Health Diagnosis, Monitoring, and Intervention

V. Ramanarayanan, D. Suendermann-Oeft, D. Pautler, O. Roesler, J. Liscombe, M. Neumann, D. Habberstad, and A. Cornish

PARKINSON'S DISEASE | American Neurological Association (ANA) 2020

Parkinson Disease Symptoms and the Company They Keep as Reported Directly by >25,000 Patients

I. Shoulson, L. Arbatti, A. Hosamath, M. Javidnia, D. Oakes, S. Eberly, C. Marras, D. Standaert, C.M. Tanner, J. Hamilton, L. Smolensky, C. Kopil, C.A. Christopher, A. Nguyen

SYSTEM | ACM International Conference on Intelligent Virtual Agents (IVA) 2019

NEMSI: A Multimodal Dialog System for Screening of Neurological or Mental Conditions

D. Suendermann-Oeft, A. Robinson, A. Cornish, D. Habberstad, D. Pautler, D. Schnelle-Walka, F. Haller, J. Liscombe, M. Neumann, M. Merrill, O. Roesler, and R. Geffarth

PARKINSON'S DISEASE | American Medical Informatics Association (AMIA) 2019

Evaluation of Automatic Speech Recognition Application Programming Interfaces (APIs) of Verbatim Reports from Parkinson Disease (PD) Patients

R. Chadaram, L. Arbatti, A. Nguyen, I. Shoulson

PARKINSON'S DISEASE | American Medical Informatics Association (AMIA) 2019

Natural Language Processing as Applied to Patient Verbatim Reports

L. Arbatti, A. Nguyen, I. Shoulson

PARKINSON'S DISEASE | World Parkinson’s Congress Journal of Parkinson’s Disease (JPD) 2019

A Cross-Sectional Natural History of Parkinson Disease as Reported by >10,000 Patients

I. Shoulson, L. Arbatti, C. Marras, D. Standaert, C.M. Tanner, L. Smolensky, C. Kopil, J. Hamilton, E. Flagg, C.A. Christopher, A. Nguyen

PARKINSON'S DISEASE | Movement Disorders Society (MDS) 2019

What Bothers Parkinson Disease Patients? Verbatim Reports from >10,000 Patients Informing Natural History and Clinical Trials

I. Shoulson, L. Arbatti, C. Marras, D. Standaert, C. Tanner, L. Smolensky, C. Kopil, J. Hamilton, E. Flagg, C.A. Christopher, A. Nguyen

PARKINSON'S DISEASE | International Society for Clinical Trials and Methodology (ISCTM) 2019

Verbatim Reports of Problems from >10,000 Parkinson Disease Patients: Informing Natural History and Clinical Outcome Assessments

I. Shoulson, L. Arbatti, C. Marras, D. Standaert, C.M. Tanner, L. Smolensky, C. Kopil, J. Hamilton, E. Flagg, C.A. Christopher, A. Nguyen

PARKINSON'S DISEASE | American Academy of Neurology (AAN) 2019

What Bothers Parkinson Patients: Clinical Curation of their Verbatim Reports

L. Arbatti, C. Marras, C. Tanner, D. Standaert, A. Nguyen, I. Shoulson

SYSTEM | International Conference on Activity and Behavior Computing (ABC) 2019

Towards Visual Behavior Detection in Human-Machine Conversations

O. Roesler and D. Suendermann-Oeft

PARKINSON'S DISEASE / HUNTINGTON DISEASE | Huntington Study Group (HSG) 2018

Problems that Bother Parkinson Disease Patients: Framework for a Patient Reported Natural History of Parkinson and Huntington Diseases

I. Shoulson, L. Arbatti, A. Nguyen, D. Standaert, C. Marras, C.M. Tanner

PARKINSON'S DISEASE | American Neurological Association (ANA) 2018

Problems that Bother Parkinson Patients: The Basis for a Patient-Reported Natural History

L. Arbatti, C. Marras, D. Standaert, C.M. Tanner, A. Nguyen, I. Shoulson

PARKINSON'S DISEASE | Movement Disorders Society (MDS) 2018

Framework for a Patient-Reported Natural History of Parkinson Disease

L. Arbatti, A. Nguyen, C. Marras, D. Standaert, C.M. Tanner, L. Smolensky, C. Kopil, L. McLaughlin, E. Flagg, C.A. Christopher

PARKINSON'S DISEASE | American Academy of Neurology (AAN) 2018

Foundations for a Patient-Reported Natural History of Parkinson Disease: Cross-sectional Analysis of Motor and Non-Motor Symptoms of the Fox Insight PD-PROP Dataset

L. Arbatti, A. Nguyen, L. McLaughlin, L. Smolensky, C. Kopil, E. Flagg, C.A. Christopher, I. Shoulson

PARKINSON'S DISEASE | International Society for Clinical Trials and Methodology (ISCTM) 2018

Foundations for a Patient-Reported Natural History of Parkinson Disease: Cross-sectional Analysis of Cardinal Signs of the MJFF Fox Insight PD-PROP Dataset

L. Arbatti, A. Nguyen, L. McLaughlin, L. Smolensky, C. Kopil, E. Flagg, C.A. Christopher, I. Shoulson

PARKINSON'S DISEASE | Parkinson’s Disease Therapeutics Conference (PDTC) 2017

Foundations for a Patient-Reported Natural History of Parkinson Disease: Cross-sectional Analysis of the MJFF Fox Insight Platform

L. Arbatti, A. Nguyen, L. McLaughlin, L. Smolensky, C. Kopil, E. Flagg, C.A. Christopher, I. Shoulson