Reimagining AAC: Using Artificial Intelligence to Support Idiosyncratic Communication Gestures

Presenter(s)

Event Details

Topic:

augmentative and alternative communication (AAC)

Format:

lecture

Subject Level:

intermediate

Age Span:

kindergarten - grade 6
grades 7-12
adult

Target Audience:

AT specialist
consultant
educator
healthcare administration
K-12 administration
occupational therapist
physical therapist
special educator
speech language pathologist
teacher of the visually impaired
university professor / personnel
vision impairment specialist

Professional Development Credits

IACET CEUs:

0.01

ACVREP CEs:

1

Presentation Length: 1 hour

Location:

Edina

Description:

Physical limitations and environmental conditions pose challenges to existing AAC systems, especially in real-world school settings such as cafeterias, playgrounds, music rooms, and gyms. This session introduces an AI-powered system developed at Penn State that uses wearable sensors and machine learning to detect user-defined gestures and translates them to spoken messages. Designed for individuals who may also communicate with unique body-based movements, the tool bridges unaided and aided AAC, shifting the burden of interpretation from the user and communication partner to the technology, offering a powerful solution for spontaneous communication in environments where traditional AAC often falls short. Participants will see how this tool can be used to support communication with unfamiliar partners in the educational environments, an essential consideration given the high turnover of paraprofessionals and support staff in schools.. Through demonstration videos, examples, and interactive discussion, attendees will join the conversation in considering practical applications for wearable gesture recognition in educational settings to increase autonomy, social participation, and access to communication for students who need or use AAC.

Learning Outcomes:

As a result of this activity, participants will be able to:

• Describe at least 3 communication challenges faced by individuals with motor and/or visual impairments when using traditional aided and unaided AAC systems.

• Explain how wearable sensor-based technologies and artificial intelligence can be used to detect and interpret individualized body-based communicative movements.

• Identify 2 potential student applications of AI-powered gesture recognition tools in real-world AAC scenarios and articulate ways to integrate these technologies into inclusive practice settings.

Disclosures:

Financial Disclosures: Sharon Redmon receives compensation from CTG for pre-conference presentations. She receives funding from the AAC Leadership grant (U.S. Department of Education, H325D220021) at Pennsylvania State University.

Non-Financial Disclosures: Sharon Redmon serves as a member of the Education Committee for USSAAC, and The Wisconsin AAC Network and is a founding member of WATRN. She has presented past webinars for AAC in the Cloud and Closing the Gap.
Disclosures: I receive a salary from Florida State University.

Non-financial disclosures: None