Abstract

The rapid growth of the aging population and the rising prevalence of Subjective Cognitive Decline (SCD) highlight the need for continuous, unobtrusive assessment of functional cognition during everyday activities. Vision-based smart home systems offer a promising pathway for monitoring behavior and supporting independent living. However, enabling real-time cognitive assistance remains challenging. It requires not only accurately interpreting complex human behaviors to detect cognitive errors, but also supporting real-time deployment on resource-constrained edge devices and under dynamic wireless network conditions. This dissertation presents Smart Kitchen, an AI-driven system for real-time cognitive error detection through the monitoring of daily cooking activities. Under this framework, the dissertation makes two primary contributions toward cognitive error detection. First, we introduce Oatmeal107, a real-world dataset collected over two years that captures natural cooking behaviors together with naturalistic cognitive sequencing errors. Second, we propose CHEF-VL, an online reasoning framework based on vision-language models that detects sequence-level cognitive errors from live video streams. To support practical real-time deployment, this dissertation further makes two complementary system contributions. We develop RT-HARE, an end-to-end framework that enables low-latency, high-accuracy human action recognition on embedded platforms by bypassing computationally expensive optical flow extraction. We also introduce Progressive Neural Compression (PNC), a rateless encoding framework that enables adaptive, timing-aware visual data offloading under volatile wireless network conditions. Together, these contributions demonstrate that AI systems can enable accurate, real-time, and non-intrusive detection of cognitive errors in daily activities. This work lays the foundation for intelligent assistive systems that enhance safety, autonomy, and quality of life for individuals experiencing cognitive decline.

Committee Chair

Chenyang Lu

Committee Members

Nathan Jacobs, Lisa Connor; Roch Guérin; Sanjoy Baruah

Degree

Doctor of Philosophy (PhD)

Author's Department

Computer Science & Engineering

Author's School

McKelvey School of Engineering

Document Type

Dissertation

Date of Award

4-29-2026

Language

English (en)

Download

Share

COinS