Abstract. Understanding scenes and human states is essential for immersive technologies such as virtual reality (VR) and augmented reality (AR). Although recent advances in machine learning have greatly improved the robustness of such estimation tasks, many conventional approaches still rely heavily on visual input. This dependence causes a significant drop in performance in environments with poor visibility, such as those affected by darkness, occlusion, or sensor limitations. In this talk, I will introduce AI techniques that use a variety of sensing methods, including acoustic signals, millimeter-wave radar, and event-based cameras. These techniques enable the estimation of scenes and human states beyond what standard video cameras can perceive. I will also present methods that use conventional cameras in combination with AI to handle occlusion. I will also discuss the current challenges and future prospects of developing AI technologies that overcome the limitations of visual perception.

Biography. Mariko Isogawa (Member, IEEE) received the B.S., M.S., and Ph.D. degrees from Osaka University, Japan, in 2011, 2013, and 2019, respectively. From 2019 to 2020, she was a Visiting Scholar at Carnegie Mellon University, USA. She is currently an Associate Professor in the Department of Information and Computer Science, Faculty of Science and Technology, Keio University, Japan. Her research interests include computer vision and pattern recognition for perceiving people and environments, particularly under challenging sensing conditions.