Jingyu Shi

me4.png

About Me


I am a Research Scientist at Meta Reality Labs Research.

I earned my Ph.D. in Electrical and Computer Engineering from Purdue University, under the supervision of Prof. Karthik Ramani. I obtained my M.S. degree in ECE in Georgia Institute of Technology, working with Prof. Patricio Vela. Prior to that, I studied Instrument Science and Technology in Beihang University and became a B.Eng.

My research interests lie in the intersection of AI and HCI. I work on multi-modal language models and embodied intelligence, as well as the interaction design of their applications.

Google Scholar / Resume / LinkedIn

news

Nov 09, 2025 One paper was accepted to 3DV 2025.
Oct 08, 2025 One paper got accepted to TMLR.
Aug 25, 2025 I am now working for Meta Reality Labs as a Research Scientist.
Jul 23, 2025 I successfully defended my Ph.D. dissertation today.
Jul 04, 2025 One paper was accepted to UIST 2025

latest posts

selected publications

  1. occludenerf.png
    OccludeNeRF: Geometry-aware 3D Scene Inpainting with Collaborative Score Distillation in NeRF
    Jingyu Shi, Achleshwar Luthra , Jiazhi Li , and 5 more authors
    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , Mar 2025
  2. caring.png
    CARING-AI: Towards Authoring Context-aware Augmented Reality INstruction through Generative Artificial Intelligence
    Jingyu Shi*, Rahul Jain* , Seunggeun Chi* , and 4 more authors
    In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems , Jan 2025
  3. transparent.png
    Transparent Barriers: Natural Language Access Control Policies for XR-Enhanced Everyday Objects
    Kentaro Taninaka , Rahul Jain , Jingyu Shi, and 2 more authors
    In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems , Jan 2025
  4. ooi.png
    Interacting objects: A dataset of object-object interactions for richer dynamic scene representations
    Asim Unmesh , Rahul Jain , Jingyu Shi, and 5 more authors
    IEEE Robotics and Automation Letters, Oct 2023
  5. hgenai.jpg
    An HCI-Centric Survey and Taxonomy of Human-Generative-AI Interactions
    Jingyu Shi*, Rahul Jain* , Hyungjun Doh , and 2 more authors
    arXiv preprint arXiv:2310.07127, Jan 2024
  6. ubitouch.png
    Ubi-TOUCH: Ubiquitous Tangible Object Utilization through Consistent Hand-object interaction in Augmented Reality
    Rahul Jain* , Jingyu Shi*, Runlin Duan , and 3 more authors
    In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology , Oct 2023
  7. ubiedge.jpg
    Ubi Edge: Authoring Edge-Based Opportunistic Tangible User Interfaces in Augmented Reality
    Fengming He , Xiyun Hu , Jingyu Shi, and 3 more authors
    In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems , May 2023