Jingyu Shi

About Me

I am a Research Scientist at Meta Reality Labs Research.

I earned my Ph.D. in Electrical and Computer Engineering from Purdue University, under the supervision of Prof. Karthik Ramani. I obtained my M.S. degree in ECE in Georgia Institute of Technology, working with Prof. Patricio Vela. Prior to that, I studied Instrument Science and Technology in Beihang University and became a B.Eng.

My research interests lie in the intersection of AI and HCI. I work on multi-modal language models and embodied intelligence, as well as the interaction design of their applications.

me4.png

news

Nov 09, 2025 One paper was accepted to 3DV 2025.
Oct 08, 2025 One paper got accepted to TMLR.
Aug 25, 2025 I am now working for Meta Reality Labs as a Research Scientist.
Jul 23, 2025 I successfully defended my Ph.D. dissertation today.
Jul 04, 2025 One paper was accepted to UIST 2025

latest posts

selected publications

  1. occludenerf.png
    OccludeNeRF: Geometry-aware 3D Scene Inpainting with Collaborative Score Distillation in NeRF
    Jingyu Shi, Achleshwar Luthra , Jiazhi Li , and 5 more authors
    In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , Mar 2025
  2. caring.png
    CARING-AI: Towards Authoring Context-aware Augmented Reality INstruction through Generative Artificial Intelligence
    Jingyu Shi*, Rahul Jain* , Seunggeun Chi* , and 4 more authors
    In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems , Jan 2025
  3. transparent.png
    Transparent Barriers: Natural Language Access Control Policies for XR-Enhanced Everyday Objects
    Kentaro Taninaka , Rahul Jain , Jingyu Shi, and 2 more authors
    In Proceedings of the 2025 CHI Conference on Human Factors in Computing Systems , Jan 2025
  4. ooi.png
    Interacting objects: A dataset of object-object interactions for richer dynamic scene representations
    Asim Unmesh , Rahul Jain , Jingyu Shi, and 5 more authors
    IEEE Robotics and Automation Letters, Oct 2023
  5. hgenai.jpg
    An HCI-Centric Survey and Taxonomy of Human-Generative-AI Interactions
    Jingyu Shi*, Rahul Jain* , Hyungjun Doh , and 2 more authors
    arXiv preprint arXiv:2310.07127, Jan 2024
  6. ubitouch.png
    Ubi-TOUCH: Ubiquitous Tangible Object Utilization through Consistent Hand-object interaction in Augmented Reality
    Rahul Jain* , Jingyu Shi*, Runlin Duan , and 3 more authors
    In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology , Oct 2023
  7. ubiedge.jpg
    Ubi Edge: Authoring Edge-Based Opportunistic Tangible User Interfaces in Augmented Reality
    Fengming He , Xiyun Hu , Jingyu Shi, and 3 more authors
    In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems , May 2023