Soyong Shin


Ph.D Student
Mechanical Engineering in Carnegie Mellon University
Research Assistant at Musculoskeletal Biomechanics Lab
Advisor: Prof. Eni Halilaj

M.S. in Mechanical Engineering in Carnegie Mellon University
* College of Engineering Fellowship awardee (ATK-Nick G. Vlahakis)

B.S. in Mechanical Engineering in Seoul National University

GitHub
Curriculum Vitae(PDF)
Email : soyongs (at) andrew (dot) cmu (dot) edu

Research Interests : Computer Vision, Biomechanics, VR/AR, Wearable Sensors

[News]
(Nov. 2025) Our work SAM 3D Body is now published!
(Nov. 2025) Two first-authored papers were accepted at 3DV 2026!
(Oct. 2025) Our work on I-MOVE-23 dataset is accepted at IEEE TBME
(Apr. 2025) I received Pre-doctoral Achievement Award at ASB 2025
(Sep. 2024) I started a Visiting Researcher position at Meta FAIR Perception Team
(Jun. 2024) I was selected one of two recipients of Meta AI Mentorship program!
(May. 2024) I started a Research Scientist Intern position at Meta Reality Labs Research
(Feb. 2024) One first-author paper WHAM was accepted at CVPR 2024.
(Feb. 2024) I am joining Meta Reality Lab as a research scientist intern from this May.
(Jan. 2024) I received CMU College of Engineering Dean's Fellowship.
(Dec. 2023) Our work "SKEL" got honorable mention at SIGGRAPH Asia 2023.

Publications


SAM 3D Body: Robust Full-Body Human Mesh Recovery
Xitong Yang, Devansh Kukreja, Don Pinkus, Anushka Sagar, Taosha Fan, Jinhyung Park, Soyong Shin, Jinkun Cao, Jiawei Liu, Nicolas Ugrinovic, Matt Feiszli, Jitendra Malik, Piotr Dollar, Kris Kitani
arXiv , 2025.
Paper / Project page
BodyContact4D: A Multi-view Video Dataset for Understanding Human and Environment Interactions
Soyong Shin, Chaeeun Lee, Holly Chen, Jyun-Ting Song, Eni Halilaj, Kris Kitani
3DV , 2026.
EgoMDM: Diffusion-based Human Motion Synthesis from Sparse Egocentric Sensors
Soyong Shin, Anuj Pahuja, Alexander Richard, Kris Kitani, Jason Saragih, Yuhua Chen, Weipeng Xu, Eni Halilaj, Timur Bagautdinov
3DV , 2026.
Impact of Multi-View Fusion and Biomechanical Modeling on Markerless Motion Tracking
Zhixiong Li, Soyong Shin, Vu Phan, Evy Meinders, Eni Halilaj
IEEE Trans. Biomedical Engineering, 2025.
Paper / Project page
MAMMA: Markerless & Automatic Multi-Person Motion Action Capture
Hanz Cuevas-Velasquez, Anastasios Yiannakidis, Soyong Shin, Giorgio Becherini, Markus Höschle, Joachim Tesch, Taylor Obersat, Tsvetelina Alexiadis, Michael J. Black
arXiv , 2025.
Paper
WHAM: Reconstructing World-grounded Humans with Accurate 3D Motion
Soyong Shin, Juyong Kim, Eni Halilaj, Michael J. Black
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , 2023.
Paper / Project page
From Skin to Skeleton: Towards Biomechanically Accurate 3D Digital Humans
Marilyn Keller, Keenon Werling, Soyong Shin, Scott Delp, Sergi Pujades, C. Karen Liu, Michael J. Black
ACM Transactions on Graphics, Proceedings in SIGGRAPH Asia, 2023.
Paper / Project page
Markerless Motion Tracking with Noisy Video and IMU Data
Soyong Shin, Eni Halilaj
IEEE Trans. Biomedical Engineering, 2023.
Paper / code
Toward Portable and Modular Biomechanics Labs: How Video and IMU Fusion will Change Gait Analysis
Eni Halilaj, Soyong Shin, Eric Rapp, Donglai Xiang
Journal of Biomechanics, 2021.
paper / code / Video (below)
Multi-view Human Pose and Shape Estimation Using Learnable Volumetric Aggregation
Soyong Shin, Eni Halilaj
arXiv, 2020.
paper / demo video (below)
Estimation of Kinematics from Inertial Measurement Units Using a Combined Deep Learning and Optimization Framework
Eric Rapp*, Soyong Shin*, Wolf Thomsen, Reed Ferber, and Eni Halilaj ( * equal contribution)
Journal of Biomechanics, Vol 116, 110229, 2021
paper / code / abstract (ASB 2020) / project page
A Multimodal Dataset for Modeling Human Pose Priors
Eni Halilaj, Soyong Shin, Eric Rapp, Xiang Donglai, and Yaadhav Raj
American Society of Biomechanics, 2020
abstract / project page

Honors and Awards


CMLH Fellowship in Digital Health
Awarded by Carnegie Mellon School of Computer Science
Selected 1 of 10 Ph.D. students studying in the field of Machine Learning and Digital Health (May. 2022)
Nick-G. Vlahakis Fellowship
Awarded by Carnegie Mellon College of Engineering
Selected 1 of 2 among CMU College of Engineering MS students (Jan. 2021)
The Korean Government Scholarship for Study Overseas
Funded by Korean National Institute for International Education
Supporting outstanding graduate students studying oversea (Adimitted in Aug. 2020, Funding from Aug. 2021 to Aug. 2023)
Minister Prize
Awarded by Korean Ministry of Science and ICT
1st prize at 2019 Korean AI Grand Challenge (Task: Optical Character Recognition)
Best Presenting Paper Award
Awarded by President of Korean Society of Computational Fluids Engineering (Oct. 2018)
1 of 5 selected at 2018 Annual Meeting of Korean Society of Computational Fluids Engineering
Excellence Award
Awarded by Dean of Engineering College, Seoul National University (Dec. 2017)
3rd prize in X-Corps Competition
Special Award
Awarded by President of Korean Society of Computational Fluids Engineering (Nov. 2017)
3rd prize at EDISON competition
The National Scholarship for Science and Engineering
Funded by Korean Student Aid Foundation (Jul. 2016)

Patents and Copyrights


Korean Copyright Comission: SNUFOAM-Vertical Axis Turbine
W. Seok, Soyong Shin, H. Yeo, and S.H. Rhee
Registered at Korean Copyright (Apr. 2019)
Korean Domestic Patent: Controllable Pitch Propeller Using Shape Memory Alloy
S.H. Ahn, Y. Park, S. Song, Soyong Shin, and M. Choi
Registered at Korean Intellectual Property Office (Dec. 2017)