Conversation

It still feels like a dream. Yes, I am working on #Vtuber tech at Google AI! 💪😎💪 Our new BlazePose GHUM Holistic model predicts full body and hand joint rotations for 3D avatars. Available soon at mediapipe.dev. 🐎💃🎵 Paper and live demo at

@CVPR

for #CVPR2022.

0:21

63.9K views

4:00 PM · Jun 22, 2022

Replying to

Watch the video explanation for our pose and hands estimation. #CVPR2022 youtube.com/watch?v=YoczNf

Rich

@yeemachine

Jun 23, 2022

Here’s what my camera looks like 😂

Rich

@yeemachine

Jun 23, 2022

The stars really had to align for this video. It was so hard to get a sunny weekend with no one in the office 😂 A couple of Googlers did see me dancing from the hallway The cringe 🫣

Rich

@yeemachine

Jun 24, 2022

Paper is finally up. It's not official until it's on arxiv 😂. arxiv.org/abs/2206.11678 I never thought I'd be working with research, especially something for Vtubers and XR. I can't thank the team enough for their hard work this past year and I'm excited for what's next!🫰

arxiv.org

BlazePose GHUM Holistic: Real-time 3D Human Landmarks and Pose Estimation

We present BlazePose GHUM Holistic, a lightweight neural network pipeline for 3D human body landmarks and pose estimation, specifically tailored to real-time on-device inference. BlazePose GHUM...

Replying to

and

Big fan of

’s Umapyoi Densetsu dance. His energy inspires me to continue developing motion capture! Side by side comparison: [ A single webcam ] vs [ OptiTrack Flex + Manus PrimeII + Rokoko Smartgloves ] We'll continue improving! ウマぴょい! 🐎💨

103

Replying to

and

I forgot to ask - could this program be used for Vroid model/avatar? I see the price for full body suit (mocap or Rokoko suit) is very expensive to own. So having something that is reasonably priced would be an absolute dream come true for me (and this is why I ask).

Replying to

and

The demo actually does use VRM models 😉. Keep in mind, this right now is actually just the tracking tech itself, not a full app. When we release it, hopefully some app developers will add it to their products!