Super stoked to release the very first episode of my Casual Robotics podcast: "Progress Towards General Purpose Robots" ft. the brilliant and .
Follow
Kevin Zakka
@kevin_zakka
Kevin Zakka’s Tweets
Meet . We help you create media in the formats you need to tell your stories. Try Genmo today with hilarious and immersive text-to-video generation 🎬 twitter.com/genmoai/status
This Tweet is unavailable.
3
15
70
Pausing this to focus on RSS, will be back after the deadline✌️
Quote Tweet
Starting "MuJoCo Weekly", a newsletter introducing MuJoCo users to new features, tips and tricks.
Follow along in this thread 
:
:Show this thread
8
Especially thrilled to announce that the has been awarded a seed grant to pursue the development of SuTr, a foundational #AI vision model for #surgicalAI.
If you’d like to help out by contributing surgical videos please get in touch! twitter.com/HiesingerLab/s
This Tweet is unavailable.
1
4
32
Tweaking reward terms and getting instantaneous feedback drastically changes the ballgame! See Taylor's thread below 👇
Quote Tweet
my @DeepMind project is out!
MuJoCo MPC (MJPC) is an interactive tool for real-time behavior synthesis with predictive control algorithms.
paper: arxiv.org/abs/2212.00541
code: github.com/deepmind/mujoc
video: dpmd.ai/mjpc
Show this thread
0:25
26.4K views
16
Week 2 👇
Quote Tweet
Week 2: "Ghostifying" an Object
Disabling contacts for a body is trivial in MuJoCo: iterate through all its geoms and set their contype and conaffinity to 0.
1/3
Show this thread
5
Steps to ghostify:
1. Duplicate the object
2. Iterate through its geoms and set contype=conaffinity=0
3. Iterate through its bodies and set mocap=True
4. Optional: make all geoms transparent
Step 3 disables the object's dynamics (it is treated as fixed by the simulator).
3/3
1
Show this thread
There are many use cases for disabling contacts (e.g. speeding up the physics), but let's talk about one particular useful instance: "ghostifying" an object for use as a visual hint. E.g. in manipulation, to render an object's goal pose: youtu.be/Bdx7DuAMB6o?t=
2/3
1
3
Show this thread
Week 2: "Ghostifying" an Object
Disabling contacts for a body is trivial in MuJoCo: iterate through all its geoms and set their contype and conaffinity to 0.
1/3
1
2
14
Show this thread
Over winter break I decided to make Notos, an #AI-powered transcription application based on ’s Whisper.
Here’s a small demo where it manages to transcribe everything flawlessly!
TestFlight: testflight.apple.com/join/GhDGhGwE
25
54
458
Show this thread
2) Use the body’s `gravcomp` attribute, which is a new feature added as of version 2.3.1. gravcomp=1.0 will exactly counteract gravity, values greater than 1 will create a buoyancy effect. The default value for gravcomp is 0.
2/3
1
6
Show this thread
Week 1: 2 ways to implement gravity compensation
1) Inject "fake" upwards forces at the center-of-mass of each body in a chain st each force counteracts the body’s weight. Write these forces to `xfrc_applied`, which are user-defined Cartesian wrenches applied to body CoMs.
1/3
1
6
Show this thread
Starting "MuJoCo Weekly", a newsletter introducing MuJoCo users to new features, tips and tricks.
Follow along in this thread 👇:
3
19
148
Show this thread
Loaded a replay buffer in one process while another process was writing to it. 3 days of training went poof 🫠 Write atomic saves kids…
1
17
Still using 3 layer MLPs and proud!
14
15
375
I'm excited to present Single-Level Differentiable Contact Simulation. It's a novel formulation that unifies contact dynamics and collision detection in a single optimization problem.
paper: arxiv.org/abs/2212.06764
code: github.com/simon-lc/Silic
video:
3
19
82
Show this thread
If you are at #CoRL2022, come to check our poster at the Poster Lobby 352 (4:45 pm - 6:00 pm).
See how our multi-finger robot hand rotates an orange peel.
Website: haozhi.io/hora
Code: github.com/HaozhiQi/hora
2
14
94
👇👇👇👇
Quote Tweet
I've been working on a small native Mac app (#Swift/#SwiftUI) to access #ChatGPT/#GPT natively from anywhere on your Desktop. Simply ⌘+⇧ and spacebar from anywhere on your Mac!
If you'd like to help me beta test it, please shoot me a DM!
Show this thread
0:40
9.3K views
1
7
OpenAI is awesome
4
2
136
A common Copilot trick I rely on: quicker Google search.
ex: I need to render at 4k resolution but forgot the exact height / width —> just add a comment saying exactly that and boom.
14
Highly enjoyable and thought provoking talk by Yuval introducing MJPC, a predictive control toolbox for MuJoCo!
Quote Tweet
Yours truly, delivering the MIT Robotics Seminar about our new MJPC tool.
youtu.be/2xVN-qY78P4
1
9
MuJoCo 2.3.1 is out! Some spoilers: speed improvements for non-AVX configurations, Python bindings for the interactive viewer (a must for sim design), and more in the link 👇
Quote Tweet
#MuJoCo 2.3.1 is out! Interactive GUI viewer is now available through Python, thanks to @levisburner. 
mujoco.readthedocs.io/en/stable/pyth
mujoco.readthedocs.io/en/stable/pythShow this thread
1
4
38
Show this thread
I do robotics + AI research and still I feel insane FOMO + left behind seeing all the advances in LLM world 😅
65
149
2,497
How can robots skewer foods of different properties?
Check out our #CoRL2022 paper on reactive, visuo-haptic skewering!
Like humans who play with their food before picking it up, our system senses multimodal properties upon contact to adapt its skewering motion on the fly.🧵1/5
GIF
1
12
38
Show this thread
Excited to announce the release of
Language-Table 🦾
This is the largest language-annotated robot dataset of its kind, by an order of magnitude.
It comes with a new sim benchmark for language control.
Dataset + env: github.com/google-researc
Colab: colab.research.google.com/github/google-
GIF
1
25
80
Show this thread
maybe you can tag someone who works on the Optimus team so we can get a URDF + 3D assets 👀
1
6
Show this thread
We have some pretty exciting models coming to Menagerie in the upcoming release! Stay tuned 😉
github.com/deepmind/mujoc
6
26
154
Show this thread
We present CARBS: Coordinated Acquisition with Reactive Bimanual Scooping, a system for robustly scooping🥄 a wide variety of foods🍇🍓🥦
Similar to how humans use a fork and spoon to eat, CARBS uses a pusher to stabilize the food position while scooping! Appearing at CoRL ‘22.
1
9
38
Show this thread
Whose at NeurIPS next week? Excited to meet lots of people, both new and familiar faces, my inbox is open, so get in touch to meet up and chat Iso Labs and ML roles! Will be giving a talk and Q&A with on Wed 12:30-2pm at NeurIPS booth as well /n
7
16
136
Show this thread
This is surreal
5
New MuJoCo documentation theme is now live, with support for dark theme: mujoco.readthedocs.io/en/latest/over
Scrolling through the XML reference in particular should be a much better experience :)
1
2
21
MuJoCo is < 5 stars away from 5k, go give it some love 🤓!!
1
1
18
Finally got around to moving my RL stack exclusively to dm_env (no more gym) and "I've never felt better. I have more energy. My skin is clearer. My eye sight has improved." 😉
3
3
69
From our demo floor at AI@, check out Code as Policies at work. This helper robot is able to compute and execute a task given via natural language. Read more → goo.gle/3U5CmCg
15
81
269
Replying to
This talk is a masterclass in doing live robot demos. "We are gonna train the robot now to walk ... BING that was 8 hrs of sim experience". Q&A? Oh let this robot causally do wheelies in the background.
1
2
6
3
6
36