Real-time physics collisions on a city scale 🤯
I built this AR experience where objects can collide with your city environment 🏙️
A smartphone is the only device needed (1/…)
Pinned Tweet
Show this thread
Follow
Click to Follow arthurbfrd
ARthur Bouffard
@arthurbfrd
Creative Technologist - AR R&D | | MSc | Member l Ambassador
ARthur Bouffard’s Tweets
4
1
38
Let me know if you have any issues 🫣 (not in London atm)
2
Show this thread
It’s out!! 🎾
Anyone in London, Los Angeles and Santa Monica can experience this Lens
Try it here: lens.snapchat.com/ebb987327c7d4b
2
10
80
Show this thread
If you have any questions, please reply or quote re tweet! (here’s a fun video of a glitch I saw)
1
12
Show this thread
This Lens will be released soon for anyone to try in London, Los Angeles and Santa Monica! (only supported cities for now) 😄
2
1
35
Show this thread
The resulting experience is visually accurate & fun, with an unlimited amount of possibilities! No post-processing was used in these video clips, and they were all recorded from an iPhone 12 (No LiDAR) 📱
1
2
19
Show this thread
To create this, I used Lens Studio’s City Scale tracking and combined it with the Physics engine. I found a way to apply colliders to the City Mesh, allowing objects to interact with your greater surroundings
1
2
42
Show this thread
The LiDAR world mesh creates very realistic and satisfying collisions on floors or surfaces (4/…)
1
1
9
Show this thread
Different hand gestures scan spawn cubes, enable and remove gravity, and can even add sideways forces to the holograms. (2/…)
1
11
Show this thread
Controlling gravity with your hands is possible with augmented reality
I made an experience which allows you to spawn holograms and control their movements with gestures 👋
All of this in real-time on (1/…)
22
99
488
Show this thread
We’re back! Up next at 12:00 p.m. PST, #LensFest expert panelists discuss building the future of brands with AR. At 12:40 p.m. PST, dive into ’s Surf AR Wellness Lens → lensfest.live
5
5
36
read image description
ALT
1
2
16
The spawning gesture is made from tracking hand gestures for opening and closing positions hand 👋🏼
A simple instantiation script then allows to spawn new balls in! (5/…)
2
2
25
Show this thread
The physics engine Lens Studio offers now is really impressive. You can set custom forces, control gravity and so on.
You can make anything into a collider such as your hands or random meshes! (4/…)
2
1
21
Show this thread
You can try the Lens out here: lens.snapchat.com/f5d8ff9ca3db44
A LiDAR device is recommended but not required here! (3/…) 📱
1
1
19
Show this thread
The kind of fidelity seen in the clip is possible due to latest update, which allows for live world meshes with physics. The world mesh here is generated with LiDAR (2/…)
1
18
Show this thread
I built an AR experience where you can spawn and hit glowing orbs with your hands 🤚
Really impressed at how the holograms bounce off the walls using real-time physics
Here’s a few details on how I made this and a link to try it yourself (1/…)
24
205
942
Show this thread
Finally; I sent out the Lens to Snapchat for immediate testing, creating this high-fidelity result. I will upload the Lens soon for your personal testing!
Thank you for reading all the way down to here :)) (6/6)
3
26
Show this thread
The next step was to use Lens Studio. Using Remote Assets was the perfect solution for such a high resolution model, as the mesh and texture quality are perfectly preserved. This led to an amazing looking result. (5/6)
1
12
Show this thread
I then added bones to each branch and used a plug-in called Wiggle Bones to move the branches around. This took a few tutorials 🫠 (4/6)
1
4
Show this thread
After the scan was processed, I exported it and brought it into Blender. I proceeded to edit out all the extra vertices and inconsistencies on the 3D mesh. ✂️ (3/6)
1
4
Show this thread
I started by using LumaAI to make a photorealistic NeRF capture of my plant. I placed it in a room and proceeded to take pictures. If you’re not in the beta using Polycam NeRFs are a great alternative💡 (2/6)
1
3
16
Show this thread
Can you tell which plant is real? Using AR with AI to create digital copies of objects is becoming incredibly simple
Here I used
⁃ Remote Assets for maximum resolution
⁃ for the NeRF capture
- Blender for animations
Here are the details (1/6)🧵
11
44
283
Show this thread
I have to say I was inspired by ’s work here! He makes really cool AI/AR projects and has awesome ideas 💡
6
1
24
Show this thread
Finally; i adjusted the material settings and lightings options for realistic results. I sent the Lens out to Snapchat and tried it in a matter of seconds! (7/7)
3
1
18
Show this thread
I then optimised the meshes and textures as they were pretty large, did this using Draco compression and image compression (6/7)
1
1
12
Show this thread
The next step: Lens Studio by Using their foot tracking ML template worked perfectly and I didn’t have to make many adjustments. It was very straightforward to import the model in the scene (5/7)
2
1
23
Show this thread
I then proceeded to edit out all the extra vertices and inconsistencies in the 3D model using the edit mode. Exported it as a gltf (4/7)
1
1
18
Show this thread
After the scan was processed, I exported it and brought it into Blender, airdropping it from my iPhone to my laptop (3/7)
2
2
16
Show this thread
First things first! I used to make a photorealistic NeRF of my shoe. I placed it on a table and proceeded with the capture. If you’re not in the beta using photogrammetry is a great alternative (2/7)
1
3
34
Show this thread
I created a photo-realistic AR try-on experience in less than 1 hour using AI & ML
Seeing how easy the process is, here’s a breakdown on how you can make this too🧵 (1/7)
23
297
1,859
Show this thread
Wanted to share how absolutely mind-blowing technology is
From a 5-min session, I was able to capture a perfectly textured 3D mesh and a coloured point cloud, all this using AI / NeRFs
This is going to be a game changer in mapping out locations in 3D
7
24
157
Just tried the Quest Pro and I have to say it’s really good, properly engineered hardware..
But working with the headset doesn’t seem great 🤔 Gaming on the other hand could be awesome
Nice to see the advancements being made in VR, will definitely be experimenting
2
2
9
Show this thread
The disco ball physics experience is now available for everyone to play with! 🪩
Try it here: lens.snapchat.com/3cd120379ea747
3
2
26