Opens profile photo
Follow
Marc Päpper 🦆
@mpaepper
I help you listen through the noise in machine learning Fighting cancer with ML as Mindpeak's CIO Organizer Data Science Meetup Founder 7 digit revenue company
Science & TechnologyHamburg, Germanypaepper.com/blogJoined November 2008
245 Following
375 Followers

Marc Päpper 🦆’s Tweets

Pinned Tweet
Marc Päpper 🦆
@mpaepper
New paper adds great ideas to stable diffusion and beats it in human evaluation: - img generation first shapes scene, then refines details, so train experts (MoDE) for these differences - use object bounding boxes to help model learn about separate parts of text prompts
Image
Quote Tweet
AK
@_akhaliq
Image
ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Experts abs: arxiv.org/abs/2210.15257 achieves a zero-shot FID-30k score of 6.75 on the MS-COCO dataset
Show this thread
85
Marc Päpper 🦆
@mpaepper
Quite awesome method: write code in a programming language called RASP and it gets compiled to neural network transformer weights. Great for interpretability, but could maybe even be used to kickstart the neural network training with some custom domain knowledge!?
Quote Tweet
Jim Fan
@DrJimFan
We train Transformers to encode algorithms in their weights, such as sorting, counting, and balancing parentheses from lots of data. I never thought we may also go in the *reverse* direction: *compile* Transformer weights directly from explicit code! Cool paper @DeepMind: 1/🧵
Show this thread
Image
6
Marc Päpper 🦆
@mpaepper
Amazing ressource for ML practitioners and researchers. I highly recommend to check it out!
Quote Tweet
Pranav Rajpurkar
@pranavrajpurkar
I am truly excited to launch my e-book, AI Research Experiences. Features 250+ pages of comprehensive notes and insights from my Harvard course, CS197. Covers technical AI toolkits and research skills to take your AI journey to the next level. It's free. docs.google.com/document/d/1uv
Image
1
Marc Päpper 🦆
@mpaepper
Very nice work to make self-supervised training more efficient especially for vision models, but also works for other modalities. I think we need a lot more of these ideas for efficient training of neural networks.
Quote Tweet
Michael Auli
@MichaelAuli
New work on efficient self-supervised learning: data2vec 2.0 pre-trains vision models 16.4x faster than the most popular existing algorithm. Blog: ai.facebook.com/blog/ai-self-s Paper: ai.facebook.com/research/publi Code/models: github.com/facebookresear with @ZloiAlexei @mhnt1580 @arunbabu1234
Show this thread
Image
2
Marc Päpper 🦆
@mpaepper
Going from GPT3 to ChatGPT needs a prompt including a good general description, the history of the chat and the user's request. Cool insights by
@sjwhitmore
who manages to replicate the virtual machine example with it:
Quote Tweet
Sam Whitmore
@sjwhitmore
ChatGPT is not new technology, just a new interface. You can replicate it pretty closely from existing APIs. All you need is: 1. a good prompt 2. a memory window Here's how to do it using @LangChainAI: 🧵
Show this thread
1
3
Marc Päpper 🦆
@mpaepper
Really awesome move by
@PyTorch
to make v2 backwards compatible without breaking changes. The speedup for compilation sounds great. I wonder how that compares to tracing your model? Any insights?
Quote Tweet
Sebastian Raschka
@rasbt
PyTorch 2.0 was just announced at the @PyTorch Conference! The focus is speed and accessibility, and the great news is that there are no breaking changes. PyTorch 2.0 will be fully backward compatible. Here's my tl;dw 1/6 twitter.com/PyTorch/status…
Show this thread
Image
1
15
Marc Päpper 🦆
@mpaepper
New stable diffusion model is out and looks awesome!
Quote Tweet
Stability AI
@StabilityAI
We are excited to announce the release of Stable Diffusion Version 2! Stable Diffusion V1 changed the nature of open source AI & spawned hundreds of other innovations all over the world. We hope V2 also provides many new possibilities! Link → stability.ai/blog/stable-di
Show this thread
Image
Marc Päpper 🦆
@mpaepper
Prompt-to-prompt works like this: You want to generate an image which looks like a previously generated one, but slightly changed according to an edit prompt. To do so, you use the same generation seed and align the cross-attention maps for the unchanged parts of the prompt
Image
1
Show this thread
Marc Päpper 🦆
@mpaepper
The result: You can give it any image with an edit instruction and it will create a matching image in seconds which matches Imagic quality which needs ~1 GPU hour per such edit. I really like how they build on all the existing bits to generate their custom training set.
1
Show this thread
Marc Päpper 🦆
@mpaepper
- GPT3 to generate prompt + edit prompt + instruction triplets - Stable diffusion (SD) with prompt-to-prompt to generate Image + edited image pairs - CLIP score guidance to generate high quality pairs of these images - Train SD w/ existing weights + additional image conditioning
Image
1
Show this thread
Marc Päpper 🦆
@mpaepper
This is like Imagic, but instead of training for an hour for each generation, you get it via inference. 🚀
Quote Tweet
AK
@_akhaliq
InstructPix2Pix: Learning to Follow Image Editing Instructions abs: arxiv.org/abs/2211.09800 project page: timothybrooks.com/instruct-pix2p InstructPix2Pix, trained on generated data, and generalizes to real images and user-written instructions at inference time
Image
Marc Päpper 🦆
@mpaepper
5) Steps More steps ~ higher quality of the generated image This is true up to a certain step count, but is not guaranteed. On the flip side, a higher step count means that the generation will take longer and costs you more money. I recommend 20-50 steps for normal use.
Image
1
1
Show this thread
Marc Päpper 🦆
@mpaepper
4) Sampler The sampler controls the pathway in the latent space by setting the amount of noise at each step. I personally favor the DDIM sampler is it generates high quality images even for low step counts, so you can save some time and money.
Image
1
1
Show this thread
Marc Päpper 🦆
@mpaepper
3) Classifier Free Guidance The classifier free guidance is a multiplier value which can be set between 0 and 20 with a default of 7.5. The higher you set it, the more closely stable diffusion will follow your prompt. I recommend setting it to 8-14.
Image
1
1
Show this thread
Marc Päpper 🦆
@mpaepper
2) Seed Stable Diffusion denoises images in an iterative fashion and the seed determines the initial noisy image. When the initial noisy image is changed, the end result changes as well. So roughly speaking - different seed, different image.
Image
1
1
Show this thread
Marc Päpper 🦆
@mpaepper
Mind blown again 🤯 Large language model trained on scientific papers. The capabilities are amazing - you can check the demo at galactica.org Best part: they opensourced it including the model weights. What a playground 🃏
Quote Tweet
Papers with Code
@paperswithcode
🪐 Introducing Galactica. A large language model for science. Can summarize academic literature, solve math problems, generate Wiki articles, write scientific code, annotate molecules and proteins, and more. Explore and get weights: galactica.org
Show this thread
Embedded video
0:43
440K views
18
Marc Päpper 🦆
@mpaepper
Machine learning is like surfing 🏄‍♂️: - you need to experiment a lot - you will fail a lot - you need to learn from these failures to improve - it's a lot of fun if you accept this
1
4
Marc Päpper 🦆
@mpaepper
Do it as he says plus write unit tests for the essentials like losses, data loading and metrics and you will succeed in training deep neural nets.
Quote Tweet
Sairam Sundaresan
@DSaience
Debugging Neural Networks 101 Over the last 2 years, I have trained 1000s of models successfully. Here's a secret - I use the same 5 steps every time. My debugging time has come down from hours to minutes (And now yours can too)
Show this thread
1
3
Marc Päpper 🦆
@mpaepper
“We should know when users leave their house, their commute to work, and everywhere they go throughout the day. Anything less is useless. We get a lot more than that from other tech companies.” Ouch...
Quote Tweet
Steve Krenzel
@stevekrenzel
With Twitter's change in ownership last week, I'm probably in the clear to talk about the most unethical thing I was asked to build while working at Twitter. 🧵
Show this thread
1