body { -ms-overflow-style: scrollbar; overflow-y: scroll; overscroll-behavior-y: none; } .errorContainer { background-color: #FFF; color: #0F1419; max-width: 600px; margin: 0 auto; padding: 10%; font-family: Helvetica, sans-serif; font-size: 16px; } .errorButton { margin: 3em 0; } .errorButton a { background: #1DA1F2; border-radius: 2.5em; color: white; padding: 1em 2em; text-decoration: none; } .errorButton a:hover, .errorButton a:focus { background: rgb(26, 145, 218); } .errorFooter { color: #657786; font-size: 80%; line-height: 1.5; padding: 1em 0; } .errorFooter a, .errorFooter a:visited { color: #657786; text-decoration: none; padding-right: 1em; } .errorFooter a:hover, .errorFooter a:active { text-decoration: underline; } #placeholder, #react-root { display: none !important; } body { background-color: #FFF !important; }

JavaScript is not available.

We’ve detected that JavaScript is disabled in this browser. Please enable JavaScript or switch to a supported browser to continue using twitter.com. You can see a list of supported browsers in our Help Center.

Terms of Service Privacy Policy Cookie Policy Imprint Ads info © 2023 Twitter, Inc.

Follow

Hila Chefer

@hila_chefer

PhD candidate @ Tel Aviv University, student researcher

, interested in Deep Learning, Computer Vision, NLP, explainable AI @Hila_Chefer@sigmoid.social

hila-chefer.github.ioJoined December 2020

Hila Chefer’s Tweets

Pinned Tweet

[1/n] Can explainability improve model accuracy? Our latest work shows the answer is yes! arxiv.org/pdf/2206.01161 github.com/hila-chefer/Ro We noticed that ViTs suffer from salient issues- their output is often based on supportive signals (background) rather than the actual object

11

100

445

Show this thread

Amazing work!

congrats 🥳

Quote Tweet

Check out our new text-to-4D work!

twitter.com/_akhaliq/statu…

5

Amazing results!

Quote Tweet

CLIPascene: Scene Sketching with Different Types and Levels of Abstraction abs: arxiv.org/abs/2211.17256 project page: clipascene.github.io/CLIPascene/

Embedded video

1:56

8.5K views

3

Happening today! Come see us at poster session 2 :) 4PM-6PM (CST) @ Hall J #428

4

Show this thread

Hila Chefer Retweeted

Sketch-Guided Text-to-Image Diffusion Models abs: arxiv.org/abs/2211.13752 project page: sketch-guided-diffusion.github.io

Embedded video

5

135

602

Hila Chefer Retweeted

SpaText: Spatio-Textual Representation for Controllable Image Generation abs: arxiv.org/abs/2211.14305 project page: omriavrahami.com/spatext/

Embedded video

1

58

251

Work done with my amazing co-authors

and Lior Wolf at

4

Show this thread

Excited to share that we will be presenting our work in person at #NeurIPS2022

! Interested in leveraging explainability to improve accuracy and robustness? Come check out our poster and chat 🥳 Code: github.com/hila-chefer/Ro

demo: huggingface.co/spaces/Hila/Ro

Quote Tweet

[1/n] Can explainability improve model accuracy? Our latest work shows the answer is yes! arxiv.org/pdf/2206.01161 github.com/hila-chefer/Ro We noticed that ViTs suffer from salient issues- their output is often based on supportive signals (background) rather than the actual object

Show this thread

4

24

130

Show this thread

info on the papers:

Quote Tweet

[1/n] @eccvconf #ECCV2022 paper thread! 1. Image-based CLIP-Guided Essence Transfer (TargetCLIP)- we extract the essence of a target while preserving realism and source identity. 2. No Token Left Behind- we use explainability to stabilize the unreliable CLIP similarity scores. twitter.com/_akhaliq/statu…

Show this thread

1

2

Show this thread

Hila Chefer Retweeted

Prompt-to-Prompt: Latent Diffusion and Stable Diffusion implementation with

diffusers is out github: github.com/google/prompt-

Ron Mokady and Amir Hertz

12

336

1,551

Hila Chefer Retweeted

[1/5] Check out our latest work: arxiv.org/pdf/2204.02849 TL;DR: We present a new approach for (1) training text-to-image model *without text* (left fig) (2) local and semantic image manipulation, without utilizing masks (right fig).

This Tweet is unavailable.

6

44

240

Show this thread

I had the pleasure of speaking at

’s vision seminar, kindly hosted by

,

. My talk focused on using Transformer explainability algorithms to improve performance of downstream tasks (e.g. image editing). Check it out :)

Hila Chefer - Transformer Explainability

August 4th, 2022. Columbia UniversityAbstract: Transformers have revolutionized deep learning research across many disciplines, starting from NLP and expandi...

4

32

[n/n] No Token Left Behind- Work done with Roni Paiss and Lior Wolf

1

Show this thread

[10/n] No Token Left Behind- Additionally, we demonstrate that using the image explainability heatmaps, it is possible to generate an image from a given layout of bounding boxes!

1

5

Show this thread

[9/n] No Token Left Behind- Using this additional explainability loss, we demonstrate that downstream tasks such as image classification and image generation and editing that use CLIP guidance can be significantly improved.

1

4

Show this thread

[8/n] No Token Left Behind- We demonstrate that explainability can be used as an additional loss to penalize similarity scores that are not based on the semantic text tokens.

1

4

Show this thread

[7/n] No Token Left Behind- RT

: twitter.com/_akhaliq/statu We find that CLIP similarity scores can be unreliable since they rely on a small subset of the text tokens. E.g., the noisy image scores higher due to the influence of the words "image of" on the similarity score.

Quote Tweet

No Token Left Behind: Explainability-Aided Image Classification and Generation abs: arxiv.org/abs/2204.04908

1

2

14

Show this thread

[6/n] TargetCLIP - Thanks,

for sharing our work! Work done with

, Roni Paiss, and Lior Wolf

1

1

2

Show this thread

[5/n] TargetCLIP - As an alternative to long optimization, we show that one can also fine-tune an inversion encoder to output the essence vector of a target, allowing for instant extraction of the essence for each target!

1

3

Show this thread

[4/n] TargetCLIP - The extracted essence vectors are global and can be applied to any source while maintaining realism, and the original identity of the source.

1

4

Show this thread

[3/n] TargetCLIP - We demonstrate that using CLIP guidance and the powerful StyleGAN, we can extract an essence vector- a vector of semantic properties that correspond to the “signature characteristics” that are also identified by humans as related to the specific target.

2

6

Show this thread

[2/n] TargetCLIP- Code: github.com/hila-chefer/Ta Humans identify characteristics independently from the target identity. For example, the joker is identified by the signature hair color, face, and makeup regardless of the identity of the actor.

1

5

Show this thread

[1/n]

#ECCV2022 paper thread! 1. Image-based CLIP-Guided Essence Transfer (TargetCLIP)- we extract the essence of a target while preserving realism and source identity. 2. No Token Left Behind- we use explainability to stabilize the unreliable CLIP similarity scores.

Quote Tweet

Image-Based CLIP-Guided Essence Transfer abs: arxiv.org/abs/2110.12427 github: github.com/hila-chefer/Ta new method creates a blending operator that is optimized to be simultaneously additive in both latent spaces

2

20

127

Show this thread

2/2 papers accepted to #ECCV2022! Works were done with

, Roni Paiss, and Lior Wolf. Details to come soon 🥳 See you in Tel Aviv! 🎉

Quote Tweet

European Conference on Computer Vision (ECCV)

#ECCV2022 list of paper IDs of accepted papers is NOW available! eccv2022.ecva.net/submission/lis

Embedded video

GIF

1

18

Hila Chefer Retweeted

Tomorrow (4pm), Amsterdam's computational linguistics seminar features Hila Chefer (Tel Aviv). She will tell us all about her fancy multimodal Transformer Explainability technique: results on CLIP, BERT, LXMERT, & ViT, and hot-of-the-press improvements in ViT robustness!

2

8

44

Show this thread

Hila Chefer Retweeted

Alireza Dehbozorgi, @BDehbozorgi83@mastodon.social

Wish all the best to

for tomorrow's Computational Linguistics Seminar (in a hybrid format), at

@illc_amsterdam

. Fingers crossed!

2

2

6

Show this thread

Hila Chefer Retweeted

submit

demos for CVPR papers by joining the organization on

here: huggingface.co/organizations/

This Tweet is unavailable.

1

3

26

Hila Chefer Retweeted

The

organization on

is now accepting

demos for CVPR papers from anyone with a chance to win prizes, deadline to submit is June 30, 2022 (AOE time). CVPR organization: huggingface.co/CVPR link to join organization: huggingface.co/organizations/

1

13

51

Amazing results!

Quote Tweet

Volumetric Disentanglement for 3D Scene Manipulation abs: arxiv.org/abs/2206.02776 project page: sagiebenaim.github.io/volumetric-dis

Embedded video

0:32

5.3K views

1

11

Hila Chefer Retweeted

a

Demo for Optimizing Relevance Maps of Vision Transformers Improves Robustness by

on

Spaces demo: huggingface.co/spaces/Hila/Ro github: github.com/hila-chefer/Ro paper: arxiv.org/abs/2206.01161

Embedded video

2

22

Check out our new

demo (huggingface.co/spaces/Hila/Ro)- Even for out of domain inputs such as images generated by DALL-E 2 or animations, our method corrects the original model to produce a plausible prediction! (code: github.com/hila-chefer/Ro)

Quote Tweet

Optimizing Relevance Maps of Vision Transformers Improves Robustness abs: arxiv.org/abs/2206.01161 github: github.com/hila-chefer/Ro show that by finetuning the explainability maps of ViTs, a significant increase in robustness can be achieved

11

35

Hila Chefer Retweeted

Popular ML resources

@popular_ML@sigmoid.social

The most popular Arxiv link yesterday:

Quote Tweet

[1/n] Can explainability improve model accuracy? Our latest work shows the answer is yes! arxiv.org/pdf/2206.01161 github.com/hila-chefer/Ro We noticed that ViTs suffer from salient issues- their output is often based on supportive signals (background) rather than the actual object

Show this thread

1

8

Thanks for sharing our work

! Check out the thread with more details on the paper 👇

demo coming soon!

Quote Tweet

[1/n] Can explainability improve model accuracy? Our latest work shows the answer is yes! arxiv.org/pdf/2206.01161 github.com/hila-chefer/Ro We noticed that ViTs suffer from salient issues- their output is often based on supportive signals (background) rather than the actual object

Show this thread

9

35

Hila Chefer Retweeted

Optimizing Relevance Maps of Vision Transformers Improves Robustness abs: arxiv.org/abs/2206.01161 github: github.com/hila-chefer/Ro show that by finetuning the explainability maps of ViTs, a significant increase in robustness can be achieved

24

102

Hila Chefer Retweeted

Theophano Mitsa

Explainability improves model accuracy! #MachineLearning

Quote Tweet

[1/n] Can explainability improve model accuracy? Our latest work shows the answer is yes! arxiv.org/pdf/2206.01161 github.com/hila-chefer/Ro We noticed that ViTs suffer from salient issues- their output is often based on supportive signals (background) rather than the actual object

Show this thread

2

5

Hila Chefer Retweeted

Elad Richardson

@EladRichardson

Cool, a clever usage of explainabiliy for fine-tuning ViT 🧠

Quote Tweet

[1/n] Can explainability improve model accuracy? Our latest work shows the answer is yes! arxiv.org/pdf/2206.01161 github.com/hila-chefer/Ro We noticed that ViTs suffer from salient issues- their output is often based on supportive signals (background) rather than the actual object

Show this thread

1

4

[n/n] joint work with

and Lior Wolf from Tel-Aviv university

1

4

Show this thread

Check out our colab notebook where you can experiment with our fine-tuned models and compare them to the original ones: colab.research.google.com/github/hila-ch

1

13

Show this thread

We only use 3 labeled examples for 500 classes, and achieve a large increase in robustness (ImageNet-A, R, ObjectNet)

1

13

Show this thread

In this work, we show that one can *directly optimize the explanations*- i.e. use a loss on the explainability signal to ensure that the classification is based on the *right reasons*- the foreground and not the background.

1

12

Show this thread