In our new paper, we introduce 𝗰𝗼𝗻𝘁𝗿𝗼𝘃𝗲𝗿𝘀𝗶𝗮𝗹 𝘀𝘁𝗶𝗺𝘂𝗹𝗶 which elicit divergent predictions from different models. Controversial stimuli (a generalization of adversarial examples) are synthesized to cause disagreement among models. https://arxiv.org/abs/1911.09288 2/10
-
-
Prikaži ovu nit
-
We used controversial stimuli to test discriminative and generative models trained on MNIST as models of human perception of handwritten digits. For each pair of models, we created controversial stimuli for each pair of digits. 3/10pic.twitter.com/QtB82cPTXJ
Prikaži ovu nit -
Each controversial stimulus was synthesized by sampling a random noise image and iteratively modifying it such that each model classified the image as a different, predetermined digit, with high confidence. 4/10pic.twitter.com/OT8VYNjkt9
Prikaži ovu nit -
For some model pairs, humans see nothing in the resulting controversial stimuli, falsifying both models. For other model pairs, human perception is aligned with one of the models. 5/10pic.twitter.com/ymt33Y78ZN
Prikaži ovu nit -
For each pair of models, we synthesized controversial stimuli for all pairs of digits... 6/10pic.twitter.com/zQaZzvchHx
Prikaži ovu nit -
We quantified human perception of 720 controversial stimuli (+ 100 MNIST images) in 30 subjects using
@Prolific. Subjects judged the probability of presence of each digit independently. The models had sigmoid (not softmax) readouts to give them the same response flexibility. 7/10pic.twitter.com/WUDWuajYJy
Prikaži ovu nit -
Each dot here is the mean error at predicting one subject's responses to all 820 images. The
@wielandbr,@bethgelab generative analysis by synthesis (ABS) model had the smallest error. But predicting humans from other humans (black dots) still beat all models. 8/10pic.twitter.com/Kwbji7Onts
Prikaži ovu nit -
𝗖𝗼𝗻𝗰𝗹𝘂𝘀𝗶𝗼𝗻 𝟭: Controversial stimuli enabled us to adjudicate among DNN models as models of human perception. This would not have been possible with natural stimuli, since all models correctly classify handwritten digits, and thus make nearly identical predictions. 9/10
Prikaži ovu nit -
𝗖𝗼𝗻𝗰𝗹𝘂𝘀𝗶𝗼𝗻 𝟮: Models employing generative internal models of the digits dominated discriminative models at accounting for human judgments. 10/10
Prikaži ovu nit
Kraj razgovora
Novi razgovor -
-
-
Hvala. Twitter će to iskoristiti za poboljšanje vaše vremenske crte. PoništiPoništi
-
Čini se da učitavanje traje već neko vrijeme.
Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.