You really, really should not be relying on AI detectors for classroom use.
This new paper shows that not only are they very easy to defeat by just prompting a couple of times, but they have insane false-positive rates against non-native English speakers. arxiv.org/abs/2304.02819
Conversation
Other reasons detectors don’t work:
1) They are often trained on GPT-3.5, so GPT-4 beats them
2) Even if they alert you to potential AI use there is no way to see if that is true
3) Students working interactively with AI defeat tests, as my class found👇
4
14
107
Also, never ask an AI to identify whether something is written by AI. It cannot do it, but might make up an answer randomly if pushed enough.
1
7
59
When I tested the detector from Winston AI in late May, I found it was highly effective. I couldn’t out prompt it. I will take a closer look at this study —and the bit about non native writers is interesting—but I caution against your categorical assertions as everything evolves.
2
2
Nope. I am willing to be categorical about it. AI detectors do not work well and have false positives. You should not use them.
3
2
58
Show replies
Can't believe that these could work at all. Definitely not after an iterative prompt
1
I would go further, not only do they not work, they WILL NEVER work. Its pretty simple, not enough entropy. Its like handing someone a number and asking if that one number was randomly generated.
1
15
Show replies
Automated
Well, duh. AI detectors are about as useful as a screen door on a submarine - they might look good, but they're not gonna do much to stop the flood of problems. Stick with good old fashioned human intuition instead. #TrustYourGut #AIisOverrated 🤖💩







