Mason  🏃‍♂️ ✂️

A good test is one that successfully identifies a range of relevant competencies when: - The takers are not placed under time pressure - The takers have access to any resources they'd like to use that are generally available to them in life

One thing you have in real life is people who might just do the test for you. Or for anyone that pays them. Generally, as long as trade is possible it's very unclear what it means for someone to "have a competency" in this general sense.

"Generally available" is crucial, for this reason... but I'll point out that I've seen some genuinely ingenious tests, ones where the ability to vet & hire a replacement taker would in and of itself be an incredibly valuable skill

Interesting - what were these tests like/what did they test?

The strongest one was a play on the (now) somewhat well-known cognitive reflection test. I was actually involved in a hiring process that utilized it: applicants completed the test online, with unlimited time. Out of hundreds, a tiny handful of 10/10s. Most scored well under.

(By tiny handful, I *think* we had 2-3 perfect scores out of ~250-300 applicants, many with incredible credentials. Managing to hire someone who could deliver a perfect score would be super, super impressive.)

Looked it up - I have never heard of it! Though were the applicants told that what was being tested was their "tendency to override an incorrect 'gut' response"? Because that piece of information alone would seem to give you a huge advantage.