Has anyone done the equivalent of an MRI study on gtp3 to see if it's neurons organize concepts like we do?
Conversation
I haven’t seen the techniques applied to GTP-3 in particular but have you seen other work to “interpret” ML models? Some examples I had bookmarked, for example distill hub
distill.pub/2020/circuits/
distill.pub/2018/building-
distill.pub/2021/multimoda
1
3
Good god. makes the *best* explanatory papers and articles. I hadn't looked at their full catalogue much but these are so well presented.
(Tangential to the point, apologies. But blown away)
Looks like they went on hiatus:
1
1
We were pretty burnt out. :/
There's successor to the Distill circuits thread though, focused on language models! transformer-circuits.pub
1
1
3
Show replies