Jason Antic

I’m motivated to point all this out because I think the importance of math prerequisites is a bit overemphasized and might be keeping a lot of otherwise talented people from giving the fields a try.

Linear algebra and calculus an important pre requisite in your opinion? Or do you reckon you can get pretty far just by applying and building upon code in the wild.

1/ I don’t think either are required and agree with @fastdotai that algebra suffices. The core concepts from linear algebra and calculus that are most useful to know are very easy to get the basic gist on quickly (derivatives and matrix multiplication).

2/ My personal approach to figuring stuff out really boils down to a combination of leaning heavily on experimentation, holding all ideas of how things as temporary and subject to change, sprinkle in cross pollination from other domains/papers, and good old fashioned problem

Hmm, I'd say a solid understanding of linear algebra and calculus are required; it saves time in the long run to be able to identify what math is decoration and what is essential. Having access to lower level building blocks also allows for faster understanding, recall.

The risk of taking a code heavy approach, and I see this a lot, is you end up thinking in terms of frameworks instead of concepts. For example, rather than decomposing tasks in terms of Pytorch functions, or worse/higher--BERT--you ask, what is self-attention really doing?

Dont disagree, but for me some of that stuff would scare me away. Better to hit the glass ceiling first then return to bare bones once that happens. The question is more how quickly will that happen or what can you achieve / build without it.

I think you're completely correct and I don't think front-loading on prerequisites works. But I do think it's necessary to approach it as wanting to understand the math in order that necessary connections become more likely to take hold. In the long run you save on effort