François CholletVerified account

Interesting idea. The system that currently plays this role is Arxiv, but it is extremely non-optimal for this purpose

Would it be better to start with best practices through libraries ex. Keras / Tensorflow i.e. code or are there best practices before code?

Also, could link back to Arxiv as a reference to why something is a best practice. Having a list/agg is extremely useful esp for newer folks

That's a good idea, but would be very time-consuming to pull off. Need a large group, and need lots of expertise.

Two problems that will arise are credit assignment and conflicts of interest (pushing one paper over another). Classic academic problems

I would start such a project with topics that are not typically the subject of papers (hence information harder to come across)

e.g. best practices for data cleaning / tokenization / vectorization, optimization / training, parameter tuning, regularization

the sort of knowledge that people tend to build from experience rather than from reading papers.