U tweetove putem weba ili aplikacija drugih proizvođača možete dodati podatke o lokaciji, kao što su grad ili točna lokacija. Povijest lokacija tweetova uvijek možete izbrisati. Saznajte više
There are errors in this post. (1) The likelihood will collapse onto the "good function" as we increase the data size if the data are from the distribution we want to fit, as increasingly fewer bad functions will be consistent with our observations.https://twitter.com/jacobmbuckman/status/1218395271890997250 …
(2) A model being able to fit noise is unremarkable and is different than having an inductive bias that favours noisy solutions. A standard GP-RBF prior over functions easily supports noise, but it favours more structured solutions.
(3) The volume of good solutions exceeds the volume of bad solutions for typical problems NNs are applied to. Neural nets were constructed to have these inductive biases to help generalization. It is wild to suggest that a NN function is "generalization agnostic".
(4) In fact, you could easily create various "generalization agnostic" priors in function space. They would behave very differently from BNNs. They would have trivial structure and indeed would not generalize.
(5) Lack of desirable posterior collapse can happen when (i) the hypothesis space does not contain a good solution, (ii) the prior is too confident about a bad solution (e.g. equal label p for any x). But NNs are expressive, and (ii) is the opposite of a vague prior on weights!
Separately from the technical discussion, I suggest tending towards asking questions and learning, and being open minded about understanding BDL. Perhaps your prior is too strong! :-)
I will also note that my original response was actually not intended to be specific to Carles, but rather to address general remarks and confusions I had recently seen about BDL.
(1) may be true in theory, but in practice, NNs are so overparameterized (relative to data) that we never see a meaningful reduction in the set of bad functions. And recent work on double descent (https://openai.com/blog/deep-double-descent/ …) indicates that this might even be the ideal regime.
The addition of data certainly affects the shape of the likelihood and the posterior, otherwise standard training wouldn’t work. I agree the posterior is often diffuse in practice, but that’s actually a great argument for marginalization, which was a key point in my note.
I am not intimately familiar with BNNs but here’s my 2cts as someone who has worked with « classical » Bayesian statistics. It does not make sense to reason about priors on parameters in isolation. Priors can only be understood in the context of the whole architecture.
What you can say is « do these priors and this architecture allow me to cover the range of acceptable predictions? ». To tale the example of classification, if you have no reason to think the dataset is imbalanced
Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.
