Sam Power

@sam_power_825

PhD student in Maths & Stats at Cambridge. Interested in Bayesian statistics, MCMC, Numerical Analysis, and links between stats and other areas of maths.

Vrijeme pridruživanja: siječanj 2018.

Tweetovi

Blokirali ste korisnika/cu @sam_power_825

Jeste li sigurni da želite vidjeti te tweetove? Time nećete deblokirati korisnika/cu @sam_power_825

  1. 31. sij

    reference request: are there nontrivial examples of problems in variational inference where one can provably solve the KL minimisation problem? i.e. find global minimiser in polynomial time. the only one i'm currently aware of is fitting a Gaussian to something log-concave.

    Poništi
  2. 28. sij

    some nice finds on arXiv this morning: i) polynomial-time rejection sampling for a class of SSMs - ii) theory for Stein Importance Sampling - iii) approximations for parametrised Gaussian fields -

    Poništi
  3. 20. sij

    I've written up a short note (available at ) which give some more detail on my earlier sequence of posts about HMC with Finite-Difference Approximations.

    Poništi
  4. 12. sij

    anyways, i'm not kidding myself that people need HMC to sample one-dimensional targets, with or without gradients, but i did think that it was a neat concept. it would be nice if this happened to be useful somewhere else, maybe further down the line, but who knows. cheers! [8/8]

    Prikaži ovu nit
    Poništi
  5. 12. sij

    by transitivity of `closeness', we can infer that the finite-difference-based leapfrog scheme will generate a reasonable trajectory, which stays moderately close to the trajectory you'd like it to. note that we _don't_ need to send h to 0 for all of this to be true. [7/8]

    Prikaži ovu nit
    Poništi
  6. 12. sij

    so, by using a finite-difference approximation to the gradient, we're actually doing a regular leapfrog scheme, but on the wrong target. moreover, geometric integration tells us that regular leapfrog is _exactly_ solving *yet another* nearby modified Hamiltonian system. [6/8]

    Prikaži ovu nit
    Poništi
  7. 12. sij

    however, in this case, there's a secret saving grace: the symmetric finite-difference approximation to the true gradient, is the _exact_ gradient of another potential, V_h, which is close to V. [5/8]

    Prikaži ovu nit
    Poništi
  8. 12. sij

    it's worth highlighting that a priori, this is something which you might not expect to work well. HMC is notoriously brittle, and there are other seemingly minor changes to the algorithm which can compromise all the nice properties of HMC, e.g. subsampling. [4/8]

    Prikaži ovu nit
    Poništi
  9. 12. sij

    sometimes, one might not have access to gradients of V, and so simulating the dynamical system might seem out of reach. a naive thing which one can try is to replace the gradient by a finite-difference approximation. [3/8]

    Prikaži ovu nit
    Poništi
  10. 12. sij

    the standard thing to do in HMC is i) write down a Hamiltonian corresponding to your sampling task, ii) write down the equations for Hamiltonian dynamics (HD) iii) numerically integrate these equations by using the leapfrog (LF) discretisation [2/8]

    Prikaži ovu nit
    Poništi
  11. 12. sij

    worked out a goofy thing about HMC in dimension 1: if you replace exact gradients by finite differences, you do _surprisingly_ well. see if you can catch why it fails when d > 1. also, if you can figure out a way to make the idea work there, let me know! [1/8]

    Prikaži ovu nit
    Poništi
  12. 6. sij

    anyways - this stuff is _super_ speculative and incomplete, but i wanted to throw the idea out there, on the off chance that somebody might have thoughts on where one could take it. very open to comments and discussion 🙂 <7/7>.

    Prikaži ovu nit
    Poništi
  13. 6. sij

    one current barrier (among many) is that i'm unsure of how one should `optimally' combine {Hessian Estimates at x_1, ..., x_N} into (Hessian Estimate at x_{N+1}). i'm not sure whether this will work out neatly (though i would like it to), but i think it's a fun idea. <6/7>

    Prikaži ovu nit
    Poništi
  14. 6. sij

    at the moment, i'm trying to work out whether one can interpret Quasi-Newton methods in this way. it doesn't quite make sense for the path itself - the x's are updated iteratively, rather than recursively - but the Hessian estimates are essentially treated recursively. <5/7>

    Prikaži ovu nit
    Poništi
  15. 6. sij

    one challenge is then to identify other sequential algorithms which are fundamentally recursive, and understand what it might mean to extend them to tree-structured problems (and whether it might make sense to apply them to general graph structures anyways, c.f. loopy BP). <4/7>

    Prikaži ovu nit
    Poništi
  16. 6. sij

    a slightly surprising instance of this, which i only came across recently, was this work on `Divide-and-Conquer SMC’: . it highlights quite nicely that the essence of SMC is about recursion, rather than sequential structure. <3/7>

    Prikaži ovu nit
    Poništi
  17. 6. sij

    some familiar examples of this are the Kalman filter and the Forward-Backward algorithm, which are both formulated for different kinds of state-space models, but each make sense for general tree graphs, once you rewrite them as an instance of belief propagation. <2/7>

    Prikaži ovu nit
    Poništi
  18. 6. sij

    i've been thinking recently about recursive algorithms, and in particular, about one neat observation: even though lots of recursive algorithms are primarily used for sequential tasks, they can typically be extended to tree-based problems <1/7>.

    Prikaži ovu nit
    Poništi
  19. 1. sij

    very interesting new paper on arXiv this morning - `Schrödinger Bridge Samplers', by E. Bernton, J. Heng, , . Roughly speaking: uses ideas from Optimal Transport + Optimal Control to devise improved SMC Samplers.

    Poništi
  20. 30. pro 2019.
    Poništi

Čini se da učitavanje traje već neko vrijeme.

Twitter je možda preopterećen ili ima kratkotrajnih poteškoća u radu. Pokušajte ponovno ili potražite dodatne informacije u odjeljku Status Twittera.

    Možda bi vam se svidjelo i ovo:

    ·