Is there a convenient way to implement SMC likelihood tempering using Gen/GenParticleFilters?
I want use SMC for parameter inference on static models with complex multi-modal posteriors and broad uninformative priors. The intuition is that very few of the particles sampled from the prior will agree well with the posterior. There will need to be a series of resampling and rejuvenation steps to help the particles settle in the right places and avoid particle degeneracy along the way.
The likelihood tempering strategy is to automatically derive a series of distributions to bridge the distance between the prior and the posterior. The bridging distributions are defined by taking any black-box model and tempering its final likelihood score. Summed up well by this figure:
which shows a prior distribution (beta=0) and a posterior (beta=1) and two intermediate bridging distributions.
The challenge I see with Gen is that we don't have an explicit likelihood function that could be tempered with a wrapper function e.g. can't write
tempered_model(model, temp=0.5) = args -> model(args) ^ temp
and I'm not sure if there's an obviously correct way to hack trace.score within an inference method to affect tempering of SMC resampling/rejuvenating/reweighting steps without screwing up some important invariants on Gen data structures.
More references to the SMC likelihood tempering idea:
Any hints about how to approach this problem?
I have experimented with a "white-box" approach of defining models with their own explicit internal tempering but I haven't found a satisfactory solution here. I think the automatic approach is very elegant and would like to find a way to make it work with Gen.
Is there a convenient way to implement SMC likelihood tempering using Gen/GenParticleFilters?
I want use SMC for parameter inference on static models with complex multi-modal posteriors and broad uninformative priors. The intuition is that very few of the particles sampled from the prior will agree well with the posterior. There will need to be a series of resampling and rejuvenation steps to help the particles settle in the right places and avoid particle degeneracy along the way.
The likelihood tempering strategy is to automatically derive a series of distributions to bridge the distance between the prior and the posterior. The bridging distributions are defined by taking any black-box model and tempering its final likelihood score. Summed up well by this figure:
which shows a prior distribution (beta=0) and a posterior (beta=1) and two intermediate bridging distributions.
The challenge I see with Gen is that we don't have an explicit likelihood function that could be tempered with a wrapper function e.g. can't write
and I'm not sure if there's an obviously correct way to hack
trace.scorewithin an inference method to affect tempering of SMC resampling/rejuvenating/reweighting steps without screwing up some important invariants on Gen data structures.More references to the SMC likelihood tempering idea:
Any hints about how to approach this problem?
I have experimented with a "white-box" approach of defining models with their own explicit internal tempering but I haven't found a satisfactory solution here. I think the automatic approach is very elegant and would like to find a way to make it work with Gen.