Interweaving Gibbs samplers
This exploration looks at the interweaving sampler of Yu and Meng. The model is Y~N(theta,1+V) with V known and two different data augmentation schemes are considered:
- Sufficient Augmentation: Y~N(Ymis,1), Ymis~N(theta,1)
- Ancillary Augmentation: Y~N(Ymist+theta,1), Ymist~N(0,V)
Ymis and Ymist (Ymis tilde) are the augmentations. The second is named ancillary since the distribution for the augmented data does not depend on the parameter. The first is named sufficient because the full conditional distribution for the parameter, theta, does not depend on the data: theta|Ymis, Yobs ~ N(Ymis, V).
First, we set up the full conditionals for SA:
Next, we set up the full conditionals for AA:
We also need the mapping between the two augmentations.
Now, we build a generic MCMC function that will run all of the schemes where AA is the MCMC associated with the ancillary augmentation, SA is the MCMC associated with the sufficient augmentation, ALT is the MCMC where a single iteration is SA followed by AA, and ASIS is the Ancillary-Sufficiency Interweaving Strategy introduced in Yu and Meng.
Notice that the only step that is different between ASIS and ALT is the 3rd step where ASIS has a deterministic step while ALT has a sampling step. Yet, as shown in the paper, ASIS leads to independent draws from the posterior while ALT leads to positively correlated draws.
Now let’s look at the autocorrelation in the chains for theta from these different MCMC schemes for various values of V.
Notice how AA has high autocorrelation when V is large, SA has high autocorrelation when V is small. Both the ALT and ASIS samplers have low autocorrelation everywhere. The difference between ALT and ASIS is not obvious here, but remember they are essentially the same sampler except ASIS has a deterministic step where ALT has a sampling step and ASIS obtains independent samples from the posterior.
blog comments powered by Disqus