DATA 352/551: Modeling and Simulation
We can make random draws from common distributions (e.g. beta) using software directly (e.g. rbeta).
Generating from common distributions can usually be done using transformation, which usually only requires drawing from uniform (0,1). See transformation table from Computational Statistics (Givens and Hoeting, 2013).
Recall, in general, the distribution of a (continuous) random variable \(X\) can be specified by its probability density function \(f(x)\). In many cases we want to sample from this target distribution.
Challenges:
Many algorithms exist to tackle these challenges and allow sampling from a general target density \(f(x)\); the distribution of the resulting sample can be exactly \(f(x)\) (e.g., acceptance-rejection sampling) or approximately \(f(x)\) (e.g., SIR algorithm).
Many simulation methods involve an acceptance-rejection step. Suppose your goal is to simulate some data \(\{x_1, x_2, \ldots x_{nsim}\}\). Key idea of acceptance-rejection:
Q: How do you write code to “accept with probability \(p(x')\)”?
Ingredients:
Algorithm:
Our goal is to sample from the density function: \(f(x) = 6x(1-x), 0<x<1\).
Discussion: sketch this function (on paper or on the computer) and come up with a proposal density.
Use the provided sample code to implement the algorithm for a total of \(nsim=10000\) updates.
Note: The larger the value of \(C\), the less efficient the algorithm is. The acceptance rate is expected to be \(1/C\).
Suppose we want to estimate the probability of rolling a 1 of a fair dice using simulation:
Alternatively, since the value of 1 is more “important” for our estimate, we want to oversample this value, and then correct for the oversampling bias:
Why? The estimate we obtained from the second method has less variation!
Importance sampling is a general variance reduction technique when we use random samples to perform some statistical inference (e.g. estimating \(P(1)\)).
The sampling importance resampling (SIR) algorithm is an algorithm that generates approximate samples from a target distribution \(f(x)\), based upon the notion of importance sampling.
Ingredients:
Algorithm: (for \(m >> n\))
Use SIR to redo the previous exercise (sample code), where the goal is to sample from the density function: \(f(x) = 6x(1-x), 0<x<1\).
Use a uniform importance sampling function to sample 10000 values from the target.
Double check the random draws: calculate the mean (should be close to \(0.5\)); plot a histogram (should be close to \(f(x)\)).
Redo the sampling with a normal distribution importance sampling function.