[1] 1 1 1 1 0 1 0 1 0 1 0 0 0 0 1 1
[1] 1.345708
DATA 352/551: Modeling and Simulation

In both projects, data is binary (e.g.: win, win, win, lose, lose, win), and a goal is to quantify “streakiness”
One solution (Bayesian Binary Segmentation Procedure for Detecting Streakiness in Sports): calculate a statistic called \(B_{10}\). Data is considered streaky if \(B_{10} > 1\)


\(B_{10}\) is a statistic (i.e., it’s calculated based on data). We need to evaluate its statistical significance by looking at p-values or confidence intervals.
If I randomly generate a data sequence (not streaky), is it possible that I get a \(B_{10}\) value greater than 1 (streaky) just by luck?
Idea: Say \(B_{10} = 1.35\) for my actual data. I will randomly sample none-streaky data and get their \(B_{10}\) values to see if \(1.35\) is considered extreme.
Write code to randomly sample none-streaky data (consider randomly flipping a coin)
Exercise: Run a simulation (1000 times) to estimate the reference distribution: i.e., repeat the previous step 1000 times and record the \(B_{10}\) value each time.
A classic example in statistics, due to R. A. Fisher: lady tasting tea
[1] "M" "M" "T" "T"
result
0 1 2 3 4
157 2316 5071 2329 127
In the “lady tasting tea” and SES examples (see homework 8), the reference distribution is easy to derive mathematically. Simulation is more useful with difficult test statistics (like \(B_{10}\)).
For the lady tasting tea example, if she gets 4 out of 4 correct, we would reject the null and conclude that “we have enough evidence that she is not guessing.” This DOES NOT prove that she is definitely not guessing. All we are saying is that “it’s difficult to get 4 out of 4 correct just by random chance.”
…if she gets 3 out of 4 correct, we would “accept” the null and conclude that “she is guessing.” This DOES NOT prove that she is definitely guessing. All we are saying is that “it’s very possible to get 3 out of 4 correct just by random chance.”
Monte Carlo methods refer to a very broad class of methods that rely on using random sampling to solve otherwise difficult problems (example: bootstrapping, Monte Carlo integration, empirical p-value).