Monte Carlo Simulation Basics, I: Historical Notes: Monte Carlo (MC) simulation provides us with a very powerful tool for solving all sorts of problems. In classical econometrics, we can use it to explore the properties of the estimators and tests that we use. More specifically, MC methods enable us to mimic (computationally) the sampling distributions of estimators and test statistics in situations that are of interest to us. In Bayesian econometrics we use this tool to actually construct the estimators themselves. I'll put the latter to one side in what follows.
To get the discussion going, suppose that we know that a particular test statistic has a sampling distribution that is Chi Square asymptotically - that is when the sample size is infinitely large - when the null hypothesis is true. That's helpful, up to a point, but it provides us with no hard evidence about that sampling distribution if the sample size that we happen to be faced with is just n = 10 (say). And if we don't know what the sampling distribution is, then we don't know what critical region is appropriate when we apply the test.
In the same vein, we usually avoid using estimators that are are "inconsistent". This implies that our estimators are (among other things) asymptotically unbiased. Again, however, this is no guarantee that they are unbiased, or even have acceptably small bias, if we're working with a relatively small sample of data. If we want to determine the bias (or variance) of an estimator for a particular finite sample size (n), then once again we need to know about the estimator's sampling distribution. Specifically, we need to determine the mean and the variance of that sampling distribution.
And there's no "magic number" above which the sample size can be considered large enough for the asymptotic results to hold. It varies from problem to problem, and often it depends on the unknown parameters that we're trying to draw inferences about. In some situations a sample size of n = 30 will suffice, but in other cases thousands of observations are needed before we can appeal to the asymptotic results.
If we can't figure the details of the sampling distribution for an estimator or a test statistic by analytical means - and sometimes that can be very, very, difficult - then one way to go forward is to conduct some sort of MC simulation experiment. Indeed, using such simulation techniques hand-in-hand with analytical tools can be very productive when we're exploring new territory.
I'll be devoting several upcoming posts to this important topic. In those posts I'll try and provide an easy-to-follow introduction to MC simulation for econometrics students.
Before proceeding further, let's take a brief look at some of the history of MC simulation. ...