Mike Love’s blog

Empirical Bayes and the James-Stein rule

Posted in statistics by mikelove on November 7, 2013

Suppose we observe 300 individual estimates y_i which are distributed \mathcal{N}(m_i, \sigma_y^2), with \sigma_y known.

Now if we assume m_i \sim \mathcal{N}(0, \sigma_m^2), the James-Stein rule gives an estimator for m_i which dominates y_i .

Below is code for a little RStudio script to see how changing the balance of variance between data y and the variance of the means changes the amount of optimal shrinkage. For more info, read the paper referenced below! It uses the RStudio’s manipulate library: info on that.

Screen Shot 2013-11-07 at 12.48.41 PM

# Stein's estimation rule and its competitors - an empirical Bayes approach
# B Efron, C Morris, Journal of the American Statistical, 1973
n <- 300
sigma.means <- 5
means <- rnorm(n, 0, sigma.means)
# sigma.y <- 5
  y <- rnorm(n,means,sigma.y)
  A <- sum((y/sigma.y)^2)/(n - 2) - 1
  B <- 1/(1 + A)
  eb <- (1 - B) * y
  plot(means, y, main="y ~ N(mean, sigma.y)\nmean ~ N(0, sigma.mean=5)")
  points(means, eb, col="red")
  legend("topleft","James-Stein estimators",col="red",pch=1)
  s <- seq(from=0,to=1,length=100)
  plot(s, sapply(s, function(b) sum((means - (1 - b)*y)^2)),
       type="l", xlab="possible values for B", ylab="sum squared error")
  points(B, sum((means - eb)^2),col="red",pch=16)
  legend("top","James-Stein B",col="red",pch=16)
}, sigma.y = slider(1,10))

4 Responses

Subscribe to comments with RSS.

  1. Austin said, on November 19, 2013 at 4:09 pm

    Hey Mike, there’s certain javascript thingies like http://www.mathjax.org/ that you can use to, on most modern browsers, display LaTeX or MathML trivially if you want!

  2. mikelove said, on November 19, 2013 at 4:29 pm

    ah yeah, I was being lazy and forgot that wordpress includes this. I just have to use dollarlatex y_i dollar

  3. Ali Gajani said, on December 20, 2013 at 1:40 pm

    Hey Mike, I was look at your Freebase Influence node, any tutorial or guidance on how to actually use that node to find out the list of people influenced & influencedBy. For example, I am looking to find the most influential man in history?

  4. Ali Gajani said, on December 20, 2013 at 1:40 pm


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s


Get every new post delivered to your Inbox.

%d bloggers like this: