What household product takes off self-tanning lotion that's been
applied too heavily?

The product should have a help/customer service phone number on it
somewhere. Try asking them.

Several years ago, I helped a friend apply self-tanning lotion, and my
hands were discolored as a result. Hydrogen peroxide solution and
vigorous scrubbing with an abrasive sponge helped me to un-tan my
hands. I don't know if this will work with all types of self-tanners,
though.

I scrubbed my hands with St. Ives Invigorating Apricot Scrub for about
10 minutes when it happened to me last year.
http://www.stives.com/products/fc/ac/01_ac_ias.cfm

After thinking about this a bit more, try Tea Tree Oil, just on a cotton wool ball.

Goodluck.

I was told to use windex window cleaner to remove the stains