What is the power spectrum of a signal consisting of normally
distributed noise?   Is there a formula that relates the standard
deviation of the normally distributed noise to the power spectrum of
the noise?

I think I have read somwhere that the power of the signal is related
to the standard deviation of the normal distribution, but I have not
been able to confirm this.

Hello and thanks for your question.


I'll assume that when you say normal noise you mean white (Gaussian)
noise, also known as Johnson noise or thermal noise.  In white noise,
by definition, the power density is constant over all ranges of
frequency.
In other words,
Noise Power= kT [delta]f
or
Noise Power per hertz = kT
where k = Boltzmann's constant in joules per Kelvin, T = temperature
in Kelvin degrees.  So again, the power density is equal throughout
the frequency spectrum, depending only on k and T:
     "Thermal Noise"
http://www.atis.org/tg2k/_thermal_noise.html

This relationship is fully explained (and the power spectrum that you
specifically asked about is illustrated) on pages 12-13 of the
following Adobe Acrobat document.  I really can't summarize it here
because you need to see the graphics and I'd need a Greek alphabet
anyway, so please look at the following:
     "Power Spectra"
http://www.upscale.utoronto.ca/GeneralInterest/Harrison/TimeSeries/PowerSpectra.pdf

And here's a longer document that provides a more complete explanation
(again, please look at the .pdf file for the equations and graphs:
     "Op Amp Noise Theory and Applications"
http://www-s.ti.com/sc/psheets/sloa082/sloa082.pdf

I should close by remarking that this is a highly complex and
technical area.  If you want to see more of the analysis, there's an
article ironically called "Basics" that goes deeper into this
correlation than I can hope to explain:
     "Basic Definitions and Theorems about ARIMA models"
http://www.xycoon.com/basics.htm


Other useful links:
     Chem 524 Notes
http://www.chem.uic.edu/chem524/notes10/notes10.html

     Distributions
http://astronomy.swin.edu.au/~pbourke/analysis/distributions/


Search terms used:
     Johnson noise
     "power spectrum" "standard deviation" noise
     "standard deviation" "power spectrum"  gaussian "johnson noise"


Please let me know if you have any questions after checking out these
sources, and thanks again!
Richard

---

Request for Answer Clarification by noisy-ga on 20 Jun 2002 15:15 PDT

Hi Richard,

Thanks for your prompt answer to my question.  After re-reading my
original question I see I didn't  word it very well.  I have read the
material you provided, and would like a bit of clarification.

 I am not sure if you've read the comments from Hedgie and West
regarding my question, but (brief recap), I was interested in
determining the power from a set of values that are normally
distributed.  Originally I was advised (by one of my lab-mates) to
generate the power spectrum of the values, and then integrate under
the curve of the power spectrum across the range of frequencies of
interest  to get the total power over that range.   Because the amount
of power in each frequency component is a function of the amplitude
(value) of the signal (time series of values), and since the
amplitudes (values) can be characterized by the standard deviation of
the normal distribution (eg: 67% of the values will be less than the
sd, 99.9% of them will be less than 3*sd), I think it should be
possible to determine the total expected power from just the standard
deviation and the range of values of interest, without having to
actually calculate the power spectrum and do the integration.

The relation that West gave between standard deviation and power for a
circuit is the closest I've seen to a formula that can be used to skip
the power spectrum-integration steps, however since my values are not
from an electrical circuit, I'm not sure if it's modifialbe to my
situation, and I'm starting to think that a nice tidy (READ:
non-Fourier transform) solution to my problem is a bit pie-in-the-sky.
 Do you have any additional insight into my problem?

Thanks for your time,
Noisy.

---

Clarification of Answer by richard-ga on 20 Jun 2002 17:54 PDT

I think West has the best approach.  The confusion I think arose from
the different possible noise distributions.  As you're aware, the
standard deviation i.e. central distribution approach gives you the
answer you seek IF we're safe in assuming that your noise is normally
distributed.
So it's the shape of the curve, not whether it's electrically or
sonically generated, that matters.

I hope you enjoyed this exercise--I certainly did!

richard-ga

Dear noisy,

Your recollection that noise power is related to the standard
deviation of the normal distribution is correct. The relationship is
as follows:

The standard deviation equals the root mean square (r.m.s.) value of
the normally distributed noise waveform. Therefore if the noise
voltage is developed across a resistor, the noise power is simply the
square of the standard deviation divided by the value of the resistor.

There are good lecture notes on Random Signals and Noise at the
following link. See Chapter 8 'Power Density Spectrum' equation 8-10
for the Power Density Spectrum.

I hope this contributes to your understanding.

Sorry noisy I forgot to give you the lin which is as follows:

http://www.eb.uah.edu/ece/courses/ee420-500/

Dear noisy

 This is just a short note, not to disagree withe my esteemed fellow
researchers, but to assure
you that does not have to be all that complex.  Einstein said :
explanation  be as simple as
 possible ..  , let's try that:

The phrase you use, normally distributed noise, is a bit ambiguous,
but most likely meaning is
that the power spectrum of that noise is normal, aka gaussian aka Bell
curve shaped.
 This is not the same as white noise, which means (usually) uniform
(flat) distribution.

So, unless you make question more exact, the best answer is: the
relation is identity:
 half-width of the power spectrum is the standard deviation of signal
around its mean.

Here is a simple into to Gaussian (normal) distributions
http://www.umich.edu/~orgolab/Chroma/chromahow3.html

 hedgie

Dear hedgie

Totally agree with keeping explanations as simple as possible.

However I must point out that for almost all noise phenomena the
Central Distribution Theorem applies.

Therefore the Normal (otherwise known as Gaussian) Distribution is
appropriate and tells us the likelihood of the value of the noise
waveform attaining any particular magnitude.

Quick self-correction

Yes, west-ga. The Gaussian  distribution in question is most likely
result of the
Central Distribution Theorem which inddeed is likely to apply. 

I am a bit woried about the noisy-ga who did not came back  to clarify
and probably regrets
s/he ever asked a question evoking statements such as 
  "(r.m.s.) value of the normally distributed noise waveform .."  the
meaning of which,
 even we, the experts, may ponder and argue about.  I, in my eagerness
to simplify have forgotten
the second half of Einstein's quote ' ..but not simpler' and said the
relationship  is identity.

Well,  depending on the meaning of the asker's term "normally
distributed noise' it may actually
be an inverse relationship -a  mathematical version  of  the
uncertainty principle.

As a pennance, I will describe a scenario: Lets imagine that the the
noisy signal is repeated measurement of some physical quantity.  The
measurement repeated at regular intervals of
will yield a gaussian distribution of measured values, as described in
any  elementary
 error analysis  recepie, e.g.
 http://www.physics.lsa.umich.edu/IP-LABS/Errordocs/stddev.html 

Now, if we do the  Fourier transform of that distribution, we get a
power spectrum, which is also
Gaussian. There is a relationship between the half-width of these two
gaussians,
 relation  between the uncertaneity in frequency and in time: 
http://sepwww.stanford.edu/sep/prof/waves/rnd/paper_html/node2.html

 But, unless noisy cames back,  we may never know what memory s/he was
chasing
 and what scenario or experiment s/he had on mind ..

Dear Hedgie-ga and West-ga,

Thank-you for your comments, and also for the links you've suggested,
I'm eager to take a look at them.  I'm sorry I haven't been able to
clarify my question sooner, but it took me a while to get through the
material suggested by Richard-ga ;).  I'll try to make the
clarification less ambiguous than the question:

The data that I am looking at are from an experiment very similar to
the one hypothesized by Hedgie.  I have a time series of values (in
this specific case they are supposed to represent some voltages) and
the values are assumed to be normally distributed (N(0,sd)).  At first
I was thinking about writing a program to generate the power spectrum
of the data, but I remembered from a course long ago something about
the power of the power spectrum of Gaussian noise being related the
the standard deviation of the time series, however I couldn't quite
remember.  I thought if I could could remember the relationship
between the standard deviaition of the noise in the time series to the
power in the power spectra (or get someone to find it for me ;) I
could save myself some programming time.

So far from the materials I've read, I haven't found a formula for the
relationship between power and standard deviation, however it has been
ages since I took a course on this type of math, so it's quite
possible I could be mistaken about that.

Thanks again for your comments, they really helped!

Dear noisy-ga
Thank you for your comment. Having grasped the gist of your experiment
and your need to find the power spectrum, I have concluded as follows:
1. Your slightly hazy memory of use of the standard deviation (sd)
actually relates to the calculation of power and not to the
calculation of the power spectrum. The sd does not appear to be useful
in arriving at the power spectrum.
2. You should proceed to use a program to arrive at the power
spectrum. A suitable program is described at the following link:

http://sprott.physics.wisc.edu/cdafaq.htm

Hello noisy-ga, richard-ga and hedgie-ga,

Just for the record, my earlier reference to the 'Central Distribution
Theorem' should have read 'Central Limit Theorem'. It's pretty obvious
my meaning has been understood. Sorry for the slip.

west-ga

I just want to say that I think there has been a great deal of
over-complication and misunderstanding here.  Noisy just wants to know
what relation there is to the standard deviation of some Gaussian
noise and the power spectrum.  Simply, the power spectrum of noise
provides an estimate (usually good) of the CONTRIBUTION OF EACH
FREQUENCY TO THE TOTAL VARIANCE.  That means that the TOTAL POWER is
equivalent to the VARIANCE.  The variance is the square of the
standard deviation.  It sounds like that is all you really needed to
know.

Now by looking at the power spectrum, you can determine whether most
of your variation is coming from high frequency events (more power at
high frequencies) or from slowly changing low frequency events (more
power at low frequencies).  You may find that it is fairly evenly
distributed over all frequencies.  This would be the case of white
noise mentioned before--white because it is an equal mixture of all
frequencies.

With that settled I would like to clear up a big mess.  There is a big
difference between the probability densty function, PDF, (e.g. the
familiar Gaussian curve which shows how many times each even ocurrs)
and the power spectrum.  The PDF has absolutely no information of how
the events have evolved in time.  As far as the PDF is concerned, the
events could have happened in ascending order, decending order or
completely random.  The PDF, a simple counting of events, will not
change.  When the PDF is normalized to the total number of events it
is called a histogram.

Now, the power spectrum has a lot of information about the time
evolution of the process, but (almost) none about the number of
individual events.  If both the power spectrum and the phases of the
frequency componants are known, then the PDF can be found by
reconstruction the original time series.  However, if only the power
spectrum is known, then NOTHING AT ALL CAN BE SAID ABOUT THE PDF!  For
those not familiar with this fact, I suggest trying it out with a
numerical simulation.  Take a given power spectrum and give it
different phases.  Then reconstruct a timeseries through inverse fft. 
You will find completely different PDF's!

It is worth saying one more thing about power spectra.  I would like
to remind everyone of the Werner-Khinchine theorem.  THE POWER
SPECTRUM IS THE FOURIER TRANSFORM OF THE AUTOCORRELATION FUNCTION. 
So, another way to view the time evolution is in terms of time lag
correlations.  By the properties of the Dirac delta function, or
simply the definition of autocorrelation, the zero-lag autocorrelation
is equal to the VARIANCE or TOTAL POWER (same thing).

As a final note, I will clear up one more thing.  It is possible to
Fourier transform a PDF.  This has ABSOLUTELY NOTHING TO DO WITH THE
POWER SPECTRUM--it is not in the time domain.  Suppose you have a
probability distribution p(x).  Then the Fourier transform P(k) is a
function of wavelength, and is called the CHARACTERISTIC FUNCTION of
the PDF and has some nice properties in general.  In specific, the
Gaussian is the unique distribution for which the shape of the PDF
p(x) is the same as that of the characteristic function P(k).  If
     p(x) = (2*pi*sigma)^-1/2 * exp[ -1/2 * ( (x - mu)/sigma )^2 ]
then
	P(k) = integral p(x) * exp(-i*k*x) = exp( -1/2*sigma^2*k^2 + i*k*mu )
I think someone brought that up, but it has nothing to do with power
spectrum (time domain) analysis.

All the best to everyone (especially noisy).  I hope this helps!

Dear abalter-ga

Firstly there is an error in the following statement in your first
paragraph:
" That means that the TOTAL POWER is equivalent to the VARIANCE". This
statement should read " That means that the TOTAL POWER is
proportional to the VARIANCE". This corrected statement was already
demonstrated by the equation in my first comment on 18 June.

Secondly nobody has suggested that noisy-ga tries to derive the Power
Spectrum from the PDF or vice versa. Basically I suggested that
noisy-ga use a program to process the data from the time series to
arrive at the Power Spectrum.

By the way I'm very pleased to see you confirm that the standard
deviation is of no use in calculating the power spectrum!