StackOverflow2013

Note that there are some explanatory texts on larger screens.

plurals

PO
text
Body
copied!<p>It's one thing to give the code, another to give an explanation. If you just want to look at the spectra, try the following:</p> <pre><code>import numpy as np import matplotlib.pyplot as plt hh = np.hanning(len(term1)) # Use a Hann window to deal with spectral leakage St = np.fft.rfft(hh*term1) Sb = np.fft.rfft(hh*baseline) FSample = 1.0/24 # Sampling frequency is 1/24th of a day deltaF = 1.0/(len(term1)*FSample) # Frequency resolution is 1/capT = 1/(NumSamples*FSample) faxis = deltaF*np.arange(len(St)) plt.plot(faxis, np.log10(np.abs(np.array([St, Sb]).T))) </code></pre> <p><img src="https://i.stack.imgur.com/2xXXA.png" alt="Spectra of term1 (blue) and baseline (green)"></p> <p>See Wikipedia's explanation of <a href="http://en.wikipedia.org/wiki/Spectral_leakage" rel="nofollow noreferrer">spectral leakage</a> and <a href="http://en.wikipedia.org/wiki/Window_function" rel="nofollow noreferrer">windowing</a> for details on <code>hh</code>. The x-axis of the plot is calibrated in days. There are two thin peaks in the <code>baseline</code> spectrum at 1 and 2 day periodicity, and a broad peak around 1 day in the <code>term1</code> spectrum.</p> <p>It wasn't clear to me from your question if the baseline data already had the real data stripped out. If so, I think this shows that there isn't much structure in one that's distinguishable from the other.</p> <p>Another thing to keep in mind is that Fourier analysis assumes the signal is stationary. Depending on the physics of what you're measuring this assumption may or may not be true. Certainly, the time-domain plot of <code>term1</code> doesn't look at all stationary, with a wild change in character starting around day 12:<img src="https://i.stack.imgur.com/bVeKW.png" alt="np.plot(np.arange(len(term1))/24.0, term1)"></p> <p>Having said all that, if you have a way to characterize the baseline data, you might be able to apply noise cancelation algorithms like the <a href="http://en.wikipedia.org/wiki/Least_mean_squares_filter" rel="nofollow noreferrer">Widrow-Hoff LMS algorithm</a>. Wikipedia presents a very theoretical overview, I'm not sure where to find a more practical application oriented explanation.</p> <p>How did you come up with the <code>term1</code>/<code>baseline</code> separation in your example data?</p>

Querying!

Guidance

An individual column

Larger individual text columns get their own page to allow for proper reading.

SQuiL has stopped working due to an internal error.

If you are curious you may find further information in the browser console, which is accessible through the devtools (F12).

Reload