Printer Friendly

A method for finding unknown signals using reinforcement FFT differencing.


This note addresses a simple yet powerful method of discovering the spectral character of an unknown but intermittent signal buried in a background made up of a distribution of other signals. Knowledge of when the unknown signal is present and when it is not, along with samples of the combined signals for each case is all that is necessary for this method. The method is based on reinforcing Fast Fourier Transform [1] (FFT) power spectra when the signal of interest occurs and subtracting spectra when it does not. Several examples are presented. This method could be used to discover spectral components of unknown chemical species within spectral analysis instruments such as Mass Spectroscopy, Fourier Transform Infrared spectroscopy (FTIR) and Gas Chromatography. In addition, this method can be used to isolate device loading signatures on power transmission lines. The examples shown within this paper are based on deterministic sinusoidal composite signals.



Let's start by introducing a simple thought problem: how might one isolate and identify the spectral trace of a recurring chemical species amongst a series of other randomly occurring chemical species. This is akin to isolating a chemical spectral fingerprint amongst a plethora of unknown chemical spectral fingerprints [2]. Moreover, such chemical fingerprints are deterministic signals, however their occurrence or lack there of can be random when samples are obtained from say composite materials such as rock samples. Similarly, one might be interested in identifying the spectral fingerprint of a device being powered on and off via a power distribution transmission line along with numerous other devices also being turned on and off at seemingly random occurrences. Once again the start-up/ shutdown signal might be primarily deterministic given standard power-on and power-off firmware actions programmed into many devices today.

The key to both of these problems is that one desires to know what portion of any given composite spectral signal contains our unknown chemical species fingerprint or device signature. This unknown signal, in this case the spectral component due to our unknown signal, is a "hidden" signal within the larger composite of many unknown or known signals. In solving this problem, we will only assume that we know when our fingerprint exists or does not exist within its sample signal segment. For the example given in Sec. 3 we also constrain the spectral components of the background signals to exist throughout the duration of each sample signal segment (but allow the content of the background signals to change from sample segment to sample segment). The examples given in Secs. 4 and 5 do not have this restriction. Instead background signals are allowed to start and stop throughout a sample segment, e.g. a particular spectral component might only exist in the first half of a signal sample segment.

This approach will work for any positive statistic, e.g., periodogram, magnitude of Fourier Transform, or a wavelet decomposition. In our efforts, we will consider the following positive statisitics: the power spectra developed from the squared magnitude of a FFT signal segment, and the square-windowed periodogram of a signal segment.


The method entails estimation of a power spectrum of an unknown signal by differencing the power spectra of combinations of signals with and without the unknown signal of interest. It is this combination that is used to reinforce/isolate the power spectral contributions of the of the unknown signal while reducing the contributions of the undesired signals, i.e. the clutter. This is done simply by subtracting the power spectra of the samples without the unknown signal from the sum formed by adding the power spectra of the samples containing the unknown signal. We only assume a priori knowledge of when the unknown signal is present and when it is not present within the power spectra being used. In practice, this knowledge might be gained by noting when an unknown chemical substance is contained within a process by a change in the substance's color or it's chemical reactivity.

This process can be stated as an average difference of a positive statistics, e.g. the power spectra, with and without the unknown signal, i.e. the statistic, [PS.sub.full], can be estimated as follows:

[PS.sub.full] = 1/N ([N.summation i=1] [PS.sub.i] - [M.summation of j=1][[bar.PS].sub.j]), (1)

where the [PS.sub.i]'s are positive statistics of signal segments containing the desired representation of the signal (the composite), and [bar.PS].sub.j]'s are the same positive statistics not containing the signal (the background). Moreover, N is the number of samples containing the desired representation of the signal, i.e. its fingerprint, and M the number of samples without it. It should be noted that N need not equal M, although the effectiveness of this method degrades as they grow apart from each other. As a clarification, both the composite and background segments are made up of background signals that start and end throughout each sample segment. Nevertheless, we will consider both cases when the background spectra durations are constant across each sample segment as well as when they are allowed to vary across a sample segment. When allowed to vary, background signals need not exist across the full segment, only during some portion of its duration. Our work does not fully investigate the limits of how short a duration is feasible, but it appears that a 50% duration does not create any issues for this method within our studies to date. In this work, we did not consider limiting the duration of the unknown signal of interest's support across a sample segment, i.e. it exists across the entire duration of the sample.

In our examples, we choose the positive statistics to be the power spectra of numerous combined signals, both with and without our signal of interest. The power spectra calculated in Secs. 3, 4 and 5 clearly show the spectral characteristics of the unknown, intermittent signal. In this paper, we use two similar statistics for PS's, a standard, square-windowed periodogram and the square magnitude of the Fourier transform. Eqs. (2) and (3) show how the latter is calculated.

[PS.sub.i] ](|FFT(sample of composite,)|).sup.2], i = 1... N (2)

[bar.PS].sub.j] ](|FFT(sample of [background.sub.j])|).sup.2], j = 1 ... M (3)


Given a composite signal containing many individual components, the method we propose can isolate and identify an itermittent but unknown component of that signal given a composite of signal samples of when the unknown signal is present and when it is not. Fig. 1 shows an example composite signal in question as the sum of the unknown signal and a multi-spectral background signal.

The unknown signal in this demonstration (Fig. 1, middle frame) shows an harmonic sum with random phases and amplitudes. It will be added to the background signal at intervals and then the method of reinforcement differencing will be applied to the composite signal to ascertain the power spectrum of the unknown signal.

The top frames of Fig. 2 show the power spectra of the unknown signal determined as described by Eq. (1). The Periodogram-based results in Fig. 2 (b) were derived by using a standard, square-windowed periodogram for PS and [bar.PS]. The FFT-based results in Fig. 2 (a) were obtained by using the square magnitude of the FFT, [(|FFT|).sup.2], for PS and [bar.PS].

For comparison, the middle frames of Fig. 2 show the power spectra derived directly from the unknown signal in Fig. 1 before being combined with the background signal. The bottom frames show the power spectra from the combined unknown and background signals. The small peaks in the top frames of Fig. 2 are a result of the algorithm's amplification of high-frequency noise, but are negligible when compared with the large peaks representing the signal. This noise is believed to be largely due to computational roundoff and spectral bleed occurring within the computational process. Additional work remains on improving the noise artifacts shown in Fig. 2. Nevertheless, the current method remains very useful.



In this section we apply the reinforcement method to a variation on the theme of the previous example. Instead of keeping the a constant background signal with time-invariant spectral components throughout each sample (while allowing it to change from sample set to sample set), we allow the spectral components to change midway through each sample set. Fig. 3 is a close-up of a particular sample showing the change in the background signal at the midpoint. The sample is namely, the sum of the unknown signal and the two partially time-invariant signals (i.e. each background signal being time-invariant over each half of the sample period).

Fig. 4 compares the power spectra estimation of the unknown signal in this new, changing background just as in Fig. 2. The top frames of each figure show the power spectra of the unknown signal determined by the method of reinforcement differencing, while the middle frames show the power spectra calculated directly from the unknown signal: The bottom frames show the power spectra of background and unknown signals combined.





As a final example of the method of reinforcement differencing, we apply it to the problem of non-intrusive load monitoring [3]. Specifically, our method is applied to the contrived problem of detecting a circuit fault in a power distribution system.

In this example, we create a series of random signals to represent the electrical current usage signal at the input to a residence as measured at the main power feed. Then we simulate a circuit breaker failure by adding an additional intermittent signal to the composite household signal and use reinforcement differencing to isolate and determine the power spectrum of the failure signal.

Once we have obtained the signal fingerprint on the main power system input, we could develop a detection system to diagnose this type of failure within a house. An additional application of this process is for monitoring of a machine's activation within a large plant so that appropriate compensation steps might be taken to load balance the overall real and imaginary power loads within the plant by starting up an additional compensating machine.

We begin by simulating a background signal by summing several distinct power signatures. Each component of the background signal could be thought of as a time trace of the current drawn by a certain device or appliance as measured at the home's main power feed. For example, the signals shown in Fig. 5 might be representative of the current drawn by an incandescent light bulb, a kitchen toaster oven, a microwave oven, a garage door opener, etc., on a typical school morning in suburbia. In our case however, these are just random signals generated for the purpose of illustration and are not actual signals from real devices.

Each of the different appliances has a distinctive fingerprint determined by the physical properties of the respective systems. However, when the signals are mixed together, it becomes difficult to distinguish one from the others, and seemingly impossible to detect any unusual behavior in the system.


If a circuit breaker began to fail in this example house on this example morning, the current drawn by the faulty circuit breaker as it shorted to the metal cabinet would leave a distinct fingerprint in the total current usage record. With the many other signals mixing together, the signal from faulty breaker might not be obvious, but the method of reinforcement differencing can isolate the signal of concern. Once the fingerprint has been isolated, other methods not discussed here can be used to monitor and warn the residents of the danger.



For this example, a priori knowledge of circuit fault occurrences is assumed. Fig. 6 shows the time periods when the "unknown" signal was present (when the arcing occured), and it denotes sampling of the signal used in calculation of the FFTs. Fig. 7 compares the power spectra determined by reinforcement differencing (top frames) with the power spectra computed directly from the unknown signal (middle frames). The bottom frames show the power spectra of the combined background and unknown signals. Fig. 7 also compares results of the reinforcement differencing method using power spectra based on the square magnitude of the FFT, (IFFTI)2, with those based on a square-windowed periodogram.

This example is more complicated than the previous examples because the background signal is the sum of many other non-stationary signals whose starting and ending points often occur within the sample used to calculate the FFT. Also, unlike actual data from chemical spectral analysis or power system load records, the various constituent signals used in the preceding examples are not continuous at their endpoints.


We have presented a method for ascertaining spectral information for an unknown signal which is intermittently present in a composite of many other signals. This method of reinforcement differencing identifies spectral content to within two percent in the examples shown in this paper, but gives only a qualitative indication of spectral power of the unknown signal. Future work will address accuracy in calculating the spectral power of the unknown signal as well as the tendency of this algorithm to amplify high-frequency noise. Results of the application of this method to chemical spectral analysis will also be presented.


The authors would like to thank Dr. Jill Scott for her technical guidance on application and technical content. The authors would also like to thank the reviewers whose constructive comments have greatly strengthened this paper and its scientific value. This work was in part supported through the INL Laboratory Directed Research and Development Program under Contract DE-AC07051D14517 with the U.S. Department of Energy, Dr. J. W. James' contributions to this work were undertaken as a post-doctoral student with the INL under a contract with Washington State University. Finally, we thank Laura Tolle for her careful editing of this note.



[1] John G. Proakis and Dimitris G. Manolakis, Digital Signal Processing Principles, Algorithms, and Applications, Macmillan Publishing Company, 866 Third Avenue, New York, New York 10022, 2nd edition, 1992.

[2] J.Michelle Kotler, NancyW. Hinman, Beizhan Yan, Daphne L. Stoner, and Jill R. Scott, "Glycine identification in natural jarosites using laser desorption fourier transform mass spectrometry: Implications for the search for life on mars," Astrobiology, vol. 8, no. 2, pp. 253-266, April 2008.

[3] C. Laughman, Kwangduk Lee, R. Cox, S. Shaw, S. Leeb, L. Norford, and P. Armstrong, "Power signature analysis," Power and Energy Magazine, IEEE, vol. 1, no. 2, pp. 56-63, Mar-Apr 2003.

Charles R.Tolle (a,1) and John W. James (b)

(a) South Dakota School of Mines and Technology Department of Electical and Computer Engineering Rapid City, SD 57701 USA (2)

(b) Washington State University stationed at the INL Department of Industrial & Material Technologies Idaho Falls, ID 83415-2210 USA (2)

(1) Corresponding author; Phone: 605-394-6133; Email: Charles.Tolle @

(2) This work was in part supported by the Idaho National Laboratory's (INL's) Laboratory Directed Research and Development (LDRD) Program under DOE Idaho Operations Office Contract DE-ACOT-051D14517. Approved for external release, STI Number: INL/JOU-08-13794
COPYRIGHT 2009 Idaho Academy of Science
No portion of this article can be reproduced without the express written permission from the copyright holder.
Copyright 2009 Gale, Cengage Learning. All rights reserved.

Article Details
Printer friendly Cite/link Email Feedback
Title Annotation:Fast Fourier Transform
Author:Tolle, Charles R.; James, John W.
Publication:Journal of the Idaho Academy of Science
Article Type:Report
Geographic Code:1USA
Date:Dec 1, 2009
Previous Article:2008 Distinguished Scientist/Engineer.
Next Article:Distribution of Pasteurella Trehalosi genotypes isolated from Bighorn Sheep in Waterton-Glacier International Peace Park.

Terms of use | Privacy policy | Copyright © 2020 Farlex, Inc. | Feedback | For webmasters