Some six years ago, in describing the testing of the Northrop Grumman Electronic Sensors and Systems Sector (Linthicum, MD) AN/ALQ-131 jammer, the GAO Report/NSIAD-1995-47, said: "Developmental testing conducted after the Band 3 entered production has shown that the system has serious performance problems. New, but preliminary, test results compiled after the draft of this report was prepared indicate some improvement in performance; however, significant problems persist." 
The test performance of the AN/ALQ-165 Airborne Self-Protection Jammer (ASPJ), a product of the joint venture team of ITT Industries' Avionics Division (Clifton, NJ) and Northrop Grumman ES , had likewise shown effectiveness testing problems. The Director of Operational Test and Evaluation (DOT&E) FY 1996 Annual Report said: "DOT&E cannot certify that the ASPJ is effective against the original ASPJ requirement." And "Key performance criteria for effectiveness and suitability were not met and the FSD [full-scale development) systems were not considered production representative." The report went on to say, "DOT&E assessed the ASPJ as not operationally effective because it did not meet the requirement threshold value for increasing the survivability, in robust multi-threat mission scenarios, of an ASPJ-equipped F/A-18 strike force over that of a non-ASPJ baseline F/A-18 strike force."
Countermeasures Of Effectiveness
Since these results were reported, a number of initiatives have been undertaken to develop software tools to assist decision makers in gaining insight into the impact of changing EW requirements. One such tool, the EW Measures Of Effectiveness (MOE) Tool,  developed with SAFAQ/AQPE support, is intended to provide access to data for performing EW system deficiency and requirements analyses. The EW MOE Tool uses data provided by sources such as government models, e.g. Enhanced Surface-To-Air Missile Simulation (ESAMS), EW contractor engagement scenarios tied to system specific digital system models, and even to test range data. The tool is described as enabling the EW decision maker to gain valuable insight into the importance of a given EW requirement, and how changing the requirement will impact EW performance and consequently the warfighter's effectiveness.
It may be observed that a number of issues and program initiatives, including Developmental and Operational T&E procedures, and Simulation Based Acquisition, each impinge on the primary concern of gaining consistently testable effectiveness results from jamming equipment in development, test, and operational use.
In addressing this merging of M&S and T&L issues, the Joint Advanced Distributed Simulation (JADS) EW Test Team was mandated to determine if Advanced Distributed Simulation could help the EW community resolve several interrelated testing process issues. The first issue of concern to the JADS EW Test Team was that of the correlation of test results for electronic-countermeasures (ECM) systems that are under test across the modeling and simulation, hardware-in-the-loop (HITL), and open-air test regimes. This concern would appear to be related to validating software simulations in relation to hardware performance as measured in HITL and open-air tests. The absence of correlation may imply, among other factors, that the software model is not valid, that some elements of the HITL simulation are not valid or that input data and test results in all three test domains were not complete or not sufficiently accurate.
The second issue of concern to the JADS EW Test Team was that of model and simulation fidelity. Adequate fidelity implies that the software simulation is a reasonably good match to the hardware that it is simulating. The lack of an adequate match between hardware and software may give rise to uncorrelatable or widely varying test results. This is particularly so if the software simulation does not model key portions of the hardware in sufficient detail. The failure to correlate output results may also result from a mismatch between the parameter values loaded in the simulation. There remains, however, the outstanding issue of identifying the key parameters and subsystems of the hardware that affect overall system performance and ECM response.
Of fundamental importance to the JADS EW Test design was the statistical correlation of EW measures of performance (MOP) across the test phases. JADS found generally poor correlation between the test results taken in the three domains despite their attempts to limit the sources of variation. They observed that the primary variance source for most measures was operator action. JADS found wide variances in the test results even when threat operator actions were constrained to a fixed set of allowable actions.
JADS further observed that their "inability to execute a test that sufficiently removed operator variance from the test casts a shadow on the EW community's ability to make statistical analysis a feasible aid for decision makers faced with determining the worth of systems designed to work against human operators." Measures such as jamming-to-signal ratio and tracking error that produce thousands of samples were reported to make the problem worse. Statistical tests were found to produce no useful comparisons in these many-sample measures.
The success or failure of deception countermeasures in dynamic engagements with threat tracking radars can be traced to one primary factor. This is the relative influence of the jamming signal and the target return signal on the radar's tracking when both signals are simultaneously present within one or more of the radar's tracking discriminators. In simple terms the tracking discriminators (in angle, range, and/or Doppler) are the components in the radar that continuously adjust the position of the radar's tracking cell (normally under servo control). The tracking cell keeps the radar signal positioned over the selected target (which may be a deception signal). This target tracking data is, in turn, used by the threat radar to control or command a missile or gun system.
A fundamental property of discriminators is that they possess very non-linear input-to-output characteristics. Furthermore, when faced with two simultaneous signals, a discriminator's characteristic distorts even further. Tracking discriminators are, in most weapon designs, an integral part of servo controlled tracking loops that are themselves modeled by differential equations. Therefore a simulation that adequately models the interaction between deception countermeasure and threat tracking radar inherently must include a number of non-linear differential equations.
A threat radar, to attain adequate fidelity, should be modeled with nonlinear differential equations. This requirement may have profound consequences on the correlation between software simulation and hardware test results. One basic property of non-linear differential equations is that they exhibit "chaotic behavior." One of the properties of non-linear differential equations, particularly for interactions that occur over a period of time, is that the outcome can be very strongly dependent on very small changes in initial input parameters or conditions.
Chaotic Behavior: A Fundamental issue
Simply put, chaotic systems are potentially unstable and exhibit a very sensitive dependence upon initial conditions. This chaotic behavior of non-linear, time-evolving, dynamic systems was first formally reported by the meteorologist, Edward Lorenz in 1963. While running computer simulations on a set of three nonlinear equations, Lorenz found that, by rounding off initial parameters from four decimal places to two, the time paths of the equations diverged exponentially over time. This finding has given rise to a new mathematical discipline, often referred to as chaos theory. Intensively studied by mathematicians for about 35 years, chaotic behavior has been found in a large variety of complex systems from weather systems to human behavior in organizations.
Because of the non-linearities of their systems, chaotic behavior is an inevitable property of the dynamic interaction of countermeasures and tracking radars, with or without a human operator in the loop. The results of the AN/ALO- 131 and AN/ALO-165 tests and the results found by the JADS EW Test Team are entirely predictable. When viewed from the perspective of the expected chaotic behavior of the nonlinear systems under test, the findings of large variances, non-correlatable results, and effectiveness criteria that are not met, should be expected. There are a number of implications of this observation.
The software simulations must adequately model various key, non-linearities in the radar system, such as those of the tracking discriminators, saturating amplifiers, and antenna-gim-bal limiters. If this is not achieved, their ability to adequately match the performance of the hardware being simulated is suspect at best. It would appear that incorporation of such non-linear models in simulations is a necessary, but not sufficient condition, for adequate fidelity. Even if such non-linearities are included in the simulations, the potential for chaotic behavior implies that increased focus must be placed on the accurate measurement of the input parameters and initial conditions. There must also be a correlation of input conditions before any expectations may be developed regarding correlation of output results from hardware and software tests.
Developmental and operational test and evaluation is carried out to demonstrate that the equipment meets specific performance criteria. When the testing involves the dynamic interaction of countermeasure and radar systems, those criteria should reflect the potential existence of chaotic behavior. In the context of the Simulation Based Acquisition process, in which simulation is applied across the spectrum of acquisition from requirements to testing, the simulations must include appropriate non-linearities at each stage. Only in this manner can the process hope to assist in ameliorating the test problems.
Chaotic and unpredictable behavior is inherent in the countermeasure-radar interaction. Given this fact, it seems reasonable to ask if this implies that there are likely be limits to the percentage of times that specific countermeasure may be effective against specific radar and weapon systems? In the absence of a body of research and engineering focused on this question, it is premature to provide a definitive answer. However, here are several thoughts:
If countermeasures can be structured to limit operation of the radar in non-linear regions, then perhaps results can be linearly extrapolated and outcomes reasonably predictable.
Finally, in the research community, particularly in the non-linear systems discipline, there are now serious efforts to control chaotic behavior of complex non-linear systems through various schemes of adaptive control. Could such adaptive control be incorporated in the mode selection process in jammers to optimize their effectiveness?
T.W. Tucker is the president of Tactical Technologies, Inc. (Ottawa, Ontario, Canada).
(1.) Under the Report heading "Band 3 Ineffective Against Some Threats"
(2.) For more descriptive material, see: www.safaq.hq.af.mil/acq_ref/aqpe/moe/ descript.htm
|Printer friendly Cite/link Email Feedback|
|Title Annotation:||changing electronic warfare requirements|
|Publication:||Journal of Electronic Defense|
|Date:||Jul 1, 2001|
|Previous Article:||Readiness Rests on Army COTS.|
|Next Article:||North Korea Seizes US Navy Vessel, Detains Crew.|