The dangers lurking in military software production.

The Dangers Lurking in Military Software Production

Viruses, Trojan Horses and Logic Bombs

Virtually all modern weapons systems incorporate semi-automatic computerized sub-systems or are computer-run in a fully automatic mode in order to achieve the shortest possible reaction time. For example, the nav/attack systems of contemporary aircraft rely for many tasks on computer inputs, and combat aircraft of the latest generation can be flown only with the assistance of computers. The General Dynamics F-16C employs roughly 300 processors, while the original F-16A design of 10 years ago had only 50 processors. Actually, the number of processors is irrelevant because without the appropriate software they remain inert slices of silicon. For efficient operation the 300 processors of the F-16C require more than 230 000 lines of computer code which have to be compiled with meticulous care by a large team of programmers and engineers. Larger combat systems require considerably more lines. The Rockwell B-1B bomber's computers come to life with about 1.5 million lines of code (incidentally three times the number required by the earlier B-1A). The combined software requirements of a cruiser for operating its weapons, navigation, surveillance and communications systems are approximately of the same order. For the SDI system it is estimated that at least 10 million lines are needed. It becomes obvious that the reliability and efficiency of any computerized weapon system rests on the expertise and meticulously performed work of the software engineers, and last but not least, on the thorough testing of the finished software product before delivery. The user of the weapon system, be he a pilot or a ship's captain, is forced to place his blind confidence in the software's integrity, the integrity of which he cannot fully test and verify. The crucial test will arrive in combat when the fate of the aircraft or ship will hang on the faultlessly operating software of its computerized systems.

Bugs

Faults in the software, referred to as bugs, have caused and still cause major damage to computer systems. Apart from incorrect inputs by human operators, bugs in the operating or application software have been at fault for gross miscalculations, scrambling of stored memories, or total collapse of complete networks. Bugs are quite often nothing more than design mistakes or oversights - as trivial as a misplaced comma. Every computer user knows that a minor punctuation mistake in a printing command sequence may have drastic consequences. In essence, such mistakes usually do not get corrected until it is too late. On the military level few facts have become known about damage caused by bugs, possibly because their mundane nature was embarrassing or because the consequences were too serious. For instance, an unconfirmed rumour has it that the loss of the "Sheffield" during the Falklands conflict was due to a bug in the radar extraction software, which prevented the detection of the approaching Exocet missile. The crash of the Swedish JAS-39 Gripen prototype was clearly traced to a software fault. It is quite likely that other fly-by-wire operated military aircraft have been lost when a bug in the software caused the computer to issue wrong commands. Two USAF F-117 stealth fighters crashed under almost identical circumstances diving at full power straight into the ground. The cause might have been a bug in the computerized flight stabilisation system which is needed to fly the aerodynamically highly unstable aircraft. Paradoxically, the F-117, as all other fly-by-wire aircraft, features up to four identical redundant computer networks operating on different channels. However, these networks run on the same software - which if bugs are present will generate identical failures.

Although the consequences of bugged software may be grim, at least they are not intentional. This is not so with purposely sabotaged software. This can be done in a number of ways, of which the best known are "Trojan Horses", "Logic Bombs" and "Reproducing Viruses". The virus method is the basis of all other approaches.

Virus programs have since been entered by cleverly written software into numerous computer systems including supposedly secure ones such as US Government and NASA networks. Industry has been hit repeatedly by virus infections.

Viruses

A virus is essentially a malicious bug placed on purpose in a software system in the guise of seemingly harmless code lines, a method called "Trojan Horse". This can then degrade the software's performance or even destroy it. If the Trojan Horse and a Reproducing Virus are coupled with a date at which they go into action or if they are set to respond to an external cue, then they become "Logic Bombs". A reproducing virus is capable of proliferating rapidly in other computer network programs and systems infesting whatever software it comes in contact with. Such viruses have one property in common with the Trojan Horses, which are usually even better camouflaged. They are very difficult to discover, and scientists agree that in more complex software systems detection is virtually impossible until it is too late to prevent an infection. Artificial intelligence and different software design procedures may eventually offer a solution.

For the military the danger posed by virus infections is always present and difficult to control. A software engineer or programmer working for a hostile nation may be instructed to enter carefully designed virus programs into military software. Hidden among the 230 000 code lines needed by the computers of the F-16, for example, which also handle the navigational system, the virus may command that to fail as soon as a certain geographical longitude or latitude is entered. Or the artificial stabilisation system may collapse with the introduction of a reproducing virus, inevitably causing the loss of the aircraft. Faults may also be written in "intermittent-sleeper" format. They would enter into force at specific dates and times, but after a random time-span would return to sleeper status, thereby making discovery of the fault virtually impossible. A missile guidance system may contain a virus commanding it to guide the weapon towards non-existing targets or an artillery fire-control system to give commands to the guns to aim short of the intended target. The faults will be blamed initially on the hardware, and it will take considerable time before the existence of a virus in the software is suspected. Even if suspicion turns certainty, the most intensive simulation and testing will not discover the virus because the commands needed or issued by a computer system do not follow logical paths in the processors. It has been calculated that in a standard navigation and guidance system up to 10 [sup.18] possible paths must be tried, which at one test per microsecond would take almost 320 000 years to complete!

To guard against the possibility of high-tech sabotage virtually all ministries of defense have issued guidelines and directives on how to make critical software trustworthy. How this is done varies from country to country, but the British approach seems to be the most complete so far. It is due to go into effect in the 1990s. Called DefStan 0055, it will apply to all software written for the British forces. The proposed standard calls for the appointment of software safety assessors collaborating with each project team. It is envisioned that a completely independent organisation will be formed eventually, employing highly skilled specialists. If this should not be feasible, then the safety assessors would have to come from a rival manufacturer (which is liable to create commercial confidentiality problems). The proposed standard also includes the use of specific testing methods and software tools, a large measure of automation, and last but not least extensive training of all those involved. Courses and degrees at universities on software safety are even being considered.

Proliferation 

Before examining possible virus countermeasures some particulars of its nature have to be explained. A virus is a program which need not necessarily but can infect other software, or which can carry its message into other programs of a predetermined nature. A virus does not require to be inserted at the design stage of a program; it can be added at any opportune occasion, e.g. during software maintenance or updating. In its simplest form the virus consists of two sub-routines and one main program. The latter calls up the sub-routines in the requisite order and acts as coordinator during the infection process. The first sub-routine contains the order to infect either all files or a number of selected executable program files which carry specific designations. A reproducing virus also checks first whether the file has already been infected. It recognises this with the help of a short code which is left as a marker during a previous infection. The use of such a code is essential because a multiple infection would inevitably lead to early discovery of the virus, the reason being that, however compactly written, the virus program would extend the run-time of the program.

The second sub-routine contains the damage command whose activation may be datelined and/or respond to other cues, also called "triggers", which are unwittingly entered by the computer's operator. In automated military systems these may also be generated by sensor inputs, firing commands, geographical locations or any other input which passes through the computer for processing. The damage may range from a message on the monitor saying "Peace on Earth", which is obviously disturbing enough during a combat situation, to faulty information, garbled displays and commands, or in the worst case to the elimination of all stored files. If the virus program has been written into the operating system software, it will multiply rapidly in the application software and automatically enter all stored files run on the computer, unless the virus' radius of action has been limited by its designer to infect specific file groups only. The latter is an additional obstacle to discovering the virus. If all storage files and application software are infected, discovery is more likely than on a small number of rarely used files, such as actual firing sequences for missiles and the like. A cueable or datelined virus entered via the operating software might lie dormant in an application software niche until this particular routine is called up and/or the dateline has been reached. Then it will begin its destructive activity. If the processor is working at maximum capacity a reproducing virus would then infect the computer and its software within minutes. The result would be the elimination of all files running in the system and their replacement by meaningless symbols. After it has done its damage and run out of files it can self-destruct, eliminating any trace of its existence. Subsequent investigation will reveal no clue about the origin of the malfunction.

An additional danger posed by manipulated software is that the virus will spread to other networks with which the infected net exchanges information, and which presumably uses the same software. In military service, where usually a common software is used, such as ADA in NATO, it may be hidden in the software used by a maintenance computer for testing a combat aircraft's nav/attack systems. The infected software, while aligning its operation with that of the software of the aircraft, will discover that the infection code lines are missing and, according to the commands written into the virus program, copy them at once!

The problem of detecting virus additions in a software package is, to use a mathematical term, "undecidable": i.e. there is absolutely no algorithm (or program) possible which is capable of establishing whether any random program has been infected by a virus or not. Though this does not seem plausible it is a grim mathematical reality on which potential saboteurs can rely. The fact that a detecting algorithm cannot be developed does not mean, however, that after the identification of its code a virus cannot be eliminated. Once the code is known, the computer can be ordered to search all files for it and delete it. But discovering the code amounts to finding a needle in the proverbial haystack. The virus can be designed to change its format with each reproduction. A very simple virus can be transformed into an "intelligent" one by adding a random number generator requiring only one additional line of code. In this case the most diligent search for a virus will prove fruitless, even if the original code has been found. Any attempt to discover sleeper viruses by searching for their possible function is also hopeless as no one can know what this function might be. A number of other approaches to virus detection have been suggested but they have proved to be either "undecidable" or impractical.

Remedies

There are only two - granted tenuous - methods of detecting or preventing a virus infection. The first is to make absolutely sure of software integrity at each step of its production and to carry out a thorough test before it leaves the design offices. The second method is the use of a non-algorithmic approach. This possibility is offered by artificial intelligence, i. e. expert systems which solve problems by mimicking human logical reasoning without resorting to algorithms and which solve difficult problems by applying facts and rules drawn from a knowledge base stored in the computer system. The artificial intelligence software base must be designed to contain all available knowledge about computer viruses, their propagation and action modes. Once a solid base has been established, additions of new facts and discoveries should become routine.

The first method is without any doubt preferable but may be difficult for the simple lack of capable programmers. The demand for software is said to be growing at the rate of 25% per annum, but the number of programmers is only increasing about 4% annually. Industry thus has to employ programmers who may not meet the high standards of individual integrity and reliability required. This in turn increases the danger of infiltration of the software design institutions and firms by potential agents and saboteurs.

This well-known problem has led to different approaches in software design. It can safely be said that, in spite of the application of computer-aided software engineering (CASE) techniques, existing management of software production is primitive compared with other aspects of modern computer technology. It is like getting a large number of knitters to work on a different part of a vast pullover without knowing what the others are doing. The resulting distortions may mean unravelling the whole garment to try and trace the fault and still not finding the one purl or plain that went wrong.

It is reckoned that correcting software system mistakes is anywhere between 50 and 100 times more expensive than doing so when most of the errors are made, namely during the design phase. To carry the above analogy further, what is needed is a stitch-by-stitch quality control performed at the work station of each knitting expert.

It is obvious that the most important requirements for achieving a high quality software are excellent communications among the programmers and engineers during its design phase and full documentation of every step in the design process. This has led to the development of PDLs (Page Description Languages) which assist in creating and documenting complex software designs and provide communication channels between all the participants. The PDL method combined with CASE is now widely used and has helped enormously to improve productivity and final quality control. This work with smaller projects but is not adequate for software with a million lines of code or more. In the early '80s a new type of PDL became available: this uses an interactive, on-line approach capable of monitoring via computer channels the activity of all programmers working on even the largest project.

Apart from greatly facilitating editing, it also offers continuous multi-user design analysis, which means that even with hundreds of people working on the software a perfect linkage between all its elements is achievable since the combined product is automatically and consistently tested for omissions, wrong syntax or mistakes - and last but not least, for undesirable tampering.

However, even this almost tamper-proof system has flaws which can be utilized by the clever saboteur. This has led to the design of chips which are resistant to accepting foreign or false information. This means that the standards hitherto applied for software design and writing have to be modified. It is also possible to write the software right into the mechanical design of the chip. This technology is well in hand and is found in every digital device which cannot be programmed, i.e. a read-only memory (ROM). Such chips might have to be designed for the most critical subsystems of weapon systems.

Another option is to use optical disks in ROM format for all critical software. If this disk has been checked out thoroughly and the software has been produced under the above-mentioned conditions, any later addition of a virus during software maintenance or modification is impossible. Optical disks are 100% tamperproof but they lack the flexibility of RAM (Random Access Memory)-oriented software. Thus a part of a weapon system's software has to be in RAM format, and even if this is infected, the virus will not be able spread to the ROM sections. The military are not enchanted with this solution because an optical disk and its ROM files are hard to destroy to prevent the information from falling into enemy hands. With magnetically stored software this poses no problem.

Several other approaches are no doubt being tested and may have been introduced to produce tamperproof and bug-free software. Most counter-measures are naturally Top Secret. An absolutely secure software may never be possible as the saboteur will always try to be a step ahead of the security engineer. The whole problem has thus developed into a permanent battle of wits for the highest stakes, fought in the abstract environment of mathematics.

PHOTO : The B-1B is heavily "computerized" and thus depends on software for functions like

PHOTO : nav/attack, aerodynamics, engine, etc.

PHOTO : CIMSA-SINTRA produces this powerful and fast 15M-125X naval weapon system computer and

PHOTO : part of its software.

PHOTO : Modern missiles like the Trident use densely packed computers for which totally bug-free

PHOTO : software is a prerequisite.

PHOTO : The General Dynamics F-16, seen here in a high angle of attack manoeuvre, requires 230 000

PHOTO : lines of code to run the 300 processors of its electronic systems.

PHOTO : Siemens offers computer-based graphic control tools operating interactively with

PHOTO : light-pens or mice. They vastly improve productivity in the design of software

PHOTO : Computing Devices produces this Milipac artillery fire-control computer. Where there is

PHOTO : little or no second-chance option a zero-fault software is a sine qua non.

PHOTO : This central avionics computer built by Texas Instruments features high-density packaging

PHOTO : and ease of maintenance thanks to the use of line-replaceable units.

PHOTO : Hughes, under a Krupp-Atlas Elektronik contract, has built for the German Forces a number

PHOTO : of Remus mobile test stations for checking software of fielded systems.