One virus engine not enough! (Security).

GFI White Paper

Whilst organizations may agree the need to protect networks from virus attacks by installing an email security product, how do they choose the appropriate solution from the wide variety of virus scanning engines available? Moreover is one anti-virus engine enough to protect the internal network from mass-mailing viruses, worms and other email-borne threats? The tests outlined in this paper indicates that each virus scanner presents its own strengths and weaknesses, demonstrating that no one anti-virus engine fully protect against all possible threats. However the simultaneous use of more than one engine can achieve greater security than is achievable when relying on only one. The use of multiple virus engines also enables security administrators to be vendor independent when it comes to virus scanning, being able to use the best of breed virus engines available on the market.

Note: The following paper analyses research findings on virus-scanning engines only, and does not examine other features found in anti-virus packages.

A Review of Current Anti-Virus Engine Tests

This paper examines the research currently available on leading anti- virus engines--namely, those developed by Trend Micro, Norton, Bit- Defender, McAfee and Norman--and studies their performance in three key areas:

* Overall detection rates of in the wild and zoo viruses;

* their ability to scan through compressed and embedded files; and

* the coverage of non-virus malware.

The results reported are based on tests conducted by the following anti-virus testing laboratories:

ICSA Labs--ICSA certification is regarded as the guarantee that a certain product is top notch and assures customers that the product has succeeded in a number of stringent tests. West Coast Labs--The West Coast Checkmark has been developed as an independent testing and standards organization. Checkmark--certified products and services can be relied upon to an identified standard.

Virus Bulletin--The Virus Bulletin 100% award is given to those anti- virus products that detect all `in the wild' viruses through both on- demand and on-access scanning during testing.

AV-Test. org--This German organization of the University of Magdeburg consistently tests anti-virus software on behalf of companies and leading IT publications for client, server, UNIX and groupware products.

Virus TestCenter--The University of Hamburg Computer Science Department runs tests on anti-virus products and publishes the results in its Virus TestCenter, with an emphasis on the detection of zoo viruses.

Test Results in Brief Considered together, the various test results show that no single anti-virus engine can fully protect against all possible threats.

For example, Trend Micro does not scan ACE, B2 or TGZ compressed files, and it does not detect viruses compressed with the increasingly popular UPX. However, it excels in the MS Office files area, capturing all OLE objects embedded in such files in the AV-Test.org tests. Trend Micro's products also obtained good (but not full) results with non-virus malware.

While Norton AntiVirus achieves a good rate at detecting both ITW and zoo viruses, it fails to detect viruses compressed vvith packages such as UPX, Shrink, and ASPack. In the tests, it achieves an average detection rate of 75% of backdoors and Trojan files.

McAfee VirusScan yielded different results from different testing organizations regarding detection of ITW viruses on different platforms. According to the AV-Test.org tests of November 2001, VirusScan caught 99.5% of the in the wild (ITW) viruses. This product does not support compression formats RAR or ACE and does not detect viruses compressed with UPX and other similar products. However McAfee achieved good results in the non-virus malware section (ActiveX, backdoors and Trojans).

Norman's main strength seems to be in maintaining a high rate at detecting ITW and zoo viruses. However Norman is less powerful when the viruses are compressed with formats other than ZIP and ARJ, or using any self-extracting (SFX) archiving method such as WinZip.

BitDefender by SOFTWIN supports several compression formats like ACE, ARJ, RAR and ZIP. It also checks through files packed using packages such as UPX, Neolite and ASPack, which are popular among virus-writers. Yet, it missed one ITW file virus and caught 92% of all zoo viruses on test.

The Case for Using Multiple Engines

Given the inability of any individual anti-virus engine to provide full coverage against all email attacks, logic dictates that combining multiple engines will produce a more complete solution. In simple terms, if anti-virus products X and Y--each stronger in one area but weaker in another--are used together, their joint strength is likely to cover a wider range of security areas, and this way they can counteract each other's weak points.

Further analysis shows the validity of this theory. The tables below use data from the AV-Test. org tests of November 2001 to show the impact of using two or three virus scanning engines to increase protection.

Email security product A1 with Norman and
BitDefender engines installed

              100% ITW   Compression     Other Malware

Norman        100%       21.6%           84%
BitDefender   99,8%      56,8%           56.3%
Total         100%       56.8% - 78.4%   84% - 100%

This email product "A1" would cover 100% of ITW viruses, between 56.8% and 78.4% of the most popular compression methods, and 84%-100% of samples from the "other malware" section.

Email security product A2 with McAfee, Norrnan and
BitDefender engines installed

              100% ITW       Compression     Other Malware

McAfee        99.5% - 100%   21.6% - 37.8%   98.7% - 99.6%
Norman        100%           22%             84%
BitDefender   99.8%          57%             56.3%
Total         100%           56.8% - 100%    98.7% - 100%

This product "A2" would cover 100% of ITW viruses, about 56.8% or more of the most popular compression methods, and 98.7%-100% of samples from the "other Malware" section.

Another email security product--"B--uses the Norton virus-scanning engine. The table below shows the total coverage with this product:

         100% ITW   Compression   Other Malware

Norton   100%       40%           83.3%

A fourth email security product, "C" uses Trend's anti-virus engine, with the following results.

        100% ITW   Compression   Other Malware

Trend   100%       51.4%         99.2%

Comparing these four products, we notice that A2 has an advantage over the rest of the products, with A1 next on the performance list.

                   100%   Compression    Other
                   ITW                   Malware

A1 (BD & Norman)   100%   56.8% - 78.4   84% - 100%
A2 (McAfee, BD &   100%   56.8% -        98.7% -
Norman)                   100%           100%
B (Norton)         100%   40%            83.3%
C (Trend)          100%   51.4%          99.2%

The table below gives a closer view of the compression area (where virus scanners tend to differ greatly in performance):

             ACE  ARJ  CAB  LHA  RAR  ZIP  UPX  ASPack  SFX

McAfee       No   No   Yes  Yes  No   Yes  No   No      1/6
Norman       No   Yes  No   No   No   Yes  No   No      0
BitDefender  Yes  Yes  Yes  Yes  Yes  Yes  Yes  Yes     1/6

Here, one sees how BitDefender covers many more of the compression formats than the rest of the virus-scanning engines being used in this analysis. The email security product A2 would therefore provide a much more complete solution than a product making use of a single virus scanner.

Comment:

It is important to run multiple anti-virus engines simultaneously, also whilst anti-virus protection is a critical component in protecting a network from email-related threats, it alone cannot fully safeguard networks from email assaults. The fact that virus scanners only cover a portion of non-virus threats is well known. Therefore a fuller email security product should include features that protect against email-borne security threats apart from viruses, as well as multiple virus scanners.

GFI MailSecurity for Exchange/SMTP provides a solution. In addition to the simultaneous use of multiple virus engines, it provides Email content & attachment checking--to quarantine dangerous emails; Exploit shield--to provide mall intrusion detection and defence; and an Email threats engine--to analyze and defuse HTML scripts, .exe files etc. www.gfi.com/mailsecurity.