Part 1 | Part 2 | Part 3 | Part 4

Part 3

How Does One Detect a Trojan?

Detecting trojans is less difficult than it initially seems. But strong knowledge of your operating system is needed; also, some knowledge of encryption can help.

If your environment is such that sensitive data resides on your server (which is never a good idea), you will want to take advanced measures. Conversely, if no such information exists on your server, you might feel comfortable employing less stringent methods. The choice breaks down to need, time, and interest. The first two of these elements represent cost. Time always costs money, and that cost will rise depending on how long it has been since your operating system was installed. This is so because in that length of time, many applications that complicate the reconciliation process have probably been installed. For example, consider updates and upgrades. Sometimes, libraries (or DLL files) are altered or overwritten with newer versions. If you were using a file-integrity checker, these files would be identified as changed. If you were not the person who performed the upgrade or update, and the program is sufficiently obscure, you might end up chasing a phantom trojan. These situations are rare, true, but they do occur.

Most forms of protection against (and prevention of) trojans are based on a technique sometimes referred to as object reconciliation . Although the term might sound intimidating, it isn't. It is a fancy way of asking "Are things still just the way I left them?" Here is how it works: Objects are either files or directories. Reconciliation is the process of comparing those objects against themselves at some earlier (or later) date. For example, take a backup tape and compare the file PS as it existed in November 1995 to the PS that now resides on your drive. If the two differ, and no change has been made to the operating system, something is amiss. This technique is invariably applied to system files that are installed as part of the basic operating system.

Object reconciliation can be easy understood if you recognize that for each time a file is altered in some way, that file's values change. For example, one way to clock the change in a file is by examining the date it was last modified. Each time the file is opened, altered, and saved, a new last-modified date emerges. However, this date can be easily manipulated. Consider manipulating this time on the PC platform. How difficult is it? Change the global time setting, apply the desired edits, and archive the file. The time is now changed. For this reason, time is the least reliable way to reconcile an object (at least, relying on the simple date-last-modified time is unreliable). Also, the last date of modification reveals nothing if the file was unaltered (for example, if it was only copied or mailed).

Another way to check the integrity of a file is by examining its size. However, this method is extremely unreliable because of how easily this value can be manipulated. When editing plain text files, it is simple to start out with a size of, say, 1,024KB and end up with that same size. It takes cutting a bit here and adding a bit there. But the situation changes radically when you want to alter a binary file. Binary files usually involve the inclusion of special function libraries and other modules without which the program will not work. Thus, to alter a binary file (and still have the program function) is a more complicated process. The programmer must preserve all the indispensable parts of the program and still find room for his or her own code. Therefore, size is probably a slightly more reliable index than time. Briefly, before I continue, let me explain the process by which a file becomes trojaned.

The most common scenario is when a semi-trusted ( known ) file is the object of the attack. That is, the file is native to your operating system distribution; it comes from the vendor (such as the file csh in UNIX or command.com in DOS). These files are written to your drive on the first install, and they have a date and time on them. They also are of a specified size. If the times, dates, or sizes of these files differ from their original values, this raises immediate suspicion.

Evil programmers know this. Their job, therefore, is to carefully examine the source code for the file (usually obtained elsewhere) for items that can be excluded (for example, they may single out commented text or some other, not-so-essential element of the file). The unauthorized code is written into the source, and the file is recompiled. The cracker then examines the size of the file. Perhaps it is too large or too small. The process then begins again, until the attacker has compiled a file that is as close to the original size as possible. This is a time-consuming process. If the binary is a fairly large one, it could take several days.

When the file has been altered, it is placed where others can obtain it. In the case of operating-system distributions, this is generally a central site for download (such as sunsite.unc.edu , which houses one of the largest collection of UNIX software on the planet). From there, the file finds its way into workstations across the void.

For reasons that must now seem obvious, the size of the file is also a poor index by which to measure its alteration. So, to recount: Date, date of last access, time, and size are all indexes without real meaning. None of these alone is suitable for determining the integrity of a file. In each, there is some flaw--usually inherent to the platform--that makes these values easy to alter. Thus, generating a massive database of all files and their respective values (time, size, date, or alteration) has only very limited value: ...a checklist is one form of this database for a UNIX system. The file content themselves are not usually saved as this would require too much disk space. Instead, a checklist would contain a set of values generated from the original file--usually including the length, time of last modification, and owner. The checklist is periodically regenerated and compared against the save copies, with discrepancies noted. However...changes may be made to the contents of UNIX files without any of these values changing from the stored values; in particular, a user gaining access to the root account may modify the raw disk to alter the saved data without it showing in the checklist.

There are other indexes, such as checksums, that one can check; these are far better indexes, but also not entirely reliable. In the checksum system, the data elements of a file are added together and run through an algorithm. The resulting number is a checksum , a type of signature for that file (bar-code readers sometimes use checksums in their scan process). On the SunOS platform, one can review the checksum of a particular file using the utility sum. sum calculates (and prints to STDOUT or other specified mediums) the checksums of files provided on the argument line.

Although checksums are more reliable than time, date, or last date of modification, these too can be tampered with. Most system administrators suggest that if you rely on a checksum system, your checksum list should be kept on a separate server or even a separate medium, accessible only by root and other trusted users. In any event, checksums work nicely for checking the integrity of a file transferred, for example, from point A to point B, but that is the extent of it.

MD5

You're probably wondering whether any technique is sufficient. I am happy to report that there is such a technique. It involves calculating the digital fingerprint , or signature, for each file. This is done utilizing various algorithms. A family of algorithms, called the MD series , is used for this purpose. One of the most popular implementations is a system called MD5 .

MD5 is a utility that can generate a digital signature of a file. MD5 belongs to a family of one-way hash functions called message digest algorithms . The MD5 system is defined in RFC 1321. Concisely stated: The algorithm takes as input a message of arbitrary length and produces as output a 128-bit "fingerprint" or "message digest" of the input. It is conjectured that it is computationally infeasible to produce two messages having the same message digest, or to produce any message having a given prespecified target message digest. The MD5 algorithm is intended for digital signature applications, where a large file must be "compressed" in a secure manner before being encrypted with a private (secret) key under a public-key cryptosystem such as RSA.

When one runs a file through an MD5 implementation, the signature emerges as a 32-character value. It looks like this:

2d50b2bffb537cc4e637dd1f07a187f4

Many sites that distribute security fixes for the UNIX operating system employ this technique. Thus, as you browse their directories, you can examine the original digital signature of each file. If, upon downloading that file, you find that the signature is different, there is a 99.9% chance that something is terribly amiss.

MD5 performs a one-way hash function. You may be familiar with these operations from other forms of encryption, including those used to encrypt password files.

Some very extreme security programs use MD4 and MD5 algorithms. One such program is S/Key, which is a registered trademark of Bell Laboratories. S/Key implements a one-time password scheme. One-time passwords are nearly unbreakable. S/Key is used primarily for remote logins and to offer advanced security along those channels of communication (as opposed to using little or no security by initiating a normal, garden-variety Telnet or Rlogin session). The process works as described in "S/Key Overview" (author unknown): S/Key uses either MD4 or MD5 (one-way hashing algorithms developed by Ron Rivest) to implement a one-time password scheme. In this system, passwords are sent cleartext over the network; however, after a password has been used, it is no longer useful to the eavesdropper. The biggest advantage of S/Key is that it protects against eavesdroppers without modification of client software and only marginal inconvenience to the users.

With or without MD5, object reconciliation is a complex process. True, on a single workstation with limited resources, one could technically reconcile each file and directory by hand (I would not recommend this if you want to preserve your sanity). However, in larger networked environments, this is simply impossible. So, various utilities have been designed to cope with this problem. The most celebrated of these is a product aptly named TripWire .

TripWire

TripWire (written in 1992) is a comprehensive system-integrity tool. It is written in classic Kernhigan and Ritchie C (you will remember from Chapter 7, "Birth of a Network: The Internet," that I discussed the portability advantages of C; it was this portability that influenced the choice of language for the authors of TripWire).

TripWire is well designed, easily understood, and implemented with minimal difficulty. The system reads your environment from a configuration file. That file contains all filemasks (the types of files that you want to monitor). This system can be quite incisive. For example, you can specify what changes can be made to files of a given class without TripWire reporting the change (or, for more wholesale monitoring of the system, you can simply flag a directory as the target of the monitoring process). The original values (digital signatures) for these files are kept within a database file. That database file (simple ASCII) is accessed whenever a signature needs to be calculated. Hash functions included in the distribution are

• MD5
• MD4
• CRC32
• MD2
• Snefru (Xerox secure hash function)
• SHA (The NIST secure hash algorithm)

It is reported that by default, MD5 and the Xerox secure hash function are both used to generate values for all files. However, TripWire documentation suggests that all of these functions can be applied to any, a portion of, or all files.

Altogether, TripWire is a very well-crafted package with many options.

TripWire is a magnificent tool, but there are some security issues. One such issue relates to the database of values that is generated and maintained. Essentially, it breaks down to the same issue discussed earlier: Databases can be altered by a cracker. Therefore, it is recommended that some measure be undertaken to secure that database. From the beginning, the tool's authors were well aware of this:

The database used by the integrity checker should be protected from unauthorized modifications; an intruder who can change the database can subvert the entire integrity checking scheme.

One method of protecting the database is extremely sound: Store the database on read-only media. This virtually eliminates any possibility of tampering. In fact, this technique is becoming a strong trend in security. In Chapter 21, "Plan 9 from Bell Labs," you will learn that the folks at Bell Labs now run their logs to one-time write or read-only media. Moreover, in a recent security consult, I was surprised to find that the clients (who were only just learning about security) were very keen on read-only media for their Web-based databases. These databases were quite sensitive and the information, if changed, could be potentially threatening to the security of other systems.

Kim and Spafford (authors of TripWire) also suggest that the database be protected in this manner, though they concede that this could present some practical, procedural problems. Much depends upon how often the database will be updated, how large it is, and so forth. Certainly, if you are implementing TripWire on a wide scale (and in its maximum application), the maintenance of a read-only database could be formidable. Again, this breaks down to the level of risk and the need for increased or perhaps optimum security.

TAMU

The TAMU suite (from Texas A&M University, of course) is a collection of tools that will greatly enhance the security of a UNIX box. These tools were created in response to a very real problem. As explained in the summary that accompanies the distribution:

Texas A&M University UNIX computers recently came under extensive attack from a coordinated group of Internet crackers. This paper presents an overview of the problem and our responses, which included the development of policies, procedures, and sdoels to protect university computers. The tools developed include `drawbridge', an advanced Internet filter bridge, `tiger scripts', extremely powerful but easy to use programs for securing individual hosts, and `xvefc', (XView Etherfind Client), a powerful distributed network monitor.

Contained within the TAMU distribution is a package of tiger scripts , which form the basis of the distribution's digital signature authentication. As the above-mentioned summary explains:

The checking performed covers a wide range of items, including items identified in CERT announcements, and items observed in the recent intrusions. The scripts use Xerox's cryptographic checksum programs to check for both modified system binaries (possible trap doors/trojans), as well as for the presence of required security related patches.

The TAMU distribution is comprehensive and can be used to solve several security problems, over and above searching for trojans. It includes a network monitor and packet filter.

NEXT