The Sleuth Kit Informer http://www.sleuthkit.org/informer http://sleuthkit.sourceforge.net/informer Brian Carrier Issue #2 March 15, 2003 CONTENTS -------------------------------------------------------------------------- - Introduction - What's New? - Autopsy 1.70 Case Management - Splitting The Disk - Part 1 INTRODUCTION -------------------------------------------------------------------------- The second issue of the Sleuth Kit Informer covers the details of case management in the 1.70 version of Autopsy. The case management has been improved to support investigations involving multiple hosts from multiple time zones. The article covers all details of the directories, configuration files, and logging. The second article in this issue examines how to break a full disk image into partitions. This will be the first in a series that examines how to extract data from full disk images. This article covers DOS/Intel partitions that are common with Windows and Linux systems. As always, feedback and suggestions are encouraged (carrier@users.sourceforge.net). WHAT'S NEW? -------------------------------------------------------------------------- TASK was recently mentioned in two articles: - CSO: Tools of Evidence by Simson Garfinkel http://www.csoonline.com/read/030103/machine.html - Security Focus: Forensics on the Windows Platform, Part Two by Jamie Morris http://www.securityfocus.com/infocus/1665 SANS expanded its forensic track to six-days, with one day for The Sleuth Kit, TCT, and Autopsy. The six-day version was presented for the first time last week in San Diego: http://www.sans.org/SANS2003/track8.php AUTOPSY 1.70 CASE MANAGEMENT -------------------------------------------------------------------------- Previous Case Management Prior to version 1.70, there was little notion of Case Management in Autopsy. Autopsy used a 'morgue' directory, which contained all file system images and output files that Autopsy created over the course of an investigation. This design had several limitations. First, all configuration was done by hand and a text editor. Second, the use of one directory for multiple purposes made if difficult to impose strict file access permissions. Lastly, the location and investigator were specified at compile time, which made it difficult to manage multiple cases at once. Current Case Management Overview The Case Management in Autopsy 1.70 was designed to handle large and global investigations with multiple systems in multiple time zones. All management is done using a graphical interface and directories have been organized so that strict permissions can be applied. There are also several types of logging and auditing being performed. The management organization uses directories and ASCII text files. It was designed to make it easy to archive cases and provide consistency. It uses an open and generic design so that any tool can take advantage of the suspect data. When Autopsy is installed, an Evidence Locker is identified. This directory will contain the files that Autopsy reads and writes and therefore should be in a partition with lots of disk space. The location can also be specified at run time with the '-d' flag. Each investigation starts with an case. The case includes information such as its name, a short description and a list of investigators that will be involved with the analysis. To handle a multiple system incident, each case can contain one or more hosts. Each host corresponds to a system whose data is being used in the analysis and each can have its own time zone and clock skew. The clock skew value is used when a host did not use NTP and the system clock was fast or slow. Autopsy will adjust the times accordingly so that it is easy to correlate events. Each host can also have a 'known good' and a 'known bad' hash database, which makes it easy to use databases that are specific to a given host or operating system. Each host can have multiple images. Each image corresponds to a file system on the suspect system. A Windows system with only a 'C:\' drive will have one image, whereas a Solaris system with five slices will have five images. Each image has a mounting point that is used for cosmetic purposes only. Evidence Locker Details The Evidence Locker contains the investigation details. It is similar to the morgue directory in previous versions, but it has more structure. All cases have a subdirectory in the Evidence Locker and the Autopsy log shows when Autopsy was started. Users that can create cases will need write permissions for the Evidence Locker. All other users only need read and execute permissions. All users will need write permissions for the Autopsy log, 'autopsy.log', but do not necessarily need read permissions for it. Case Details Each case is given a directory in the Evidence Locker. The name of the directory corresponds with the name of the case. To rename a case, simply rename the directory. # mv case-old case-new The 'case.aut' file is the case configuration file. It contains the date that the case was created on and directory names that are reserved for future use. This file must exist for all cases. The 'case.log' file is the audit log for the case and its contents are discussed in the Logging Details section. The 'investigators.txt' file contains a list of the investigators that are working on this case. These names are used for logging only and not authentication. To restrict access, Unix groups should be used. After all, if the permissions on the analysis stations allows the user to write to the file system images, the user can always bypass any authentication that Autopsy requires. The 'investigators.txt' file contains an investigators name on each line. If no investigators were specified when the case was created, then this file will not exist. To add or remove users, simply edit this file by hand. To delete the case, simply delete the directory (using 'rm -rf CASE'). To archive the case, the 'tar' command can be used (where CASE is replaced by the actual case name): # tar cf CASE.tar CASE # gzip CASE.tar Users that need to add hosts to the case will need write permissions for the case directory. All other users just need read and execute permissions. All users need write permissions for the case log, but do not need read permissions. All users will need read permissions for the configuration file, but do not need write permissions. Host Details Each host is given a directory in the case directory. The host name is the same as the directory name. Therefore, to rename the host after it has been created, simply rename the directory. # mv host-old host-new The host directory contains five directories: images: Where all file system images are located logs: Where the host log, investigator log, and investigator notes are stored. Its contents are discussed in the Logging Details section. mnt: Not currently used by Autopsy, but can be used to manually mount images in loopback in Linux. Future versions of Autopsy will mount the images here. output: Where all Autopsy output files are saved. This includes unallocated space files, strings files, timeline body files, timelines, and temporary files. reports: Not currently used by Autopsy, but will contain reports that are created in future versions of Autopsy. The 'images' and 'output' directories can each have an 'md5.txt' file. This file contains MD5 values for the files in the directory so that the image integrity can be verified during the investigation. The host configuration file, 'host.aut', is located in the host directory. It contains entries for the files used by the host and the time zone information. The first column of each line specifies the configuration type. The following are valid configuration types: Type: desc Description: Description of the host Arguments: The description string Example: desc The DNS server from the New York data center Type: image Description: File system images Arguments: Path to file system image, file system type, and mounting point Example: image images/part1.dd ntfs C: Type: dls Description: Unallocated data from file system images Arguments: Path to 'dls' image and path to original file system image Example: dls output/part1.dls images/part1.dd Type: strings Description: Strings file of file system or dls images Arguments: Path to strings file and path to original image Example: strings output/part1.str images/part1.dd Type: body Description: Body files used when making timelines Arguments: Path to body file Example: body output/body Type: timeline Description: Timeline files of activity Arguments: Path to timeline file Example: timeline output/timeline.march15 Type: timezone Description: The time zone that the suspect system is from Arguments: Time zone variable (legal TZ values) Example: timezone EST5EDT Type: timeskew Description: The clock skew of the suspect system in seconds Arguments: Positive or negative integer Example: timeskew -24 Type: alert_db Description: The 'known bad' hash database Arguments: Path to database Example: alert_db /usr/local/forensics/database/linux-rootkits.md5 Type: exclude_db Description: The 'known good' hash database Arguments: Path to database Example: alert_db /usr/local/forensics/database/redhat-8.0.md5 After the images have been added to the host, users do not need write permissions to the 'images' directory. All users will likely need write permissions to the 'output' directory though. All users need read, write, and execute permissions for the 'logs' directory and each user needs write and read permissions for their specific log and notes file. All users will need write permissions to the general host log. All users will need read and write permissions to the host configuration file. Image Details Each image must be located in the 'images' directory of the host directory. Either the actual image must be located there or a symbolic link to the image. As was previously seen, the details of the image, such as mounting point and file system type are saved in the host configuration file, 'host.aut'. When the image file is analyzed, all output data is saved in the 'output' directory. Therefore, the 'images' directory can have read-only permissions after all images are added to it. Logging Details The Evidence Locker directory contains a log, 'autopsy.log', with entries for each time Autopsy is started and stopped. This log identifies when Autopsy was started, on what port, and to which host. Case logs are stored in the case directory with the name 'case.log'. The log contains an entry for when the case is created, when it is opened and when hosts in the case are opened. Other actions are saved in the host log files. There are two types of host log files. Both are saved in the 'logs' directory of the host. The 'host.log' file is the generic host log file and has entries for when the host is opened and for when images are opened. The 'logs' directory also contains a log file for each investigator. The log file is named with the investigator name and a '.log' extension. It contains entries for all actions done by the specific user, such as which directories are listed and what files are viewed. Any notes that are created by the investigator are saved in a text file in the 'logs' directory. The file is named with the investigator name and a '.notes' extension. The following table shows which logs record a given high-level action. Autopsy starting Evidence Locker Log Case creation Case Log Case opening Case Log Host creation Case Log, Host Log Host opening Case Log, Host Log, Investigator Log Image creation Host Log Image opening Host Log, Investigator Log Directory listing Investigator Log File viewing Investigator Log Meta Data viewing Investigator Log Data Unit viewing Investigator Log Keyword search Investigator Log File Type sorting Investigator Log Timeline creation Investigator Log Notes creation Investigator Log, Investigator Notes Conclusion Autopsy 1.70 has case management integrated into its design. The case management has been designed to make it easy for investigators to use other tools and view the configuration using standard text editing tools. Future versions of Autopsy will integrate archival and modification abilities from the graphical interface. References http://autopsy.sourceforge.net SPLITTING THE DISK - PART 1 -------------------------------------------------------------------------- Introduction The Sleuth Kit currently requires a raw partition image as input. Support for a raw disk image has been on the todo list for a while, but has not been implemented. As the full disk is typically imaged during acquisition for simplicity and to ensure that all data is copied, many investigators are faced with the problem of breaking the full disk into partitions. This section will be the first in a series that uses Linux to extract partitions from a full disk. This issue will cover the extraction of DOS/Intel partitions and future issues will cover the extraction of slices from BSD disk labels (FreeBSD and OpenBSD) and Solaris Virtual Table of Contents. Partitions DOS/Intel partitions are used in Windows and Linux systems and are described in a partition table. The master table is located in the first sector of the disk and it has four entries. The first sector of the disk is typically called the Master Boot Record. There are three major-types of partitions: primary, secondary, and extended. Primary and secondary partitions contain file systems and are the ones that we will extract. Extended partitions are used to overcome the limitations of only having four entries in the master partition table. An extended partition can be imagined as a virtual disk because it has a secondary partition table and its own partitions. Inside the boudaries of an extended partition can be a secondary partition (which contains a file system) and another extended partition. Extended partitions typically consume the remainder of the disk when the partition table is full and its partition table is then used to describe additional partitions. For example, we have a 3 GB disk drive and want to make six 500 MB partitions. The first three entries in the master partition table will each describe a 500 MB primary partition. The fourth master partition table entry will be for an extended partition that consumes the remaining 1.5 GB of the disk. Inside the 1.5GB extended partition will be a partition table with an entry for a 500 MB secondary partition and an entry for a 1 GB extended partition. The 1GB extended partition will contain a partition table with an entry for a 500 MB secondary partition and an entry for a 500 MB extended partition. The 500 MB extended partition will have a partition table that contains an entry for the final secondary partition. Extracting Partitions The 'fdisk' command will list all of the partitions on a disk (including some extended ones). The '-l' flag forces it to list the details and the '-u' flag forces it to output in sectors. 'fdisk' will typically default to these flags when run on a disk image created by 'dd', but it doesn't hurt to specify them. # fdisk -lu disk1.dd Disk disk1.dd: 0 heads, 0 sectors, 0 cylinders Units = sectors of 1 * 512 bytes Device Boot Start End Blocks Id System disk1.dd1 * 63 16064999 8032468+ 83 Linux disk1.dd2 16065000 32129999 8032500 7 NTFS disk1.dd3 32130000 40162499 4016250 83 Linux disk1.dd4 40162500 58621184 9229342+ 5 Extended disk1.dd5 40162563 48194999 4016218+ 83 Linux disk1.dd6 48195063 58621184 5213061 83 Linux We see that this example has five file systems and an extended partition. You may notice though that there should be a second extended partition that contains the last partition. It is there, but 'fdisk' does not report it. Notice that there are 63 unused sectors between the end of disk1.dd5 and start of disk1.dd6. That is where the extended partition table is. To extract the partitions, 'dd' will be used. The 'skip' flag will specify the starting sector and the 'count' flag will specify the size of each partition. The starting sector is given in the 'fdisk' output. We'll have to do a little math to get the size value though. The 'fdisk' output gives the starting and ending sectors, so we have to subtract them and add one. For example, the first partition has the following size: 16064999 - 63 + 1 = 16064937 To extract the above partitions, the following are used: # dd if=disk1.dd of=part1.dd bs=512 skip=63 count=16064937 # dd if=disk1.dd of=part2.dd bs=512 skip=16065000 count=16065000 # dd if=disk1.dd of=part3.dd bs=512 skip=32130000 count=8032500 # dd if=disk1.dd of=part4.dd bs=512 skip=40162563 count=8032437 # dd if=disk1.dd of=part5.dd bs=512 skip=48195063 count=10426122 Remember to skip the extended partitions and to calculate the MD5 value for each partition. # md5sum part?.dd | tee part.md5 MD5 (part1.dd) = 356317f7369e9d831e6925622e0779cd MD5 (part2.dd) = c4a6761b486de3c6abf7cf2c554289e5 MD5 (part3.dd) = c7d67b58552a754a1165d8870d2b0aaf MD5 (part4.dd) = fb129325adad04ea02c4ba544c2b2f39 MD5 (part5.dd) = b40170b5b8ec196c1c22739ce259dde3 In Linux, an alternative to extracting each partition is to create a loopback device for each. The 'losetup' tool allows one to create a device that can be read like a normal file. In this case, we will supply a byte offset so that when the beginning of the device is read, the data is really coming from the middle of the disk image where the partition starts. The '-o' flag to 'losetup' is used to specify the byte offset. Since the offset is in bytes and not sectors, we will have to multiply the 'fdisk' start location by 512. For example, the offset for the first partition is: 63 * 512 = 32256 The first step is to identify if the loopback device is being used. The loopback devices are named '/dev/loopX', where X is replaced by a number: # losetup /dev/loop0 The next step is to configure the devices for each partition: # losetup -o 32256 /dev/loop0 disk1.dd # losetup -o 8225280000 /dev/loop1 disk1.dd ... This is clearly faster to setup and requires less disk space, but there is a downside. The device will read data from the disk image until there is no more. For example, if you were to read the device for the first partition, then you would receive all of the data for the first partition, the second, and the third. Therefore, you cannot restrict a 'grep' keyword search to just one partition using 'losetup'. Future versions of The Sleuth Kit will have tools to list the partition details from non-Linux systems and file system tools that can take a disk image as input. Future editions of The Sleuth Kit Informer will have steps for dealing with Solaris and BSD disk labels. [Ed: The January 15, 2004 issue (#12) of The Sleuth Kit Informer expands on this article by showing how to use the 'mmls' tool that now comes with The Sleuth Kit.] -------------------------------------------------------------------------- Copyright (c) 2003 by Brian Carrier. All Rights Reserved