Hash Lookup Module

What Does It Do

The Hash Lookup Module calculates MD5 hash values for files and looks up hash values in a database to determine if the file is notable, known (in general), or unknown.

Configuration

The Hash Sets tab on the Options panel is where you can set and update your hash set information. Hash sets are used to identify files that are 'known' or 'notable'.

• Known good files are those that can be safely ignored. This set of files frequently includes standard OS and application files. Ignoring such uninteresting-to-the-investigator files, can greatly reduce image analysis time.
• Notable (or known bad) files are those that should raise awareness. This set will vary depending on the type of investigation, but common examples include contraband images and malware.

Importing Hash Sets

To import an existing hash set, use the "Import Database" button on the Hash Sets options panel. This will bring up a dialog to import the file.

Database Path - The path to the hash set you are importing. Autopsy supports the following formats:

• Text: One hash starting each line. For example, the output from running the md5, md5sum, or md5deep program on a set of files (*.txt)
• Index only: Generated by Sleuth Kit/Autopsy. The NSRL is available in this format for use with Autopsy (see below) (*.idx)
• Sleuth Kit/Autopsy format database: SQLite hash sets created by Autopsy (*.kdb)
• EnCase: An EnCase hash set file (*.hash)
• HashKeeper: Hash set file conforming to the HashKeeper standard (*.hsh)

Destination - The Destination field refers to where the hash set will be stored.

• Local: The hash set file will be used from original the location on disk
• Remote: The hash set will be copied into the central repository. When using a PostgreSQL central repository, this allows multiple users to easily share the same hash sets.

Name - Display name of the hash set. One will be suggested based on the file name, but this can be changed.

Version - The version of the hash set can only be entered when importing the hash set into the central repository. Additionally, no version can be entered if the hash set is not read-only.

Source Organization - The organization can only be entered when importing the hash set into the central repository. See the section on managing organizations for more information.

Type of database - All entries in the hash set should either be "known" (can be safely ignored) or "notable" (could be indicators of suspicious behavior).

Make database read-only - The read-only setting is only active when importing the hash set into the central repository. A read-only database can not have new hashes added to it through either the Hash Sets options panel or the context menu. For locally imported hash sets, whether they can be written to is dependent on the type of hash set. Autopsy format databases (*.kdb) can be edited, but all other types will be read-only.

Send ingest inbox message for each hit - Determines whether a message is sent for each matching file. This can not be enabled for a "known" hash set.

Copy hash set into user configuration folder - Makes a copy of the hash set instead of using the existing one. This is intended to be used with a Creating a Live Triage Drive drive.

Indexing

After importing the hash set, you may have to index it before it can be used. For most hash set types, Autopsy needs an index of the hash set to actually use a hash set. It can create the index if you import only the hash set. Any hash sets that require an index will be displayed in red, and their "Index Status" will indicate that an index needs to be created. This is done simply by using the Index button.

Autopsy uses the hash set management system from The Sleuth Kit. You can manually create an index using the 'hfind' command line tool or you can use Autopsy. If you attempt proceed without indexing a hash set, Autopsy will offer to automatically produce an index for you. You can also specify only the index file and not use the full hash set - the index file is sufficient to identify known files. This can save space. To do this, specify the .idx file from the Hash Sets option panel.

Creating Hash sets

New hash sets can be created using the "New Hash Set" button. The fields are mostly the same as the import dialog described above.

In this case, the Database Path is where the new database will be stored. If the central repository is being used then this field is not needed.

Using Hash Sets

There is an ingest module that will hash the files and look them up in the hash sets. It will flag files that were in the notable hash set and those results will be shown in the Results tree of the Tree Viewer. Other ingest modules are able to use the known status of a file to decide if they should ignore the file or process it. You can also see the results in the File Search window. There is an option to choose the 'known status'. From here, you can do a search to see all 'notable' files. From here, you can also choose to ignore all 'known' files that were found in the NSRL. You can also see the status of the file in a column when the file is listed.

NIST NSRL

Autopsy can use the NIST NSRL to detect 'known files'. The NSRL contains hashes of 'known files' that may be good or bad depending on your perspective and investigation type. For example, the existence of a piece of financial software may be interesting to your investigation and that software could be in the NSRL. Therefore, Autopsy treats files that are found in the NSRL as simply 'known' and does not specify good or bad. Ingest modules have the option of ignoring files that were found in the NSRL.

To use the NSRL, you may download a pre-made index from http://sourceforge.net/projects/autopsy/files/NSRL. Download the NSRL-XYZm-autopsy.zip (where 'XYZ' is the version number. As of this writing, it is 247) and unzip the file. Use the "Tools", "Options" menu and select the "Hash Sets" tab. Click "Import Database" and browse to the location of the unzipped NSRL file. You can change the Hash Set Name if desired. Select the type of database desired, choosing "Send ingest inbox message for each hit" if desired, and then click "OK".

Using the Module

Ingest Settings

When hash sets are configured, the user can select the hash sets to use during the ingest process.

Seeing Results

Results show up in the tree as "Hashset Hits", grouped by the name of the hash set.