Autopsy  4.19.3
Graphical digital forensics platform for The Sleuth Kit and other tools.
Classes | Public Member Functions | Private Member Functions | Static Private Member Functions | Private Attributes | Static Private Attributes | List of all members
org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule Class Reference

Inherits org.sleuthkit.autopsy.ingest.FileIngestModule.

Classes

class  FindInHashsetsResult
 
class  IngestJobTotals
 

Public Member Functions

ProcessResult process (AbstractFile file)
 
void shutDown ()
 
void startUp (org.sleuthkit.autopsy.ingest.IngestJobContext context) throws IngestModuleException
 

Private Member Functions

void calculateHashes (AbstractFile file, IngestJobTotals totals) throws TskCoreException
 
boolean createArtifactIfNotExists (AbstractFile file, String comment, HashDb db)
 
FindInHashsetsResult findInHashsets (AbstractFile file, AtomicLong totalCount, AtomicLong totalLookupTime, List< HashDb > hashSets, TskData.FileKnown statusIfFound, Function< AbstractFile, String > lookupErrorMessage)
 
String generateComment (HashHitInfo hashInfo)
 
Score getScore (HashDb.KnownFilesType knownFilesType)
 
void initializeHashsets (List< HashDb > allHashSets)
 
void postHashSetHitToBlackboard (AbstractFile abstractFile, String md5Hash, HashDb db, String comment)
 
void reportLookupError (TskException ex, AbstractFile file, Function< AbstractFile, String > lookupErrorMessage)
 
boolean shouldSkip (AbstractFile file)
 

Static Private Member Functions

static synchronized IngestJobTotals getTotalsForIngestJobs (long ingestJobId)
 
static synchronized void postSummary (long jobId, List< HashDb > knownBadHashSets, List< HashDb > noChangeHashSets, List< HashDb > knownHashSets)
 

Private Attributes

Blackboard blackboard
 
final HashDbManager hashDbManager = HashDbManager.getInstance()
 
long jobId
 
final List< HashDbknownBadHashSets = new ArrayList<>()
 
final Function< AbstractFile, String > knownBadLookupError = (file) -> Bundle.HashDbIngestModule_lookingUpKnownBadHashValueErr(file.getName())
 
final List< HashDbknownHashSets = new ArrayList<>()
 
final Function< AbstractFile, String > knownLookupError = (file) -> Bundle.HashDbIngestModule_lookingUpKnownHashValueErr(file.getName())
 
final List< HashDbnoChangeHashSets = new ArrayList<>()
 
final Function< AbstractFile, String > noChangeLookupError = (file) -> Bundle.HashDbIngestModule_lookingUpNoChangeHashValueErr(file.getName())
 
final IngestServices services = IngestServices.getInstance()
 
final HashLookupModuleSettings settings
 
final SleuthkitCase skCase
 

Static Private Attributes

static final Logger logger = Logger.getLogger(HashDbIngestModule.class.getName())
 
static final int MAX_COMMENT_SIZE = 500
 
static final IngestModuleReferenceCounter refCounter = new IngestModuleReferenceCounter()
 
static final HashMap< Long, IngestJobTotalstotalsForIngestJobs = new HashMap<>()
 

Detailed Description

File ingest module to mark files based on hash values.

Definition at line 66 of file HashDbIngestModule.java.

Member Function Documentation

void org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.calculateHashes ( AbstractFile  file,
IngestJobTotals  totals 
) throws TskCoreException
private

Generates hashes for the given file if they haven't already been set. Hashes are saved to the AbstractFile object.

Parameters
fileThe file in order to determine the hash.
totalsThe timing metrics for this process.

Definition at line 464 of file HashDbIngestModule.java.

References org.sleuthkit.autopsy.healthmonitor.HealthMonitor.getTimingMetric(), org.sleuthkit.autopsy.healthmonitor.HealthMonitor.submitNormalizedTimingMetric(), and org.sleuthkit.autopsy.healthmonitor.HealthMonitor.submitTimingMetric().

boolean org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.createArtifactIfNotExists ( AbstractFile  file,
String  comment,
HashDb  db 
)
private

Creates a BlackboardArtifact if artifact does not already exist.

Parameters
fileThe file that had a hash hit.
commentThe comment to associate with this artifact.
dbthe database in which this file was found.
Returns
True if the operation occurred successfully and without error.

Definition at line 433 of file HashDbIngestModule.java.

References org.sleuthkit.autopsy.ingest.IngestMessage.createErrorMessage(), org.sleuthkit.autopsy.modules.hashdatabase.HashLookupModuleFactory.getModuleName(), and org.sleuthkit.autopsy.ingest.IngestServices.postMessage().

FindInHashsetsResult org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.findInHashsets ( AbstractFile  file,
AtomicLong  totalCount,
AtomicLong  totalLookupTime,
List< HashDb hashSets,
TskData.FileKnown  statusIfFound,
Function< AbstractFile, String >  lookupErrorMessage 
)
private

Attempts to find an abstract file in a list of HashDB objects.

Parameters
fileThe file to find.
totalCountThe total cound of files found in this type
totalLookupTimeThe counter tracking the total amount of run time for this operation.
hashSetsThe HashDB objects to cycle through looking for a hash hit.
statusIfFoundThe FileKnown status to set on the file if the file is found in the hashSets.
lookupErrorMessageThe function that generates a message should there be an error in looking up the file in the hashSets.
Returns
Whether or not the file was found and whether or not there was an error during the operation.

Definition at line 369 of file HashDbIngestModule.java.

String org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.generateComment ( HashHitInfo  hashInfo)
private

Generates a formatted comment.

Parameters
hashInfoThe HashHitInfo.
Returns
The formatted comment.

Definition at line 407 of file HashDbIngestModule.java.

Score org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.getScore ( HashDb.KnownFilesType  knownFilesType)
private

Converts HashDb.KnownFilesType to a Score to be used to create an analysis result.

Parameters
knownFilesTypeThe HashDb KnownFilesType to convert.
Returns
The Score to use when creating an AnalysisResult.

Definition at line 505 of file HashDbIngestModule.java.

static synchronized IngestJobTotals org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.getTotalsForIngestJobs ( long  ingestJobId)
staticprivate

Definition at line 103 of file HashDbIngestModule.java.

void org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.initializeHashsets ( List< HashDb allHashSets)
private

Cycle through list of hashsets and place each HashDB in the appropriate list based on KnownFilesType.

Parameters
allHashSetsList of all hashsets from DB manager

Definition at line 162 of file HashDbIngestModule.java.

void org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.postHashSetHitToBlackboard ( AbstractFile  abstractFile,
String  md5Hash,
HashDb  db,
String  comment 
)
private
static synchronized void org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.postSummary ( long  jobId,
List< HashDb knownBadHashSets,
List< HashDb noChangeHashSets,
List< HashDb knownHashSets 
)
staticprivate
ProcessResult org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.process ( AbstractFile  file)

Processes a file. Called between calls to startUp() and shutDown(). Will be called for each file in a data source.

IMPORTANT: In addition to returning ProcessResult.OK or ProcessResult.ERROR, modules should log all errors using methods provided by the org.sleuthkit.autopsy.coreutils.Logger class. Log messages should include the name and object ID of the data being processed and any other information that would be useful for debugging. If an exception has been caught by the module, the exception should be sent to the logger along with the log message so that a stack trace will appear in the application log.

Parameters
fileThe file to analyze.
Returns
A result code indicating success or failure of the processing.

Implements org.sleuthkit.autopsy.ingest.FileIngestModule.

Definition at line 195 of file HashDbIngestModule.java.

References org.sleuthkit.autopsy.ingest.IngestMessage.createErrorMessage(), org.sleuthkit.autopsy.ingest.IngestModule.ProcessResult.ERROR, org.sleuthkit.autopsy.casemodule.Case.getCurrentCaseThrows(), org.sleuthkit.autopsy.modules.hashdatabase.HashLookupModuleFactory.getModuleName(), org.sleuthkit.autopsy.casemodule.Case.getSleuthkitCase(), org.sleuthkit.autopsy.ingest.IngestModule.ProcessResult.OK, org.sleuthkit.autopsy.ingest.IngestServices.postMessage(), org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.IngestJobTotals.totalKnownBadCount, org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.IngestJobTotals.totalLookuptime, and org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.IngestJobTotals.totalNoChangeCount.

void org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.reportLookupError ( TskException  ex,
AbstractFile  file,
Function< AbstractFile, String >  lookupErrorMessage 
)
private

Reports an error when an issue is encountered looking up a file.

Parameters
exThe exception thrown in the error.
fileThe file for which this error applies.
lookupErrorMessageThe function that generates an error message specific to which piece of the ingest processing failed.

Definition at line 308 of file HashDbIngestModule.java.

References org.sleuthkit.autopsy.ingest.IngestMessage.createErrorMessage(), org.sleuthkit.autopsy.modules.hashdatabase.HashLookupModuleFactory.getModuleName(), and org.sleuthkit.autopsy.ingest.IngestServices.postMessage().

boolean org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.shouldSkip ( AbstractFile  file)
private

Returns true if this file should be skipped for processing.

Parameters
fileThe file to potentially skip.
Returns
True if this file should be skipped.

Definition at line 274 of file HashDbIngestModule.java.

void org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.shutDown ( )

Invoked by Autopsy when an ingest job is completed (either because the data has been analyzed or because the job was cancelled), before the ingest module instance is discarded. The module should respond by doing things like releasing private resources, submitting final results, and posting a final ingest message.

IMPORTANT: If the module instances must share resources, the modules are responsible for synchronizing access to the shared resources and doing reference counting as required to release those resources correctly. Also, more than one ingest job may be in progress at any given time. This must also be taken into consideration when sharing resources between module instances. See IngestModuleReferenceCounter.

Implements org.sleuthkit.autopsy.ingest.IngestModule.

Definition at line 649 of file HashDbIngestModule.java.

References org.sleuthkit.autopsy.ingest.IngestModuleReferenceCounter.decrementAndGet().

void org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.startUp ( org.sleuthkit.autopsy.ingest.IngestJobContext  context) throws IngestModuleException

Invoked by Autopsy to allow an ingest module instance to set up any internal data structures and acquire any private resources it will need during an ingest job. If the module depends on loading any resources, it should do so in this method so that it can throw an exception in the case of an error and alert the user. Exceptions that are thrown from startUp() are logged and stop processing of the data source.

IMPORTANT: If the module instances must share resources, the modules are responsible for synchronizing access to the shared resources and doing reference counting as required to release those resources correctly. Also, more than one ingest job may be in progress at any given time. This must also be taken into consideration when sharing resources between module instances. See IngestModuleReferenceCounter.

IMPORTANT: Start up IngestModuleException messages are displayed to the user, if a user is present. Therefore, an exception to the policy that exception messages are not localized is appropriate in this method. Also, the exception messages should be user-friendly.

Parameters
contextProvides data and services specific to the ingest job and the ingest pipeline of which the module is a part.
Exceptions
org.sleuthkit.autopsy.ingest.IngestModule.IngestModuleException

Implements org.sleuthkit.autopsy.ingest.IngestModule.

Definition at line 127 of file HashDbIngestModule.java.

References org.sleuthkit.autopsy.ingest.IngestMessage.createWarningMessage(), org.sleuthkit.autopsy.modules.hashdatabase.HashDbManager.getAllHashSets(), org.sleuthkit.autopsy.modules.hashdatabase.HashLookupModuleFactory.getModuleName(), org.sleuthkit.autopsy.ingest.IngestModuleReferenceCounter.incrementAndGet(), and org.sleuthkit.autopsy.ingest.IngestServices.postMessage().

Member Data Documentation

Blackboard org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.blackboard
private

Definition at line 90 of file HashDbIngestModule.java.

final HashDbManager org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.hashDbManager = HashDbManager.getInstance()
private

Definition at line 82 of file HashDbIngestModule.java.

long org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.jobId
private

Definition at line 87 of file HashDbIngestModule.java.

final List<HashDb> org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.knownBadHashSets = new ArrayList<>()
private

Definition at line 84 of file HashDbIngestModule.java.

final Function<AbstractFile, String> org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.knownBadLookupError = (file) -> Bundle.HashDbIngestModule_lookingUpKnownBadHashValueErr(file.getName())
private

Definition at line 71 of file HashDbIngestModule.java.

final List<HashDb> org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.knownHashSets = new ArrayList<>()
private

Definition at line 85 of file HashDbIngestModule.java.

final Function<AbstractFile, String> org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.knownLookupError = (file) -> Bundle.HashDbIngestModule_lookingUpKnownHashValueErr(file.getName())
private

Definition at line 77 of file HashDbIngestModule.java.

final Logger org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.logger = Logger.getLogger(HashDbIngestModule.class.getName())
staticprivate

Definition at line 68 of file HashDbIngestModule.java.

final int org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.MAX_COMMENT_SIZE = 500
staticprivate

Definition at line 79 of file HashDbIngestModule.java.

final List<HashDb> org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.noChangeHashSets = new ArrayList<>()
private

Definition at line 86 of file HashDbIngestModule.java.

final Function<AbstractFile, String> org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.noChangeLookupError = (file) -> Bundle.HashDbIngestModule_lookingUpNoChangeHashValueErr(file.getName())
private

Definition at line 74 of file HashDbIngestModule.java.

final IngestModuleReferenceCounter org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.refCounter = new IngestModuleReferenceCounter()
staticprivate

Definition at line 89 of file HashDbIngestModule.java.

final IngestServices org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.services = IngestServices.getInstance()
private

Definition at line 80 of file HashDbIngestModule.java.

final HashLookupModuleSettings org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.settings
private

Definition at line 83 of file HashDbIngestModule.java.

final SleuthkitCase org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.skCase
private

Definition at line 81 of file HashDbIngestModule.java.

final HashMap<Long, IngestJobTotals> org.sleuthkit.autopsy.modules.hashdatabase.HashDbIngestModule.totalsForIngestJobs = new HashMap<>()
staticprivate

Definition at line 88 of file HashDbIngestModule.java.


The documentation for this class was generated from the following file:

Copyright © 2012-2022 Basis Technology. Generated on: Thu Sep 29 2022
This work is licensed under a Creative Commons Attribution-Share Alike 3.0 United States License.