Autopsy
4.19.3
Graphical digital forensics platform for The Sleuth Kit and other tools.
|
Inherits org.sleuthkit.autopsy.ingest.FileIngestModule.
Classes | |
class | Indexer |
enum | IngestStatus |
enum | StringsExtractOptions |
enum | UpdateFrequency |
Public Member Functions | |
ProcessResult | process (AbstractFile abstractFile) |
void | shutDown () |
void | startUp (IngestJobContext context) throws IngestModuleException |
Private Member Functions | |
void | cleanup () |
Optional< TextExtractor > | getExtractor (AbstractFile abstractFile) |
boolean | isLimitedOCRFile (AbstractFile aFile, String mimeType) |
void | postIndexSummary () |
Static Private Member Functions | |
static void | putIngestStatus (long ingestJobId, long fileId, IngestStatus status) |
Private Attributes | |
IngestJobContext | context |
FileTypeDetector | fileTypeDetector |
Indexer | indexer |
Ingester | ingester = null |
boolean | initialized = false |
int | instanceNum = 0 |
long | jobId |
final IngestServices | services = IngestServices.getInstance() |
final KeywordSearchJobSettings | settings |
boolean | startedSearching = false |
Lookup | stringsExtractionContext |
Static Private Attributes | |
static final List< String > | ARCHIVE_MIME_TYPES |
static final String | IMAGE_MIME_TYPE_PREFIX = "image/" |
static final Map< Long, Map< Long, IngestStatus > > | ingestStatus = new HashMap<>() |
static final AtomicInteger | instanceCount = new AtomicInteger(0) |
static final int | LIMITED_OCR_SIZE_MIN = 100 * 1024 |
static final Logger | logger = Logger.getLogger(KeywordSearchIngestModule.class.getName()) |
static final List< String > | METADATA_DATE_TYPES |
static final Map< String, BlackboardAttribute.ATTRIBUTE_TYPE > | METADATA_TYPES_MAP |
static final ImmutableSet< String > | OCR_DOCUMENTS |
static final IngestModuleReferenceCounter | refCounter = new IngestModuleReferenceCounter() |
An ingest module on a file level Performs indexing of allocated and Solr supported files, string extraction and indexing of unallocated and not Solr supported files Index commit is done periodically (determined by user set ingest update interval) Runs a periodic keyword / regular expression search on currently configured lists for ingest and writes results to blackboard Reports interesting events to Inbox and to viewers
Definition at line 89 of file KeywordSearchIngestModule.java.
|
private |
Common cleanup code when module stops or final searcher completes
Definition at line 459 of file KeywordSearchIngestModule.java.
|
private |
Definition at line 549 of file KeywordSearchIngestModule.java.
References org.sleuthkit.autopsy.ingest.IngestJobContext.fileIngestIsCancelled(), org.sleuthkit.autopsy.textextractors.TextExtractorFactory.getExtractor(), org.sleuthkit.autopsy.keywordsearch.KeywordSearchJobSettings.isOCREnabled(), and org.sleuthkit.autopsy.textextractors.configs.ImageConfig.setOCREnabled().
|
private |
Returns true if file should have OCR performed on it when limited OCR setting is specified.
aFile | The abstract file. |
mimeType | The file mime type. |
Definition at line 474 of file KeywordSearchIngestModule.java.
|
private |
Posts inbox message with summary of text_ingested files
Definition at line 490 of file KeywordSearchIngestModule.java.
References org.sleuthkit.autopsy.ingest.IngestMessage.createMessage(), org.sleuthkit.autopsy.coreutils.MessageNotifyUtil.Notify.error(), org.sleuthkit.autopsy.ingest.IngestMessage.MessageType.INFO, org.sleuthkit.autopsy.ingest.IngestServices.postMessage(), and org.sleuthkit.autopsy.coreutils.MessageNotifyUtil.Notify.warn().
ProcessResult org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.process | ( | AbstractFile | file | ) |
Processes a file. Called between calls to startUp() and shutDown(). Will be called for each file in a data source.
IMPORTANT: In addition to returning ProcessResult.OK or ProcessResult.ERROR, modules should log all errors using methods provided by the org.sleuthkit.autopsy.coreutils.Logger class. Log messages should include the name and object ID of the data being processed and any other information that would be useful for debugging. If an exception has been caught by the module, the exception should be sent to the logger along with the log message so that a stack trace will appear in the application log.
file | The file to analyze. |
Implements org.sleuthkit.autopsy.ingest.FileIngestModule.
Definition at line 355 of file KeywordSearchIngestModule.java.
References org.sleuthkit.autopsy.ingest.IngestJobContext.fileIngestIsCancelled(), org.sleuthkit.autopsy.modules.filetypeid.FileTypeDetector.getMIMEType(), org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.Indexer.indexFile(), org.sleuthkit.autopsy.keywordsearch.KeywordSearchJobSettings.isOCREnabled(), org.sleuthkit.autopsy.ingest.IngestModule.ProcessResult.OK, and org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.IngestStatus.SKIPPED_ERROR_INDEXING.
|
staticprivate |
Records the ingest status for a given file for a given ingest job. Used for final statistics at the end of the job.
ingestJobId | id of ingest job |
fileId | id of file |
status | ingest status of the file |
Definition at line 227 of file KeywordSearchIngestModule.java.
void org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.shutDown | ( | ) |
After all files are ingested, execute final index commit and final search Cleanup resources, threads, timers
Implements org.sleuthkit.autopsy.ingest.IngestModule.
Definition at line 420 of file KeywordSearchIngestModule.java.
References org.sleuthkit.autopsy.ingest.IngestModuleReferenceCounter.decrementAndGet(), org.sleuthkit.autopsy.ingest.IngestJobContext.fileIngestIsCancelled(), org.sleuthkit.autopsy.keywordsearch.KeywordSearch.getServer(), org.sleuthkit.autopsy.keywordsearch.Server.queryNumIndexedChunks(), and org.sleuthkit.autopsy.keywordsearch.Server.queryNumIndexedFiles().
void org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.startUp | ( | IngestJobContext | context | ) | throws IngestModuleException |
Initializes the module for new ingest run Sets up threads, timers, retrieves settings, keyword lists to run on
Implements org.sleuthkit.autopsy.ingest.IngestModule.
Definition at line 256 of file KeywordSearchIngestModule.java.
References org.sleuthkit.autopsy.ingest.IngestMessage.createWarningMessage(), org.sleuthkit.autopsy.casemodule.Case.getCaseDirectory(), org.sleuthkit.autopsy.casemodule.Case.getCaseType(), org.sleuthkit.autopsy.casemodule.Case.getCurrentCaseThrows(), org.sleuthkit.autopsy.ingest.IngestJobContext.getJobId(), org.sleuthkit.autopsy.keywordsearch.Server.getMultiUserServerProperties(), org.sleuthkit.autopsy.keywordsearch.KeywordSearch.getServer(), org.sleuthkit.autopsy.ingest.IngestModuleReferenceCounter.incrementAndGet(), org.sleuthkit.autopsy.casemodule.Case.CaseType.MULTI_USER_CASE, org.sleuthkit.autopsy.ingest.IngestServices.postMessage(), org.sleuthkit.autopsy.keywordsearch.Server.queryNumIndexedDocuments(), org.sleuthkit.autopsy.textextractors.configs.StringsConfig.setExtractUTF16(), org.sleuthkit.autopsy.textextractors.configs.StringsConfig.setExtractUTF8(), org.sleuthkit.autopsy.textextractors.configs.StringsConfig.setLanguageScripts(), and org.sleuthkit.autopsy.keywordsearchservice.KeywordSearchService.tryConnect().
|
staticprivate |
generally text extractors should ignore archives and let unpacking modules take care of them
Definition at line 98 of file KeywordSearchIngestModule.java.
|
private |
Definition at line 206 of file KeywordSearchIngestModule.java.
Referenced by org.sleuthkit.autopsy.keywordsearch.KeywordSearchIngestModule.Indexer.extractStringsAndIndex().
|
private |
Definition at line 194 of file KeywordSearchIngestModule.java.
|
staticprivate |
Definition at line 151 of file KeywordSearchIngestModule.java.
|
private |
Definition at line 193 of file KeywordSearchIngestModule.java.
|
private |
Definition at line 192 of file KeywordSearchIngestModule.java.
|
staticprivate |
Definition at line 217 of file KeywordSearchIngestModule.java.
|
private |
Definition at line 201 of file KeywordSearchIngestModule.java.
|
staticprivate |
Definition at line 203 of file KeywordSearchIngestModule.java.
|
private |
Definition at line 204 of file KeywordSearchIngestModule.java.
|
private |
Definition at line 202 of file KeywordSearchIngestModule.java.
|
staticprivate |
Definition at line 91 of file KeywordSearchIngestModule.java.
|
staticprivate |
Definition at line 190 of file KeywordSearchIngestModule.java.
|
staticprivate |
Definition at line 133 of file KeywordSearchIngestModule.java.
|
staticprivate |
Definition at line 138 of file KeywordSearchIngestModule.java.
|
staticprivate |
Definition at line 154 of file KeywordSearchIngestModule.java.
|
staticprivate |
Definition at line 205 of file KeywordSearchIngestModule.java.
|
private |
Definition at line 191 of file KeywordSearchIngestModule.java.
|
private |
Definition at line 200 of file KeywordSearchIngestModule.java.
|
private |
Definition at line 198 of file KeywordSearchIngestModule.java.
|
private |
Definition at line 199 of file KeywordSearchIngestModule.java.
Copyright © 2012-2022 Basis Technology. Generated on: Thu Mar 30 2023
This work is licensed under a
Creative Commons Attribution-Share Alike 3.0 United States License.