How it works
Processing is organized by the Hadoop framework. The input data is staged by zipping it in archives of a set size. Then in processing each file is read from the archive, assigned a unique ID, and processed with Tika, which extracts text and metadata. Metadata, text, and the file itself are delivered as processed results.
The current and future building blocks of the system are HDFS, Hadoop, Tika, Lucene, Hive.
Indexing
Each FreeEed project will create its own Lucene/Solr index for later searches.
Output
Metadata results are output as a CSV file, while the native files and the extracted text are stored in a zip file(s). The end results can be used for culling and producing native files for legal review.
With the compilation and professional support available for enterprise use, FreeEed brings high performance, scalability and reliability to data processing at a fraction of the cost of proprietary products.
Supported file formats
MS Office and other formats (over 300)
PST processing.
Other capabilities
PST processing.
Text extraction
Data culling
Native/Text/Metadata results delivery
Optical Character Recognition (OCR)
Imaging (PDF creation)
Instant search
Deduplication (configurable for emails)