Aiimi Insight Engine
User GuidesAiimi
  • Introducing Aiimi Insight Engine
  • Architecture
    • Overview and Key Concepts
    • Hosting Options
    • Architecture and How It Works
      • Agent Servers
        • Security Agent
        • Source Agent
        • Content Agent
        • Enrichment Agent
        • Job Agent
        • OCR Agent
        • Migration Agent
        • Tika Agent
      • Repository
        • Data Node
        • Proxy Node
        • Kibana Node
      • Gateway and User Interface
      • Document and Data Sources
    • Deployment Options
    • Security
      • User Security
      • Data and Document Security
      • Source System Security
      • Firewalling
      • Agent Servers
      • Repository
      • Gateway (Web Server)
      • Tools & Utilities
  • Installation
    • Elastic and Kibana Install (Windows)
    • Aiimi Insight Engine Installation (Windows)
      • Installation Security
      • Certificates in a Key Vault
      • SAR Configuration
      • CSOM Bridge Set Up
      • AI Studio
    • AI Services
      • Prerequisites
      • AI Enrichment Service
        • Installation and Setup
        • Enabling Enrichment Steps
        • Using AI Enrichment Steps
        • Performance and Concurrency
      • AI Model Service
        • Installation and Setup
        • Enabling Providers
        • Private Generative AI
        • Azure Open AI
        • Enabling AI History
        • HTML Cleaner Service
      • Configuration of Logging
      • Offline Set-up of Models
      • Using SSL
      • Running as a Service (Windows)
      • Using GPUs
      • AI and Semantic Search Set Up
        • Open & Closed Book AI
        • Semantic Search
          • Vectors for Semantic Search
          • Source Configuration
          • Sentence Transformer Models
          • Enrichment
          • Kibana
          • Final Search Flow
    • Email Threading Upgrade
  • Run Books
    • SharePoint Online Connector
      • Migrating ACS to Azure AD with Sites.Read.All
      • Migrating ACS to Azure AD with Sites.FullControl.All
  • Control Hub
    • Agents
      • Configurations
        • Config Management
        • Security Configurations
          • Security - General
          • Security - Source
            • Active Directory
            • Atlassian
            • Azure Active Directory
            • Builtin Security
            • Miro Security
            • Google Directory
            • SharePoint Security
            • Slack Security
          • Security - Sync
          • Security - Agents
          • Security - Scheduling
        • Source Configurations
          • Source - General
          • Source - Source
            • Alfresco Kafka
            • Azure Blob Storage
            • BBC Monitoring
            • Big Query Cataloguer
            • BIM360
            • CSV Data Loader
            • Confluence
            • Content Server
            • Data File Cataloguer
            • Document Store
            • DocuSign
            • Dropbox
            • Exchange 365
            • Filesystem
            • Google Bucket
            • Google Drive
            • Google Vault
            • Jira
            • JSON Data Loader
            • Livelink
            • Microsoft Teams
            • Mimecast
            • Miro
            • ODBC Data Loader
            • PowerBi Cataloguer
            • Reuters Connect
            • ShareFile
            • SharePoint
              • Azure Portal and Azure AD Authentication
              • Sensitivity Labels
            • SharePoint Legacy
            • SQL Server Cataloguer
            • Slack
            • Versioned Document Store
            • Websites
            • XML Data Loader
          • Source - Crawl
          • Source - Agents
          • Source - Schedule
          • Source - Advanced
        • Enrichment Configurations
          • Creating a Pipeline
            • General
            • Steps
              • AccessMiner
              • AI Classification
              • Apply Sensitivity Label
              • Anonymiser
              • CAD Extractor
              • Checksum
              • Content Retrieval
              • Copy
              • Data Rule Processor
              • Delete
              • Direct Copy
              • Email Extractor
              • Entity Rule Processor
              • External Links
              • Geotag
              • Google NLP Extractor
              • Google Vision Extractor
              • Metrics Calculation
              • Microsoft Vision Extractor
              • OcrRest
              • Office Metadata
              • PCI Extractor
              • REST
              • Set Document Risk
              • Text Cleaner
              • Tika Text Extraction
              • Trie Entity Extractor
              • Update Metadata
            • Filters
            • Agents
            • Schedule
            • Advanced
        • OCR Engine
        • Job Configurations
          • General
          • Job
            • AutomatedSearchJob
            • Command Job
            • ElasticJob
            • Extended Metrics Job
            • File Extractor
            • GoogleVaultSAR
            • Google Drive Last Access Date
            • Nightly Events Processor Job
            • Notifications Processor Job
            • Portal Sync Job
            • Purge Job
            • SAR Archiving
            • Text Content Merge Job
          • Output
          • Agents
          • Scheduling
        • Migration Configuration
          • General
          • Filter
          • Metadata Mappings
          • Agents
          • Scheduling
          • Advanced
      • Stats
        • Data Views
    • Security
      • User Settings
      • Credentials
      • Authentication
      • Application Access
      • Auditing
      • Descriptor Groups
      • Uploads
    • Mappings
      • Entities
        • Manage Entity Groups
        • Create an Entity
        • Manage an Entity
      • Models
        • Create a New Model
        • Find a Model
        • Enable or Disable a Model
      • Vectors
      • Rank Features
    • Search Settings
      • Search Relevancy
        • Core Settings
        • Makers Algorithm
        • Filename Boost Layer
        • Minimum Matching Terms Filter
        • Field Boost
        • Modified Date Boosting
        • Hit Highlighting
        • Why My Search Matched
        • Data Search Strategy
      • Bulk Search
        • Managing a Bulk Search
      • Filtering
      • Search Performance
      • Related Results
      • Featured Links
    • AI Settings
      • Search Flows
        • Search Flow Types
        • General Configuration
        • Query and Prompt Classification
        • Search Steps
        • Smart Filtering
        • Model Steps
        • Result Templates
        • System Prompt
      • Tools
        • Concepts
        • Import OOTB Tools
        • Built In Functions and Tools
        • Create and Edit Tools
      • Classifications
        • Class
        • Class Rules
        • AI Classification
    • User Interface
      • Thumbnails
      • Code of Conduct
      • Visualisations
        • Related Result Connections Diagram
        • Event Timeline
        • Timeline Lens Activity Chart
        • Relationship Map
      • Map Lens
      • Theming
      • User Avatar
    • Global Settings
      • General
      • App Settings
      • Presets
      • Metrics
      • Viewer
      • SAR
        • Importing Data For A SAR
        • SAR Disclosure Document Storage
        • Getting SAR data from Google Vault
        • SAR Configuration Access
        • SAR File Status
      • Disclosure Portal
        • Disclosure Portal Set Up
        • SARs From The Portal
        • Email Delivery Settings
          • Delivery Settings
          • Brand Settings
          • Customise Emails
        • SMS Delivery Settings
        • Requestor Message Limit
        • Attachment Configuration
        • Password Configuration
        • File Scanner Configurator
      • Collections
      • Notifications
      • OData API
  • AI Studio
    • Classifications
      • Classifications
      • Classification Rules
    • Jobs
  • Labels
  • API Guides
    • Insight API Guide
      • Swagger Documentation
      • Trying Some Endpoints
      • Search Filter
      • Hits / Items
      • Inspecting REST Calls
    • Data Science API Guide
      • REST Interface
        • Login
        • Datasets
        • Fields
        • Field Statistics
        • Search
        • Scroll
        • Update
      • Python Library
      • Data Science API Wrapper
        • Login
        • Datasets
        • Fields
        • Field Statistics
        • Search
        • Scroll
        • Scroll Search
        • Update Single Document
        • Bulk Update
      • Search API Wrapper
        • Login
        • Privileged Access
        • Search
        • Collection
        • ChatBot Class
      • Admin API Wrapper
      • AI Model Server API Wrapper
      • Utilities
        • Query Builders
        • Azure Key Vault Wrapper
    • Creating a Native Enrichment Step
      • Creating an Enrichment Step
        • Creating the Core Classes
        • Extending our Enrichment Step
        • Adding a Configuration Template
        • Adding the Enrichment Step
        • Creating an Enrichment Pipeline
      • Other Tasks
        • Entities, Metadata and Data
        • Accessing the Repository
      • Example Code
      • Troubleshooting
    • Creating a Python Enrichment Step
      • Creating an Enrichment Step
        • Running the Example from Command Line
        • Running the Example
      • Creating Your Own Step
      • Adding or Changing Entities, Metadata
  • Whitepapers and Explainers
    • From a Billion To One – Mastering Relevancy
    • Methods for Text Summarization
      • Application
      • Technology Methods
      • Commercial Tools
      • Key Research Centres
      • Productionisation
      • Related Areas of Text Analytics
      • Conclusion
      • References
Powered by GitBook
On this page
  1. Control Hub
  2. Agents
  3. Configurations
  4. Job Configurations
  5. Job

File Extractor

PreviousExtended Metrics JobNextGoogleVaultSAR
CtrlK
  • Archives
  • Email Attachments
  • Collections
  • Upload Report

You can control what happens to PST or Zipped files when they are added to a collection. By allowing them to be extracted in a collection users can freely see the files and process them as required.

SAR Prerequisites

The SAR Import settings must be enabled and the storage locations set.

Use our guide on Importing Data For a SAR for help setting up SAR imports.

  • The File Import Size Limit should be the same size or bigger than the maximum archive size set in the job.

  • The Source System needs to be a location that both the Agent and Web services can access.

  • Within Advanced on the source make sure write, update and delete are checked.

Collection Prerequisites

You can allow extractions in both normal and redactable collections.

  • For collections the destination source must be a Document Store.

  • Within the source check Enable for Configurable Collection to allow the source to be selected within the job.

  • Within Advanced on the source make sure write, update and delete are checked.

Archives

  1. Process Email Archives - If checked email archives (PST, OST) will be processed as part of this job.

  2. Extract File Archives - If checked file archives (ZIP, RAR, 7z) will be processed as part of this job.

  3. Maximum Archive Size - Enter the maximum size of an archive that can be processed in bytes.

    • Set to 0 for no limit.

Email Attachments

  1. Extract Email Attachments - If checked all email attachments will be extracted and stored as separate files to the email.

  2. Excluded Attachment Names - Use Regular Expressions to choose what attachments are excluded from processing.

    • This is based on the name of the attachment.

    • If left blank all will be processed.

Collections

  1. Collection Filter - Select what type of collection this feature is enabled on from the dropdown.

    • All - Run the process when an eligible file is added to any collection.

    • Redaction Enabled - Only run the process when an eligible file is added to a collection with redaction enabled.

    • SAR Only - Only run the process when an eligible file is added to a SAR collection.

Upload Report

  1. Generate Upload Report - If checked an upload report for each extracted archive will be generated.