Aiimi Insight Engine Habanero
User GuidesAiimi
  • Introducing Aiimi Insight Engine
  • Architecture
    • Overview and Key Concepts
    • Search Flows
      • Search Flow Types
      • Smart Filtering
      • Query and Prompt Classification
      • Search Algorithms
      • Extractive and Generative Models
    • Hosting Options
    • Architecture and How It Works
      • Agent Servers
        • Security Agent
        • Source Agent
        • Content Agent
        • Enrichment Agent
        • Job Agent
        • OCR Agent
        • Migration Agent
        • Tika Agent
      • Repository
        • Data Node
        • Proxy Node
        • Kibana Node
      • Gateway and User Interface
      • Document and Data Sources
    • Deployment Options
    • Security
      • User Security
      • Data and Document Security
        • Progressive Access
        • Privileged Access
      • Source System Security
      • Firewalling
      • Agent Servers
        • Security Agent
        • Source Agent
        • Content Agent
        • Enrichment Agent
        • Job Agent
        • OCR Agent
        • Migration Agent
        • Tika Agent
      • Repository
      • Gateway (Web Server)
      • Tools & Utilities
  • Installation
    • Elastic and Kibana Install (Windows)
    • Aiimi Insight Engine Installation (Windows)
      • Installation Security
      • Certificates in a Key Vault
      • SAR Configuration
      • CSOM Bridge Set Up
    • AI Services
      • Prerequisites
      • AI Enrichment Service
        • Installation and Setup
        • Enabling Enrichment Steps
        • Using AI Enrichment Steps
        • Performance and Concurrency
      • AI Model Service
        • Installation and Setup
        • Enabling Providers
        • Private Generative AI
        • Azure Open AI
        • Enabling AI History
        • HTML Cleaner Service
      • Configuration of Logging
      • Offline Set-up of Models
      • Using SSL
      • Running as a Service (Windows)
      • Using GPUs
      • AI and Semantic Search Set Up
        • Open & Closed Book AI
        • Semantic Search
          • Vectors for Semantic Search
          • Source Configuration
          • Sentence Transformer Models
          • Enrichment
          • Kibana
          • Final Search Flow
    • Email Threading Upgrade
  • Control Hub
    • Configurations
      • Config Management
      • Security Configurations
        • Security - General
        • Security - Source
          • Active Directory
          • Atlassian
          • Azure Active Directory
          • Builtin Security
          • Miro Security
          • Google Directory
          • Slack Security
        • Security - Sync
        • Security - Agents
        • Security - Scheduling
      • Source Configurations
        • Source - General
        • Source - Source
          • Alfresco Kafka
          • Azure Blob Storage
          • BBC Monitoring
          • Big Query Cataloguer
          • BIM360
          • CSV Data Loader
          • Confluence
          • Content Server
          • Data File Cataloguer
          • Document Store
          • DocuSign
          • Dropbox
          • Exchange 365
          • Filesystem
          • Google Bucket
          • Google Drive
          • Google Vault
          • Jira
          • JSON Data Loader
          • Livelink
          • Microsoft Teams
          • Mimecast
          • Miro
          • ODBC Data Loader
          • PowerBi Cataloguer
          • Reuters Connect
          • ShareFile
          • SharePoint
            • Azure Portal and Azure AD Authentication
          • SharePoint Legacy
          • SQL Server Cataloguer
          • Slack
          • Versioned Document Store
          • Websites
          • XML Data Loader
        • Source - Crawl
        • Source - Agents
        • Source - Schedule
        • Source - Advanced
      • Enrichment Configurations
        • Creating a Pipeline
          • General
          • Steps
            • AccessMiner
            • AI Classification
            • Anonymiser
            • CAD Extractor
            • Checksum
            • Content Retrieval
            • Copy
            • Data Rule Processor
            • Delete
            • Email Extractor
            • Entity Rule Processor
            • External Links
            • Geotag
            • Google NLP Extractor
            • Google Vision Extractor
            • Metrics Calculation
            • Microsoft Vision Extractor
            • OcrRest
            • Office Metadata
            • PCI Extractor
            • REST
            • Set Document Risk
            • Text Cleaner
            • Tika Text Extraction
            • Trie Entity Extractor
            • Update Metadata
          • Filters
          • Agents
          • Schedule
          • Advanced
      • OCR Engine
      • Job Configurations
        • General
        • Job
          • AutomatedSearchJob
          • Command Job
          • ElasticJob
          • Extended Metrics Job
          • File Extractor
          • GoogleVaultSAR
          • Google Drive Last Access Date
          • Nightly Events Processor Job
          • Notifications Processor Job
          • Portal Sync Job
          • Purge Job
          • Text Content Merge Job
        • Output
        • Agents
        • Scheduling
      • Migration Configuration
        • General
        • Filter
        • Metadata Mappings
        • Agents
        • Scheduling
        • Advanced
    • Credentials
      • Create a Credential
      • Find a Credential
      • Edit a Credential
      • Delete a Credential
    • Mappings
      • Entities
        • Group Management
        • Create an Entity
        • Managing Entities
      • Models
        • Create a New Model
        • Find a Model
        • Enable or Disable a Model
      • Vectors
      • Rank Features
    • Featured Links
    • AI Settings
      • Classifications
      • Class
      • Class Rules
      • AI Classification
    • User Settings
    • Stats
      • Data Views
    • Global Settings
      • General
        • Stackdriver
        • Document Recommendations
        • Searchable PDF Storage
        • Versioning
        • Results
        • Marking Useful Results
        • Folder Browsing
        • Cascading Search
        • Search Suggestions
        • Miscellaneous
      • Authentication
      • App Settings
      • Application Access
      • Thumbnails
      • Presets
      • Code of Conduct
      • Metrics
      • Viewer
        • Preview Configurations
        • Redacting Information
        • Watermarking
      • SAR
        • Importing Data For A SAR
        • SAR Disclosure Document Storage
        • Getting SAR data from Google Vault
        • SAR Access
        • SAR File Status
      • Disclosure Portal
        • Disclosure Portal Set Up
        • SARs From The Portal
        • Email Delivery Settings
          • Delivery Settings
          • Brand Settings
          • Customise Emails
        • SMS Delivery Settings
        • Requestor Message Limit
        • Attachment Configuration
        • Password Configuration
        • File Scanner Configurator
      • Collections
      • Visualisations
        • Related Result Connections Diagram
        • Event Timeline
        • Timeline Lens Activity Chart
        • Relationship Map
      • Notifications
      • Map Lens
      • Theming
        • General
        • Layout
        • Site Message
        • Colours
      • User Avatar
      • OData API
      • Uploads
      • Security
    • Search Settings
      • Search Relevancy
        • Core Settings
        • Makers Algorithm
        • Filename Boost Layer
        • Minimum Matching Terms Filter
        • Field Boost
        • Modified Date Boosting
        • Hit Highlighting
        • Why My Search Matched
        • Data Search Strategy
      • Bulk Search
        • Managing a Bulk Search
      • Search Flows
        • Create a Search Flow
          • General
          • Query Classification Step
          • Search Steps
          • Model Steps
      • Filtering
      • Search Performance
      • Related Results
  • API Guides
    • Insight API Guide
      • Swagger Documentation
      • Trying Some Endpoints
      • Search Filter
      • Hits / Items
      • Inspecting REST Calls
    • Data Science API Guide
      • REST Interface
        • Login
        • Datasets
        • Fields
        • Field Statistics
        • Search
        • Scroll
        • Update
      • Python Wrapper
        • Login
        • Datasets
        • Fields
        • Field Statistics
        • Search
        • Query Builders
        • Scroll
        • Scroll Search
        • Update Single Document
        • Bulk Update
    • Creating a Native Enrichment Step
      • Creating an Enrichment Step
        • Creating the Core Classes
        • Extending our Enrichment Step
        • Adding a Configuration Template
        • Adding the Enrichment Step
        • Creating an Enrichment Pipeline
      • Other Tasks
        • Entities, Metadata and Data
        • Accessing the Repository
      • Example Code
      • Troubleshooting
    • Creating a Python Enrichment Step
      • Creating an Enrichment Step
        • Running the Example from Command Line
        • Running the Example
      • Creating Your Own Step
      • Adding or Changing Entities, Metadata
  • whitepapers and explainers
    • From a Billion To One – Mastering Relevancy
    • Methods for Text Summarization
      • Application
      • Technology Methods
      • Commercial Tools
      • Key Research Centres
      • Productionisation
      • Related Areas of Text Analytics
      • Conclusion
      • References
Powered by GitBook
On this page
  • Size and Quality
  • Processing Mode
  • Change to an External Process
  • Storage
  • File System Storage
  • Azure Storage
  • Google Storage
  1. Control Hub
  2. Global Settings

Thumbnails

PreviousApplication AccessNextPresets

Needed for initial set up

There are a few things that need to be configured for thumbnails within Aiimi Insight Engine. This includes where they are stored. Thumbnails are shown next to as many files shown in a results list as possible. Some documents can't have a thumbnail created so a default is applied. If a file does not have a thumbnail Aiimi Insight Engine will create one based of the file and the data within it.

Any changes made to the settings regarding thumbnails will not work retroactively. Enrichment will need to be re-run for any updates to existing data.

Size and Quality

  1. You can choose to enable or disable thumbnails with the Enable Thumbnails checkbox.

  2. Choose the Thumbnail Size that is shown on the screen.

    • Choose between Tiny, Small, Medium or Large.

  3. Use the Quality slider to determine how clear the thumbnail is.

    • The smaller the size the less someone can see so quality can be sacrificed.

  4. You can choose to show no thumbnail for files modified over a certain number of months ago. To set a modified date range enter the Maximum Modified Age in months.

    • To show all thumbnails despite modified date leave Maximum Modified Age empty.

    • If you enter 6, files last modified within 6 months will show a thumbnail.

  5. Set a Max degree of Parallelism to determine the number of thumbnailing process that can run at the same time.

    • This is capped at the maximum number of logical processors available in your content agent.


Processing Mode

Thumbnails can be processed in the content agent when it runs or as an external process. We recommend and default to the InProcess process mode.

If you see issues like runaway memory usage, consistent high CPU usage but no files processed then you should consider changing your Process mode.

Change to an External Process

The External Process is slower as it runs the thumbnail and document conversion processes separately. When using an external process it can be stopped by setting memory and runtime limits.

When changing your JSON file remember to keep any existing settings and make all the necessary changes so that it is still valid.

  1. Select mode and choose ExternalProcess.

  2. Set limits to automatically stop the external process.

    • Change the memory limit in GB to suit your needs. By default this is set to 5GB.

    • Change the RunTime limit in seconds to suit your needs. By default this is set to 60 seconds.

  3. Open your appsettings.json

  4. Update the json to point to the external process.

    • Add your external process as a contentUtilitesPath.

For example:

"contentUtilitesPath": "C:\\InsightMaker\\Utils\\InsightMaker.ContentUtilities\\InsightMaker.ContentUtilities.exe"

Storage

Configuring the storage for the Content Retrieval enrichment step happens here. The input folder is an area that holds documents and data that needs a thumbnail. Then there are ContentAgent resources available the files in that root folder will have thumbnails created and stored in the output folder.

  1. Select the Storage for the queue of documents that should have thumbnails generated from the Input Storage Configuration dropdown.

    • Choose between File System Storage, Azure Storage or Google Storage.

  2. Select the storage for the thumbnails from the Output Storage Configuration dropdown.

    • Choose between File System Storage, Azure Storage or Google Storage.

You can have a different storage configuration for the input and output. Depending on your chosen storage you will need to populate different information.

File System Storage

  1. Enter the root path of the folder within Store Root.

  2. Choose the amount of Reserved Disk Space for the folder.

    • Enter the reserve amount in bytes.

    • You may want to reserve disk space for the input and output folders. Input only holds temporary data and is removed once the thumbnail is made.

  3. Check Error on full storage if you want to surface an error.

Azure Storage

  1. Enter the Endpoint Suffix for the storage account.

  2. Enter the name of the storage account to crawl within Account Name.

  3. Select the Account Key Type from the dropdown.

  4. Choose the related credential from the Select Credential dropdown list.

  5. Enter the root Azure Container ID for the Azure container.

Google Storage

  1. Enter the Project ID from the Google Cloud Platform.

  2. Enter the Bucket Location from the Google Cloud Platform.

  3. Enter the Bucket Prefix to be added to the beginning of all buckets.