Aiimi Insight Engine
User GuidesAiimi
  • Introducing Aiimi Insight Engine
  • Architecture
    • Overview and Key Concepts
    • Search Flows
      • Search Flow Types
      • Smart Filtering
      • Query and Prompt Classification
      • Search Algorithms
      • Extractive and Generative Models
    • Hosting Options
    • Architecture and How It Works
      • Agent Servers
        • Security Agent
        • Source Agent
        • Content Agent
        • Enrichment Agent
        • Job Agent
        • OCR Agent
        • Migration Agent
        • Tika Agent
      • Repository
        • Data Node
        • Proxy Node
        • Kibana Node
      • Gateway and User Interface
      • Document and Data Sources
    • Deployment Options
    • Security
      • User Security
      • Data and Document Security
      • Source System Security
      • Firewalling
      • Agent Servers
      • Repository
      • Gateway (Web Server)
      • Tools & Utilities
  • Installation
    • Elastic and Kibana Install (Windows)
    • Aiimi Insight Engine Installation (Windows)
      • Installation Security
      • Certificates in a Key Vault
      • SAR Configuration
      • CSOM Bridge Set Up
      • AI Studio
    • AI Services
      • Prerequisites
      • AI Enrichment Service
        • Installation and Setup
        • Enabling Enrichment Steps
        • Using AI Enrichment Steps
        • Performance and Concurrency
      • AI Model Service
        • Installation and Setup
        • Enabling Providers
        • Private Generative AI
        • Azure Open AI
        • Enabling AI History
        • HTML Cleaner Service
      • Configuration of Logging
      • Offline Set-up of Models
      • Using SSL
      • Running as a Service (Windows)
      • Using GPUs
      • AI and Semantic Search Set Up
        • Open & Closed Book AI
        • Semantic Search
          • Vectors for Semantic Search
          • Source Configuration
          • Sentence Transformer Models
          • Enrichment
          • Kibana
          • Final Search Flow
    • Email Threading Upgrade
  • Run Books
    • SharePoint Online Connector
  • Control Hub
    • Configurations
      • Config Management
      • Security Configurations
        • Security - General
        • Security - Source
          • Active Directory
          • Atlassian
          • Azure Active Directory
          • Builtin Security
          • Miro Security
          • Google Directory
          • Slack Security
        • Security - Sync
        • Security - Agents
        • Security - Scheduling
      • Source Configurations
        • Source - General
        • Source - Source
          • Alfresco Kafka
          • Azure Blob Storage
          • BBC Monitoring
          • Big Query Cataloguer
          • BIM360
          • CSV Data Loader
          • Confluence
          • Content Server
          • Data File Cataloguer
          • Document Store
          • DocuSign
          • Dropbox
          • Exchange 365
          • Filesystem
          • Google Bucket
          • Google Drive
          • Google Vault
          • Jira
          • JSON Data Loader
          • Livelink
          • MFiles
          • Microsoft Teams
          • Mimecast
          • Miro
          • ODBC Data Loader
          • PowerBi Cataloguer
          • Reuters Connect
          • ShareFile
          • SharePoint
            • Azure Portal and Azure AD Authentication
            • Sensitivity Labels
          • SharePoint Legacy
          • SQL Server Cataloguer
          • Slack
          • Versioned Document Store
          • Websites
          • XML Data Loader
        • Source - Crawl
        • Source - Agents
        • Source - Schedule
        • Source - Advanced
      • Enrichment Configurations
        • Creating a Pipeline
          • General
          • Steps
            • AccessMiner
            • AI Classification
            • Anonymiser
            • CAD Extractor
            • Checksum
            • Content Retrieval
            • Copy
            • Data Rule Processor
            • Delete
            • Email Extractor
            • Entity Rule Processor
            • External Links
            • Geotag
            • Google NLP Extractor
            • Google Vision Extractor
            • Metrics Calculation
            • Microsoft Vision Extractor
            • OcrRest
            • Office Metadata
            • PCI Extractor
            • REST
            • Set Document Risk
            • Text Cleaner
            • Tika Text Extraction
            • Trie Entity Extractor
            • Update Metadata
          • Filters
          • Agents
          • Schedule
          • Advanced
      • OCR Engine
      • Job Configurations
        • General
        • Job
          • AutomatedSearchJob
          • Command Job
          • ElasticJob
          • Extended Metrics Job
          • File Extractor
          • GoogleVaultSAR
          • Google Drive Last Access Date
          • Nightly Events Processor Job
          • Notifications Processor Job
          • Portal Sync Job
          • Purge Job
          • Text Content Merge Job
        • Output
        • Agents
        • Scheduling
      • Migration Configuration
        • General
        • Filter
        • Metadata Mappings
        • Agents
        • Scheduling
        • Advanced
    • Credentials
    • Mappings
      • Entities
        • Manage Entity Groups
        • Create an Entity
        • Manage an Entity
      • Models
        • Create a New Model
        • Find a Model
        • Enable or Disable a Model
      • Vectors
      • Rank Features
    • Featured Links
    • AI Settings
      • Classifications
      • Class
      • Class Rules
      • AI Classification
    • User Settings
    • Stats
      • Data Views
    • Global Settings
      • General
      • Authentication
      • App Settings
      • Application Access
      • Thumbnails
      • Presets
      • Code of Conduct
      • Metrics
      • Viewer
      • SAR
        • Importing Data For A SAR
        • SAR Disclosure Document Storage
        • Getting SAR data from Google Vault
        • SAR Access
        • SAR File Status
      • Disclosure Portal
        • Disclosure Portal Set Up
        • SARs From The Portal
        • Email Delivery Settings
          • Delivery Settings
          • Brand Settings
          • Customise Emails
        • SMS Delivery Settings
        • Requestor Message Limit
        • Attachment Configuration
        • Password Configuration
        • File Scanner Configurator
      • Collections
      • Visualisations
        • Related Result Connections Diagram
        • Event Timeline
        • Timeline Lens Activity Chart
        • Relationship Map
      • Notifications
      • Map Lens
      • Theming
      • User Avatar
      • OData API
      • Uploads
      • Security
    • Search Settings
      • Search Relevancy
        • Core Settings
        • Makers Algorithm
        • Filename Boost Layer
        • Minimum Matching Terms Filter
        • Field Boost
        • Modified Date Boosting
        • Hit Highlighting
        • Why My Search Matched
        • Data Search Strategy
      • Bulk Search
        • Managing a Bulk Search
      • Search Flows
        • Create a Search Flow
          • General
          • Query Classification Step
          • Search Steps
          • Model Steps
      • Filtering
      • Search Performance
      • Related Results
  • AI Studio
    • Classifications
      • Classifications
      • Classification Rules
    • Jobs
  • Labels
  • API Guides
    • Insight API Guide
      • Swagger Documentation
      • Trying Some Endpoints
      • Search Filter
      • Hits / Items
      • Inspecting REST Calls
    • Data Science API Guide
      • REST Interface
        • Login
        • Datasets
        • Fields
        • Field Statistics
        • Search
        • Scroll
        • Update
      • Python Wrapper
        • Login
        • Datasets
        • Fields
        • Field Statistics
        • Search
        • Query Builders
        • Scroll
        • Scroll Search
        • Update Single Document
        • Bulk Update
    • Creating a Native Enrichment Step
      • Creating an Enrichment Step
        • Creating the Core Classes
        • Extending our Enrichment Step
        • Adding a Configuration Template
        • Adding the Enrichment Step
        • Creating an Enrichment Pipeline
      • Other Tasks
        • Entities, Metadata and Data
        • Accessing the Repository
      • Example Code
      • Troubleshooting
    • Creating a Python Enrichment Step
      • Creating an Enrichment Step
        • Running the Example from Command Line
        • Running the Example
      • Creating Your Own Step
      • Adding or Changing Entities, Metadata
  • Whitepapers and Explainers
    • From a Billion To One – Mastering Relevancy
    • Methods for Text Summarization
      • Application
      • Technology Methods
      • Commercial Tools
      • Key Research Centres
      • Productionisation
      • Related Areas of Text Analytics
      • Conclusion
      • References
Powered by GitBook
On this page
  • Vault Setup
  • Azure Key Vault
  1. Installation
  2. Aiimi Insight Engine Installation (Windows)

Certificates in a Key Vault

You can use a Key Vault to store and manage passwords (or secrets) and certificates. This reduces any risk associated with storing and sensitive information on a local filesystem. At the moment we only support Azure Key Vault, this means we are unable to support password protected certificates.

Credentials and certificates stored within the Control Hub Credentials are not impacted by this.

When loading certificates from an Azure key vault, they cannot be password protected as the password is removed by the vault. When loading a certificate from the filesystem, it can be password protected.

Vault Setup

Aiimi Insight Engine determines how to retrieve passwords and certificates based on the configurations in your appsettings.json files. If a key vault is configured, that is tried first. If that fails or is not configured it will revert to looking locally for passwords and certificates.

  1. Set up certificates - Within your Key Vault you need to set up your certificates, secrets and passwords.

    • We recommend adding a year to the certificate names. This can help with certificate management and switching certificates.

  2. Vault Access - You need to grant access to the vault, this varies depending on the vault plugin.

  3. Update appsettings.json - Add the Key Vault details to the root of each appsetting.json file.

    • This can be done manually or via a bulk update.

Manual appsettings.json Update

We recommend starting with the index utils file. This way you can confirm the settings are correct before changing all of the others.

  1. Add the below settings to every appsettings.json file:

    • Replace the values between the <> for your chosen key vault.

{
  "keyVault": {
    "type": "<AzureKeyVault>",
    "enableTracing": false,
    "azureKeyVault": {
      "vaultUri": "https://<vaultId>.vault.azure.net/"
    }
  }
}
Bulk appsettings.json Update

You can add the keyVault settings to every file using a JSON file and PowerShell. We recommend running a back up before completing this incase something goes wrong.

  1. Create a JSON file containing the below information:

    • Replace the values between the <> for your chosen key vault.

{
  "keyVault": {
    "type": "<AzureKeyVault>",
    "enableTracing": false,
    "azureKeyVault": {
      "vaultUri": "https://<vaultId>.vault.azure.net/"
    }
  }
}
  1. Save this a .json file.

  2. Open PowerShell as an admin.

  3. Run: im-settings add --installation-folder <F:\InsightMaker\> <C:\tmp\akv.json>

    • Replace the values between the <> to match your file paths. The first should be where your insight maker files are stored. The second is the path for the json file you just created.

During set up you can set "enableTracing" to true to help debugging.

This could log sensitive information and must be set to false once it is working correctly.

  1. Replace any certificate and password values with the certificate or secret name in the vault.

    • Certificate passwords are handled by the vault and should be set to "".

Annotated Example
  1. The certificate used to validate connections to the Elastic server. It is retrieved from the vault by downloading the certificate elastic-stack-ca.

  2. The certificate password is not needed as it's managed by the vault.

  3. The elastic user password is retrieved from the vault by downloading the secret elasticPassword.

  4. The certificate to encrypt HTTPS connections to the API endpoint is retrieved from the vault by downloading the certificate elastic-certificates.

  5. The system secret is retrieved from the vault by downloading the secret systemSecret.

  6. The vault configuration section. This uses the AzureKeyVault plugin to access a vault at https://pandora.vault.azure.net/.

{
  "elastic": {
    "certificate": {
      "path": "elastic-stack-ca", 1️⃣
      "password": "" 2️⃣
    },
    "password": "elasticPassword", 3️⃣
    "prefix": "dev",
    "server": [
      "https://im.aiimi.com:9200"
    ],
    "username": "elastic",
    "enableTracing": false
  },
  "plugins": {
    "locations": [
      "c:\\InsightMaker\\Plugins"
    ]
  },
  "remoteApi": {
    "AllowedOrigins": [
      "*"
    ],
    "BindAddresses": [
      "0.0.0.0"
    ],
    "Port": 2221,
    "RemoteAddress": "https://im.aiimi.com",
    "certificate": {
      "path": "elastic-certificates", 4️⃣
      "password": "" 2️⃣
    }
  },
  "systemSecret": "systemSecret", 5️⃣
  "licenseKey": "",
  "licenseSig": "",
  "keyVault": { 6️⃣
    "type": "AzureKeyVault", 
    "azureKeyVault": {
      "vaultUri": "https://pandora.vault.azure.net/"
    }
  }
}

Azure Key Vault

This plugin allows Aiimi Insight Engine to retrieve certificates and passwords from Azure Key Vault. It accesses it via a URI provided in the config, and supports DefaultAzureCredential. This allows you to control access via secrets or passwords stored in environment variables, managed identities (if running in an Azure environment), Azure CLI/PowerShell or interactive.

This configuration requires the access to be encompassed by DefaultAzure Credentials. However, You should follow your existing procedures when configuring access to the vault.

There is no authentication section to grant access to the vault, this must be setup via environment variables, managed identities, something compatible with DefaultAzureCredentials.

PreviousInstallation SecurityNextSAR Configuration