Aiimi Insight Engine Ghost
User GuidesAiimi
  • Introducing Aiimi Insight Engine
  • Architecture
    • Overview and Key Concepts
    • Search Flows
      • Search Flow Types
      • Smart Filtering
      • Query and Prompt Classification
      • Search Algorithms
      • Extractive and Generative Models
    • Hosting Options
    • Architecture and How It Works
      • Agent Servers
        • Security Agent
        • Source Agent
        • Content Agent
        • Enrichment Agent
        • Job Agent
        • OCR Agent
        • Migration Agent
        • Tika Agent
      • Repository
        • Data Node
        • Proxy Node
        • Kibana Node
      • Gateway and User Interface
      • Document and Data Sources
    • Deployment Options
    • Security
      • User Security
      • Data and Document Security
        • Progressive Access
        • Privileged Access
      • Source System Security
      • Firewalling
      • Agent Servers
        • Security Agent
        • Source Agent
        • Content Agent
        • Enrichment Agent
        • Job Agent
        • OCR Agent
        • Migration Agent
        • Tika Agent
      • Repository
      • Gateway (Web Server)
      • Tools & Utilities
  • Installation
    • Elastic and Kibana Install (Windows)
    • Aiimi Insight Engine Installation (Windows)
      • Installation Security
      • SAR Configuration
      • CSOM Bridge Set Up
    • AI Services
      • Prerequisites
      • AI Enrichment Service
        • Installation and Setup
        • Enabling Enrichment Steps
        • Using AI Enrichment Steps
        • Performance and Concurrency
      • AI Model Service
        • Installation and Setup
        • Enabling Providers
        • Private Generative AI
        • Azure Open AI
        • HTML Cleaner Service
      • Configuration of Logging
      • Offline Set-up of Models
      • Using SSL
      • Running as a Service (Windows)
      • Using GPUs
      • AI and Semantic Search Set Up
        • Open & Closed Book AI
        • Semantic Search
          • Vectors for Semantic Search
          • Source Configuration
          • Sentence Transformer Models
          • Enrichment
          • Kibana
          • Final Search Flow
  • Control Hub
    • Configurations
      • Config Management
      • Security Configurations
        • Security - General
        • Security - Source
          • Active Directory
          • Atlassian
          • Azure Active Directory
          • Builtin Security
          • Miro Security
          • Google Directory
          • Slack Security
        • Security - Sync
        • Security - Agents
        • Security - Scheduling
      • Source Configurations
        • Source - General
        • Source - Source
          • Alfresco Kafka
          • Azure Blob Storage
          • BBC Monitoring
          • Big Query Cataloguer
          • BIM360
          • CSV Data Loader
          • Confluence
          • Content Server
          • Data File Cataloguer
          • Document Store
          • DocuSign
          • Dropbox
          • Exchange 365
          • Filesystem
          • Google Bucket
          • Google Drive
          • Google Vault
          • Jira
          • JSON Data Loader
          • Livelink
          • Microsoft Teams
          • Mimecast
          • Miro
          • ODBC Data Loader
          • PowerBi Cataloguer
          • Reuters Connect
          • ShareFile
          • SharePoint
          • SharePoint Legacy
          • SQL Server Cataloguer
          • Slack
          • Versioned Document Store
          • Websites
          • XML Data Loader
        • Source - Crawl
        • Source - Agents
        • Source - Schedule
        • Source - Advanced
      • Enrichment Configurations
        • Creating a Pipeline
          • General
          • Steps
            • AccessMiner
            • AI Classification
            • Anonymiser
            • CAD Extractor
            • Checksum
            • Content Retrieval
            • Copy
            • Data Rule Processor
            • Delete
            • Email Extractor
            • Entity Rule Processor
            • External Links
            • Geotag
            • Google NLP Extractor
            • Google Vision Extractor
            • Metrics Calculation
            • Microsoft Vision Extractor
            • OcrRest
            • Office Metadata
            • PCI Extractor
            • REST
            • Set Document Risk
            • Text Cleaner
            • Tika Text Extraction
            • Trie Entity Extractor
            • Update Metadata
          • Filters
          • Agents
          • Schedule
          • Advanced
      • OCR Engine
      • Job Configurations
        • General
        • Job
          • AutomatedSearchJob
          • Command Job
          • ElasticJob
          • Extended Metrics Job
          • GoogleVaultSAR
          • Google Drive Last Access Date
          • Nightly Events Processor Job
          • Notifications Processor Job
          • Portal Sync Job
          • Purge Job
          • Text Content Merge Job
        • Output
        • Agents
        • Scheduling
      • Migration Configuration
        • General
        • Filter
        • Metadata Mappings
        • Agents
        • Scheduling
        • Advanced
    • Credentials
      • Create a Credential
      • Find a Credential
      • Edit a Credential
      • Delete a Credential
    • Mappings
      • Entities
        • Managing Groups
        • Create an Entity
        • Managing Entities
      • Models
        • Create a New Model
        • Find a Model
        • Enable or Disable a Model
      • Vectors
      • Rank Features
    • Featured Links
    • AI Settings
      • Classifications
      • Class
      • Class Rules
      • AI Classification
    • User Settings
    • Stats
      • Data Views
    • Global Settings
      • General
        • Stackdriver
        • Document Recommendations
        • Searchable PDF Storage
        • Versioning
        • Results
        • Marking Useful Results
        • Folder Browsing
        • Cascading Search
        • Search Suggestions
        • Miscellaneous
      • Authentication
      • App Settings
      • Application Access
      • Thumbnails
      • Presets
      • Code of Conduct
      • Metrics
      • Viewer
        • Redacting Information
        • Watermarking
      • SAR
        • Importing Data For A SAR
        • SAR Disclosure Document Storage
        • Getting SAR data from Google Vault
        • SAR Access
        • SAR File Status
      • Disclosure Portal
        • Disclosure Portal Set Up
        • SARs From The Portal
        • Email Delivery Settings
          • Delivery Settings
          • Brand Settings
          • Customise Emails
        • SMS Delivery Settings
        • Requestor Message Limit
        • Attachment Configuration
        • Password Configuration
        • File Scanner Configurator
      • Collections
      • Visualisations
        • Related Result Connections Diagram
        • Event Timeline
        • Timeline Lens Activity Chart
        • Relationship Map
      • Notifications
      • Map Lens
      • Theming
        • General
        • Layout
        • Colours
      • User Avatar
      • OData API
      • Uploads
      • Security
    • Search Settings
      • Search Relevancy
        • Core Settings
        • Makers Algorithm
        • Filename Boost Layer
        • Minimum Matching Terms Filter
        • Field Boost
        • Modified Date Boosting
        • Hit Highlighting
        • Why My Search Matched
        • Data Search Strategy
      • Bulk Search
        • Managing a Bulk Search
      • Search Flows
        • Create a Search Flow
          • General
          • Query Classification Step
          • Search Steps
          • Model Steps
      • Filtering
      • Search Performance
      • Related Results
  • API Guides
    • Insight API Guide
      • Swagger Documentation
      • Trying Some Endpoints
      • Search Filter
      • Hits / Items
      • Inspecting REST Calls
    • Data Science API Guide
      • REST Interface
        • Login
        • Datasets
        • Fields
        • Field Statistics
        • Search
        • Scroll
        • Update
      • Python Wrapper
        • Login
        • Datasets
        • Fields
        • Field Statistics
        • Search
        • Query Builders
        • Scroll
        • Scroll Search
        • Update Single Document
        • Bulk Update
    • Creating a Native Enrichment Step
      • Creating an Enrichment Step
        • Creating the Core Classes
        • Extending our Enrichment Step
        • Adding a Configuration Template
        • Adding the Enrichment Step
        • Creating an Enrichment Pipeline
      • Other Tasks
        • Entities, Metadata and Data
        • Accessing the Repository
      • Example Code
      • Troubleshooting
    • Creating a Python Enrichment Step
      • Creating an Enrichment Step
        • Running the Example from Command Line
        • Running the Example
      • Creating Your Own Step
      • Adding or Changing Entities, Metadata
  • whitepapers and explainers
    • From a Billion To One – Mastering Relevancy
    • Methods for Text Summarization
      • Application
      • Technology Methods
      • Commercial Tools
      • Key Research Centres
      • Productionisation
      • Related Areas of Text Analytics
      • Conclusion
      • References
Powered by GitBook
On this page
  • Extract Disclosure Portal Zips
  • Initialise SAR Collection
  • Privacy Portal Certificate
  • Configuring IIS
  • Generate root and child certificates
  • Delete Configuration Files
  • Log4net.config
  • Web.config
  • Appsettings.json
  • Installing .NET Hosting Bundle
  • Configure Insight Engine Agents and APIs
  1. Installation
  2. Aiimi Insight Engine Installation (Windows)

SAR Configuration

To have SARs on your Aiimi Insight Engine system there are a few things that need to be set up.

Extract Disclosure Portal Zips

  1. Extract the ‘disclosure-portal.zip’ folder to C:\

    • The paths should be:

      • C:\DisclosurePortal\admin-api

      • C:\DisclosurePortal\client-api

      • C:\DisclosurePortal\client-app

Initialise SAR Collection

  1. Within the Control Hub go to Mappings, Models.

  2. Under the Collection Business Area check if the dsarCollection exists.

  3. Initialise it by running the following commands on the Aiimi Insight Engine server:

cd C:\InsightMaker\Utils\InsightMaker.IndexUtilities\ 

.\InsightMaker.IndexUtilities.exe initialise --dsar --publish 

This will create the model, publish it, and populate default values for objects such as Subject Types, Universal Policies etc.

Privacy Portal Certificate

Configuring IIS

  1. Open the Internet Information Services (IIS).

  2. Right click Sites then Add Website.

    1. Site name: PrivacyPortal (or similar, useful name)

    2. Physical path: C:\DisclosurePortal\client-app

    3. Binding:

      • Type: https (not http)

      • IP address: All Unassigned

      • Port: 444 (Usually on Port 443 unless on the same server)

      • SSL certificate: InsightEngine20

  3. Click OK

  4. Right Click on PrivacyPortal then Add Application.

    1. Alias: api

    2. Physical path: C:\DisclosurePortal\client-api

  5. Click OK

  6. Click Admin

  7. Under Management select Configuration Editor

  8. Section: Select system.webServer > serverRuntime

  9. uploadReadAheadSize: Change to 2147483647

  10. Select apply

  11. Select admin again

  12. Under the IIS Section select SSL Settings

  13. Enable Require SSL and select Require under client certificates.

Generate root and child certificates

If deploying on a separate web server to Aiimi Insight Engine, copy the folder ‘C:\InsightMaker\scripts’ to the server.

If deploying on the same server as AIE, simply use the existing ‘C:\InsightMaker\scripts’ folder.

  1. Create folder ‘certs’ under C:\DisclosurePortal

  2. Open PowerShell as an Administrator.

  3. Run the following command

C:\InsightMaker\scripts\create-root-child-certs.ps1 
  1. Confirm the certificate files exist under C:\DisclosurePortal\certs\

  2. Right click root.crt and select Install Certificate

  3. Select Local Machine then Next.

  4. Select 'Place all certificates in the following store'.

  5. Click Browse and select ‘Trusted Root Certification Authorities’.

  6. Click OK, then Next and Finish.

Deploying on a separate web server to Aiimi Insight Engine

  1. Copy the Elastic cert ‘elastic-certificiates.p12’ file to the server under C:\DisclosurePortal\certs\

    • e.g. C:\Apps\elasticsearch-8.11.1\config\certs\elastic-certificates.p12

  2. Copy the cert file ‘child.pfx’ to the Aiimi Insight Engine server (or AIE web app AND agents servers, if multi-server deployment)

Deploying on the same server as Aiimi Insight Engine

  1. Use the existing ‘elastic-certificates.p12’ path and password

Delete Configuration Files

Log4net.config

  1. Rename the log4net.default.config files within \client-api and \admin-api to log4net.config.

  2. Open \client-api\log4net.config in Notepad++ (or similar application).

  3. Update the file value on line 4 to a valid location.

    • e.g. C:/tmp/logs/DisclosurePortal/ (the log name in the file)

  4. Save and close

  5. Repeat steps 1 to 4 for the ‘\admin-api\log4net.config’ file.

Web.config

  1. Rename the web.default.config files within \client-api and \admin-api to web.config.

Appsettings.json

If you are deploying on a separate web server to Aiimi Insight Engine, copy the folder ‘C:\InsightMaker\Plugins’ to the server.

  1. Rename the appsettings.default.json files within \client-api and \admin-api to appsettings.json.

  2. Open \client-api\appsettings.json in Notepad++ (or similar application).

    1. Set the plugins.locations to C:\\InsightMaker\\Plugins (line 10).

    2. Set the disclosureSettingsPath to C:\\DisclosurePortal\\disclosuresettings.json.

    3. Set the disclosureRoot to C:\\tmp\\disclosures.

  3. Create an empty file called disclosuresettings.json and save this within C:\DisclosurePortal\.

  4. Create a folder path of C:\tmp\disclosures\.

  5. Save and close.

  6. Create a folder path of C:\tmp\RequestStore\.

  7. Create a file called verificationStore.json and save this within C:\tmp\.

    1. Set the contents of this file to:

{ 

  "Requests": {} 

} 
  1. Save and close.

  2. Open ‘\admin-api\appsettings.json’ in Notepad

    1. Set the disclosureSettingsPath to ‘C:\\DisclosurePortal\\disclosuresettings.json’.

    2. Set the disclosureRoot to ‘C:\\tmp\\disclosures’.

    3. Set the plugins.locations to ‘C:\\InsightMaker\\Plugins’ (line 6).

    4. Set the remoteApi.certificate.path to ‘C:\\DisclosurePortal\\certs\\elastic-certificates.p12’

      • (or the existing path of ‘elastic-certificates.p12’ if deploying on the same server as Aiimi Insight Engine)

    5. Set the remoteApi.certificate.password value to the password of ‘elastic-certificates.p12’

  3. Save and close

Installing .NET Hosting Bundle

Deploying on a separate web server to Aiimi Insight Engine

  1. Run ‘dotnet-hosting-8.0.xx-win.exe’

    • It's important that the Hosting Bundle is installed after IIS has been installed and enabled.

  2. Perform an iisreset.

    1. The login screen should be displayed.

Configure Insight Engine Agents and APIs

  1. For the Admin and Search API’s and Content and Job Agents, open ‘appsettings.json’ in Notepad.

    • Admin API (e.g. C:\InsightMaker\Apps\Admin\api\appsettings.json)

    • Search API (e.g. C:\InsightMaker\Apps\Search\api\appsettings.json)

    • ContentAgent (e.g. C:\InsightMaker\ContentAgent\appsettings.json)

    • JobAgent (e.g. C:\InsightMaker\JobAgent\appsettings.json)

  2. Add/update the advanced.disclosureClientCertificate section with the correct path to ‘child.pfx’ and its password:

"advanced": { 
    "redactionLicencedKey": "See Agreement", 
    "disclosureClientCertificate": { 
      "path": "C:\\DisclosurePortal\\certs\\child.pfx", 
      "password": "changeme" 
    } 
  } 
  1. Restart IIS where Search and Admin are deployed, and restart the Job and Content agents.

PreviousInstallation SecurityNextCSOM Bridge Set Up

Last updated 5 months ago

Download .NET Hosting Bundle 8.x from (actual version number may vary from screenshot)

Browse to

https://dotnet.microsoft.com/en-us/download/dotnet/8.0
https://localhost:444/#/login