Aiimi Insight Engine Installation (Windows)

This section presents a guide for setting up a simple single node Aiimi Insight Engine instance on Windows 2016 or later.

If you are designing and installing a production environment, then you will be installing individual agents and components on specific nodes. Further guidance on this can be provided on request.

Prerequisites

• Notepad++ (or your text editor of choice)

• Latest Java Development Kit and Java Runtime Environment - you can use something from https://adoptopenjdk.net if you wish. If you are installing Elastic on the same machine, use the one in the Elastic install folder.

• Search on Windows for 'Turn Windows Features On/Off' and enable Internet Information Services (IIS) in that dialog (used for managing websites and apis).

• .NET SDK (v8.0)

• .NET Core and Hosting bundle (.NET installs used to run InsightMaker agents and middleware as well as host web components) (v8.0)

Downloads

• Download NSSM to run the Tika Agent as a service

• Obtain your Aiimi Insight Engine distribution (Reach out to your contact at Aiimi)

• Obtain the Aiimi Aiimi Insight Engine Tika distribution

• Obtain your AIE licence from Aiimi

Download and Extract

The following guide will assume everything is being stored in the C: drive. You can use any drive but we recommend installing to the root of a drive.

1. Once you have your Aiimi Insight Engine distribution extract it to C:\InsightMaker.

2. Run the following scripts in an an administrator PowerShell window. These commands stop Windows silently blocking files.

   • gci c:\InsightMaker\Plugins\*.dll | Unblock-File

   • gci c:\InsightMaker\Scripts\*.ps1 | Unblock-File

3. Run the following Script in an admin PowerShell to update the Agent file names.

$Filename = "C:\InsightMaker\ContentAgent";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

$Filename = "C:\InsightMaker\EnrichmentAgent";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

$Filename = "C:\InsightMaker\JobAgent";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

$Filename = "C:\InsightMaker\MigrationAgent";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

$Filename = "C:\InsightMaker\OCRAgent";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

$Filename = "C:\InsightMaker\SecurityAgent";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

$Filename = "C:\InsightMaker\SourceAgent";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

4. Run the following script in an admin PowerShell to upate the app file names.

$Filename = "C:\InsightMaker\Apps\Admin\Api";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;
Copy-Item -Path "$filename\web.default.config" -Destination "$filename\web.config" -Force

$Filename = "C:\InsightMaker\Apps\Admin\app";
Copy-Item -Path "$filename\web.default.config" -Destination "$filename\web.config" -Force;

$Filename = "C:\InsightMaker\Apps\search\api";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;
Copy-Item -Path "$filename\web.default.config" -Destination "$filename\web.config" -Force;

$Filename = "C:\InsightMaker\Apps\web\api";
Copy-Item -Path "$filename\web.default.config" -Destination "$filename\web.config" -Force;

$Filename = "C:\InsightMaker\Utils\InsightMaker.IndexUtilities";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

$Filename = "C:\InsightMaker\Utils\InsightMaker.Security.BuiltinSecurityUtilities";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

Aiimi Insight Engine Configurations

Some values need to be encrypted to ensure security. .\InsightMaker.SettingsUtility encryptstring <value>

1. Open appsettings.default.json in Notepad++.

2. Update the Plugins path points to the plugins folder (likely C:\InsightMaker\Plugins).

3. Run the following script in an admin PowerShell to update your app settings Json file.

   1. Update where the script below says "Text input" with the relevant information.

   2. Ensure the correct Elastic password is set.

      • This password must be encrypted.

   3. The Elasticsearch credentials are set to the elastic login previously created.

      • The password must be encrypted.

   4. Your license signature should be a unique value for this system.

   5. Your license secret must be encrypted.

   6. The jwt configuration key must be encrypted and at least 32 characters.

.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "elastic.certificate.path" -v "C:\Apps\certs\elastic-stack-ca.p12";
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "elastic.certificate.password" -v "Text input";
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "elastic.password" -v "Text input";
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "elastic.prefix" -v "Text input";
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "plugins.locations" -l "C:\InsightMaker\Plugins"
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "remoteApi.certificate.path" -v "C:\Apps\certs\elastic-certificates.p12";
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "remoteApi.certificate.password" -v "Text input";
.\InsightMaker.SettingsUtilities.exe patch -c -i C:\InsightMaker\ -a "licenseKey" -v "Text input";
.\InsightMaker.SettingsUtilities.exe patch -c -i C:\InsightMaker\ -a "licenseSig" -v "Text input";

Configure Log File Storage

Each agent has a log4net.default.config file. This file defines the path for the log directoy. This is C:\tmp\logs by default.

1. Run the following script to update the logs into their own subfolders.

$Filename = "C:\InsightMaker\ContentAgent\log4net.config";
((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.ContentAgent.log','C:/tmp/logs/agents/InsightMaker.ContentAgent.log') | Set-Content -Path $filename;
$Filename = "C:\InsightMaker\EnrichmentAgent\log4net.config";((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.EnrichmentAgent.log','C:/tmp/logs/agents/InsightMaker.EnrichmentAgent.log') | Set-Content -Path $filename;
$Filename = "C:\InsightMaker\JobAgent\log4net.config";
((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.JobAgent.log','C:/tmp/logs/agents/InsightMaker.JobAgent.log') | Set-Content -Path $filename;
$Filename = "C:\InsightMaker\MigrationAgent\log4net.config";
((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.MigrationAgent.log','C:/tmp/logs/agents/InsightMaker.MigrationAgent.log') | Set-Content -Path $filename;
$Filename = "C:\InsightMaker\OCRAgent\log4net.config";
((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.OCRAgent.log','C:/tmp/logs/agents/InsightMaker.OCRAgent.log') | Set-Content -Path $filename
$Filename = "C:\InsightMaker\SecurityAgent\log4net.config";
((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.SecurityAgent.log','C:/tmp/logs/agents/InsightMaker.SecurityAgent.log') | Set-Content -Path $filename;

2. In Utils\InsightMaker.IndexUtilities, run the following from an admin command prompt. These commands configure the default elastic mappings for InsightMaker indices and create some useful default entities etc.

• .\InsightMaker.IndexUtilities.exe map

• .\InsightMaker.IndexUtilities.exe initialise -fmsci

• .\InsightMaker.IndexUtilities.exe initialise --collection-types;

• .\InsightMaker.IndexUtilities.exe initialise --theme;

• .\InsightMaker.IndexUtilities.exe upgrade --generate-entity-groups;

Install Services

1. Run the following script in an Admin PowerShall to update the logs into their own subfolders.

sc.exe create InsightMakerContentAgent binPath= c:\InsightMaker\ContentAgent\InsightMaker.ContentAgent.exe DisplayName= "InsightMaker Content Agent" start= auto
sc.exe create InsightMakerEnrichmentAgent binPath= c:\InsightMaker\EnrichmentAgent\InsightMaker.EnrichmentAgent.exe DisplayName= "InsightMaker Enrichment Agent" start= auto
sc.exe create InsightMakerJobAgent binPath= c:\InsightMaker\JobAgent\InsightMaker.JobAgent.exe DisplayName= "InsightMaker Job Agent" start= auto
sc.exe create InsightMakerMigrationAgent binPath= c:\InsightMaker\MigrationAgent\InsightMaker.MigrationAgent.exe DisplayName= "InsightMaker Migration Agent" start= auto
sc.exe create InsightMakerOcrAgent binPath= c:\InsightMaker\OcrAgent\InsightMaker.OcrAgent.exe DisplayName= "InsightMaker Ocr Agent" start= auto
sc.exe create InsightMakerSecurityAgent binPath= c:\InsightMaker\SecurityAgent\InsightMaker.SecurityAgent.exe DisplayName= "InsightMaker Security Agent" start= auto
sc.exe create InsightMakerSourceAgent binPath= c:\InsightMaker\SourceAgent\InsightMaker.SourceAgent.exe DisplayName= "InsightMaker Source Agent" start= auto

sc.exe description InsightMakerContentAgent "InsightMaker Content Agent"
sc.exe description InsightMakerEnrichmentAgent "InsightMaker Enrichment Agent"
sc.exe description InsightMakerJobAgent "InsightMaker Job Agent"
sc.exe description InsightMakerMigrationAgent "InsightMaker Migration Agent"
sc.exe description InsightMakerOcrAgent "InsightMaker Ocr Agent"
sc.exe description InsightMakerSecurityAgent "InsightMaker Security Agent"
sc.exe description InsightMakerSourceAgent "InsightMaker Source Agent"

Configuring IIS

Initially: Ensure that '.NET Core and HostingBundle' has been installed, otherwise middleware will not be able to run in IIS.

1. Ensure the account that IIS runs under has read access to your Elasticsearch certificates.

2. Open Internet Information Services (IIS).

3. Open dropdowns in the left window and select Default Website.

4. On the right hand side, open 'Bindings' and add a new binding.

5. If you want to use SSL (production platform for example):

   1. Set type to https and use the default fields (port 443)

   2. Set SSL certificate

6. Open basic settings on the right and set the physical path to the 'app' folder within the Search directory.

7. Right click Default Website and add a new application and call this alias 'api' and set the physical path to the 'api' folder within the Search directory.

8. Right click Default Website and add a new application and call this alias 'admin' and add the path to the 'app' folder of the Admin directory.

9. Right click 'admin' and add one more website and call this alias 'api' and point it to the api folder in the Admin directory.

This sets up the default website on the server and can be accessed using 'localhost' in the navigation bar in any browser, and the other apps using their alias (for example admin is accessed using localhost/admin).

If you only see a blank screen it's possible that IIS didn't set itself up correctly, first try restarting IIS and, if that doesn't resolve the issue, follow the steps at the start of the guide to disable and re-enable IIS. Ensure that most of the 'Common Http Features' section gets enabled inside IIS.

Installing Odata API (optional)

1. Navigate to C:/Insightmaker/Apps/OData.

2. Open appsettings.default.json in a source code editor like NotePad++.

3. Check the details are correct:

   • The Plugins path points to the plugins folder. Likely C:\InsightMaker\Plugins

   • The elastic certificate path points to the elastic-stack-ca.p12 file. This was created when you installed Elasticsearch.

   • The correct password is set.

   • The Elasticsearch credentials are set to the elastic login you created.

   • The prefix is set to the correct value and consistent across all components.

4. Add your two Aiimi Insight Engine licences to the root of the json:

   • "licenseKey": "your key"

   • "licenseSig": "your sig"

5. Check the remoteAddress is set to ‘http://localhost/analytics/‘.

6. Save this file, do not change the name or location.

7. Within C:/Insightmaker/Apps/OData open log4net.default.config.

8. Change the file paths to where you’d like log files to save.

   • This can not be the in InsightMaker folder.

9. Save this file, do not change the name or location.

10. Remove the '.default' from each of the following file names:

    • 'appsettings.default.json'

    • 'log4net.default.config'

    • 'web.default.config'

11. Open the web.config file.

    • Check the module is modules="AspNetCoreModuleV2".

12. Open Internet Information Services (IIS)

13. Open the dropdown on the left and Right click on default website.

14. Click add new application, call the alias “analytics”.

15. Set the physical path to the ‘API’ folder in InsightMaker/Apps/OData.

    • You should be able to view the OData metadata document using localhost/analytics/odata/$metadata.

Installing Tika

1. Unzip the Tika distribution.

2. Create a directory to store IM utilities, outside of the InsightMaker directory to avoid adding any complication to upgrades (eg. C:\Utils).

3. Move tika.jar and run-tika.bat to this folder.

4. Modify run-tika.bat to include the correct path to tika.

   • If your Elasticsearch is stored to a folder with spaces in it, then the Java home path will need to be encased in quotation marks)

5. Run nssm install from an admin command prompt.

6. Point the path field to run-tika.bat and set up service/display names.

If you want to run Tika without installing then from a command prompt navigate to the folder where you have placed Tika, then run .\run-tika.bat serve. This will allow you to identify any issues which are less obvious if you just go straight for the install (Ctrl + C to close down once tested).

7. By default Tika will listen on post 9998.

8. Access Tika in your web browser in http://localhost:9998 to test your Tika service.

Accessing Control Hub

1. Navigate to http://localhost/admin and login with your elastic credentials (elastic:changeme by default).

2. Assuming you can login correctly, then you should see your agents on the configuration page. They will all be empty. To get started, please see the Quick Start Guide in the Administration Guide.