Aiimi Insight Engine

CtrlK

User Guides Aiimi

Aiimi Insight Engine Installation (Windows)

This guide walks you through setting up a single node Aiimi Insight Engine instance. It is for Windows 2016 or later.

For a production environment you must install agents and components on specific nodes. Reach out to your Aiimi contact for further guidance on this.

Prerequisites

You need your Aiimi Insight Engine licence from Aiimi.
.NET SDK (v8.0) is downloaded and installed.
.NET Core and Hosting Bundle (v9.0) is downloaded and Installed.
- This runs the Aiimi Insight Engine agents and middleware and hosts web components.

If you installed Elasticsearch and Kibana you may already have the following downloads.

Notepad++ or similar text editing software installed.
NSSM is downloaded and installed to run the Tika Agent as a service.
Your Aiimi Insight Engine distribution is downloaded. Your Aiimi contact can help you with this if you don't have it.
The Tika file in the Aiimi Insight Engine distribution is downloaded.

Download Software

The following guide assumes everything is being stored in the C: drive. You can use any drive but we recommend installing to the root.

1

Check if your Aiimi Insight Engine distribution is extracted into C:\InsightMaker.

2

If it's not, open PowerShell as an Admin.

3

Run the following script to extract the zip file to C:\InsightMaker.

Check the storage path in this script before running it. For example: (C:\).

Expand-Archive -Force C:\Downloads\insightmaker-win*.zip C:\InsightMaker

4

Run the following script to download the redistributables and dotnet bundle.

Check the storage path in this script before running it. For example: (C:\).

Start-BitsTransfer -Source "https://aka.ms/vs/17/release/vc_redist.x86.exe" -destination "C:\Downloads";
Start-BitsTransfer -Source "https://aka.ms/vs/17/release/vc_redist.x64.exe" -destination "C:\Downloads";
Start-BitsTransfer -Source "https://builds.dotnet.microsoft.com/dotnet/aspnetcore/Runtime/8.0.16/dotnet-hosting-8.0.16-win.exe" -destination "C:\Downloads"

Prepare Files

1

Run the following script to stop Windows silently blocking files.

Check the storage path in this script before running it. For example: (C:\).

gci C:\InsightMaker\Plugins\*.dll | Unblock-File;
gci C:\InsightMaker\Scripts\*.ps1 | Unblock-File

2

Run the following script to update the Agent config file names.

Check the storage path in this script before running it. For example: (C:\).

$Filename = "C:\InsightMaker\ContentAgent";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

$Filename = "C:\InsightMaker\EnrichmentAgent";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

$Filename = "C:\InsightMaker\JobAgent";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

$Filename = "C:\InsightMaker\MigrationAgent";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

$Filename = "C:\InsightMaker\OCRAgent";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

$Filename = "C:\InsightMaker\SecurityAgent";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

$Filename = "C:\InsightMaker\SourceAgent";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

3

Run the following script to update the Apps and Utils config file names.

Check the storage path in this script before running it. For example: (C:\).

$Filename = "C:\InsightMaker\Apps\Admin\Api";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;
Copy-Item -Path "$filename\web.default.config" -Destination "$filename\web.config" -Force

$Filename = "C:\InsightMaker\Apps\Admin\app";
Copy-Item -Path "$filename\web.default.config" -Destination "$filename\web.config" -Force;

$Filename = "C:\InsightMaker\Apps\search\api";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;
Copy-Item -Path "$filename\web.default.config" -Destination "$filename\web.config" -Force;

$Filename = "C:\InsightMaker\Utils\InsightMaker.IndexUtilities";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

$Filename = "C:\InsightMaker\Utils\InsightMaker.Security.BuiltinSecurityUtilities";
Copy-Item -Path "$filename\appsettings.default.json" -Destination "$filename\appsettings.json" -Force;
Copy-Item -Path "$filename\log4net.default.config" -Destination "$filename\log4net.config" -Force;

Aiimi Insight Configurations

1

Run the following script to update your AppSettings.json files.

Check the storage path and update it where it says "Text input" with the relevant information. Ensure the following information is input correctly:

Elastic Certificate Password, Elastic password, Elastic Prefix, RemoteApi Certificate Password, License Key and License Signature.
- The license signature should be a unique value for this system.

cd C:\InsightMaker\Utils\InsightMaker.SettingsUtilities;
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "elastic.certificate.path" -v "C:\Apps\certs\elastic-stack-ca.p12";
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "elastic.certificate.password" -v "Text input";
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "elastic.password" -v "Text input";
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "elastic.prefix" -v "Text input";
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "elastic.server" -l "https://localhost:9200";
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "plugins.locations" -l "C:\InsightMaker\Plugins"
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "remoteApi.certificate.path" -v "C:\Apps\certs\elastic-certificates.p12";
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "remoteApi.RemoteAddress" -v "https://localhost";
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\Apps\Admin\api -a "remoteApi.RemoteAddress" -v "https://localhost/admin/api";
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\Apps\Search\api -a "remoteApi.RemoteAddress" -v "https://localhost/api";
.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "remoteApi.certificate.password" -v "Text input";
.\InsightMaker.SettingsUtilities.exe patch -c -i C:\InsightMaker\ -a "licenseKey" -v "Text input";
.\InsightMaker.SettingsUtilities.exe patch -c -i C:\InsightMaker\ -a "licenseSig" -v "Text input";

Json Web Token Config

1

Run the following script to update your JWT config key.

Check the storage path and update it where it says "Text input" with the relevant information.

The JWT configuration key must be at least 32 characters.
- For help creating a random 32 character key see https://www.random.org.

.\InsightMaker.SettingsUtilities.exe patch -i C:\InsightMaker\ -a "jwtConfiguration.Key" -v "Text input";

Configure Log File Storage

Each agent has a log4net.default.config file. This file defines the path for the log directory and is C:\tmp\logs by default.

1

Run the following script to update the logs into their own subfolders by

Check the storage path in this script before running it. For example: (C:). The logs cannot be output to the InsightMaker folder.

$Filename = "C:\InsightMaker\ContentAgent\log4net.config";
((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.ContentAgent.log','C:/tmp/logs/agents/InsightMaker.ContentAgent.log') | Set-Content -Path $filename;

$Filename = "C:\InsightMaker\EnrichmentAgent\log4net.config";((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.EnrichmentAgent.log','C:/tmp/logs/agents/InsightMaker.EnrichmentAgent.log') | Set-Content -Path $filename;

$Filename = "C:\InsightMaker\JobAgent\log4net.config";
((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.JobAgent.log','C:/tmp/logs/agents/InsightMaker.JobAgent.log') | Set-Content -Path $filename;

$Filename = "C:\InsightMaker\MigrationAgent\log4net.config";
((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.MigrationAgent.log','C:/tmp/logs/agents/InsightMaker.MigrationAgent.log') | Set-Content -Path $filename;

$Filename = "C:\InsightMaker\OCRAgent\log4net.config";
((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.OCRAgent.log','C:/tmp/logs/agents/InsightMaker.OCRAgent.log') | Set-Content -Path $filename;

$Filename = "C:\InsightMaker\SecurityAgent\log4net.config";
((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.SecurityAgent.log','C:/tmp/logs/agents/InsightMaker.SecurityAgent.log') | Set-Content -Path $filename;

2

Run the following script to update the remaining logs into their own subfolders.

$Filename = "C:\InsightMaker\SourceAgent\log4net.config";
((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.SourceAgent.log','C:/tmp/logs/agents/InsightMaker.SourceAgent.log') | Set-Content -Path $filename;

$Filename = "C:\InsightMaker\Apps\Search\api\log4net.config";
((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.Middleware.search.log','C:/tmp/logs/agents/InsightMaker.Middleware.search.log') | Set-Content -Path $filename;

$Filename = "C:\InsightMaker\Apps\Admin\api\log4net.config";
((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.Middleware.Admin.log','C:/tmp/logs/InsightMaker.Middleware.Admin.log') | Set-Content -Path $filename;

$Filename = "C:\InsightMaker\Utils\InsightMaker.IndexUtilities\log4net.config";
((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.IndexUtilities.log','C:/tmp/logs/InsightMaker.IndexUtilities.log') | Set-Content -Path $filename;

$Filename = "C:\InsightMaker\Utils\InsightMaker.Security.BuiltinSecurityUtilities\log4net.config";
((Get-Content -path $filename -Raw) -replace 'c:/tmp/logs/InsightMaker.Security.BuiltinSecurityUtilities','C:/tmp/logs/InsightMaker.Security.BuiltinSecurity.log') | Set-Content -Path $filename;

Initialise Control Hub Parameter

1

Run the following script to initialise the Control Hub.

They configure the default elastic mappings for Aiimi Insight Engine indices. They also create some useful default entities for your system.

Check the path of the storage location in this script before running it. For example: (C:\).

cd C:\InsightMaker\Utils\InsightMaker.IndexUtilities;
.\InsightMaker.IndexUtilities.exe map
.\InsightMaker.IndexUtilities.exe initialise -fmsci
.\InsightMaker.IndexUtilities.exe initialise --collection-types;
.\InsightMaker.IndexUtilities.exe initialise --theme;
.\InsightMaker.IndexUtilities.exe upgrade --generate-entity-groups;

Agent Services

1

Run the following script to create all the agents.

Check the path of the storage location in this script before running it. For example: (C:\).

cd C:\InsightMaker\Utils\InsightMaker.IndexUtilities; 
 
sc.exe create InsightMakerContentAgent binPath= C:\InsightMaker\ContentAgent\InsightMaker.ContentAgent.exe DisplayName= "InsightMaker Content Agent" start= auto; 
 
sc.exe create InsightMakerEnrichmentAgent binPath= C:\InsightMaker\EnrichmentAgent\InsightMaker.EnrichmentAgent.exe DisplayName= "InsightMaker Enrichment Agent" start= auto; 
 
sc.exe create InsightMakerJobAgent binPath= C:\InsightMaker\JobAgent\InsightMaker.JobAgent.exe DisplayName= "InsightMaker Job Agent" start= auto; 
 
sc.exe create InsightMakerMigrationAgent binPath= C:\InsightMaker\MigrationAgent\InsightMaker.MigrationAgent.exe DisplayName= "InsightMaker Migration Agent" start= auto; 
 
sc.exe create InsightMakerOcrAgent binPath= C:\InsightMaker\OcrAgent\InsightMaker.OcrAgent.exe DisplayName= "InsightMaker Ocr Agent" start= auto; 
 
sc.exe create InsightMakerSecurityAgent binPath= C:\InsightMaker\SecurityAgent\InsightMaker.SecurityAgent.exe DisplayName= "InsightMaker Security Agent" start= auto;  
 
sc.exe create InsightMakerSourceAgent binPath= C:\InsightMaker\SourceAgent\InsightMaker.SourceAgent.exe DisplayName= "InsightMaker Source Agent" start= auto;

Update Descriptions

1

Run the following script to update the descriptions of these agents.

sc.exe description InsightMakerContentAgent "InsightMaker Content Agent"; 
 
sc.exe description InsightMakerEnrichmentAgent "InsightMaker Enrichment Agent"; 
 
sc.exe description InsightMakerJobAgent "InsightMaker Job Agent"; 
 
sc.exe description InsightMakerMigrationAgent "InsightMaker Migration Agent"; 
 
sc.exe description InsightMakerOcrAgent "InsightMaker Ocr Agent"; 
 
sc.exe description InsightMakerSecurityAgent "InsightMaker Security Agent"; 
 
sc.exe description InsightMakerSourceAgent "InsightMaker Source Agent";

Install Tika

1

Run the following script to move the Tika server file.

Check the storage path and release names match the required versions. For example: (C:), (tika-server-standard-3.1.0).

mkdir C:\Utils\Tika; 
copy-item "C:\InsightMaker\Tika\tika-server-standard-3.1.0.jar" -Destination "C:\Utils\Tika";

2

Run the following script to create a run-tika.bat.

Check the storage path and release names match the required versions. For example: (C:), (tika-server-standard-3.1.0).

If your Elasticsearch is in a folder with spaces in the name, the Java path will need to be in quotation marks.

'"C:\Apps\elasticsearch-8.17.3\jdk\bin\Java" -jar C:\\Utils\Tika\tika-server-standard-3.1.0.jar' | Out-File -FilePath C:\Utils\Tika\run-tika.bat -encoding ascii;

3

Run the following script to create the Tika service.

This will open an NSSM service installer modal.

Check the path of the storage location in this script before running it. For example: (C:\).

C:\Utils\nssm-2.24\win64\nssm.exe install InsightMakerTika;

4

Path: Select the explore (...) button.

We recommend you don't copy and paste the file path.

5

Navigate to C:Utils\Tika\run-tika.bat.

6

Startup Directory: Select the explore (...) button

We recommend you don't copy and paste the file path.

7

Navigate to C:Utils\Tika.

If you want to run Tika without installing it from PowerShell navigate to the Tika folder. Run the .\run-tika.bat serve file. This can help you to identify any issues which are less obvious when going straight for the install.

8

Run the following script to start the Tika service.

C:\Utils\nssm-2.24\win64\nssm.exe start InsightMakerTika;

Check Tika Services

1

Open your web browser and navigate to http://localhost:9998 to check the status of Tika.

It should say: "Welcome to the Apache Tika X.X.X-SNAPSHOT Server".

IIS Configuration

The '.NET Core and HostingBundle' must be installed before you continue. If it's not, the middleware will not be able to run in IIS.

1

Run the following script to enable IIS.

Enable-WindowsOptionalFeature -Online -FeatureName IIS-WebServerRole, IIS-WebServer, IIS-CommonHttpFeatures, IIS-ManagementConsole, IIS-HttpErrors, IIS-HttpRedirect, IIS-WindowsAuthentication, IIS-StaticContent, IIS-DefaultDocument, IIS-HttpCompressionStatic, IIS-DirectoryBrowsing

2

Run the following command to create a self-signed certificate.

This is used later and can be updated for production builds.

$newcert = New-SelfSignedCertificate -dnsname "InsightEngine20" -KeyLength 2048 -CertStoreLocation cert:\LocalMachine\My -NotAfter (Get-Date).AddYears(20)

3

Run the following script to create a new https (443) binding and remove the http (80) binding.

New-WebBinding -Name "Default Web site" -IP "*" -Port 443 -Protocol https
Get-WebBinding -Port 80 -Name "Default Web Site" | Remove-WebBinding

IIS Application Pools Setup

1

Run the following script to create a new application pool.

$newAppPool = New-WebAppPool -Name "Admin";
$newAppPool.autoStart = "true";
$newAppPool | Set-Item;

2

Run the following script to update the applications IIS settings.

Import-Module WebAdministration;

Set-ItemProperty "IIS:\Sites\Default Web site" -name physicalPath -value "C:\InsightMaker\Apps\Search\app";
New-WebApplication -Name "API" -Site "Default Web Site" -PhysicalPath "C:\InsightMaker\Apps\Search\api" -ApplicationPool "DefaultAppPool";
New-WebApplication -Name "Admin" -Site "Default Web Site" -PhysicalPath "C:\InsightMaker\Apps\Admin\app" -ApplicationPool "Admin";
New-WebApplication -Name "Admin\api" -Site "Default Web Site" -PhysicalPath "C:\InsightMaker\Apps\Admin\api" -ApplicationPool "Admin";

IIS Assign Certificate to Binding

This sets up the default website on the server. It means it can be accessed using 'localhost' in any browser and the other apps using their alias. For example admin is accessed using localhost/admin.

1

The account IIS runs on needs read access to your Elasticsearch certificates.

2

Open IIS and navigate to the Default web site.

3

Select Bindings under Edit Site on the actions menu.

4

Select port 443 and select Edit.

5

SSL certificate: Select the new certificate you created.

6

Open your web browser and navigate to https://localhost/#/login to check this is set up correctly.

If you see a blank screen it's possible the IIS didn't set up correctly. You may need to restart IIS.

Password Encryption

We recommend you encrypt your build. For help encrypting your build see our Installation Security area. Once you have completed those steps return to this guide.

Restart Agent Services

1

Run the following script to restart all the agent services.

restart-service InsightMakerContentAgent;
restart-service InsightMakerEnrichmentAgent;
restart-service InsightMakerJobAgent;
restart-service InsightMakerMigrationAgent;
restart-service InsightMakerOcrAgent;
restart-service InsightMakerSecurityAgent;
restart-service InsightMakerSourceAgent;
iisreset;

Accessing Control Hub

1

Navigate to http://localhost/admin.

2

Login with your elastic username and password.

When you login for the first time you should see your agents on the configuration page. They will all be empty.

3

You can now close any PowerShell windows you have open.

Installing OData API (Optional)

1

Navigate to C:/Insightmaker/Apps/OData.

2

Open appsettings.default.json in a source code editor like NotePad++.

3

Check the following details:

The Plugins path points to the plugins folder. Likely C:\InsightMaker\Plugins.
The elastic certificate path points to the elastic-stack-ca.p12 file. This was created when you installed Elasticsearch.
The correct password is set.
The Elasticsearch credentials are set to the elastic login you created.
The prefix is set to the correct value and consistent across all components.

4

Add your two Aiimi Insight Engine licences to the root of the json:

"licenseKey": "your key"
"licenseSig": "your sig"

5

Check the remoteAddress is set to ‘http://localhost/analytics/‘.

6

Save this file, do not change the name or location.

7

Within C:/Insightmaker/Apps/OData open log4net.default.config.

8

Change the file paths to be where you’d like to save log files.

This can not be the in InsightMaker folder.

9

Save this file, do not change the name or location.

10

Remove the '.default' from each of the following file names:

'appsettings.default.json'
'log4net.default.config'
'web.default.config'

11

Open the web.config file.

Check the module is modules="AspNetCoreModuleV2".

12

Open Internet Information Services (IIS).

13

Navigate to the Default website.

14

Click add new application.

15

Alias - Enter 'analytics' as the name of this application.

16

Physical Path - Set this to the API folder within InsightMaker/Apps/OData.

You should be able to view the OData metadata document using localhost/analytics/odata/$metadata.

PreviousElastic and Kibana Install (Windows)NextInstallation Security

Last updated 21 days ago