Architecture and How It Works

Introduction

Aiimi Insight Engine is made up of services that do ‘backend processing’, a repository and a set of web apps that provide access. The web apps consist of an API and an APP. The Aiimi Insight Engine agents and web apps run on Microsoft Windows or Linux operating systems. This can be a server or desktop version providing it can run a suitable web server such as Internet Information server of Apache.

• A desktop version such as Windows 10 would only be used for demo or development environments.

  • The repository, which is Elasticsearch, can run on Windows or Linux. At Aiimi we have no preference between the two.

• Java is required to run Elasticsearch. The 7 generation of Elasticsearch is packaged with a suitable JVM to run the software.

  • You can use an external version but note, new licencing models apply to Oracle Java versions. The Elasticsearch tool Kibana cis useful for admins to inspect indexes and perform ad-hoc reporting. For production platforms access to this would be locked down.

High Level Architecture

• Agent Servers – One or more servers running the Aiimi Insight Engine agents:

  • Security Agent – Responsible for synchronising users and groups from the credential stores. It is also responsible for servicing authentication requests.

  • Source Agent – Connects to source systems to discover new, modified and deleted data and documents. It also fetches data and documents for enrichment and when users want to preview a document.

  • Content Agent – A support service for the Source Agent. It handles thumbnails, caching, proxying document & data fetch and preview requests.

  • Enrichment Agent – Runs enrichment on data and documents stored within Aiimi Insight Engine. It runs the configured pipelines, each containing one or more enrichment steps. For example, document classification, NER or image recognition.

  • Job Agent – Runs maintenance and other jobs that the platform requires to be run. For example index backup, recommendations, and notifications. It may also run customer specific scripts.

  • OCR Agent – Responsible for running one or more OCR engines. Out of the box IronOCR is provided, but plugging in other engines such as ABBY Reader is supported.

  • Migration Agent – Runs migration jobs which move documents and data between systems.

  • Tika Agent – Converts documents from their native format to plain indexable text. This is used by the enrichment agent (Tika Text Conversion Step).

  • Repository Servers:

    • One or more Elasticsearch nodes

    • One or more Elasticsearch proxy nodes

    • One Kibana node (which usually shares with a proxy node)

  • Web Servers

    • One or more servers running the APIs and the end user apps.

The following schematic illustrates an example architecture for Aiimi Insight Engine.