The enrichment agent sits at the heart of Aiimi Insight Engine and enriches the data and documents with additional context. This context is stored against the item in the repository as a series of labels and data structures. These are used by the user interfaces and other processes for a few reasons. It helps structure information, drive recommendations, push insights, and support data science activities.
A core part of the enrichment agent is like a pipeline, it is a container for a series of enrichment steps. Each step is responsible for enriching the item being passed through the pipeline. For example, classification, named entity recognition, adding geotags and so on.
You configure your list of pipeline steps with a simple drag and drop interface. Each step will have some configuration that also needs completing.
As with security and sources, pipeline steps are based on a framework. It is easy for Aiimi, our customers and partners to create a specific framework for their needs.
A Microsoft.NET framework can be used to implement a pipeline step.
Or, you can leverage the REST step which allows you to call out to any REST/JSON based service.
Aiimi use the REST based approach for a series of Python based machine learning steps. For example, Document classification, phrase and topic extraction, statistical named entity recognition, document summarisation and sentiment analysis. These are in the Python REST Service which ships with Aiimi Insight Engine. You can find an example that you can use as the basis of your own custom steps in here.
Enrichment can be computationally expensive (CPU and Memory) depending on the steps in your pipelines. Enterprise production platform commonly have more than one server for crawling and enrichment. The exact number will depend on your volumetrics.
