Aiimi Insight Engine Habanero
User GuidesAiimi
  • Introducing Aiimi Insight Engine
  • Architecture
    • Overview and Key Concepts
    • Search Flows
      • Search Flow Types
      • Smart Filtering
      • Query and Prompt Classification
      • Search Algorithms
      • Extractive and Generative Models
    • Hosting Options
    • Architecture and How It Works
      • Agent Servers
        • Security Agent
        • Source Agent
        • Content Agent
        • Enrichment Agent
        • Job Agent
        • OCR Agent
        • Migration Agent
        • Tika Agent
      • Repository
        • Data Node
        • Proxy Node
        • Kibana Node
      • Gateway and User Interface
      • Document and Data Sources
    • Deployment Options
    • Security
      • User Security
      • Data and Document Security
        • Progressive Access
        • Privileged Access
      • Source System Security
      • Firewalling
      • Agent Servers
        • Security Agent
        • Source Agent
        • Content Agent
        • Enrichment Agent
        • Job Agent
        • OCR Agent
        • Migration Agent
        • Tika Agent
      • Repository
      • Gateway (Web Server)
      • Tools & Utilities
  • Installation
    • Elastic and Kibana Install (Windows)
    • Aiimi Insight Engine Installation (Windows)
      • Installation Security
      • Certificates in a Key Vault
      • SAR Configuration
      • CSOM Bridge Set Up
    • AI Services
      • Prerequisites
      • AI Enrichment Service
        • Installation and Setup
        • Enabling Enrichment Steps
        • Using AI Enrichment Steps
        • Performance and Concurrency
      • AI Model Service
        • Installation and Setup
        • Enabling Providers
        • Private Generative AI
        • Azure Open AI
        • Enabling AI History
        • HTML Cleaner Service
      • Configuration of Logging
      • Offline Set-up of Models
      • Using SSL
      • Running as a Service (Windows)
      • Using GPUs
      • AI and Semantic Search Set Up
        • Open & Closed Book AI
        • Semantic Search
          • Vectors for Semantic Search
          • Source Configuration
          • Sentence Transformer Models
          • Enrichment
          • Kibana
          • Final Search Flow
    • Email Threading Upgrade
  • Control Hub
    • Configurations
      • Config Management
      • Security Configurations
        • Security - General
        • Security - Source
          • Active Directory
          • Atlassian
          • Azure Active Directory
          • Builtin Security
          • Miro Security
          • Google Directory
          • Slack Security
        • Security - Sync
        • Security - Agents
        • Security - Scheduling
      • Source Configurations
        • Source - General
        • Source - Source
          • Alfresco Kafka
          • Azure Blob Storage
          • BBC Monitoring
          • Big Query Cataloguer
          • BIM360
          • CSV Data Loader
          • Confluence
          • Content Server
          • Data File Cataloguer
          • Document Store
          • DocuSign
          • Dropbox
          • Exchange 365
          • Filesystem
          • Google Bucket
          • Google Drive
          • Google Vault
          • Jira
          • JSON Data Loader
          • Livelink
          • Microsoft Teams
          • Mimecast
          • Miro
          • ODBC Data Loader
          • PowerBi Cataloguer
          • Reuters Connect
          • ShareFile
          • SharePoint
            • Azure Portal and Azure AD Authentication
          • SharePoint Legacy
          • SQL Server Cataloguer
          • Slack
          • Versioned Document Store
          • Websites
          • XML Data Loader
        • Source - Crawl
        • Source - Agents
        • Source - Schedule
        • Source - Advanced
      • Enrichment Configurations
        • Creating a Pipeline
          • General
          • Steps
            • AccessMiner
            • AI Classification
            • Anonymiser
            • CAD Extractor
            • Checksum
            • Content Retrieval
            • Copy
            • Data Rule Processor
            • Delete
            • Email Extractor
            • Entity Rule Processor
            • External Links
            • Geotag
            • Google NLP Extractor
            • Google Vision Extractor
            • Metrics Calculation
            • Microsoft Vision Extractor
            • OcrRest
            • Office Metadata
            • PCI Extractor
            • REST
            • Set Document Risk
            • Text Cleaner
            • Tika Text Extraction
            • Trie Entity Extractor
            • Update Metadata
          • Filters
          • Agents
          • Schedule
          • Advanced
      • OCR Engine
      • Job Configurations
        • General
        • Job
          • AutomatedSearchJob
          • Command Job
          • ElasticJob
          • Extended Metrics Job
          • File Extractor
          • GoogleVaultSAR
          • Google Drive Last Access Date
          • Nightly Events Processor Job
          • Notifications Processor Job
          • Portal Sync Job
          • Purge Job
          • Text Content Merge Job
        • Output
        • Agents
        • Scheduling
      • Migration Configuration
        • General
        • Filter
        • Metadata Mappings
        • Agents
        • Scheduling
        • Advanced
    • Credentials
      • Create a Credential
      • Find a Credential
      • Edit a Credential
      • Delete a Credential
    • Mappings
      • Entities
        • Group Management
        • Create an Entity
        • Managing Entities
      • Models
        • Create a New Model
        • Find a Model
        • Enable or Disable a Model
      • Vectors
      • Rank Features
    • Featured Links
    • AI Settings
      • Classifications
      • Class
      • Class Rules
      • AI Classification
    • User Settings
    • Stats
      • Data Views
    • Global Settings
      • General
        • Stackdriver
        • Document Recommendations
        • Searchable PDF Storage
        • Versioning
        • Results
        • Marking Useful Results
        • Folder Browsing
        • Cascading Search
        • Search Suggestions
        • Miscellaneous
      • Authentication
      • App Settings
      • Application Access
      • Thumbnails
      • Presets
      • Code of Conduct
      • Metrics
      • Viewer
        • Preview Configurations
        • Redacting Information
        • Watermarking
      • SAR
        • Importing Data For A SAR
        • SAR Disclosure Document Storage
        • Getting SAR data from Google Vault
        • SAR Access
        • SAR File Status
      • Disclosure Portal
        • Disclosure Portal Set Up
        • SARs From The Portal
        • Email Delivery Settings
          • Delivery Settings
          • Brand Settings
          • Customise Emails
        • SMS Delivery Settings
        • Requestor Message Limit
        • Attachment Configuration
        • Password Configuration
        • File Scanner Configurator
      • Collections
      • Visualisations
        • Related Result Connections Diagram
        • Event Timeline
        • Timeline Lens Activity Chart
        • Relationship Map
      • Notifications
      • Map Lens
      • Theming
        • General
        • Layout
        • Site Message
        • Colours
      • User Avatar
      • OData API
      • Uploads
      • Security
    • Search Settings
      • Search Relevancy
        • Core Settings
        • Makers Algorithm
        • Filename Boost Layer
        • Minimum Matching Terms Filter
        • Field Boost
        • Modified Date Boosting
        • Hit Highlighting
        • Why My Search Matched
        • Data Search Strategy
      • Bulk Search
        • Managing a Bulk Search
      • Search Flows
        • Create a Search Flow
          • General
          • Query Classification Step
          • Search Steps
          • Model Steps
      • Filtering
      • Search Performance
      • Related Results
  • API Guides
    • Insight API Guide
      • Swagger Documentation
      • Trying Some Endpoints
      • Search Filter
      • Hits / Items
      • Inspecting REST Calls
    • Data Science API Guide
      • REST Interface
        • Login
        • Datasets
        • Fields
        • Field Statistics
        • Search
        • Scroll
        • Update
      • Python Wrapper
        • Login
        • Datasets
        • Fields
        • Field Statistics
        • Search
        • Query Builders
        • Scroll
        • Scroll Search
        • Update Single Document
        • Bulk Update
    • Creating a Native Enrichment Step
      • Creating an Enrichment Step
        • Creating the Core Classes
        • Extending our Enrichment Step
        • Adding a Configuration Template
        • Adding the Enrichment Step
        • Creating an Enrichment Pipeline
      • Other Tasks
        • Entities, Metadata and Data
        • Accessing the Repository
      • Example Code
      • Troubleshooting
    • Creating a Python Enrichment Step
      • Creating an Enrichment Step
        • Running the Example from Command Line
        • Running the Example
      • Creating Your Own Step
      • Adding or Changing Entities, Metadata
  • whitepapers and explainers
    • From a Billion To One – Mastering Relevancy
    • Methods for Text Summarization
      • Application
      • Technology Methods
      • Commercial Tools
      • Key Research Centres
      • Productionisation
      • Related Areas of Text Analytics
      • Conclusion
      • References
Powered by GitBook
On this page
  • Access Controls
  • Progressive and Privileged Access
  • Source Scope and Visibility Within Apps
  • Classification
  • Advanced Security Controls - Security Classifications
  • Risk Ratings
  • Redaction and Anonymisation
  • Automated Redaction
  • Banned Words
  • Mark as Sensitive
  • Audit Controls
  1. Architecture
  2. Security

Data and Document Security

Access Controls

Fundamental to Aiimi Insight Engine is the enforced access control rules. The access control rules dictate who can see a piece of data or content.

How it works:

Users are members of groups, that are usually synchronised from an Active Directory. Items of data and content have a list of groups that have read access. When searching users only see items that match the permissions from their user group list.


Progressive and Privileged Access

Progressive Access

  • Tells a user if items exist for their search but they cannot access them. They can then ask the owner for permission​ to see items.

Privileged Access

  • Privileged access gives a select set of users a controlled way to bypass permissions.

  • This could be for audit requirements, legal discovery, or analysis after a cyber attack​.

Both progressive and privileged access are locked down to specific groups. They are not available to any user by default​.

The use of progressive and privileged access is fully audited. You can use this to monitor and report on it.


Source Scope and Visibility Within Apps

Sources represent either different repositories or different areas of a repository. For example, we may have a source for SharePoint HR and a source for SharePoint Asset.

  • You can control which sources are visible for each app.

  • In the screenshot below we only surface our ‘Pension Documents’ in the PII and PCI apps.

    • It's important to note that even if a user has underlying permissions to see the pension documents, they will never see this content via the Search app.

  • You can see here we are unable to select the ‘Pension Documents’ source in the Search app.


Classification

Aiimi Insight Engine can automatically classify data and content. This can be business classifications, such as types of accounts payable document, i.e., invoice, purchase order, goods receipts. Or it can be a security classification, such as public, internal, restricted and top secret.

Using classifications, we can build additional controls such as who can see items, or where information can be sent. We can also use these classifications to help inform and automate information security policies.


Advanced Security Controls - Security Classifications

Security classifications allow you to apply additional security on top of the standard access control lists.

  • Users are given the right to see one or more security classifications.

    • For a user to have access they must have access via the standard access control list. Or they must have all the security classifications that appear on the data or content.

  • Data and content may also have one or more security classifications applied.


Risk Ratings

This is based on the items PII data:

  • The number of people referenced

  • Amount of personal information

  • Visibility to your workforce

  • Frequency of use

  • Specific keywords that indicate risk

It's a key feature of our PII and DSAR solution. Risk ratings can also be used in a similar way to classifications, and used to restrict what people can see (using the security classifications mechanism).


Redaction and Anonymisation

You can redact the information in your documents within a SAR or Collection. You can redact specific parts like PII or PCI or select the sections of content you want to redact.

Automated Redaction

  • For example, during enrichment, if a 'high-risk' piece of content is found it can be automatically anonymised.


Banned Words

You can create a list of banned words. If these words appear in a document, the document is marked sensitive.

  • Sensitive documents do not appear in a users search result, even if they match a search.

This feature is a useful safety guard against things that may end up in the wrong place. Typical banned words include things such as; p45, harassment, CV, Disciplinary and so on.


Mark as Sensitive

Users may find results in their search that they consider sensitive. Users can mark a result as sensitive where it will be temporarily hidden until it is investigated.

This is a good way of crowd sourcing and quickly removing items that require review. Administrators can review files that have been marked as sensitive and then reinstate them if they are deemed not to be a risk.


Audit Controls

The actions of all users' are audited and stored in an audit log. It can be used to detect misuse, show the use and accuracy of the system and support advanced recommendation algorithms.

The auditing stores a users activity, what they searched, what they opened and If they have collaborated on someones collection or DSAR.

The Aiimi Insight Engine can calculate the potential risk of a piece of data or content. For more information on Risk Ratings see

Redacted items are only available on Aiimi Insight Engine. The source is not affected. For support setting up redaction use

The removal of PII and other sensitive keywords can be automated using the anonymisation enrichment step. For support creating an anonymiser see

our enrichment guide for Setting Document Risk.
our guide on redacting information.
our guide on the enrichment anonymiser.
An example of a user with the ‘secret’ classification and a corresponding document with the protective classification ‘secret'.