Aiimi Insight Engine
User GuidesAiimi
  • Introducing Aiimi Insight Engine
  • Architecture
    • Overview and Key Concepts
    • Search Flows
      • Search Flow Types
      • Smart Filtering
      • Query and Prompt Classification
      • Search Algorithms
      • Extractive and Generative Models
    • Hosting Options
    • Architecture and How It Works
      • Agent Servers
        • Security Agent
        • Source Agent
        • Content Agent
        • Enrichment Agent
        • Job Agent
        • OCR Agent
        • Migration Agent
        • Tika Agent
      • Repository
        • Data Node
        • Proxy Node
        • Kibana Node
      • Gateway and User Interface
      • Document and Data Sources
    • Deployment Options
    • Security
      • User Security
      • Data and Document Security
      • Source System Security
      • Firewalling
      • Agent Servers
      • Repository
      • Gateway (Web Server)
      • Tools & Utilities
  • Installation
    • Elastic and Kibana Install (Windows)
    • Aiimi Insight Engine Installation (Windows)
      • Installation Security
      • Certificates in a Key Vault
      • SAR Configuration
      • CSOM Bridge Set Up
      • AI Studio
    • AI Services
      • Prerequisites
      • AI Enrichment Service
        • Installation and Setup
        • Enabling Enrichment Steps
        • Using AI Enrichment Steps
        • Performance and Concurrency
      • AI Model Service
        • Installation and Setup
        • Enabling Providers
        • Private Generative AI
        • Azure Open AI
        • Enabling AI History
        • HTML Cleaner Service
      • Configuration of Logging
      • Offline Set-up of Models
      • Using SSL
      • Running as a Service (Windows)
      • Using GPUs
      • AI and Semantic Search Set Up
        • Open & Closed Book AI
        • Semantic Search
          • Vectors for Semantic Search
          • Source Configuration
          • Sentence Transformer Models
          • Enrichment
          • Kibana
          • Final Search Flow
    • Email Threading Upgrade
  • Run Books
    • SharePoint Online Connector
  • Control Hub
    • Configurations
      • Config Management
      • Security Configurations
        • Security - General
        • Security - Source
          • Active Directory
          • Atlassian
          • Azure Active Directory
          • Builtin Security
          • Miro Security
          • Google Directory
          • Slack Security
        • Security - Sync
        • Security - Agents
        • Security - Scheduling
      • Source Configurations
        • Source - General
        • Source - Source
          • Alfresco Kafka
          • Azure Blob Storage
          • BBC Monitoring
          • Big Query Cataloguer
          • BIM360
          • CSV Data Loader
          • Confluence
          • Content Server
          • Data File Cataloguer
          • Document Store
          • DocuSign
          • Dropbox
          • Exchange 365
          • Filesystem
          • Google Bucket
          • Google Drive
          • Google Vault
          • Jira
          • JSON Data Loader
          • Livelink
          • MFiles
          • Microsoft Teams
          • Mimecast
          • Miro
          • ODBC Data Loader
          • PowerBi Cataloguer
          • Reuters Connect
          • ShareFile
          • SharePoint
            • Azure Portal and Azure AD Authentication
            • Sensitivity Labels
          • SharePoint Legacy
          • SQL Server Cataloguer
          • Slack
          • Versioned Document Store
          • Websites
          • XML Data Loader
        • Source - Crawl
        • Source - Agents
        • Source - Schedule
        • Source - Advanced
      • Enrichment Configurations
        • Creating a Pipeline
          • General
          • Steps
            • AccessMiner
            • AI Classification
            • Anonymiser
            • CAD Extractor
            • Checksum
            • Content Retrieval
            • Copy
            • Data Rule Processor
            • Delete
            • Email Extractor
            • Entity Rule Processor
            • External Links
            • Geotag
            • Google NLP Extractor
            • Google Vision Extractor
            • Metrics Calculation
            • Microsoft Vision Extractor
            • OcrRest
            • Office Metadata
            • PCI Extractor
            • REST
            • Set Document Risk
            • Text Cleaner
            • Tika Text Extraction
            • Trie Entity Extractor
            • Update Metadata
          • Filters
          • Agents
          • Schedule
          • Advanced
      • OCR Engine
      • Job Configurations
        • General
        • Job
          • AutomatedSearchJob
          • Command Job
          • ElasticJob
          • Extended Metrics Job
          • File Extractor
          • GoogleVaultSAR
          • Google Drive Last Access Date
          • Nightly Events Processor Job
          • Notifications Processor Job
          • Portal Sync Job
          • Purge Job
          • Text Content Merge Job
        • Output
        • Agents
        • Scheduling
      • Migration Configuration
        • General
        • Filter
        • Metadata Mappings
        • Agents
        • Scheduling
        • Advanced
    • Credentials
    • Mappings
      • Entities
        • Manage Entity Groups
        • Create an Entity
        • Manage an Entity
      • Models
        • Create a New Model
        • Find a Model
        • Enable or Disable a Model
      • Vectors
      • Rank Features
    • Featured Links
    • AI Settings
      • Classifications
      • Class
      • Class Rules
      • AI Classification
    • User Settings
    • Stats
      • Data Views
    • Global Settings
      • General
      • Authentication
      • App Settings
      • Application Access
      • Thumbnails
      • Presets
      • Code of Conduct
      • Metrics
      • Viewer
      • SAR
        • Importing Data For A SAR
        • SAR Disclosure Document Storage
        • Getting SAR data from Google Vault
        • SAR Access
        • SAR File Status
      • Disclosure Portal
        • Disclosure Portal Set Up
        • SARs From The Portal
        • Email Delivery Settings
          • Delivery Settings
          • Brand Settings
          • Customise Emails
        • SMS Delivery Settings
        • Requestor Message Limit
        • Attachment Configuration
        • Password Configuration
        • File Scanner Configurator
      • Collections
      • Visualisations
        • Related Result Connections Diagram
        • Event Timeline
        • Timeline Lens Activity Chart
        • Relationship Map
      • Notifications
      • Map Lens
      • Theming
      • User Avatar
      • OData API
      • Uploads
      • Security
    • Search Settings
      • Search Relevancy
        • Core Settings
        • Makers Algorithm
        • Filename Boost Layer
        • Minimum Matching Terms Filter
        • Field Boost
        • Modified Date Boosting
        • Hit Highlighting
        • Why My Search Matched
        • Data Search Strategy
      • Bulk Search
        • Managing a Bulk Search
      • Search Flows
        • Create a Search Flow
          • General
          • Query Classification Step
          • Search Steps
          • Model Steps
      • Filtering
      • Search Performance
      • Related Results
  • AI Studio
    • Classifications
      • Classifications
      • Classification Rules
    • Jobs
  • Labels
  • API Guides
    • Insight API Guide
      • Swagger Documentation
      • Trying Some Endpoints
      • Search Filter
      • Hits / Items
      • Inspecting REST Calls
    • Data Science API Guide
      • REST Interface
        • Login
        • Datasets
        • Fields
        • Field Statistics
        • Search
        • Scroll
        • Update
      • Python Wrapper
        • Login
        • Datasets
        • Fields
        • Field Statistics
        • Search
        • Query Builders
        • Scroll
        • Scroll Search
        • Update Single Document
        • Bulk Update
    • Creating a Native Enrichment Step
      • Creating an Enrichment Step
        • Creating the Core Classes
        • Extending our Enrichment Step
        • Adding a Configuration Template
        • Adding the Enrichment Step
        • Creating an Enrichment Pipeline
      • Other Tasks
        • Entities, Metadata and Data
        • Accessing the Repository
      • Example Code
      • Troubleshooting
    • Creating a Python Enrichment Step
      • Creating an Enrichment Step
        • Running the Example from Command Line
        • Running the Example
      • Creating Your Own Step
      • Adding or Changing Entities, Metadata
  • Whitepapers and Explainers
    • From a Billion To One – Mastering Relevancy
    • Methods for Text Summarization
      • Application
      • Technology Methods
      • Commercial Tools
      • Key Research Centres
      • Productionisation
      • Related Areas of Text Analytics
      • Conclusion
      • References
Powered by GitBook
On this page
  • Access Controls
  • Progressive and Privileged Access
  • Progressive Access
  • Privileged Access
  • Source Scope and Visibility Within Apps
  • Classification
  • Advanced Security Controls
  • Security Classifications
  • Risk Ratings
  • Redaction and Anonymisation
  • Banned Words
  • Mark as Sensitive
  • Audit Controls
  1. Architecture
  2. Security

Data and Document Security

Last updated 1 month ago

There are a few ways you can control your data and document security. You can limit the data certain users can access, add flags for high risk items, anonymise information and much more.

Access Controls

Fundamental to Aiimi Insight Engine is the enforced access control rules. The access control rules dictate who can see a piece of data or content.

Users are members of groups, that are usually synchronised from an Active Directory. Every piece of data and content has a list of groups that can access it. When a user performs a search they will only see items where their group has the relevant permissions.


Progressive and Privileged Access

The use of progressive and privileged access is fully audited and must be explicitly granted by an administrator.

Progressive Access

If a user has been granted progressive access they can see items they can't access in their results list but, they won't be able to open them. They can request access to an item from the item's owner.

For more information

Progressive access is an optional feature and must be explicitly granted by an administrator.

Privileged Access

Privileged access gives select users a controlled way to bypass permissions. Users with privileged access can see all search results even if they don’t have permission to access an item. Unlike progressive access, they will see the item, preview, and metadata.

Privileged access is an optional feature and must be explicitly granted by an administrator. It is not available to any user by default​.


Source Scope and Visibility Within Apps

Sources represent either different repositories or areas of a repository. For example, you may have a source for SharePoint HR and SharePoint Asset.

  • Control which sources are visible within each app of Aiimi Insight Engine.

    • You can limit the application it is available within from the source configuration.

    • If you disable a source in Enterprise Search, even if a user has permission to see the documents, they will never see this content via the Search app.

  • Limit who can see and access a source anywhere in the application. Within a source's configuration, you can add which users and groups can access it.


Classification

Aiimi Insight Engine can automatically classify data and content. This could be business classifications, such as types of accounts payable document, i.e., invoice, purchase order, goods receipts. Or it could be security classifications, such as public, internal, restricted and top secret.

Using classifications, you can also add additional controls like, who can see items, or where information can be sent. You can also use these classifications to help inform and automate information security policies.


Advanced Security Controls

Security Classifications

Security classifications allow you to apply additional security on top of the standard access control lists.

  • Users are given the right to see one or more security classification.

    • For a user to have access they must have access via the standard access control list. Or they must have all the security classifications that appear on the data or content.

  • Data and content may also have one or more security classifications applied.

Risk Ratings

The Aiimi Insight Engine can calculate the potential risk of a piece of data or content.

It's based on the items PII data:

  • The number of people referenced

  • Amount of personal information

  • Visibility to your workforce

  • Frequency of use

  • Specific keywords that indicate risk

It's a key feature of our PII and DSAR solution. Risk ratings can also be used in a similar way to classifications, and used to restrict what people can see.

Redaction and Anonymisation

Redact the information in your documents within a SAR or Collection. You can redact specific parts like PII and PCI or select the sections of content you want to redact. The redacted items are only available on Aiimi Insight Engine, the source is not affected.

Automated Redaction

PII and other sensitive keywords can be auto removed using the anonymisation enrichment step.

  • For example, during enrichment, if a 'high-risk' piece of content is found it can be automatically anonymised.

Banned Words

You can create a list of banned words, if these words appear in a document, the document is marked sensitive.

  • Sensitive documents do not appear in a users results, even if they match a search.

This feature is a useful safety guard against things that may end up in the wrong place. Typical banned words include things such as; p45, harassment, CV, Disciplinary etc.

Mark as Sensitive

Users may find results in their search that they consider sensitive. They can manually mark a result as sensitive, it is temporarily hidden until it is investigated.

This is a good way of crowd sourcing and quickly removing items that require review. Administrators can review files that have been marked as sensitive. They are reinstated if they are deemed not to a risk.

Audit Controls

The actions of all users' are audited and stored in an audit log. It can be used to detect misuse, show the use and accuracy of the system and support advanced recommendation algorithms.

The audit stores a users activity, what they searched, what they opened and if they have collaborated on a collection or DSAR.

For more information

For more information

For more information .

For more information on Risk Ratings see

For support setting up redaction use

For support creating an anonymiser see

see our guide on application access.
see our guide on configuring a source.
see our guide on AI Studio Classifications
our enrichment guide for Setting Document Risk.
our guide on the enrichment anonymiser.
see our guide on application access.
our guide on redacting information.
An example of a user with the ‘secret’ classification and a corresponding document with the protective classification ‘secret'.