Aiimi Insight Engine Bell
User GuidesAiimi
  • Introducing Aiimi Insight Engine
  • Architecture
    • Overview and Key Concepts
    • Hosting Options
    • Architecture and How It Works
      • Agent Servers
        • Security Agent
        • Source Agent
        • Content Agent
        • Enrichment Agent
        • Job Agent
        • OCR Agent
        • Migration Agent
        • Tika Agent
      • Repository
        • Data Node
        • Proxy Node
        • Kibana Node
      • Gateway and User Interface
      • Document and Data Sources
    • Deployment Options
    • Security
      • Source System Security
      • Firewalling
      • Agent Servers
        • Security Agent
        • Source Agent
        • Content Agent
        • Enrichment Agent
        • Job Agent
        • OCR Agent
        • Migration Agent
        • Tika Agent
      • Repository
      • Gateway (Web Server)
      • Tools & Utilities
  • Installation
    • Elasticsearch Installation (Windows)
    • Aiimi Insight Engine Installation (Windows)
    • AI Services
      • Prerequisites
      • AI Enrichment Service
        • Installation and Setup
        • Enabling Enrichment Steps
        • Using AI Enrichment Steps
        • Performance and Concurrency
      • AI Model Service
        • Installation and Setup
        • Enabling Providers
        • Private Generative AI
        • Azure Open AI
      • Configuration of Logging
      • Offline Set-up of Models
      • Using SSL
      • Running as a Service (Windows)
      • Using GPUs
    • HTML Cleaner Service
  • security
    • Users
    • Data and Documents
      • Progressive Access
      • Privileged Access
  • Control Hub
    • Configurations
      • Config Management
      • Security Configurations
        • Security - General
        • Security - Source
          • Active Directory
          • Azure Active Directory
          • Builtin Security
          • Miro Security
          • Google Directory
        • Security - Sync
        • Security - Agents
        • Security - Scheduling
      • Source Configurations
        • Source - General
        • Source - Source
          • Azure Blob Storage
          • BBC Monitoring
          • Big Query Cataloguer
          • BIM360
          • CSV Data Loader
          • Confluence
          • Content Server
          • Data File Cataloguer
          • Document Store
          • DocuSign
          • Dropbox
          • Exchange 365
          • Filesystem
          • Google Bucket
          • Google Drive
          • Google Vault
          • JSON Data Loader
          • Miro
          • ODBC Data Loader
          • PowerBi Cataloguer
          • Reuters Connect
          • ShareFile
          • SharePoint
          • SharePoint Legacy
          • Slack
          • SQL Server Cataloguer
          • Websites
          • XML Data Loader
        • Source - Crawl
        • Source - Agents
        • Source - Schedule
        • Source - Advanced
      • Enrichment Configurations
        • Creating a Pipeline
          • General
          • Steps
            • AccessMiner
            • Anonymiser
            • CAD Extractor
            • Checksum
            • Content Retrieval
            • Copy
            • Data Rule Processor
            • Delete
            • Email Extractor
            • Entity Rule Processor
            • External Links
            • Geotag
            • Google NLP Extractor
            • Google Vision Extractor
            • Metrics Calculation
            • Microsoft Vision Extractor
            • OcrRest
            • Office Metadata
            • PCI Extractor
            • REST
            • Set Document Risk
            • Text Cleaner
            • Tika Text Extraction
            • Trie Entity Extractor
          • Filters
          • Agents
          • Schedule
          • Advanced
      • OCR Engine
      • Job Configurations
        • General
        • Job
          • AutomatedSearchJob
          • Command Job
          • ElasticJob
          • Extended Metrics Job
          • GoogleVaultSAR
          • Nightly Events Processor Job
          • Notifications Processor Job
          • Portal Sync Job
          • Purge Job
          • Text Content Merge Job
        • Output
        • Agents
        • Scheduling
      • Migration Configuration
        • General
        • Filter
        • Metadata Mappings
        • Agents
        • Scheduling
        • Advanced
      • Content Server
    • Credentials
      • Create a Credential
      • Find a Credential
      • Edit a Credential
      • Delete a Credential
    • Mappings
      • Entities
        • Managing Groups
        • Create an Entity
        • Managing Entities
      • Models
        • Create a New Model
        • Find a Model
        • Enable or Disable a Model
      • Vectors
      • Rank Features
    • Featured Links
    • Global Settings
      • General
        • Stackdriver
        • Document Recommendations
        • Searchable PDF Storage
        • Versioning
        • Results
        • Marking Useful Results
        • Folder Browsing
        • Cascading Search
        • Search Suggestions
        • Delve Settings
        • Collections
        • Miscellaneous
      • Authentication
      • Application Access
      • Search Relevancy
        • Core Settings
        • Makers Algorithm
        • Filename Boost Layer
        • Minimum Matching Terms Filter
        • Field Boost
        • Modified Date Boosting
        • Hit Highlighting
        • Why My Search Matched
        • Data Search Strategy
      • Search Performance
      • Filtering
      • Thumbnails
      • Presets
      • Code of Conduct
      • Metrics
      • Viewer
        • Redacting Information
      • SAR
        • Importing Data For A SAR
        • Getting SAR data from Google Vault
        • SAR Access
      • Privacy Portal
        • Activate the Privacy Portal
        • Disclosure
        • Submit SARs From The Privacy Portal
        • Email Delivery Settings
          • Delivery Settings
          • Brand Settings
          • Customise Emails
        • SMS Delivery Settings
        • Requestor Message Limit
        • Attachment Configuration
        • Password Configuration
        • File Scanner Configurator
      • Visualisations
        • Related Result Connections Diagram
        • Event Timeline
        • Create and Modified Date Activity Chart
        • Relationship Map
      • Notifications
      • Map Lens
      • App
      • Theming
        • Links
        • Layout
        • Colours
      • Related Results
      • OData API
      • Bulk Search
        • Managing a Bulk Search
      • Search Flows
    • User Settings
    • Stats
      • Data Views
  • API Guides
    • Insight API Guide
      • Swagger Documentation
      • Trying Some Endpoints
      • Search Filter
      • Hits / Items
      • Inspecting REST Calls
    • Data Science API Guide
      • REST Interface
        • Login
        • Datasets
        • Fields
        • Field Statistics
        • Search
        • Scroll
        • Update
      • Python Wrapper
        • Login
        • Datasets
        • Fields
        • Field Statistics
        • Search
        • Query Builders
        • Scroll
        • Scroll Search
        • Update Single Document
        • Bulk Update
    • Creating a Native Enrichment Step
      • Creating an Enrichment Step
        • Creating the Core Classes
        • Extending our Enrichment Step
        • Adding a Configuration Template
        • Adding the Enrichment Step
        • Creating an Enrichment Pipeline
      • Other Tasks
        • Entities, Metadata and Data
        • Accessing the Repository
      • Example Code
      • Troubleshooting
    • Creating a Python Enrichment Step
      • Creating an Enrichment Step
        • Running the Example from Command Line
        • Running the Example
      • Creating Your Own Step
      • Adding or Changing Entities, Metadata
  • whitepapers and explainers
    • From a Billion To One – Mastering Relevancy
    • Methods for Text Summarization
      • Application
      • Technology Methods
      • Commercial Tools
      • Key Research Centres
      • Productionisation
      • Related Areas of Text Analytics
      • Conclusion
      • References
Powered by GitBook
On this page
  • Prerequisites
  • Connecting Google Drive with Aiimi Insight Engine
  • Connection
  • Security Synchronisation
  • Drives
  • Advanced
  1. Control Hub
  2. Configurations
  3. Source Configurations
  4. Source - Source

Google Drive

PreviousGoogle BucketNextGoogle Vault

Last updated 11 months ago

Connect your Google Drive system to Aiimi Insight Engine to make the most of the data.

Public and Private Google Drives must be configured separately.

Prerequisites

Your Google cloud environment must be configured to allow the connector access to various APIs, services and scopes. Before running the Google Drive Connector these 5 things must be in place.

Google Cloud Project

Aiimi Insight Engine's Google Drive Connector needs a project. A Google Cloud Project is required for Google Cloud Services such as managing APIs and resource permissions.

For information on creating a project . ()

Required APIs

Aiimi Insight Engine's Google Drive Connector requires 3 APIs to be enabled on the relevant project.

  1. Activity API

  2. Google Drive API

  3. Admin SDK API

For information on enabling APIs . ()

Service Account

A service account associated with the relevant project is needed to perform tasks for the connector. The delegated user used in conjunction with the Service Account Credentials need 2 custom roles. These roles will need relevant Admin privileges granted.

Custom role examples:

Google Drive Connector Role This can be Organisational Unit specific. Admin API Privileges - Users - Read

Google Drive Connector Groups This is for all Organisational Units. Groups are domain wide and not limited to a unit. Admin API Privileges - Groups - Read

Further Information

Any role intended to be Organisation Unit specific can only include the following privileges:

  1. Users

  2. User Security Management

  3. Organizational Units

  4. Chrome Management

  5. Shared device settings

Personal Google Drives - The delegated user will be limited to Personal Google Drives within their Organisational Unit. This ensures only the intended drives are discovered and crawled by Aiimi Insight Engine.

Shared/Team Drives - The delegated user must be a member with at least "Viewer" level access of each drive. This ensures only the intended drives are discovered and crawled by Aiimi Insight Engine.

For file actions such as Delete - The delegated user must be a "Manager" of the relevant drive. Only "Managers" are able to delete files from a Shared Google Drive. This ensures that only the Shared Drives connected to Aiimi Insight Engine can have files deleted.

For more information on service accounts . ()

API Secret Key

Your service account will require an API secret key to allow a secure connection. The secret key will be used as a credential in Aiimi Insight Engine.

For support setting up a secret only credential

We recommend you download the key as a JSON file when prompted.

Once generated your private key will be downloaded to your machine. You must store this securely as google do not store it and cannot regenerate it.

For more information on assigning keys . ()

Client Domain-Wide Delegation

To get the most out of your connection the service account must have domain-wide delegation and the correct scopes authorised.

A super admin must delegate domain-wide authority ensuring the correct Client ID is used for the service account.

The following scopes are required:

Connecting Google Drive with Aiimi Insight Engine

  1. Source System: Select Google Drive Public or Google Drive Personal from the dropdown.

Connection

  1. Select Credential: Select the credential with the service account details for your Google Drive project.

  2. Delegated User: Enter the username of the Service Account user used for domain level operations.

Security Synchronisation

  1. Security Configuration: Select the security configuration the crawler will use to synchronise Google Directory and Aiimi Insight Engine users.

Drives

Indexing

  1. Crawlable Drives List: Add all the Google Drives that will be crawled.

    • Add usernames for personal drives. For example user@domain.com.

    • Add the drive IDs for public drives.

  2. Uncrawlable Drives List: Add the Google Drives that will not be crawled.

    • Add usernames for personal drives. For example user@domain.com.

    • Add the drive IDs for public drives.

Deleting

  1. Delete Orphaned Drives: Check this to remove any orphaned Google Drives from Aiimi Insight Engine.

Advanced

Parallelism

  1. Parallel Drive Crawling: Enter the maximum number of drives that should be crawled at once.

  2. Parallel Folder Crawling: Enter the maximum number of folders, from one drive, that should be crawled at once.

  3. Parallel Folder Query: Enter the number of Elastic queries, from one drive, that should be crawled at once. This can impact the Elastic performance.

  4. Parallel Drive Deletion: Enter the number of folders that can be deleted at once. This can impact the Elastic performance.

For more information on delegating authority ()

The GoogleDirectory security must be configured before.

see Google's documentation on creating and managing projects
https://cloud.google.com/resource-manager/docs/creating-managing-projects
see Google's documentation on enabling an API in your Google Cloud project
https://cloud.google.com/endpoints/docs/openapi/enable-api
see Google's documentation on Creating a service account
https://developers.google.com/identity/protocols/oauth2/service-account#creatinganaccount
see our guide on creating secret only credentials.
see Google's documentation on Creating a service account
https://developers.google.com/identity/protocols/oauth2/service-account#creatinganaccount
https://www.googleapis.com/auth/drive
https://www.googleapis.com/auth/drive.readonly
https://www.googleapis.com/auth/admin.directory.user.readonly
https://www.googleapis.com/auth/admin.directory.group.readonly
https://www.googleapis.com/auth/drive.activity.readonly
https://www.googleapis.com/auth/admin.directory.orgunit.readonly
see Google's documentation on Delegating domain-wide authority
https://developers.google.com/identity/protocols/oauth2/service-account#delegatingauthority
See our guide on Configuring the Google Directory Security.
See our guide on Configuring the Google Directory Security.