Big Query Cataloguer

Connect a Big Query Cataloguer to Aiimi Insight Engine and make the most of your data. Once you have selected a Source System type more detail will expand to customise this.

1. Enter the Google Cloud project in Project.

   • This should be the project containing the BigQuery dataset that needs cataloguing.

2. Enter the Service Account for Google Cloud.

   • This is the client email in the JSON file that is generated when creating a service account.

   • This account must have the right access to the dataset, query tables and views, read metadata and read permissions.

3. Select the Credential to use in Select Credential.

4. Enter the Datasets to be included.

5. Enter Label Filters to filter the data set.

   • See Google Clouds documentation for syntax details.

6. Check Include hidden datasets to also catalogue hidden datasets.

7. Check Include Views to include views and tables in the catalogue.

8. Check Use TABLESAMPLE to use it when sampling data from tables.

   • Currently Alpha.

9. Enter the number of Sample Rows to use.

   • Enter the number of sample rows to extract from each table.

   • If set to 0 no sample will be generated.

   • Existing samples will be preserved where possible.

10. Enter a Max Text Length.

    • This is the maximum number of characters that will be taken from a text column.

Optional Mapping

Map the BigQuery labels for datasets.

1. Select Add New Item.

2. Enter the ID of the destination Aiimi Insight Engine database model property in the left column.

3. Enter the source BigQuery label key name in the right column.

Map the BigQuery labels for tables and views.

1. Select Add New Item.

2. Enter the ID of the destination Aiimi Insight Engine database model property in the left column.

3. Enter the source BigQuery label key name in the right column.