Integration for Google BigQuery

Automating your privacy operations by integrating MineOS with Google BigQuery

This integration allows you to:

• Automate data subject requests (DSR) for user data from Google BigQuery.
• Preview and search user data in BigQuery.
• Perform data classification to detect data types stored in your BigQuery tables.

Before you start

• Make sure your MineOS plan supports integrations.
• Make sure you have access to Google Cloud Platform and Google BigQuery.

Setting up

To connect the BigQuery integration, follow these steps:

1. On the left sidebar, click Data Inventory and then Data Sources
2. Click on Add data source
3. Select Google Big Query from the catalog, then enter the Google Big Query page from your data sources list
4. In the RequestHandling tab, check the Handle this data source in privacy requests checkbox and choose the Integration handling style.
5. Click "Connect" and follow the on-screen instructions for granting the required permissions.
6. Some organizations have a session timeout enabled for Google Cloud scopes. To avoid session timeout for this integration, you need to mark it as a Trusted App, as well as exempt trusted apps from session timeout. Refer to Google's article for help.
   1. Note: failing to do so will expire this integration and you will have to reconnect it every time.
7. Enter your Google Cloud Project ID, click Test & Save

Optional: Defining queries for DSR flow

To manage your DSR handling actions, you need to define the queries that will run in Big Query for each possible action:

Optional: Preview Query

The Preview query will be used when opening the ticket processing screen and is responsible for showing how many records were found as well as showing a sample (preview) of the data.

Example Query:

SELECT name, age, country FROM `dataset.table` WHERE email = {{endUserEmail}} 

* Preview will return the count of the number of records returned in the query response

* Preview will show the first 3 values from the query response, it supports strings and long types

* The variable is mandatory

Optional: Copy Query

The Copy query will be used for the Copy action. Copy action is running on the ticket processing page in ticket of type Copy when clicking on Generate Copy

Example Query:

SELECT * FROM `dataset.table` WHERE email = {{endUserEmail}} 

* Copy will show all records returned in the query response

* The variable is mandatory

Required: Delete Query

The Delete query will be used for the Delete action. Delete action is running on the ticket processing page in ticket of type Deletion when clicking on Delete from X sources

Example Query:

DELETE FROM `dataset.table` WHERE email = {{endUserEmail}} 

* The variable is mandatory

Paste the project id and queries in the Request handling tab in the correct inputs and click Save

Working with custom identifiers

If you wish to use this integration with a user identifier that is not email, you will need to provide it for each request by making an UpdateMetadata API call.

When making the API call, make sure to use the following field name: $googlebq

"customFields": {
    "$googlebq": "1234567"
  }

You can use any value you like, and it will be used to replace the {endUserName} parameter when running queries.

Note: you still need to use the {endUserName} parameter in queries.

Permissions required to use this integration

Mine will request the following BigQuery permissions when you activate this integration:

What's next?

Read more about the deletion process using integrations here.

Read more about the get a copy process using integrations here.

Talk to us if you need any help with Integrations via our chat or at portal@saymine.com, and we'll be happy to assist!🙂