Google BigQuery

Prerequisites

To connect BigQuery to Gravity, you need the following:

• A BigQuery account or a Google Apps account

• Gravity account owner permission to add destinations

Setup instructions

Grant Access

1. Go to your Google Cloud Console's projects list.

2. Create a Service Account - See Google's Creating a service account documentation and save the Private key (JSON) for your service account.

3. Find your Project ID and make a note of it. Go back to the IAM & admin tab, and go to the project members list.

4. Select + Add.

5. In the New Members field, enter the name of the Service Account you created in Step 2, which is the entire email address.

6. Click Select a role > BigQuery > BigQuery User.

Set bucket access control

In the GCP console, set the access control for the bucket to Fine-grained

Assign permissions to service account

You must give the service account Storage Object Admin permission for the bucket, so that it can read and write the data from the bucket.

1. In your Google Cloud Console, go Storage > Browser to see the list of buckets in your current project.

2. Select the bucket you want to use.

3. Go to Permissions and then click Add Members.

4. In the Add members window, enter the Service Account name

5. From the Select a role dropdown, select Storage Object Admin.

Set the life cycle of objects in the bucket

You must set a lifecycle rule so that data older than one day is deleted from your bucket.

1. In your Google Cloud Console, go Storage > Browser to see the list of buckets in your current project.

2. In the list, find the bucket you are using for Gravity, and in the Lifecycle rules column, select its rules.

3. Click Add rule. A detail view will open.

4. In Select object conditions, select Age and enter 1.

5. Click Continue.

6. In Select action, select Delete.

7. Click Continue and then click Save.

Complete Gravity setup

1. Choose Service Account Key file

2. Insert Service Account Email

3. Insert Project Name

4. Set Timeout

5. Click Save, Test, and Publish

   • Test checks connectivity to BigQuery and permission to create datasets and tables on your BigQuery warehouse.

   • create a dataset to check if we have bigquery.datasets.create permissions on your data warehouse.

   • create a table in the dataset (bigquery.tables.create permissions)