How to connect to a BigQuery instance using a service account in Google Cloud Platform (GCP)

FAQs
1

How to connect to a BigQuery instance using a service account in Google Cloud Platform (GCP)

BigQuery is a fully managed enterprise data warehouse that helps manage and analyze data with built-in features like machine learning, geospatial analysis, and business intelligence. BigQuery’s serverless architecture allows using SQL queries answer important questions with zero infrastructure management by using scalable, distributed analysis engine to query terabytes in seconds and petabytes in minutes.

BigQuery interfaces include Google Cloud console interface and the BigQuery command-line tool. Developers and data scientists can use client libraries with familiar programming including Python, Java, JavaScript, and Go, as well as BigQuery’s REST API and RPC API to transform and manage data. ODBC and JDBC drivers provide interaction with existing applications including third-party tools and utilities.

For more information on BigQuery, click on the link below:

BigQuery

To connect to a BigQuery instance using a service account in Google Cloud Platform, follow the steps below:

1. Create a Service Account:

  • Navigate to the Google Cloud Console.
  • Open the project where your BigQuery instance is located.
  • In the left-hand navigation pane, go to “IAM & Admin” > “Service accounts.”
  • Click on “Create Service Account.”
  • Provide a name and description for the service account, then click “Create.”

2. Grant BigQuery Permissions:

  • In the IAM & Admin section, locate the newly created service account.
  • Click on the pencil icon in the “Actions” column to edit permissions.
  • Add the “BigQuery User” role or a custom role with the necessary BigQuery permissions.

3. Generate and Download JSON Key File:

  • Still in the service account details, click on “Add Key” > “JSON.”
  • This will download a JSON key file containing the credentials for your service account. Keep this file secure.

4. Set Up Authentication Locally:

  • Store the downloaded JSON key file in a secure location on your local machine.
  • Set the GOOGLE_APPLICATION_CREDENTIALS environment variable to point to the path of the JSON key file.
export GOOGLE_APPLICATION_CREDENTIALS="[PATH_TO_JSON_KEY_FILE]"

5. Install the BigQuery Client Library:

  • If not done already, install the BigQuery client library for the programming language. For example, following Python command can be used:
pip install --upgrade google-cloud-bigquery

6. Write Code to Connect to BigQuery:

  • Use the client library in your preferred programming language (Python, Java, etc.) to write code that connects to BigQuery using the service account.
  • Use the service account email and the path to the JSON key file for authentication.

7. Test the Connection

  • Execute your code to test the connection to BigQuery using the service account.
  • Ensure that you can successfully query or interact with your BigQuery instance.