I am trying to setup a bigquery data transfer configuration using terraform. I am using my personal gcp account. I have a setup of terraform on my laptop so that terraform and gcp can work together.

Trying this below code in main.tf,

terraform { required_providers { google = { source = "hashicorp/google" version = "4.18.0" } } } provider "google" { # Configuration options project="gcp-project-100" region="us-central1" zone="us-central1-a" credentials = "keys.json" } data "google_project" "project" { } resource "google_project_iam_member" "permissions" { project = data.google_project.project.project_id role = "roles/iam.serviceAccountShortTermTokenMinter" member = "serviceAccount:service-${data.google_project.project.number}@gcp-sa-bigquerydatatransfer.iam.gserviceaccount.com" } resource "google_bigquery_data_transfer_config" "query_config" { depends_on = [google_project_iam_member.permissions] display_name = "my-query" location = "US" data_source_id = "scheduled_query" schedule = "every wednesday 09:30" service_account_name = "service-${data.google_project.project.number}@gcp-sa-bigquerydatatransfer.iam.gserviceaccount.com" destination_dataset_id = "practice" params = { destination_table_name_template = "test_gsod" write_disposition = "WRITE_TRUNCATE" query = "select station_number , year , month,day, mean_temp,mean_dew_point ,mean_visibility from `bigquery-public-data.samples.gsod`" } } 

terraform apply is failing with below details

google_bigquery_data_transfer_config.query_config: Creating... ╷ │ Error: Error creating Config: googleapi: Error 403: The caller does not have permission │ │ with google_bigquery_data_transfer_config.query_config, │ on main.tf line 27, in resource "google_bigquery_data_transfer_config" "query_config": │ 27: resource "google_bigquery_data_transfer_config" "query_config" {

Can someone please help me , how to do this setup properly.

1 Answer

The issue is resolved now. I have used below piece of code along with bigquery admin role for my terraform service account.

 terraform { required_providers { google = { source = "hashicorp/google" version = "4.18.0" } } } provider "google" { # Configuration options project="gcp-project-100" region="us-central1" zone="us-central1-a" credentials = "keys.json" } resource "google_bigquery_data_transfer_config" "query_config" { display_name = "my-query" location = "US" data_source_id = "scheduled_query" schedule = "every 15 mins" destination_dataset_id = "practice" params = { destination_table_name_template = "test_gsod" write_disposition = "WRITE_TRUNCATE" query = "select station_number , year , month,day, mean_temp,mean_dew_point ,mean_visibility from `bigquery-public-data.samples.gsod`" } } Now it is working fine. Thanks. 

ncG1vNJzZmirpJawrLvVnqmfpJ%2Bse6S7zGiorp2jqbawutJobmtoYmuEcYGOoJytrJmjtG6x0aumq2WnnbatsYytqbKhnpx6tbuMnKapsV2ZrrWtjK6qoqaXYrSwu8alnGaamZy%2BtrHRsmSdmaSWerW%2BwKeqn52iYrCwusU%3D