Skip to main content

Insert BigQuery Dataset

Creates a new empty dataset.

External Documentation

To learn more, visit the GCP documentation.

Basic Parameters​

ParameterDescription
Project IDProject ID of the new dataset.

Advanced Parameters​

ParameterDescription
Access[Optional] An array of objects that define dataset access for one or more entities. You can set this property when inserting or updating a dataset in order to control who is allowed to access the data. If unspecified at dataset creation time, BigQuery adds default dataset access for the following entities: access.specialGroup: projectReaders; access.role: READER; access.specialGroup: projectWriters; access.role: WRITER; access.specialGroup: projectOwners; access.role: OWNER; access.userByEmail: [dataset creator email]; access.role: OWNER;.
Dataset Reference Dataset ID[Required] A unique ID for this dataset, without the project name. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters.
Dataset Reference Project ID[Optional] The ID of the project containing this dataset.
Default Collation[Output-only] The default collation of the dataset.
Default Encryption Configuration Kms Key Name[Optional] Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key.
Default Partition Expiration Ms[Optional] The default partition expiration for all partitioned tables in the dataset, in milliseconds. Once this property is set, all newly-created partitioned tables in the dataset will have an expirationMs property in the timePartitioning settings set to this value, and changing the value will only affect new tables, not existing ones. The storage in a partition will have an expiration time of its partition time plus this value. Setting this property overrides the use of defaultTableExpirationMs for partitioned tables: only one of defaultTableExpirationMs and defaultPartitionExpirationMs will be used for any new partitioned table. If you provide an explicit timePartitioning.expirationMs when creating or updating a partitioned table, that value takes precedence over the default partition expiration time indicated by this property.
Default Table Expiration Ms[Optional] The default lifetime of all tables in the dataset, in milliseconds. The minimum value is 3600000 milliseconds (one hour). Once this property is set, all newly-created tables in the dataset will have an expirationTime property set to the creation time plus the value in this property, and changing the value will only affect new tables, not existing ones. When the expirationTime for a given table is reached, that table will be deleted automatically. If a table's expirationTime is modified or removed before the table expires, or if you provide an explicit expirationTime when creating a table, that value takes precedence over the default expiration time indicated by this property.
Description[Optional] A user-friendly description of the dataset.
Friendly Name[Optional] A descriptive name for the dataset.
LabelsThe labels associated with this dataset. You can use these to organize and group your datasets. You can set this property when inserting or updating a dataset. See Creating and Updating Dataset Labels for more information.
LocationThe geographic location where the dataset should reside. The default value is US. See details at https://cloud.google.com/bigquery/docs/locations.

Example Output​

{
    "access": [
        {
            "dataset": {
                "dataset": {
                    "datasetId": "[Required] A unique ID for this dataset, without the project name. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters.",
                    "projectId": "[Optional] The ID of the project containing this dataset."
                },
                "target_types": [
                    {
                        "targetType": "[Required] Which resources in the dataset this entry applies to. Currently, only views are supported, but additional target types may be added in the future. Possible values: VIEWS: This entry applies to all views in the dataset."
                    }
                ]
            },
            "domain": "[Pick one] A domain to grant access to. Any users signed in with the domain specified will be granted the specified access. Example: \"example.com\". Maps to IAM policy member \"domain:DOMAIN\".",
            "groupByEmail": "[Pick one] An email address of a Google Group to grant access to. Maps to IAM policy member \"group:GROUP\".",
            "iamMember": "[Pick one] Some other type of member that appears in the IAM Policy but isn't a user, group, domain, or special group.",
            "role": "[Required] An IAM role ID that should be granted to the user, group, or domain specified in this access entry. The following legacy mappings will be applied: OWNER  roles/bigquery.dataOwner WRITER  roles/bigquery.dataEditor READER  roles/bigquery.dataViewer This field will accept any of the above formats, but will return only the legacy format. For example, if you set this field to \"roles/bigquery.dataOwner\", it will be returned back as \"OWNER\".",
            "routine": {
                "datasetId": "[Required] The ID of the dataset containing this routine.",
                "projectId": "[Required] The ID of the project containing this routine.",
                "routineId": "[Required] The ID of the routine. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 256 characters."
            },
            "specialGroup": "[Pick one] A special group to grant access to. Possible values include: projectOwners: Owners of the enclosing project. projectReaders: Readers of the enclosing project. projectWriters: Writers of the enclosing project. allAuthenticatedUsers: All authenticated BigQuery users. Maps to similarly-named IAM members.",
            "userByEmail": "[Pick one] An email address of a user to grant access to. For example: fred@example.com. Maps to IAM policy member \"user:EMAIL\" or \"serviceAccount:EMAIL\".",
            "view": {
                "datasetId": "[Required] The ID of the dataset containing this table.",
                "projectId": "[Required] The ID of the project containing this table.",
                "tableId": "[Required] The ID of the table. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters."
            }
        }
    ],
    "creationTime": "[Output-only] The time when this dataset was created, in milliseconds since the epoch.",
    "datasetReference": {
        "datasetId": "[Required] A unique ID for this dataset, without the project name. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters.",
        "projectId": "[Optional] The ID of the project containing this dataset."
    },
    "defaultCollation": "[Output-only] The default collation of the dataset.",
    "defaultEncryptionConfiguration": {
        "kmsKeyName": "[Optional] Describes the Cloud KMS encryption key that will be used to protect destination BigQuery table. The BigQuery Service Account associated with your project requires access to this encryption key."
    },
    "defaultPartitionExpirationMs": "[Optional] The default partition expiration for all partitioned tables in the dataset, in milliseconds. Once this property is set, all newly-created partitioned tables in the dataset will have an expirationMs property in the timePartitioning settings set to this value, and changing the value will only affect new tables, not existing ones. The storage in a partition will have an expiration time of its partition time plus this value. Setting this property overrides the use of defaultTableExpirationMs for partitioned tables: only one of defaultTableExpirationMs and defaultPartitionExpirationMs will be used for any new partitioned table. If you provide an explicit timePartitioning.expirationMs when creating or updating a partitioned table, that value takes precedence over the default partition expiration time indicated by this property.",
    "defaultTableExpirationMs": "[Optional] The default lifetime of all tables in the dataset, in milliseconds. The minimum value is 3600000 milliseconds (one hour). Once this property is set, all newly-created tables in the dataset will have an expirationTime property set to the creation time plus the value in this property, and changing the value will only affect new tables, not existing ones. When the expirationTime for a given table is reached, that table will be deleted automatically. If a table's expirationTime is modified or removed before the table expires, or if you provide an explicit expirationTime when creating a table, that value takes precedence over the default expiration time indicated by this property.",
    "description": "[Optional] A user-friendly description of the dataset.",
    "etag": "[Output-only] A hash of the resource.",
    "friendlyName": "[Optional] A descriptive name for the dataset.",
    "id": "[Output-only] The fully-qualified unique name of the dataset in the format projectId:datasetId. The dataset name without the project name is given in the datasetId field. When creating a new dataset, leave this field blank, and instead specify the datasetId field.",
    "isCaseInsensitive": false,
    "kind": "bigquery#dataset",
    "labels": {},
    "lastModifiedTime": "[Output-only] The date when this dataset or any of its tables was last modified, in milliseconds since the epoch.",
    "location": "The geographic location where the dataset should reside. The default value is US. See details at https://cloud.google.com/bigquery/docs/locations.",
    "satisfiesPZS": false,
    "selfLink": "[Output-only] A URL that can be used to access the resource again. You can use this URL in Get or Update requests to the resource."
}

Workflow Library Example​

Insert Bigquery Dataset with Gcp and Send Results Via Email

Workflow LibraryPreview this Workflow on desktop