+7SrSSKJr SSKJr SSKJr SSKJr SSKrSSKrSSKrSSK J r SSK J r SS K Jr SSKJ r SS KJr SS KJr SS KJr S rSr"SS\ R.5r"SS\ R.5r"SS\ R.5r"SS\ R.5rSrSrSrSrSr Sr!Sr"Sr#Sr$S r%S!r&S"r'S*S#jr(S$r)S%r*S&r+S'r,S(r-S)r.g)+z+Declarative Hooks for BQ surface arguments.)absolute_import)division)print_function)unicode_literalsN)encoding) exceptions)util)log)yaml)timesz gcloud-bqz Invalid Schema change. Only adding additional columns or relaxing `required` fields on existing columns is supported: For more details on BigQuery schemas see: https://cloud.google.com/bigquery/docs/schemas.c\rSrSrSrSrg)PermissionsFileError,z4Error if a permissions file is improperly formatted.N__name__ __module__ __qualname____firstlineno____doc____static_attributes__r*lib/googlecloudsdk/command_lib/bq/hooks.pyrr,s>+TZZ\-=-= >& -rc"[R"S5Rn/n[R"US5nUR SS5nU(a[ U[5(d [S5eUHTn[R"X5n[U5(aURU5 M<[SRU55e [USS9$![Ran[S RU55eSnAff=f) aBuilds a bigquery AccessValueListEntry array from input file. Expects YAML or JSON formatted file. Args: input_file: input file contents from argparse namespace. Raises: PermissionsFileError: if the file contents are not a valid JSON or YAML file. Returns: [AccessValueListEntry]: Array of AccessValueListEntry messages specifying access permissions defined input file. DatasetraccessNz>Error parsing permissions file: no access list defined in filezBError parsing permissions file: invalid permission definition [{}]cUR$N)rolexs r*PermissionsFileProcessor..ss166rkeyz#Error parsing permissions file [{}])r%r&AccessValueListEntryr loadget isinstancelistrrPyValueToMessage_ValidatePermissionappendr.sortedYAMLParseError) input_fileaccess_value_msgpermissions_arraypermissions_from_file access_yaml permissionypes rPermissionsFileProcessorrSOs ++I6KK IIjm4155hE 3H$(O(O J - ,,-=Kj Z ( (  ," F:&  - #)9 ::   -44S9 sB8CD/D  DcUR=(dE UR=(d2 UR=(d UR=(d UR=(a UR nU$r:)domain userByEmail specialGroupview groupByEmailr;)permission_objis_valids rrHrHzsc%  # #%  $ $%   %  $ $     /rc[R"[RR5U5n[ [ R "UR555S-$)z;Convert commandline duration into epoch timeoffset (in ms).i)r GetDateTimePlusDurationdatetimenowinttimemktime timetuple)expire_durationts rProcessTableExpirationrfsB ##H$5$5$9$9$;_M! T[[ ' (4 //rc R[R"S5n[R"S5n[R"U5nUR SS5nU(a[ U[ 5(d [S5e/nUH2nU"USUSUR SS 5S 9nURU5 M4 U"[US S 9S 9$![Ran[SRU55eSnAf[[4an[SRU55eSnAff=f)1Convert Input JSON file into TableSchema message. TableSchemaTableFieldSchemaschemaNz?Error parsing schema file: no schema field list defined in filenametypemodeNULLABLE)rlrmrncUR$r:rlr<s rr>,BqTableSchemaFileProcessor..sQVVrr@fieldszError parsing schema file [{}]z8Error parsing schema file, invalid field definition [{}])r%r&r rCrDrErFrrIrJrKr.AttributeErrorKeyError) file_argtable_schema_typeschema_field_type schema_json all_fieldsfield new_fieldrRes rBqTableSchemaFileProcessorrs,,];,,-?@))H%K//(D1K jd;;  K J#V}V}yy,i  "  F:;K$L MM  H :AA#F GG ( # BII!L s$BCD&C44D&D!!D&c [R"S5nURn[R"S5n[R"U5nU(a[ U[ 5(d [S5e/nUH,nURU"[R"Xc5S95 M. U$![Ran[SRU55eSnAff=f)rhTableDataInsertAllRequest JsonObjectz8Error parsing data file: no data records defined in file)jsonzError parsing data file [{}]N) r%r&RowsValueListEntryr rCrErFrrIr DictToMessagerKr.)rwdata_insert_request_typeinsert_row_type data_row_type data_jsonrowsrowrRs rBqTableDataFileProcessorrs%334OP,??/((6-I (#I Jy$77  D  D kk x55cI J K  I ;BB3G HHIsA2B--C CC cLAUR(dSURlU$)zCEnsure that view parameters are set properly tables create request.N)rXtable)refargsrequests rSetViewParametersrs GMM .rcAURRRnURnUR(a[ X45(a [ X45 U$)z.Process the overwrite flag on datasets create.)datasetdatasetReference datasetId projectId overwrite_DatasetExists_TryDeleteDataset)rrr dataset_id project_ids rProcessDatasetOverwritersD //99*  * ^^j-- / .rcURnUR5nURnUR(a[ X4U5(a [ X4U5 U$)z,Process the overwrite flag on tables create.)rNamerr _TableExists_TryDeleteTable)rrrrtable_idrs rProcessTableOverwritersB}}* XXZ(}}* ^^J*55jJ7 .rcrURnURnUR(d U(dU$[UR 5R 5UR 5UR 5nURn[XVU5n[R"S5nU"US9URlU$)aSProcess schema Updates (additions/mode changes) for the request. Retrieves the current table schema for ref and attempts to merge in the schema provided in the requests. This is necessary since the API backend does not handle PATCH semantics for schema updates (e.g. process the deltas) so we must always send the fully updated schema in the requests. Args: ref: resource reference for table. args: argparse namespace for requests request: BigqueryTablesPatchRequest object Returns: request: updated requests Raises: SchemaUpdateError: table not found or invalid an schema change. rirs) r relax_columnsrk_TryGetCurrentSchemaParentrr_GetUpdatedSchemar%r&) rrrrrelaxed_columnsoriginal_schemanew_schema_columnsupdated_fieldsrxs rProcessSchemaUpdaters& --%&&/ o N( jjl388:s}}/||$?.,,];*.A'-- .rc[R"5nURn[R"S5nU"XUS9nUR U5nU(aUR S:wa[ SRU55eUR$![Ra [ SRX U55ef=f)aTry to retrieve the current BigQuery TableSchema for a table_ref. Tries to fetch the schema of an existing table. Raises SchemaUpdateError if table is not found or if table is not of type 'TABLE'. Args: dataset_id: the dataset id containing the table. table_id: the table id for the table. project_id: the project id containing the dataset and table. Returns: schema: the table schema object Raises: SchemaUpdateError: table not found or invalid table type. BigqueryTablesGetRequestrtableIdrTABLEzBSchema modifications only supported on TABLE objects received [{}]#Table with id [{}:{}:{}] not found.) r% GetApiClienttablesr&Getrmr!r.apitools_exceptionsHttpNotFoundErrorrk)rrrclientserviceget_request_type get_requestrs rrrs"  "& MM'++,FG  +  KK $E EJJ')  ++16%= *   . . -44 H  s AB0B?cU(a'URVs0sHo3RU_M snO0nU(a [X$5nU(a[URU5n[ UR 5SS9$s snf)zAUpdate original_schema by adding and/or relaxing mode on columns.cUR$r:rqr<s rr>#_GetUpdatedSchema..Dsqvvrr@)rtrl_GetRelaxedCols_AddNewColsToSchemarJvalues)r new_columnsrforig_field_maps rrr8sn6E/0010Qvvqy01"$_EN(););^LN %%'-= >>2sA9crUR5nUH nX1;a SX#lM[[5e U$)aChange mode to `NULLABLE` for columns in existing schema. Tries set mode on existing columns in orig_schema_map to `NULLABLE`. Raises SchemaUpdateError if column is not found in orig_schema_map. Args: relaxed_columns: [string] the list columns to relax required mode for. orig_schema_map: {string: TableSchemaField} map of field name to TableSchemaField objects representing the original schema. Returns: updated_schema_map: {string: TableSchemaField} map of field name to TableSchemaField objects representing the updated schema. Raises: SchemaUpdateError: if any of the fields to be relaxed are not in the original schema. ro)copyrnr!_INVALID_SCHEMA_UPDATE_MESSAGE)rorig_schema_mapupdated_schema_mapcols rrrGsC&'++- c %/" < ==  rcUR5nUH0nURU;a[[5eX2UR'M2 U$)a_Add new columns to an existing schema. Tries add new fields to an existing schema. Raises SchemaUpdateError if column already exists in the orig_schema_map. Args: new_fields: [TableSchemaField] the list columns add to schema. orig_schema_map: {string: TableSchemaField} map of field name to TableSchemaField objects representing the original schema. Returns: updated_schema_map: {string: TableSchemaField} map of field name to TableSchemaField objects representing the updated schema. Raises: SchemaUpdateError: if any of the fields to be relaxed are not in the original schema. )rrlr!r) new_fieldsrrr}s rrrcsI&'++-i~~( < ==)2y~~& rc [R"5nURn[R"S5nU"XS9nUR U5 g![ R a( [R"SRX55 gf=f)GValidate a resource of the given type with specified ID already exists.BigqueryDatasetsGetRequest)rrTz"Dataset with id [{}:{}] not found.F) r%rdatasetsr&rrrr infor.)rrrrrrs rrr~s}  "& OO'++,HI :L+ KK    . .HH,33JK  sA9B  B c$[R"5nURn[R"S5nU"XUS9nUR U5 g![ R a) [R"SRX U55 gf=f)rrrTrF) r%rrr&rrrr rr.)rrrrrrrs rrrs  "& MM'++,FG  + KK    . .HH-44 H  sA:BBc[R"5nURn[R"S5nU"XSS9nUR U5 [ R "SRX55 g)6Try to delete a dataset, propagating error on failure.BigqueryDatasetsDeleteRequestT)rrdeleteContentszDeleted dataset [{}:{}]N)r%rrr&Deleter rr.)rrrrdelete_request_typedelete_requests rrrs`  "& OO' ../NO&. .. (( $ + +J CDrc[R"5nURn[R"S5nU"XUS9nUR U5 [ R "SRX U55 g)rBigqueryTablesDeleteRequestrzDeleted table [{}:{}:{}]N)r%rrr&rr rr.)rrrrrrrs rrrsb  "& MM' ../LM& . .. (( % , ,ZX NOr)NN)/r __future__rrrrr^rar0apitools.base.pyrrrgooglecloudsdk.api_lib.bqr r%googlecloudsdk.corer r googlecloudsdk.core.utilr r/rErrorrrrr!r,r5rSrHrfrrrrrrrrrrrrrrrrrrs2&%' %>6*#$* "5=:++=8j&&8<))<( ((( !(V0 @I4  $N%P ?86"* E Pr