H7l SrSSKJr SSKJr SSKJr SSKrSSKrSSKrSSKrSSK r SSK J r J r J r JrJrJr SSKrSSKJr SSKJr SSKrSS KJr SS KJr SS KJr SS KJr SS KJr SSKJ r SSKJ!r! StS\RDS\ \#\#4S\\$S\\$4Sjjr%S\RD4Sjr&SuS\RDS\\ RNRPS\\$S\\#S\\\\#\#4S\\#S\\#S\\)S\\#4Sjjr*SuS\RDS\\ RNRPS\\$S\\#S\\\\#\#4S\\#S\\#S\\)S\\#4Sjjr+SvS\RDS \\ RNRXS!\ RNRXS"\\#S#\\#S$\\)S%\\#S&\\#4S'jjr-S\RDS\ RNR\S(\)4S)jr/SwS\RDS\ RNR\S*\\)4S+jjr0SxS\RDS,\\#S-\\#S.\\#S/\\#4 S0jjr1SyS\RDS1\#S2\\)S3\\)S4\\)S5\\#S6\\$S\\$S7\\$S8\\$S9\\$S:\\2S,\\#S;\\)S/\\#S<\\RDRfS=\\#S>\\)4$S?jjr4SzS\RDS.\\#S,\\#S\\$S7\\$S/\\#4 S@jjr5SxS\RDS,\\#S-\\#S.\\#S/\\#4 SAjjr6SzS\RDSB\\)S,\\#S-\\#S.\\#S/\\#4 SCjjr7S{S\RDS,\\#S.\\#S/\\#4SDjjr8SE\RrS4S\RDSF\ RNR\SG\#SH\$SI\\ /\Rt44 SJjjr;S|S\RDSF\ RNR\SG\#SH\$4SKjjrS}S\RDS1\#SO\\#S"\\#S#\\#SP\\#S4\\)SQ\\)S2\\)S3\\)S:\\2SR\\)SS\\$S6\\$S;\\)ST\\\#SU\\ \#\#4S8\\$S9\\$S>\\)S<\\RDRfS=\\#4,SVjjr?S~S\RDS=\\#SW\)S(\\#4SXjjr@SS\RDSY\ RNRXSZ\#S"\\#S#\\#S[\\#S\\\)S]\\#S^\\#S_\\$S`\\)Sa\\#Sb\\)Sc\\)Sd\\)Se\\\#Sf\\)ST\\\#Sg\\#Sh\\\#Si\\)Sj\\#S=\\#Sk\\)Sl\\#Sm\\A\$44SnjjrBSuS\RDS\ RNRXSo\#Sp\\)S[\\#Sq\\#Sr\\#Si\\)S=\\#4SsjjrCg)z$The BigQuery CLI job client library.)absolute_import)division)print_functionN)AnyCallableDictListOptionalUnion)flags)http)bigquery_client) table_reader)utils) wait_printer)bq_error) bq_id_utils)bq_processor_utilsbqclientjob_dict start_rowmax_rowscUc [S5eUc [S5eU(dSnO*[RRR"S0UD6n[ R R(a5U(a.[R"URURUU5nO,[R"URURU5nURX#5$)a9Convenience method to get the schema and rows from job query result. Arguments: bqclient: A BigqueryClient to get state and request clients from. job_dict: job reference dictionary. start_row: first row to read. max_rows: number of rows to read. result_first_page: the first page of the result of a query job. Returns: A tuple where the first item is the list of fields and the second item a list of rows. Raises: ValueError: will be raised if start_row is not explicitly provided. ValueError: will be raised if max_rows is not explicitly provided. Nzstart_row is requiredzmax_rows is required) ValueErrorrApiClientHelper JobReferenceCreater FLAGS$jobs_query_use_results_from_responsebq_table_readerQueryTableReader apiclientmax_rows_per_requestJobTableReaderReadSchemaAndRows)rrrrresult_first_pagejob_refreaders !platform/bq/clients/client_job.pyReadSchemaAndJobRowsr+s. , --  + ,, 8token unreachabler)bq_client_utilsNormalizeProjectReferencer typecheckrProjectReferencer MAX_RESULTSPrepareListRequest isinstancestrlowerApplyParametersr#rHr/executegetlenextenddict)rr5r6r7r8r9r:r;r<requestsresultr>responses r*r?r?sF77)!!22  '333&22k  1 1j ',$$!'')l)56Aggil6$$ ! "!2 "!2     " " $ ) ) 4G 4 < < >& JJvr "' V #G {(B)CL8gl#O4gk!!&&(--88@@Bf nnVZZ+, V #G {(B ' "(/HWf$]3H] /97s.Hsource_referencesdest_referencecreate_dispositionwrite_dispositionignore_already_existsoperation_typedestination_expiration_timec  UH1n [R"U [RRSS9 M3 [R"U[RRSS9 [ U5UV s/sHn [ U 5PM sn S.n U(aXkS'U(aX{S'U(aXS'[ R "U UUS9 [USU 040U D6$s sn f![Ran U(aS n A g U eS n A ff=f) aCopies a table. Args: bqclient: A BigqueryClient to get state and request clients from. source_references: TableReferences of source tables. dest_reference: TableReference of destination table. create_disposition: Optional. Specifies the create_disposition for the dest_reference. write_disposition: Optional. Specifies the write_disposition for the dest_reference. ignore_already_exists: Whether to ignore "already exists" errors. encryption_configuration: Optional. Allows user to encrypt the table from the copy table command with Cloud KMS key. Passed as a dictionary in the following format: {'kmsKeyName': 'destination_kms_key'} **kwds: Passed on to ExecuteJob. Returns: The job description, or None for ignored errors. Raises: BigqueryDuplicateError: when write_disposition 'WRITE_EMPTY' is specified and the dest_reference table already exists. CopyTablerB)destinationTable sourceTables"destinationEncryptionConfiguration operationTypedestinationExpirationTime)rcrdcopyN) rrPrTableReferencer\rrW ExecuteJobrBigqueryDuplicateError) rrarbrcrdreencryption_configurationrfrgr3src_ref copy_configes r*ririsF#g##22# !!00  ~.4EF4EtG}4EF+8P45#1  /J+,$$+)  h 5 > >>%G&  ( (   G s$4CCD.C<:C<<Dreturnc[R"U[RRSS9 URR 5R "S0[U5D6R5$![Ra gf=f)zReturns true if the job exists. JobExistsrBFr) rrPrrr#rHrYr\rXrBigqueryNotFoundError)rr5s r*ryry"sr ,,99+    " " $ ( ( ;4 ? ; C C EE  ' ' sAA11BBignore_not_foundc([R"U[RRSS9 URR 5R "S0[U5D6R5 g![Ra U(degf=f)aqDeletes JobReference reference. Args: bqclient: A BigqueryClient to get state and request clients from. reference: the JobReference to delete. ignore_not_found: Whether to ignore "not found" errors. Raises: BigqueryTypeError: if reference is not a JobReference. bq_error.BigqueryNotFoundError: if reference does not exist and ignore_not_found is False. DeleteJobrBNr) rrPrrr#rHdeleter\rXrrz)rr5r{s r*r}r}0sw",,99+  $$7tI7??A  ' '    sAA22BB project_id upload_filejob_idlocationcU=(d URnU(d[R"S5eUR5nUR(a[ SUR55US'SU0nU=(d UR n[U[R5(aURU5nUbXBS.nXvS'U(aXWS'SnU(aHURn [R"U5RS :XaS n [R "US U S 9nUR"R%5R'XhUS 9n U(aW (a[(R*"U 5n U $U R-5n U $)aStart a job with the given configuration. Args: bqclient: A BigqueryClient to get state and request clients from. configuration: The configuration for a job. project_id: The project_id to run the job under. If None, bqclient.project_id is used. upload_file: A file to include as a media upload to this request. Only valid on job requests that expect a media upload file. job_id: A unique job_id to use for this job. If a JobIdGenerator, a job id will be generated from the job configuration. If None, a unique job_id will be created for this request. location: Optional. The geographic location where the job should run. Returns: The job resource returned from the insert job request. If there is an error, the jobReference field will still be filled out with the job reference used in the request. Raises: bq_error.BigqueryClientConfigurationError: if project_id and bqclient.project_id are None. z(Cannot start a job without a project id.c3N# UHoRS5SSS2v M g7f)=rN) partition).0props r* StartJob..ws&'.CdsADqD!.Cs#% properties configuration)jobId projectId jobReferencerrFzapplication/octet-stream)filenamemimetype resumable)body media_bodyr)rr BigqueryClientConfigurationErrorro job_propertyr\job_id_generatorrTrNJobIdGeneratorGenerateenable_resumable_uploadsosstatst_size http_requestMediaFileUploadr#rHinsertrexecute_in_chunks_with_progressrX) rrrrrr job_request job_reference media_uploadrr]r_s r*StartJobrQsn>0X00*   3 32  $$&- "&'.6.C.C'#M,!-0+  .X..&6677 __] +F $>M"/"*J,11I ww{##q(i//+L    # # % , , : - 'Y  9 9' BF -__ F -r,querydry_run use_cachepreserve_nulls request_idmaximum_bytes_billed timeout_msjob_timeout_ms max_slotsmin_completion_ratiouse_legacy_sqljob_creation_modereservation_idcreate_sessionc FU =(d URn U (d[R"S5eSU0nU (aU US'U(aUUS'UR(a#[R "UR5US'Uc=[ RR(a[[R"55n[UU5n[R"U40SU_SU_S U_S U_S U_S U _S U _SU_SU_SU _SU_SU_SU_SU_SU_SU_SUcSO U(aSOS_6 [R"UUS9 [R"UUS9 [R "SUU U5 UR"R%5R&"SUU S.UD6R)5$)a Executes the given query using the rpc-style query api. Args: bqclient: A BigqueryClient to get state and request clients from. query: Query to execute. dry_run: Optional. Indicates whether the query will only be validated and return processing statistics instead of actually running. use_cache: Optional. Whether to use the query cache. Caching is best-effort only and you should not make assumptions about whether or how long a query result will be cached. preserve_nulls: Optional. Indicates whether to preserve nulls in input data. Temporary flag; will be removed in a future version. request_id: Optional. The idempotency token for jobs.query maximum_bytes_billed: Optional. Upper limit on the number of billed bytes. max_results: Maximum number of results to return. timeout_ms: Timeout, in milliseconds, for the call to query(). job_timeout_ms: Optional. How long to let the job run. max_slots: Optional. Cap on target rate of slot consumption by the query. min_completion_ratio: Optional. Specifies the minimum fraction of data that must be scanned before a query returns. This value should be between 0.0 and 1.0 inclusive. project_id: Project id to use. external_table_definitions_json: Json representation of external table definitions. udf_resources: Array of inline and external UDF code resources. use_legacy_sql: The choice of using Legacy SQL for the query is optional. If not specified, the server will automatically determine the dialect based on query information, such as dialect prefixes. If no prefixes are found, it will default to Legacy SQL. location: Optional. The geographic location where the job should run. connection_properties: Optional. Connection properties to use when running the query, presented as a list of key/value pairs. A key of "time_zone" indicates that the query will be run with the default timezone corresponding to the value. job_creation_mode: Optional. An option for job creation. The valid values are JOB_CREATION_REQUIRED and JOB_CREATION_OPTIONAL. reservation_id: Optional. An option to set the reservation to use when execute the job. Reservation should be in the format of "project_id:reservation_id", "project_id:location.reservation_id", or "reservation_id". create_session: Optional. True to create a session for the query. query_parameters: parameter values for use_legacy_sql=False queries. positional_parameter_mode: If true, set the parameter mode to POSITIONAL instead of the default NAMED. destination_encryption_configuration: Optional. Allows user to encrypt the table created from a query job with a Cloud KMS key. **kwds: Extra keyword arguments passed directly to jobs.Query(). Returns: The query response. Raises: bq_error.BigqueryClientConfigurationError: if project_id and bqclient.project_id are None. bq_error.BigqueryError: if query execution fails. z(Cannot run a query without a project id.rtableDefinitionsuserDefinedFunctionResourcesdefaultDatasetNrrruse_query_cacherrrr6rrr reservationrrquery_parameters$destination_encryption_configurationparameter_mode POSITIONALNAMEDconnection_properties)rz3Calling bqclient.apiclient.jobs().query(%s, %s, %s))rrr)rrr dataset_idrNGetQueryDefaultDatasetr rjobs_query_use_request_idrUuuiduuid4_GetReservationPathrrWloggingdebugr#rHrrX)rrrrrrrr6rrrrrexternal_table_definitions_json udf_resourcesrrrrrrrpositional_parameter_moderr3r]reservation_paths r*_StartQueryRpcrsj0X00*   3 32 e '$"AG .;G *+  / F F!G  EKKAATZZ\"J( $$ #0     $$0*#$ (!",P#& # *5L7),$$ %:$$Wg> --;   u 9Z9379wyr,c  U=(d URnU(d[R"S5e0n[R"UUUUUUS9 UR R 5R"S0UD6R5$)a.Waits for a query job to run and returns results if complete. By default, waits 10s for the provided job to complete and either returns the results or a response where jobComplete is set to false. The timeout can be increased but the call is not guaranteed to wait for the specified timeout. Args: bqclient: A BigqueryClient to get state and request clients from. job_id: The job id of the query job that we are waiting to complete. project_id: The project id of the query job. max_results: The maximum number of results. timeout_ms: The number of milliseconds to wait for the query to complete. location: Optional. The geographic location of the job. Returns: The getQueryResults() result. Raises: bq_error.BigqueryClientConfigurationError: if project_id and bqclient.project_id are None. z.Cannot get query results without a project id.)rrrr6rr) rrrrrWr#rHgetQueryResultsrX)rrrr6rrr3s r*GetQueryResultsr2s<0X00*   3 38  $$$      " 2 2 :T : B B DDr,c [UUUUUUS9nUSSS:wa![R"U5n[X5n[R "U5$)aStarts a job and waits for it to complete. Args: bqclient: A BigqueryClient to get state and request clients from. configuration: The configuration for a job. project_id: The project_id to run the job under. If None, bqclient.project_id is used. upload_file: A file to include as a media upload to this request. Only valid on job requests that expect a media upload file. job_id: A unique job_id to use for this job. If a JobIdGenerator, a job id will be generated from the job configuration. If None, a unique job_id will be created for this request. location: Optional. The geographic location where the job should run. Returns: job, if it did not fail. Raises: BigQueryError: if the job fails. rrrrstatusstateDONE)rrr1WaitJobrNRaiseIfJobError)rrrrrrr_rs r*RunJobSynchronouslyras_8    & Hg&(&??GM X -F  ( ( 00r,syncc Uc URnU(a[UUUUUUS9nU$[UUUUUUS9n[R"U5 U$)z,Execute a job, possibly waiting for results.r)rrrrNr)rrrrrrrjobs r*rqrqsk \ ==D   C$ *   C##C( *r,cU=(d URnU(d[R"S5eU(d[R"S5e[RR R XUS9nURR5R"S 0[U5D6R5SnUSSS:wa0UR(a[R"U5n[XS9nU$) aAttempt to cancel the specified job if it is running. Args: bqclient: A BigqueryClient to get state and request clients from. project_id: The project_id to the job is running under. If None, bqclient.project_id is used. job_id: The job id for this job. location: Optional. The geographic location of the job. Returns: The job resource returned for the job for which cancel is being requested. Raises: bq_error.BigqueryClientConfigurationError: if project_id or job_id are None. z)Cannot cancel a job without a project id.z%Cannot cancel a job without a job id.)rrrrrrr)rr)rrrrrrrr#rHcancelr\rXrrr1r)rrrrrr_s r* CancelJobrs,0X00*   3 33    3 3/ --::AA8B-&&=m)<=EEGN  Hg&(X]]&??GM X ;F -r,rrrwaitwait_printer_factoryc.[R"U[RRSS9 [R"5nSnU(aU"5nOUR 5n[ R"[ R"SS5[SSS5[ R"S55nS n S n S n X::a[XX#S 9upUS Sn S n U (aURURX5 O|[['U55HMn[R"5U- n URURX5 [R("S5 MO X::aM[+SU <35eUR-5 U$![Ra!n [R "SU 5 Sn A NSn A f[R"a!n [R "SU 5 Sn A NSn A f[R$an U (aeSn Sn A GNSn A ff=f)aPoll for a job to run until it reaches the requested status. Arguments: bqclient: A BigqueryClient to get state and request clients from. job_reference: JobReference to poll. status: (optional, default 'DONE') Desired job status. wait: (optional, default maxint) Max wait time. wait_printer_factory: (optional, defaults to bqclient.wait_printer_factory) Returns a subclass of WaitPrinter that will be called after each job poll. Returns: The job object returned by the final status call. Raises: StopIteration: If polling does not reach the desired state before timing out. ValueError: If given an invalid wait value. rrBNrrUNKNOWNF)rrrrz+Transient error during job status check: %sTz1Wait timed out. Operation not finished, in state )rrPrrtimer itertoolschainrepeatrangePollJobprintrrBigqueryCommunicationErrorrwarningBigqueryBackendErrorBigqueryServiceErrornextsleep StopIterationdone)rrrrr start_timerprinterwaits current_waitcurrent_statusin_error_staterrv_s r*rrs6!!..  yy{* #"$G++-G //aU1b!_b %,..(&Lid8}W-nn  m))<H  "4; YY[:-l mmM''F jjm - 6     ,,. *1  . .H ooCQGG  ( (H ooCQGG  ( ( n s0=9FHF55H G((H? HHc([R"U[RRSS9 [R "U5nUR R5R"S0[U5D6R5nUSSnXR:HU4$)aPoll a job once for a specific status. Arguments: bqclient: A BigqueryClient to get state and request clients from. job_reference: JobReference to poll. status: (optional, default 'DONE') Desired job status. wait: (optional, default 0) Max server-side wait time for one poll call. Returns: Tuple (in_state, job) where in_state is True if job is in the desired state. Raises: ValueError: If given an invalid wait value. rrBrrr) rrPrrrN NormalizeWaitr#rHrYr\rX)rrrrrcurrents r*rr-s*!!..   & &t ,$!%%<](;<DDF# M' "'  S !!r,c X[U5nSUS'[U40UD6n[UUSUUS9$)aHRun a query job synchronously, and return the result. Args: bqclient: A BigqueryClient to get state and request clients from. start_row: first row to read. max_rows: number of rows to read. **kwds: Passed on to Query. Returns: A tuple where the first item is the list of fields and the second item a list of rows. Trr)rr)r\Queryr+)rrrr3new_kwdsrs r*RunQueryrRsD$$Z((6 h#(##  .  r,max_single_waitc UR(d[R"S5eU(d[R"S5eUb5[RR (d[R"S5eU (aU "5nOUR 5n[R"5nSnSnSnUcSO[R"5U- nUU- nU b.[[UU 5S-5nUS:a[RnUS:a [S5eUcUnURb/Uc URnO[UR[U55n[S)0SU_S U_S U_S U_S U_S U_SU_SU _SU_SU_SU_SU _SU _SU_SU_SU_SU_UD6nU(aG[[[USS9S9S9nSU;aUSUSS S'SU;aUSUSS S'//U4$S U;a-[ R"R$R&"S)0US D6nO4UR)UR*US!5 [-UUR*UUUS"9nUS#(a=[/UU(a [U5O0SUUS$9unn0n S%U;aUS%U S%'S&U US'.nUUU4$GM![R0a!n![2R4"S(U!5 Sn!A!N3Sn!A!f[R6a!n![2R4"S(U!5 Sn!A!NfSn!A!ff=f)*a% Executes the given query using the rpc-style query api. Args: bqclient: A BigqueryClient to get state and request clients from. query: Query to execute. dry_run: Optional. Indicates whether the query will only be validated and return processing statistics instead of actually running. use_cache: Optional. Whether to use the query cache. Caching is best-effort only and you should not make assumptions about whether or how long a query result will be cached. preserve_nulls: Optional. Indicates whether to preserve nulls in input data. Temporary flag; will be removed in a future version. request_id: Optional. Specifies the idempotency token for the request. maximum_bytes_billed: Optional. Upper limit on maximum bytes billed. max_results: Optional. Maximum number of results to return. wait: (optional, default maxint) Max wait time in seconds. min_completion_ratio: Optional. Specifies the minimum fraction of data that must be scanned before a query returns. This value should be between 0.0 and 1.0 inclusive. wait_printer_factory: (optional, defaults to bqclient.wait_printer_factory) Returns a subclass of WaitPrinter that will be called after each job poll. max_single_wait: Optional. Maximum number of seconds to wait for each call to query() / getQueryResults(). external_table_definitions_json: Json representation of external table definitions. udf_resources: Array of inline and remote UDF resources. location: Optional. The geographic location where the job should run. connection_properties: Optional. Connection properties to use when running the query, presented as a list of key/value pairs. A key of "time_zone" indicates that the query will be run with the default timezone corresponding to the value. job_creation_mode: Optional. An option for job creation. The valid values are JOB_CREATION_REQUIRED and JOB_CREATION_OPTIONAL. reservation_id: Optional. An option to set the reservation to use when execute the job. Reservation should be in the format of "project_id:reservation_id", "project_id:location.reservation_id", or "reservation_id". job_timeout_ms: Optional. How long to let the job run. max_slots: Optional. Cap on target rate of slot consumption by the query. **kwds: Passed directly to ExecuteSyncQuery. Raises: bq_error.BigqueryClientError: if no query is provided. StopIteration: if the query does not complete within wait seconds. bq_error.BigqueryError: if query fails. Returns: A tuple (schema fields, row results, execution metadata). For regular queries, the execution metadata dict contains the 'State' and 'status' elements that would be in a job result after FormatJobInfo(). For dry run queries schema and rows are empty, the execution metadata dict contains statistics z7Running RPC-style query asynchronously is not supportedNo query string providedNzrequest_id is not yet supportedriz#Wait timed out. Query not finished.rrrrrrrrrrr6rrrrrrtotalBytesProcessed)r )r) statisticscacheHitr schemarRUNNING)r6rr jobComplete)rrr'errorsSUCCESS)Staterrz Transient error during query: %sr)rrBigqueryClientErrorr rrrrintminsysmaxsizerr$rr\rrrrrrrr+rrrr)"rrrrrrrr6rrrrrrrrrrrrr3rr elapsed_timercurrent_wait_msremaining_time rows_to_readr_ executionr rowsrrvs" r* RunQueryRpcrpsb   & &A    & &'A BBEKK$I$I  & &'H II"$G++-Gyy{*,-/ \='/QTYY[:5Mll*n  $c./BTIJ Q KK/ ! ABB  #  ( ( 4  !#88Lx<OPL   * "  "6     "6 *   % -L (  #8 0! "*% ( *01F*G )6 !;A*;MIl #G ,Z 8  9?9IIl #G ,X 6b)$ $ V #%55BBII ~& -  m))<C    #&    - #0D b $   v #H-& ) i((% K p  . .= oo8!<<  ( (= oo8!<<=s,9D0J*B.JL1K  L$LLdestination_tablepriorityallow_large_resultsflatten_resultsmaximum_billing_tierschema_update_optionslabelsc  XU(d[R"S5eSU0n!UR(a#[R"UR5U!S'U (aU U!S'U (aU U!S'U(a#[R "XS9n"[U"5U!S 'U(aUU!S 'U(aUU!S 'U(aURU!S '[R"U!40SU_SU_SU_SU_SU_SU _SU _SU_SU_SU_SU_SU_SU_SU_SU_SU _SU_SU_S U_6 [R"U!US!9 SU!0n$[UUS"S#9n%[R"U$UUUU%S$9 [R"U$US%9 [UU$40U D6$![R a#n#[R "SU<SU#<35eS n#A#ff=f)&a$Execute the given query, returning the created job. The job will execute synchronously if sync=True is provided as an argument or if bqclient.sync is true. Args: bqclient: A BigqueryClient to get state and request clients from. query: Query to execute. destination_table: (default None) If provided, send the results to the given table. create_disposition: Optional. Specifies the create_disposition for the destination_table. write_disposition: Optional. Specifies the write_disposition for the destination_table. priority: Optional. Priority to run the query with. Either 'INTERACTIVE' (default) or 'BATCH'. preserve_nulls: Optional. Indicates whether to preserve nulls in input data. Temporary flag; will be removed in a future version. allow_large_results: Enables larger destination table sizes. dry_run: Optional. Indicates whether the query will only be validated and return processing statistics instead of actually running. use_cache: Optional. Whether to use the query cache. If create_disposition is CREATE_NEVER, will only run the query if the result is already cached. Caching is best-effort only and you should not make assumptions about whether or how long a query result will be cached. min_completion_ratio: Optional. Specifies the minimum fraction of data that must be scanned before a query returns. This value should be between 0.0 and 1.0 inclusive. flatten_results: Whether to flatten nested and repeated fields in the result schema. If not set, the default behavior is to flatten. external_table_definitions_json: Json representation of external table definitions. udf_resources: Array of inline and remote UDF resources. maximum_billing_tier: Upper limit for billing tier. maximum_bytes_billed: Upper limit for bytes billed. use_legacy_sql: The choice of using Legacy SQL for the query is optional. If not specified, the server will automatically determine the dialect based on query information, such as dialect prefixes. If no prefixes are found, it will default to Legacy SQL. schema_update_options: schema update options when appending to the destination table or truncating a table partition. labels: an optional dict of labels to set on the query job. query_parameters: parameter values for use_legacy_sql=False queries. time_partitioning: Optional. Provides time based partitioning specification for the destination table. clustering: Optional. Provides clustering specification for the destination table. destination_encryption_configuration: Optional. Allows user to encrypt the table created from a query job with a Cloud KMS key. range_partitioning: Optional. Provides range partitioning specification for the destination table. script_options: Optional. Options controlling script execution. job_timeout_ms: Optional. How long to let the job run. continuous: Optional. Whether the query should be executed as continuous query. job_creation_mode: Optional. An option for job creation. The valid values are JOB_CREATION_REQUIRED and JOB_CREATION_OPTIONAL. reservation_id: Optional. An option to set the reservation to use when execute the job. Reservation should be in the format of "project_id:reservation_id", "project_id:location.reservation_id", or "reservation_id". If reservation_id is "none", the job will be executed without assigned reservation using the on-demand slots. **kwds: Passed on to ExecuteJob. Raises: bq_error.BigqueryClientError: if no query is provided. Returns: The resulting job info. rrrrr)rA identifierzInvalid value z for destination_table: Nrjrl scriptOptionsjobCreationModer rcrrrdrr!r"rrr#rtime_partitioning clusteringrr continuousrrange_partitioningrF)check_reservation_project)rr$rr)r) rrrrNrGetTableReference BigqueryErrorr\namerrWrrq)&rrrrcrdrrr rrrr!rrr"rrr#r$rr)rr*r,script_optionsrrrrr+rrr3 query_configr5rvr]rs& r*rr4sJX   & &'A BB5!, %4%K%K&L!"%'FL#$3@L/0!33i(,IL#$),56$2L!&7&<&:.` to the fully qualified reservation path `projects//locations//reservations/`. The special value "none" is returned as is. Args: bqclient: A BigqueryClient to get state and request clients from. reservation_id: The reservation id to convert. check_reservation_project: Whether to validate the reservation project. Returns: The fully qualified reservation path or "none" if reservation_id is "none". nonerAr&default_locationr-)rNGetReservationReferencebq_flagsLOCATIONvaluepath)rrr-r5s r*rrsN"~7 55((.. 9 )  r,destination_table_referencesourcefield_delimiterskip_leading_rowsencodingquotemax_bad_recordsallow_quoted_newlines source_formatallow_jagged_rows!preserve_ascii_control_charactersignore_unknown_valuesprojection_fields autodetect null_marker null_markersuse_avro_logical_typesjson_extensioncopy_files_onlysource_column_matchtimestamp_target_precisionc, [R"U[RR5 S[ U50n-[ R "U5n.U.SR[ R5(aU.U-S'Sn/OU.Sn/UbS[R"U50U-S'UbUU-S'UbUU-S'U$bU$U-S 'UbUU-S 'UbUU-S 'U&bU&U-S 'UU-S 'U(aUU-S'UbUU-S'U bU U-S'U!bU!U-S'U"bU"U-S'U#bU#U-S'U*bU*U-S'U+bU+U-S'[ R"U-40SU_SU_SU_SU_SU_SU _SU _SU _SU _SU _S U_S!U_S"U_S#U_S$U_S%U_S&U_S'U_S(U_S)U_S*U_S+U%_S,U'_S-U)_S.U&_6 S/U-0n0U(bB[R"UU([RRS0S19n1U1R!5U0S2'[#U4U0U/S3.U,D6$)4aLoad the given data into BigQuery. The job will execute synchronously if sync=True is provided as an argument or if bqclient.sync is true. Args: bqclient: A BigqueryClient to get state and request clients from. destination_table_reference: TableReference to load data into. source: String specifying source data to load. schema: (default None) Schema of the created table. (Can be left blank for append operations.) create_disposition: Optional. Specifies the create_disposition for the destination_table_reference. write_disposition: Optional. Specifies the write_disposition for the destination_table_reference. field_delimiter: Optional. Specifies the single byte field delimiter. skip_leading_rows: Optional. Number of rows of initial data to skip. encoding: Optional. Specifies character encoding of the input data. May be "UTF-8" or "ISO-8859-1". Defaults to UTF-8 if not specified. quote: Optional. Quote character to use. Default is '"'. Note that quoting is done on the raw binary data before encoding is applied. max_bad_records: Optional. Maximum number of bad records that should be ignored before the entire job is aborted. Only supported for CSV and NEWLINE_DELIMITED_JSON file formats. allow_quoted_newlines: Optional. Whether to allow quoted newlines in CSV import data. source_format: Optional. Format of source data. May be "CSV", "DATASTORE_BACKUP", or "NEWLINE_DELIMITED_JSON". allow_jagged_rows: Optional. Whether to allow missing trailing optional columns in CSV import data. preserve_ascii_control_characters: Optional. Whether to preserve embedded Ascii Control characters in CSV import data. ignore_unknown_values: Optional. Whether to allow extra, unrecognized values in CSV or JSON data. projection_fields: Optional. If sourceFormat is set to "DATASTORE_BACKUP", indicates which entity properties to load into BigQuery from a Cloud Datastore backup. autodetect: Optional. If true, then we automatically infer the schema and options of the source files if they are CSV or JSON formats. schema_update_options: schema update options when appending to the destination table or truncating a table partition. null_marker: Optional. String that will be interpreted as a NULL value. null_markers: Optional. List of strings that will be interpreted as a NULL value. time_partitioning: Optional. Provides time based partitioning specification for the destination table. clustering: Optional. Provides clustering specification for the destination table. destination_encryption_configuration: Optional. Allows user to encrypt the table created from a load job with Cloud KMS key. use_avro_logical_types: Optional. Allows user to override default behaviour for Avro logical types. If this is set, Avro fields with logical types will be interpreted into their corresponding types (ie. TIMESTAMP), instead of only using their raw types (ie. INTEGER). reference_file_schema_uri: Optional. Allows user to provide a reference file with the reader schema, enabled for the format: AVRO, PARQUET, ORC. range_partitioning: Optional. Provides range partitioning specification for the destination table. hive_partitioning_options: (experimental) Options for configuring hive is picked if it is in the specified list and if it supports the precision and the scale. STRING supports all precision and scale values. If none of the listed types supports the precision and the scale, the type supporting the widest range in the specified list is picked, and if a value exceeds the supported range when reading the data, an error will be returned. This field cannot contain duplicate types. The order of the decimal_target_types: (experimental) Defines the list of possible SQL data types to which the source decimal values are converted. This list and the precision and the scale parameters of the decimal field determine the target type. In the order of NUMERIC, BIGNUMERIC, and STRING, a type is picked if it is in the specified list and if it supports the precision and the scale. STRING supports all precision and scale values. If none of the listed types supports the precision and the scale, the type supporting the widest range in the specified list is picked, and if a value exceeds the supported range when reading the data, an error will be returned. This field cannot contain duplicate types. The order of the types in this field is ignored. For example, ["BIGNUMERIC", "NUMERIC"] is the same as ["NUMERIC", "BIGNUMERIC"] and NUMERIC always takes precedence over BIGNUMERIC. Defaults to ["NUMERIC", "STRING"] for ORC and ["NUMERIC"] for the other file formats. json_extension: (experimental) Specify alternative parsing for JSON source format. To load newline-delimited JSON, specify 'GEOJSON'. Only applicable if `source_format` is 'NEWLINE_DELIMITED_JSON'. column_name_character_map: Indicates the character map used for column names. Specify 'STRICT' to use the latest character map and reject invalid column names. Specify 'V1' to support alphanumeric + underscore and name must start with a letter or underscore. Invalid column names will be normalized. Specify 'V2' to support flexible column name. Invalid column names will be normalized. file_set_spec_type: Set how to discover files for loading. Specify 'FILE_SYSTEM_MATCH' (default behavior) to expand source URIs by listing files from the underlying object store. Specify 'NEW_LINE_DELIMITED_MANIFEST' to parse the URIs as new line delimited manifest files, where each line contains a URI (No wild-card URIs are supported). thrift_options: (experimental) Options for configuring Apache Thrift load, which is required if `source_format` is 'THRIFT'. parquet_options: Options for configuring parquet files load, only applicable if `source_format` is 'PARQUET'. connection_properties: Optional. ConnectionProperties for load job. reservation_id: Optional. An option to set the reservation to use when execute the job. Reservation should be in the format of "project_id:reservation_id", "project_id:location.reservation_id", or "reservation_id". copy_files_only: Optional. True to configures the load job to only copy files to the destination BigLake managed table, without reading file content and writing them to new files. source_column_match: Optional. Controls the strategy used to match loaded columns to the schema. timestamp_target_precision: Precision (maximum number of total digits in base 10) for second of TIMESTAMP type. Available for the formats: CSV. **kwds: Passed on to ExecuteJob. Returns: The resulting job info. rjr sourceUrisNfieldsr useAvroLogicalTypesreference_file_schema_urifileSetSpecType jsonExtensioncolumnNameCharacterMapparquetOptionsdecimalTargetTypesrltimeZone dateFormatdatetimeFormat timeFormattimestampFormatsourceColumnMatchtimestampTargetPrecisionrcrdr>r?r@rArBrDrCrErFrGrHr#rJrKr)r*rIr,hive_partitioning_optionsthrift_optionsrrNparquet_optionsloadFr5r)rr)rrPrrpr\rProcessSources startswithGCS_SCHEME_PREFIXrN ReadSchemarWr7r8r9r:r;rq)2rr<r=r rcrdr>r?r@rArBrCrDrErFrGrHrIr#rJrKr)r*rrLrUr,rbdecimal_target_typesrMcolumn_name_character_map time_zone date_formatdatetime_format time_formattimestamp_formatfile_set_spec_typercrdrrrNrOrPr3 load_configsourcesrrr5s2 r*Loadrts F!;#>#>#M#M$T*E%FG+  - -f 5' QZ-??@@ 'K K!*K %'A'A&'IJK')?K%&*/HK+,#%7K!"#1K *,EK() $3K !&:+"#),45'K  +K  $3K ! +K !%5K!"$':K#$+.HK*+$$+*&  *    &"2*)J2*2 !" #$*%&'()*,+,!:-.$/0212&34&58;'-77!!**00"' I $->>#3M-  + HL r,destination_uris print_headerdestination_format compressionc [R"U[RR[RR4SS9 UR S5n U H[n U R [R5(aM)[R"SRU [R55e 0n[U[RR5(aS[U50nOn[U[RR5(aES[U50nU(aURSSU005 U(aURSS U005 [R"UU UUUUU S 9 S U0nU bB[ R""UU [$R&R(S S 9nUR+5US'[-U4SU0U D6$)aExtract the given table from BigQuery. The job will execute synchronously if sync=True is provided as an argument or if bqclient.sync is true. Args: bqclient: A BigqueryClient to get state and request clients from. reference: TableReference to read data from. destination_uris: String specifying one or more destination locations, separated by commas. print_header: Optional. Whether to print out a header row in the results. field_delimiter: Optional. Specifies the single byte field delimiter. destination_format: Optional. Format to extract table to. May be "CSV", "AVRO" or "NEWLINE_DELIMITED_JSON". trial_id: Optional. 1-based ID of the trial to be exported from a hyperparameter tuning model. add_serving_default_signature: Optional. Whether to add serving_default signature for BigQuery ML trained tf based models. compression: Optional. The compression type to use for exported files. Possible values include "GZIP" and "NONE". The default value is NONE. use_avro_logical_types: Optional. Whether to use avro logical types for applicable column types on extract jobs. reservation_id: Optional. An option to set the reservation to use when execute the job. Reservation should be in the format of "project_id:reservation_id", "project_id:location.reservation_id", or "reservation_id". **kwds: Passed on to ExecuteJob. Returns: The resulting job info. Raises: bq_error.BigqueryClientError: if required parameters are invalid. ExtractrB,z2Illegal URI: {}. Extract URI must start with "{}". sourceTable sourceModelmodelExtractOptionstrialIdaddServingDefaultSignature)rurwrvr>rxrLextractFr5rr)rrPrrpModelReferencesplitrgrrhrrformatrTr\updaterWrNr7r8r9r:r;rq)rr5rurvr>rwtrial_idadd_serving_default_signaturerxrLrr3urisuriextract_configrs r*rzrzs`  % % 4 4  % % 4 4     $$ c >>,>> ? ?  ( ( > E E%77  . ;66EEFF#T)_5N)[88GGHH#T)_5N2Y4IJK$ *,I"  $$+%3n--77!!**00"' I $->>#3M- H BM BT BBr,r.)NNNNNNNN)NNFNCOPYN)F)NNNN)NNNNNNNNNNNNNNNNNNNNNN)NNNNN)NNN)rr)NNNNNNNNNNNNNNNNNNNNNNNNNNNNNN)T))NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN)D__doc__ __future__rrrrrrrrtypingrrrr r r rabslr googleapiclientr rr8clientsrrr!rrNrrrrBigqueryClientrUrr+r4rrQboolr2r?rprirryr}rfloatJobCreationModerrrrqrr WaitPrinterrrrrrrr/rtrzrr,r*rs *&% == 0#3, $ ,,,,38n,},sm ,^/88IM!% $48'+'+ $#'+,,+ 33DDE+#+ + 5cC01 +  } + }+~+C=+`IM!% $48'+'+ $#'R,,R 33DDER#R R 5cC01 R  } R }R~RC=Rr)-'+,1!$*15F ,,F K77FFGF  //>>F ! F  } F $D> F SMF "*#F R ,, **77   "(- ,, **77 tn H!%!% " I,,I I# I SM I sm I^# $%) $*.!% $$(#,0 $$(%)" $(%)")-5R,,R Rd^R~ R TN R  R#3-R#R RSMR}R#5/R R TN!R"sm#R& &&66'R,SM-R.TN/Rn! $!% $" ,E,,,E SM,E ,E# ,E  ,E sm ,Ed!%!% " '1,,'1 '1# '1 SM '1 sm '1Z $!% "  ,,   4.    #  SM  sm  J!% " ),,) ) SM)sm )^  P ,,P ..;;P  P   P #\---. P l ",,"..;;" "  "J,,B# $%) $*.!% ,0 %)$(" $($(#1A=,,A= A=d^A=~ A= TN A=  A=#3-A=#A= A=#5/A=#\---.A=c]A="sm#A=& &&66'A=,SM-A=.SM/A=0}1A=N(,(,'+"%)*." $,0&*$(*.*.%)15'+)-$(#%) $(Es/,,s/ s/ }s/! s/  } s/ sm s/TNs/"$s/d^s/~s/#5/s/d^s/#3-s/ #3-!s/"TN#s/$$DI.%s/& T#s(^ $'s/4SM5s/6}7s/8TN9s/> &&66?s/DSMEs/r'+,,SM $c] @ (,'+%)(,"%),0#'(,8<,0-1!%15!%(,)--1""$("$(&*)-6:Yx,,x!,!xC=x ~x(0~x $D>!x" S *#x$%x&$DI.'x(#)x*49%+x2%TN3x<SM=xRSMSxTd^UxV"#WxX!)c 3YxB$(%)(, "&!%-1$(_C,,_C**99_C_C4. _C c] _C ! _C#_C%TN_CSM_Cr,