8 SrSSKJr SSKJr SSKJr SSKrSSKrSSKrSSKJ r J r J r J r J r Jr SSKJr SSKJr S rS rS S S SSSSS.r\R+5VVs0sHupX_M snnr\R/5rS\S\4SjrS\S\4SjrS\S\4SjrS\S\S\4SjrS\S\4SjrSGSjrSSS \S\4S!jr S"\ \S\ \\ \\ \\ 4444S#jr!"S$S%\ 5r"S&\ \S"\S\"4S'jr#S(r$S)r%S*\S\ \4S+jr&S,r'S-r(S.r)S/r*SHS0\ \+S1\ \S2\ \4S3jjr,"S4S5\5r-SHS6\R\R^S \S7\ \+S1\ \S8\ \S\-4 S9jjr0SIS2\ \S\ \ \4S:jjr1SHS6\S;\ \S0\ \+S1\ \S<\ \4 S=jjr2SHS6\S0\ \+S1\ \S>\ \4S?jjr3S@\SA\ \\ 44SBjr4SC\SD\ \\ 4SA\ \\ 4SE\4SFjr5gs snnf)Jz#Bigquery Client library for Python.)absolute_import)division)print_functionN)AnyDictList NamedTupleOptional TypedDict)bq_error) bq_id_utilsizgs://cloudSqlawsazure sqlDataSource cloudSpanner cloudResourcespark) CLOUD_SQLAWSAzureSQL_DATA_SOURCE CLOUD_SPANNERCLOUD_RESOURCESPARK iam_role_idreturnc SU-$)zReturns properties for a connection with IAM role id. Args: iam_role_id: IAM role id. Returns: JSON string with properties to create a connection with IAM role id. z#{"accessRole": {"iamRoleId": "%s"}})rs 'platform/bq/utils/bq_processor_utils.pyMakeAccessRolePropertiesJsonr!)s / << tenant_idc SU-$)zReturns properties for a connection with tenant id. Args: tenant_id: tenant id. Returns: JSON string with properties to create a connection with customer's tenant id. z{"customerTenantId": "%s"}r)r#s r MakeTenantIdPropertiesJsonr%6s & 11r"federated_app_client_idc SU-$)zReturns properties for a connection with a federated app (client) id. Args: federated_app_client_id: federated application (client) id. Returns: JSON string with properties to create a connection with customer's federated application (client) id. z&{"federatedApplicationClientId": "%s"}r)r&s r +MakeAzureFederatedAppClientIdPropertiesJsonr(Ds 24K KKr"cSU<SU<S3$)a#Returns properties for a connection with tenant and federated app ids. Args: tenant_id: tenant id federated_app_client_id: federated application (client) id. Returns: JSON string with properties to create a connection with customer's tenant and federated application (client) ids. z{"customerTenantId": "z%", "federatedApplicationClientId" : "z"}r)r#r&s r 4MakeAzureFederatedAppClientAndTenantIdPropertiesJsonr*Ts r"namec4[R"SSU5$)z-Convert a name with underscores to camelcase.z_[a-z]cFURS5SR5$)Nr)groupupper)matchs r ToLowerCamel..ls Aq(9(?(?(Ar")resub)r+s r ToLowerCamelr6js A4 HHr"c PURSUR555 g)zAdds all kwds to config dict, adjusting keys to camelcase. Note this does not remove entries that are set to None, however. Args: config: A configuration dict. **kwds: A dict of keys and values to set in the config. c3J# UHupUcM [U5U4v M g7fN)r6).0kvs r "ApplyParameters..xs O$a!$s ##N)updateitems)configkwdss r ApplyParametersrCos --OOOr"project_referencez,bq_id_utils.ApiClientHelper.ProjectReferencelocationc,SUR-S-U-$)aFormats a project identifier for data transfers. Data transfer API calls take in the format projects/(projectName), so because by default project IDs take the format (projectName), add the beginning format to perform data transfer commands Args: project_reference: The project id to format for data transfer commands. location: The location id, e.g. 'us' or 'eu'. Returns: The formatted project name for transfers. z projects/z /locations/) projectId)rDrEs r #FormatProjectIdentifierForTransfersrH{s $ (22 2] BX MMr" json_stringc[R"U5$![a#n[R"SU<SU<35eSnAff=f)zAWrapper for standard json parsing, may throw BigQueryClientError.z Error decoding JSON from string : N)jsonloads ValueErrorr BigqueryClientError)rIes r ParseJsonrQsA ::k ""   & &4?C s AAAc0\rSrSr%\\\S'\\S'Srg) InsertEntry insert_idrecordrN) __name__ __module__ __qualname____firstlineno__r str__annotations__object__static_attributes__rr"r rSrSs c] .r"rSrUc[R"U5n[U[5(d[R "S5e[ X5$![a(n[R "S[U5<35eSnAff=f)zParses a JSON encoded record and returns an InsertEntry. Arguments: insert_id: Id for the insert, can be None. json_string: The JSON encoded data to be converted. Returns: InsertEntry object for adding to a table. zValue is not a JSON objectzCould not parse object: N) rLrM isinstancedictr rOrSrNr[)rUrIrowrPs r JsonToInsertEntryrcsjQ **[ !C c4  ( ()E FF y && Q  & &s1v'O PPQsA A B#A;;BcVURS05nSU;aSUS;aUSS$g)zHelper to return the session id if the job is part of one. Args: job: a job resource to get statistics and sessionInfo from. Returns: sessionId, if the job is part of a session. statistics sessionInfo sessionIdN)get)jobstatss r GetSessionIdrks; '', #%e u]/C C   ,, r"c[S5n[URS05R55RU5R 5$![ a gf=f)zHelper for job printing code.)extractloadquerycopy configurationN)setrhkeys intersectionpopKeyError)job_info job_namess r GetJobTypeNamerysX67) HLL" - 2 2 45 i   sAA A A  source_stringcURS5Vs/sHoR5PM nnUVs/sH oR[5(dMUPM" nnU(d[R "S5eU(aC[ U5[ U5:wa)[R "SR[55eU$USn[ U5S:a#[R "S[ U54-5e[RRU5(d[R "SU<35e[RRU5(d[R "SU<35eU$s snfs snf) atTake a source string and return a list of URIs. The list will consist of either a single local filename, which we check exists and is a file, or a list of gs:// uris. Args: source_string: A comma-separated list of URIs. Returns: List of one or more valid URIs, as strings. Raises: bq_error.BigqueryClientError: if no valid list of sources can be determined. ,zNo sources specifiedz(All URIs must begin with "{}" if any do.rr.z7Local upload currently supports only one file, found %dzSource file not found: zSource path is not a file: ) splitstrip startswithGCS_SCHEME_PREFIXr rOlenformatospathexistsisfile)rzsourcesourcesgs_uriss r ProcessSourcesrsK +8*=*=c*B C*B\\^*B' C" "&7&78I&Jf7     & &'= >>  7|s7|#  ( ( 4 ; ;>& ! !  ( ()/ 1  77>>& ! !  ( (-3 5  .7 D sE8E=E=c*URS5S$)z#Convert a kind to just a type name.#) partition)kinds r KindToNamers  Q r"cP[R5HupX ;dM Us $ gr9)CONNECTION_TYPE_TO_PROPERTY_MAPr@) connectiontps r GetConnectionTypers&-335da h6 r"cSU;a0[US5nUS-nX ;a[SU<SU<SU<35eO[SnUVs/sHo3RS5(dMUPM nn[U5S:wa[S[U5<SU<35eUS nUS R 5USS -n[ [ RUS 5nUc[S U<35eUR"S 0XD6$s snf) z-Construct a Reference from a server response.r Referencez Cannot find z in object of type rKr.zExpected one Reference, found rNzUnknown reference type: r) rrNendswithrr0getattrr ApiClientHelperCreate) object_infotypename lower_camelr;rs upper_camelreference_types r ConstructObjectReferencer s {+f-.H[(K%  (K 1 & H" >{!jj&=A{D > 4yA~ 47It D q'KA$$&QR8+;66 TJ. X? @@    :!9 :: ?s C,C,czURRnUSR5USS-nU[U50$)z,Construct an Object from an ObjectReference.rr.N) __class__rWlowerra) referencerrs r ConstructObjectInfor&s?  ) )( !!#hqrl2+ tI ''r" max_results page_tokenfilter_expressioncF[U5nUbXS'UbX4S'UbX$S'U$)#Create and populate a list request. maxResultsfilter pageToken)ra)rrrrrequests r PrepareListRequestr-s< O''L")H%K .r"cV\rSrSr%\\S'\\\S'\\\S'\\\\S'Sr g)TransferListRequestiCparentpageSizer dataSourceIdsrN) rWrXrYrZr[r\r intrr^rr"r rrCs* + SM c]$s)$$r"rr page_sizedata_source_idsc[[X5S9nUbX%S'UbX5S'UbPURS5nUSS:XaUSRS5nXES'U$[R"S US-5eU$) rrrr:rrr.r|zSInvalid filter flag values: '%s'. Expected format: '--filter=dataSourceIds:id1,id2')rarHr}r BigqueryError)rrErrrrs r PrepareTransferListRequestrMs  0 E '#J%K %++C0Oq_,'*005o!0o .  " " > A    .r"cUcgURS5(a$URS5SRS5$[R"SU-S-5e![a"n[R"SU-S-5UeSnAff=f) a5Parses the state filter for list jobs. Args: filter_expression: A string containing the state filter, e.g., 'state:done'. Returns: A single state filter or a list of filters to apply. Returns None if no filter is provided. Raises: bq_error.BigqueryClientError: if the filter expression is invalid. Nzstates:rr.r|Invalid flag argument ""z*, the expression must start with "states:")rr} IndexErrorr r)rrPs r ParseStateFilterExpressionrks !!),,  $ $S )! , 2 23 77  !   6 7    " " #&7 7# =  s"A B%BB run_attemptstatesc~[US9nXS'UbU[:a[nX%S'Ub[U5US'UbX5S'U$)z0Create and populate a transfer run list request.r runAttemptrrr)ra MAX_RESULTSr)rrrrrrs r PrepareTransferRunListRequestrsY  "'%,[ k%J 26:GH%K .r" message_typecP[US9nUbU[:a[nXS'UbX$S'UbLSU;a*URS5SRS5nX4S 'U$[R "S U-S -5eU$![a"n[R "S U-S -5UeSnAff=f) z0Create and populate a transfer log list request.rNrrz messageTypes:rr.r| messageTypesrr)rarr}rr r)rrrrrrPs r PrepareListTransferLogRequestrs  "'[ k%J%K,&#))#.q177< ". .  " " #l 2S 8  .$$ % 4s :  s'A99 B%B  B%paramsr@c[R"U5nX!S'U$![an[R"S5UeSnAff=f)aProcesses the params flag. Args: params: The user specified parameters. The parameters should be in JSON format given as a string. Ex: --params="{'param':'value'}". items: The body that contains information of all the flags set. Returns: items: The body after it has been updated with the params flag. Raises: bq_error.BigqueryError: If there is an error with the given params. zWParameters should be specified in JSON format when creating the transfer configuration.Nr)rLrM Exceptionr r)rr@ parsed_paramsrPs r ProcessParamsFlagrsR JJv&M "/ ,    #    s A?Arefresh_window_daysdata_source_info data_sourcecSU;a(USS:XaXS'U$[R"SU-5e[R"SU-5e)aProcesses the Refresh Window Days flag. Args: refresh_window_days: The user specified refresh window days. data_source_info: The data source of the transfer config. items: The body that contains information of all the flags set. data_source: The data source of the transfer config. Returns: items: The body after it has been updated with the refresh window days flag. Raises: bq_error.BigqueryError: If the data source does not support (custom) window days. dataRefreshTypeCUSTOM_SLIDING_WINDOWdata_refresh_window_daysz=Data source '%s' does not support custom refresh window days.z6Data source '%s' does not support refresh window days.)r r)rrr@rs r ProcessRefreshWindowDaysFlagrse***)*.EE*= &' l  " " I     @;N r")rN)NNNr9)6__doc__ __future__rrrrLrr4typingrrrr r r utilsr r rrrr@CONNECTION_PROPERTY_TO_TYPE_MAPrsCONNECTION_TYPESr[r!r%r(r*r6rCrHrQrSrcrkryrrrrrrrrrProjectReferencerrrrrr)rrs00r rs{*&% CC    &#% #5::<#(,} d3iF"& $ ##  SM ."& $"& # 3- <c$sCx.2!!38n! S>! !i #sG