h8SrSSKJr SSKJr SSKJr SSKrSSKrSSKrSSKrSSK r SSK J r SSK J r SSKJr SSKrSS KJr S rS r"S S \R*5r\"\R0\R2/5r\"\R6\R8/5r\"\Vs/sHoR<PM sn5rSr Sr!\RD"S5r#\RD"S5r$\RD"S5r%\RD"S5r&Sr'"SS\RP"\RR55r*"SS\*5r+"SS\*5r,"SS\,5r-"SS\,5r."SS \*5r/"S!S"\/5r0S#r1S*S$jr2\/Rf4S%jr4S&r5S'r6S(r7S)r8gs snf)+z*File and Cloud URL representation classes.)absolute_import)division)unicode_literalsN)errors)log) platforms)urllibz://z_.gstmpc4\rSrSrSrSrSrSrSrSr Sr S r S r g ) ProviderPrefix'z)Provider prefix strings for storage URLs.filegshdfshttphttpsposixs3N) __name__ __module__ __qualname____firstlineno____doc__FILEGCSHDFSHTTPHTTPSPOSIXS3__static_attributes__r5lib/googlecloudsdk/command_lib/storage/storage_url.pyr r 's(1 $ # $ $ % % "r"r /zblob.core.windows.netz$(?P.+)#(?P[0-9]+)$z (?P.+)#(?P.+)$z;^(?Parn:aws:s3::.+:accesspoint\/(?:.+\.mrap))z.*\/\/(?P.+)$c[RRU5=(a4 [R"[R"U5R 5$N)ospathexistsstatS_ISFIFOst_mode)r(s r# is_named_piper-Gs1   F$-- 0E0E"FFr"c\rSrSrSr\R S5r\S5r \S5r \R S5r \R S5r Sr S rS rS rS rg ) StorageUrlKz4Abstract base class for file and Cloud Storage URLs.cg)z"Returns the delimiter for the url.Nrselfs r# delimiterStorageUrl.delimiterNr"c[ezFReturns True if the URL points to a named pipe (FIFO) or other stream.NotImplementedErrorr2s r# is_streamStorageUrl.is_streamR  r"c[e)2Returns True if the URL points to stdin or stdout.r9r2s r#is_stdioStorageUrl.is_stdioWr=r"cg)2Returns the string representation of the instance.Nrr2s r# url_stringStorageUrl.url_string\r6r"cg)zFReturns the string representation of the instance without the version.Nrr2s r#versionless_url_string!StorageUrl.versionless_url_string`r6r"c[URUR5nURUR5(aUSSOUnSR X RU5n[ U5$)aEAppends part at the end of url_string. The join is performed in 3 steps: 1) Strip off one delimiter (if present) from the right of the url_string. 2) Strip off one delimiter (if present) from the left of the part. 3) Join the two strings with delimiter in between. Note that the behavior is slight different from os.path.join for cases where the part starts with a delimiter. os.path.join('a/b', '/c') => '/c' But this join method will return a StorageUrl with url_string as 'a/b/c'. This is done to be consistent across FileUrl and CloudUrl. The delimiter of the instance will be used. So, if you are trying to append a Windows path to a CloudUrl instance, you have to make sure to convert the Windows path before passing it to this method. Args: part (str): The part that needs to be appended. Returns: A StorageUrl instance. N{}{}{})rstrip_one_delimiterrGr4 startswithformatstorage_url_from_string)r3partleftrightnew_url_strings r#joinStorageUrl.joindsX0  ; ;T^^ LD77DHTE__T>>5AN "> 22r"ct[U[U55(d[$URUR:H$r&) isinstancetypeNotImplementedrD)r3others r#__eq__StorageUrl.__eq__s. eT$Z ( (  ??e.. ..r"c,[UR5$r&)hashrDr2s r#__hash__StorageUrl.__hash__s   r"cUR$r&rDr2s r#__str__StorageUrl.__str__s ??r"rN)rrrrrabcabstractpropertyr4propertyr;r@rDrGrTr[r_rcr!rr"r#r/r/Ks<--    ==QQ3:/ !r"r/c^\rSrSrSrU4SjrSr\S5r\S5r \S5r Sr S r \S 5r \S 5rS rU=r$) FileUrlaFile URL class providing parsing and convenience methods. This class assists with usage and manipulation of an (optionally wildcarded) file URL string. Depending on the string contents, this class represents one or more directories or files. Attributes: scheme (ProviderPrefix): This will always be "file" for FileUrl. bucket_name (str): None for FileUrl. resource_name (str): The file/directory path. generation (str): None for FileUrl. c|>[[U] 5 [RUlSUlSUlURS5(aU[S5SnOUn[RR5(a&URS[R5UlOX lUR#5 g)z^Initialize FileUrl instance. Args: url_string (str): The string representing the filepath. Nzfile://r$)superri__init__r rscheme bucket_name generationrMlenrOperatingSystem IsWindowsreplacer'sep resource_name$_warn_if_unsupported_double_wildcard)r3rDfilename __class__s r#rmFileUrl.__init__s  '4!# %%DKDDOY''C NO,hh   **,,#++C8d#--/r"cZUR(dgURUR-UR-nURSRURS95nSR U5nSU;a4[ R "SR[R55 gg)z4Log warning if ** use may lead to undefined results.Nz{delim}**{delim})delimz**zv** behavior is undefined if directly preceded or followed by with characters other than / in the cloud and {} locally.) rvr4splitrNrTrwarningr'ru)r3delimiter_bounded_url split_url+removed_correct_double_wildcards_url_strings r#rw,FileUrl._warn_if_unsupported_double_wildcards     NNT-?-??$..P%++!!!79I24'')2D/ :: kk GGMvffH;r"c"[R$)z8Returns the pathname separator character used by the OS.)r'rur2s r#r4FileUrl.delimiters  66Mr"cRUR=(d [UR5$r8)r@r-rvr2s r#r;FileUrl.is_streams == =M$*<*<==r"c URS:H$)r?-)rvr2s r#r@FileUrl.is_stdios    $$r"cT[RRUR5$)z*Returns True if the file/directory exists.)r'r(r)rvr2s r#r)FileUrl.existss 77>>$,, --r"cT[RRUR5$)z0Returns True if the path represents a directory.)r'r(isdirrvr2s r#r FileUrl.isdirs 77==++ ,,r"clSRURR[UR5$rCrKrNrnvalueSCHEME_DELIMITERrvr2s r#rDFileUrl.url_string- ??4;;,,.>-- //r"cUR$znReturns the string representation of the instance. Same as url_string because these files are not versioned. rbr2s r#rGFileUrl.versionless_url_string ??r"rorprvrn)rrrrrrmrwrgr4r;r@r)rrDrGr! __classcell__rys@r#riris 04(   > > % %.- / /   r"ric\^\rSrSrSrU4Sjr\S5r\S5r\S5r Sr U=r $)BaseHdfsAndPosixUrlzBase class designed for HDFS and POSIX file system URLs. Attributes: scheme (ProviderPrefix): The cloud provider, must be either POSIX or HDFS. bucket_name (str): None. resource_name (str): The file/directory path. generation (str): None. c d>[[U] 5 XlSUlSUlU[ UR[-5SUl UR[R[R4;a#[R"SUR-5eURRUR 5(dj["R$"SR'URR(URR[UR UR55 gg)z*Initialize BaseHadoopAndPosixUrl instance.NUnrecognized scheme "%s"zE{} URLs typically start at the root directory. Did you mean: {}{}{}{})rlrrmrnrorprqrrrvr rrrInvalidUrlErrorrMr4rrrNname)r3rnrDrys r#rmBaseHdfsAndPosixUrl.__init__s t-/KDDO#C (8)9%:%;//1D1DEE  " "#= #K LL    ( ( 8 8 kk fkkkknn     9r"cg)z@Returns the pathname separator character used by POSIX and HDFS.r$rr2s r#r4BaseHdfsAndPosixUrl.delimiters r"clSRURR[UR5$rrr2s r#rDBaseHdfsAndPosixUrl.url_stringrr"cUR$rrbr2s r#rG*BaseHdfsAndPosixUrl.versionless_url_stringrr"r) rrrrrrmrgr4rDrGr!rrs@r#rrsI,   / /   r"rc,^\rSrSrSrU4SjrSrU=r$)HdfsUrli#aHDFS URL class providing parsing and convenience methods. Attributes: scheme (ProviderPrefix): This will always be "hdfs" for HdfsUrl. bucket_name (str): None for HdfsUrl. resource_name (str): The file/directory path. generation (str): None for HdfsUrl. cJ>[[U] [RU5 g)z^Initialize HdfsUrl instance. Args: url_string (str): The string representing the filepath. N)rlrrmr rr3rDrys r#rmHdfsUrl.__init__-s  '4!."5"5zBr"rrrrrrrmr!rrs@r#rr#sCCr"rc,^\rSrSrSrU4SjrSrU=r$)PosixFileSystemUrli6aURL class representing local and external POSIX file systems. *Intended for transfer component.* This class is different from FileUrl in many ways: 1) It supports only POSIX file systems (not Windows). 2) It can represent file systems on external machines. 3) It cannot run checks on the address of the URL like "exists" or "is_stream" because the URL may point to a different machine. 4) The class is intended for use in "agent transfers". This is when a Transfer Service customer installs agents on one machine or multiple and uses the agent software to upload and download files on the machine(s). We implement this class in the "storage" component for convenience and because the "storage" and "transfer" products are tightly coupled. Attributes: scheme (ProviderPrefix): This will always be "posix" for PosixFileSystemUrl. bucket_name (None): N/A resource_name (str): The file/directory path. generation (None): N/A cJ>[[U] [RU5 g)zfInitialize PosixFileSystemUrl instance. Args: url_string (str): Local or external POSIX file path. N)rlrrmr rrs r#rmPosixFileSystemUrl.__init__Ns  d,^-A-A:Nr"rrrs@r#rr6s.OOr"rc^\rSrSrSrSrSU4Sjjr\SSj5rSr Sr \ S5r \ S 5r \ S 5r\ S 5r\ S 5rS rSrSrSrSrU=r$)CloudUrliWayCloud URL class providing parsing and convenience methods. This class assists with usage and manipulation of an (optionally wildcarded) cloud URL string. Depending on the string contents, this class represents a provider, bucket(s), or object(s). This class operates only on strings. No cloud storage API calls are made from this class. Attributes: scheme (ProviderPrefix): The cloud provider. bucket_name (str|None): The bucket name if url represents an object or bucket. resource_name (str|None): The object name if url represents a resource or prefix. generation (str|None): The generation number if present. r$c<>[[U] 5 U(aUOSUlU(aUOSUlU(aUOSUlU(aUOSUlU(aUOSUlU(a [ U5OSUlUR5 UR5 gr&) rlrrmrnrorvstrrp_validate_scheme_validate_resource_name)r3rnrorvrprys r#rmCloudUrl.__init__ksq (D"$"&DK&1{tD"&DK&1{tD*7TD)3c*oDO  "r"cL[U5nU[UR[-5SnUR S5(a*[ R "SR[U55eU[R:Xa[RU5OSnU(aCURS5n[RU5nU(aURS5OSnSn O=UR[5upjn[!X85upUcU(a[!UU5upiU"X6X5$)aVParse the url string and return the storage url object. Args: url_string (str): Cloud storage url of the form gs://bucket/object. is_bucket_gen_parsing_allowed (bool): If true, bucket generation parsing is allowed in the url. Returns: CloudUrl object Raises: InvalidUrlError: Raised if the url_string is not a valid cloud url. Nr$zOCloud URL scheme should be followed by colon and two slashes: "{}". Found: "{}" access_pointkey)_get_scheme_from_url_stringrqrrrMrrrNr r _S3_MRAP_ARN_REGEX_ACCESS_POINTmatchgroup_S3_MRAP_ARN_REGEX_KEY partitionCLOUD_URL_DELIMITERget_generation_number_from_name) clsrDis_bucket_gen_parsing_allowedrnschemeless_url_strings3_mrap_ap_matchros3_mrap_key_matchrvrp_s r#from_url_stringCloudUrl.from_url_stringws1) 4F's6<<:J+J'K'MN'',,  " "# F#Z 0   ^&& & (--.CD  %**>:k0667LM,=  ! !% (4j'<&E&E '#km#B #m  #@#B  #  vM >>r"crUR[;a#[R"SUR-5eg)Nr)rnVALID_CLOUD_SCHEMESrrr2s r#rCloudUrl._validate_schemes/ {{--  " "#= #K LL.r"cURS:XdURS:Xa#[R"SUR-5eg)N.z..z(%s is an invalid root-level object name.)rvrrr2s r#r CloudUrl._validate_resource_namesF S D$6$6$$>  " "#M#'#5#5$6 77%?r"cg)z@Cloud URLs cannot represent named pipes (FIFO) or other streams.Frr2s r#r;CloudUrl.is_stream r"cg)z,Cloud URLs cannot represent stdin or stdout.Frr2s r#r@CloudUrl.is_stdiorr"c2UR(a{UR5(a@SRURR[ UR UR5$SRURUR5$UR$)Nz {}{}{}#{}z{}#{})rp is_bucketrNrnrrrorGr2s r#rDCloudUrl.url_stringsr    !! KK       OO   ^^D77 II  ( ((r"c>UR5(a*SRURR[5$UR 5(a5SRURR[UR 5$UR5(a@SRURR[UR UR5$SRURR[UR UR5$)Nz{}{}z{}{}{}/z {}{}{}//{}z {}{}{}/{}) is_providerrNrnrrrrois_s3_mrap_bucketrvr2s r#rGCloudUrl.versionless_url_strings  ]]4;;,,.> ??       ++  -t/?/?    ! !  ++                  r"cFUR5(agUR$)Nz//)rCLOUD_URL_DELIMr2s r#r4CloudUrl.delimiters      r"c\[UR=(a UR(+5$r&boolrorvr2s r#rCloudUrl.is_buckets!   ;););%; <[[U] XX45 U(aUOSUlU(d[R "S5eX`lg)Nz(Azure URLs must contain an account name.)rlrrmsnapshotrraccount)r3rnrorvrprrrys r#rmAzureUrl.__init__s@ (D"]!)HdDM   " "#M NNLr"c [U5n[RX5 U[UR[ -5SnUR [5upEnUR S5un nUR [5upn U R S5upn [RRU 5n U"UUU SU ;aU SSOSSU ;a U SSUS9$SUS9$)aParses the url string and return the storage URL object. Args: url_string (str): Azure storage URL of the form: http://account.blob.core.windows.net/container/blob Returns: AzureUrl object Raises: InvalidUrlError: Raised if the url_string is not a valid cloud URL. Nr? versionIdrr)rorvrprr) rrvalidate_url_stringrqrrrrr parseparse_qs) rrDrnrhostnamerpath_and_paramsr containerblob_and_paramsblobparams params_dicts r#rAzureUrl.from_url_string$s) 4F   4 's6<<:J+J'K'LM $9#B#B$ H&&s+MGQ%4$=$=%!I/&//4ODV,,''/K  + %{+A.+/  $Z(+ +/ r"cU[;$r&)VALID_HTTP_SCHEMES)rrns r#is_valid_schemeAzureUrl.is_valid_schemeXs ' ''r"c[RUR5(d/[R"SR UR55eg)NzInvalid Azure scheme "{}")rrrnrrrNr2s r#rAzureUrl._validate_scheme\sD  # #DKK 0 0  " "#>#E#E ++$  1r"c[RU5 [U;a[RU5(d%[R"SR U55eg)NzInvalid Azure URL: "{}")rr AZURE_DOMAINrrrN)rrDrns r#rAzureUrl.validate_url_stringasK V$ J &8+C+CF+K+K  " "#<#C#CJ#O PP,Lr"cl[[RRUR55n0nUR (aUR US'UR (aUR US'[RRU5US'[RRU5$)Nrr) listr rurlsplitrGrpr urlencode urlunsplit)r3 url_partsurl_parameterss r#rDAzureUrl.url_stringgs}V\\**4+F+FGHIN $(OOn[! }}#'==nZ <<)).9IaL << " "9 --r"cUR5(a:SRURR[UR [ 5$UR5(aESRURR[UR [ UR5$SRURR[UR [ URUR5$)Nz {}{}{}.{}z {}{}{}.{}/{}z{}{}{}.{}/{}/{}) rrNrnrrrr rrorvr2s r#rGAzureUrl.versionless_url_stringss     1 13C $ l<<     " "4;;#4#46F#'<<t?O?OQQ  # #DKK$5$57G$(LL,$($4$4d6H6H JJr")rr)NNNNN)rrrrrrmrrrrrrgrDrGr!rrs@r#rrs $11f(( QQ   .  .  J  Jr"rcUR[5nUS:Xa[R$USUR 5nU[ ;a%[ R"SRU55e[U5$)z)Returns scheme component of a URL string.rzUnrecognized scheme "{}") findrr rlower VALID_SCHEMESrrrN)rDend_scheme_idx prefix_strings r#rrsn??#34.r   q0668MM)  " " $ + +M : << - ((r"cv[U5nU[R:Xa [U5$U[R:Xa [ U5$U[R :Xa [U5$U[;a[RU5$U[;a[RXS9$[R"S5e)a6Static factory function for creating a StorageUrl from a string. Args: url_string (str): Cloud url or local filepath. is_bucket_gen_parsing_allowed (bool): If true, bucket generation parsing is allowed in the url. Returns: StorageUrl object. Raises: InvalidUrlError: Unrecognized URL scheme. )rzUnrecognized URL scheme.)rr rrirrrrrrrrrrr)rDrrns r#rOrOs 'z 2& ~""" :  ~### j )) ~""" :  !!  # #J // ""  # # $  9::r"cPURU5(aUS[U5*$U$)zStrip one delimiter char from the end. Args: string (str): String on which the action needs to be performed. delimiter (str): A delimiter char. Returns: str: String with trailing delimiter removed. N)endswithrq)stringr4s r#rLrLs, __Y "C N? ## -r"cpURR[5up#[SR UR [U55nUR UR :XaU$URUR UR 5n[SR UR [U55$)aReturns best-effort new StorageUrl based on original with new scheme. This relies strongly on "storage_url_from_string" and will probably fail for unusual formats like Azure URL. However, delimiter replacement is handled for cases like converting Windows to cloud URLs. Ignores versioning info embedded in URLs because each URL type tends to have non-translatable syntax for its versions. Args: original_url (StorageUrl): URL to convert. new_scheme (ProviderPrefix): Scheme to update URL with. probably fail or have unexpected results because URL formats tend to have non-translatable versioning syntax. Returns: StorageUrl with updated scheme and best-effort transformation. rK)rGr~rrOrNrr4rt) original_url new_schemerold_url_string_no_schemeunprocessed_new_url*old_url_string_no_scheme_correct_delimiters r# switch_schemer+s&!- C C I I!!/(*B1DE2<<< /G/O/O1;;0=, (0"2 33r"cN[U;aU$UR[5u pU$)z7Removes ProviderPrefix or other scheme from URL string.)rr)rDrschemeless_urls r# remove_schemer.s,Z' #--.>?!Q r"cb[U;aU$[RR[-U-$)z;Returns a string with gs:// prefixed, if URL has no scheme.)rr rrrbs r#add_gcs_scheme_if_missingr0s-#     ! !$4 4z AAr"cU(dgU[R:Xa [nSnO!U[R:Xa [nSnOUS4$UR U5nUb%UR S5nUR U5nXV4$US4$)agParses a cloud storage resource name (bucket or object) into its components. Args: scheme (str): Scheme of URL such as gs and s3. resource_name (str): Name of the resource (bucket or object) in the format 'name' or 'name#generation'. Returns: A tuple containing the parsed resource name and generation number (or None if not present). )NNrp version_idNr)r rGS_GENERATION_REGEXr S3_VERSION_REGEXrr)rnrvpattern_to_match group_namegeneration_matchrrps r#rrs   ~!!!*J """'J $ %++M:!  ! !& )D!'' 3J    r"r)9r __future__rrrreenumr'rer*"googlecloudsdk.command_lib.storagergooglecloudsdk.corergooglecloudsdk.core.utilrsix six.movesr rTEMPORARY_FILE_SUFFIXEnumr frozensetrr rrrrrrrr compiler3r4rrr-with_metaclassABCMetar/rirrrrrrrOrrLr+r.r0r)rns0r#rFs 1&' 5#. ! TYY  !3!3^5F5F GH 3 3^5I5IJKnEnF<,4+C+C 3DB"sFs F