CYSrSSKJr SSKJr SSKJr SSKrSSKrSSKrSSKJ r SSKJ r SSKJ r SS K Jr SSKJ r SSKJ r SS KJr SSKJ r SS KJr SS KJr SS KJr SSKJr SSKr"SS\R:5r"SS\5r"SS\5r"SS\5r "SS\5r!Sr"Sr#"SS\$5r%g)aUtilities for interacting with Google Cloud Storage. This makes use of both the Cloud Storage API as well as the gsutil command-line tool. We use the command-line tool for syncing the contents of buckets as well as listing the contents. We use the API for checking ACLs. )absolute_import)division)unicode_literalsN) exceptions) list_pager)transfer) storage_util)iam_util)log) properties) transports)scaled_integerc\rSrSrSrSrg)Error0z&Base exception for storage API module.N__name__ __module__ __qualname____firstlineno____doc____static_attributes__r1lib/googlecloudsdk/api_lib/storage/storage_api.pyrr0s.rrc\rSrSrSrSrg)BucketNotFoundError4z6Error raised when the bucket specified does not exist.rNrrrrrr4s>rrc\rSrSrSrSrg)ListBucketError8zFError raised when there are problems listing the contents of a bucket.rNrrrrr r 8sNrr c\rSrSrSrSrg) UploadError<z5Error raised when there are problems uploading files.rNrrrrr#r#<s=rr#c\rSrSrSrSrg)BucketInWrongProjectError@a<Error raised when a bucket exists in a project the user doesn't own. Specifically, this applies when a command creates a bucket if it doesn't exist, or returns the existing bucket otherwise. If the bucket exists but is owned by a different project, it could belong to a malicious user squatting on the bucket name. rNrrrrr&r&@srr&cH[R"U5upU=(d S$)Nzapplication/octet-stream) mimetypes guess_type) local_path mime_type_s r _GetMimetyper.Js %%j1,)  000rc[RR[R"U55$![R a& [ R"SRU55ef=f)Nz![{0}] not found or not accessible) ospathgetsizesix ensure_strerrorrBadFileExceptionformat)r+s r _GetFileSizer8OsV; 77??3>>*5 66 ;  % %&I'-vj'9 ;;;s 25:A/c\rSrSrSrSSjrSrSrSrSr S r S r SS jr S r SS jrSSjrSrSSjrSrSrSrSrSrSrSrg) StorageClientWz$Client for Google Cloud Storage API.NcU=(d [R"5UlU=(d [R"5UlgN)r GetClientclient GetMessagesmessages)selfr?rAs r__init__StorageClient.__init__Zs,4L224DK: 8 8 :DMrcSn[R"[RRR R 55nUS:XaSnU$X!-S:wa X!X!-- - nU$)zReturns the property defined chunksize corrected for server granularity. Chunk size for GCS must be a multiple of 256 KiB. This functions rounds up the property defined chunk size to the nearest chunk size interval. irN)r ParseIntegerr VALUESstorageupload_chunk_sizeGet)rBgcs_chunk_granularity chunksizes r _GetChunkSizeStorageClient._GetChunkSize^so '++!!33779;IA~i   *a /I,MNNi rc## URRUS9n[R"URR USS9HnUv M g7f)z3List the buckets associated with the given project.)projectN batch_size)rAStorageBucketsListRequestr YieldFromListr?buckets)rBrPrequestbs r ListBucketsStorageClient.ListBucketsmsImm55g5FG  % %dkk&9&9&-$@ g@sAAc URRRURR UR UR UR UR S95$)zCopy one GCS object to another. Args: src: Resource, the storage object resource to be copied from. dst: Resource, the storage object resource to be copied to. Returns: Object, the storage object that was copied to. ) sourceBucket sourceObjectdestinationBucketdestinationObject)r?objectsCopyrAStorageObjectsCopyRequestbucketobject)rBsrcdsts rr`StorageClient.CopytsT ;;   # # //!jj!jj 0   rc .SnURRRURR UR UR UR UR US95nUR(a UR$URnM)aiRewrite one GCS object to another. This method has the same result as the Copy method, but can handle moving large objects that may potentially timeout a Copy request. Args: src: Resource, the storage object resource to be copied from. dst: Resource, the storage object resource to be copied to. Returns: Object, the storage object that was copied to. N)r[r\r]r^ rewriteToken) r?r_RewriterAStorageObjectsRewriteRequestrbrcdoneresourcerh)rBrdre rewrite_tokenresps rriStorageClient.RewritesM  [[ ( ( -- 4 4:::: # # ( 5  d }}''m rcURRRURR UR UR S95$)zGets an object from the given Cloud Storage bucket. Args: object_ref: storage_util.ObjectReference, The user-specified bucket to download from. Returns: Object: a StorageV1 Object message with details about the object. rbrc)r?r_rJrAStorageObjectsGetRequestrbrcrB object_refs r GetObjectStorageClient.GetObjectsJ ;;   " "4==#I#I    $J$" ##rc [U5nURRUS9n[U5nUR 5n[ R R[R"U5XVS9nURRURURUS9nSRURURS9n [R"SRUU S95 UR R"R%XS9n UR8R;5 U R<U:waU[R."SRU R<U55 [>R@"SRU55eU $![&R(a# [+S RURS 95e[&R,atn [R."S RX[0R2"U 5S 95 [5S RU R6X[0R2"U SS9S95eSn A ff=f!UR8R;5 f=f)aUpload a file to the GCS results bucket using the storage API. Args: local_path: str, the path of the file to upload. File must be on the local filesystem. target_obj_ref: storage_util.ObjectReference, the path of the file on GCS. Returns: Object, the storage object that was copied to. Raises: BucketNotFoundError if the user-specified bucket does not exist. UploadError if the file upload is not successful. exceptions.BadFileException if the uploaded file size does not match the size of the local file. )size)r,rL)rbnamercz{bucket}/{target_path})rb target_pathz#Uploading [{local_file}] to [{gcs}] local_filegcs)uploadz8Could not upload file: [{bucket}] bucket does not exist.rbz4Could not upload file [{local_file}] to [{gcs}]: {e})r|r}ezA{code} Could not upload file [{local_file}] to [{gcs}]: {message}{status_message} error_format)coder|r}messageNz6Response size: {0} bytes, but local file is {1} bytes.zYCloud storage upload failure. Uploaded file does not match local file: {0}. Please retry.)!r8rAObjectr.rMrUploadFromFiler3r4StorageObjectsInsertRequestrbrcr7r infor?r_Insertapi_exceptionsHttpNotFoundErrorr HttpErrordebughttp_exc HttpExceptionr# status_codestreamcloserxrr6) rBr+target_obj_ref file_sizesrc_objr,rLr~ insert_reqgsc_pathresponseerrs r CopyFileToGCSStorageClient.CopyFileToGCSs:"Z(Imm"" "2GZ(I""$I __ % % z"i&NF::$$  " ";J (..$$.2G2G/HHH 2 9 9Z>F:HI$$++J+Fh$ mm}} ! iiHOO --$%  ' ' %%+VJ%7 99 O3  + +1  D 6..6 / 11  # #> iiFMM  " "3 'N)*  M 6s:"00(:<  = >> > mms&*#FA I A/II  II+c UR5n[RRX$US9n[R "SS9UlURRURURS9nSRURURS9n[R"SRX'S95 URRR!XeS 9 URRR!U5nUR.R15 [3U5n UR4U :waT[R6"S RXR455 [&R("S RU55eg!["R$a=n [&R("S RX'[*R,"U 5S 95eSn A ff=f!UR.R15 f=f)aDownload a file from the given Cloud Storage bucket. Args: source_obj_ref: storage_util.ObjectReference, the path of the file on GCS to download. local_path: str, the path of the file to download to. Path must be on the local filesystem. overwrite: bool, whether or not to overwrite local_path if it already exists. Raises: BadFileException if the file download is not successful. )rL overwriteNresponse_encodingrqz{bucket}/{object_path})rb object_pathz%Downloading [{gcs}] to [{local_file}]r{downloadz=Could not copy [{gcs}] to [{local_file}]. Please retry: {err})r|r}rz9Download size: {0} bytes, but expected size is {1} bytes.zhCloud Storage download failure. Downloaded file [{0}] does not match Cloud Storage object. Please retry.)rMrDownloadrr GetApitoolsTransport bytes_httprArrrbrcr7r rr?r_rJrrrr6rrrrr8rxr) rBsource_obj_refr+rrLrget_reqrrrrs rCopyFileFromGCSStorageClient.CopyFileFromGCSs""$I  ))9*>H$99 Hmm44$$$$5&G(..$$.2G2G/HHH 4 ; ;<-.  kkg9$$((1hooZ(I}} ! ii ==9;  ' ' 66AFG"%8GG""G%%Hc X[R"5nUR5n[RR X#S9n[ R"SS9UlURRURURS9n[R"SU5 URR R#XTS9 UR3S 5 U$![$R&a=n[(R*"SR-U[.R0"U5S95eSnAff=f) zRead a file from the given Cloud Storage bucket. Args: object_ref: storage_util.ObjectReference, The object to read from. Raises: BadFileException if the file read is not successful. Returns: file-like object containing the data read. )rLNrrqz Reading [%s]rz/Could not read [{object_}]. Please retry: {err})object_rr)ioBytesIOrMrr FromStreamr rrrArrrbrcr rr?r_rJrrrr6r7rrseek)rBrtdatarLrrrs r ReadObjectStorageClient.ReadObject"s ::# UHoRT:Hv M g7fr=)id).0rWrbs r 8StorageClient.CreateBucketIfNotExists..s9'8!ttv~'8szEUnable to create bucket [{}] as it already exists in another project.)r rGcorerPrJr?rUrArrrStorageBucketsInsertRequestBucketIamConfigurationValueUniformBucketLevelAccessValueuniformBucketLevelAccesspublicAccessPreventionrbiamConfigurationcorsSoftDeletePolicyValuesoftDeletePolicyrHttpConflictErrorrSListanyitemsr&r7) rBrbrPrcheck_ownershipenable_uniform_level_accessenable_public_access_preventionsoft_delete_durationrstorage_buckets_insert_requestiam_configurationbucket_list_req bucket_lists ` rCreateBucketIfNotExists%StorageClient.CreateBucketIfNotExistsYsZJ**//77;;T;JG- kkdmmDDE \  mm==>(O++%%**?;K 9{'8'89 9 9 % VF^ %% :e  + +) -- 3 3]]))v3;*= 4 >% --..DDF $ 059]]5I5I5_5_5}5}/6~6 2 ) 4 *5?  25@  2 mm22HHJ J  '-->  59--2  ) MM 6 6)= 7  '-->  ""#AB  - -   F F!G!    A)s+>""h&<& 6q}}Z->->"00&8:  ; <<@1Jf@N4"&&* n%`'4!rs ''  9'%7>.36#*63 /HNN/?%?OeO>%>1 ;IFIr