$^SrSSKJr SSKJr SSKJr SSKrSSKrSSKrSSKrSSK J r SSK J r SSK J r SS K Jr SS K Jr SS K Jr SS K Jr SS K Jr SSK Jr SSK Jr SSKJr SSKJr SSKJr SSKJr SSKJr SSKJr SSKJ r SSKJ!r! SSS.r"Sr#Sr$Sr%S Sjr&Sr'g)!z2Utility functions for performing upload operation.)absolute_import)division)unicode_literalsN) api_factory) cloud_api)request_config_factory)buffered_upload_stream)component_stream)errors)fast_crc32c_util) hash_util)progress_callbacks) upload_stream) task_status) delete_task)crc32c) properties)files)hashing) platforms)scaled_integerz text/markdownzapplication/gzip)z.mdz.tgzcP[R"[RRR R 55nX:aB[RRUR;a[RR$[RR$)zDetermines if resumbale uplaod should be performed. Args: api (CloudApi): An api instance to check if it supports resumable upload. object_length (int): Length of the data to be uploaded. Returns: bool: True if resumable upload can be performed. )r ParseIntegerrVALUESstorageresumable_thresholdGetr CapabilityRESUMABLE_UPLOAD capabilitiesUploadStrategy RESUMABLESIMPLE)api object_lengthrs >lib/googlecloudsdk/command_lib/storage/tasks/cp/upload_util.pyget_upload_strategyr'5sw'3333779;*++s/?/??  # # - --  # # * **c<U(a[R$[R5Hup#UR U5(dMUs $ [ R R5(d{[RRRR5(aD[R"SSSU/S[RSS9nUR R#5nO[$R&"U5up5U(aU$[R$)a Gets a file's MIME type. Favors returning the result of `file -b --mime ...` if the command is available and users have enabled it. Otherwise, it returns a type based on the file's extension. Args: source_path (str): Path to file. May differ from file_resource.storage_url if using a temporary file (e.g. for gzipping). is_stream (bool): If the source file is a pipe (typically FIFO or stdin). Returns: A MIME type (str). If a type cannot be guessed, request_config_factory.DEFAULT_CONTENT_TYPE is returned. filez-bz--mimeT)checkstdoutuniversal_newlines)rDEFAULT_CONTENT_TYPECOMMON_EXTENSION_RULESitemsendswithrOperatingSystem IsWindowsrrr use_magicfileGetBool subprocessrunPIPEr,strip mimetypes guess_type) source_path is_stream extension content_typeoutput_s r&get_content_typerBHs" ! 6 66"8!=!=!?iI&& "@  # # - - / /--5577 ^^VT8[A"&#-??/35F==&&(L**;7OL   4 44r(cURRn[RRR R 5(aURROSn[R"X#5n[R"[RRRR55n[RRU;a~[ R""5 U[RR$:Xd[ R&"5(a/[(R*R,[ R."50$0$UR0(d<[RR2U;dU[RR4:Xa0$[(R*R6[8R:"50$)aYGets appropriate hash objects for upload validation. Args: source_resource (resource_reference.FileObjectResource): The upload source. destination_resource (resource_reference.ObjectResource): The upload destination. Returns: A dict[hash_util.HashAlgorithm, hash object], the values of which should be updated with uploaded bytes. N) storage_urlschemerrr#enable_zonal_buckets_bidi_streamingr5 bucket_namerget_capabilities CheckHashes check_hashesrrrAPPENDABLE_UPLOADr log_or_raise_crc32c_issuesALWAYScheck_if_will_use_fast_crc32cr HashAlgorithmCRC32C get_crc32cmd5_hashCLIENT_SIDE_HASH_VALIDATIONNEVERMD5rget_md5)source_resourcedestination_resourceproviderrGr rJs r& get_digestersrZpsP" - - 4 4(    " " F F N N P P&&22  --hD,'',,0024,++|;//1 ..555  9 9 ; ;%%,,.>.I.I.K LL I66,Fj,,222 I  ! ! % %w'8 99r(c U(ay[R"UU=(d SUURURUU[RR [ R"5[R"5S9 nOSnURR(a[ R"SS5n O*[R"URR5n URR(a][ R""[$R&R(R*R-55n [.R0"U U UUS9$Uc[2R4"U UUUS9$[6R8"XXUS9$)aGets a stream to use for an upload. Args: source_resource (resource_reference.FileObjectResource): Contains a path to the source file. length (int|None): The total number of bytes to be uploaded. offset (int|None): The position of the first byte to be uploaded. digesters (dict[hash_util.HashAlgorithm, hash object]|None): Hash objects to be populated as bytes are read. task_status_queue (multiprocessing.Queue|None): Used for sending progress messages. If None, no messages will be generated or sent. destination_resource (resource_reference.ObjectResource|None): The upload destination. Used for progress reports, and should be specified if task_status_queue is. component_number (int|None): Identifies a component in composite uploads. total_components (int|None): The total number of components used in a composite upload. Returns: An UploadStream wrapping the file specified by source_resource. r) status_queueoffsetlength source_urldestination_urlcomponent_numbertotal_componentsoperation_name process_id thread_idNrb)max_buffer_size digestersprogress_callback)r^rhri)r]r^rhri)rFilesAndBytesProgressCallbackrDr OperationName UPLOADINGosgetpid threading get_identis_stdiofdopenrBinaryFileReader resource_namer=rParseBinaryIntegerrrrupload_chunk_sizerr BufferedUploadStreamr UploadStreamr ComponentStream) rWr^r]rhtask_status_queuerXrarbri source_streamrgs r& get_streamr|sH:*HH&{"..,88))"00::99;%%'   ))IIa&M**##113M  **$77!!33779;O ! 6 6'+  -- ~  % %+  --  + +V+ --r(cU(dg[RRU;a>[R"U[RR5nURnO[RR U;a>[ R"U[RR 5nURnOB[R"SRSRUR5555e[R"URR UU5 g![R"a/ [$R&"UR5R)US9 ef=f)zHRaises error if hashes for uploaded_resource and digesters do not match.Nz4Unsupported hash algorithm(s) found in digesters: {}z, )rz)r rOrUget_base64_hash_digest_stringrRrPrget_hash crc32c_hashr Errorformatjoinkeysvalidate_object_hashes_matchrD url_stringHashMismatchErrorrDeleteObjectTaskexecute)rhuploaded_resourcerzcalculated_digestdestination_hashs r&validate_uploaded_objectrs1    I-!??)))--.0(11%%2)))001)44 ,,>EE IIinn& '     **%%002C  ! !   !2!>!>?GG+H  s ,D11AE4)NNNNNNN)(__doc__ __future__rrrr:rmr6rogooglecloudsdk.api_lib.storagerrr"googlecloudsdk.command_lib.storager r r r r rr(googlecloudsdk.command_lib.storage.tasksr+googlecloudsdk.command_lib.storage.tasks.rmrgooglecloudsdk.command_lib.utilrgooglecloudsdk.corergooglecloudsdk.core.utilrrrrr/r'rBrZr|rr(r&rs 9&' 64AE?5?8A<@C2**,.3   +&%5P$:P!%$( $ $D-N r(