]&SrSSKJr SSKJr SSKJr SSKrSSKrSSKrSSKJ r SSK J r SSK J r SS KJr SS KJr SS KJr S r"S S\R(5r"SS\5rSrSSjrSSjrSSjrSSjrSrSr\"5r\RAS5r!\RAS5r"\RAS5r#SSjr$g)aUtilities that augment the core CRC32C functionality for storage commands. The core CRC32C utility provides a hashlib-like functionality for CRC32C calculation but will at times fall back to a slow, all-Python implementation. This utility provides several mitigation strategies to avoid relying on the slow implementation of CRC32C, including adding a "deferred" strategy that uses the component gcloud-crc32c on files after they are downloaded. )absolute_import)division)unicode_literalsN) info_holder)errors)crc32c)binary_operations)log) propertiesz gcloud-crc32cc6^\rSrSrSrU4SjrSSjrSrU=r$)GcloudCrc32cOperation,z1Operation for hashing a file using gcloud-crc32c.c :>[[U] "SS[0UD6 g)Nbinary)superr __init__ BINARY_NAME)selfkwargs __class__s :lib/googlecloudsdk/command_lib/storage/fast_crc32c_util.pyrGcloudCrc32cOperation.__init__/s /M{MfMc 4S[U5S[U5U/$)Nz-oz-l)str)r file_pathoffsetlengthrs r_ParseArgsForCommand*GcloudCrc32cOperation._ParseArgsForCommand2s #f+tS[) <L)structpackr.r2s rdigestDeferredCrc32c.digest`s ;;tTYY ''rcVSRUR5RS5$)zReturns a checksum like `digest` except as a bytestring of double length. Returns: A sixteen byte digest string, containing only hex digits. z{:08x}ascii)formatr.encoder2s r hexdigestDeferredCrc32c.hexdigestjs" ??499 % , ,W 55rr-N)r) r"r#r$r%r&rr3r7r@rErKr'rrrr*r*6s% ) ;&(6rr*cz[[R"[5;$![Ra gf=f)zCReturns if gcloud-crc32c is installed and does not attempt install.F)rr CheckForInstalledBinaryMissingExecutableExceptionrrr_is_gcloud_crc32c_installedrPss6 +CCKP PP  5 5 s #::c:[[R"[US9;$![RaL [R "S5 [ RRRRS5 g [R "S5 g=f)zCReturns True if gcloud-crc32c is installed and optionally installs.install_if_missingzygcloud-crc32c can not be found on the path. The user may have opted out of installation or lack component manager access.Fz.Failed to check if gcloud-crc32c is available.) rr rNrOr debugr VALUESstorageuse_gcloud_crc32cSetrRs r!_check_if_gcloud_crc32c_availablerY{s@ +CC(:   5 5 ; II A//33E: @II>? s"ABBc[R=(dF [RRR R 5SL=(a [U5$)NF)rIS_FAST_GOOGLE_CRC32C_AVAILABLEr rUrVrWGetBoolrYrRs rcheck_if_will_use_fast_crc32cr]sG  / / 1199;5H@ +,> ? rc[RRRR 5nUSLagUc[ R (agU(a [SS9$[5$)zReturns True if gcloud-crc32c should be used and installs if needed. Args: install_if_missing (bool): Install gcloud-crc32c if not already present. Returns: True if the Go binary gcloud-crc32c should be used. FTrR) r rUrVrWr\rr[rYrP)rSuser_wants_gcloud_crc32cs rshould_use_gcloud_crc32cr`s]1199;& & ,,  , EE $ &&rcb[SS9nU(a [5$[R"U5$)aDWraps the crc32c.get_crc32c() method to allow fallback to gcloud-crc32c. DO NOT USE for streaming downloads, as this relies on file-based hashing and does not take whether or not streaming is enabled into account. Args: initial_data (bytes): The CRC32C object will be initialized with the checksum of the data. Returns: A DeferredCrc32c instance if hashing can be deferred. Otherwise it returns a google_crc32c.Checksum instance if google-crc32c (https://github.com/googleapis/python-crc32c) is available and a predefined.Crc instance from crcmod library if not. TrR)r`r*r get_crc32c) initial_data should_defers rrbrbs) *TB,) Nv/@/@/NNrc[R"5nURRnU(aG[R R USS5nSRURRU5$g)zReturns the command to install google-crc32c library. This will typically only be called if gcloud-crc32c is missing and can't be installed for some reason. It requires user intervention which is why it's not a preferred option. lib third_partyz5{} -m pip install google-crc32c --upgrade --target {}N) r InfoHolder installationsdk_rootospathjoinrIbasicpython_location)sdk_inforjthird_party_paths r!get_google_crc32c_install_commandrrsc # # %(  " " + +( ww||He]C B I I&&(8 :: rc~[5nSn[R"SRU(aUS95$US95$)z CRC32C warnings share this text.z'gcloud components install gcloud-crc32cz This copy {{}} since fast hash calculation tools are not installed. You can change this by running: $ {crc32c_step} You can also modify the "storage/check_hashes" config setting.) crc32c_step)rrtextwrapdedentrI)google_crc32c_install_stepgcloud_crc32c_install_steps r_get_hash_check_warning_baseryse ABH HIO '1IOI  *IOI  rzwill not be validatedz may be slowz was skippedc$[SS9(ag[RRRR 5nU[R RR:Xa"U(a[R"[5 gU[R RR:Xa[R"[5 gU[R RR:Xa[R "["5eg)aInforms user about slow hashing if requested. Args: warn_for_always (bool): User may not want to see a warning about slow hashes if they have the "always check hashes" property set because (1) they intentionally set a property and (2) it could duplicate a warning in FileDownloadTask. Raises: errors.Error: IF_FAST_ELSE_FAIL set, and CRC32C binary not present. See error message for more details. TrRN)r]r rUrV check_hashesGet CheckHashesALWAYSvaluer warning_SLOW_HASH_CHECK_WARNINGIF_FAST_ELSE_SKIP_NO_HASH_CHECK_WARNINGIF_FAST_ELSE_FAILrr:_NO_HASH_CHECK_ERROR)warn_for_alwaysr{s rlog_or_raise_crc32c_issuesrs#d; ""**77;;=,Z++22888_KK()z--??EEEKK&'z--??EEE ,,+ ,,Fr)F)r)T)%r& __future__rrrrkrCrugooglecloudsdk.command_libr"googlecloudsdk.command_lib.storagergooglecloudsdk.command_lib.utilr&googlecloudsdk.command_lib.util.anthosr googlecloudsdk.corer r rBinaryBackedOperationr objectr*rPrYr]r`rbrrry_HASH_CHECK_WARNING_BASErIrrrrrrrrs'' 252D#* =-CC=:6V:6z,'0O(  $891884::=I/66}E-r