D' J S r SSKJr SSKJr SSKJr SSKJr SSKrSSKrSSKrSSK r SSK
r
SSK J r SSK
Jr SS KJr SS
KJr SS KJr SS KJr SS
KJr SSKJr SSKJr SSKJ r SSKJ r SSKJ r SSKJ r SSK J!r! SSK J"r" SSKJ#r# \ RH r%Sr&S\&- S- r'S\#RP " 5 - S- S - \#RP " 5 - S - S - S - S - S!- \#RR " 5 - S"- S#- S$- S%- S!- \#RR " 5 - S&- S'- r* " S( S)\ 5 r+g)*zEImplementation of hash command for calculating hashes of local files. )absolute_import)print_function)division)unicode_literalsN)Command)CommandArgument) ApiSelector)CommandException)FileProgressCallbackHandler)ProgressCallbackWithTimeout)StorageUrlFromString) FileMessage) FinalMessage) boto_util) constants)hashing_helper)parallelism_framework_util)GcloudStorageFlag)GcloudStorageMap) shim_utilz*
gsutil hash [-c] [-h] [-m] filename...
z
SYNOPSIS
a
DESCRIPTION
Calculate hashes on local files, which can be used to compare with
``gsutil ls -L`` output. If a specific hash option is not provided, this
command calculates all gsutil-supported hashes for the files.
Note that gsutil automatically performs hash validation when uploading or
downloading files, so this command is only needed if you want to write a
script that separately checks the hash.
If you calculate a CRC32c hash for files without a precompiled crcmod
installation, hashing will be very slow. See "gsutil help crcmod" for details.
OPTIONS
-c Calculate a CRC32c hash for the specified files.
-h Output hashes in hex format. By default, gsutil uses base64.
-m Calculate a MD5 hash for the specified files.
z>--format=value[separator="",terminator=""](digest_format.sub("z", "Hashes ["),z url.sub("z ", "] for ").sub("$", ":^\^n"),z,md5_hash.yesno(yes=" Hash (md5): ", no=""),zmd5_hash.yesno(no=""),z$md5_hash.yesno(yes="NEWLINE", no="")z.sub("NEWLINE", "z"),z2crc32c_hash.yesno(yes=" Hash (crc32c): ", no=""),zcrc32c_hash.yesno(no=""),z'crc32c_hash.yesno(yes="NEWLINE", no="")z"))c ^ \ rS rSrSr\R " S/ \S\R SSSS\
R /\
R \ R " 5 /S 9 r\R " SS
/S S \0 S
9rU 4S jr\S 5 rS rS rSrU =r$ ) HashCommandY z&Implementation of gsutil hash command.hash chmTFr ) command_name_aliasesusage_synopsismin_argsmax_argssupported_sub_args file_url_okprovider_url_okurls_start_arggs_api_supportgs_default_apiargparse_argumentschecksum command_helpzCalculate file hashes) help_namehelp_name_aliases help_typehelp_one_line_summary help_textsubcommand_help_textc b > [ SS[ /[ S5 S S S.S9n[ U R U R
VVs/ s H u p#UPM snn- 5 nSU; a SU; a U=R S/-
sl O"SU; a SU; a U=R S /-
sl [ TU ]! U5 $ s snnf )
Nstorager z--hex)-h-c-m)gcloud_commandflag_mapr4 r5 z
--skip-md5z
--skip-crc32c) r _GCLOUD_FORMAT_STRINGr setargssub_optsr6 superget_gcloud_storage_args)selfgcloud_storage_mapflag_args_set __class__s &platform/gsutil/gslib/commands/hash.pyr= #HashCommand.get_gcloud_storage_argst s ) !
$G,
499DMM BMM BBCH xD 0''L>9'
d( 2''O+<<' 7 *+= >>
Cs B+ c 2 SnSnS nS nSnSnU( a3 U H- u pU S:X a SnSnM U S:X a
SnS nS
nM! U S :X d M) SnSnM/ U( d SnSnU( a9 [ R " 5 ( d UR [ R 5 X4XVU4$ ) a* Returns behavior variables based on input options.
Args:
sub_opts: getopt sub-arguments for the command.
logger: logging.Logger for the command.
Returns:
Tuple of
calc_crc32c: Boolean, if True, command should calculate a CRC32c checksum.
calc_md5: Boolean, if True, command should calculate an MD5 hash.
format_func: Function used for formatting the hash in the desired format.
cloud_format_func: Function used for formatting the hash in the desired
format.
output_format: String describing the hash output format.
Fc J [ R " U R 5 5 $ N)r Base64EncodeHash hexdigestdigests rD (HashCommand._ParseOpts.. s ~66v7G7G7IJ c U $ rH rK s rD rM rN s vrO base64r4 Tr3 hexc " U R 5 $ rH )rJ rK s rD rM rN s
v'7'7'9rO c L [ R " U 5 R S5 $ )Nascii)r Base64ToHexHashdecoderK s rD rM rN s ,,V4;;GDrO r5 )r UsingCrcmodExtensionwarnr SLOW_CRCMOD_WARNING) clsr; logger calc_crc32ccalc_md5 format_funccloud_format_funcfound_hash_option
output_formatounused_as rD
_ParseOptsHashCommand._ParseOpts s " KHJ - M!+! 9 + "
$Y -9+
$Y( "
" kh999;; kk.445 +- OOrO c 0 nU( a" [ R R S5 US' U( a [ R " 5 US' U$ )a Constructs the dictionary of hashes to compute based on the arguments.
Args:
calc_crc32c: If True, CRC32c should be included.
calc_md5: If True, MD5 should be included.
Returns:
Dictionary of {string: hash digester}, where string the name of the
digester algorithm.
zcrc-32ccrc32cmd5)crcmod
predefinedCrcr GetMd5)r> r^ r_ hash_dicts rD _GetHashClassesFromArgs#HashCommand._GetHashClassesFromArgs sC I " - - 1 1) " 5 RA S U5 GM U( a UR8 U
S ' U( a UR: U
S
' [C SU< SU
< S35 [D RF " U
5 H< u nn[C SU< SU R 5 ( a U" U5 OU" U5 < 35 M> GM GM U( d [I S5 e[K U R R [M [" R" " 5 5 5 g! , ( d f GN= f)z)Command entry point for the hash command.F)ri customerEncryptionmd5Hashsize)bucket_listing_fieldsTN)ru finished message_typeHashing)src_urloperation_namerb)callback_processorzNo hashes present for %srj ri zHashes [z] for :z Hash (z): zNo files matchedr )'rf r; r] r: WildcardIterator IterObjectsr
storage_url object_name IsFileUrlospathgetsize
gsutil_api status_queueputr time FILE_HASHr r callrp openr CalculateHashesFromContents root_objectru rt ri logging getLoggerrZ printsix iteritemsr
_PutToQueueWithTimeoutr )r> r^ r_ r` ra rc matched_oneurl_strfile_refurl file_name file_sizer} ro fp obj_metadata md5_presentcrc32c_presentnamerL s rD
RunCommandHashCommand.RunCommand s t}}dkkB[KK99++G4@@! A
( "7+ ((44 ( 2 2 4 4ggooi0)
//
&
&
*
*# ))+ (#('2'<'< > ? ;)$//*F*F2Fw2O9BDDHD J
22;I)It$ 6 62D
F % //
&
&
*
*# ))+ (#''2'<'< > ? )!--,"'')$,,D8+'..d:. ^ $ $%? I
+ 3 3Ie
"."5"5Ih
}i@A MM) 4LD&
@C
V)<.?.G*HJ K 5c
n / 004??77' 46 C %$s -L99
M rQ )__name__
__module__ __qualname____firstlineno____doc__r CreateCommandSpec _SYNOPSISr NO_MAXr JSONr MakeZeroOrMoreFileURLsArgument command_specHelpSpec_DETAILED_HELP_TEXT help_specr= classmethodrf rp r __static_attributes__
__classcell__)rC s @rD r r Y s . ** !&&' %%)HHJK M, # 3#
)?* .P .P`$A
A
rO r ),r
__future__r r r r r r r rk r
gslib.commandr gslib.command_argumentr gslib.cs_api_mapr gslib.exceptionr
gslib.progress_callbackr r gslib.storage_urlr
gslib.thread_messager r gslib.utilsr r r r gslib.utils.shim_utilr r r PutToQueueWithTimeoutr r r get_format_flag_caretget_format_flag_newliner8 r rQ rO rD r sy L & % '
! 2 ( , ? ? 2 , - ! ! & 2 3 2 !3II
4 O
##%&(9:
AF F <
< #LL $;;=> AEE HKK t
' t
rO