zSrSSKJr SSKJr SSKJr SSKrSSKrSSKrSSKJ r SSK J r SSK J r SS K Jr SS K Jr SS K Jr SS K Jr SS K Jr SSK Jr SSK Jr SSK Jr SSK Jr SSK Jr SSK Jr SSKJr SSKJr SSKJr SSKJr SSKJ r SSKJ!r! SSK"J#r# SSK"J$r$ Sr%Sr&Sr'Sr(S r)S!r*S"r+S#r,S$r-S7S%jr.S7S&jr/S7S'jr0\ RbRd4S(jr3\ RbRd4S)jr4S*r5S+r6S,r7\RpS-5r9S.r:S7S/jr;S0rS7S3jr?S4r@S5rAS7S6jrBg)8zxGeneric logic for cp and mv command surfaces. Uses command surface tests. Ex: cp_test.py, not cp_command_util_test.py. )absolute_import)division)unicode_literalsN) cloud_api) arg_parsers)base)encryption_util)errors) errors_util)flags) folder_util)name_expansion)plurality_checkable_iterator)rm_command_util)stdin_iterator) storage_url)user_request_args_factory) task_executor)task_graph_executor) task_status)copy_task_iterator)log) properties)files) platformsaXCopy all source versions from a source bucket or folder. If not set, only the live version of each source object is copied. Note: This option is only useful when the destination bucket has Object Versioning enabled. Additionally, the generation numbers of copied versions do not necessarily match the order of the original generation numbers. a%Applies gzip transport encoding to any file upload whose extension matches the input extension list. This is useful when uploading files with compressible content such as .js, .css, or .html files. This also saves network bandwidth while leaving the data uncompressed in Cloud Storage. When you specify the `--gzip-in-flight` option, files being uploaded are compressed in-memory and on-the-wire only. Both the local files and Cloud Storage objects remain uncompressed. The uploaded objects retain the `Content-Type` and name of the original files.a5Applies gzip transport encoding to file uploads. This option works like the `--gzip-in-flight` option described above, but it applies to all uploaded files, regardless of extension. CAUTION: If some of the source files don't compress well, such as binary data, using this option may result in longer uploads.aApplies gzip content encoding to any file upload whose extension matches the input extension list. This is useful when uploading files with compressible content such as .js, .css, or .html files. This saves network bandwidth and space in Cloud Storage. When you specify the `--gzip-local` option, the data from files is compressed before it is uploaded, but the original files are left uncompressed on the local disk. The uploaded objects retain the `Content-Type` and name of the original files. However, the `Content-Encoding` metadata is set to `gzip` and the `Cache-Control` metadata set to `no-transform`. The data remains compressed on Cloud Storage servers and will not be decompressed on download by gcloud storage because of the `no-transform` field. Since the local gzip option compresses data prior to upload, it is not subject to the same compression buffer bottleneck of the in-flight gzip option.amApplies gzip content encoding to file uploads. This option works like the `--gzip-local` option described above, but it applies to all uploaded files, regardless of extension. CAUTION: If some of the source files don't compress well, such as binary data, using this option may result in files taking up more space in the cloud than they would if left uncompressed.aOutputs a manifest log file with detailed information about each item that was copied. This manifest contains the following information for each item: - Source path. - Destination path. - Source size. - Bytes transferred. - MD5 hash. - Transfer start time and date in UTC and ISO 8601 format. - Transfer completion time and date in UTC and ISO 8601 format. - Final result of the attempted transfer: OK, error, or skipped. - Details, if any. If the manifest file already exists, gcloud storage appends log items to the existing file. Objects that are marked as "OK" or "skipped" in the existing manifest file are not retried by future commands. Objects marked as "error" are retried. agCauses POSIX attributes to be preserved when objects are copied. With this feature enabled, gcloud storage will copy several fields provided by the stat command: access time, modification time, owner UID, owner group GID, and the mode (permissions) of the file. For uploads, these attributes are read off of local files and stored in the cloud as custom metadata. For downloads, custom cloud metadata is set as POSIX attributes on files after they are downloaded. On Windows, this flag will only set and restore access time and modification time because Windows doesn't have a notion of POSIX UID, GID, and mode. aPreserve symlinks instead of copying what they point to. With this feature enabled, uploaded symlinks will be represented as placeholders in the cloud whose content consists of the linked path. Inversely, such placeholders will be converted to symlinks when downloaded while this feature is enabled, as described at https://cloud.google.com/storage-transfer/docs/metadata-preservation#posix_to. Directory symlinks are only followed if this flag is specified. CAUTION: No validation is applied to the symlink target paths. Once downloaded, preserved symlinks will point to whatever path was specified by the placeholder, regardless of the location or permissions of the path, or whether it actually exists. This feature is not supported on Windows. cURSSS[S9 URSSS[R"5[S9 g ) z&Adds flags for gzip parsing in flight.z-Jz--gzip-in-flight-all store_trueactionhelpz-jz--gzip-in-flightFILE_EXTENSIONSmetavartyper N) add_argument_GZIP_IN_FLIGHT_ALL_HELP_TEXTrArgList$_GZIP_IN_FLIGHT_EXTENSIONS_HELP_TEXTparsers 9lib/googlecloudsdk/command_lib/storage/cp_command_util.pyadd_gzip_in_flight_flagsr,sN   (        / cBSnSnURSSSU(aUOUS9 g)zAdds --include-managed-folders flag to the parser. Args: parser: (parser_arguments.ArgumentInterceptor): Parser passed to surface. hns_feature_warning: (bool) Whether to add a warning for HNS buckets to the flag. aIncludes managed folders in command operations. For transfers, gcloud storage will set up managed folders in the destination with the same IAM policy bindings as the source. Managed folders are only included with recursive cloud-to-cloud transfers. Please note that for hierarchical namespace buckets, managed folders are always included. Hence this flag would not be applicable to hierarchical namespace buckets.zIncludes managed folders in command operations. For transfers, gcloud storage will set up managed folders in the destination with the same IAM policy bindings as the source. Managed folders are only included with recursive cloud-to-cloud transfers.z--include-managed-foldersrFrdefaultr Nr%)r*hns_feature_warninghns_feature_warning_help_text help_texts r+ add_include_managed_folders_flagr5s<7   ! ,? (Y r-c(URSSUSS9 g)z(Adds flag for skipping copying symlinks.z--ignore-symlinksrz;Ignore file symlinks instead of copying what they point to.r/Nr1parser_or_groupr0s r+add_ignore_symlinks_flagr9s!  G r-c0URSSU[S9 g)z"Adds flag for preserving symlinks.z--preserve-symlinksrr/N)r%_PRESERVE_SYMLINKS_HELP_TESTr7s r+add_preserve_symlinks_flagr<s!  ' r-cF[R"U5 [R"U5 [R"XS9 [R"U5 UR SSSS9 UR SSSS S 9 UR S S S[ S 9 UR S SSSS 9 g)z'Adds flags shared by cp, mv, and rsync. release_trackz --content-md5 MD5_DIGESTzManually specified MD5 hash digest for the contents of an uploaded file. This flag cannot be used when uploading multiple files. The custom digest is used by the cloud provider for validation.)r#r z-nz --no-clobberrzDo not overwrite existing files or objects at the destination. Skipped items will be printed. This option may perform an additional GET request for cloud objects before attempting an upload.rz-Pz--preserve-posixz-Uz--skip-unsupportedz+Skip objects with unsupported object types.N)r add_additional_headers_flagadd_continue_on_error_flagadd_object_metadata_flagsadd_precondition_flagsr%_PRESERVE_POSIX_HELP_TEXT)r*r?s r+add_cp_mv_rsync_flagsrFs##F+""6*!!&Fv& KM           $      8 r-cURSSSS9 URSSS9 [XS9 URS S S [S 9 URS S SS 9 URSSS SS 9 [USS9 UR SSS9n[ U5 [ U5 URSS[S9 URSSS SS 9 URSSSS9 UR SS9n[U5 URSS S [S 9 URS!S"S#[R"5[S$9 UR 5n[R"U5 [R"U5 [R "U5 [R""US%S&9 g')(z1Adds flags to cp, mv, or other cp-based commands.source*zThe source path(s) to copy.)nargsr destinationzThe destination path.)r r>z-Az--all-versionsrrz--do-not-decompressz6Do not automatically decompress downloaded gzip files.z-Dz --daisy-chainaCopy in "daisy chain" mode, which means copying an object by first downloading it to the machine where the command is run, then uploading it to the destination bucket. The default mode is a "copy in the cloud," where data is copied without uploading or downloading. During a copy in the cloud, a source composite object remains composite at its destination. However, you can use daisy chain mode to change a composite object into a non-composite object. Note: Daisy chain mode is automatically used when copying between providers.T)r2zNFlags to influence behavior when handling symlinks. Only one value may be set.)mutexr z-Lz--manifest-pathz-vz--print-created-messagez7Prints the version-specific URL for each copied object.z-sz--storage-classzSpecify the storage class of the destination object. If not specified, the default storage class of the destination bucket is used. This option is not valid for copying to non-cloud destinations.)rLz-Zz--gzip-local-allz-zz --gzip-localr!r"aRead the list of resources to copy from stdin. No need to enter a source argument if this flag is present. Example: "storage cp -I gs://bucket/destination" Note: To copy the contents of one file directly from stdin, use "-" as the source argument without the "-I" flag.)r4N)r%rF_ALL_VERSIONS_HELP_TEXTr5 add_groupr9r<_MANIFEST_HELP_TEXTr,_GZIP_LOCAL_ALL_HELP_TEXTrr' _GZIP_LOCAL_EXTENSIONS_HELP_TEXTr add_predefined_acl_flagadd_preserve_acl_flagadd_encryption_flagsadd_read_paths_from_stdin_flag)r*r?symlinks_groupgzip_flags_groupacl_flags_groups r+add_cp_and_mv_flagsrY shc0MNm*AB< \8OQ  CE      #6tD##  $.>*^,d-4GH   D F    OP%%D%1+,   $        +  $$&/0o.V$&&  ; r-c*URSSSSSS9 g)zAdds flag for copying with recursion. Not used by mv. Args: parser (parser_arguments.ArgumentInterceptor): Parser passed to surface. z-Rz-rz --recursiverzWRecursively copy the contents of any directories that match the source path expression.rNr1r)s r+add_recursion_flagr[Zs(     %  r-cVUR(dg[U[R5(a%[R "SR U55e[USS5(a[R "S5eUH]n[R"U5n[U[R5(dM:[R "SR U55e UR(d[R "S5e[R"URU5 g)zGValidates that arguments are consistent with managed folder operations.Nz?Cannot include managed folders with a non-cloud destination: {}read_paths_from_stdinzCannot include managed folders when reading paths from stdin, as this would require storing all paths passed to gcloud storage in memory.z:Cannot include managed folders with a non-cloud source: {}z;Cannot include managed folders unless recursion is enabled.) include_managed_folders isinstancerFileUrlr Errorformatgetattrstorage_url_from_string recursiver raise_error_if_not_gcs command_path)argsraw_source_urlsraw_destination_url url_stringurls r+ validate_include_managed_foldersrmns  % % #[%8%899 ,,I # $   T*D11 ,, O  $j  - -j 9C#{**++ LL F M M  $  ,,E $$T%6%68KLr-cUR(a'UR(a[R"S5eUR(a9[ R R5(a[R"S5e[U[R5(a6UR(a%[R"SRU55e[XRU5 g)z*Raises errors if invalid flags are passed.z;Cannot specify both generation precondition and no-clobber.z2Symlink preservation is not supported for Windows.z*> ? ?  $ $ (B(B(F(F F 4++A tO#."9"9"; . D D*??KK"00@@',*;;>> ; ! #/"; : # "? kkU/%002,BB(==II ..>>%*(99==2  - 9 8 kk r-chUHnXS;aX5(aX5X%'URS5(dURS5(dMEU(a!URS5(aUSSOUSSnO URS5(aUSSOUSSnXa;dMXc;dMX6(dMX6X%'M g)aCorrects the results of url match tracker for copy and renames. Args: original_sources: Original sources given by the user. updated_sources: Updated sources after filtering and modifying for folders only. url_found_match_tracker: The common url found match tracker. folders_tracker: The url found match tracker we have created specifically for folders feature. is_rename_folders: Is the rename folders case. /**/**/**N)endswith)original_sourcesrrfolders_trackerrrlpossible_updated_urls r+rrs$c /"6%4%9" ||Ecll622 +.<<+=+=s3Bx3s8 ,/<<+=+=s3Bx3s8  1"533'6'L$)r-c/nUHn[[R"U55(dM)URS5(dURS5(abU(a!URS5(aUSSOUSSnO URS5(aUSSOUSSnX@;aUR U5 MMUR U5 M U$)aFilters and modifies sources urls for the purpose of HNS bucket rename_folder. We filter out any source URL which is not a GCS URL as rename folders is only applicable to HNS buckets which is a GCS feature. Apart from this, if the given URL ends with a **, we change it to a single * to match the filesystem behaviour. In case of a regular Linux filesystem, a ** or a * will rename folders under the given path to the destination. But in case of Gcloud, we would recursively list all sub-directories under it and try to renma them. This is not required for rename_folder operations. Hence, by replacing them with a *, we can simply rename the given sub-folders. Args: sources: List of source URLs given by the user to the command. is_rename_folders: Boolean indicating if the operation is for renaming folders. Returns: A list of source URLs which are filtered and modified for the purpose of rename_folders only operation. rrrNrrr)is_gcs_cloud_urlrrdrappend)sourcesrupdated_source_listrHupdated_sources r+rrs0f K??G H H u!8!8 (.(=(=6#2;(.(=(=6#2;  &"">2 '  ( r-c /nURH:n[[R"U55(dM)UR U5 M< U(dg[ R "U[RR[RRS[ RRUS9n[R"U[ R"R%55$)aHelper method to remove folders from HNS buckets. Removing folders is only applicable for HNS buckets. In the case where a source is a FileURL, we will see a failure in the DeleteTaskIteratorFactory as we try to call is_bucket() on a FileURL. This happens specifically in the case where we use --no-clobber flag and where the destination already exists. For such cases, we would skip deletion of the source. So the NameExpansionIterator will contain a FileURL which the DeleteTaskIteratorFactory will not be able to handle. Hence, we are skipping a given source if it's not a CloudURL since Folders are only applicable to CloudURLs (which are GCS) and running any attempt to find and delete a Folder is of no use on any other type of URL. Args: args: User provided arguments for the command. url_found_match_tracker: The common url found match tracker. Returns: Exit code for the operation. rFr)rHrrrdrrrr rrrrrrrremove_foldersrrr)rhr cloud_urlsrkfolder_expansion_iterators r+_remove_foldersrs6*KKj ;;JGHH #   ,BB(==II ..>>%*(99==5   ' '11779 r-c[U[R5=(a& UR[RR L$)zIReturns True if the URL is of type CloudURL and has a GCS ProviderPrefix.)r_rrrrr)rls r+rr+s6k**+7 ** 2266 6r-c [R"UR5n[X5 [R "U5 [ R"5n[XX#5 UR(a![X5n[USSUUS9nU(aU$[R"[R"UR UR"55n[R"UR%55n['XU5nUR((a[*R,R.n O[*R,R0n [2R4"UU UR6[8R:R<[8R>R@[BRD"U5URFURH(a[2RJRLO[2RJRNUS9 n[UUUUUS9nU(a[QX5n U S:waU nUR(a[2R4"UR [8R:RR[8R>R<S[2RJRLUS9n [TRV"UU [XRZR]55nU$)zCRuns implementation of cp surface with tweaks for similar commands.F)rhrrrjr) fields_scopeignore_symlinksrr object_staterqrrrr)/rrdrKrur initialize_key_storerrrr^rrrrrget_urls_iterablerHr]peekr preserve_aclr FieldsScopeFULLNO_ACLrrrr rrrLIST_AS_PREFIXESr get_object_state_from_flagsrqrerrNO_WITH_WARNINGrrrremove_managed_foldersrrr) rhrrjrr exit_coderaw_source_string_iteratorrrrfolder_exit_code!managed_folder_expansion_iterators r+run_cpr3s\#;;D>).,==AA"9 +'"88  +  5 5 ; ; =i r-)F)C__doc__ __future__rrrr contextlibrzgooglecloudsdk.api_lib.storagergooglecloudsdk.callioperr"googlecloudsdk.command_lib.storager r r r r rrrrrr(googlecloudsdk.command_lib.storage.tasksrrr+googlecloudsdk.command_lib.storage.tasks.cprgooglecloudsdk.corerrgooglecloudsdk.core.utilrrrMr(r&rQrPrOrEr;r,r5r9r< ReleaseTrackGArFrYr[rmrucontextmanagerrrrrrrrrrrr-r+rsW '' 4/(>5:4:=K>=:HBH@J#**. ($!C$K "$(  $"!H 150A0A0D0D"J/3.?.?.B.BL^("MJK( 2.n 5p kf &MT %&R1h\r-