Cx xSrSSKJr SSKJr SSKJr SSKrSSKrSSKrSSKrSSK r SSK r SSK r SSK J r SSKJr SSKJr SS KJr SS KJr SS KJr SS KJr SS KJr SSKJr SSKJr SSKJr SSKJ r! Sr"Sr#Sr$Sr%Sr&Sr'Sr(Sr)Sr*\RVRX\RZ"S\R\R^S\R\R`S\R\RbS05\RVRd\RZ"S\R\R^S \R\R`S!\R\RbS"050r3S#r4S$r5S%r6S&r7S'r8S7S(jr9\RVRX4S)jr:/S*Qr;S+r<S8S,jr=S-r>S.r?S/r@S0rAS1rBS2rC\R\R"\RR5"S3S4\R555rI\R\R"\RR5"S5S6\I555rKg)9z-Command to install on-premise Transfer agent.)absolute_import)division)unicode_literalsN)agent_pools_util)apis) arg_parsers)base) agents_util) creds_util)log) properties) gce_cache)universe_descriptor) platforms)clienta Specify the number of agents to install on your current machine. System requirements: 8 GB of memory and 4 CPUs per agent. Note: If the 'id-prefix' flag is specified, Transfer Service increments a number value after each prefix. Example: prefix1, prefix2, etc. a Specify the path to the service account's credentials file. No input required if authenticating with your user account credentials, which Transfer Service will look for in your system. Note that the credentials location will be mounted to the agent container. a If you want to grant agents access to specific parts of your filesystem instead of the entire filesystem, specify which directory paths to mount to the agent container. Multiple paths must be separated by commas with no spaces (e.g., --mount-directories=/system/path/to/dir1,/path/to/dir2). When mounting specific directories, gcloud transfer will also mount a directory for logs (either /tmp or what you've specified for --logs-directory) and your Google credentials file for agent authentication. It is strongly recommended that you use this flag. If this flag isn't specified, gcloud transfer will mount your entire filesystem to the agent container and give the agent root access. a" Specify the network to connect the container to. This flag maps directly to the `--network` flag in the underlying '{container_managers} run' command. If binding directly to the host's network is an option, then setting this value to 'host' can dramatically improve transfer performance. zP Could not find project ID. Try adding the project flag: --project=[project-id] a} Specify the HTTP URL and port of a proxy server if you want to use a forward proxy. For example, to use the URL 'example.com' and port '8080' specify 'http://www.example.com:8080/' Ensure that you specify the HTTP URL and not an HTTPS URL to avoid double-wrapping requests in TLS encryption. Double-wrapped requests prevent the proxy server from sending valid outbound requests. zm Credentials file not found at {creds_file_path}. {fix_suggestion}. Afterwards, re-run {executed_command}. a To confirm your agents are connected, go to the following link in your browser, and check that agent status is 'Connected' (it can take a moment for the status to update and may require a page refresh): https://console.cloud.google.com/transfer/on-premises/agent-pools/pool/{pool}/agents?project={project} If your agent does not appear in the pool, check its local logs by running "{logs_command}". The container ID is the string of random characters printed by step [2/3]. The container ID can also be found by running "{list_command}". a Allow the agent to work with S3-compatible sources. This flag blocks the agent's ability to work with other source types (e.g., file systems). When using this flag, you must provide source credentials either as environment variables `AWS_ACCESS_KEY_ID` and `AWS_SECRET_ACCESS_KEY` or as default credentials in your system's configuration files. To provide credentials as environment variables, run: ``` AWS_ACCESS_KEY_ID="id" AWS_SECRET_ACCESS_KEY="secret" gcloud transfer agents install --s3-compatible-mode ``` cg)N'https://docs.docker.com/engine/install/r&lib/surface/transfer/agents/install.pyrs9rrz^https://docs.docker.com/engine/install/binaries/#install-server-and-client-binaries-on-windowszQhttps://docs.docker.com/engine/install/binaries/#install-client-binaries-on-macoscg)Nz$https://podman.io/docs/installation/rrrrrrs6rz8https://podman.io/docs/installation/#installing-on-linuxz,https://podman.io/docs/installation/#windowsz*https://podman.io/docs/installation/#macosa The agent runs inside a {container_manager} container, so you'll need to install {container_manager} before finishing agent installation. See the installation instructions at {installation_guide_url} and re-run '{executed_command}' after {container_manager} installation. c@U(aSOSnUURSUS3$)aYReturns the container command for the given subcommand and container manager. Args: use_sudo (bool): Whether to use sudo in the command. container_manager (agents_util.ContainerManager): The container manager. subcommand (str): The subcommand to run. Returns: str: The container command for the given subcommand and container manager. zsudo z container z [container ID])value)use_sudocontainer_manager subcommand sudo_prefixs r_get_container_subcommandr s4$+  '--./ _&rcUcg[RR[RRU55$)zConverts relative and symbolic paths to absolute paths. Args: path (str|None): The path to expand. If None, returns None. Returns: str|None: The absolute path or None if path is None. N)ospathabspath expanduserr#s r _expand_pathr's. \  ++D1 22rc@SR[R5$)z}Returns the run command. Does not include environment variables. Returns: str: The command that was executed by the user.  )joinsysargvrrr_get_executed_commandr-s #(( rcl[R"SRSRU555 g)zoLogs the command used to create the agent. Args: command (list[str]): The command used to create the agent. zCreated agent with command: {}r)N)r infoformatr*)commands r_log_created_agentr2s% (( , 3 3CHHW4E FGrcU(aY[U5n[RRU5(d(Sn[ [ R UU[5S95eU$[R"5n[RRU5(aU$[R"SS9(agSn[ [ R UU[5S95e)zEnsures agent will be able to authenticate and returns creds. Args: creds_file_supplied_by_user (str): The path to the credentials file. Returns: str: The path to the credentials file. Raises: OSError: If the credentials file is not found. z:Check for typos and ensure a creds file exists at the path)creds_file_pathfix_suggestionexecuted_commandF) check_ageNzRTo generate a credentials file, please run `gcloud auth application-default login`) r'r"r#existsOSErrorMISSING_CREDENTIALS_ERROR_TEXTr0r- oauth2_client_get_well_known_filerGetOnGCE)creds_file_supplied_by_userr4r5s r%_authenticate_and_get_creds_file_pathr?s!"#>?O 77>>/ * * F  ( / /-+46 0 8 99 !668/WW^^O$$ %( ?.$++)'02,4 55rcURn[R"U5(ag[R"SR UR 555 [U[R"5[5S9n[U5e)zChecks for binary identified by container_manager is in system PATH. Args: container_manager (agents_util.ContainerManager): The container manager. Raises: OSError: If the binary is not found. Nz[2/3] {} not found)r current_osr6) rshutilwhichr errorr0title._get_help_text_for_container_manager_not_foundrOperatingSystemr-r9)rr1help_strs r(_check_if_container_manager_is_installedrIsm  # #' \\' )) ' '  89 ;)**,,.( r) )enable_multipart--enable-multipart)hdfs_data_transfer_protection--hdfs-data-transfer-protection)hdfs_namenode_uri--hdfs-namenode-uri) hdfs_username--hdfs-username)kerberos_config_file--kerberos-config-file)kerberos_keytab_file--kerberos-keytab-file)kerberos_service_principal--kerberos-service-principal)kerberos_user_principal--kerberos-user-principal)!max_concurrent_small_file_uploadsz--entirefile-fr-parallelismc[H6up#[XS5nUcMURSRX455 M8 g)zAdds user flags values directly to Docker/Podman command. Args: user_args (argparse.Namespace): The user arguments. container_args (list[str]): The container arguments. Nz{}={})_ADD_IF_PRESENT_PAIRSgetattrappendr0) user_argscontainer_argsuser_argcontainer_flag user_values r'_add_container_flag_if_user_arg_presentrd7s:#8hd3JGNN>FG#8rc /nU(aURS5 [RRU5nUR UR SSSR UR5SS/5 [R"5upgU(a1URS5 URSR U55 U(a1URS5 URS R U55 UR(a*URS R UR55 [U5n[UR5n [RR[R 5n [R n UR"(+=(d# XR";=(d XR";n U (aURS 5 OuS R U 5/n UbU RSR US95 UR"H!nU RSR US95 M# UR U 5 UR$(a;URS5 URSR UR$55 Sn[&R("5(d2[*R,"5nSUR.SUR0S3nUS3SR UR25SR [4R6"55SR U 5SR U5/nUb URSR U55 U (aURS5 UR8(aIUR:bUR8S-nO UR8nURSR U55 [=US S 5nU(a URS!R U55 [?UU5 UR@(aURS"5 [&R("5(dR[&RBRDRFRI5nURS#R U55 UU-$)$aGReturns container run command from user arguments and generated values. When `elevate_privileges` is True, the command will be run with sudo and SELinux will be disabled by passing appropriate security-opt flags. This is needed for running the agent in a container that is not owned by the user. Args: args (argparse.Namespace): The user arguments. project (str): The project to use for the agent. creds_file_path (str): The path to the credentials file. elevate_privileges (bool): Whether to use sudo and disable SELinux. Returns: list[str]: The container run command. sudorunz--ulimitz memlock={}z--rmz-dz--envzAWS_ACCESS_KEY_ID={}zAWS_SECRET_ACCESS_KEY={}z --network={}z-v=/:/transfer_rootz -v={}:/tmpNz&-v={creds_file_path}:{creds_file_path})r4z-v={path}:{path}r&zHTTPS_PROXY={}zgcr.io/zdocker./z/cloud-ingest/zcloud-ingest/tsop-agent:latestz--agent-pool={}z --hostname={}z --log-dir={}z--project-id={}z--creds-file={}z--enable-mount-directory0z--agent-id-prefix={}gcs_api_endpointz--gcs-api-endpoint={}z --enable-s3z--universe-domain={})%r^r ContainerManager from_argsextendrr0 memlock_limitr get_default_aws_credsnetworkr'logs_directoryr"r#r$sepmount_directoriesproxyr IsDefaultUniverserGetUniverseDomainDescriptorartifact_registry_domainproject_prefixpoolsocket gethostname id_prefixcountr]rds3_compatible_modeVALUEScoreuniverse_domainGet)argsprojectr4elevate_privilegesbase_container_commandraws_access_keyaws_secret_keyexpanded_creds_file_pathexpanded_logs_directory_pathroot_with_driveroot_without_drivemount_entire_filesystem mount_flagsr#docker_uri_prefixuniverse_descriptor_obj agent_argsagent_id_prefixrjrs r_get_container_run_commandrDs$!!&)!22<'!!+. ZZ!!'*!!"2"9"9$**"EF   % % ' '1MMO )BBC # 2 2 3> C 9:tyy)V//1289w' *)'../GHI01 ^^ zz,oo,33ODET#5t<-445EFG)$ ; m$  % % ' ' '',,<<@@BO,33ODE * ,,rc[XU5n[R"USS9nURS:Xa [ U5 U$[ R RU5n[RRSRURR555 [XUSS9n[R"USS9nURS:Xa [ U5 U$SRU5n[SU35e) aMGenerates, executes, and returns agent install and run command. Args: args (argparse.Namespace): The user arguments. project (str): The project to use for the agent. creds_file_path (str): The path to the credentials file. Returns: list[str]: The container run command. Raises: OSError: If the command fails to execute. Fcheckrz2 Could not execute {} command. Trying with "sudo".T)rr)zError executing command: )r subprocessrg returncoder2r rkrlr statusPrintr0rrEr*r9) rrr4container_run_commandcompleted_processr)elevated_privileges_container_run_commandelevated_prev_completed_process command_strs r_execute_container_commandrs5 _!nn%:%H!!Q&,-  !22<Sn[U5HupEURS5(dMUn O [SU5H]n[R"U5nU(aSR U[ U55Xs'[R"USS9 [U5 M_ g)zCreates multiple identical agents. Args: agent_count (int): The number of agents to create. agent_id_prefix (str): The prefix to add to the agent ID. container_command (list[str]): The container command to execute. z--agent-id-prefix=z--agent-id-prefix={}{}TrN) enumerate startswithrangecopydeepcopyr0strrrgr2) agent_countrcontainer_commandidx_agent_prefixidxtokenr}container_command_copys r_create_additional_agentsrs/0jc ,-- 1 Q $e!]]+<= # ) )/3u: F.NN)6-.%rcU[;a[SU35e[UUn[RURR 5UUS9$)aReturns the help text for when the container manager is not found. Args: container_manager (agents_util.ContainerManager): The container manager. current_os (platforms.OperatingSystem): The current operating system. executed_command (str): The command that was executed. Returns: str: The help text for when the container manager is not found. Raises: ValueError: If the container manager is not supported. z!Container manager not supported: )rinstallation_guide_urlr6),CONTAINER_MANAGER_INSTALLATION_GUIDE_URL_MAP ValueError%CONTAINER_MANAGER_NOT_FOUND_HELP_TEXTr0rrE)rrAr6rs rrFrFsm JJ 89J8KL MMH / 5 5)//5573' 6 rz Install Transfer Service agents to enable you to transfer data to or from POSIX filesystems, such as on-premises filesystems. Agents are installed locally on your machine and run inside {container_managers} containers. a| To create an agent pool for your agent, see the `gcloud transfer agent-pools create` command. To install an agent that authenticates with your user account credentials and has default agent parameters, run: $ {command} --pool=AGENT_POOL You will be prompted to run a command to generate a credentials file if one does not already exist. To install an agent that authenticates with a service account with credentials stored at '/example/path.json', run: $ {command} --creds-file=/example/path.json --pool=AGENT_POOL To install an agent using service account impersonation, run: $ {command} --creds-file=/example/path.json --pool=CUSTOM_AGENT_POOL --impersonate-service-account=impersonated-account@project-id.iam.gserviceaccount.com Note : The `--impersonate-service-account` flag only applies to the API calls made by gcloud during the agent installation and authorization process. The impersonated credentials are not passed to the transfer agent's runtime environment. The agent itself does not support impersonation and will use the credentials provided via the `--creds-file` flag or the default gcloud authenticated account for all of its operations. To grant the agent permissions, you must provide a service account key with the required direct roles (e.g., Storage Transfer Agent, Storage Object User) cU[RR:HnU(aSOSn[R US9nU[ S.$)zReturns the detailed help dictionary for the install command based on the release track. Args: release_track (base.ReleaseTrack): The release track. Returns: dict[str, str]: The detailed help dictionary for the install command. zDocker or PodmanDockercontainer_managers) DESCRIPTIONEXAMPLES)r ReleaseTrackALPHAINSTALL_CMD_DESCRIPTION_TEXTr0INSTALL_CMD_EXAMPLES_TEXT) release_trackis_alphardescription_texts r_get_detailed_help_textrIsPd//55 5(-5)8188+9&+ rc\rSrSrSr\"\RR5r \ \RR4Sj5r Sr Sr g)Installi] Install Transfer Service agents.cvURSSSS9 URS[[S9 URS[S9 U[R R :Xa"URS S [RS S 9S 9 URS[RSS9 URSSS9 URSSSS9 URSS[SS9 URS[R"5S[S9 URS[S9 URSS [S9 URS!S"S#9nURS$S%S9 URS&S'S9 URS(/S)QS*S+9 URS,S"S#9nURS-S.S9 URS/S0S9 URS1S2S9 URS3S4S9 g5)6zAdd arguments for the install command. Args: parser (argparse.ArgumentParser): The argument parser for the command. release_track (base.ReleaseTrack): The release track. z--poolTzThe agent pool to associate with the newly installed agent. When creating transfer jobs, the agent pool parameter will determine which agents are activated.)requiredhelpz--counttyperz --creds-filerz--docker-networkrpdockerrdestrrKaSplit up files and transfer the resulting chunks in parallel before merging them at the destination. Can be used make transfers of large files faster as long as the network and disk speed are not limiting factors. If unset, agent decides when to use the feature.)actionrz --id-prefixzEAn optional prefix to add to the agent ID to help identify the agent.z--logs-directoryz/tmpzSpecify the absolute path to the directory you want to store transfer logs in. If not specified, gcloud transfer will mount your /tmp directory for logs.)defaultrz--memlock-limitizSet the agent container's memlock limit. A value of 64000000 (default) or higher is required to ensure that agent versions 1.14 or later have enough locked memory to be able to start.)rrrz--mount-directorieszMOUNT-DIRECTORIES)rmetavarrz--proxyz--s3-compatible-mode store_trueHDFSF)category sort_argsrOaA URI representing an HDFS cluster including a schema, namenode, and port. Examples: "rpc://my-namenode:8020", "http://my-namenode:9870". Use "http" or "https" for WebHDFS. If no schema is provided, the CLI assumes native "rpc". If no port is provided, the default is 8020 for RPC, 9870 for HTTP, and 9871 for HTTPS. For example, the input "my-namenode" becomes "rpc://my-namenode:8020".rQz/". Realm is mapped from your Kerberos config. Any supplied realm is ignored. If not passed in, it will default to "hdfs/" (fqdn = fully qualified domain name).N) add_argumentintCOUNT_FLAG_HELP_TEXTCREDS_FILE_FLAG_HELP_TEXTr rGANETWORK_HELP_TEXTr0rStoreTrueFalseActionArgListMOUNT_DIRECTORIES_HELP_TEXTPROXY_FLAG_HELP_TEXTS3_COMPATIBLE_HELP_TEXT add_group)parserr hdfs_groupkerberos_groups rArgs Install.Argsdsx '(   2FG -FG)),,,   ''8'D  //NO  $%   H I   "# (    (<=  $& !!"J )  K): & %%&N 'F   8 # " &  rc URbURS:a [S5e[RRR R 5nU(d[[5e[R"SS5n[R"UR5RURRR :wa[SUR-5e[#UR$5n[&R(R+S5 [,R.R1U5n[3U5 [&R(R+SR5UR6R9555 [;XU5nURb![=URUR>U5 [&R(R+S 5 US S :Hn[&R(R+[@R5URU[CUUS 5[CUUS 5S95 g)zVInstalls the agent. Args: args (argparse.Namespace): The arguments to the command. Nrz&Agent count must be greater than zero.transferv1zAgent pool not found: u[1/3] Credentials found ✓u[2/3] {} found ✓u&[3/3] Agent installation complete! ✓rrflogslist)ryr logs_command list_command)"r}rr rrrrMISSING_PROJECT_ERROR_TEXTrGetMessagesModulerapi_getrystate AgentPoolStateValueValuesEnumCREATEDr? creds_filer rrr rkrlrIr0rrErrr|&CHECK_AGENT_CONNECTED_HELP_TEXT_FORMATr )selfrrmessagesr4rrrs rRun Install.Runs  zz$**q. ? @@$$,,002G  1 22%%j$7H  +11//77 8 /$)); <<;DOOLOJJ23 $44>>tD,->?JJ)001B1H1H1N1N1PQR2  zz DNN!#v-HJJ.552! 3! 6 rrN)__name__ __module__ __qualname____firstlineno____doc__rr rr detailed_help staticmethodrr__static_attributes__rrrrr]sF))$*;*;*>*>?-!%!2!2!5!5xxt7rrc`\rSrSrSr\"\RR5r \ S5r Sr g) InstallAlphairc [RU[RRS9 UR S[ SS9 UR S[[RVs/sHoRPM sn5[RRRSS9 UR SS [RS S 9S 9 UR S SS9 gs snf)zwAdd arguments for the install command. Args: parser (argparse.ArgumentParser): The argument parser for the command. )rz#--max-concurrent-small-file-uploadszAdjust the maximum number of files less than or equal to 32 MiB large that the agent can upload in parallel. Not recommended for users unfamiliar with Google Cloud's rate limiting.rz--container-managerz0The container manager to use for running agents.)rrrz --networkrpz(docker or podman)rrz--gcs-api-endpointzThe API endpoint for Google Cloud Storage. Override to use a regional endpoint, ensuring data remains within designated geographic boundaries.rN)rrr rrrrsortedr rkrDOCKERrr0)roptions rrInstallAlpha.Args s LLt'8'8'>'>L? - ?@ (3(D(D E(Df\\(D E ,,3399 ?     % %9M % N   & Fs%CrN) rrrrrrr rrrrrrrrrr r s/))$*;*;*A*AB-%%rr )N)F)Lr __future__rrr collectionsrr"rBrzrr+googlecloudsdk.api_lib.transferrgooglecloudsdk.api_lib.utilrgooglecloudsdk.callioperr #googlecloudsdk.command_lib.transferr r googlecloudsdk.corer r googlecloudsdk.core.credentialsr'googlecloudsdk.core.universe_descriptorrgooglecloudsdk.core.utilr oauth2clientrr;rrrrrrr:rrrkr  defaultdictrGLINUXWINDOWSMACOSXPODMANrrr r'r-r2r?rIr\rdrrrrFrrrUniverseCompatible ReleaseTracksrrCommandrrr rrrr#sZ4&'  <,/(;:#*5G.0 " *& $  '')@)@9  % % + +9  % % - -p  % % , ,c *   '')@)@6  % % + +J  % % - ->  % % , ,< *#!0,H)%$ 3H)5Z"22996  H8=u-p'  @(D%%(()wdllw*wtD%%++,+7+-+r