SrSSKJr SSKJr SSKJr SSKrSSKrSSKrSSKrSSK J r SSK J r SSK J r SS K Jr SS K Jr SS KJr S rSS jrSSjrg)z,Utilities for running training jobs locally.)absolute_import)division)unicode_literalsN)execution_utils)log) properties)encoding)files)rangec>[RRRR 5nU=(d2 [ R "S5=(d [R"5nUSS/n[ R"[R5n[R"U[RUS9nSSS5 WR5nUS:wa&[ R""SR%U55 gUR&R)5nS [+U5;aUR-S 5nUR/S 5(ag UR/S 5(ag[ R""SR%U55 g!,(df  N=f)aGet the primary node name. Returns: str, the name of the primary node. If running in tensorflow 1.x, return 'master'. If running in tensorflow 2.x, return 'chief'. If tensorflow is not installed in local envrionment, it will return the default name 'chief'. Raises: ValueError: if there is no python executable on the user system thrown by execution_utils.GetPythonExecutable. pythonz-cz2import tensorflow as tf; print(tf.version.VERSION))stdoutstderrNrau Cannot import tensorflow under path {}. Using "chief" for cluster setting. If this is not intended, Please check if tensorflow is installed. Please also verify if the python path used is correct. If not, to change the python path: use `gcloud config set ml_engine/local_python $python_path` Eg: gcloud config set ml_engine/local_python /usr/bin/python3chiefdecodezutf-8z1.masterz2.zgUnexpected tensorflow version {}, using the default primary node name, aka "chief" for cluster settings)rVALUES ml_engine local_pythonGetr FindExecutableOnPathrGetPythonExecutable FileWriterosdevnull subprocessPopenPIPEwaitrwarningformatrreaddirr startswith) exe_overridepython_executablecmdfproc return_code tf_versions 7lib/googlecloudsdk/command_lib/ml_engine/local_train.pyGetPrimaryNodeNamer-"sP"",,99==?,,e00:,))+  = ?#  #q   C  BD$ +AKKE FLVF  {{!* Z ""7+J4  T"" ++55;VJ5GI /$#s %F Fc ,Uc/n[RRRR 5nU=(d2 [ R "S5=(d [R"5nUSU/U-n XS.U(aXES.O0U=(d 0SS.n [R"SRSRU 5[R"U S S S 9S 95 [R R#5n [R"U 5U S 'U[%5:Xa[R&"U 4U S US.UD6$[(R*"U 5n [,R."U 4U US.UD6n [0R2"[R4U 5 U $)aMake a Popen object that runs the module, with the correct env. If task_type is primary instead replaces the current process with the subprocess via execution_utils.Exec Args: module_name: str. Name of the module to run, e.g. trainer.task package_root: str. Absolute path to the package root for the module. used as CWD for the subprocess. args: [str]. Additional user args. Any relative paths will not work. cluster: dict. Cluster configuration dictionary. Suitable for passing to tf.train.ClusterSpec. task_type: str. Task type of this process. Only relevant if cluster is specified. index: int. Task index of this process. **extra_popen_args: extra args passed to Popen. Used for testing. Returns: a subprocess.Popen object corresponding to the subprocesses or an int corresponding to the return value of the subprocess (if task_type is primary) Raises: ValueError: if there is no python executable on the user system thrown by execution_utils.GetPythonExecutable. r z-m)job_nameargs)typeindexcloud)jobtaskcluster environmentzr@)rrrrrr rrrrinfor!joinjsondumpsrenvironcopyr-Execr EncodeEnvrratexitregisterKillSubprocess) module_name package_rootr0r6 task_typer2extra_popen_argsr%r&r'r<r>r5s r, MakeProcessrPOs< \ D"",,99==?,,e00:,))+ D+.5#% 45upU [5:wdM[[ U 55Hn [ UUUU U U S9 M M@ [U5Hn [ UUUS U U S9 M [ UUU[5SU S9$s snfs snf) a-Create a cluster configuration and start processes for the cluster. Args: module_name: str. Python module to use as the task. package_root: str. Absolute path to the package root of the module. num_ps: int. Number of parameter servers num_workers: int. Number of workers. num_evaluators: int. Number of evaluators. start_port: int. First port for the contiguous block of ports used by the cluster. user_args: [str]. Additional user args for the task. Any relative paths will not work. Returns: int. the retval of primary subprocess zlocalhost:{port}r)portpsworkerN)r0rNr2r6 evaluator)listr r-r!itemslenrP) rLrMnum_ps num_workersnum_evaluators start_port user_argsportspr6rN addressesis r,RunDistributedrdsW, uZf!4{!BQ!FG H%/66E!H6EF Afqj) +) & &A & .) + ,.,Q$***2,.  '&mmoi&((S^$!K "'#  %%.  a % ! ! [!#13$  &&+ +.s D )D)NNNN)N)__doc__ __future__rrrrIrCrrgooglecloudsdk.corerrrgooglecloudsdk.core.utilr r six.movesr r-rPrdrQr,rksV3&' /#*-**^ AT" 3&rQ