| SrSSKJr SSKJr SSKJr SSKrSSKrSSKrSSKrSSK r SSK J r SSK J r SSK Jr SS KJr SS KJr SS KJr SS KJr SS KJr SSKJr SSKJr SSKJr SSKJr SSK J!r! SSK"J#r# SSK$J%r% SSK$J&r& SSK$J'r' Sr(Sr)Sr*Sr+Sr,Sr-"SS\%R\5r/"SS \R`5r1S!r2\Rf"S"S#S$S%S&9r4\Rf"S'\5S(S)9r6\Rf"S*\5S+S)9r7\Rf"S,\5S-S)9r8\Rf"S.\5S/S0S19r9\Rf"S2S3S49r:\Rf"S5S6S49r;\Rf"S7S8S49r<\Rf"S9S:S49r=\Rf"S;/\R|"5S9r?S?r@S@rASSAjrB\Rf"SBS$SCSD9rC\Rf"SESFS49rD\Rf"SB\R"5S$\ R"SG5SH9rG\Rf"SIS#S$S$\ R"SJ5SK9rHSSMjrISSNjrJSSOjrK\Rf"SPSQS49rL\Rf"SRSSS49rM\Rf"ST\R"SU\RSLSV9S$SW\R"\'RRR5SXSY9rT\Rf"SZS#S[S\9rU\Rf"S]S$SS^S_9rVS`SaSbSc.rW\!R"Sd\ R"5RR\WSeSf9r\Sgr]Shr^SSijr_Sjr`\Rf"Sk\RRSlS)9rc\Rf"SmS$\dSnSo9reSprfSqrgSrrhSsriStrjSurkSvrlSwrmSSxjrnSSyjroSzrpS{rqS|rrS}rsS~rtSruSSjrv\!R"S\ R"5RRSSS$S9ry\!R"S\ R"5RRSSS$S9rz\!R"S\ R"5RRSSS$S9r}Sr~SrSrSrSrSSjrSrSrSrSrSrSrSrSrSrSrSrSrSrSrSrg)z>Provides common arguments for the AI Platform command surface.)absolute_import)division)unicode_literalsN)jobs) versions_api) storage_util)actions) arg_parsers)base)concepts) completers)iam_util) resource_args) constants) models_util) arg_utils)concept_parsers) exceptions)log) propertieszptable[box,title="Job Overview"]( jobId, createTime, startTime, endTime, state, {INPUT}, {OUTPUT}) a(trainingInput:format='table[box,title="Training Input Summary"]( runtimeVersion:optional, region, scaleTier:optional, pythonModule, parameterServerType:optional, parameterServerCount:optional, masterType:optional, workerType:optional, workerCount:optional, jobDir:optional )' ztrainingOutput:format='table[box,title="Training Output Summary"]( completedTrialCount:optional:label=TRIALS, consumedMLUnits:label=ML_UNITS)' {HP_OUTPUT} z,trainingOutput.trials.sort(trialId):format='table[box,title="Training Output Trials"]( trialId:label=TRIAL, finalMetric.objectiveValue:label=OBJECTIVE_VALUE, finalMetric.trainingStep:label=STEP, hyperparameters.list(separator=" "))' zpredictionInput:format='table[box,title="Predict Input Summary"]( runtimeVersion:optional, region, model.basename():optional, versionName.basename(), outputPath, uri:optional, dataFormat, batchSize:optional )' zpredictionOutput:format='table[box,title="Predict Output Summary"]( errorCount, nodeHours, outputPath, predictionCount )' c\rSrSrSrg) ArgumentErrornN)__name__ __module__ __qualname____firstlineno____static_attributes__r1lib/googlecloudsdk/command_lib/ml_engine/flags.pyrrnsr rc(^\rSrSrU4SjrSrU=r$)MlEngineIamRolesCompleterrc P>[[U] "S[RSS.UD6 g)Nmodel)resource_collection resource_destr)superr#__init__rMODELS_COLLECTION)selfkwargs __class__s r!r*"MlEngineIamRolesCompleter.__init__ts. #T3'99 r r)rrrrr*r __classcell__)r.s@r!r#r#rs r r#c J[R"SSSSRUS9S9$)Nz --descriptionFzDescription of the {noun}.nounrequireddefaulthelp)r Argumentformatr2s r!GetDescriptionFlagr:{s-  ' . .D . 9  ;;r z --distributed store_trueFzwRuns the provided code in distributed mode by providing cluster configurations as environment variables to subprocesses)actionr6r7--parameter-server-countzdNumber of parameter servers with which to run. Ignored if --distributed is not specified. Default: 2typer7--worker-countzZNumber of workers with which to run. Ignored if --distributed is not specified. Default: 2z--evaluator-countz]Number of evaluators with which to run. Ignored if --distributed is not specified. Default: 0z --start-porti.jzStart of the range of ports reserved by the local cluster. This command will use a contiguous block of ports equal to parameter-server-count + worker-count + 1. If --distributed is not specified, this flag is ignored. )r?r6r7 operationzName of the operation.r7z--configa2Path to the job configuration file. This file should be a YAML document (JSON also accepted) containing a Job resource as defined in the API (all fields are optional): https://cloud.google.com/ml/reference/rest/v1/projects.jobs EXAMPLES: JSON: { "jobId": "my_job", "labels": { "type": "prod", "owner": "alice" }, "trainingInput": { "scaleTier": "BASIC", "packageUris": [ "gs://my/package/path" ], "region": "us-east1" } } YAML: jobId: my_job labels: type: prod owner: alice trainingInput: scaleTier: BASIC packageUris: - gs://my/package/path region: us-east1 If an option is specified both in the configuration file **and** via command line arguments, the command line arguments override the configuration file. jobzName of the job.z--package-pathaXPath to a Python package to build. This should point to a *local* directory containing the Python source for the job. It will be built using *setuptools* (which must be installed) using its *parent* directory as context. If the parent directory contains a `setup.py` file, the build will use that; otherwise, it will use a simple built-in one. z --packagesPACKAGEzPath to Python archives used for training. These can be local paths (absolute or relative), in which case they will be uploaded to the Cloud Storage bucket given by `--staging-bucket`, or Cloud Storage URLs ('gs://bucket-name/path/to/package.tar.gz'). )r6r?metavarr7a>Google Cloud region of the regional endpoint to use for this command. If unspecified, the command uses the global endpoint of the AI Platform Training and Prediction API. Learn more about regional endpoints and see a list of available regions: https://cloud.google.com/ai-platform/prediction/docs/regional-endpoints a Google Cloud region of the regional endpoint to use for this command. For the global endpoint, the region needs to be specified as `global`. Learn more about regional endpoints and see a list of available regions: https://cloud.google.com/ai-platform/prediction/docs/regional-endpoints cU(a([R"S[R[S9$[R"S[R [ S9$)zAAdds --region flag to determine endpoint for models and versions.z--region)choicesr7)r r8rSUPPORTED_REGIONS_WITH_GLOBAL!_REGION_FLAG_WITH_GLOBAL_HELPTEXTSUPPORTED_REGIONS_REGION_FLAG_HELPTEXT)include_globals r! GetRegionArgrMsG ==77 . 00 ))  ""r z--service-accountz:Specifies the service account for resource access control.r5r7z --networkaFull name of the Google Compute Engine network (https://cloud.google.com/vpc/docs) to which the Job is peered with. For example, ``projects/12345/global/networks/myVPC''. The format is of the form projects/{project}/global/networks/{network}, where {project} is a project number, as in '12345', and {network} is network name. Private services access must already have been configured (https://cloud.google.com/vpc/docs/configure-private-services-access) for the network. If unspecified, the Job is not peered with any network. ad The email address of a service account to use when running the training appplication. You must have the `iam.serviceAccounts.actAs` permission for the specified service account. In addition, the AI Platform Training Google-managed service account must have the `roles/iam.serviceAccountAdmin` role for the specified service account. [Learn more about configuring a service account.](https://cloud.google.com/ai-platform/training/docs/custom-service-account) If not specified, the AI Platform Training Google-managed service account is used by default. r?r5r7z--enable-web-accessa Whether you want AI Platform Training to enable [interactive shell access] (https://cloud.google.com/ai-platform/training/docs/monitor-debug-interactive-shell) to training containers. If set to `true`, you can access interactive shells at the URIs given by TrainingOutput.web_access_uris or HyperparameterOutput.web_access_uris (within TrainingOutput.trials). )r<r5r6r7Tc.[R"SUSS9$)Nz --module-namezName of the module to run.rNr r8r5s r!GetModuleNameFlagrS7s /K MMr cSRSU(aSOS-S9nU(aUS- nU(a[nO-[R"[R R SS9n[R"S X2S 9$) a Get base.Argument() for `--job-dir`. If allow_local is provided, this Argument gives a str when parsed; otherwise, it gives a (possibly empty) ObjectReference. Args: upload_help: bool, whether to include help text related to object upload. Only useful in remote situations (`jobs submit training`). allow_local: bool, whether to allow local directories (only useful in local situations, like `local train`) or restrict input to directories in Cloud Storage. Returns: base.Argument() for the corresponding `--job-dir` flag. ap{dir_type} in which to store training outputs and other data needed for training. This path will be passed to your TensorFlow program as the `--job-dir` command-line arg. The benefit of specifying this field is that AI Platform will validate the path for use in training. However, note that your training program will need to parse the provided `--job-dir` argument. zCloud Storage pathz or local_directory)dir_typezf If packages must be uploaded and `--staging-bucket` is not provided, this path will be used instead. T)allow_empty_objectz --job-dirr>) r9str functoolspartialrObjectReference FromArgumentr r8) upload_help allow_localhelp_type_s r! GetJobDirFlagra<s}   F"(3$= ? E  E   l::GG15 7E { ;;r cbU(aSnOSn[R"S[RUS9$)NzAdditional user arguments to be forwarded to user code. Any relative paths will be relative to the *parent* directory of `--package-path`. z6Additional user arguments to be forwarded to user code user_args)nargsr7)r r8argparse REMAINDER)local help_texts r! GetUserArgsrifs4 I II      r versionzName of the model version.z--runtime-versionzAI Platform runtime version for this job. Must be specified unless --master-image-uri is specified instead. It is defined in documentation along with the list of supported versions: https://cloud.google.com/ai-platform/prediction/docs/runtime-version-listz--polling-interval unlimited<zKNumber of seconds to wait between efforts to fetch the latest log messages.)r?r5r6r<r7z--allow-multiline-logsz0Output multiline log messages as single records.)r<r7z --task-namez7If set, display only the logs for this particular task.r4 tensorflowz scikit-learnxgboost) TENSORFLOW SCIKIT_LEARNXGBOOSTz --frameworkz`ML framework used to train this version of the model. If not specified, defaults to 'tensorflow')custom_mappingshelp_strcRSRSS5n[R"XUS9 g)Nz{} must hold permission {}z/The 'AI Platform Service Agent' service accountz)'Cloud KMS CryptoKey Encrypter/Decrypter')permission_info)r9kms_resource_argsAddKmsKeyResourceArg)parserresourcerws r! AddKmsKeyFlagr|s-077713/(( 9r cnSRUS9n[R"SUS9nURU5 g)Na Version of Python used {context}. Choices are 3.7, 3.5, and 2.7. However, this value must be compatible with the chosen runtime version for the job. Must be used with a compatible runtime version: * 3.7 is compatible with runtime versions 1.15 and later. * 3.5 is compatible with runtime versions 1.4 through 1.14. * 2.7 is compatible with runtime versions 1.15 and earlier. )contextz--python-versionrB)r9r r8 AddToParser)rzr~rurjs r!AddPythonVersionFlagrs@   && !  MM  ' fr chSnU(a[R"SUS9$[R"SX!S9$)NzName of the model.r&rB--model)r7r5rQ) positionalr5rhs r! GetModelNamers,") ==y 11 == FFr cURbg[UR5S:a[R"S5 [ [ R RUR55Ulgg)z?Flatten PACKAGES flag and warn if multiple arguments were used.NrkzoUse of --packages with space separated values is deprecated and will not work in the future. Use comma instead.)packageslenrwarninglist itertoolschain from_iterable)argss r!ProcessPackagesrsT ]] 4==A kk66t}}EFDM r z--staging-bucketz Bucket in which to stage training archives. Required only if a file upload is necessary (that is, other flags include local paths) and no other flags implicitly specify an upload path. z--signature-namea" Name of the signature defined in the SavedModel to use for this job. Defaults to DEFAULT_SERVING_SIGNATURE_DEF_KEY in https://www.tensorflow.org/api_docs/python/tf/compat/v1/saved_model/signature_constants, which is "serving_default". Only applies to TensorFlow models. r5r?r7c0[R"SSSSS9$)Nz --summarizer;Fa Summarize job output in a set of human readable tables instead of rendering the entire resource as json or yaml. The tables currently rendered are: * `Job Overview`: Overview of job including, jobId, status and create time. * `Training Input Summary`: Summary of input for a training job including region, main training python module and scale tier. * `Training Output Summary`: Summary of output for a training job including the amount of ML units consumed by the job. * `Training Output Trials`: Summary of hyperparameter trials run for a hyperparameter tuning training job. * `Predict Input Summary`: Summary of input for a prediction job including region, model verion and output path. * `Predict Output Summary`: Summary of output for a prediction job including prediction count and output path. This flag overrides the `--format` flag. If both are present on the command line, a warning is displayed. )r<r5r7rQrr r!GetSummarizeFlagrs!     r cT[R[[RSS9S9$)z0Get tabular format for standard ml training job.rU HP_OUTPUTINPUTOUTPUT) _JOB_SUMMARYr9_JOB_TRAIN_INPUT_SUMMARY_FORMAT _JOB_TRAIN_OUTPUT_SUMMARY_FORMATrr r!GetStandardTrainingJobSummaryrs0    + - 4 4r 4 B  DDr c\[R[[R[S9S9$)z4Get tablular format to HyperParameter tuning ml job.rr)rr9!_JOB_PREDICT_INPUT_SUMMARY_FORMATr_JOB_TRAIN_OUTPUT_TRIALS_FORMATrr r!GetHPTrainingJobSummaryrs3    - - 4 4355  66r c:[R[[S9$)z'Get table format for ml prediction job.r)rr9r"_JOB_PREDICT_OUTPUT_SUMMARY_FORMATrr r!GetPredictJobSummaryrs    - /  11r c,[R"SSS9$)Nr&zModel for the {resource}.namerhr ResourceParameterAttributeConfigrr r!ModelAttributeConfigrs  2 2 + --r c,[R"SSS9$)NrjzVersion for the {resource}.rrrr r!VersionAttributeConfigrs  2 2 - //r cn[R"SS[5[5[RS9$)Nzml.projects.models.versionsrj) resource_name versionsIdmodelsId projectsId)r ResourceSpecrr DEFAULT_PROJECT_ATTRIBUTE_CONFIGrr r!GetVersionResourceSpecr"s0   #')#%::  <) r r8r ArgDictrXintr9join_OP_ACCELERATOR_TYPE_MAPPERrG)cs r!GetAcceleratorFlagrs     #8  % vdii:BBCB!V]]1BCEF% GG&DsBc [RVs/sHoPM nnUcgURSS5nX2;aI[SR SR UVs/sHnSR U5PM sn555eURSS5nUS::a [S 5e[ R"5Rn[R"X5R5nU"UUS 9$s snfs snf) z:Validates and returns a accelerator config message object.Nr?zBThe type of the accelerator can only be one of the following: {}. rrrrkrz5The count of the accelerator must be greater than 0. )rr?) _ACCELERATOR_TYPE_MAPPERrGgetrr9rrGetMessagesModule GoogleCloudMlV1AcceleratorConfigr ChoiceToEnumTypeValueValuesEnum) acceleratorrtypesraw_typeaccelerator_countaccelerator_msgaccelerator_types r!ParseAcceleratorFlagrs.66 7616% 7  __VT *(   F49961fmmA&6 78 ::"oogq1!  "33555++335    # 87s C-C2c x[R"S/SQSSS9RU5 [R"S[R "S[ RSS 9S SS S 9RU5 [R"S [R "S[ RSS 9S SSS 9RU5 g)z'Add args that configure explainability.z--explanation-method)zintegrated-gradientszsampled-shapleyxraiFz Enable explanations and select the explanation method to use. The valid options are: integrated-gradients: Use Integrated Gradients. sampled-shapley: Use Sampled Shapley. xrai: Use XRAI. )rGr5ruz--num-integral-stepsrkTrl2z Number of integral steps for Integrated Gradients. Only valid when `--explanation-method=integrated-gradients` or `--explanation-method=xrai` is specified. )r?r6r5r7z --num-pathsz Number of paths for Sampled Shapley. Only valid when `--explanation-method=sampled-shapley` is specified. N)r ChoiceArgumentrr8r BoundedIntsysmaxsizerzs r!AddExplainabilityFlagsrsA K--  ! !!S[[D A K--  ! !!S[[D A Kr c:[5RU5 [5RU5 [5RU5 [ 5RU5 [ 5RU5 [ 5RU5 [5RU5 [5RU5 [5RU5 [5RU5 U(a[5RU5 gg)z%Add Custom container flags to parser.N) GetMasterMachineTyperGetMasterAcceleratorGetMasterImageUri#GetParameterServerMachineTypeConfigGetParameterServerAcceleratorGetParameterServerImageUriGetWorkerMachineConfigGetWorkerAcceleratorGetWorkerImageUriGetUseChiefInTfConfigGetTpuTfVersion)rzsupport_tpu_tf_versions r!AddCustomContainerFlagsrs$$V,$$V,!!&)%'33F;!--f5**62&&v.$$V,!!&)%%f-!!&)r zgeneric-acceleratorz Available types of accelerators.c US:g$)NACCELERATOR_TYPE_UNSPECIFIEDrxs r!rs Q"@@r )ruinclude_filterr5c$URS5$)NNVIDIA) startswithrs r!rrs Q\\(3r zautoscaling-metric-namezThe available metric names.c US:g$)NMETRIC_NAME_UNSPECIFIEDrrs r!rrs Q";;r z Hardware accelerator config for the {worker_type}. Must specify both the accelerator type (TYPE) for each server and the number of accelerators to attach to each server (COUNT). ch[R"U[RS[RS9$)Nr) item_type valid_choices)rrrenumrG) choice_strs r!"_ConvertAcceleratorTypeToEnumValuers.    ,D,I,I*7.F.N.N PPr cT[U5nUS::a[R"S5eU$)Nrz4The count of the accelerator must be greater than 0.)rr ArgumentTypeError)rrs r!_ValidateAcceleratorCountrs1  % aZ  ' '> @@ ,r c [[R5nX;aI[SR SR UVs/sHnSR U5PM sn555eU$s snf)z(Value validation for Metric target name.zBThe autoscaling metric name can only be one of the following: {}. rr)r"_OP_AUTOSCALING_METRIC_NAME_MAPPERrGrr9r)keynamesrs r!_ValidateMetricTargetKeyrsb 199 :%  F49961fmmA&6 78 :: *7sA*c[U5nUS:dUS:a[SU-5eU$![[4a [SU-5ef=f)z)Value validation for Metric target value.z4Metric target percentage value %s is not an integer.rdz0Metric target value %s is not between 0 and 100.)r TypeError ValueErrorr)valueresults r!_ValidateMetricTargetValuer si ZF  aZ6C< :UB DD - Z  N s )Ac SRSR[R55nSn[R "UU[ R"[[S.SS/S9SRXUS 9S 9nU$) z=Creates an ArgDict representing an AcceleratorConfig message.z1*type*::: Type of the accelerator. Choices are {},zd*count*::: Number of accelerators to attach to each machine running the job. Must be greater than 0.rr?rrz'{arg_help} {type_help} {count_help} )arg_help type_help count_helpr) r9rrrGr r8r rrr)arg_namer!r5r"r#args r!_MakeAcceleratorArgConfigArgr&.sAHH hh'//02)C*    4,%( *  FHjFI K# *r c2Sn[R"SSUS9$)zBuild master-machine-type flag.z Specifies the type of virtual machine to use for training job's master worker. You must set this value when `--scale-tier` is set to `CUSTOM`. z--master-machine-typeFrNrQrhs r!rrFs#) I ??r c<[RSS9n[SUS9$)zBuild master-accelerator flag.z master worker worker_typez--master-acceleratorr!_ACCELERATOR_TYPE_HELPr9r&r(s r!rrQs'$+++H) %y 22r c.[R"SSSS9$)zBuild master-image-uri flag.z--master-image-uriFzDocker image to run on each master worker. This image must be in Container Registry. Only one of `--master-image-uri` and `--runtime-version` must be specified.rNrQrr r!rrXs!  N PPr c [R"SSSS9n[R"S[R"S[R SS9SSS 9n[R "S S S9nURU5 URU5 U$) z1Build parameter-server machine type config group.z--parameter-server-machine-typeTzType of virtual machine to use for training job's parameter servers. This flag must be specified if any of the other arguments in this group are specified machine to use for training job's parameter servers.rNr=rkrlz8Number of parameter servers to use for the training job.rOFz1Configure parameter server machine type settings.r r8r rrrrr machine_type machine_countmachine_type_groups r!rrbs' 13,--  ! !!S[[D A F I- )) >@  .  / r c<[RSS9n[SUS9$)z(Build parameter-server-accelerator flag.zparameter serversr*z--parameter-server-acceleratorr,r-r(s r!rrzs($++8K+L) %& <HTTP path to send prediction requests to inside the container.z--health-routez8HTTP path to send health checks to inside the container.N)add_argument_group add_argumentr r UpdateActionrr)rzcontainer_group route_groups r!AddContainerFlagsrIsh-- 4.6/        % %       % %        % % E    K,B,B1e,L M  % %  )) *+ K Er c URSS9nURS[SS9 URS[SS9 URSS [R"[ [ S 9[R0S S 9 g )z:Adds flags related to autoscaling to the specified parser.z2Configure the autoscaling settings to be deployed.rBz --min-nodeszz --max-nodeszz--sampling-percentagezPercentage of requests to be logged, expressed as a number between 0 and 1. For example, set this value to 1 in order to log all requests or to 0.1 to log 10% of requests. N)rDrErXfloat)rzgroups r!AddRequestLoggingConfigFlagsrR(sX  # # 0 $ 2%    r )F)T)TF)r&)TFrU)__doc__ __future__rrrrerYrrtextwrap googlecloudsdk.api_lib.ml_enginerrgooglecloudsdk.api_lib.storagergooglecloudsdk.callioper r r googlecloudsdk.calliope.conceptsr googlecloudsdk.command_lib.iamr iam_completersr core_iam_utilgooglecloudsdk.command_lib.kmsrrx$googlecloudsdk.command_lib.ml_enginerr$googlecloudsdk.command_lib.util.apisr(googlecloudsdk.command_lib.util.conceptsrgooglecloudsdk.corerrrrrrrrrErrorrIamRolesCompleterr#r:r8 DISTRIBUTEDr PARAM_SERVERSWORKERS EVALUATORS START_PORTOPERATION_NAMECONFIGJOB_NAME PACKAGE_PATHrPACKAGESrKrIrMSERVICE_ACCOUNTNETWORKGetIamAccountFormatValidatordedentTRAINING_SERVICE_ACCOUNTENABLE_WEB_ACCESSrSrari VERSION_NAMERUNTIME_VERSIONrr StorePropertyVALUES ml_enginepolling_intervalPOLLING_INTERVALALLOW_MULTILINE_LOGS TASK_NAME_FRAMEWORK_CHOICESChoiceEnumMapperrGoogleCloudMlV1VersionFrameworkValueValuesEnumFRAMEWORK_MAPPERr|rrrBucketReferencer\STAGING_BUCKETrXSIGNATURE_NAMErrrrrrrrrrrrrrrrrrrrrGoogleCloudMlV1MetricSpecNameValueValuesEnumrr.rrrrr&rrrrrrrrrrrr?rIrNrRrr r!rsFE&' 197+/(5GDM:<:D*#*   #$ # %!&"J$$ @ @;mm   D F    BD --  B D ]]  BD ]]    {1IJ ' ) T ==%7 8}}   ==        %! "--  EG --   ==  3 3 5      MM       M '> *;  &"P    0?2P0<F.2F<6r<r