_SrSSKJr SSKJr SSKJr SSKrSSKJr SSKJ r SSK J r SS K J r S r S rS rS \S\4SjrS\S\4SjrSr"SS\5rg)z*Utilities for Vertex AI Model Garden APIs.)absolute_import)division)unicode_literalsN) list_pager)apis) constants)flagszis_hf_wildcard(true)zis_hf_wildcard(false)z=labels.VERIFIED_DEPLOYMENT_CONFIG=VERIFIED_DEPLOYMENT_SUCCEED model_namereturncB[[R"SU55$)z2Returns whether the model is a Hugging Face model.z^[^/]+/[^/@]+$boolrematch)r s 4lib/googlecloudsdk/api_lib/ai/model_garden/client.pyIsHuggingFaceModelr#s bhh(*5 66modelcB[[R"SU55$)z4Returns whether the model is a custom weights model.z^gs://r )rs rIsCustomWeightsModelr(s bhhy%( ))rc UR5nURUS9UlU(a1URUR UR UUUS9SS9S9UlURSUSU3US9n URU 5$) zDeploys a custom weights model.)gcsUri) machineTypeacceleratorTypeacceleratorCount) machineSpecminReplicaCount)dedicatedResources projects/ /locations/ destination)googleCloudAiplatformV1beta1DeployRequest) )GoogleCloudAiplatformV1beta1DeployRequest4GoogleCloudAiplatformV1beta1DeployRequestCustomModel customModel5GoogleCloudAiplatformV1beta1DeployRequestDeployConfig.GoogleCloudAiplatformV1beta1DedicatedResources'GoogleCloudAiplatformV1beta1MachineSpec deployConfig(AiplatformProjectsLocationsDeployRequestDeploy) messagesprojects_locations_servicer machine_typeaccelerator_typeaccelerator_countprojectlocationdeploy_requestrequests rDeployCustomWeightsModelr7-sEEG.CCD "*"`"`#RR HH( 0!2I  S #a #N  = =gYk(<0> > ' $ * *7 33rc\rSrSrSr\R 4SjrS SjrS Sjr S Sjr Sr g) ModelGardenClientRz3Client used for interacting with Model Garden APIs.c[R"[R[RU5nUR UlURUlURUl g)N) rGetClientInstancerAI_PLATFORM_API_NAMEAI_PLATFORM_API_VERSIONMESSAGES_MODULE _messagespublishers_models_publishers_models_serviceprojects_locations_projects_locations_service)selfversionclients r__init__ModelGardenClient.__init__UsT  # #&&))'2F++DN&,&>&>D#'-'@'@D$rNcpURRUUUUS9nURRU5$)aGet a publisher model. Args: model_name: The name of the model to get. The format should be publishers/{publisher}/models/{model} is_hugging_face_model: Whether the model is a hugging face model. include_equivalent_model_garden_model_deployment_configs: Whether to include equivalent Model Garden model deployment configs for Hugging Face models. hugging_face_token: The Hugging Face access token to access the model artifacts for gated models unverified by Model Garden. Returns: A publisher model. )nameisHuggingFaceModel2includeEquivalentModelGardenModelDeploymentConfigshuggingFaceToken)r@$AiplatformPublishersModelsGetRequestrBGet)rEr is_hugging_face_model8include_equivalent_model_garden_model_deployment_configshugging_face_tokenr6s rGetPublisherModel#ModelGardenClient.GetPublisherModel^sC,nnAA 0;s+ BG  * * . .w 77rc jSnU(GaURRUUUS9nU(aUUlU(aUUlU(a3UVs/sH nURR UUUS9PM" snUlU(a/UV s/sHn URR U S9PM sn UlU(a/UV s/sHn URR U S9PM sn UlU(a[U5S-Ul U(aUUl U(dU(dU(aBSn!U(aURRUS9n!URRU!UUS9UlU(dU(dU(aBSn"U(aURRUS9n"URRU"UUS9Ul[!U5(a&[#URUR$UUUUUU5$['U5(aURR)US9n#OURR)US 9n#URR+U UUS 9U#lURR/UU S 9U#lURR3URR5URR7UUU[8R:"U [<R>5S 9S U S9U S9U#l URRCSUSU3U#S9n$UR$REU$5$s snfs sn fs sn f)a+0Deploy an open weight model. Args: project: The project to deploy the model to. location: The location to deploy the model to. model: The name of the model to deploy or its gcs uri for custom weights. accept_eula: Whether to accept the end-user license agreement. accelerator_type: The type of accelerator to use. accelerator_count: The number of accelerators to use. machine_type: The type of machine to use. endpoint_display_name: The display name of the endpoint. hugging_face_access_token: The Hugging Face access token. spot: Whether to deploy the model on Spot VMs. reservation_affinity: The reservation affinity to use. use_dedicated_endpoint: Whether to use a dedicated endpoint. enable_fast_tryout: Whether to enable fast tryout. container_image_uri: Immutable. URI of the Docker image to be used as the custom container for serving predictions. This URI must identify an image in Artifact Registry or Container Registry. Learn more about the [container publishing requirements](https://cloud.google.com/vertex- ai/docs/predictions/custom-container-requirements#publishing), including permissions requirements for the Vertex AI Service Agent. The container image is ingested upon ModelService.UploadModel, stored internally, and this original path is afterwards not used. To learn about the requirements for the Docker image itself, see [Custom container requirements](https://cloud.google.com/vertex- ai/docs/predictions/custom-container-requirements#). You can use the URI to one of Vertex AI's [pre-built container images for prediction](https://cloud.google.com/vertex-ai/docs/predictions/pre- built-containers) in this field. container_command: Specifies the command that runs when the container starts. This overrides the container's [ENTRYPOINT](https://docs.docker. com/engine/reference/builder/#entrypoint). Specify this field as an array of executable and arguments, similar to a Docker `ENTRYPOINT`'s "exec" form, not its "shell" form. If you do not specify this field, then the container's `ENTRYPOINT` runs, in conjunction with the args field or the container's [`CMD`](https://docs.docker.com/engine/reference/builder/#cmd), if either exists. If this field is not specified and the container does not have an `ENTRYPOINT`, then refer to the Docker documentation about [how `CMD` and `ENTRYPOINT` interact](https://docs.docker.com/engine/reference/builder/#understand- how-cmd-and-entrypoint-interact). If you specify this field, then you can also specify the `args` field to provide additional arguments for this command. However, if you specify this field, then the container's `CMD` is ignored. See the [Kubernetes documentation about how the `command` and `args` fields interact with a container's `ENTRYPOINT` and `CMD`](https://kubernetes.io/docs/tasks/inject-data-application/define- command-argument-container/#notes). In this field, you can reference [environment variables set by Vertex AI](https://cloud.google.com/vertex-ai/docs/predictions/custom- container-requirements#aip-variables) and environment variables set in the env field. You cannot reference environment variables set in the Docker image. In order for environment variables to be expanded, reference them by using the following syntax: $( VARIABLE_NAME) Note that this differs from Bash variable expansion, which does not use parentheses. If a variable cannot be resolved, the reference in the input string is used unchanged. To avoid variable expansion, you can escape this syntax with `$$`; for example: $$(VARIABLE_NAME) This field corresponds to the `command` field of the Kubernetes Containers [v1 core API](https://kubernetes.io/docs/reference/generated/kubernetes- api/v1.23/#container-v1-core). container_args: Specifies arguments for the command that runs when the container starts. This overrides the container's [`CMD`](https://docs.docker.com/engine/reference/builder/#cmd). Specify this field as an array of executable and arguments, similar to a Docker `CMD`'s "default parameters" form. If you don't specify this field but do specify the command field, then the command from the `command` field runs without any additional arguments. See the [Kubernetes documentation about how the `command` and `args` fields interact with a container's `ENTRYPOINT` and `CMD`](https://kubernetes.io/docs/tasks/inject-data- application/define-command-argument-container/#notes). If you don't specify this field and don't specify the `command` field, then the container's [`ENTRYPOINT`](https://docs.docker.com/engine/reference/builder/#cmd) and `CMD` determine what runs based on their default behavior. See the Docker documentation about [how `CMD` and `ENTRYPOINT` interact](https://docs.docker.com/engine/reference/builder/#understand- how-cmd-and-entrypoint-interact). In this field, you can reference [environment variables set by Vertex AI](https://cloud.google.com/vertex-ai/docs/predictions/custom- container-requirements#aip-variables) and environment variables set in the env field. You cannot reference environment variables set in the Docker image. In order for environment variables to be expanded, reference them by using the following syntax: $( VARIABLE_NAME) Note that this differs from Bash variable expansion, which does not use parentheses. If a variable cannot be resolved, the reference in the input string is used unchanged. To avoid variable expansion, you can escape this syntax with `$$`; for example: $$(VARIABLE_NAME) This field corresponds to the `args` field of the Kubernetes Containers [v1 core API](https://kubernetes.io/docs/reference/generated/kubernetes- api/v1.23/#container-v1-core).. container_env_vars: List of environment variables to set in the container. After the container starts running, code running in the container can read these environment variables. Additionally, the command and args fields can reference these variables. Later entries in this list can also reference earlier entries. For example, the following example sets the variable `VAR_2` to have the value `foo bar`: ```json [ { "name": "VAR_1", "value": "foo" }, { "name": "VAR_2", "value": "$(VAR_1) bar" } ] ``` If you switch the order of the variables in the example, then the expansion does not occur. This field corresponds to the `env` field of the Kubernetes Containers [v1 core API](https://kubernetes.io/docs/reference/generated/kubernetes- api/v1.23/#container-v1-core). container_ports: List of ports to expose from the container. Vertex AI sends any http prediction requests that it receives to the first port on this list. Vertex AI also sends [liveness and health checks](https://cloud.google.com/vertex-ai/docs/predictions/custom- container-requirements#liveness) to this port. If you do not specify this field, it defaults to following value: ```json [ { "containerPort": 8080 } ] ``` Vertex AI does not use ports other than the first one listed. This field corresponds to the `ports` field of the Kubernetes Containers [v1 core API](https://kubernetes.io/docs/reference/generated/kubernetes- api/v1.23/#container-v1-core). container_grpc_ports: List of ports to expose from the container. Vertex AI sends any grpc prediction requests that it receives to the first port on this list. Vertex AI also sends [liveness and health checks](https://cloud.google.com/vertex-ai/docs/predictions/custom- container-requirements#liveness) to this port. If you do not specify this field, gRPC requests to the container will be disabled. Vertex AI does not use ports other than the first one listed. This field corresponds to the `ports` field of the Kubernetes Containers [v1 core API](https://kubernetes.io/docs/reference/generated/kubernetes- api/v1.23/#container-v1-core). container_predict_route: HTTP path on the container to send prediction requests to. Vertex AI forwards requests sent using projects.locations.endpoints.predict to this path on the container's IP address and port. Vertex AI then returns the container's response in the API response. For example, if you set this field to `/foo`, then when Vertex AI receives a prediction request, it forwards the request body in a POST request to the `/foo` path on the port of your container specified by the first value of this `ModelContainerSpec`'s ports field. If you don't specify this field, it defaults to the following value when you deploy this Model to an Endpoint: /v1/endpoints/ENDPOINT/deployedModels/DEPLOYED_MODEL:predict The placeholders in this value are replaced as follows: * ENDPOINT: The last segment (following `endpoints/`)of the Endpoint.name][] field of the Endpoint where this Model has been deployed. (Vertex AI makes this value available to your container code as the [`AIP_ENDPOINT_ID` environment variable](https://cloud.google.com/vertex-ai/docs/predictions/custom- container-requirements#aip-variables).) * DEPLOYED_MODEL: DeployedModel.id of the `DeployedModel`. (Vertex AI makes this value available to your container code as the [`AIP_DEPLOYED_MODEL_ID` environment variable](https://cloud.google.com/vertex- ai/docs/predictions/custom-container-requirements#aip-variables).) container_health_route: HTTP path on the container to send health checks to. Vertex AI intermittently sends GET requests to this path on the container's IP address and port to check that the container is healthy. Read more about [health checks](https://cloud.google.com/vertex- ai/docs/predictions/custom-container-requirements#health). For example, if you set this field to `/bar`, then Vertex AI intermittently sends a GET request to the `/bar` path on the port of your container specified by the first value of this `ModelContainerSpec`'s ports field. If you don't specify this field, it defaults to the following value when you deploy this Model to an Endpoint: /v1/endpoints/ENDPOINT/deployedModels/ DEPLOYED_MODEL:predict The placeholders in this value are replaced as follows * ENDPOINT: The last segment (following `endpoints/`)of the Endpoint.name][] field of the Endpoint where this Model has been deployed. (Vertex AI makes this value available to your container code as the [`AIP_ENDPOINT_ID` environment variable](https://cloud.google.com/vertex-ai/docs/predictions/custom- container-requirements#aip-variables).) * DEPLOYED_MODEL: DeployedModel.id of the `DeployedModel`. (Vertex AI makes this value available to your container code as the [`AIP_DEPLOYED_MODEL_ID` environment variable](https://cloud.google.com/vertex- ai/docs/predictions/custom-container-requirements#aip-variables).) container_deployment_timeout_seconds (int): Deployment timeout in seconds. container_shared_memory_size_mb (int): The amount of the VM memory to reserve as the shared memory for the model in megabytes. container_startup_probe_exec (Sequence[str]): Exec specifies the action to take. Used by startup probe. An example of this argument would be ["cat", "/tmp/healthy"] container_startup_probe_period_seconds (int): How often (in seconds) to perform the startup probe. Default to 10 seconds. Minimum value is 1. container_startup_probe_timeout_seconds (int): Number of seconds after which the startup probe times out. Defaults to 1 second. Minimum value is 1. container_health_probe_exec (Sequence[str]): Exec specifies the action to take. Used by health probe. An example of this argument would be ["cat", "/tmp/healthy"] container_health_probe_period_seconds (int): How often (in seconds) to perform the health probe. Default to 10 seconds. Minimum value is 1. container_health_probe_timeout_seconds (int): Number of seconds after which the health probe times out. Defaults to 1 second. Minimum value is 1. Returns: The deploy long-running operation. N) healthRouteimageUri predictRoute)rKvalue) containerPorts)command)exec_ periodSecondstimeoutSeconds)huggingFaceModelId)publisherModelName)huggingFaceAccessToken acceptEula containerSpec)endpointDisplayNamededicatedEndpointEnabled)rrrreservationAffinityr)rrspot)rfastTryoutEnabledr r!r")#r@.GoogleCloudAiplatformV1beta1ModelContainerSpecr]args"GoogleCloudAiplatformV1beta1EnvVarenv GoogleCloudAiplatformV1beta1Portports grpcPortsstrdeploymentTimeoutsharedMemorySizeMb+GoogleCloudAiplatformV1beta1ProbeExecAction!GoogleCloudAiplatformV1beta1Probe startupProbe healthProberr7rDrr%4GoogleCloudAiplatformV1beta1DeployRequestModelConfig modelConfig7GoogleCloudAiplatformV1beta1DeployRequestEndpointConfigendpointConfigr(r)r*r ParseReservationAffinityFlagr BETA_VERSIONr+r,r-)%rEr3r4r accept_eular1r2r0endpoint_display_namehugging_face_access_tokenrireservation_affinityuse_dedicated_endpointenable_fast_tryoutcontainer_image_uricontainer_commandcontainer_argscontainer_env_varscontainer_portscontainer_grpc_portscontainer_predict_routecontainer_health_route$container_deployment_timeout_secondscontainer_shared_memory_size_mbcontainer_startup_probe_exec&container_startup_probe_period_seconds'container_startup_probe_timeout_secondscontainer_health_probe_exec%container_health_probe_period_seconds&container_health_probe_timeout_secondscontainer_speckportstartup_probe_exechealth_probe_execr5r6s% rr-ModelGardenClient.Deploy|s|N .. G G0*2 H  !2 ,  (  ( NN = =03 > (   ( ' NN ; ;$ ; O'  -$ , NN ; ;$ ; O,$   . 4 5 ; ( ),K) & 3 4! 'nnHH6I  NN < <(DF =  # & 2 3  &nnHH5I  NN < <'CE =  "E"" % ..  * *         E " "~~OO"Pn~~OO"Pn KK#<"( L  NN 5%; O ! #'.."f"f>>XXNN( 0!2$)$F$F()*@*@% OY  -#g#NnnEEy H:>2@FG  + + 2 27 ;;c  $ s 'L&#L+=#L0c [nU(aSR[[/5nU(a USUSUS3n[R "UR URRSSUS9SS UUS 9$) a~List publisher models in Model Garden. Args: limit: The maximum number of items to list. None if all available records should be yielded. batch_size: The number of items to list per page. list_hf_models: Whether to only list Hugging Face models. model_filter: The filter on model name to apply on server-side. Returns: The list of publisher models in Model Garden.. z AND z AND (model_user_id=~"(?i).*z.*" OR display_name=~"(?i).*z.*")z publishers/*T)parentlistAllVersionsfilterpublisherModelspageSize)fieldbatch_size_attribute batch_sizelimit) _NATIVE_MODEL_FILTERjoin_HF_WILDCARD_FILTER_VERIFIED_DEPLOYMENT_FILTERr YieldFromListrBr@%AiplatformPublishersModelsListRequest)rErrlist_hf_models model_filter filter_strs rListPublisherModels%ModelGardenClient.ListPublisherModelss&&J<<  ; <j L4\NC##/. 6  # # '' <<!  =  '  r)r@rDrB)FTN)NNNNNNNNNNNNNNNN)NdFN) __name__ __module__ __qualname____firstlineno____doc__rr~rHrTr-r__static_attributes__rrr9r9Rsq;&33A"?C 8Z"!+/&*#'-1.2"&,0-1=]rsp1&' ',3/,.C 73747 *** "4Jqqr