mHSrSSKJr SSKJr SSKJr SSKrSSKrSSKrSSKrSSK J r SSK J r SSK Jr SS K Jr SS KJr SS KJr SS KJr SS KJr SS KJr SSKrSSKrSSKrSSKr\"SSSSS\R8"S5\R8"S5\R8"S5S\R8"S51 5r\"\S1-5r"SS\R>5r "SS\R>5r!"SS\R>5r""SS \R>5r#"S!S"\#5r$"S#S$\#5r%"S%S&\R>5r&"S'S(\RN5r(S)r)\RTS*5r+"S+S,\,5r-"S-S.\,5r."S/S0\,5r/"S1S2\,5r0S4S3jr1g)5a2 Library code to support App Engine Flex runtime builders. The App Engine Flex platform runs a user's application that has been packaged into a docker image. At the lowest level, the user provides us with a source directory complete with Dockerfile, which we build into an image and deploy. To make development easier, Google provides blessed language runtimes that the user can extend in their Dockerfile to get a working base image for their application. To further make development easier, we do not require users to author their own Dockerfiles for "canonical" applications for each of the Silver Languages. In order for this to be possible, preprocessing must be done prior to the Docker build to inspect the user's source code and automatically generate a Dockerfile. Flex runtime builders are a per-runtime pipeline that covers the full journey from source directory to docker image. They are stored as templated .yaml files representing CloudBuild Build messages. These .yaml files contain a series of CloudBuild build steps. Additionally, the runtime root stores a `runtimes.yaml` file which contains a list of runtime names and mappings to the corresponding builder yaml files. Such a builder will look something like this (note that denote values to be filled in by the builder author, and $DOLLAR_SIGNS denote a literal part of the template to be substituted at runtime): steps: - name: 'gcr.io/google_appengine/python-builder:' env: ['GAE_APPLICATION_YAML_PATH=${_GAE_APPLICATION_YAML_PATH}'] - name: 'gcr.io/cloud-builders/docker:' args: ['build', '-t', '$_OUTPUT_IMAGE', '.'] images: ['$_OUTPUT_IMAGE'] To test this out in the context of a real deployment, do something like the following (ls/grep steps just for illustrating where files are): $ ls /tmp/runtime-root runtimes.yaml python-v1.yaml $ cat /tmp/runtime-root/runtimes.yaml schema_version: 1 runtimes: python: target: file: python-v1.yaml $ gcloud config set app/use_runtime_builders true $ gcloud config set app/runtime_builders_root file:///tmp/runtime-root $ cd $MY_APP_DIR $ grep 'runtime' app.yaml runtime: python $ grep 'env' app.yaml env: flex $ gcloud beta app deploy A (possibly) easier way of achieving the same thing if you don't have a runtime_builders_root set up for development yet: $ cd $MY_APP_DIR $ export _OUTPUT_IMAGE=gcr.io/$PROJECT/appengine/placeholder $ gcloud container builds submit \ --config=<(envsubst < /path/to/cloudbuild.yaml) . $ gcloud app deploy --image-url=$_OUTPUT_IMAGE Or (even easier) use a 'custom' runtime: $ cd $MY_APP_DIR $ ls cloudbuild.yaml app.yaml $ rm -f Dockerfile $ grep 'runtime' app.yaml runtime: custom $ gcloud beta app deploy )absolute_import)division)unicode_literalsN)cloudbuild_util)config) storage_api) storage_util) exceptions)log) properties)yaml aspnetcorephpnodejsrubyjavaz(python|python-.+)$z (go|go1\..+)$z^gs://ztest-gaz test-re-[ab]z test-betac\rSrSrSrSrg) FileReadError{z.Error indicating a file read operation failed.N__name__ __module__ __qualname____firstlineno____doc____static_attributes__r2lib/googlecloudsdk/api_lib/app/runtime_builders.pyrr{s6rrc\rSrSrSrSrg) ManifestErrorz9Error indicating a problem parsing or using the manifest.rNrrrrr!r!Arr!c\rSrSrSrSrg)ExperimentsErrorzBError indicating a problem parsing or using the experiment config.rNrrrrr%r%sJrr%c\rSrSrSrSrg)CloudBuildLoadErrorzHError indicating an issue loading the runtime Cloud Build specification.rNrrrrr(r(sPrr(c\rSrSrSrSrg)CloudBuildFileNotFoundz,Error indicating a missing Cloud Build file.rNrrrrr+r+s4rr+c,^\rSrSrSrU4SjrSrU=r$)InvalidRuntimeBuilderURIzGError indicating that the runtime builder URI format wasn't recognized.cJ>[[U] SRU55 g)Nz[{}] is not a valid runtime builder URI. Please set the app/runtime_builders_root property to a URI with either the Google Cloud Storage (`gs://`) or local file (`file://`) protocol.)superr.__init__format)selfuri __class__s rr2!InvalidRuntimeBuilderURI.__init__s# "D2 F3K !rr)rrrrrr2r __classcell__)r6s@rr.r.sO!!rr.c\rSrSrSrSrg)BuilderResolveErrorz9Error indicating that a build file could not be resolved.rNrrrrr:r:r#rr:c:\rSrSrSrSrSrSrSrSr Sr S r S r g ) RuntimeBuilderStrategyz-Enum indicating when to use runtime builders.cXRLa[$XRLa[$[ SR U55e)aReturn the allowlist of runtimes for this strategy. The allowlist is kept as a constant within this module. Returns: list of str, the names of runtimes that are allowed for this strategy. Raises: ValueError: if this strategy is not allowlist-based. z5RuntimeBuilderStrategy {} is not an allowed strategy.) ALLOWLIST_GA_ALLOWLISTED_RUNTIMES_GAALLOWLIST_BETA_ALLOWLISTED_RUNTIMES_BETA ValueErrorr3r4s r _GetAllowlist$RuntimeBuilderStrategy._GetAllowlistsE    %% $$ $ '' ?FFtL NNrcUR5HnURU5(a gM g![a X:Xa gM6f=f)NTF)rJmatchAttributeError)r4runtimeallowlisted_runtimes r _IsAllowed!RuntimeBuilderStrategy._IsAllowedsX#113  $ $W - - . 4   ) *s2AAc(US:Xa(XRURUR4;aU$XRLagXRLdXRLaURU5$XRLag[ SR U55e)akReturns True if runtime should use runtime builders under this strategy. For the most part, this is obvious: the ALWAYS strategy returns True, the ALLOWLIST_${TRACK} strategies return True if the given runtime is in the list of _ALLOWLISTED_RUNTIMES_${TRACK}, and the NEVER strategy returns False. However, in the case of 'custom' runtimes, things get tricky: if the strategy *is not* NEVER, we return True only if there is no `Dockerfile` in the current directory (this method assumes that there is *either* a `Dockerfile` or a `cloudbuild.yaml` file), since one needs to get generated by the Cloud Build. Args: runtime: str, the runtime being built. needs_dockerfile: bool, whether the Dockerfile in the source directory is absent. Returns: bool, whether to use the runtime builders. Raises: ValueError: if an unrecognized runtime_builder_strategy is given customTFz&Invalid runtime builder strategy [{}].)ALWAYSrFrDrQNEVERrHr3)r4rOneeds_dockerfiles rShouldUseRuntimeBuilders/RuntimeBuilderStrategy.ShouldUseRuntimeBuilderss6(t (,(;(;(,(9(9(; ; {{  $$ $0A0A(A __W %%    ?FFtL MMrrN) rrrrrrVrFrDrUrJrQrXrrrrr=r=s*5 %., &N$'Nrr=clSRUVs/sHoRS5PM sn5$s snf)z:Join parts of a gs:// Cloud Storage or local file:// path./)joinstrip)argsargs r_Joinr`s+ T2Tc99S>T2 332s1c## URS5(aU[R"[RR R RU55nUv SSS5 gURS5(ag[R"5n[RRU5n[R"URU55nUv SSS5 g[U5e!,(df  g=f!,(df  g=f![RR RR [RR RR"[$R&4a9n[(R*"SSS9 [-[R."U55eSnAff=f7f)aRead a file/object (local file:// or gs:// Cloud Storage path). >>> with _Read('gs://builder/object.txt') as f: ... assert f.read() == 'foo' >>> with _Read('file:///path/to/object.txt') as f: ... assert f.read() == 'bar' Args: uri: str, the path to the file/object to read. Must begin with 'file://' or 'gs://' Yields: a file-like context manager. Raises: FileReadError: If opening or reading the file failed. InvalidRuntimeBuilderPath: If the path is invalid (doesn't begin with an appropriate prefix). zfile://Ngs://Texc_info) startswith contextlibclosingsixmovesurllibrequesturlopenr StorageClientr ObjectReferenceFromUrl ReadObjectr.error HTTPErrorURLErrorcalliope_exceptionsBadFileExceptionr debugr text_type)r5reqstorage_clientobject_fes r_Readr~s=** ~~i    cii..66>>sC D  E D  "002n,,44S9g   n77@ AQ B A %S )) E D B A ))   * *CII,<,<,B,B,K,K  . . 0*IIb4  a( ))*sGAD!C7&D.G/A/DD#D+G, D7 DDGD DDGDA/G4F<<GGc:\rSrSrSrS SjrSrSrSrSr S r g) BuilderReferenceiz6A reference to a specific cloudbuild.yaml file to use.Nc(XlX lX0lg)aQConstructs a BuilderReference. Args: runtime: str, The runtime this builder corresponds to. build_file_uri: str, The full URI of the build configuration or None if this runtime existed but no longer can be built (deprecated). deprecation_message: str, A message to print when using this builder or None if not deprecated. NrObuild_file_urideprecation_message)r4rOrrs rr2BuilderReference.__init__sL(2rc<UR(d"[SRURS95e[R "5n[ UR5n[R"X2US9nSSS5 WRcUR5Ul URRRURl URHnSnSnURHHnUR!S5n ["R$"S['U 5-5 SU ;aS nS U ;dMFS nMJ U(dURR)S 5 U(aMS U;dMURR)S 5 M U$!,(df  GN=f)aLoads the Cloud Build configuration file for this builder reference. Args: params: dict, a dictionary of values to be substituted in to the Cloud Build configuration template corresponding to this runtime version. Returns: Build message, the parsed and parameterized Cloud Build configuration file. Raises: CloudBuildLoadError: If the Cloud Build configuration file is unknown. FileReadError: If reading the configuration file fails. InvalidRuntimeBuilderPath: If the path of the configuration file is invalid. z:There is no build file associated with runtime [{runtime}]rO)messagesparamsNF=zEnv var in build step: GAE_APPLICATION_YAML_PATHTGOOGLE_RUNTIME_VERSIONz7GAE_APPLICATION_YAML_PATH=${_GAE_APPLICATION_YAML_PATH}_GOOGLE_RUNTIME_VERSIONz1GOOGLE_RUNTIME_VERSION=${_GOOGLE_RUNTIME_VERSION})rr(r3rOrGetMessagesModuler~cloudbuild_configLoadCloudbuildConfigFromStreamoptions BuildOptions!SubstitutionOptionValueValuesEnum ALLOW_LOOSEsubstitutionOptionstepsenvsplitr rwstrappend) r4rrdatabuildstep has_yaml_pathhas_runtime_versionrpartss rLoadCloudBuildBuilderReference.LoadCloudBuild(sO$     F 6$,,6 ' ))002H t"" #t>> &2e $ }}++-em 77CC MM$ m!# # +c%j89 &% /- #u , $    E G %>&%H KL L- $ #s F  FchUR(a![R"UR5 gg)zFWarns that this runtime is deprecated (if it has been marked as such).N)rr warningrIs rWarnIfDeprecated!BuilderReference.WarnIfDeprecatedWs"  kk$**+ rcURUR:H=(a9 URUR:H=(a URUR:H$Nrr4others r__eq__BuilderReference.__eq__\sL LLEMM ) B   5#7#7 7 B  $ $(A(A ACrc.URU5(+$r)rrs r__ne__BuilderReference.__ne__as{{5! !!r)rrrOr) rrrrrr2rrrrrrrrrrs > 3-^, C "rrc>\rSrSrSrSr\S5rSrSr Sr Sr g ) ManifestieaLoads and parses a runtimes.yaml manifest. To resolve a builder configuration file to use, a given runtime name is looked up in this manifest. For each runtime, it either points to a configuration file directly, or to another runtime. If it points to a runtime, resolution continues until a configuration file is reached. The following is the proto-ish spec for the yaml schema of the mainfest: # Used to determine if this client can parse this manifest. If the number is # less than or equal to the version this client knows about, it is compatible. int schema_version; # Required # The registry of all the runtimes that this manifest defines. The key of the # map is the runtime name that appears in app.yaml. runtimes { # Determines which builder this runtime points to. Target target { oneof { # A path relative to the manifest's location of the builder spec to use. string file; # Another runtime registered in this file that should be resolved and # used for this runtime. string runtime; } } # Specifies deprecation information about this runtime. Deprecation deprecation { # A message to be displayed to the user on use of this runtime. string message; } } r?c[R"SU5 [U5n[R"X!S9nSSS5 U"UW5$!,(df  N=f)zLoads a manifest from a gs:// or file:// path. Args: uri: str, A gs:// or file:// URI Returns: Manifest, the loaded manifest. z#Loading runtimes manifest from [%s] file_hintN)r rwr~r load)clsr5r|rs r LoadFromURIManifest.LoadFromURIsCII3S9 sq YYq (d  sD> s A  AcXlX lURRSS5nUc[SR U55eU[ R :a'[SR [ R US95eg)0Use LoadFromFile, not this constructor directly.schema_versionNz+Unable to parse the runtimes manifest: [{}]zUnable to parse the runtimes manifest. Your client supports schema version [{supported}] but requires [{required}]. Please update your SDK to a later version. supportedrequired)_uri_datagetr!r3rSCHEMA_VERSIONr4r5rrequired_versions rr2Manifest.__init__sIJzz~~&6=  7 > >s C EE(111  $$*FX5L5L4D%+%F GG2rch[URRS05R55$)zSGet all registered runtimes in the manifest. Returns: [str], The runtime names. runtimes)listrrkeysrIs rRuntimesManifest.Runtimess'  z2.335 66rc*URRS05nUnU1nURUS5nU(d"[R"SX0R5 gURS05RSS5nU(aI[R"SX65 Xd;a[ SR US95eURU5 UnMURS 05RS S5nURS05RS S5nU(aV[[RRUR5U5n [R"S X95 [X9U5$[R"S U5 [USU5$)aGets the associated reference for the given runtime. Args: runtime: str, The name of the runtime. Returns: BuilderReference, The reference pointed to by the manifest, or None if the runtime is not registered. Raises: ManifestError: if a problem occurred parsing the manifest. rNz'Runtime [%s] not found in manifest [%s]targetrOz!Runtime [%s] is an alias for [%s]zSA circular dependency was found while resolving the builder for runtime [{runtime}]r deprecationmessagefilez1Resolved runtime [%s] as build configuration [%s]z0Resolved runtime [%s] has no build configuration) rrr rwrr!r3addr`ospathdirnamer) r4rOrcurrent_runtimeseen runtime_def new_runtimedeprecation_msg build_filefull_build_uris rGetBuilderReferenceManifest.GetBuilderReferencesazz~~j"-HO  D LL$7k  ;!99 .OOHb155iFk  5! 0  $$*F7F$;= = %# r:>>y$Oo??8R044VTBj rwwtyy9:F E! 3 _> >  iiB! ot_ EErrrN) rrrrrr classmethodrr2rrrrrrrres2%L.   G74FrrcJ\rSrSrSrSrSrSr\S5r Sr Sr S S jr S r g ) ExperimentsiakRuntime experiment configs as read from a gs:// or a file:// source. The experiment config file follows the following protoish schema: # Used to determine if this client can parse this manifest. If the number is # less than or equal to the version this client knows about, it is compatible. int schema_version; # Required # Map of experiments and their rollout percentage. # The key is the name of the experiment, the value is an integer between 0 # and 100 representing the rollout percentage # In case no experiments are defined, an empty 'experiments:' section needs to # be present. experiments r?zexperiments.yamltrigger_build_server_sidec[XR5n[R"SU5 [ U5n[ R "X2S9nSSS5 U"UW5$!,(df  N=f![an[SRX%55eSnAf[ Ran[SRX%55eSnAff=f)zLoads a runtime experiment config from a gs:// or file:// path. Args: dir_uri: str, A gs:// or file:// URI pointing to a folder that contains the file called Experiments.CONFIG_FILE Returns: Experiments, the loaded runtime experiments config. z,Loading runtimes experiment config from [%s]rNz>Unable to read the runtimes experiment config: [{}], error: {}) r` CONFIG_FILEr rwr~r rrr%r3YAMLParseError)rdir_urir5r|rr}s rrExperiments.LoadFromURIs  )CII      J 6#> s: A0AA0 A-)A00 C :BC +CC cXlX lURRSS5nUc[SR U55eU[ R :a'[SR [R US95eg)rrNzXUnable to parse the runtimes experiment config due to missing schema_version field: [{}]zUnable to parse the runtimes experiments config. Your client supports schema version [{supported}] but requires [{required}]. Please update your SDK to a newer version.r)rrrr%r3rrrrs rr2Experiments.__init__sIJzz~~&6=  ''-vc{ 44+444  77=v //:J8>8L MM5rc8URRS5$)zqGet all experiments and their rollout percentage. Returns: dict[str,int] Experiments and their rollout state. experiments)rrrIs rrExperiments.Experiments)s ::>>- ((rcdURRS5U$![a Us$f=f)zGet the rollout percentage of an experiment or return 'default'. Args: experiment: the name of the experiment default: the value to return if the experiment was not found Returns: int the percent of the experiment r)rrKeyError)r4 experimentdefaults rGetExperimentPercentWithDefault+Experiments.GetExperimentPercentWithDefault1s3 ZZ^^M *: 66  ns  //rN)r)rrrrrrrTRIGGER_BUILD_SERVER_SIDErrr2rrrrrrrrs;."+92M ) rrcT\rSrSrSrSrSrSrSSjrSr Sr S r S r S r S rS rg)ResolveriAzResolves the location of a builder configuration for a runtime. There are several possible locations that builder configuration can be found for a given runtime, and they are checked in order. Check GetBuilderReference for the locations checked. z runtimes.yamlzruntimes_buildpacks.yamlzcloudbuild.yamlc`Xl[RRU5UlX0l[ RRRRSS9Ul X@l [R"SUR5 [R"SUR5 g)aInstantiates a resolver. Args: runtime: str, The name of the runtime to be resolved. source_dir: str, The local path of the source code being deployed. legacy_runtime_version: str, The value from runtime_config.runtime_version in app.yaml. This is only used in legacy mode. use_flex_with_buildpacks: bool, if true, use the build-image and run-image built through buildpacks. Returns: Resolver, The instantiated resolver. T)rz#Using use_flex_with_buildpacks [%s]zUsing runtime builder root [%s]N)rOrrabspath source_dirlegacy_runtime_versionr VALUESappruntime_builders_rootGetbuild_file_rootuse_flex_with_buildpacksr rw)r4rOrrrs rr2Resolver.__init__OsLggooj1DO"8%,,00FFJJKD$<!II3++-II/1E1EFrcUR5=(d> UR5=(d' UR5=(d UR5nU(d"[ SR UR S95eU$)zResolve the builder reference. Returns: BuilderReference, The reference to the builder configuration. Raises: BuilderResolveError: if this fails to resolve a builder. z4Unable to resolve a builder for runtime: [{runtime}]r)_GetReferenceCustom_GetReferencePinned_GetReferenceFromManifest_GetReferenceFromLegacyr:r3rO)r4 builder_defs rrResolver.GetBuilderReferencehsw   " '   " ' &&( ' $$&    @ 6$,,6 ' )) rc "URS:Xa[R"S[R5 [ UR[ SURRSS5RS5-[R55$g)aTries to resolve the reference for runtime: custom. If the user has an app.yaml with runtime: custom we will look in the root of their source directory for a custom build pipeline named cloudbuild.yaml. This should only be called if there is *not* a Dockerfile in the source root since that means they just want to build and deploy that Docker image. Returns: BuilderReference or None rTz5Using local cloud build file [%s] for custom runtime.zfile:///\r[N) rOr rwrCLOUDBUILD_FILErr`rreplacer]rIs rrResolver._GetReferenceCustom~su ||x iiG((*  ,,  T__44T3?EEcJJ(( *++ rcURRS5(aA[R"SUR5 [ URUR5$g)aNTries to resolve the reference for when a runtime is pinned. Usually a runtime is looked up in the manifest and resolved to a configuration file. The user does have the option of 'pinning' their build to a specific configuration by specifying the absolute path to a builder in the runtime field. Returns: BuilderReference or None rbz#Using pinned cloud build file [%s].N)rOrfr rwrrIs rrResolver._GetReferencePinnedsD ||w'' ii5t||D dllDLL 99 rcjUR(a[RO[Rn[ UR U5n[ R"SU5 [RU5nURUR5$![a [ R"SSS9 gf=f)a Tries to resolve the reference by looking up the runtime in the manifest. Calculate the location of the manifest based on the builder root and load that data. Then try to resolve a reference based on the contents of the manifest. Returns: BuilderReference or None zUsing manifest_uri [%s]rcTrdN) rrBUILDPACKS_MANIFEST_NAME MANIFEST_NAMEr`rr rwrrrrOr)r4manifest_file_name manifest_urimanifests rr"Resolver._GetReferenceFromManifests  ( ( ))  # # --/ABLII'6%%l3h  ) )$,, 77  iiT" s /BB21B2c:UR(aURUR5$[R"SUR5 URS-n[ UR U5n[U5nUR5R5R5nSSS5 [R"SWUR5 URU5$!,(df  NA=f![a [R"SSS9 gf=f)aFTries to resolve the reference by the legacy resolution process. TODO(b/37542861): This can be removed after all runtimes have been migrated to publish their builders in the manifest instead of .version files. If the runtime is not found in the manifest, use legacy resolution. If the app.yaml contains a runtime_config.runtime_version, this loads the file from '-.yaml' in the runtime builders root. Otherwise, it checks '.version' to get the default version, and loads the configuration for that version. Returns: BuilderReference or None z0Fetching version for runtime [%s] in legacy modez.versionNrcTrdz2Using version [%s] for runtime [%s] in legacy mode) r"_GetReferenceFromLegacyWithVersionr rwrOr`rr~readdecoder]r)r4version_file_nameversion_file_urir|versions rr Resolver._GetReferenceFromLegacys  ""  4 4  % %''II@$,,O z1T113DE ! "a&&(//#))+ #IIBt||%  2 27 ;; # "  iiT" s04 C8?-C',C8' C51C85C88DDcSRURU/5S-n[URU5n[R "SU5 [ URU5$)zGets the name of configuration file to use for legacy mode. Args: version: str, The pinned version of the configuration file. Returns: BuilderReference -z.yamlz7Calculated builder definition using legacy version [%s])r\rOr`rr rwr)r4r file_namefile_uris rr+Resolver._GetReferenceFromLegacyWithVersionsU$,,01G;IT))95HIIG DLL( 33r)rrrOrrNF)rrrrrrrr r2rrrrrrrrrrrrAsC"-7%/).G2,* 4">00.%#&&'8$?+/ gooz.0(  % % ''rr#)2r __future__rrrrgenumrre!googlecloudsdk.api_lib.cloudbuildrrrgooglecloudsdk.api_lib.storagerr googlecloudsdk.callioper rugooglecloudsdk.corer r r risix.moves.urllib.errorsix.moves.urllib.parsesix.moves.urllib.request frozensetcompilerErGErrorrr!r%r(r+r.r:Enumr=r`contextmanagerr~objectrrrrr+rrrr<sGR'' =I67E*#*$ %5(FFZZ&'ZZ !ZZ >* ,- 'M 7J$$7BJ$$BKz''KQ***Q505!2!B***BJNTYYJNZ4  "*"*JK"vK"\CFvCFLS&Slm4vm4`(r