-SrSSKJr SSKJr SSKJr SSKrSSKrSSKrSSKJ r SSK J r SSK Jr SS K Jr SS KJr SS KJr SSKJ r SS KJr SS KJr SSKJr SSKJr SSKJr SSKJr SSKJr Sr \RB"S5r""SS\RF5r$"SS\RF5r%Sr&Sr'Sr(Sr)"SS\*5r+Sr,S"S jr-S!r.g)#z/Utility methods used by the deploy_app command.)absolute_import)division)unicode_literalsN) exceptions) metric_names) storage_api) storage_util) context_util)storage_parallel)log)metrics) propertiesencoding)files)times)mapc(^\rSrSrU4SjrSrU=r$)LargeFileError1cH>[[U] SRXUS95 g)NzCannot upload file [{path}], which has size [{size}] (greater than maximum allowed size of [{max_size}]). Please delete the file or add to the skip_files entry in your application .yaml file and try again.)pathsizemax_size)superr__init__format)selfrrr __class__s 9lib/googlecloudsdk/api_lib/app/deploy_app_command_util.pyrLargeFileError.__init__3s( .$( &d&A D__name__ __module__ __qualname____firstlineno__r__static_attributes__ __classcell__r!s@r"rr1sDDr$rc(^\rSrSrU4SjrSrU=r$) MultiError;c>[U5S:aSRU5nOSRU5nSR[[U55n[ [ RU]#X4-5 X l g)NrzMultiple errors occurred {0} zAn error occurred {0} z ) lenrjoinrstrrcore_exceptionsErrorrerrors)r operation_descriptionr7msg errors_stringr!s r"rMultiError.__init__=sb 6{Q , 3 34I Jc % , ,-B CcKKC 01M /  /0CDKr$)r7r&r-s@r"r/r/;s r$r/c0nSRUR5nUHqn[RR X5n[ R RU[RS9nSR XX/5n U US.U[U5'Ms [R"USUS9n U Hn [RRU 5nXd;a[R"S5 M?[ R RU [RS9nSR XX/5n U US.U[U5'M U$)aBuilds a deployment manifest for use with the App Engine Admin API. Args: upload_dir: str, path to the service's upload directory source_files: [str], relative paths to upload. bucket_ref: The reference to the bucket files will be placed in. tmp_dir: A temp directory for storing generated files (currently just source context files). Returns: A deployment manifest (dict) for use with the Admin API. z"https://storage.googleapis.com/{0}) algorithm/) sourceUrlsha1SumN) source_dirz7Source context already exists. Using the existing file.)rbucketosrr3 file_utilsChecksumHashSingleFilehashlibsha1_FormatForManifestr CreateContextFilesbasenamer debug) upload_dir source_files bucket_reftmp_dirmanifest bucket_urlrel_path full_path sha1_hash manifest_path context_files context_files r"_BuildDeploymentManifestrYGs-(3:::;L;LM*h Z2I##229=D\\3KIHHj45M".H  )* 11 t ,-#lww -H iiIJ%%44\?F||5Mihh 67m$0h!(+,$ /r$cURRRURR UR S9URR SS9S9nUR(dgURRnUVs/sHioDRRcMURRS:dM7URRS:XdMSURRPMk nnU(a[R "[#U55$S$![Ra gf=fs snf)aNGet the TTL of objects in days as specified by the lifecycle policy. Only "delete by age" policies are accounted for. Args: storage_client: storage_api.StorageClient, API client wrapper. bucket_ref: The GCS bucket reference. Returns: datetime.timedelta, TTL of objects in days, or None if no deletion policy on the bucket. )rB lifecycle)fields)request global_paramsNrDelete)clientbucketsGetmessagesStorageBucketsGetRequestrBStandardQueryParametersapitools_exceptionsHttpForbiddenErrorr[rule conditionageactiontypedatetime timedeltamin)storage_clientrOrBrulesrhagess r"_GetLifecycleDeletePolicyrsvs$  " " * * . .''@@$$A&$--EEF /!F         %%* %*Tnn.@.@ nnA"&++"2"2h">dnnU +/  CI &8D8  / /   s*AD$ D>(D>D> D>$D;:D;cUcg[R"[R5nU[- nX!R- U:*$)aDetermines whether a GCS object is close to end-of-life. In order to reduce false negative rate (objects that are close to deletion but aren't marked as such) the returned filter is forward-adjusted with _TTL_MARGIN. Args: ttl: datetime.timedelta, TTL of objects, or None if no TTL. obj: storage object to check. Returns: True if the ojbect is safe or False if it is approaching end of life. T)rNowUTC _TTL_MARGIN timeCreated)ttlobjnowdeltas r" _IsTTLSafer}s: [  %))# %  E ))r$c "^0n[R"5n[Xb5m[U4SjUR U555nSupUGH<n [ R RX5n [ R R[R"U SS95(d[ R RX:5n [ R R[R"U SS95n U(aX:a [XU5eX Sn X- n X;a([R"SRU S95 X- nOXU 'U (dGM [R "SR[#S U-U - S 5S 95 GM? U$) aBuilds a map of files to upload, indexed by their hash. This skips already-uploaded files. Args: manifest: A dict containing the deployment manifest for a single service. source_dir: The relative source directory of the service. bucket_ref: The GCS bucket reference to upload files into. tmp_dir: The path to a temporary directory where generated files may be stored. If a file in the manifest is not found in the source directory, it will be retrieved from this directory instead. max_file_size: int, File size limit per individual file or None if no limit. Raises: LargeFileError: if one of the files to upload exceeds the maximum App Engine file size. Returns: A dict mapping hashes to file paths that should be uploaded. c3b># UH$n[TU5(dMURv M& g7fN)r}name).0orys r" &_BuildFileUploadMap..s(.'L!$S!,qvv'Ls//)rrzutf-8rr@zSkipping upload of [{f}])fz)Incremental upload skipped {pct}% of datagY@)pct)r StorageClientrsset ListBucketrCrr3existsrEncodegetsizerr rLrinforound)rQrArOrP max_file_sizefiles_to_uploadrpexisting_items skipped_size total_sizerSrTrrUrys @r"_BuildFileUploadMaprsT,/,,..!.=#.~'@'@'L...!,h Z2I 77>>(//)gF G G'',,w1i 77??8??9wG HD- 9M ::"9-IJ" ii*11H1=>l#,i z hh:AAEL(:5q9B;<+. r$c\rSrSrSrSrg)FileUploadTaskc(XlX lX0lgr)rUrrR)r rUrrRs r"rFileUploadTask.__init__sNI Or$)rRrrUN)r'r(r)r*rr+r%r$r"rrs!r$rc[RRRR 5=(d [ R n/n[UR55HLupE[RRUU5n[ R"XV5nURU5 MN [ R"X2SS9 g)a7Uploads files to App Engine Cloud Storage bucket using threads. Args: files_to_upload: dict {str: str}, map of checksum to local path bucket_ref: storage_api.BucketReference, the reference to the bucket files will be placed in. Raises: MultiError: if one or more errors occurred during file upload. T) num_threadsshow_progress_barN)rVALUESappnum_file_upload_threadsGetIntr DEFAULT_NUM_THREADSsorteditemsr ObjectReference FromBucketRefrappend UploadFiles)rrOrtasksrUr dest_obj_reftasks r"_UploadFilesThreadsrs""&&>>EEG6!55 %  5 5 78oi//==j>GIL  * *4 >D LL 9 u157r$c[R"[R5 [R "5n[ XX$5n[XPX$U5n[Xb5 SSS5 [RRS5 [R"SRW55 [R"[R5 U$!,(df  Nx=f)aZCopies application files to the Google Cloud Storage code bucket. Use the Cloud Storage API using threads. Consider the following original structure: app/ main.py tools/ foo.py Assume main.py has SHA1 hash 123 and foo.py has SHA1 hash 456. The resultant GCS bucket will look like this: gs://$BUCKET/ 123 456 The resulting App Engine API manifest will be: { "app/main.py": { "sourceUrl": "https://storage.googleapis.com/staging-bucket/123", "sha1Sum": "123" }, "app/tools/foo.py": { "sourceUrl": "https://storage.googleapis.com/staging-bucket/456", "sha1Sum": "456" } } A 'list' call of the bucket is made at the start, and files that hash to values already present in the bucket will not be uploaded again. Args: upload_dir: str, path to the service's upload directory source_files: [str], relative paths to upload. bucket_ref: The reference to the bucket files will be placed in. max_file_size: int, File size limit per individual file or None if no limit. Returns: A dictionary representing the manifest. NzFile upload done.zManifest: [{0}])r CustomTimedEventrCOPY_APP_FILES_STARTrDTemporaryDirectoryrYrrr statusPrintrrCOPY_APP_FILES)rMrNrOrrPrQrs r"CopyFilesToCodeBucketrsT <<<=$$&''*7H)j=BO4 ' **&'((  # #H -. <667 /'&s %C Ccf[RRS:XaURSS5$U$)zHReformat a filename for the deployment manifest if it is Windows format.\r>)rCrsepreplace)filenames r"rIrI:s*WW[[D   D# && /r$r)/__doc__ __future__rrrrmrGrCapitools.base.pyrrfgooglecloudsdk.api_lib.apprgooglecloudsdk.api_lib.storagerr googlecloudsdk.appengine.toolsr "googlecloudsdk.command_lib.storager googlecloudsdk.corer5r r rgooglecloudsdk.core.utilrrrDr six.movesr_DEFAULT_NUM_THREADSrnrwr6rr/rYrsr}robjectrrrrIr%r$r"rs 6&' >3677?=#'*-8*  # D_**D && ,^9>**3l!V!74596rr$