SrSSKJr SSKJr SSKJr SSKrSSKrSSKJr SSKJ r SSK J r SS K J r SSKrSS KJr \R$R&r"S S \5r"S S\ R.5rSSjrSrSSjrg)zCommon ML file upload logic.)absolute_import)division)unicode_literalsN) storage_api) storage_util) exceptions)files)zipc\rSrSrSrSrg)MissingStagingBucketException$zIndicates that a staging bucket was not provided with a local path. It doesn't inherit from core.exceptions.Error because it should be caught and re-raised at the call site with an actionable message. N__name__ __module__ __qualname____firstlineno____doc____static_attributes__r3lib/googlecloudsdk/command_lib/ml_engine/uploads.pyr r $srr c\rSrSrSrSrg)BadDirectoryError,z9Indicates that a provided directory for upload was empty.rNrrrrrr,sArrc[R"[RS9nUHupEUR U5 M Ub!SR X#R 5/5nOUR 5n[R"5n/nUH}upH[RRUSR X(/55n URXI5n URSR SU RU R/55 M U$)aUploads files at the local path to a specifically prefixed location. The prefix is 'cloudmldist/'. Args: upload_pairs: [(str, str)]. Pairs of absolute paths to local files to upload and corresponding path in Cloud Storage (that goes after the prefix). For example, ('/path/foo', 'bar') will upload '/path/foo' to '/bar' in Cloud Storage. bucket_ref: storage_util.BucketReference. Files will be uploaded to this bucket. gs_prefix: str. Prefix to the GCS Path where files will be uploaded. Returns: [str]. A list of fully qualified gcs paths for the uploaded files, in the same order they were provided. ) algorithm/zgs:/) file_utilsChecksumhashlibsha256AddFileContentsjoin HexDigestr StorageClientrObjectReference FromBucketRef CopyFileToGCSappendbucketname) upload_pairs bucket_ref gs_prefixchecksum local_path_storage_clientdests uploaded_pathobj_refobjs r UploadFilesr70s$ 7>> :(#mj Z($)%7%7%9:;I""$I,,.. %#/j**88CHHi78:G  & &z ;C LL63::sxx89: $0 ,rc /n[R"[R"U55HZup#nUHNn[RR X%5nUR [RRX`S95 MP M\ U$)aQReturn all the descendents of root, relative to its path. For instance, given the following directory structure /path/to/root/a /path/to/root/a/b /path/to/root/c This function would return `['a', 'a/b', 'c']`. Args: root: str, the path to list descendents of. Returns: list of str, the paths in the given directory. )start)oswalksix text_typepathr#r)relpath)rootpathsdirpathr1 filenamesfilenameabs_paths r_GetFilesRelativerFUsh" %!wws}}T':;g)g0h ll277??8?89 < ,rc0URS5(aU$Uc [S5e[RR U5(d[ SR U55e[U5nUVs/sHoDR[S5PM nnUVs/sHn[RX/5PM nn[[[Xe55UUS9nU(d[ SR U55eUSS[US5*$s snfs snf) a"Uploads path to Cloud Storage if it isn't already there. Translates local file system paths to Cloud Storage-style paths (i.e. using the Unix path separator '/'). Args: path: str, the path to the directory. Can be a Cloud Storage ("gs://") path or a local filesystem path (no protocol). staging_bucket: storage_util.BucketReference or None. If the path is local, the bucket to which it should be uploaded. gs_prefix: str, prefix for the directory within the staging bucket. Returns: str, a Cloud Storage path where the directory has been uploaded (possibly prior to the execution of this function). Raises: MissingStagingBucketException: if `path` is a local path, but staging_bucket isn't found. BadDirectoryError: if the given directory couldn't be found or is empty. zgs://NzKThe path provided was local, but no staging bucket for upload was provided.z[{}] is not a valid directory.r)r.zYCannot upload contents of directory [{}] to Google Cloud Storage; directory has no files.r) startswithr r:r>isdirrformatrFreplace _PATH_SEPr#r7listr len)r>staging_bucketr.r fr3 full_filesuploaded_pathss rUploadDirectoryIfNecessaryrSns, __W K '   t   <CCDI JJ D !%/4 4e99Y $e% 43895a y)5*9tC $:;-)24.   ""(&, 00  ?Sq]N ++ 59s 4D D)N)NN)r __future__rrrr r:googlecloudsdk.api_lib.storagerrgooglecloudsdk.corergooglecloudsdk.core.utilr rr< six.movesr r>seprL Exceptionr Errorrr7rFrSrrrr\sj#&' 67*8  GGKK IB ((B"J24,r