o l@s:dZddlmZddlmZddlmZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl mZddl mZddlmZdd lmZdd lmZdd lmZdd lmZddlZGd ddejZGdddeZGdddeZGdddeZddZGdddeZ GdddeZ!ddZ"dS)zManages the state of what is installed in the cloud SDK. This tracks the installed modules along with the files they created. It also provides functionality like extracting tar files into the installation and tracking when we check for updates. )absolute_import)division)unicode_literalsN)config) exceptions) console_io) installers) snapshots)encoding)filesc@seZdZdZdS)Errorz*Base exception for the local_state module.N)__name__ __module__ __qualname____doc__rrD/tmp/google-cloud-sdk/lib/googlecloudsdk/core/updater/local_state.pyr /sr c eZdZdZfddZZS)InvalidSDKRootErrorzGError for when the root of the Cloud SDK is invalid or cannot be found.ctt|ddS)NzThe components management action could not be performed because the installation root of the Cloud SDK could not be located. If you previously used the Cloud SDK installer, you could re-install the SDK and retry again.)superr__init__self __class__rrr7 zInvalidSDKRootError.__init__r rrrr __classcell__rrrrr4rcr)InvalidDownloadErrorz9Exception for when the SDK that was download was invalid.cr)Nz#The Cloud SDK download was invalid.)rr rrrrrrBrzInvalidDownloadError.__init__rrrrrr ?rr cr)PermissionsErrorzCError for when a file operation cannot complete due to permissions.cstt|dj||ddS)zInitialize a PermissionsError. Args: message: str, The message from the underlying error. path: str, The absolute path to a file or directory that needs to be operated on, but can't because of insufficient permissions. zh{message}: [{path}] Ensure you have the permissions to access the file and that the file is not in use.messagepathN)rr!rformat)rr#r$rrrrJs zPermissionsError.__init__rrrrrr!Grr!csfdd}|S)a7Use this decorator for functions that deal with files. If an exception indicating file permissions is raised, this decorator will raise a PermissionsError instead, so that the caller only has to watch for one type of exception. Args: func: The function to decorate. Returns: A decorator. c sz|i|WStjy5}z |jdd}|ddr0tt|dtj |ddd}~wt t fy^}z|j t j krYttt|jttj |jdd}~ww)Nrz [Errno 13]r")shutilr args startswithrreraiser!osr$abspathOSErrorIOErrorerrnoEACCESr Decodestrerrorfilename)r(kwargsefuncrr_TryFuncfs.  z)_RaisesPermissionsError.._TryFuncr)r7r8rr6r_RaisesPermissionsErrorXs r9c@sJeZdZdZejjZdZdZ dZ dZ dZ e ddZd d Ze d d Zed dZeddZeddZddZeddZeddZd:ddZed:ddZed:ddZed d!Zd"d#Zd$d%Zed:d&d'Zed:d(d)Z ed:d*d+Z!e  ,d;d-d.Z"e d:d/d0Z#ed:d1d2Z$ed3d4Z%d5d6Z&dInstallationManifest_state_directory VersionStringloggingdebugsysexc_info) component_idstaterrrVersionForInstalledComponentsz.InstallationState.VersionForInstalledComponentcCstj|s td|t||_tj|jt j |_ tj|j t j |_ tj|j t j|_tj|jt j|_t|j|j |_dS)zInitializes the installation state for the given sdk install. Args: sdk_root: str, The file path of the root of the SDK installation. Raises: ValueError: If the given SDK root does not exist. z.The given Cloud SDK root does not exist: [{0}]N)r+r$isdir ValueErrorr%r r1_InstallationState__sdk_rootjoinr:STATE_DIR_NAMErCBACKUP_DIR_NAMEr@TRASH_DIR_NAME#_InstallationState__trash_directorynormpathSTAGING_ROOT_SUFFIX$_InstallationState__sdk_staging_rootrComponentInstaller_component_installer)rr<rrrrs(       zInstallationState.__init__cCs"tj|jst|jdSdS)z1Creates the state directory if it does not exist.N)r+r$rLrC file_utilsMakeDirrrrr_CreateStateDirsz!InstallationState._CreateStateDircCs|jS)zvGets the root of the SDK that this state corresponds to. Returns: str, the path to the root directory. )rNrrrrr<szInstallationState.sdk_rootcs6tjjs gStj}fdd|D}|S)zReturns the files in the state directory that have the given suffix. Args: suffix: str, The file suffix to match on. Returns: list of str, The file names that match. cs2g|]}tjtjj|r|r|qSr)r+r$isfilerOrCendswith).0frsuffixrr s  z5InstallationState._FilesForSuffix..)r+r$rLrClistdir)rrar matchingrr`r_FilesForSuffixs   z!InstallationState._FilesForSuffixcCsB|tj}i}|D]}|dttj }t|j|||<q |S)zGets all the components that are currently installed. Returns: A dictionary of component id string to InstallationManifest. N)rer:COMPONENT_SNAPSHOT_FILE_SUFFIXlenrBrC)rsnapshot_files manifestsr_rIrrrInstalledComponentss z%InstallationState.InstalledComponentscCs tj|S)zFGenerates a ComponentSnapshot from the currently installed components.)r ComponentSnapshotFromInstallStaterrrrSnapshot s zInstallationState.SnapshotNcCs|j||dS)azGenerates a ComponentSnapshotDiff from current state and the given state. Args: latest_snapshot: snapshots.ComponentSnapshot, The current state of the world to diff against. platform_filter: platforms.Platform, A platform that components must match in order to be considered for any operations. Returns: A ComponentSnapshotDiff. )platform_filter)rm CreateDiff)rlatest_snapshotrnrrrDiffCurrentStates z"InstallationState.DiffCurrentStatec Cs|tO}tj|d}tj|d}t|ddg\}}tj |||dd}tj |||dt |} t | dkr@t tj|| d} t| |jWdn1sZwYt|j} | || | S) aCreates a new staging area from a fresh download of the Cloud SDK. Args: url: str, The url to download the new SDK from. progress_callback: f(float), A function to call with the fraction of completeness. Returns: An InstallationState object for the new install. Raises: installers.URLFetchError: If the new SDK could not be downloaded. InvalidDownloadError: If the new SDK was malformed. z .downloadz.extractzcomponents.reinstallprogress_callback command_pathrtrN) _ClearStagingrYTemporaryDirectoryr+r$rOrSplitProgressBarr DownloadTar ExtractTarrcrgr MoveDirrVr:r[CopyMachinePropertiesTo) rurlrtt download_dir extract_dirdownload_callbackextract_callbackdownloaded_tarr r< staging_sdkrrrCreateStagingFromDownloads.     z+InstallationState.CreateStagingFromDownloadcCsh|||||t|j|j|r"|dt|j|j|r2|ddSdS)aReplaces this installation with the given other installation. This moves the current installation to the backup directory of the other installation. Then, it moves the entire second installation to replace this one on the file system. The result is that the other installation completely replaces the current one, but the current one is snapshotted and stored as a backup under the new one (and can be restored later). Args: other_install_state: InstallationState, The other state with which to replace this one. progress_callback: f(float), A function to call with the fraction of completeness. g?g?N)r[ ClearBackup ClearTrashrYr|rNr@)rother_install_statertrrr ReplaceWithEs zInstallationState.ReplaceWithcCsb|sdS|t|j|jt|j}||t|j |j t|j |j dS)ajRestore the backup from this install state if it exists. If this installation has a backup stored in it (created by and update that used ReplaceWith(), above), it replaces this installation with the backup, using a temporary staging area. This installation is moved to the trash directory under the installation that exists after this is done. The trash directory can be removed at any point in the future. We just don't want to delete code that is running since some platforms have a problem with that. Returns: bool, True if there was a backup to restore, False otherwise. FT) r?rwrYr|r@rVr:r[rrNrS)r staging_staterrr RestoreBackupcs zInstallationState.RestoreBackupcCtj|jS)zDetermines if this install has a valid backup that can be restored. Returns: bool, True if there is a backup, False otherwise. )r+r$rLr@rrrrr?zInstallationState.HasBackupcCs|r|jSdS)zGets the backup directory of this installation if it exists. Returns: str, The path to the backup directory or None if it does not exist. N)r?r@rrrrBackupDirectorysz!InstallationState.BackupDirectorycC.tj|jr t|j|r|ddSdS)zDeletes the current staging directory if it exists. Args: progress_callback: f(float), A function to call with the fraction of completeness. rrN)r+r$existsrVrYRmTreerrtrrrrw   zInstallationState._ClearStagingcCr)zDeletes the current backup if it exists. Args: progress_callback: f(float), A function to call with the fraction of completeness. rrN)r+r$rLr@rYrrrrrrrzInstallationState.ClearBackupcCr)zDeletes the current trash directory if it exists. Args: progress_callback: f(float), A function to call with the fraction of completeness. rrN)r+r$rLrSrYrrrrrrrzInstallationState.ClearTrashunknowncCs(|||}|jj|||d}|S)aDownloads the given component based on the given snapshot. Args: snapshot: snapshots.ComponentSnapshot, The snapshot that describes the component to install. component_id: str, The component to install from the given snapshot. progress_callback: f(float), A function to call with the fraction of completeness. command_path: the command path to include in the User-Agent header if the URL is HTTP Returns: Optional[str], The path of the downloaded archive, or None if the component has no actual sources. Raises: installers.URLFetchError: If the component associated with the provided component ID has a URL that is not fetched correctly. rs)r[ComponentFromIdrXDownload)rsnapshotrIrtru componentdownloaded_archiverrrrs zInstallationState.DownloadcCs,|jj||d}t|j|}|||dS)aInstalls the archive previously downloaded from self.Download(). Args: snapshot: snapshots.ComponentSnapshot, The snapshot that describes the component to install. component_id: str, The component to install from the given snapshot. downloaded_archive: Optional[str], The path to the archive downloaded previously. progress_callback: f(float), A function to call with the fraction of completeness. rvN)rXExtractrBrC MarkInstalled)rrrIrrtr manifestrrrInstalls  zInstallationState.InstallcCst|j|}|}t|}|j}t}t}t|ddD]q\} } tj || } tj | s4tj | rwt | tj tj| } | drb| d} tj | rWt | |tj || d| rv|tj || tj | } | sdntj| r|tj| |r|| |q|D]}tj|rtj |st|qt|tddD]}tj|rtj |st|st|q|dS) a,Uninstalls the given component. Deletes all the files for this component and marks it as no longer being installed. Args: component_id: str, The id of the component to uninstall. progress_callback: f(float), A function to call with the fraction of completeness. rr)startz.pyc __pycache__T)keyreverseN)rBrCInstalledPathsrgrNset enumerater+r$rOr\islinkremovedirnamerTr]addrLrYrsortedrcrmdirMarkUninstalled)rrIrtrpaths total_pathsrootdirs_to_remove pycache_dirsnumpr$dir_pathpyc_pathdrrr UninstallsF         "  zInstallationState.UninstallcCs6|jD]}tj|j|}tj|rt|qdS)z=Clear deprecated directories that were not removed correctly.N)DEPRECATED_DIRSr+r$rOr<rLrYr)rrr$rrrClearDeprecatedDirs"s   z%InstallationState.ClearDeprecatedDirscCsHtj|jtjj}tj|jtjj}tj|sdSt ||dS)aNCopy this state's properties file to another state. This is primarily intended to be used to maintain the machine properties file during a schema-change-induced reinstall. Args: other_state: InstallationState, The installation state of the fresh Cloud SDK that needs the properties file mirrored in. N) r+r$rOr<rr;CLOUDSDK_PROPERTIES_NAMErr'copyfile)r other_state my_propertiesother_propertiesrrrr}*s   z)InstallationState.CopyMachinePropertiesToFc Cstjr td}ntjddkrtd}ntjddkr#td}nd}t|j5t j dd t j d d d d g}|durGt t dn|}|D] }tj||d||dqKWddS1scwYdS)ajAttempts to compile all the python files into .pyc files. Args: force: boolean, passed to force option of compileall.compiledir, workers: int, can be used to explicitly set number of worker processes; otherwise we determine it automatically. Only set for testing. This does not raise exceptions if compiling a given file fails. a((httplib2/python3|typing/python3|platform/bq/third_party/yaml/lib3|third_party/google/api_core|third_party/google/auth|third_party/google/oauth2|third_party/overrides|third_party/proto|dulwich|gapic|pubsublite|pubsub/lite_subscriptions.py|logging_v2|platform/bundledpythonunix|pubsub_v1/services)rrz.*a (kubernetes/utils/create_from_yaml.py|platform/google_appengine|gslib/vendored/boto/boto/iam/connection.py|gslib/vendored/boto/tests/|third_party/.*/python2/|third_party/yaml/[a-z]*.py|third_party/yaml/lib2/|third_party/antlr3/|appengine/|google/cloud/appengine_|google/cloud/bigquery_logging_v1|third_party/fancy_urllib/|platform/bq/third_party/gflags|platform/ext-runtime/nodejs/test/|platform/gsutil/third_party/apitools/ez_setup|platform/gsutil/third_party/pyparsing|platform/gsutil/third_party/crcmod_osx/crcmod/test)Nbin bootstrappingdataclilibplatformr&)rxquietforceworkers)sixPY2recompilerG version_inforYChDirr<r+r$rOmin cpu_count compileall compile_dir)rrrregex_exclusion to_compile num_workersrrrrCompilePythonFiles<s,     "z$InstallationState.CompilePythonFiles)N)Nr)FN)(r rrrrr;CLOUDSDK_STATE_DIRrPrQrRrUrfr staticmethodr>rArKr9rr[propertyr<rerjrmrqrrrr?rrwrrrrrrr}rrrrrr:}sh         '         8 r:c@sLeZdZdZdZddZddZddZd d Zd d Z d dZ ddZ dS)rBzDClass to encapsulate the data stored in installation manifest files.z .manifestcCs@||_||_tj|j|tj|_tj|j|tj |_ dS)zCreates a new InstallationManifest. Args: state_dir: str, The directory path where install state is stored. component_id: str, The component id that you want to get the manifest for. N) state_diridr+r$rOr:rf snapshot_filerBMANIFEST_SUFFIX manifest_file)rrrIrrrrs zInstallationManifest.__init__cCs^t|j}t|D] }||dq Wdn1swY|j|j|jddS)azMarks this component as installed with the given snapshot and files. This saves the ComponentSnapshot and writes the installed files to a manifest so they can be removed later. Args: snapshot: snapshots.ComponentSnapshot, The snapshot that was the source of the install. files: list of str, The files that were created by the installation.  N)rI)rY FileWriterr_NormalizeFileListwrite WriteToFilerr)rrr fpr_rrrrs  z"InstallationManifest.MarkInstalledcCs,|j|jfD] }tj|rt|qdS)zMarks this component as no longer being installed. This does not actually uninstall the component, but rather just removes the snapshot and manifest. N)rrr+r$r\r)rr_rrrrs   z$InstallationManifest.MarkUninstalledcCr)zLoads the local ComponentSnapshot for this component. Returns: The snapshots.ComponentSnapshot for this component. )r rkFromFilerrrrrrkrz&InstallationManifest.ComponentSnapshotcCs||jS)zLoads the ComponentSnapshot and get the schemas.Component this component. Returns: The schemas.Component for this component. )rkrrrrrrComponentDefinitionsz(InstallationManifest.ComponentDefinitioncCs |jjS)zGets the version string of this component as it was installed. Returns: str, The installed version of this component. )rversionversion_stringrrrrrDs z"InstallationManifest.VersionStringcCs@t|j}dd|D}Wd|S1swY|S)zGets the list of files and dirs created by installing this component. Returns: list of str, The files and directories installed by this component. cSsg|]}|qSr)rstrip)r^linerrrrbsz7InstallationManifest.InstalledPaths..N)rY FileReaderr)rr_r rrrrs  z#InstallationManifest.InstalledPathsN) r rrrrrrrrkrrDrrrrrrBs  rBcCstg}tg}tg}|D]0}t|}|dr"||dn||tj|}|r>||dtj|}|s/qt|||BS)z=Removes non-empty directory entries and sorts resulting list./) r posixpathrTr]rr+r$rr) file_listparent_directories directoriesr r_norm_file_pathrrrrs     r)#r __future__rrrrr/rEr+rrr'rGgooglecloudsdk.corerrgooglecloudsdk.core.consolergooglecloudsdk.core.updaterrr googlecloudsdk.core.utilr r rYrr rr r!r9objectr:rBrrrrrs@           % R