o S@sndZddlmZddlmZddlmZddlZddlZddlmZddlm Z ddlm Z dd lm Z dd l m Z dd l mZdd lmZdd lmZddlZzddl mZWn eycdZYnwersdeejddvrseZneZdZdZGdddeZGdddeZGdddeZGdddeZeej GdddeZ!Gdddej"Z#d"d d!Z$dS)#ae The Cloud SDK resource cache. A resource is an object maintained by a service. Each resource has a corresponding URI. A URI is composed of one or more parameters. A service-specific resource parser extracts the parameter tuple from a URI. A corresponding resource formatter reconstructs the URI from the parameter tuple. Each service has an API List request that returns the list of resource URIs visible to the caller. Some APIs are aggregated and return the list of all URIs for all parameter values. Other APIs are not aggregated and require one or more of the parsed parameter tuple values to be specified in the list request. This means that getting the list of all URIs for a non-aggregated resource requires multiple List requests, ranging over the combination of all values for all aggregate parameters. A collection is list of resource URIs in a service visible to the caller. The collection name uniqely identifies the collection and the service. A resource cache is a persistent cache that stores parsed resource parameter tuples for multiple collections. The data for a collection is in one or more tables. +---------------------------+ | resource cache | | +-----------------------+ | | | collection | | | | +-------------------+ | | | | | table | | | | | | (key,...,col,...) | | | | | | ... | | | | | +-------------------+ | | | | ... | | | +-----------------------+ | | ... | +---------------------------+ A resource cache is implemented as a ResourceCache object that contains Collection objects. A Collection is a virtual table that contains one or more persistent cache tables. Each Collection is also an Updater that handles resource parsing and updates. Updates are typically done by service List or Query requests that populate the tables. The Updater objects make this module resource agnostic. For example, there could be updater objects that are not associated with a URI. The ResourceCache doesn't care. If the List request API for a collection aggregates then its parsed parameter tuples are contained in one table. Otherwise the collection is stored in multiple tables. The total number of tables is determined by the number of aggregate parameters for the List API, and the number of values each aggregate parameter can take on. )absolute_import)division)unicode_literalsN)config)log) module_util) properties) exceptions) file_cache)encoding)files) sqlite_cachesqlCLOUDSDK_CACHE_IMPLEMENTATIONsqliteizgooglecloudsdk.resource-1.0c@s4eZdZdZd ddZd ddZdd Zd d ZdS) ParameterInfoaAn object for accessing parameter values in the program state. "program state" is defined by this class. It could include parsed command line arguments and properties. The class also can also map between resource and program parameter names. Attributes: _additional_params: The list of parameter names not in the parsed resource. _updaters: A parameter_name => (Updater, aggregator) dict. NcCs|pg|_|pi|_dSN)_additional_params _updaters)selfadditional_paramsupdatersrE/tmp/google-cloud-sdk/lib/googlecloudsdk/core/cache/resource_cache.py__init__ss zParameterInfo.__init__TcC~~dS)zReturns the program state string value for parameter_name. Args: parameter_name: The Parameter name. check_properties: Check the property value if True. Returns: The parameter value from the program state. Nr)rparameter_namecheck_propertiesrrrGetValuews zParameterInfo.GetValuecCs|jS)aReturn the list of parameter names not in the parsed resource. These names are associated with the resource but not a specific parameter in the resource. For example a global resource might not have a global Boolean parameter in the parsed resource, but its command line specification might require a --global flag to completly qualify the resource. Returns: The list of parameter names not in the parsed resource. )rrrrrGetAdditionalParamss z!ParameterInfo.GetAdditionalParamscCs|j|dS)a1Returns the updater and aggregator property for parameter_name. Args: parameter_name: The Parameter name. Returns: An (updater, aggregator) tuple where updater is the Updater class and aggregator is True if this updater must be used to aggregate all resource values. NN)rget)rrrrr GetUpdaters zParameterInfo.GetUpdaterr!)T)__name__ __module__ __qualname____doc__rrr r#rrrrrgs   rc@seZdZdZdddZdS) ParameterzA parsed resource tuple parameter descriptor. A parameter tuple has one or more columns. Each has a Parameter descriptor. Attributes: column: The parameter tuple column index. name: The parameter name. rNcCs||_||_dSr)columnname)rr)r*rrrrs zParameter.__init__)rN)r$r%r&r'rrrrrr(s r(cs eZdZdZfddZZS)_RuntimeParameteraA runtime Parameter. Attributes: aggregator: True if parameter is an aggregator (not aggregated by updater). generate: True if values must be generated for this parameter. updater_class: The updater class. value: A default value from the program state. cs4tt|j|j|jdd|_||_||_||_dS)N)r*F) superr+rr)r*generate updater_classvalue aggregator)r parameterr.r/r0 __class__rrrs  z_RuntimeParameter.__init__)r$r%r&r'r __classcell__rrr2rr+s r+c@seZdZdZdS) BaseUpdaterz(A base object for thin updater wrappers.N)r$r%r&r'rrrrr5sr5cseZdZdZdddddeffdd ZdddZdd Zd d Zdd d Z dddZ ddZ ddZ ddZ dddZejdddZZS)UpdateraA resource cache table updater. An updater returns a list of parsed parameter tuples that replaces the rows in one cache table. It can also adjust the table timeout. The parameters may have their own updaters. These objects are organized as a tree with one resource at the root. Attributes: cache: The persistent cache object. collection: The resource collection name. columns: The number of columns in the parsed resource parameter tuple. parameters: A list of Parameter objects. timeout: The resource table timeout in seconds, 0 for no timeout (0 is easy to represent in a persistent cache tuple which holds strings and numbers). NrcsFtt|||_||_|r|nd|_||_|pg|_|pd|_dS)aUpdater constructor. Args: cache: The persistent cache object. collection: The resource collection name that (1) uniquely names the table(s) for the parsed resource parameters (2) is the lookup name of the resource URI parser. Resource collection names are unique by definition. Non-resource collection names must not clash with resource collections names. Prepending a '.' to non-resource collections names will avoid the clash. columns: The number of columns in the parsed resource parameter tuple. Must be >= 1. column: If this is an updater for an aggregate parameter then the updater produces a table of aggregate_resource tuples. The parent collection copies aggregate_resource[column] to a column in its own resource parameter tuple. parameters: A list of Parameter objects. timeout: The resource table timeout in seconds, 0 for no timeout. rN) r,r6rcache collectioncolumnsr) parameterstimeout)rr8r9r:r)r;r<r2rrrs zUpdater.__init__cCs4|jr|jg}nt|g}|r||d|S)aReturns the table name; the module path if no collection. Args: suffix_list: a list of values to attach to the end of the table name. Typically, these will be aggregator values, like project ID. Returns: a name to use for the table in the cache DB. .)r9r GetModulePathextendjoin)r suffix_listr*rrr _GetTableNames     zUpdater._GetTableNamecCsLg}|jD]}||j\}}|j|j|d}t||||}||q|S)aRConstructs and returns the _RuntimeParameter list. This method constructs a muable shadow of self.parameters with updater_class and table instantiations. Each runtime parameter can be: (1) A static value derived from parameter_info. (2) A parameter with it's own updater_class. The updater is used to list all of the possible values for the parameter. (3) An unknown value (None). The possible values are contained in the resource cache for self. The Select method combines the caller supplied row template and the runtime parameters to filter the list of parsed resources in the resource cache. Args: parameter_info: A ParamaterInfo object for accessing parameter values in the program state. Returns: The runtime parameters shadow of the immutable self.parameters. )r)r;r#r*rr+append)rparameter_inforuntime_parametersr1r.r0r/runtime_parameterrrr_GetRuntimeParameters s  zUpdater._GetRuntimeParameterscCstS)z"Returns the parameter info object.)rrrrrr*szUpdater.ParameterInfocCs|sg}td|jddd|Dz||WStjyB|||}|dur9|| || |j|ddYSw)aReturns the list of rows matching row_template in table. Refreshes expired tables by calling the updater. Args: table: The persistent table object. row_template: A row template to match in Select(). parameter_info: A ParamaterInfo object for accessing parameter values in the program state. aggregations: A list of aggregation Parameter objects. Returns: The list of rows matching row_template in table. z cache table=%s aggregations=[%s] cSsg|] }d|j|jqS)z{}={})formatr*r/.0xrrr Az'Updater.SelectTable..NT)ignore_expiration) rinfor*r@Selectr CacheTableExpiredUpdate DeleteRowsAddRowsValidate)rtable row_templaterD aggregationsrowsrrr SelectTable.s    zUpdater.SelectTablecCst|}|jt|kr|dg|jt|7}tdddd|Dgg}g}||}t|D]\}}d|_|j rn||j dvrn|j ||j <td|j |j |j |j |j rm| |d |_|D]} | |j qdq3|j r| |d |_td|j |j |j |j |j|jd } dg| j} ||j | | j <td |j |j |j g} | |d||| |D] \} }| | | |q| }q3|sd d|D}d|vrgS|j|d }|jj||j|j|jd}|||||Sg}|||||D] \}}||qtd||S)a%Returns the list of rows matching row_template in the collection. All tables in the collection are in play. The row matching done by the cache layer conveniently prunes the number of tables accessed. Args: row_template: A row template tuple. The number of columns in the template must match the number of columns in the collection. A column with value None means match all values for the column. Each column may contain these wildcard characters: * - match any string of zero or more characters ? - match any character The matching is anchored on the left. parameter_info: A ParamaterInfo object for accessing parameter values in the program state. Returns: The list of rows that match the template row. Nzcache template=[%s]z, cSsg|]}d|qS)z'{}')rI)rKtrrrrMdz"Updater.Select..F)N*z2cache parameter=%s column=%s value=%s aggregate=%sT)r8z)cache parameter=%s column=%s aggregate=%scSsg|]}|jqSr)r/rJrrrrMsrAr:keysr<z cache rows=%s)listr:lenrrPr@rG enumerater-r/r)r*r0rCr.r8 YieldSelectTableFromPermutations ExtendValuesrBTabler<r[r?)rrXrDtemplatevaluesrYr;ir1vupdater sub_template new_valuespermselectedaggregation_values table_namerWrZ_rrrrQLs|           zUpdater.SelectcCs,||}|D] }|j|kr|jSqdS)z9Get this updater's column number for a certain parameter.N)rGr*r))rrDrupdater_parametersr1rrr_GetParameterColumns   zUpdater._GetParameterColumncsFfdd|D}tdj|fdd|D}||dS)z?Add selected values to a template and extend the selected rows.csg|]}|jqSr)r))rKrowrrrrMr]z(Updater.ExtendValues..z$cache collection={} adding values={}csg|]}|gqSrrrKval)rorrrMr]N)rrPrIr9r?)rrirorpvalsrkr)rorrrfs zUpdater.ExtendValuesc cs|D]Q}dd|D}|jj|j|d|j|j|jd}g}|D]%} | jr>||| j} | dur2q!|d|| <|| | _ | j rF| | q!| ||||} || fVqdS)aSelects completions from tables using multiple permutations of values. For each vector in values, e.g. ['my-project', 'my-zone'], this method selects rows matching the template from a leaf table corresponding to the vector (e.g. 'my.collection.my-project.my-zone') and yields a 2-tuple containing that vector and the selected rows. Args: parameters: [Parameter], the list of parameters up through the current updater belonging to the parent. These will be used to iterate through each permutation contained in values. values: list(list()), a list of lists of valid values. Each item in values corresponds to a single permutation of values for which item[n] is a possible value for the nth generator in parent_parameters. template: list(str), the template to use to select new values. parameter_info: ParameterInfo, the object that is used to get runtime values. Yields: (perm, list(list)): a 2-tuple where the first value is the permutation currently being used to select values and the second value is the result of selecting to match the permutation. cSsg|]}|qSrrrwrrrrMsz.r_r`Nr) r8rgrBr:r<r-rur*popr/rCr[) rr;rirhrDro temp_permrWrYr1r)rprrrres.    z(Updater.YieldSelectTableFromPermutationsTcs@||}fdd|D}|jj|j|d|j|j|j|dS)a$Returns the table for row. Args: row: The fully populated resource row. parameter_info: A ParamaterInfo object for accessing parameter values in the program state. create: Create the table if it doesn't exist if True. Returns: The table for row. csg|] }|jr|jqSr)r0r))rKprvrrrMrNz*Updater.GetTableForRow..r_)r:rar<create)rGr8rgrBr:r<)rrvrDr~r;rirr}rGetTableForRows  zUpdater.GetTableForRowcCr)z;Returns the list of all current parsed resource parameters.Nr)rrDrYrrrrSszUpdater.UpdaterNTr!)r$r%r&r'DEFAULT_TIMEOUTrrBrGrr[rQrurfrerabcabstractmethodrSr4rrr2rr6s( "  V /r6cs.eZdZdZdfdd ZeddZZS) ResourceCachezA resource cache object.NTcs&|s|}tt|j||tddS)aResourceCache constructor. Args: name: The persistent cache object name. If None then a default name conditioned on the account name is used. /cache//resource.cache create: Create the cache if it doesn't exist if True. r*r~versionN)GetDefaultNamer,rrVERSION)rr*r~r2rrrs    zResourceCache.__init__cCsTtjg}tjjjjdd}|r||t t j j ||dt j j |S)z(Returns the default resource cache name.F)requiredzresource.cache)rPaths cache_dirrVALUEScoreaccountGetrCr MakeDirospathr@)rrrrrr s    zResourceCache.GetDefaultNamer)r$r%r&r'r staticmethodrr4rrr2rrs rc CsT|st}ttfD]}|sq z|j|dtdWdStjy'Yq wdS)zGDeletes the current persistent resource cache however it's implemented.FrN) rrr r CacherDeleter CacheInvalid)r*implementationrrrrs rr)%r' __future__rrrrrgooglecloudsdk.corerrrrgooglecloudsdk.core.cacher r googlecloudsdk.core.utilr r sixr ImportErrorGetEncodedValueenvironPERSISTENT_CACHE_IMPLEMENTATIONrrobjectrr(r+r5 add_metaclassABCMetar6rrrrrrrsL 5           8 8