^SrSSKJr SSKJr SSKJr SSKrSSKJrJr SSK J r SSK J r SS K Jr SS KJr SS KJr S rS rSrSrSr\R4"SS5r"SS5rg)z-Client for interacting with Storage Insights.)absolute_import)division)unicode_literalsN)SequenceOptional) list_pager)errors)client)apis) propertiesi'CSV options cannot be set with parquet.cLUb URS5RS5$S$)z;Returns the ASCII string unescaping any escaped characters.Nasciizunicode-escape)encodedecode)strings 2lib/googlecloudsdk/api_lib/storage/insights_api.py_get_unescaped_asciir%s3"- w  & & 8378c@SRXR55$)Nzprojects/{}/locations/{})formatlower)projectlocations r_get_parent_stringr+s $ * *7NN4D EErc[R"5nURU5n[URR URR R55$N)gcs_json_client JsonClient get_bucketrmetadata projectNumberrr)bucket gcs_clientbucket_resources r_get_parent_string_from_bucketr&2sT))+*))&1/ ,,''--/ rReportFormatOptionscsvparquetc%>^\rSrSrSrU4SjrS0S\S\S\S\S\S \\S \ S \\ \S \\ \S \\ \S\\ \S\\ \S\\ \S\\ \S\\ \S\ S\S\4$Sjjr Sr Sr SrSrS1SjrS2S \ S\\ \S\\ \S\\ \S \\ \S\\ \S\\ \S\\ \S\\ \S\ S\S \\S\4SjjrS2S \ S \\ \S \\ \S\\ \S \\ \S\\ \S\\ \S\\ \S\\ \S\ S\S \\S\4SjjrS rS3S!jrS"rS4S#jrS$rS5S%jrS&rS'rS(rS6S)jrS7S*jrS+rS,rS-r S.r!S/r"U=r#$)8 InsightsApiBz Client for Storage Insights API.c>[[U] 5 [R"SS5Ul[R "SS5Ulg)Nstorageinsightsv1)superr,__init__ core_apisGetClientInstancer GetMessagesModulemessages)self __class__s rr2InsightsApi.__init__Es: +t%'--.?FDK//0A4HDMrdataset_config_namerdestination_projectorganization_numberretention_periodactivity_data_retention_periodorganization_scopesource_projects_listsource_folders_list!include_buckets_prefix_regex_list!exclude_buckets_prefix_regex_listinclude_buckets_name_listexclude_buckets_name_listinclude_source_locationsexclude_source_locationsauto_add_new_buckets identity_type descriptionc UbMURRRUR55nURRUS9nOAURRURRRRS9nU(aURR US9OSnU (aURR U S9OSnURRUUUUUUUUU(aUOSS9 nUbUUlU (dU (aU =(d /Vs/sHnURRUS9PM nnUU =(d /Vs/sHnURRUS9PM sn- nURRUS9Ul U (dU (aU =(d /Vs/sHnURRUS9PM nnUU =(d /Vs/sHnURRUS9PM sn- nURRUS9Ul U(aURRUS 9UlU(aURRUS 9UlURR!UU[#X25S 9nUR$R&R)U5$s snfs snfs snfs snf) a Creates a dataset config. Args: dataset_config_name: Name for the dataset config being created. location: The location where insights data will be stored in a GCS managed BigQuery instance. destination_project: The project in which the dataset config is being created and by extension the insights data will be stored. organization_number: Organization number of the organization to which all source projects / folders must belong. retention_period: No of days for which insights data is to be retained in BigQuery instance. activity_data_retention_period: No of days for which activity data is to be retained in BigQuery instance. If not set, retention period will be used by the API. organization_scope: If True, Insights data will be collected for all resources in the given organization. source_projects_list: List of source project numbers. Insights data is to be collected for buckets that belong to these projects. source_folders_list: List of source folder numbers. Insights data is to be collected for buckets that belong to these folders. include_buckets_prefix_regex_list: List of bucket prefix regex patterns which are to be included for insights processing from the source projects. We can either use included or excluded bucket parameters. exclude_buckets_prefix_regex_list: List of bucket prefix regex patterns which are to be excluded from insights processing from the source projects. We can either use included or excluded bucket parameters. include_buckets_name_list: List of bucket names which are to be included for insights processing from the source projects. We can either use included or excluded bucket parameters. exclude_buckets_name_list: List of bucket names which are to be excluded from insights processing from the source projects. We can either use included or excluded bucket parameters. include_source_locations: List of bucket locations which are to be included for insights processing from the source projects. We can either use included or excluded location parameters. exclude_source_locations: List of bucket locations which are to be excluded from insights processing from the source projects. We can either use included or excluded location parameters. auto_add_new_buckets: If True, auto includes any new buckets added to source projects that satisfy the include/exclude criterias. identity_type: Option for how permissions need to be setup for a given dataset config. Default option is IDENTITY_TYPE_PER_CONFIG. description: Human readable description text for the given dataset config. Returns: An instance of Operation message N)typeprojectNumbers folderNumbers) rJidentityincludeNewlyCreatedBucketsnameorganizationNumberretentionPeriodDayssourceProjects sourceFoldersorganizationScope bucketNamebucketPrefixRegexcloudStorageBuckets locations) datasetConfigdatasetConfigIdparent)r6IdentityTypeValueValuesEnumupperIDENTITY_TYPE_PER_CONFIGSourceProjects SourceFolders DatasetConfigactivityDataRetentionPeriodDaysCloudStorageBucketCloudStorageBucketsexcludeCloudStorageBucketsincludeCloudStorageBucketsCloudStorageLocationsexcludeCloudStorageLocationsincludeCloudStorageLocations;StorageinsightsProjectsLocationsDatasetConfigsCreateRequestrr !projects_locations_datasetConfigsCreate)r7r:rr;r<r=r>r?r@rArBrCrDrErFrGrHrIrJidentity_type_enumsource_projectssource_foldersdataset_config excluded_nameexcluded_storage_bucketsexcluded_regex included_nameincluded_storage_bucketsincluded_regexcreate_requests rcreate_dataset_config!InsightsApi.create_dataset_configJsP ==11EE    mm,,2D,Emmm,,}}%%99RR-m mm22+3#' ]]00)1"&]]00#7 .,&$0B,1 N&1 (4 !$E 9>B>">m -- * *m * D>" A GR G# Gn -- * *^ * L G# -- + +": , / !$E 9>B>">m -- * *m * D>" A GR G# Gn -- * *^ * L G# -- + +": , /  -- - -0 . 1  -- - -0 . 1 ]]^^$+!"5@_N ;; 8 8 ? ? OOa"#"#s%#K#K 9#K,#Kc~URRUS9nURRR U5$)z Creates the dataset config link.rS)r6@StorageinsightsProjectsLocationsDatasetConfigsLinkDatasetRequestr rt LinkDatasetr7dataset_config_relative_namerequests rcreate_dataset_config_link&InsightsApi.create_dataset_config_links<mm\\ )]G ;; 8 8 D DW MMrc~URRUS9nURRR U5$)zDeletes the dataset config.r)r6;StorageinsightsProjectsLocationsDatasetConfigsDeleteRequestr rtDeleters rdelete_dataset_config!InsightsApi.delete_dataset_configs<mmWW )XG ;; 8 8 ? ? HHrc~URRUS9nURRR U5$)z Deletes the dataset config link.r)r6BStorageinsightsProjectsLocationsDatasetConfigsUnlinkDatasetRequestr rt UnlinkDatasetrs rdelete_dataset_config_link&InsightsApi.delete_dataset_config_links<mm^^ )_G ;; 8 8 F Fw OOrczURRRURR US95$)zGets the dataset config.r)r rtGetr68StorageinsightsProjectsLocationsDatasetConfigsGetRequest)r7rs rget_dataset_configInsightsApi.get_dataset_configs: ;; 8 8 < < NN- O  rcUb=[[RRRR 5U5nO<[[RRRR 5S5n[ R"URRURRUS9UbUO[SSS9$)zLists the dataset configs. Args: location (str): The location where insights data will be stored in a GCS managed BigQuery instance. page_size (int|None): Number of items per request to be returned. Returns: List of dataset configs. -rcpageSizedatasetConfigs batch_sizebatch_size_attributefield) rr VALUEScorerrr YieldFromListr rtr69StorageinsightsProjectsLocationsDatasetConfigsListRequest PAGE_SIZE)r7r page_sizercs rlist_dataset_configInsightsApi.list_dataset_configs!    ( ( , , .f"*"3"3"8"8"@"@"D"D"FLf  # # 55 OO P !* 599' rrwrxc /nSU4SUSL4SUSL4SUSL=(d USL4SUSL=(d USL4SUSL4SU SL4S U SL4S U SL4S U SL4S U SL4/ nUH unnU(dMURU5 M" U$) zReturns the update_mask list.rXrVNrWrornrrrqrRrUrkrJ)append)r7r?rwrxrDrBrErCrFrGrHr=r>rJ update_maskmask_conditionsmask conditions r_get_dataset_config_update_mask+InsightsApi._get_dataset_config_update_mask#s"K 01 ?$67 .45 ( %T 1 =0< ) %T 1 =0< ()A)MN ')A)MN %';4'GH  0 <= - *$ 6   4/0-O2+i 4 + rcURUUUUUUUU U U U U US9 nU(d%[R"SRU55eUbURR US9OSnUbURR US9OSnURRU UUUU S9nU(aSUlU bU Ul U(dU(aU=(d /Vs/sHnURRUS9PM nnUU=(d /Vs/sHnURRUS 9PM sn- nURRUS 9Ul OU(dU(aU=(d /Vs/sHnURRUS9PM nnUU=(d /Vs/sHnURRUS 9PM sn- nURRUS 9Ul U (aURRU S 9UlU (aURRU S 9UlURR#UUS R%U5S 9nUR&R(R+U5$s snfs snfs snfs snf)aE Updates the dataset config. Args: dataset_config_relative_name: The relative name of the dataset config to be modified. organization_scope: If True, Insights data will be collected for all resources in the given organization. If this field is Empty/None, Either the sourceProjects or the sourceFolders must be set to update the scope. source_projects_list: List of source project numbers. Insights data is to be collected for buckets that belong to these projects. If this field is Empty/None, Either the organizationScope will be True or the sourceFolders must be set to update the scope. source_folders_list: List of source folder IDs. Insights data is to be collected for buckets that belong to these organisational folders. If this field is Empty/None, Either the organizationScope will be True or the sourceProjects must be set to update the scope. include_buckets_name_list: Optional list of bucket names which are to be included for insights processing from the source projects. Can either use included or excluded bucket parameters. include_buckets_prefix_regex_list: Optional list of bucket prefix regex patterns which are to be included for insights processing from the source projects. Can either use included or excluded bucket parameters. exclude_buckets_name_list: Optional list of bucket names which are to be excluded from insights processing from the source projects. Can either use included or excluded bucket parameters. exclude_buckets_prefix_regex_list: Optional list of bucket prefix regex patterns which are to be excluded from insights processing from the source projects. Can either use included or excluded bucket parameters. include_source_locations: Optional list of bucket locations which are to be included for insights processing from the source projects. Can either use included or excluded location parameters. exclude_source_locations: Optional list of bucket locations which are to be excluded from insights processing from the source projects. Can either use included or excluded location parameters. auto_add_new_buckets: If True, auto includes any new buckets added to source projects that satisfy the include/exclude criterias. retention_period: No of days for which insights data is to be retained in BigQuery instance. activity_data_retention_period: No of days for which activity data is to be retained in BigQuery instance. description: Human readable description text for the given dataset config. Returns: An instance of Operation message. ) r?rwrxrDrBrErCrFrGrHr=r>rJz(Nothing to update for dataset config: {}NrMrO)rUrJrVrWrRTrYr[r]r_,)rSra updateMask)rr InsightApiErrorrr6rhrirjrXrkrlrmrnrorprqrr:StorageinsightsProjectsLocationsDatasetConfigsPatchRequestjoinr rtPatch)r7rr?r@rArDrBrErCrFrGrHr=r>rJrrwrxryrzr{r|r}r~rrs rupdate_dataset_config!InsightsApi.update_dataset_configUs H66-,*";*K";*K!9!91)'E7K   " " 4 ; ;*    )mm22+3/3  (]]00)1.2]]00,&$#7 1N)-n&%1 (4!$E 9>B>">m -- * *m * D>" A GR G# Gn -- * *^ * L G# -- + +": , / #&G 9>B>">m -- * *m * D>" A GR G# Gn -- * *^ * L G# -- + +": , /  -- - -0 . 1  -- - -0 . 1 mmVV )$88K(WG ;; 8 8 > >w GG_"#"#s#I;#J)#J#J cU(aHURR5nU(dU(dU(a[R"S5eSnO(Sn[ U5nURR UUUS9n[ UUS9$)z%Returns ReportFormatOptions instance.r N) delimiterheaderRequiredrecordSeparatorr()r6ParquetOptionsr GcsApiErrorr CSVOptionsr')r7 csv_separator csv_delimiter csv_headerr*parquet_options csv_optionsunescaped_separators r_get_report_format_options&InsightsApi._get_report_format_optionss~ 446o }   !JKKko0?MM,,!#--k   rc RURRURRURURUR S9URRURURUR S9[ URRRUR55S9n URRUURRURURS9URRUS9S9n URXxX5nURRUR UR"U U U S9nURR%['U5US9nUR(R*R-U5$)aCreates a report config. Args: source_bucket (str): Source bucket name for which reports will be generated. destination_url (storage_url.CloudUrl): The destination url where the generated reports will be stored. metadata_fields (list[str]): Fields to be included in the report. start_date (datetime.datetime.date): The date to start generating reports. end_date (datetime.datetime.date): The date after which to stop generating reports. frequency (str): Can be either DAILY or WEEKLY. csv_separator (str): The character used to separate the records in the CSV file. csv_delimiter (str): The delimiter that separates the fields in the CSV file. csv_header (bool): If True, include the headers in the CSV file. parquet (bool): If True, set the parquet options. display_name (str): Display name for the report config. Returns: The created ReportConfig object. yearmonthday startDateendDate frequencyr#destinationPath)r#)metadataFieldsstorageDestinationOptionsstorageFilters csvOptionsparquetOptions displayNamefrequencyOptionsobjectMetadataReportOptions)rc reportConfig)r6FrequencyOptionsDaterrrgetattrFrequencyValueValuesEnumrfObjectMetadataReportOptionsCloudStorageDestinationOptions bucket_name resource_nameCloudStorageFiltersr ReportConfigr)r*:StorageinsightsProjectsLocationsReportConfigsCreateRequestr&r projects_locations_reportConfigsru)r7 source_bucketdestination_urlmetadata_fields start_dateend_daterrrrr* display_namefrequency_optionsobject_metadata_report_optionsreport_format_options report_configrs rcreate_inventory_report#InsightsApi.create_inventory_reportsJ 66--$$ (8(8jnn%N ""hnn(,,#H MM * * C C OO  7 &*]]%N%N&"&--"N"N"..+99#O#;}}88 9" &O&#"!;;j;MM..(,,,44 *$B /DM ]]]]-m<"^$N ;; 7 7 > > rcZ/nUb*URSRUR55 UbaURSRUR55 URb*URSRUR55 U(aSR U5$g)z1Returns the filter string used for list API call.Nz6objectMetadataReportOptions.storageFilters.bucket="{}"zAobjectMetadataReportOptions.storageDestinationOptions.bucket="{}"zJobjectMetadataReportOptions.storageDestinationOptions.destinationPath="{}"z AND )rrrrr)r7r destination filter_lists r_get_filters_for_list!InsightsApi._get_filters_for_listWsK  B I I'' )*  {6679  " " . ##)6+*C*C#D F \\+ && rc Ub=[[RRRR 5U5nO$[ Ub URO UR5n[R"URRURRUURX5S9UbUO[SSS9$)aLists the report configs. Args: source_bucket (storage_url.CloudUrl): Source bucket for which reports will be generated. destination (storage_url.CloudUrl): The destination url where the generated reports will be stored. location (str): The location for which the report configs should be listed. page_size (int|None): Number of items per request to be returend. Returns: List of Report configs. )rcfilterr reportConfigsr)rr rrrrr&rrrr rr68StorageinsightsProjectsLocationsReportConfigsListRequestrr)r7rrrrrcs rlist_inventory_reports"InsightsApi.list_inventory_reportsms"!*"3"3"8"8"@"@"D"D"F"*,f.  &  # #,7,C,CEf  # # 44 NN--mI O K!* 599' rczURRRURR US95$)zGets the report config.r)r rrr67StorageinsightsProjectsLocationsReportConfigsGetRequestr7report_config_names rget_inventory_report InsightsApi.get_inventory_reports: ;; 7 7 ; ; MM# N % &&rc~URRXS9nURRR U5$)zDeletes the report config.)rSforce)r6:StorageinsightsProjectsLocationsReportConfigsDeleteRequestr rr)r7rr rs rdelete_inventory_report#InsightsApi.delete_inventory_reportsA C C# D 2  ;; 7 7 > >w GGrc/nUbKURRURURURS9nUR S5 OSnUbKURRURURURS9nUR S5 OSnUbJ[ URRRUR55nUR S5 OSnURRUUUS9U4$)zBReturns a tuple of messages.FrequencyOptions and update_mask list.NrzfrequencyOptions.startDatezfrequencyOptions.endDatezfrequencyOptions.frequencyr) r6rrrrrrrrrf)r7rrrrstart_date_messageend_date_messagefrequency_messages r&_get_frequency_options_and_update_mask2InsightsApi._get_frequency_options_and_update_masksK==--j&6&6JNN.L56++}}HNN ,F34! -- ( ( A A // 56 &&($' ' )   rc/nU(aURS5 UbQURRURURS9nURS5 URS5 OSnURR UUS9U4$)zHReturns a tuple of messages.ObjectMetadataReportOptions and update_mask.z*objectMetadataReportOptions.metadataFieldsNrz:>#('^P^P^P ^P  ^P  ^P'/sm^P^P%Xc]3^P$HSM2^P*2(3-)@^P*2(3-)@^P"*(3-!8^P"*(3-!8^P!)# 7^P !)# 7!^P"!#^P$%^P&'^P@NIP<"&1504;?CG;?CG:>:>#'"6:00  .0x}- 0 "*(3-!8 0 *2(3-)@ 0"*(3-!80*2(3-)@0!)# 70!)# 70!00'/sm00j"&6:59;?CG;?CG:>:>#'"6:fHfH%Xc]3 fH $HSM2 fH "*(3-!8 fH*2(3-)@fH"*(3-!8fH*2(3-)@fH!)# 7fH!)# 7fH!fHfH'/smfHfHP4BH.LPB& H<M00(GGR &   rr,)rE __future__rrr collectionstypingrrapitools.base.pyrgooglecloudsdk.api_lib.storager 'googlecloudsdk.api_lib.storage.gcs_jsonr rgooglecloudsdk.api_lib.utilr r3googlecloudsdk.corer r_CSV_PARQUET_ERROR_MESSGErrr& namedtupler'r,rrrVsj4&'%'1M9*  E8 F",,- T T r