g;SrSSKrSSKrSSKrSSKrSSKrSSKJr SSK r SSK J r SSK J r "SS\RR5r"SS \RR5r"S S \RR5r"S S \RR5r"SS\RR5r"SS\RR5r"SS\RR5r"SS\5r\RR0R3\\5 "SS\5r\RR0R3\\5 g)z0Classes for dealing with I/O from ML pipelines. N) cStringIO) _decoders)_filec*\rSrSrSrSrSrSrSrg)ExampleProtoCoderz8A coder to encode and decode TensorFlow Example objects.c.SSKnURUlgNr) tensorflowtrain _tf_train)selftfs ,lib/third_party/ml_sdk/cloud/ml/io/coders.py__init__ExampleProtoCoder.__init__"sXXDNc"UR5$)zEncodes Tensorflow example object to a serialized string. Args: example_proto: A Tensorflow Example object Returns: String. )SerializeToString)r example_protos rencodeExampleProtoCoder.encode&s  * * ,,rc\URR5nURU5 U$)zDecodes a serialized string into a Tensorflow Example object. Args: serialized_str: string Returns: Tensorflow Example object. )r ExampleParseFromString)rserialized_strexamples rdecodeExampleProtoCoder.decode1s)nn$$&G N+ Nr)r N __name__ __module__ __qualname____firstlineno____doc__rrr__static_attributes__rrrrs@ - rrc.\rSrSrSrSSjrSrSrSrg) JsonCoder?z1A coder to encode and decode JSON formatted data.NcXlgN_indent)rindents rrJsonCoder.__init__BsLrc@[R"XRSS9$)zbEncodes a python object into a JSON string. Args: obj: python object. Returns: JSON string. ),z: )r/ separators)jsondumpsr.robjs rrJsonCoder.encodeEs ::c,,; GGrc.[R"U5$)zlDecodes a JSON string to a python object. Args: json_string: A JSON string. Returns: A python object. )r4loads)r json_strings rrJsonCoder.decodeQs ::k ""rr-r,r r'rrr)r)?s9 H #rr)cJ\rSrSrSr"SS\5rS SjrSrSr Sr g ) CsvCoder]z3A coder to encode and decode CSV formatted data. c0\rSrSrSrSrSrSrSrSr g) CsvCoder._WriterWrapperaz?A wrapper for csv.writer / csv.DictWriter to make it picklable.cXU4Ul[5UlU(a'[R"URUSUS9Ulg[R "URSUS9Ulg)N)lineterminator delimiter)_stater_buffercsv DictWriter_writerwriter)r column_namesrFdecode_to_dicts rr CsvCoder._WriterWrapper.__init__dsY!n=dk[dl ~~ LL  ! zz LL! rcURRU5 URR5nURR S5 URR S5 U$r )rKwriterowrHgetvalueseektruncate)rrecordvalues r encode_record%CsvCoder._WriterWrapper.encode_recordssO llF#ll##%e ll llA lrcUR$r,)rGrs r __getstate__$CsvCoder._WriterWrapper.__getstate__{s [[rc"UR"U6 gr,)r)rstates r __setstate__$CsvCoder._WriterWrapper.__setstate__~s mmUr)rHrGrKN) r!r"r#r$r%rrWr[r_r&r'rr_WriterWrapperrAasI !rracj[R"XX4XV5UlURUUUS9Ulg)aInitializes CsvCoder. Args: column_names: Tuple of strings. Order must match the order in the file. numeric_column_names: Tuple of strings. Contains column names that are numeric. Every name in numeric_column_names must also be in column_names. delimiter: A one-character string used to separate fields. decode_to_dict: Boolean indicating whether the docoder should generate a dictionary instead of a raw sequence. True by default. fail_on_error: Whether to fail if a corrupt row is found. Default is True. skip_initial_space: When True, whitespace immediately following the delimiter is ignored when reading. )rMrFrNN)r CsvDecoder_decoderra_encoder)rrMnumeric_column_namesrFrN fail_on_errorskip_initial_spaces rrCsvCoder.__init__s@"((I+DM''!%('DMrc8URRU5$)aDecode csv line into a python dict. Args: csv_line: String. One csv line from the file. Returns: Python dict where the keys are the column names from the file. The dict values are strings or numbers depending if a column name was listed in numeric_column_names. Missing string columns have the value '', while missing numeric columns have the value None. If there is an error in parsing csv_line, a python dict is returned where every value is '' or None. Raises: Exception: The number of columns to not match. )rdr)rcsv_lines rrCsvCoder.decodes" ==   ))rc8URRU5$)a7Encode python dict to a csv-formatted string. Args: python_data: A python collection, depending on the value of decode_to_dict it will be a python dictionary where the keys are the column names or a sequence. Returns: A csv-formatted string. The order of the columns is given by column_names. )rerW)r python_datas rrCsvCoder.encodes == & &{ 33r)rdreN)r2TTF) r!r"r#r$r%objectrarrrr&r'rrr>r>]s0v@DG26"''2*& 4rr>c*\rSrSrSrSrSrSrSrg) YamlCoderz1A coder to encode and decode YAML formatted data.c[R(a+[RUl[RUlg[ R"S5 [RUl[RUlg)zTrying to use the efficient libyaml library to encode and decode yaml. If libyaml is not available than we fallback to use the native yaml library, use with caution; it is far less efficient, uses excessive memory, and leaks memory. zpCan't find libyaml so it is not used for YamlCoder, the implementation used is far slower and has a memory leak.N) yaml__with_libyaml__ CSafeDumper _safe_dumper CSafeLoader _safe_loaderloggingwarning SafeDumper SafeLoaderrZs rrYamlCoder.__init__sV **d**d oo EF//d//drcD[R"USSURS9$)zbEncodes a python object into a YAML string. Args: obj: python object. Returns: YAML string. Fzutf-8)default_flow_styleencodingDumper)rudumprxr6s rrYamlCoder.encodes' 99     ""rc>[R"XRS9$)zlDecodes a YAML string to a python object. Args: yaml_string: A YAML string. Returns: A python object. )Loader)ruloadrz)r yaml_strings rrYamlCoder.decodes 99[):): ;;r)rxrzNr r'rrrrrrs9*$ " >r)rrrrrrrrrrrrrrrrr)r!r"r#r$r%rrrpropertyrrrrrr&r'rrrrpsu ! ( 2 22 ># ##=+~   ?rrc0\rSrSrSrSrSrSrSrSr g) rizResult of training a model.c<SUlSUlSUlSUlgr,rtraining_job_metadataerrorrrZs rrTrainingJobResult.__init__s% D"&D DJ#DrcJSHn[X5[X5:wdM g g)NrFTrrs rrTrainingJobResult.__eq__s'%  WQ] *% rc X:X+$r,r'rs rrTrainingJobResult.__ne__rrc/nURR5H%up#URSRX#55 M' SRSR U55$)NrzTrainingJobResult({0})rrrs rrTrainingJobResult.__repr__sQ F '') mmI$$Q*+* # * *499V+< ==r)rrrrN) r!r"r#r$r%rrrrr&r'rrrrs# $>rr)r%rIrr4r{ apache_beambeam six.movesrrugoogle.cloud.ml.utilrrcodersCoderrr)r>rrrrrrprregistryregister_coderrr'rrr s!   *& ))@# !!#<[4t{{  [4|-< !!-<`.1DKK%%.1b' dkk//' T' T[[..' T_?_?D ##$68OP > >F ##$57MNr