xSrSSKrSSKrSSKr"SS\5r"SS\5r"SS\5r"S S \5r g) zDataflow-related utilities. Nc\rSrSrSrSrg) DecodeErrorzBase decode error.N)__name__ __module__ __qualname____firstlineno____doc____static_attributes__r1lib/third_party/ml_sdk/cloud/ml/util/_decoders.pyrrsr rc\rSrSrSrSrg)PassthroughDecodercU$Nrselfxs rdecodePassthroughDecoder.decodes Hr rN)rrr r rr rr rrrs r rc\rSrSrSrSrSrg) JsonDecoder!z"A decoder for JSON formatted data.c.[R"U5$r)jsonloadsrs rrJsonDecoder.decode$s ::a=r rN)rrr r r rr rr rrr!s *r rc\\rSrSrSr"SS\5r"SS\5rSrSr S r S r S r g ) CsvDecoder(z$A decoder for CSV formatted data. c0\rSrSrSrSrSrSrSrSr g) CsvDecoder._LineGenerator.zCA csv line generator that allows feeding lines to a csv.DictReader.c/Ulgr_linesrs r__init__"CsvDecoder._LineGenerator.__init__1s dkr c`UR(aeURRU5 gr)r(append)rlines r push_line#CsvDecoder._LineGenerator.push_line4s _ kkr cU$rrr)s r__iter__"CsvDecoder._LineGenerator.__iter__9s kr c[UR5nUS:Xa [S5eUS:Xd SU-5eURR5$)Nrz@Columns do not match specified csv headers: empty line was foundzUnexpected number of lines %s)lenr(rpop)r line_lengths rnextCsvDecoder._LineGenerator.next<sX $k   NP P A L>LL [[__ r r'N) rrr r r r*r/r2r9r rr r_LineGeneratorr$.sM  r r;c0\rSrSrSrSrSrSrSrSr g) CsvDecoder._ReaderWrapperJz?A wrapper for csv.reader / csv.DictReader to make it picklable.cXX4U4UlXlU(a%[R"X[ U5US9Ulg[R "U[ U5US9Ulg)N) delimiterskipinitialspace)_state_line_generatorcsv DictReaderstr_readerreader)rline_generator column_namesr@decode_to_dictskip_initial_spaces rr*"CsvDecoder._ReaderWrapper.__init__MsV#9')dk+ ~~ C N/1 zz.C N3EG r clURRU5 URR5$r)rCr/rGr9rs r read_record%CsvDecoder._ReaderWrapper.read_recordZs) $$Q' \\    r cUR$r)rBr)s r __getstate__&CsvDecoder._ReaderWrapper.__getstate__^s [[r c"UR"U6 gr)r*)rstates r __setstate__&CsvDecoder._ReaderWrapper.__setstate__as mmUr )rCrGrBN) rrr r r r*rOrRrVr rr r_ReaderWrapperr=JsI G!r rXcXl[U5UlURUR 5XUU5UlX@lXPlg)aQInitializer. Args: column_names: Tuple of strings. Order must match the order in the file. numeric_column_names: Tuple of strings. Contains column names that are numeric. Every name in numeric_column_names must also be in column_names. delimiter: String used to separate fields. decode_to_dict: Boolean indicating whether the docoder should generate a dictionary instead of a raw sequence. True by default. fail_on_error: Whether to fail if a corrupt row is found. skip_initial_space: When True, whitespace immediately following the delimiter is ignored. N) _column_namesset_numeric_column_namesrXr;rG_decode_to_dict_fail_on_error)rrJnumeric_column_namesr@rK fail_on_errorrLs rr*CsvDecoder.__init__dsK"&!$%9!:D&& |DL*'r cjUR(a [U5e[R"SU5 g)aFHandle corrupt rows. Depending on whether the decoder is configured to fail on error it will raise a DecodeError or return None. Args: message: String, the error message to raise. Returns: None, when the decoder is not configured to fail on error. Raises: DecodeError: when the decoder is configured to fail on error. zDiscarding invalid row: %sN)r^rloggingwarning)rmessages r_handle_corrupt_rowCsvDecoder._handle_corrupt_row}s,    oo2G< r ctU(aUR5(dgXR;a [U5$U$r)stripr\float)r column_namevalues r _get_valueCsvDecoder._get_values/  000 5\ Lr cxURRU5n[ U5[ UR 5:wa#URSUR <SU<35$UR (acXR Sc#URSUR <SU<35$UR5Hup4URX45X'M U$[UR 5HupSXnURX45X'M U$![a"nURU<SU<35sSnA$SnAff=f)aTDecodes the given string. Args: record: String to be decoded. Returns: Serialized object corresponding to decoded string. Or None if there's an error and the decoder is configured not to fail on error. Raises: DecodeError: If columns do not match specified csv headers. ValueError: If some numeric column has non-numeric data. z: Nz,Columns do not match specified csv headers: z -> ) rGrO Exceptionrfr6rZr] iteritemsrm enumerate)rrecordenamerlindexs rrCsvDecoder.decodes(>||''/f 6{c$,,--  % %  & *++  ""2& ' /''""F ,- - ))++$t3 , M#4#5#56+% 4 7 M- >  % %!V&< ==>sD D9D4.D94D9)rZr]r^r\rGN) rrr r r objectr;rXr*rfrmrr rr rr!r!(s3 v8v4(2(&r r!) r rDrrcrqrryrrr!rr rrzsG )   &[[r