Thank you Fabian, I have one followup question.
You wrote: *isUtcTimestamp denotes whether timestamps should be represented asSQL UTC timestamps.* Quetion: So, if *isUtcTimestamp *is set to false, how timestamps are represented? Regards, Krzysztof Chmielewski wt., 25 sty 2022 o 11:56 Fabian Paul <fp...@apache.org> napisał(a): > Hi Krzysztof, > > sorry for the late reply. The community is very busy at the moment > with the final two weeks of Flink 1.15. > > The parameters you have mentioned are mostly relevant for the internal > conversion or representation from Parquet types to Flink's SQL type > system. > > - isUtcTimestamp denotes whether timestamps should be represented as > SQL UTC timestamps > - batchSize is an internal number of how many rows are put into one > vector. Vectors are used internally in Flink SQL for performance > reasons to enable faster execution on batches i.e. for Hive we use the > following default value [1] > - isCaseSensitive is used to map the field/column names from parquet > and match them to columns in Flink > > I have also included @jingsongl...@gmail.com who is more familiar with > the parquet format. > > Best, > Fabian > > [1] > https://github.com/apache/flink/blob/d8a031c2b7d7b73fe38a3f894913d3dcaa5a4111/flink-table/flink-table-common/src/main/java/org/apache/flink/table/data/columnar/vector/VectorizedColumnBatch.java#L46 > > On Mon, Jan 24, 2022 at 4:32 PM Krzysztof Chmielewski > <krzysiek.chmielew...@gmail.com> wrote: > > > > Hi, > > I would like to bump this up a little bit. > > > > The isCaseSensitive is rather clear. If this is false, then column read > in parquet file is case insensitive. > > batchSize - how many records we read from the Parquet file before > passing it to the upper classes right? > > > > Could someone describe what timestamp flab does with some examples? > > > > Regards, > > Krzysztof Chmielewski > > > > > > pon., 10 sty 2022 o 14:59 Krzysztof Chmielewski < > krzysiek.chmielew...@gmail.com> napisał(a): > >> > >> Hi, > >> I would like to ask for some more details regarding three > ParquetColumnarRowInputFormat contruction parameters. > >> > >> The parameters are: > >> batchSize, > >> isUtcTimestamp, > >> isCaseSensitive > >> > >> The parametr names gives some hint about their purpose but there is no > description in docs (java, flink page). > >> > >> Could you provide me some information about the batching process and > other two boolean flags? > >> > >> Regards, > >> Krzysztof Chmielewski >
