Hi Community,
I am trying to use flink-parquet for reading and writing parquet files from
the Flink filesystem connectors.
In File source, I would be decoding parquet files and converting them to
avro records and similarly in file sink i would be encoding avro records to
parquet files.
For collection i am using
BulkFormat<Object, FileSourceSplit> bulkFormat =
new
StreamFormatAdapter<>(AvroParquetReaders.forSpecificRecord(recordClazz));
FileSource<Object> source = FileSource.forBulkFileFormat(bulkFormat,
path).build();
and for sinking i am using
FileSink<Object> sink = FileSink.forBulkFormat(path,
AvroParquetWriters.forSpecificRecord(recordClazz)).build()
Query: The StreamFormatAdapter class is marked @Internal and,
AvroParquetReaders
and AvroParquetWriters classes are marked @Experimental – does it mean that
in future flink releases these classes can be changed in a non-backward
compatible way like plugging of any other 3PP rather than “parquet-avro” or
changing the API structure thus impacting the application code ?
Would it be safe to use the code as specified above ?
Thanks and Regards
Anuj
