Bulk storage of protobuf records in files

2023-05-26 Thread Ryan Skraba via user
Hello all! I discovered while investigating FLINK-32008[1] that we can write to the filesystem connector with the protobuf format, but today, the resulting file is pretty unlikely to be useful or rereadable. There's no real standard for storing many protobuf messages in a single file container, a

Re: [ACCOUNCE] Apache Flink Elasticsearch Connector 3.0.0 released

2022-11-10 Thread Ryan Skraba via user
Excellent news -- welcome to the new era of easier, more timely and more feature-rich releases for everyone! Great job! Ryan On Thu, Nov 10, 2022 at 3:15 PM Leonard Xu wrote: > Thanks Chesnay and Martijn for the great work! I believe the > flink-connector-shared-utils[1] you built will help

Re: Flink support on Avro1.11

2022-08-08 Thread Ryan Skraba via user
Hello! I've updated the Avro version to the most recent release (1.11.1) in the PR mentioned my Martijn[1] -- there's an end-to-end TPC-DS test that has failed. I'm not sure it's related, but I'm taking a look. All my best, Ryan [1]: https://github.com/apache/flink/pull/18133 On Thu, Aug 4, 20