Hello,
My situation is following:
* I write data in ORC format by Flink into HDFS:
* I implements Vectorizer interface for processing my data and converting
it into VectorizedRowBatch
* I create OrcBulkWriter:
OrcBulkWriterFactory orcBulkWriterFactory = new
OrcBulkWriterFactory<>(
Hi Yang,
(redirecting this to user mailing list as this is not a dev question)
I am not sure why the state loading is stuck after enabling the compaction
filter
but the background cleanup of RocksDB state with TTL will not work without
activating the filter.
This happens on RocksDB opening in Fli
For cross-referencing, here is the SO thread[1]. Unfortunately, I don't
have a good answer for you, except try to align the ORC versions somehow.
[1]
https://stackoverflow.com/questions/65126074/orc-files-generated-by-flink-can-not-be-used-in-hive
On Fri, Dec 4, 2020 at 9:00 AM Сергей Чернов wro
Let me try this out on my standalone Hive. I remember reading something
similar on SO[1]. In this case, it was an external ORC generated by Spark
and an external table was created using CDH. The OP answered referring to a
community post[2] on Cloudera. It may be worth checking.
[1]
https://stackov
Hi everyone!
We'll be hosting a meetup on the upcoming Apache Flink 1.12 release on
December 9.
Join the Ask-Me-Anything (AMA) session with Aljoscha Krettek, Timo Walther,
Stephan Ewen, Arvid Heise and Robert Metzger.
The AMA will be live-streamed on Youtube (link TBA in the meetup groups
below).
Hi Jafee,
Can u please help me out with the sample code how you have written the
custom sink and how you using this broadcast pattern to update schema at
run time. It will help me.
On Sat, Oct 17, 2020 at 1:55 PM Piotr Nowojski wrote:
> Hi Julian,
>
> Glad to hear it worked! And thanks for comi
Hey AJ,
Depending on your control messages and what you’re trying to accomplish you can
simplify the application even further by stripping out the second broadcast and
letting operator chaining guarantee that control messages flow appropriately.
This results in
___
Hello,
I'm tuning flink for parallelism right now and when I look at the
JobManager I see
*taskmanager.cpu.cores* 1.7976931348623157E308
Which looks like the maximum double number.
We have 8 cpu cores, so we figured we'd bump to 16 for hyper threading. We
have 37 operators so we rounded up and se
We're running this in a local environment so that may be contributing to
what we're seeing.
On Fri, Dec 4, 2020 at 10:41 PM Rex Fenley wrote:
> Hello,
>
> I'm tuning flink for parallelism right now and when I look at the
> JobManager I see
> *taskmanager.cpu.cores* 1.7976931348623157E308
> Which