Hi,
We are seeing an issue with Flink on our production. The version is 1.7
which we use.
We started seeing sudden lag on kafka, and the consumers were no longer
working/accepting messages. On trying to enable debug mode, the below
errors were seen
[image: image.jpeg]
I am not sure why this occur
Dawid Wysakowicz created FLINK-19339:
Summary: Support Avro's unions with logical types
Key: FLINK-19339
URL: https://issues.apache.org/jira/browse/FLINK-19339
Project: Flink
Issue Type:
It seems my questions are misunderstood to be about details on
implementations, but actually my concerns are from the users' view,
especially how to understand the new APIs and how to choose/use them.
To better express my concern, let's list the new APIs this FLIP proposes to
introduce:
*env.get
Arvid Heise created FLINK-19338:
---
Summary: New source interface cannot unregister unregistered source
Key: FLINK-19338
URL: https://issues.apache.org/jira/browse/FLINK-19338
Project: Flink
Issu
Hi Husky,
Module is a mechanism to support built-in functions which should always be
in the classpath.
So I'm afraid it may conflict with the current mechanism to support dynamic
loading for modules.
IIUC, what you want is the `CREATE FUNCTION ... USING JAR` which is
discussed in FLINK-14055 [1].
Aljoscha/Guowei,
I think we are pretty close with aligning on the Iceberg sink requirements.
This new sink API can really benefit and simplify Iceberg sink
implementation. Looking forward to the initial scope with 1.12 release.
> CommitResult commit(GlobalCommittableT);
I like the CommitResult
sunjincheng created FLINK-19337:
---
Summary: Make a little bit improvement for PyFlink package
structure and class name.
Key: FLINK-19337
URL: https://issues.apache.org/jira/browse/FLINK-19337
Project: Fl
Benchao Li created FLINK-19336:
--
Summary: EncodingUtils#encodeObjectToString should propagate inner
exception
Key: FLINK-19336
URL: https://issues.apache.org/jira/browse/FLINK-19336
Project: Flink
Husky Zeng created FLINK-19335:
--
Summary: Automatically get udf's resource files from hdfs when
running a job that uses hive-udf
Key: FLINK-19335
URL: https://issues.apache.org/jira/browse/FLINK-19335
P
hailong wang created FLINK-19334:
Summary: Translate page 'Streaming File Sink' into Chinese
Key: FLINK-19334
URL: https://issues.apache.org/jira/browse/FLINK-19334
Project: Flink
Issue Type:
http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Is-there-a-way-to-avoid-submit-hive-udf-s-resources-when-we-submit-a-job-td38204.html
As the mail say，I plan to resolve this problem in my production environment
, is there any suggestion ？
--
Sent from: http://apache-flink-mai
Huang Xingbo created FLINK-19333:
Summary: Introduce
BatchArrowPythonOverWindowAggregateFunctionOperator
Key: FLINK-19333
URL: https://issues.apache.org/jira/browse/FLINK-19333
Project: Flink
Zekun Yu created FLINK-19332:
Summary: Special characters issue using Kinesis Data Analytics for
Apache Flink
Key: FLINK-19332
URL: https://issues.apache.org/jira/browse/FLINK-19332
Project: Flink
Seth Wiesman created FLINK-19331:
Summary: State processor api has native resouce leak when working
with RocksDB
Key: FLINK-19331
URL: https://issues.apache.org/jira/browse/FLINK-19331
Project: Flink
Hi everyone,
This vote is canceled. We have found few blocking
issues [FLINK-19330], [FLINK-19327]
a new RC would be out tomorrow.
Thanks,
Igal.
On Mon, Sep 21, 2020 at 8:56 AM Tzu-Li (Gordon) Tai
wrote:
> Hi everyone,
>
> Please review and vote on the release candidate #1 for the version 2.2.0
Igal Shilman created FLINK-19330:
Summary: Recovery with async operations fails due to unitialized
runtimeContext
Key: FLINK-19330
URL: https://issues.apache.org/jira/browse/FLINK-19330
Project: Flink
To me, the simplifications made by Seth sound good and do make a lot of
sense.
We should really break this down to a few orthogonal guides, then it is
easy for users:
- Metadata always goes through the JobManager, no matter what
CheckpointStorage.
- The JobManagerCheckpointStorage has the opti
Hi all,
I'll try and summarize my thoughts after Guowei, Yun, Kostas, Dawid, and
me had an offline discussion about this.
Also, I would like to give credit to Guowei for initially coming up with
the idea of a topology sink in the context of this discussion. I think
it's a good idea and we sh
Igal Shilman created FLINK-19329:
Summary: FunctionGroupOperator#dispose might throw an NPE
Key: FLINK-19329
URL: https://issues.apache.org/jira/browse/FLINK-19329
Project: Flink
Issue Type:
Hi Yu,
Let me address your comments one at a time.
I think I can address comments one and two with a single answer. This FLIP
does not change any runtime data structures or implementations. As such, it
only provides new user-facing factory classes for those components.
StateBackend (the interface
We use a Cloudera 6.3 cluster in prod. I'd guess that it's still widely used in
prod as those cloudera upgrades for major versions are planned long time ahead
and take a significant amount of resources in big data lakes.
On that 6.3. cluster, if I open spark-shell, I still see scala 2.11 in use
cellen created FLINK-19328:
--
Summary: flink sql
Key: FLINK-19328
URL: https://issues.apache.org/jira/browse/FLINK-19328
Project: Flink
Issue Type: Bug
Components: Table SQL / API
Affec
Igal Shilman created FLINK-19327:
Summary: Helm charts specify a too low value for the job manager's
heap size
Key: FLINK-19327
URL: https://issues.apache.org/jira/browse/FLINK-19327
Project: Flink
Aljoscha Krettek created FLINK-19326:
Summary: Allow explicitly configuring time behaviour on CEP
PatternStream
Key: FLINK-19326
URL: https://issues.apache.org/jira/browse/FLINK-19326
Project: Fli
Congxian Qiu(klion26) created FLINK-19325:
-
Summary: Optimize the consumed time for checkpoint completion
Key: FLINK-19325
URL: https://issues.apache.org/jira/browse/FLINK-19325
Project: Flink
Xintong Song created FLINK-19324:
Summary: Map requested/allocated containers with priority on YARN
Key: FLINK-19324
URL: https://issues.apache.org/jira/browse/FLINK-19324
Project: Flink
Issu
Yingjie Cao created FLINK-19323:
---
Summary: Small optimization of network layer record serialization
Key: FLINK-19323
URL: https://issues.apache.org/jira/browse/FLINK-19323
Project: Flink
Issue
+1
I could see this is a well written document after a long and thorough
discussion [1]. Thanks for driving this all along, Becket!
Best Regards,
Yu
[1]
http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/DISCUSS-FLIP-33-Standardize-connector-metrics-td26869.html
On Mon, 21 Sep 2020
+1， Thanks for driving this, Becket!
Best,
Jincheng
Becket Qin 于2020年9月21日周一 上午11:31写道：
> Hi all,
>
> I would like to start the voting thread for FLIP-33 which proposes to
> standardize the metrics of Flink connectors.
>
> In short, we would like to provide a convention and guidance of Flink
>
+1
On Mon, Sep 21, 2020 at 11:31 AM Becket Qin wrote:
> Hi all,
>
> I would like to start the voting thread for FLIP-33 which proposes to
> standardize the metrics of Flink connectors.
>
> In short, we would like to provide a convention and guidance of Flink
> connector metrics. It will help sim
YufeiLiu created FLINK-19322:
Summary: Specified parallelism in create table operation
Key: FLINK-19322
URL: https://issues.apache.org/jira/browse/FLINK-19322
Project: Flink
Issue Type: Improveme
Fabian Hueske created FLINK-19321:
-
Summary: CollectSinkFunction does not define serialVersionUID
Key: FLINK-19321
URL: https://issues.apache.org/jira/browse/FLINK-19321
Project: Flink
Issue
Yingjie Cao created FLINK-19320:
---
Summary: Remove clearBuffers from the public interfaces of
RecordWriter
Key: FLINK-19320
URL: https://issues.apache.org/jira/browse/FLINK-19320
Project: Flink
Aljoscha Krettek created FLINK-19319:
Summary: Deprecate
StreamExecutionEnvironment.setStreamTimeCharacteristic()
Key: FLINK-19319
URL: https://issues.apache.org/jira/browse/FLINK-19319
Project: F
Aljoscha Krettek created FLINK-19318:
Summary: Deprecate timeWindow() operations in DataStream API
Key: FLINK-19318
URL: https://issues.apache.org/jira/browse/FLINK-19318
Project: Flink
I
Aljoscha Krettek created FLINK-19317:
Summary: Make EventTime the default StreamTimeCharacteristic
Key: FLINK-19317
URL: https://issues.apache.org/jira/browse/FLINK-19317
Project: Flink
I
Aljoscha Krettek created FLINK-19316:
Summary: FLIP-134: Batch execution for the DataStream API
Key: FLINK-19316
URL: https://issues.apache.org/jira/browse/FLINK-19316
Project: Flink
Issu
Chesnay Schepler created FLINK-19315:
Summary: Move slot-bookkeeping into separate component
Key: FLINK-19315
URL: https://issues.apache.org/jira/browse/FLINK-19315
Project: Flink
Issue T
Chesnay Schepler created FLINK-19314:
Summary: Add compatibility layer
Key: FLINK-19314
URL: https://issues.apache.org/jira/browse/FLINK-19314
Project: Flink
Issue Type: Task
Co
Chesnay Schepler created FLINK-19313:
Summary: Implement declarative SlotPool
Key: FLINK-19313
URL: https://issues.apache.org/jira/browse/FLINK-19313
Project: Flink
Issue Type: Sub-task
Yingjie Cao created FLINK-19312:
---
Summary: Introduce BufferWritingResultPartition which wraps the
logic of writing buffers to ResultSubpartition
Key: FLINK-19312
URL: https://issues.apache.org/jira/browse/FLINK-1931
Chesnay Schepler created FLINK-19311:
Summary: Add ResourceRequirement(s)
Key: FLINK-19311
URL: https://issues.apache.org/jira/browse/FLINK-19311
Project: Flink
Issue Type: Sub-task
Chesnay Schepler created FLINK-19310:
Summary: Add new TaskManagerSlot implementation
Key: FLINK-19310
URL: https://issues.apache.org/jira/browse/FLINK-19310
Project: Flink
Issue Type: Ta
Chesnay Schepler created FLINK-19309:
Summary: Add TaskExecutorAllocator
Key: FLINK-19309
URL: https://issues.apache.org/jira/browse/FLINK-19309
Project: Flink
Issue Type: Task
Chesnay Schepler created FLINK-19308:
Summary: Add SlotTracker
Key: FLINK-19308
URL: https://issues.apache.org/jira/browse/FLINK-19308
Project: Flink
Issue Type: Task
Components
Hi Guowei,
> I believe that we could support such an async sink writer
> very easily in the future. What do you think?
How would you see the expansion in the future? Do you mean just adding
`isAvailable()` method with a default implementation later on?
Piotrek
pon., 21 wrz 2020 o 02:39 Steven W
Chesnay Schepler created FLINK-19307:
Summary: Add RequirementsTracker
Key: FLINK-19307
URL: https://issues.apache.org/jira/browse/FLINK-19307
Project: Flink
Issue Type: Task
Co
Chesnay Schepler created FLINK-19306:
Summary: Implement DeclarativeSlotManager
Key: FLINK-19306
URL: https://issues.apache.org/jira/browse/FLINK-19306
Project: Flink
Issue Type: Sub-task
Chesnay Schepler created FLINK-19305:
Summary: Setup Azure build for running core and flink-tests
Key: FLINK-19305
URL: https://issues.apache.org/jira/browse/FLINK-19305
Project: Flink
Is
Chesnay Schepler created FLINK-19304:
Summary: Add feature toggle
Key: FLINK-19304
URL: https://issues.apache.org/jira/browse/FLINK-19304
Project: Flink
Issue Type: Sub-task
R
FLIP-138 has been accepted with 3 binding votes and no disapproving votes:
- zhu zhu
- xintong
- chesnay
On 9/17/2020 10:59 AM, Chesnay Schepler wrote:
+1
On 9/4/2020 9:40 AM, Zhu Zhu wrote:
+1
Thanks,
Zhu
Xintong Song 于2020年9月4日周五 下午1:26写道：
Thanks for starting this vote.
+1 from my si
Thanks for the update Seth, and let me further clarify my comments /
concerns around the new `CheckpointStorage`.
1. In the existing `MemoryStateBackend`, there's a `maxStateSize` field
which limits the maximal state size sent to JM from one single memory
backend, with the default size of 5MB. Ple
Hi all,
The voting time for FLIP-134 [1] has passed. I'm closing the vote now.
Including my implicit vote, there were 7 + 1 votes, 4 of which are binding:
- Dawid Wysakowicz (binding)
- Gao Yun
- Ma Guowei
- David Anderson (binding)
- Kostas Kloudas (binding)
- Peter Huang
- Aljoscha Krettek (b
Hi Jark,
Thank you for your e-mail. I agree, let's engage all interested parties in this
discussion - I'm writing this e-mail to both Flink and Calcite dev mailing
lists.
I'll repeat myself to present the proposal to the Calcite community.
I would like to propose an enrichment of existing Flin
Juha Mynttinen created FLINK-19303:
--
Summary: Disable WAL in RocksDB recovery
Key: FLINK-19303
URL: https://issues.apache.org/jira/browse/FLINK-19303
Project: Flink
Issue Type: Improvement
55 matches
Mail list logo
