Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/6405
cc @GJL @StephanEwen
---
GitHub user azagrebin opened a pull request:
https://github.com/apache/flink/pull/6405
[FLINK-8439] Add Flink shading to AWS credential provider s3 hadoop câ¦
## What is the purpose of the change
This PR refactors S3 Hadoop and Presto file systems and adds Flink shading
GitHub user azagrebin opened a pull request:
https://github.com/apache/flink/pull/6379
[FLINK-9637] Add public user documentation for state TTL feature
## What is the purpose of the change
Public user documentation for first iteration of state TTL feature
## Brief
GitHub user azagrebin opened a pull request:
https://github.com/apache/flink/pull/6361
[FLINK-9858] [tests] State TTL End-to-End Test
## What is the purpose of the change
This PR introduces end to end test for state TTL feature, heap and rocksdb
backends.
## Brief
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6313#discussion_r202081209
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/heap/HeapKeyedStateBackend.java
---
@@ -240,7 +243,7 @@ private boolean
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6313#discussion_r202041863
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlMapState.java
---
@@ -84,52 +86,89 @@ public boolean contains(UK key) throws
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6313#discussion_r202040710
--- Diff:
flink-core/src/main/java/org/apache/flink/api/common/state/StateDescriptor.java
---
@@ -92,6 +93,10 @@
@Nullable
private String
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6308#discussion_r201798531
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/OperatorBackendSerializationProxy.java
---
@@ -62,62 +67,62 @@ public int getVersion
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6308#discussion_r202035286
--- Diff:
flink-runtime/src/test/java/org/apache/flink/runtime/state/SerializationProxiesTest.java
---
@@ -126,11 +131,11 @@ public void
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6308#discussion_r202033136
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/metainfo/StateMetaInfoSnapshotReadersWriters.java
---
@@ -0,0 +1,412
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/6186
Hi @Aitozi, the main issue FLINK-9510 contains a design doc for this effort
with roadmap
---
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r201941972
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlListState.java
---
@@ -0,0 +1,171 @@
+/*
+ * Licensed to the Apache
GitHub user azagrebin opened a pull request:
https://github.com/apache/flink/pull/6313
[FLINK-9701] Add TTL in state descriptors
## What is the purpose of the change
This PR activates TTL feature in state descriptors.
## Brief change log
- add method
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/6277
LGTM
cc @StefanRRichter
---
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6277#discussion_r201272235
--- Diff:
flink-core/src/main/java/org/apache/flink/api/common/state/StateTtlConfiguration.java
---
@@ -93,4 +97,82 @@ public Time getTtl
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6277#discussion_r201257564
--- Diff:
flink-core/src/main/java/org/apache/flink/api/common/state/StateTtlConfiguration.java
---
@@ -93,4 +97,82 @@ public Time getTtl
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6277#discussion_r200950760
--- Diff:
flink-core/src/main/java/org/apache/flink/api/common/state/StateTtlConfiguration.java
---
@@ -93,4 +101,59 @@ public Time getTtl
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6277#discussion_r200974407
--- Diff:
flink-core/src/main/java/org/apache/flink/api/common/state/StateTtlConfiguration.java
---
@@ -93,4 +101,59 @@ public Time getTtl
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6277#discussion_r200949235
--- Diff:
flink-core/src/main/java/org/apache/flink/api/common/state/StateTtlConfiguration.java
---
@@ -61,7 +61,15 @@
private final
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6228#discussion_r199567018
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/heap/HeapPriorityQueue.java
---
@@ -0,0 +1,341 @@
+/*
+ * Licensed to the
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6228#discussion_r199567167
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/heap/HeapPriorityQueue.java
---
@@ -0,0 +1,341 @@
+/*
+ * Licensed to the
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6228#discussion_r199806792
--- Diff:
flink-runtime/src/test/java/org/apache/flink/runtime/state/InternalPriorityQueueTestBase.java
---
@@ -0,0 +1,452 @@
+/*
+ * Licensed to
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6228#discussion_r199567240
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/heap/KeyGroupPartitionedPriorityQueue.java
---
@@ -0,0 +1,280
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6228#discussion_r199567286
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/heap/KeyGroupPartitionedPriorityQueue.java
---
@@ -0,0 +1,280
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6228#discussion_r199567420
--- Diff:
flink-state-backends/flink-statebackend-rocksdb/src/main/java/org/apache/flink/contrib/streaming/state/RocksDBOrderedSetStore.java
---
@@ -0,0
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6228#discussion_r199806702
--- Diff:
flink-runtime/src/test/java/org/apache/flink/runtime/state/InternalPriorityQueueTestBase.java
---
@@ -0,0 +1,452 @@
+/*
+ * Licensed to
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6228#discussion_r199806862
--- Diff:
flink-runtime/src/test/java/org/apache/flink/runtime/state/InternalPriorityQueueTestBase.java
---
@@ -0,0 +1,452 @@
+/*
+ * Licensed to
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6228#discussion_r199807071
--- Diff:
flink-runtime/src/test/java/org/apache/flink/runtime/state/InternalPriorityQueueTestBase.java
---
@@ -0,0 +1,452 @@
+/*
+ * Licensed to
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6228#discussion_r199567205
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/heap/HeapPriorityQueueSet.java
---
@@ -0,0 +1,186 @@
+/*
+ * Licensed to the
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6228#discussion_r199473489
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/InternalPriorityQueue.java
---
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6228#discussion_r199806903
--- Diff:
flink-runtime/src/test/java/org/apache/flink/runtime/state/InternalPriorityQueueTestBase.java
---
@@ -0,0 +1,452 @@
+/*
+ * Licensed to
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6228#discussion_r199566880
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/KeyExtractorFunction.java
---
@@ -0,0 +1,36 @@
+/*
+ * Licensed to the
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6228#discussion_r199566980
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/heap/HeapPriorityQueue.java
---
@@ -0,0 +1,341 @@
+/*
+ * Licensed to the
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6228#discussion_r199806997
--- Diff:
flink-runtime/src/test/java/org/apache/flink/runtime/state/InternalPriorityQueueTestBase.java
---
@@ -0,0 +1,452 @@
+/*
+ * Licensed to
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6228#discussion_r199806942
--- Diff:
flink-runtime/src/test/java/org/apache/flink/runtime/state/InternalPriorityQueueTestBase.java
---
@@ -0,0 +1,452 @@
+/*
+ * Licensed to
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/6196
@StefanRRichter
I added more precomputed fields to `CompositeSerializer` constructor and
included `TtlStateFactory` in TTL tests.
---
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/6196
Regarding CI, seems to be unrelated test
`YARNSessionCapacitySchedulerITCase`, [passed in my
CI](https://travis-ci.org/azagrebin/flink/builds/398352328)
---
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6196#discussion_r199231639
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlStateFactory.java
---
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/6196
@StefanRRichter @sihuazhou
Thanks guys for the helpful review.
I refactored the `CompositeSerializer` to rely rather on loops than streams
and added tests for it.
Please, have a look
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6196#discussion_r199228508
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlStateFactory.java
---
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6196#discussion_r199227970
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlStateFactory.java
---
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6196#discussion_r199227770
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlStateFactory.java
---
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6196#discussion_r199226982
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlStateFactory.java
---
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6196#discussion_r199226833
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlStateFactory.java
---
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/5920
@trionesadam
is there still any interest in working on this PR or it can be closed?
---
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r199082008
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlListState.java
---
@@ -0,0 +1,172 @@
+/*
+ * Licensed to the Apache
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r198947650
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlFoldFunction.java
---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r198757244
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlFoldFunction.java
---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r198751723
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlMapState.java
---
@@ -0,0 +1,132 @@
+/*
+ * Licensed to the Apache
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r198439894
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/AbstractTtlDecorator.java
---
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r198439148
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlStateVisibility.java
---
@@ -0,0 +1,29 @@
+/*
+ * Licensed to the
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r198436490
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlUpdateType.java
---
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r198431042
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/AbstractTtlDecorator.java
---
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r198430741
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlConfig.java
---
@@ -0,0 +1,66 @@
+/*
+ * Licensed to the Apache
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6196#discussion_r197498439
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlStateFactory.java
---
@@ -0,0 +1,207 @@
+/*
+ * Licensed to the
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/6202
cc @tillrohrmann
---
GitHub user azagrebin opened a pull request:
https://github.com/apache/flink/pull/6202
[FLINK-9493] Forward cause when releasing a TaskManager at the SlotPool
## What is the purpose of the change
This PR is to propagate original cause from components calling
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/6186
Thanks for review @sihuazhou, I think now all concerns should have been
addressed.
CI failure is unrelated, works in [my CI for the same
commit](https://travis-ci.org/azagrebin/flink/builds
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r197179238
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlValue.java ---
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache
GitHub user azagrebin opened a pull request:
https://github.com/apache/flink/pull/6196
[FLINK-9513] Implement TTL state wrappers factory and serializer for value
with TTL
## What is the purpose of the change
This PR introduces a state factory for wrapping state objects
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r197110068
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlListState.java
---
@@ -0,0 +1,171 @@
+/*
+ * Licensed to the Apache
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r197109577
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/AbstractTtlDecorator.java
---
@@ -0,0 +1,98 @@
+/*
+ * Licensed to the
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r197108498
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlListState.java
---
@@ -0,0 +1,171 @@
+/*
+ * Licensed to the Apache
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r197102118
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlListState.java
---
@@ -0,0 +1,171 @@
+/*
+ * Licensed to the Apache
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r197079917
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlValue.java ---
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r197077377
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlListState.java
---
@@ -0,0 +1,171 @@
+/*
+ * Licensed to the Apache
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r196826339
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/TtlListState.java
---
@@ -0,0 +1,171 @@
+/*
+ * Licensed to the Apache
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6186#discussion_r196819320
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/ttl/AbstractTtlDecorator.java
---
@@ -0,0 +1,95 @@
+/*
+ * Licensed to the
GitHub user azagrebin opened a pull request:
https://github.com/apache/flink/pull/6186
[FLINK-9514,FLINK-9515,FLINK-9516] State ttl wrappers
## What is the purpose of the change
This PR introduces TTL logic wrappers for state objects.
## Brief change log
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/6186
cc @StefanRRichter
---
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/6173
Thanks @sihuazhou! I have added changes to address the comments
---
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/6173
cc @StefanRRichter @aljoscha
---
GitHub user azagrebin opened a pull request:
https://github.com/apache/flink/pull/6173
[FLINK-9571] Refactor StateBinder to internal backend specific state
factories
## What is the purpose of the change
Remove StateBinder and StateDescriptor.bind method from State user API
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/6169
cc @tillrohrmann
---
GitHub user azagrebin opened a pull request:
https://github.com/apache/flink/pull/6169
[FLINK-8573] Extend ProgramInvocationException message with Job ID where
applicable
## What is the purpose of the change
Print job id when program execution fails where it is possible
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6159#discussion_r195443441
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/AbstractKeyGroupPartitionedSnapshot.java
---
@@ -0,0 +1,76
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/6159#discussion_r195441098
--- Diff:
flink-runtime/src/main/java/org/apache/flink/runtime/state/AbstractKeyGroupPartitionedSnapshot.java
---
@@ -0,0 +1,76
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/6156
Thanks for review, @sihuazhou! I added fixing commit to address your
comments.
---
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/6156
cc @StefanRRichter
---
GitHub user azagrebin opened a pull request:
https://github.com/apache/flink/pull/6156
[FLINK-9572] Extend InternalAppendingState with internal stored state access
## What is the purpose of the change
Extend InternalAppendingState with get and update methods for internal
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/5421
@zhangminglei
what do you think about creation one more `ProgramInvocationException`
overloaded constructor with JobId as parameter? It could always append to the
message like `MESSAGE
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/6024
cc @zentol
---
GitHub user azagrebin opened a pull request:
https://github.com/apache/flink/pull/6024
[FLINK-9301] [e2e test] Add back "not co mini cluster" test with reduced
parallelism
â¦ced to 25 parallelism and memory usage logging
## What is the purpose of
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/5947
agree, I added constant check of previous non-null state into its update
method
---
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/5947
Thanks for review and good points @StefanRRichter
I updated the PR to address the comments.
The resume state e2e test also checks operator <-> state correspondence
upon state resto
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/5947
cc @StefanRRichter
---
GitHub user azagrebin opened a pull request:
https://github.com/apache/flink/pull/5947
[FLINK-8978] Stateful generic stream job upgrade e2e test
## What is the purpose of the change
e2e test for generic state job upgrade and state recovery based operator
uid
Github user azagrebin commented on the issue:
https://github.com/apache/flink/pull/5921
cc @StefanRRichter, @aljoscha
---
GitHub user azagrebin opened a pull request:
https://github.com/apache/flink/pull/5921
[FLINK-9254] Move NotSoMiniClusterIterations to be an end-to-end test
## What is the purpose of the change
This PR moves `NotSoMiniClusterIterations` to nightly e2e tests
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/5912#discussion_r184043414
--- Diff:
flink-streaming-java/src/test/java/org/apache/flink/streaming/runtime/tasks/StreamTaskTest.java
---
@@ -178,7 +172,7
Github user azagrebin commented on a diff in the pull request:
https://github.com/apache/flink/pull/5912#discussion_r184042695
--- Diff: pom.xml ---
@@ -452,7 +452,7 @@ under the License.
org.javassist
GitHub user azagrebin opened a pull request:
https://github.com/apache/flink/pull/5912
[FLINK-9041] Refactor StreamTaskTest to not use scala and akka
## What is the purpose of the change
Get rid of scala/akka dependency in StreamTaskTest
## Brief change log
GitHub user azagrebin opened a pull request:
https://github.com/apache/flink/pull/5906
[FLINK-9213] [REST] [docs] Revert checkpoint details URL from
'/jobs/:jobid/checkpoints/:checkpointid' to
'/jobs/:jobid/checkpoints/details/:checkpointid' as it was in 1.5
##
93 matches
Mail list logo
