It seems that we have done that (?). The
BatchTableEnvironment.registerTableSource(name, tableSource) only accept a
BatchTableSource. In contrast, the
StreamTableEnvironment.registerTableSource(name, tableSource) only accept a
StreamTableSource. So that, if a TableSource implements both batch
Hi Shen,
What you've read in the documentation is correct. The managed memory is
only used by the batch operators.
For streaming, we have different state backends with different
characteristics.
The FsStateBackend keeps the state (including window contents) on the heap
and makes backups to the (di
Ted Yu created FLINK-4531:
-
Summary: Thread.stop() is deprecated
Key: FLINK-4531
URL: https://issues.apache.org/jira/browse/FLINK-4531
Project: Flink
Issue Type: Bug
Reporter: Ted Yu
> Will work log updates still lead to notifications on the mailing list?
I don't know but it is most likely configurable. I would prefer if we
didn't get notifications for work log items because these would again
be duplicates of the GitHub notifications.
On Mon, Aug 29, 2016 at 5:39 PM, Neelesh
This limitation doesn't exist anymore in the latest master. Jobs may
be monitored for infinite amount of time now. Note that it wouldn't
cancel the job if the submission timeout had been reached before the
job completed.
On Mon, Aug 29, 2016 at 6:06 PM, Till Rohrmann wrote:
> If I'm not mistaken
If I'm not mistaken then this was a limitation of Akka's ask futures. The
maximum timeout was as specified. It might have changed with a new version,
though. Usually, you should rarely run into a case where this actually
matters.
Cheers,
Till
On Mon, Aug 29, 2016 at 5:44 PM, Greg Hogan wrote:
>
AkkaUtils.scala
val INF_TIMEOUT = 21474835 seconds
That is job submit timeout 248.55 days.
Why is this number?
Hi,
I have a question regarding memory management in the Streaming mode. The
documents state that the memory management module is only used in the Batch
mode, and Flink Streaming directly operates on JVM heap. Then, what if the
volume of data in a window becomes too large to fit in the heap? Will
Thanks Max for coordinating the effort.
+1 as well.
On Mon, Aug 29, 2016 at 6:57 AM, Robert Metzger wrote:
> +1
>
> I didn't know that we can put the comments into the work log.
> Will work log updates still lead to notifications on the mailing list?
>
> On Mon, Aug 29, 2016 at 11:52 AM, Maximi
Could be rewritten as "val INFO_TIMEOUT = Integer.MAX_VALUE seconds"?
On Mon, Aug 29, 2016 at 4:22 AM, 时金魁 wrote:
>
>
> AkkaUtils.scala
> val INF_TIMEOUT = 21474835 seconds
>
>
> That is job submit timeout 248.55 days.
>
>
> Why is this number?
>
>
>
>
Till Rohrmann created FLINK-4530:
Summary: Generalize TaskExecutorToResourceManagerConnection to be
reusable
Key: FLINK-4530
URL: https://issues.apache.org/jira/browse/FLINK-4530
Project: Flink
Till Rohrmann created FLINK-4529:
Summary: Move JobMaster, TaskExecutor and ResourceManager out of
the rpc package
Key: FLINK-4529
URL: https://issues.apache.org/jira/browse/FLINK-4529
Project: Flink
Hi,
I am running Flink on a cluster of 5 nodes.
Here is my config:
*taskmanager.numberOfTaskSlots: 1parallelism.default: 1*
My Flink dashboard shows the following:
*Task Managers: 5*
*Task Slots: 5*
*Available Task Slots: 5*
I have the following questions:
1. Why does a job with 8 tasks
+1
I didn't know that we can put the comments into the work log.
Will work log updates still lead to notifications on the mailing list?
On Mon, Aug 29, 2016 at 11:52 AM, Maximilian Michels wrote:
> From what I understand so far, the message mirroring can be adjusted
> in the follow parts:
>
> 1
Till Rohrmann created FLINK-4528:
Summary: Mark runAsync and scheduleAsync methods protected
Key: FLINK-4528
URL: https://issues.apache.org/jira/browse/FLINK-4528
Project: Flink
Issue Type: S
Stephan Ewen created FLINK-4527:
---
Summary: Drop the "flinkAccumulators" from the Execution
Key: FLINK-4527
URL: https://issues.apache.org/jira/browse/FLINK-4527
Project: Flink
Issue Type: Impro
Maximilian Michels created FLINK-4526:
-
Summary: ApplicationClient: remove redundant proxy messages
Key: FLINK-4526
URL: https://issues.apache.org/jira/browse/FLINK-4526
Project: Flink
Is
At first glance, I thought we are losing the possibility to distingish
between choosing a batch or streaming table if a TableSource implements
both. Because currently you are using a StreamTableSource as default if
a TableSource implements both types. I think it would be better to
determine bat
Hi Timo,
Yes, you are right. The STREAM keyword is invalid now. If there is a STREAM
keyword in the query, the parser will throw "can’t convert table xxx to stream"
exception. Because we register the table as a regular table not streamable.
- Jark Wu
> 在 2016年8月29日，下午8:13，Timo Walther 写道：
>
Hi Jark,
your code looks good and it also simplifies many parts. So the STREAM
keyword is not optional but invalid now, right? What happens if there is
keyword in the query?
Timo
Am 29/08/16 um 05:40 schrieb Jark Wu:
Hi Fabian, Timo,
I have created a prototype for removing STREAM keyword
Stephan Ewen created FLINK-4525:
---
Summary: Drop the "eager split pre-assignment" code paths
Key: FLINK-4525
URL: https://issues.apache.org/jira/browse/FLINK-4525
Project: Flink
Issue Type: Impr
>From what I understand so far, the message mirroring can be adjusted
in the follow parts:
1) GitHub PR description
2) GitHub Diff comments
3) GitHub Main thread comments
We can choose to put these as
1) The full GitHub comment
2) Only supply a link to the GitHub comment
Next, we can store all
Gordon fixed the Kinesis consumer already over the weekend. I just merged
the change.
I'll push a hotfix for the jodatime+quickstart issue to both branches.
On Fri, Aug 26, 2016 at 6:35 PM, Tzu-Li (Gordon) Tai
wrote:
> Next week Monday evening should be fine to get the Kinesis issue fix in.
>
>
Thanks for reporting Niels. We'll look into it ASAP.
On Mon, Aug 29, 2016 at 11:33 AM, Maximilian Michels
wrote:
> Danke für's Weiterleiten.
>
> On Mon, Aug 29, 2016 at 11:25 AM, Till Rohrmann
> wrote:
>> For the attention of the YARN shepherd.
>>
>> -- Forwarded message --
>> F
Thanks for reporting Niels. We'll look into it ASAP.
On Mon, Aug 29, 2016 at 10:31 AM, Niels Basjes wrote:
> Hi,
>
> Last week I brought down one of our Yarn nodes because of this problem:
> https://issues.apache.org/jira/browse/FLINK-4485
>
> The Yarn node no longer accepted any Flink/Yarn jobs
Thanks Stephan, for your reply.
I am actually trying to run Apache Samoa algorithm on a Flink runner. There
I have access to just the jar file that is created, and the
samoa-flink.properties file. No other access point.
~ Bhupesh
On Mon, Aug 29, 2016 at 2:33 PM, Stephan Ewen wrote:
> Hi!
>
> R
Hi!
Right now, you would have to activate it on the StreamExecutionEnvironment.
We did not at a global setting for that, because each job may choose
different settings (like checkpointing interval).
Can you intercept the job anywhere?
Stephan
On Mon, Aug 29, 2016 at 8:47 AM, Bhupesh Chawda wr
Robert Metzger created FLINK-4524:
-
Summary: JobManagerProcessReapingTest.testReapProcessOnFailure
unstable
Key: FLINK-4524
URL: https://issues.apache.org/jira/browse/FLINK-4524
Project: Flink
Hi Vinay,
it's not only done by enabling encryption of the Akka messages since the
TaskManager to TaskManager communication utilizes Netty. There is a JIRA
issue which tracks the progress on implementing secured communication
within Flink: https://issues.apache.org/jira/browse/FLINK-3931. Feel fre
Hi,
Last week I brought down one of our Yarn nodes because of this problem:
https://issues.apache.org/jira/browse/FLINK-4485
The Yarn node no longer accepted any Flink/Yarn jobs until we stopped the
jobmaster.
I classified this as a 'blocker' because it makes the yarn-session very
short lived (a
30 matches
Mail list logo
