Hi, Milinda,
We have recently some discussions on the MillWheel model:
http://www.infoq.com/presentations/millwheel.
It is very interesting talk and have one striking point that we did not
think about before: handle late arrivals as a "correction" to the earlier
results. Hence, if we follow that
Hi Yi,
Please find my comments inline.
On Thu, Mar 5, 2015 at 1:18 PM, Yi Pan wrote:
> Hi, Milinda,
>
> We have recently some discussions on the MillWheel model:
> http://www.infoq.com/presentations/millwheel.
Yes. Above is a very interesting talk. I asked the above question regarding
the lan
Hi Samza devs
How can I get container id such as 0, 1, 2, ... up to yarn.container.count?
I tried to get some environment variable but it didn't work well.
If it's not easy to get the container id, I have to use process id to
distinguish containers from the same job running in the same machine b
Hey Jae,
We expose this in SamzaContainerContext. Unfortunately, this class never
gets exposed to the StreamTask. The plan was to expose this class via
TaskContext, but we haven't done that yet.
Have you tried using System.getProperty("samza.container.id")?
Cheers,
Chris
On Thu, Mar 5, 2015 at
Hey Jae,
Also, this should work:
System.getenv(ShellCommandConfig.ENV_CONTAINER_ID).toInt
This is actually how SamzaContainer gets its containerId variable. BTW,
I've opened:
https://issues.apache.org/jira/browse/SAMZA-586
To track exposing SamzaContainerContext.
Cheers,
Chris
Hey all,
No majore objections. I'm moving forward with this. I've moved all 0.9.0
JIRAs to 0.10.0, except for the ones in the above list. Here's a list of
all open 0.9.0 bugs:
https://issues.apache.org/jira/issues/?jql=project%20%3D%20SAMZA%20AND%20resolution%20%3D%20Unresolved%20AND%20fixVersion
ENV_CONTAINER_ID does not look existing in 0.8.0. I will try
ENV_CONTAINER_NAME.
On Thu, Mar 5, 2015 at 5:09 PM, Chris Riccomini
wrote:
> Hey Jae,
>
> Also, this should work:
>
> System.getenv(ShellCommandConfig.ENV_CONTAINER_ID).toInt
>
> This is actually how SamzaContainer gets its container
Hey Jae,
Ah, yes, you are correct. 0.8.0 has container names, not IDs. In YARN,
these names will be the YARN container names. I'm not sure what they're set
to in Mesos--I'd have to check the Mesos JobRunner stuff.
Cheers,
Chris
On Thu, Mar 5, 2015 at 5:47 PM, Bae, Jae Hyeon wrote:
> ENV_CONTAI
