log size, as in the worst case,
the change log will grow as fast as the input log.
Chen
--
Chen Song
> org.apache.hadoop.security.AccessControlException: Client cannot
> > > authenticate via:[TOKEN, KERBEROS]; Host Details : local host is:
> > > "talend-cdh-datanode8/62.210.141.237"; destination host is:
> > > "talend-cdh-namenode":8020;
> > >
> > > java.io.IOException: Failed on local exception: java.io.IOException:
> > > org.apache.hadoop.security.AccessControlException: Client cannot
> > > authenticate via:[TOKEN, KERBEROS]; Host Details : local host is:
> > > "cdh-datanode8/62.210.141.237"; destination host is:
> > "cdh-namenode":8020;
> > >
> > > at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:772)
> > >
> > > ..
> > >
> > >
> > >
> > > Anyone knows how to solve this?
> > >
> > >
> > > Qi FU
> > >
> >
>
--
Chen Song
Thanks Yan. I will take a look shortly.
On Sat, Jul 25, 2015 at 1:20 AM, Yan Fang wrote:
> Hi Chen Song,
>
> If you can work on this issue, it will be great.
>
> 1. the related ticket is https://issues.apache.org/jira/browse/SAMZA-727
>
> 2. most of the change will happe
Maybe a dumb question.
Is there a way to set an ENV for samza containers?
We want to set LD_LIBRARY_PATH to include hadoop native libs.
--
Chen Song
-apache-hadoop-util-nativec
.
If anyone has any thoughts on this error, please advise.
Chen
On Thu, Jul 30, 2015 at 5:15 PM, Yan Fang wrote:
> Hi Chen Song,
>
> I do not think there is a way in Samza with which you can specify the ENV
> for Samza container.
>
> And currently S
resource manager's security token as the renewer.
Diffs
-
samza-yarn/src/main/scala/org/apache/samza/job/yarn/ClientHelper.scala
a2b92798a78d2f489e8a8fc082b2bb44a734a6cc
Diff: https://reviews.apache.org/r/37026/diff/
Testing
---
Thanks,
Chen Song
Dump question.
When running samza job on YARN, I found that container logs gone after
days. Is there a YARN config to keep the logs from being deleted for
long-lived streaming job?
--
Chen Song
e for each message and this will result
the changelog to grow to the same size of input log. Even with compaction
turned on changelog, it is not very scalable. I am wondering if there is a
way to mitigate this problem.
On Wed, Jul 22, 2015 at 2:12 PM, Yan Fang wrote:
> Hi Chen Song,
>
>
ce the size (both
> for the KV-store and the changelog).
>
> Regards,
> -Yi
>
> On Tue, Aug 4, 2015 at 7:25 AM, Chen Song wrote:
>
> > Thanks Yan.
> >
> > Very good explanation on 1).
> >
> > For 2), I understand that users can tune the
a bug in the master branch that is
> pending resolution in https://issues.apache.org/jira/browse/SAMZA-723
>
> Cheers!
> Navina
>
>
> On Mon, Aug 3, 2015 at 2:26 PM, Chen Song wrote:
>
> > Dump question.
> >
> > When running samza job on YARN, I found that c
Also, I have YARN aggregated enabled. But because logs are gone locally,
after the job is killed or failed. I don't have aggregated logs on HDFS.
On Tue, Aug 25, 2015 at 5:39 PM, Chen Song wrote:
> Thanks Navina.
>
> What I found is that containers logs were gone after days local
27;t sound a solid way
in the long
term.
Any thoughts?
Chen
-
--
Chen Song
task is reading from the same partition of a topic). I am
wondering how is that achieved?
Chen
On Sat, Oct 24, 2015 at 5:52 AM, Yan Fang wrote:
> Hi Chen Song,
>
>
> Sorry for the late reply. What you describe is a typical bootstrap use
> case. Check
> http://samza.apache.org/l
pic is a special one that the containers only read during
> the start up phase. Hence, it is not considered as part of the
> SystemStreamPartitions that are assigned to the tasks. As Yan mentioned,
> broadcast stream in 0.10 is the solution to your use case.
>
> Thanks!
>
>
> &g
According to the documentation, each Samza container is single threaded.
Why giving yarn.container.cpu.cores as a config option and what is the
implication
to set this to a value > 1?
--
Chen Song
> itself. However, you may end-up under-utilizing your Yarn cluster
> resources.
>
> HTH!
> Navina
>
> On Mon, Nov 16, 2015 at 2:32 PM, Chen Song wrote:
>
> > According to the documentation, each Samza container is single threaded.
> > Why giving yarn.container.c
: samza
Description
---
Use resource manager's security token as the renewer.
Diffs (updated)
-
samza-yarn/src/main/scala/org/apache/samza/job/yarn/ClientHelper.scala
74a0676
Diff: https://reviews.apache.org/r/37026/diff/
Testing
---
Thanks,
Chen Song
/yarn/YarnJob.scala 62ddb26
samza-yarn/src/test/scala/org/apache/samza/job/yarn/TestClientHelper.scala
PRE-CREATION
samza-yarn/src/test/scala/org/apache/samza/job/yarn/TestSamzaAppMasterService.scala
7f5d9f4
Diff: https://reviews.apache.org/r/37026/diff/
Testing
---
Thanks,
Chen Song
/yarn/YarnJob.scala 62ddb26
samza-yarn/src/test/scala/org/apache/samza/job/yarn/TestClientHelper.scala
PRE-CREATION
samza-yarn/src/test/scala/org/apache/samza/job/yarn/TestSamzaAppMasterService.scala
7f5d9f4
Diff: https://reviews.apache.org/r/37026/diff/
Testing
---
Thanks,
Chen Song
/yarn/YarnJob.scala 62ddb26
samza-yarn/src/test/scala/org/apache/samza/job/yarn/TestClientHelper.scala
PRE-CREATION
samza-yarn/src/test/scala/org/apache/samza/job/yarn/TestSamzaAppMasterService.scala
7f5d9f4
Diff: https://reviews.apache.org/r/37026/diff/
Testing
---
Thanks,
Chen Song
---
Thanks,
Chen Song
: https://reviews.apache.org/r/46282/diff/
Testing
---
Thanks,
Chen Song
can reuse the code in ClientHelper since they are
> > almost identical
Same as above. Remove the deletion of staging directory from the AM main
method. The ideal way to handle the clean up of staging directory would be.
* The client side, if we make JobRunner keep polling status of appli
https://reviews.apache.org/r/37026/diff/
Testing
---
Thanks,
Chen Song
---
Thanks,
Chen Song
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46282/#review130660
---
On April 15, 2016, 10:09 p.m., Chen Song wrote:
>
> ---
> This
--
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37026/#review130672
---
On May 8, 2016, 12:50 a.m., Chen Song wrote:
>
> -
stream. Once we have coordinator stream secured, we can revisit
this.
- Chen
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46282/#review130660
---
> > online docs for general YARN APP launch sequence w/ Kerberos.
>
> Chen Song wrote:
> Yes, that is my understanding. Each container, once initialized, will use
> whatever HDFS delegation token passed in the launch context. However, once it
> expires, the container won&
/learn/documentation/versioned/yarn/yarn-security.md PRE-CREATION
Diff: https://reviews.apache.org/r/46282/diff/
Testing
---
Thanks,
Chen Song
--
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/37026/#review134501
-------
On May 8, 2016, 12:50 a.m., Chen Song wrote:
>
>
ation is missing in the doc patch.
>
> Chen Song wrote:
> Will update the doc patch.
Sorry this config is auto-generated by ClientHelper and sent to the coordinator
stream. It is not set explicitly as a configuration by user. I will elaborate
it a bit more in the yarn-security.md doc page. H
Diff: https://reviews.apache.org/r/37026/diff/
Testing
---
Thanks,
Chen Song
/documentation/versioned/yarn/isolation.md 1eb3bf5
docs/learn/documentation/versioned/yarn/yarn-security.md PRE-CREATION
Diff: https://reviews.apache.org/r/46282/diff/
Testing
---
Thanks,
Chen Song
---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/46282/#review135081
---
On June 15, 2016, 2:25 a.m., Chen Song wrote:
>
> --
35 matches
Mail list logo
