OK. Here is what I have inside my configuration in order to start NewRelic
at the time of the container start:
task.opts=-Xmx2048m -XX:+UseConcMarkSweepGC
*-javaagent:/usr/home/vaibhav/newrelic/newrelic.jar*
Now, I have specified the absolute path of the NewRelic jar. There are 2
challenges that I see with this approach:
1. The NewRelic jar and config would have to be present on all the Yarn
slave nodes since that container can start at any of the Yarn nodes.
2. A single Yarn cluster can be used to run multiple Samza applications, so
I would have to keep configuration for each application at each of the Yarn
slave nodes.
In order to fix this, I am bundling NewRelic inside my tar file:
// download new relic dependencies
task downloadNewrelic(type: Download) {
mkdir "newrelic"
src "
https://download.newrelic.com/newrelic/java-agent/newrelic-agent/current/newrelic-java.zip
"
dest file("newrelic")
}
task buildTar(dependsOn: [build, downloadNewrelic], type: Tar) {
// unzip new relic dependencies and add to the archive
into(".") {
from zipTree(file("newrelic/newrelic-java.zip"))
}
}
With this build configuration, I have an additional *newrelic* directory
inside my archive. Here is the directory structure:
my-job-name.tar.gz
bin
config
lib
newrelic
newrelic.jar
newrelic.yml
...
What I would like to achieve is to:
- Remove hardcoding the *javaagent* inside *task.opts* in the container
config.
- Specify the path of NewRelic agent in *task.opts* relative to where the
container files are present on the Yarn slave node.
I hope I was clearer this time. Please let me know if any other
clarification is needed.
Thanks,
Vaibhav Garg
+91-9505020924
vaibhavgar...@gmail.com
LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
On Sat, Feb 29, 2020 at 5:22 AM Yi Pan <nickpa...@gmail.com> wrote:
> Hi, Vaibhav,
>
> Not quite sure whether I understand your ask. How do you package and
> distribute NewRelic jar in the YARN cluster? It would be much helpful if
> you can share your job configure that worked and the places that you would
> like to change.
>
> Best,
>
> -Yi
>
> On Fri, Feb 28, 2020 at 11:55 AM Vaibhav Garg <vaibhavgar...@gmail.com>
> wrote:
>
> > OK. I have been able to use NewRelic by specifying -javaagent option in
> > task.opts.
> >
> > However, I had to give the absolute path to the NewRelic jar file in
> order
> > to make it work.
> >
> > How can I give a path that is relative to the container since I would
> like
> > to integrate NewRelic dependencies along with other container files?
> >
> > Thanks,
> > Vaibhav Garg
> > +91-9505020924
> > vaibhavgar...@gmail.com
> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> >
> >
> > On Sat, Feb 29, 2020 at 12:41 AM Vaibhav Garg <vaibhavgar...@gmail.com>
> > wrote:
> >
> > > Yes. That's exactly what I am looking for.
> > >
> > > Let me try it out and I will get back to you if I face any issues.
> > >
> > > Thanks again
> > >
> > > Vaibhav Garg
> > > +91-9505020924
> > > vaibhavgar...@gmail.com
> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >
> > >
> > > On Fri, Feb 28, 2020 at 1:23 PM Yi Pan <nickpa...@gmail.com> wrote:
> > >
> > >> Hi, Vaibhav,
> > >>
> > >> Check the description of task.opts in the configuration doc here:
> > >>
> > >>
> >
> http://samza.apache.org/learn/documentation/latest/jobs/samza-configurations.html
> > >>
> > >> Is this what you are looking for?
> > >>
> > >> On Thu, Feb 27, 2020 at 9:54 PM Vaibhav Garg <vaibhavgar...@gmail.com
> >
> > >> wrote:
> > >>
> > >> > Hi,
> > >> >
> > >> > Any help here would be great.
> > >> >
> > >> > Thanks,
> > >> > Vaibhav Garg
> > >> > +91-9505020924
> > >> > vaibhavgar...@gmail.com
> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >> >
> > >> >
> > >> > On Thu, Feb 27, 2020 at 10:20 AM Vaibhav Garg <
> > vaibhavgar...@gmail.com>
> > >> > wrote:
> > >> >
> > >> > > Dear Bharath,
> > >> > >
> > >> > > Thanks for your reply. I now realize that I put a very ambiguous
> > >> question
> > >> > > to the community. Here is another attempt:
> > >> > >
> > >> > > I have set up a Yarn cluster that is configured to run containers
> of
> > >> > > multiple Samza jobs.
> > >> > >
> > >> > > I would like to treat all the containers of the job as a single
> > >> > > application in New Relic (as expected).
> > >> > >
> > >> > > Since the containers of the application can be killed on one Yarn
> > node
> > >> > and
> > >> > > can start on another Yarn node, I would have to specify New Relic
> > >> > > environment settings at the container start.
> > >> > >
> > >> > > Now, I am not sure how to specify java agent (to include New Relic
> > >> java
> > >> > > agent) and pass additional arguments such as New Relic environment
> > >> every
> > >> > > time the container starts.
> > >> > >
> > >> > > Please let me know if my understanding is wrong or some other
> > >> > > clarification is needed.
> > >> > >
> > >> > > Thanks in advance,
> > >> > > Vaibhav Garg
> > >> > > +91-9505020924
> > >> > > vaibhavgar...@gmail.com
> > >> > > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >> > >
> > >> > >
> > >> > > On Thu, Feb 27, 2020 at 8:25 AM Bharath Kumara Subramanian <
> > >> > > codin.mart...@gmail.com> wrote:
> > >> > >
> > >> > >> Hi,
> > >> > >>
> > >> > >> I am not sure I fully understand the ask.
> > >> > >>
> > >> > >> IIUC, Samza doesn't have native integration with New relic.
> > However,
> > >> you
> > >> > >> should still be able to integrate your application with New relic
> > on
> > >> > your
> > >> > >> end without native support.
> > >> > >> If you are particularly looking to integrate native Samza metrics
> > w/
> > >> New
> > >> > >> relic, you might need to implement your own custom metrics
> > reporter.
> > >> You
> > >> > >> can find more details here
> > >> > >> <
> > >> > >>
> > >> >
> > >>
> >
> https://samza.apache.org/learn/documentation/latest/operations/monitoring.html#customreporter
> > >> > >> >
> > >> > >>
> > >> > >> Thanks,
> > >> > >> Bharath
> > >> > >>
> > >> > >> On Wed, Feb 26, 2020 at 3:15 AM Vaibhav Garg <
> > >> vaibhavgar...@gmail.com>
> > >> > >> wrote:
> > >> > >>
> > >> > >> > Hi,
> > >> > >> >
> > >> > >> > I want to integrate New Relic in my Samza jobs. Can anyone help
> > >> with
> > >> > >> this,
> > >> > >> > please?
> > >> > >> >
> > >> > >> > Thanks,
> > >> > >> > Vaibhav Garg
> > >> > >> > +91-9505020924
> > >> > >> > vaibhavgar...@gmail.com
> > >> > >> > LinkedIn <https://www.linkedin.com/in/vaibhavgarg90/>
> > >> > >> >
> > >> > >>
> > >> > >
> > >> >
> > >>
> > >
> >
>
