Re: Samza ZooKeeper Connection Inquiry

2015-06-03 Thread Yan Fang
Hi Chas, The actually connection does not happen in the StreamTask, but the Consumer/Producer layer. One way of understanding this is that, * Consumers are run in a separate thread, which keeps the connection to the server (kafka broker, wiki api, etch) and fetches messages from them; * StreamTas

Re: Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

2015-06-03 Thread Yi Pan (Data Infrastructure)
> On June 3, 2015, 11:44 p.m., Guozhang Wang wrote: > > Since the operator topology will sit inside a single node, most of the time > > we should be facing some simple topology structures like > > window-window-join, but not as complicated as a complete workflow DAG. So I > > think we do not n

Re: Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

2015-06-03 Thread Yi Pan (Data Infrastructure)
> On May 29, 2015, 1:28 a.m., Navina Ramesh wrote: > > samza-sql-core/src/main/java/org/apache/samza/sql/api/data/EntityName.java, > > line 160 > > > > > > How is this "anonymous" system used? This will go away when

Re: Review Request 34500: SAMZA-552 Operator API change: builder and simplified operator classes

2015-06-03 Thread Guozhang Wang
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34500/#review86504 --- Since the operator topology will sit inside a single node, most of t

Samza ZooKeeper Connection Inquiry

2015-06-03 Thread Chas Pezanko
Hello, So I am currently working on learning Samza for work and there is a question that I am looking for some direction through. In a Kafka Consumer/Producer in order to connect to ZooKeeper you can simply do so through a localhost port configuration and that creates a static connection for a

Re: Nothing occurs from collector.send()

2015-06-03 Thread Yan Fang
Hi Garrett, 1. "systems.kafka.streams.metrics-samza.samza.msg.serde=metrics" seems a little problematic for me. I did not see where the metrics part is configured. such as "# Define a metrics reporter called "snapshot", which publishes metrics # every 60 seconds. metrics.reporters=snapshot metric

Re: Review Request 34564: SAMZA-401: add utilization metrics for the event loop

2015-06-03 Thread Luis De Pombo
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/34564/ --- (Updated June 3, 2015, 5:13 p.m.) Review request for samza. Repository: samza

Re: Review Request 34564: SAMZA-401: add utilization metrics for the event loop

2015-06-03 Thread Luis De Pombo
> On May 22, 2015, 7:36 a.m., Yan Fang wrote: > > samza-core/src/main/scala/org/apache/samza/container/SamzaContainerMetrics.scala, > > line 41 > > > > > > do we want to create a new Gauge every loop? I think one Gauge

Re: Nothing occurs from collector.send()

2015-06-03 Thread Garrett Barton
It must have been junk data. I started using a new topic for everything (metrics/source/dest) and data is flowing fine now. When I switch it back to the other topics I get the hung behavior with nothing erroring out in the logs. On Wed, Jun 3, 2015 at 10:10 AM, Garrett Barton wrote: > Thanks al

Re: Did 0.9.1 ever get released?

2015-06-03 Thread Guozhang Wang
Hi Tommy, I am currently working on 0.9.1 release now, and expecting it to be out in a week or two. Guozhang On Wed, Jun 3, 2015 at 7:16 AM, Tommy Becker wrote: > Did 0.9.1 ever get released? I don't see it in Maven Central, nor has the > site been updated... > > -- > Tommy Becker > Senior So

Did 0.9.1 ever get released?

2015-06-03 Thread Tommy Becker
Did 0.9.1 ever get released? I don't see it in Maven Central, nor has the site been updated... -- Tommy Becker Senior Software Engineer Digitalsmiths A TiVo Company www.digitalsmiths.com tobec...@tivo.com ___

Re: Nothing occurs from collector.send()

2015-06-03 Thread Garrett Barton
Thanks all for the responses. As you can probably tell from the job name I am doing a validation process which is taking in json and testing for a ton of things. This is why I am using String serde's instead of Samza's built in json support, the json could be malformed and I need to still operate