[VOTE] [RESULT] Samza 0.10.0 Release Candidate 2

Cool! Thanks everyone! Now the vote has been more than 72 hours and we got +1 (binding) x 3 and +1 (non-binding) x 5. Samza 0.10.0 officially passed the VOTE! Thanks! -Yi On Thu, Dec 17, 2015 at 5:03 PM, Navina Ramesh wrote: > +1 binding > > Just a note: The integration test didn't run becaus

Re: [VOTE] Samza 0.10.0 Release Candidate 2

+1 binding Just a note: The integration test didn't run because I am somehow pulling in a newer version of Zopkio. Need to update our int tests for the next release! It works fine with a lower version of Zopkio. Thanks! Navina On Wed, Dec 9, 2015 at 8:12 PM, Chris Riccomini wrote: > +1 binding

Re: Configuring RocksDB SST file size

Hi, Kishore, Could you open a JIRA for this small SST files issue? It is good to track it s.t. we won't forget this one. Thanks! -Yi On Thu, Dec 17, 2015 at 4:16 AM, Kishore N C wrote: > Hi Tao, > > > I am not sure what do you mean by ulimit issues > > When so many small SST files are created

Re: samza-hello-samza build cannot find samza 0.10.0-SNAPSHOT artifacts on maven

The snapshot versions are also found on Apache's public repo: apache-repo apache-repo https://repository.apache.org/content/groups/public Cheers, KN. On Fri, Aug 28, 2015 at 11:36 PM, Yan Fang wrote: > run ./gradlew publishToMavenLocal ? > > Fang, Yan > yanfang...@gmail.com > >

Re: Configuring RocksDB SST file size

Hi Tao, > I am not sure what do you mean by ulimit issues When so many small SST files are created, I run into limits on maximum open files (ulimit -n). I dug into RocksDB's (plethora of) options today and identified the option that causes the 2.3 MB sizes: target_file_size_base

Re: Statefull system consumer

Hi, Anton, It seems to me that the best option would possibly use the row number as the IncomingMessageEnvelope's offset. Then, when Samza commits the checkpoint, it will commit the row number as the offset. When the Samza job restarts, the row number would be read from the checkpoint topic and yo

Re: SystemConsumer questions

Hi, Ivan, Sorry to reply late. Could you explain what state that you have to maintain in SystemConsumer, not in KV-store and checkpoint topics? Samza's SystemConsumer is designed as a "pump" to simply pump the messages in to Samza StreamTasks, where the main stateful process is executed. Why and w

Re: KafkaSystemProducer partitioning

Hi, Michal, Sorry to reply late. Actually, you are right that the "partition.class" configuration is not used in Samza to determine the outgoing partition. In Samza, partition is defined by the following code sections: {code} val topicName = envelope.getSystemStream.getStream val partitions: java