Re: Error handling

2019-03-25 Thread Tom Davis
I am using Samza 1.0, yes. The stacktrace is: 19:24:49.326 [Samza StreamProcessor Container Thread-0] ERROR org.apache.samza.processor.StreamProcessor - Container: org.apache.samza.container.SamzaContainer@3e923d9e failed with an exception. Stopping the stream processor: c13057a8-42c5-4b68-9

Re: Empty Kafka topic partition Warning

2019-03-25 Thread Jeremiah Adams
It looks like my configuration for changelog.replication.factor is not being applied. Instead the default seems to be applied. I have messages on 2/3 partitions. I am not seeing what I have incorrectly configured. stores.redelivery-store.factory=org.apache.samza.storage.kv.RocksDbKeyValueStorage

[GitHub] [samza] srinipunuru commented on a change in pull request #966: Moving the Fixes from internal AvroRelConverter

2019-03-25 Thread GitBox
srinipunuru commented on a change in pull request #966: Moving the Fixes from internal AvroRelConverter URL: https://github.com/apache/samza/pull/966#discussion_r268728229 ## File path: samza-sql/src/main/java/org/apache/samza/sql/avro/AvroRelConverter.java ## @@ -170,22

[GitHub] [samza] srinipunuru commented on a change in pull request #966: Moving the Fixes from internal AvroRelConverter

2019-03-25 Thread GitBox
srinipunuru commented on a change in pull request #966: Moving the Fixes from internal AvroRelConverter URL: https://github.com/apache/samza/pull/966#discussion_r268731436 ## File path: samza-sql/src/main/java/org/apache/samza/sql/avro/AvroRelConverter.java ## @@ -271,45

[GitHub] [samza] srinipunuru commented on a change in pull request #966: Moving the Fixes from internal AvroRelConverter

2019-03-25 Thread GitBox
srinipunuru commented on a change in pull request #966: Moving the Fixes from internal AvroRelConverter URL: https://github.com/apache/samza/pull/966#discussion_r268731436 ## File path: samza-sql/src/main/java/org/apache/samza/sql/avro/AvroRelConverter.java ## @@ -271,45

Fwd: SSL with Samza 0.14.1?

2019-03-25 Thread Prateek Maheshwari
Forwarding again. Original email did not show up on the OSS mailing list. -- Forwarded message - From: Deshpande, Omkar Date: Fri, Mar 22, 2019 at 5:08 PM Subject: Fwd: SSL with Samza 0.14.1? To: prateek...@gmail.com ++Prateek gmail -- *From:* LeVeck

Re: Empty Kafka topic partition Warning

2019-03-25 Thread Jagadish Venkatraman
Hi Jeremiah, >> It looks like my configuration for changelog.replication.factor is not being applied. Instead the default seems to be applied. I have messages on 2/3 partitions. I am not seeing what I have incorrectly configured. The "replication.factor" config you mention determines how many bro

Re: SSL with Samza 0.14.1?

2019-03-25 Thread Prateek Maheshwari
Hi Matt, It's possible that the old Kafka AdminClient does not support SSL for ZK out of the box. I'll check if this is the case, and if this is something that can be configured. In the mean time, can you tell us the following: 1. Kafka broker version you're running. 2. Kafka client version for t

Re: SSL with Samza 0.14.1?

2019-03-25 Thread Jagadish Venkatraman
Hi Matt, >> We’re trying to connect to a cluster that has SSL enabled, and we’re getting some errors that are indicative of SSL connectivity failing. Can you share some more details on the errors - eg: log files, stack-traces? Do you get the error when Samza attempts to create the topic? >> Is

[GitHub] [samza] cameronlee314 commented on a change in pull request #955: SAMZA-2131: [Scala cleanup] Convert FileSystemCheckpointManagerConfig.scala and SystemConfig.scala to Java

2019-03-25 Thread GitBox
cameronlee314 commented on a change in pull request #955: SAMZA-2131: [Scala cleanup] Convert FileSystemCheckpointManagerConfig.scala and SystemConfig.scala to Java URL: https://github.com/apache/samza/pull/955#discussion_r268763050 ## File path: samza-core/src/main/java/org/apache

[GitHub] [samza] mynameborat commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0

2019-03-25 Thread GitBox
mynameborat commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0 URL: https://github.com/apache/samza/pull/951#discussion_r268764920 ## File path: samza-test/src/test/java/org/apache/samza/test/framework/StreamApplicationIntegrationTestHarness.java

Re: Error handling

2019-03-25 Thread Prateek Maheshwari
Hi Tom, Unfortunately this exception only shows that the SamzaContainer tried to shut down a second time due to a processing timeout. This by itself is fine, and should be handled by the framework already. We'll need to look at rest of the framework logs to tell what state the application was in

[GitHub] [samza] mynameborat commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0

2019-03-25 Thread GitBox
mynameborat commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0 URL: https://github.com/apache/samza/pull/951#discussion_r268772818 ## File path: samza-test/src/test/scala/org/apache/samza/test/integration/StreamTaskTestUtil.scala ## @@ -133,43 +1

[GitHub] [samza] srinipunuru merged pull request #966: Moving the Fixes from internal AvroRelConverter

2019-03-25 Thread GitBox
srinipunuru merged pull request #966: Moving the Fixes from internal AvroRelConverter URL: https://github.com/apache/samza/pull/966 This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [samza] mynameborat commented on a change in pull request #955: SAMZA-2131: [Scala cleanup] Convert FileSystemCheckpointManagerConfig.scala and SystemConfig.scala to Java

2019-03-25 Thread GitBox
mynameborat commented on a change in pull request #955: SAMZA-2131: [Scala cleanup] Convert FileSystemCheckpointManagerConfig.scala and SystemConfig.scala to Java URL: https://github.com/apache/samza/pull/955#discussion_r268788137 ## File path: samza-core/src/main/java/org/apache/

[GitHub] [samza] dengpanyin commented on issue #961: SAMZA-2134:Enable table rate limiter by default.

2019-03-25 Thread GitBox
dengpanyin commented on issue #961: SAMZA-2134:Enable table rate limiter by default. URL: https://github.com/apache/samza/pull/961#issuecomment-476328264 Add @sborya to take a look as well. This is an automated message from

Re: Empty Kafka topic partition Warning

2019-03-25 Thread Jeremiah Adams
Thanks for the input Jagadish. This makes sense now. I'm seeing one of our input topic partitions empty. One of our upstream jobs is likely configured incorrectly. Jeremiah Adams Software Engineer www.helixeducation.com Blog | Twitter | Facebook | LinkedIn _

[GitHub] [samza] vjagadish1989 commented on a change in pull request #961: SAMZA-2134:Enable table rate limiter by default.

2019-03-25 Thread GitBox
vjagadish1989 commented on a change in pull request #961: SAMZA-2134:Enable table rate limiter by default. URL: https://github.com/apache/samza/pull/961#discussion_r268843384 ## File path: samza-api/src/main/java/org/apache/samza/table/descriptors/RemoteTableDescriptor.java ##

[GitHub] [samza] vjagadish1989 commented on a change in pull request #961: SAMZA-2134:Enable table rate limiter by default.

2019-03-25 Thread GitBox
vjagadish1989 commented on a change in pull request #961: SAMZA-2134:Enable table rate limiter by default. URL: https://github.com/apache/samza/pull/961#discussion_r268846328 ## File path: samza-api/src/main/java/org/apache/samza/table/descriptors/RemoteTableDescriptor.java ##

[GitHub] [samza] vjagadish1989 commented on a change in pull request #961: SAMZA-2134:Enable table rate limiter by default.

2019-03-25 Thread GitBox
vjagadish1989 commented on a change in pull request #961: SAMZA-2134:Enable table rate limiter by default. URL: https://github.com/apache/samza/pull/961#discussion_r268846328 ## File path: samza-api/src/main/java/org/apache/samza/table/descriptors/RemoteTableDescriptor.java ##

[GitHub] [samza] vjagadish1989 commented on a change in pull request #961: SAMZA-2134:Enable table rate limiter by default.

2019-03-25 Thread GitBox
vjagadish1989 commented on a change in pull request #961: SAMZA-2134:Enable table rate limiter by default. URL: https://github.com/apache/samza/pull/961#discussion_r268846328 ## File path: samza-api/src/main/java/org/apache/samza/table/descriptors/RemoteTableDescriptor.java ##

[GitHub] [samza] dengpanyin commented on a change in pull request #961: SAMZA-2134:Enable table rate limiter by default.

2019-03-25 Thread GitBox
dengpanyin commented on a change in pull request #961: SAMZA-2134:Enable table rate limiter by default. URL: https://github.com/apache/samza/pull/961#discussion_r268856771 ## File path: samza-api/src/main/java/org/apache/samza/table/descriptors/RemoteTableDescriptor.java #

[GitHub] [samza] dengpanyin commented on a change in pull request #961: SAMZA-2134:Enable table rate limiter by default.

2019-03-25 Thread GitBox
dengpanyin commented on a change in pull request #961: SAMZA-2134:Enable table rate limiter by default. URL: https://github.com/apache/samza/pull/961#discussion_r268860718 ## File path: samza-api/src/main/java/org/apache/samza/table/descriptors/RemoteTableDescriptor.java #

Re: Empty Kafka topic partition Warning

2019-03-25 Thread Jagadish Venkatraman
Hey Jeremiah, That's great to hear that you've root-caused it! Please let us know should you run into further issues :-) Best, Jagadish On Mon, Mar 25, 2019 at 1:16 PM Jeremiah Adams wrote: > Thanks for the input Jagadish. This makes sense now. I'm seeing one of our > input topic partitions e

[GitHub] [samza] mynameborat opened a new pull request #972: [SAMZA-2146] - Expose physical name in StreamDescriptor

2019-03-25 Thread GitBox
mynameborat opened a new pull request #972: [SAMZA-2146] - Expose physical name in StreamDescriptor URL: https://github.com/apache/samza/pull/972 Refer to https://issues.apache.org/jira/browse/SAMZA-2146 for more details This

[GitHub] [samza] asfgit closed pull request #972: [SAMZA-2146] - Expose physical name in StreamDescriptor

2019-03-25 Thread GitBox
asfgit closed pull request #972: [SAMZA-2146] - Expose physical name in StreamDescriptor URL: https://github.com/apache/samza/pull/972 This is an automated message from the Apache Git Service. To respond to the message, plea

[GitHub] [samza] shanthoosh opened a new pull request #973: Remove semicolon from sub-headings of 1.1 blog post.

2019-03-25 Thread GitBox
shanthoosh opened a new pull request #973: Remove semicolon from sub-headings of 1.1 blog post. URL: https://github.com/apache/samza/pull/973 This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [samza] shanthoosh commented on issue #973: Remove semicolon from sub-headings of 1.1 blog post.

2019-03-25 Thread GitBox
shanthoosh commented on issue #973: Remove semicolon from sub-headings of 1.1 blog post. URL: https://github.com/apache/samza/pull/973#issuecomment-476411309 Thanks for the quick review Daniel. Since the change involves minor documentation cleanup, merging it to trunk. ---

[GitHub] [samza] shanthoosh merged pull request #973: Remove semicolon from sub-headings of 1.1 blog post.

2019-03-25 Thread GitBox
shanthoosh merged pull request #973: Remove semicolon from sub-headings of 1.1 blog post. URL: https://github.com/apache/samza/pull/973 This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [samza] mynameborat commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0

2019-03-25 Thread GitBox
mynameborat commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0 URL: https://github.com/apache/samza/pull/951#discussion_r268891069 ## File path: samza-kafka/src/main/java/org/apache/samza/system/kafka/KafkaSystemAdmin.java ## @@ -628,10 +589,12 @

[GitHub] [samza] mynameborat commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0

2019-03-25 Thread GitBox
mynameborat commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0 URL: https://github.com/apache/samza/pull/951#discussion_r268861935 ## File path: samza-test/src/test/java/org/apache/samza/test/harness/IntegrationTestHarness.java ## @@ -0,0 +1,192

[GitHub] [samza] mynameborat commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0

2019-03-25 Thread GitBox
mynameborat commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0 URL: https://github.com/apache/samza/pull/951#discussion_r268893914 ## File path: samza-kafka/src/main/java/org/apache/samza/system/kafka/KafkaSystemAdmin.java ## @@ -628,10 +589,12 @

[GitHub] [samza] mynameborat commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0

2019-03-25 Thread GitBox
mynameborat commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0 URL: https://github.com/apache/samza/pull/951#discussion_r268880746 ## File path: samza-test/src/test/java/org/apache/samza/test/harness/IntegrationTestHarness.java ## @@ -0,0 +1,192

[GitHub] [samza] mynameborat commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0

2019-03-25 Thread GitBox
mynameborat commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0 URL: https://github.com/apache/samza/pull/951#discussion_r268890341 ## File path: gradle/dependency-versions.gradle ## @@ -34,7 +34,7 @@ jodaTimeVersion = "2.2" joptSimpleVersion

[GitHub] [samza] cameronlee314 commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0

2019-03-25 Thread GitBox
cameronlee314 commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0 URL: https://github.com/apache/samza/pull/951#discussion_r268902437 ## File path: samza-test/src/test/java/org/apache/samza/test/framework/StreamApplicationIntegrationTestHarness.java ##

[GitHub] [samza] cameronlee314 commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0

2019-03-25 Thread GitBox
cameronlee314 commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0 URL: https://github.com/apache/samza/pull/951#discussion_r268901338 ## File path: samza-kafka/src/main/java/org/apache/samza/system/kafka/KafkaSystemAdmin.java ## @@ -594,7 +593,13

[GitHub] [samza] cameronlee314 commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0

2019-03-25 Thread GitBox
cameronlee314 commented on a change in pull request #951: SAMZA-2127: Upgrade to Kafka 2.0 URL: https://github.com/apache/samza/pull/951#discussion_r268900710 ## File path: samza-kafka/src/main/java/org/apache/samza/system/kafka/KafkaSystemAdmin.java ## @@ -594,7 +593,13

[VOTE] SEP-21: Samza Async API for High Level

2019-03-25 Thread Bharath Kumara Subramanian
Hi all, This is a call for a vote for SEP-21: Samza Async API for High Level SEP-21 has been discussed and implemented using SAMZA-2055. For reference, the design document can be found - https://cwiki.apache.org/confluence/display/SAMZA/SEP-21%3A+Samza+Async+API+for+High+Level Thanks, Bharath

[VOTE] SEP-21: Samza Async API for High Level

2019-03-25 Thread Bharath Kumarasubramanian
Hi all, This is a call for a vote for SEP-21: Samza Async API for High Level. SEP-21 has been discussed and implemented using SAMZA-2055. For reference, the design document can be found - https://cwiki.apache.org/confluence/display/SAMZA/SEP-21%3A+Samza+Async+API+for+High+Level Thanks, Bharath

[GitHub] [samza] shanthoosh opened a new pull request #974: SAMZA-2147: Improve the documentation of the samza release process.

2019-03-25 Thread GitBox
shanthoosh opened a new pull request #974: SAMZA-2147: Improve the documentation of the samza release process. URL: https://github.com/apache/samza/pull/974 Currently the RELEASE.md which outlines the samza release process in open-source has the following problems: * There is no cle

[GitHub] [samza] shanthoosh commented on issue #974: SAMZA-2147: Improve the documentation of the samza release process.

2019-03-25 Thread GitBox
shanthoosh commented on issue #974: SAMZA-2147: Improve the documentation of the samza release process. URL: https://github.com/apache/samza/pull/974#issuecomment-476433492 @dxichen Please take a look when you have a chance.

[GitHub] [samza] shanthoosh commented on issue #974: SAMZA-2147: Improve the documentation of the samza release process.

2019-03-25 Thread GitBox
shanthoosh commented on issue #974: SAMZA-2147: Improve the documentation of the samza release process. URL: https://github.com/apache/samza/pull/974#issuecomment-476434197 @dxichen Here is the screen shot of the `RELEASE.md` file rendered after this patch. ![screencapture-file-hom

Re: [VOTE] SEP-21: Samza Async API for High Level

2019-03-25 Thread Jagadish Venkatraman
+1 (binding); thanks Bharath for the proposal and the implementation. LGTM; On Monday, March 25, 2019, Bharath Kumara Subramanian < codin.mart...@gmail.com> wrote: > Hi all, > > > This is a call for a vote for SEP-21: Samza Async API for High Level > > > SEP-21 has been discussed and implemente

[GitHub] [samza] dxichen commented on a change in pull request #974: SAMZA-2147: Improve the documentation of the samza release process.

2019-03-25 Thread GitBox
dxichen commented on a change in pull request #974: SAMZA-2147: Improve the documentation of the samza release process. URL: https://github.com/apache/samza/pull/974#discussion_r268911456 ## File path: RELEASE.md ## @@ -103,48 +127,54 @@ from reading keyboard input (becaus

[GitHub] [samza] dxichen commented on a change in pull request #974: SAMZA-2147: Improve the documentation of the samza release process.

2019-03-25 Thread GitBox
dxichen commented on a change in pull request #974: SAMZA-2147: Improve the documentation of the samza release process. URL: https://github.com/apache/samza/pull/974#discussion_r268911624 ## File path: RELEASE.md ## @@ -22,78 +22,102 @@ Before you start, here are a few pre

[GitHub] [samza] shanthoosh opened a new pull request #975: SAMZA-2148: Fix the rendering of bash and sql scripts in samza-sql release validation page.

2019-03-25 Thread GitBox
shanthoosh opened a new pull request #975: SAMZA-2148: Fix the rendering of bash and sql scripts in samza-sql release validation page. URL: https://github.com/apache/samza/pull/975 Currently, the bash/sql scripts available in samza-sql release validation page were not using the proper mar

[GitHub] [samza] shanthoosh commented on issue #975: SAMZA-2148: Fix the rendering of bash and sql scripts in samza-sql release validation page.

2019-03-25 Thread GitBox
shanthoosh commented on issue #975: SAMZA-2148: Fix the rendering of bash and sql scripts in samza-sql release validation page. URL: https://github.com/apache/samza/pull/975#issuecomment-476437090 @srinipunuru Please take a look when you have a chance. I've attached the screenshot of the

[GitHub] [samza] shanthoosh edited a comment on issue #975: SAMZA-2148: Fix the rendering of bash and sql scripts in samza-sql release validation page.

2019-03-25 Thread GitBox
shanthoosh edited a comment on issue #975: SAMZA-2148: Fix the rendering of bash and sql scripts in samza-sql release validation page. URL: https://github.com/apache/samza/pull/975#issuecomment-476437090 @srinipunuru @atoomula Please take a look when you have a chance. I've attach

[GitHub] [samza] shanthoosh commented on a change in pull request #974: SAMZA-2147: Improve the documentation of the samza release process.

2019-03-25 Thread GitBox
shanthoosh commented on a change in pull request #974: SAMZA-2147: Improve the documentation of the samza release process. URL: https://github.com/apache/samza/pull/974#discussion_r268913484 ## File path: RELEASE.md ## @@ -103,48 +127,54 @@ from reading keyboard input (bec

[GitHub] [samza] shanthoosh commented on a change in pull request #974: SAMZA-2147: Improve the documentation of the samza release process.

2019-03-25 Thread GitBox
shanthoosh commented on a change in pull request #974: SAMZA-2147: Improve the documentation of the samza release process. URL: https://github.com/apache/samza/pull/974#discussion_r268913522 ## File path: RELEASE.md ## @@ -22,78 +22,102 @@ Before you start, here are a few

[GitHub] [samza] shanthoosh edited a comment on issue #975: SAMZA-2148: Fix the rendering of bash and sql scripts in samza-sql release validation page.

2019-03-25 Thread GitBox
shanthoosh edited a comment on issue #975: SAMZA-2148: Fix the rendering of bash and sql scripts in samza-sql release validation page. URL: https://github.com/apache/samza/pull/975#issuecomment-476437090 @srinipunuru @atoomula Please take a look when you have a chance. I've attach

[GitHub] [samza] shanthoosh merged pull request #974: SAMZA-2147: Improve the documentation of the samza release process.

2019-03-25 Thread GitBox
shanthoosh merged pull request #974: SAMZA-2147: Improve the documentation of the samza release process. URL: https://github.com/apache/samza/pull/974 This is an automated message from the Apache Git Service. To respond to t

[GitHub] [samza] shanthoosh merged pull request #975: SAMZA-2148: Fix the rendering of bash and sql scripts in samza-sql release validation page.

2019-03-25 Thread GitBox
shanthoosh merged pull request #975: SAMZA-2148: Fix the rendering of bash and sql scripts in samza-sql release validation page. URL: https://github.com/apache/samza/pull/975 This is an automated message from the Apache Git

[GitHub] [samza] vjagadish1989 commented on a change in pull request #952: Improved standby-aware container allocation for active-containers on job redeploys

2019-03-25 Thread GitBox
vjagadish1989 commented on a change in pull request #952: Improved standby-aware container allocation for active-containers on job redeploys URL: https://github.com/apache/samza/pull/952#discussion_r268930336 ## File path: samza-core/src/main/java/org/apache/samza/clustermanager/St

[GitHub] [samza] vjagadish1989 commented on a change in pull request #952: Improved standby-aware container allocation for active-containers on job redeploys

2019-03-25 Thread GitBox
vjagadish1989 commented on a change in pull request #952: Improved standby-aware container allocation for active-containers on job redeploys URL: https://github.com/apache/samza/pull/952#discussion_r268927712 ## File path: samza-core/src/main/java/org/apache/samza/clustermanager/St