Re: Alerting and Monitoring Samza Checkpointing?

Hi Jeremiah, +1 to what Tom said. Samza currently does not rely on Kafka consumer's checkpointing behavior and exposes its own notion of a "lag". This is reported as a per-partition metric under KafkaSystemConsumerMetrics#messagesBehindHighWatermark

[GitHub] samza pull request #823: State restore improvements

GitHub user rmatharu opened a pull request: https://github.com/apache/samza/pull/823 State restore improvements This PR makes the following changes: * Consumer consolidation to ensure 1 storeConsumer per system, earlier it was 1 consumer per SSP per store. * Refactorin

Re: Alerting and Monitoring Samza Checkpointing?

I am referring to the "Lag" that can exist when a Consumer Offset is significantly less than the Log Size. This difference is Lag and is often symptomatic of a problem - processing has stopped or being overwhelmed etc. Our Legacy Node.js system uses Consumer Groups, (same as say older Spark).

[GitHub] samza pull request #822: SAMZA-2004: Add ability to disable table metrics

Github user asfgit closed the pull request at: https://github.com/apache/samza/pull/822 ---

[GitHub] samza pull request #822: SAMZA-2004: Add ability to disable table metrics

GitHub user weisong44 opened a pull request: https://github.com/apache/samza/pull/822 SAMZA-2004: Add ability to disable table metrics For jobs with very high throughput, it is desirable to disable metrics on tables. We would introduce the option to disable all metrics for a table o

[GitHub] samza pull request #813: SAMZA-2004: Add ability to disable table metrics

Github user weisong44 closed the pull request at: https://github.com/apache/samza/pull/813 ---

[GitHub] samza pull request #821: SAMZA-2007: Samza-sql - Fix Samza-SQL to not expect...

GitHub user atoomula opened a pull request: https://github.com/apache/samza/pull/821 SAMZA-2007: Samza-sql - Fix Samza-SQL to not expect LogicalTableModify in the Calcite plan. This is required for supporting schema evolution without failing the jobs. You can merge this pull reques

[GitHub] samza pull request #820: Ensure that DOC pages for older releases are easy t...

Github user asfgit closed the pull request at: https://github.com/apache/samza/pull/820 ---

[GitHub] samza pull request #820: Ensure that DOC pages for older releases are easy t...

GitHub user vjagadish1989 opened a pull request: https://github.com/apache/samza/pull/820 Ensure that DOC pages for older releases are easy to discover You can merge this pull request into a Git repository by running: $ git pull https://github.com/vjagadish1989/samza website-r

[GitHub] samza pull request #819: Use consistent font /heading sizes for all pages

GitHub user vjagadish1989 opened a pull request: https://github.com/apache/samza/pull/819 Use consistent font /heading sizes for all pages You can merge this pull request into a Git repository by running: $ git pull https://github.com/vjagadish1989/samza website-reorg34 Alter

[GitHub] samza pull request #819: Use consistent font /heading sizes for all pages

Github user asfgit closed the pull request at: https://github.com/apache/samza/pull/819 ---

[GitHub] samza pull request #818: Commit for website publish in 1.0.0

Github user vjagadish1989 closed the pull request at: https://github.com/apache/samza/pull/818 ---

[GitHub] samza pull request #818: Commit for website publish in 1.0.0

GitHub user vjagadish1989 opened a pull request: https://github.com/apache/samza/pull/818 Commit for website publish in 1.0.0 You can merge this pull request into a Git repository by running: $ git pull https://github.com/vjagadish1989/samza website-reorg33 Alternatively you

[GitHub] samza pull request #817: Clean up docs for standalone

Github user asfgit closed the pull request at: https://github.com/apache/samza/pull/817 ---

[GitHub] samza pull request #817: Clean up docs for standalone

GitHub user vjagadish1989 opened a pull request: https://github.com/apache/samza/pull/817 Clean up docs for standalone You can merge this pull request into a Git repository by running: $ git pull https://github.com/vjagadish1989/samza website-reorg32 Alternatively you can rev