Re: Is this a decent use case for Kafka Streams?

Unf this notion isn't applicable: "...At the end of a time window..." If you comb through the archives of this group you'll see many questions about notifications for the 'end of an aggregation window' and a similar number of replies from the Kafka group stating that such a notion doesn't really e

network errors?

What do these messages mean: WARN kafka.network.Processor - Attempting to send response via channel for which there is no open connection, connection id 2

"... since it is no longer fetchable"

Attempting to run a KStream app and seeing lots of this sort of error message: > Resuming partition -# > Pausing partition -# > Not returning fetched records for assigned partition -# since it is no longer fetchable This cycles through all the partitions. It seems to get _some_ data from the top

joining two windowed aggregations

I want to collect data in two windowed groups - 4 hours with a one hour overlap and a 5 minute / 1 minute. I want to compare the values in the _oldest_ window for each group. Seems like this would be a standard join operation but Im not clear on how to limit which window the join operates on. I co

kafka consumers as kubernetes pods - best practice

Im looking for suggestions as to how to manage k-consumers when they are run as kubernetes pods - especially in an auto-scaling environment. Looking at the output of our logging it seems like we're dropping data when a pod is moved between hosts despite doing (what I believe is) an orderly shutdown

Joining on non-keyed values - how to lookup fields

Id like to further my immersion in kafka-as-database by doing more extensive key/val joins. Specifically there are many instances in the DB world where one is given a numeric field and needs to lookup the appropriate string translation / value. Imagine a record of student/class data where al the co

Re: Understanding ReadOnlyWindowStore.fetch

there configuration settings? Some properties file to setup RocksDB? Im not getting any errors - just not getting any data. On Wed, Mar 29, 2017 at 12:52 PM, Jon Yeargers wrote: > So '.until()' is based on clock time / elapsed time (IE record age) / > something else? > > The

Re: Understanding ReadOnlyWindowStore.fetch

based on > TimeWindows.until(..). > > Thanks, > Damian > > On Wed, 29 Mar 2017 at 17:24 Jon Yeargers > wrote: > > > To be a bit more specific: > > > > If I call this: KTable kt = > > sourceStream.groupByKey().reduce(..., "somekeystore");

Re: Understanding ReadOnlyWindowStore.fetch

pect to find there? On Wed, Mar 29, 2017 at 8:34 AM, Jon Yeargers wrote: > But if a key shows up in a KTable->forEach should it be available in the > StateStore (from the KTable)? > > On Wed, Mar 29, 2017 at 6:31 AM, Michael Noll > wrote: > >> Jon, >> >> t

Re: Understanding ReadOnlyWindowStore.fetch

examples/blob/3. > 2.x/kafka-streams/src/test/java/io/confluent/examples/streams/ > ValidateStateWithInteractiveQueriesLambdaIntegrationTest.java > (this is for Confluent 3.2 / Kafka 0.10.2). > > -Michael > > > > On Wed, Mar 29, 2017 at 3:12 PM, Jon Yeargers > wrot

Re: Understanding ReadOnlyWindowStore.fetch

ams.metadataForKey(...) to find the instance that has the key you > are looking for. > > Thanks, > Damian > > > > On Tue, 28 Mar 2017 at 22:37 Jon Yeargers > wrote: > > > Im probing about trying to find a way to solve my aggregation -> db > issue. > &g

Understanding ReadOnlyWindowStore.fetch

Im probing about trying to find a way to solve my aggregation -> db issue. Looking at the '.fetch()' function Im wondering about the 'timeFrom' and 'timeTo' params as not a lot is mentioned about 'proper' usage. The test in https://github.com/confluentinc/examples/blob/master/kafka-streams/src/te

WindowStore and retention

How long does a given value persist in a WindowStore? Does it obey the '.until()' param of a windowed aggregation/ reduction? Please say yes.

using a state store for deduplication

Ive been (re)reading this document( http://docs.confluent.io/3.2.0/streams/developer-guide.html#state-stores) hoping to better understand StateStores. At the top of the section there is a tantalizing note implying that one could do deduplication using a store. At present we using Redis for this as

Iterating stream windows

Im hoping to support external queries into a windowed state store aggregator. Thanks to a previous question here I see where to use a ReadOnlyWindowStore but Im not clear on how to define the boundaries for the call. Assumie I have a one hour window with a 5 minute 'slide' between new windows. If

Re: YASSQ (yet another state store question)

Also - if I run this on two hosts - what does it imply if the response to 'streams.allMetadata()' from one host includes both instances but the other host only knows about itself? On Sun, Mar 26, 2017 at 5:58 AM, Jon Yeargers wrote: > If the '.state()' function returns &q

Re: YASSQ (yet another state store question)

ted$ > 20to$20another$20instance%7Csort:relevance/confluent- > platform/LglWC_dZDKw/qsPuCRT_DQAJ <https://groups.google.com/ > forum/?pli=1#!searchin/confluent-platform/migrated$ > 20to$20another$20instance|sort:relevance/confluent- > platform/LglWC_dZDKw/qsPuCRT_DQAJ>. > > Tha

YASSQ (yet another state store question)

I've setup a KTable as follows: KTable, String> outTable = sourceStream.groupByKey(). reduce(rowReducer, TimeWindows.of(5 * 60 * 1000L).advanceBy(1 * 60 * 1000).until(10 * 60 * 1000L), "AggStore"); I can confirm its presence via 'streams.allMetadata()' (accessible

Re: APPLICATION_SERVER_CONFIG ?

he- > way-we-treat-data-and-services/ > that lists some of the reasoning behind this school of thinking. Again, > YMMV. > > Personally, I think there's no simple true/false here. The decisions > depend on what you need, what your context is, etc. Anyways, since you > alre

Re: APPLICATION_SERVER_CONFIG ?

g/unifying-stream-processing- > and-interactive-queries-in-apache-kafka/ > > Happy to answer any follow up question. > > > -Matthias > > On 3/23/17 11:51 AM, Jon Yeargers wrote: > > What does this config param do? > > > > I see it referenced / used in some samp

APPLICATION_SERVER_CONFIG ?

What does this config param do? I see it referenced / used in some samples and here ( https://cwiki.apache.org/confluence/display/KAFKA/KIP-67%3A+Queryable+state+for+Kafka+Streams )

Getting current value of aggregated key

If I have an aggregation : KTable, VideoLogLine> outTable = sourceStream.groupByKey().reduce(rowReducer, TimeWindows.of(60 * 60 * 1000L).until(10 * 60 * 1000L), "HourAggStore"); how would I go about getting some value from this with a separate process? I have the "

Re: clearing an aggregation?

of 3000ms have predictable window boundaries > `[0;5000),[3000;8000),...` > > — and not `[1000;6000),[4000;9000),...` or even something “random” like > > `[1452;6452),[4452;9452),...`. > > Would that help you? > > -Michael > > > > [1] http://docs.confluent.io

clearing an aggregation?

Is this possible? Im wondering about gathering data from a stream into a series of windowed aggregators: minute, hour and day. A separate process would start at fixed intervals, query the appropriate state store for available values and then hopefully clear / zero / reset everything for the next in

understanding consumer rebalance trigger(s)

Im wondering what the parameters are to instantiate a consumer rebalance. I have a topic that turns roughly 50K / minute across 6 partitions. Each is serviced by a separate dockerized consumer. Roughly every 8-12 min this goes into a rebalance that may take up to a minute. When it returns it often

Re: hitting the throughput limit on a cluster?

; only thing that is going to get you more performance is more spindles (or > faster spindles). This is either more disks or more brokers, but at the end > of it you need to eliminate the disk IO bottleneck. > > -Todd > > > On Tue, Feb 21, 2017 at 7:29 AM, Jon Yeargers > wro

hitting the throughput limit on a cluster?

Running 3x 8core on google compute. Topic has 16 partitions (replication factor 2) and is consumed by 16 docker containers on individual hosts. System seems to max out at around 4 messages / minute. Each message is ~12K - compressed (snappy) JSON. Recently moved from 12 to the above 16 parti

conflicts between consumer groups? seeing many duplicate records

Is it possible that using the same group name for two topics could cause a conflict? I have a situation where Im seeing vast numbers of records (more than 2x) get duplicated in a topic. I was looking at consumer lag using 'kafka-consumer-groups ... --new-consumer' and noticed that I had another ap

Re: KTable and cleanup.policy=compact

gt;> no records where there should be record. > >> > >> Mathieu > >> > >> > >> On Wed, Feb 8, 2017 at 12:15 PM, Eno Thereska > >> wrote: > >> > >>> If you fail to set the policy to compact, there shouldn't be any > &

Re: KTable and cleanup.policy=compact

What are the ramifications of failing to do this? On Tue, Feb 7, 2017 at 9:16 PM, Matthias J. Sax wrote: > Yes, that is correct. > > > -Matthias > > > On 2/7/17 6:39 PM, Mathieu Fenniak wrote: > > Hey kafka users, > > > > Is it correct that a Kafka topic that is used for a KTable should be set >

"auto offset commit failed"

This message seems to come and go for various consumers: WARN o.a.k.c.c.i.ConsumerCoordinator - Auto offset commit failed for group : Commit offsets failed with retriable exception. You should retry committing offsets. Since Im not tracking offsets - how would I go about retrying them?

Re: Understanding output of KTable->KTable join

e (unrelated to > GlobalKTable): https://issues.apache.org/jira/browse/KAFKA-3705 > > Maybe the discussion helps you do implement you own join. > > > -Matthias > > On 1/30/17 11:05 AM, Jon Yeargers wrote: > > I want to do a one:many join between two streams. Th

Understanding output of KTable->KTable join

I want to do a one:many join between two streams. There should be ~ 1:100 with < 1% having no match. My topology is relatively simple: KTable1.join(KTable2)->to("other topic") \ \---> toStream().print() In the join it takes both Value1 and Value2 as JSON, converts

Re: Strategy for true random producer keying

(cont'd) meant to say mod%partition count of System.currentTimeMillis(). Having said that - is there any disadvantage to true random distribution of traffic for a topic? On Tue, Jan 24, 2017 at 11:17 AM, Jon Yeargers wrote: > It may be picking a random partition but it sticks

Re: Strategy for true random producer keying

On Jan 24, 2017, at 11:18, Jon Yeargers > wrote: > > > > If I don't specify a key when I call send a value to kafka (something > akin > > to 'kafkaProducer.send(new ProducerRecord<>(TOPIC_PRODUCE, jsonView))') > how > > is it keyed? > >

Strategy for true random producer keying

If I don't specify a key when I call send a value to kafka (something akin to 'kafkaProducer.send(new ProducerRecord<>(TOPIC_PRODUCE, jsonView))') how is it keyed? I am producing to a topic from an external feed. It appears to be heavily biased towards certain values and as a result I have 2-3 par

Re: Messages are lost

Make sure you don't have an orphaned process holding onto the various kafka/zk folders. If it won't respond and you can't kill it then this might have happened. On Tue, Jan 24, 2017 at 6:46 AM, Ghosh, Achintya (Contractor) < achintya_gh...@comcast.com> wrote: > Can anyone please answer this? > >

Re: Consumer not associating? Perpetual rebalance

ommit", "true"); consumerProperties.put("auto.commit.interval.ms", "1000"); // this is the critical view (I think) consumerProperties.put("max.poll.records", "10"); On Tue, Jan 10, 2017 at 8:55 AM, Jon Yeargers wrote: > Single app wi

Consumer not associating? Perpetual rebalance

Single app with single consumer. Pulling ~30 records / min. When I enter 'kafka-topics ... --new-consumer --group --describe' it always tells me "Consumer group is rebalancing". If I enter "kafka-consumer-offset-checker ...--topic --group "it responds with appropriate consumer position(s) but

Re: What to do when it won't rebalance "properly"

On Sat, Jan 7, 2017 at 7:48 AM, Jon Yeargers wrote: > Have been messing about with Kubernetes on google-cloud. Launched a pod > with 6 consumer nodes and watched the lag using 'kafka-topics .. > --new-consumer --describe'. Topic has assigned all (12 in this case) nodes >

What to do when it won't rebalance "properly"

Have been messing about with Kubernetes on google-cloud. Launched a pod with 6 consumer nodes and watched the lag using 'kafka-topics .. --new-consumer --describe'. Topic has assigned all (12 in this case) nodes to the same consumer while the other 5 are sitting idle. This has been the case for ~2

0.10.2.0-SNAPSHOT - rocksdb exception(s)

2017-01-01 18:19:13,206 [StreamThread-1] ERROR o.a.k.c.c.i.ConsumerCoordinator - User provided listener org.apache.kafka.streams.processor.internals.StreamThread$1 for group RtDetailBreako utProcessor failed on partition assignment org.apache.kafka.streams.errors.ProcessorStateException: Error op

0.10.2.0-SNAPSHOT - "log end offset should not change while restoring"

java.lang.IllegalStateException: task [0_6] Log end offset of RtDetailBreakoutProcessor-table_stream-changelog-6 should not change while restoring: old end offset 26883455, current offset 2 6883467 at org.apache.kafka.streams.processor.internals.ProcessorStateManager.restoreActiveState(Pr

EOF exceptions - 0.10.2.0-SNAPSHOT

(Ive been testing against the latest from github as 0.10.1.1 is too buggy) Seeing quite a few of these this morning: 2017-01-01 16:56:53,299 [StreamThread-1] DEBUG o.a.kafka.common.network.Selector - Connection with / disconnected java.io.EOFException: null at org.apache.kafka.common.ne

Re: Interesting error message du jour

> -Ewen > > On Fri, Dec 30, 2016 at 9:45 AM, Jon Yeargers > wrote: > > > Attaching the debug log > > > > On Fri, Dec 30, 2016 at 6:39 AM, Jon Yeargers > > wrote: > > > >> Using 0.10.2.0-snapshot: > >> > >> java.la

Re: Interesting error message du jour

Attaching the debug log On Fri, Dec 30, 2016 at 6:39 AM, Jon Yeargers wrote: > Using 0.10.2.0-snapshot: > > java.lang.IllegalStateException: Incorrect state transition from > ASSIGNING_PARTITIONS to ASSIGNING_PARTITIONS > > at org.apache.kafka.streams.p

Interesting error message du jour

Using 0.10.2.0-snapshot: java.lang.IllegalStateException: Incorrect state transition from ASSIGNING_PARTITIONS to ASSIGNING_PARTITIONS at org.apache.kafka.streams.processor.internals.StreamThread.setState(StreamThread.java:163) at org.apache.kafka.streams.processor.internals.Stre

Re: Memory / resource leak in 0.10.1.1 release

means it is not relevant to your aggregate function. > > > Guozhang > > > On Sun, Dec 25, 2016 at 4:15 AM, Jon Yeargers > wrote: > > > I narrowed this problem down to this part of the topology (and yes, it's > > 100% repro - for me): > > >

Re: Memory / resource leak in 0.10.1.1 release

o find / fix this. On Thu, Dec 22, 2016 at 9:26 AM, Jon Yeargers wrote: > Yes - that's the one. It's 100% reproducible (for me). > > > On Thu, Dec 22, 2016 at 8:03 AM, Damian Guy wrote: > >> Hi Jon, >> >> Is this for the topology where

Re: Memory / resource leak in 0.10.1.1 release

aggregate(minute) -> foreach > \-> groupByKey.aggregate(hour) -> foreach > > I'm trying to understand how i could reproduce your problem. I've not seen > any such issues with 0.10.1.1, but then i'm not sure what you are doing. > > Thanks, > Damian > > On Thu, 22 Dec

Memory / resource leak in 0.10.1.1 release

Im still hitting this leak with the released version of 0.10.1.1. Process mem % grows over the course of 10-20 minutes and eventually the OS kills it. Messages like this appear in /var/log/messages: Dec 22 13:31:22 ip-172-16-101-108 kernel: [2989844.793692] java invoked oom-killer: gfp_mask=0x24

Re: spontaneous / unwanted exits from KStream apps

oom-killer: gfp_mask=0x24201ca, order=0, oom_score_adj=0 Some stack dump bits as well. This bug (or one v much like it) was fixed in the 0.10.2.0-SNAPSHOT I was previously using. The unhappy build comes from 0.10.1.1-RC1 and trunk. On Wed, Dec 21, 2016 at 4:32 AM, Jon Yeargers wrote: > Found

Re: spontaneous / unwanted exits from KStream apps

Found this treasure lurking in my app folder: Shows that the process was OOM-killed by the OS. Have restarted to see if it will reproduce - so far memory usage seems stable. On Wed, Dec 21, 2016 at 3:05 AM, Jon Yeargers wrote: > I upgraded my app(s) to 0.10.1.1-rc1 but I'm still seei

spontaneous / unwanted exits from KStream apps

I upgraded my app(s) to 0.10.1.1-rc1 but I'm still seeing these error-free shutdowns. Something is taking my app down after varying lengths of time (10 minutes to several hours). Doesn't matter if Im running one or many instances. Suggestions on where to look? I've sent several debug logs.

Re: effect of high IOWait on KStream app?

gt; I haven't tried this on AWS unfortunately so I don't know what to expect > there. > > Eno > > > > On 17 Dec 2016, at 15:39, Jon Yeargers wrote: > > > > stateDir=/tmp/kafka-streams-simple-benchmark > > > > numRecords=1000 > > > &

Re: checking consumer lag on KStreams app?

pped in releases following 0.9.0. Use > > >>>>>> ConsumerGroupCommand instead. (kafka.tools. > ConsumerOffsetChecker$) > > >>>>>> SLF4J: Class path contains multiple SLF4J bindings. > > >>>>>> SLF4J: Found binding in > > >>>>>> > > >>>>>> [jar:file:/home/testuser/kafka/kafka_2.10-0.10.0.1/ > > >>>>> libs/logback-classic-1.0.3.jar!/org/slf4j/impl/ > > >>> StaticLoggerBinder.class] > > >>>>>> SLF4J: Found binding in > > >>>>>> > > >>>>>> [jar:file:/home/testuser/kafka/kafka_2.10-0.10.0.1/ > > >>>>> libs/slf4j-log4j12-1.7.21.jar!/org/slf4j/impl/ > > >>> StaticLoggerBinder.class] > > >>>>>> SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for > > >> an > > >>>>>> explanation. > > >>>>>> SLF4J: Actual binding is of type > > >>>>>> [ch.qos.logback.classic.selector.DefaultContextSelector] > > >>>>>> Exiting due to: org.apache.zookeeper.KeeperException$ > > >>> NoNodeException: > > >>>>>> KeeperErrorCode = NoNode for /consumers/test/owners. > > >>>>>> > > >>>>>> Please let me know where I may be going wrong. > > >>>>>> I have the kafka logs set in folder > > >>>>>> /data01/testuser/kafka-logs > > >>>>>> > > >>>>>> Under kafka-logs I see many folders with name something like > > >>>>>> consumer_offsets_* > > >>>>>> > > >>>>>> I have the stream dir set in folder > > >>>>>> /data01/testuser/kafka-streams/test > > >>>>>> > > >>>>>> Thanks > > >>>>>> Sachin > > >>>>>> > > >>>>>> > > >>>>>> On Sun, Dec 11, 2016 at 2:19 AM, Matthias J. Sax < > > >>>> matth...@confluent.io> > > >>>>>> wrote: > > >>>>>> > > >>>>>>> It's basically just a consumer as any other. The application.id > > >> is > > >>>>> used > > >>>>>>> as consumer group.id. > > >>>>>>> > > >>>>>>> So just use the available tools you do use to check consumer lag. > > >>>>>>> > > >>>>>>> > > >>>>>>> -Matthias > > >>>>>>> > > >>>>>>> On 12/9/16 5:49 PM, Jon Yeargers wrote: > > >>>>>>>> How would this be done? > > >>>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > > > > > > >

Re: effect of high IOWait on KStream app?

fetching metadata with correlation id 1 : {joinSourceTopic2kTableKTable=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient:709) Streams KTableKTable LeftJoin Performance [MB/s joined]: 6.530348031376133 On Sat, Dec 17, 2016 at 6:39 AM, Jon Yeargers wrote: > I'd be happy to but the

Re: effect of high IOWait on KStream app?

rs? The benchmark is called SimpleBenchmark.java, it's > included with the source, and it will start a couple of streams apps. It > requires a ZK and a broker to be up. Then you run it: > org.apache.kafka.streams.perf.SimpleBenchmark > . > > Thanks > Eno > > On 16 Dec 2016,

effect of high IOWait on KStream app?

Looking for reasons why my installations seem to be generating so many issues: Starting an app which is stream->aggregate->filter->foreach While it's running the system in question (AWS) averages >10% IOWait with spikes to 60-70%. The CPU load is in the range of 3/2/1 (8 core machine w/ 16G RAM

Re: What makes a KStream app exit?

nly be a few > days until the fixes get merged. > > > -Matthias > > On 12/16/16 9:10 AM, Jon Yeargers wrote: > > Have started having this issue with another KStream based app. Digging > > through logs I ran across this message: > > > > When I've seen it

Re: What makes a KStream app exit?

hread [StreamThread-1] Shutting down 2016-12-16 17:04:51,821 [StreamThread-1] DEBUG o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] shutdownTasksAndState: shutting down all active tasks [[0_1, 0_2, 1_1, 0_5, 0_6]] and standby tasks [[]] On Fri, Dec 16, 2016 at 4:53 AM, Jon Yeargers

Re: What makes a KStream app exit?

ly... I just can't seem to see it. On Fri, Dec 16, 2016 at 2:10 AM, Jon Yeargers wrote: > Im seeing instances where my apps are exiting (gracefully, mind you) > without any obvious errors or cause. I have debug logs from many instances > of this and have yet to find a reason to expla

What makes a KStream app exit?

Im seeing instances where my apps are exiting (gracefully, mind you) without any obvious errors or cause. I have debug logs from many instances of this and have yet to find a reason to explain what's happening. - nothing in the app log - nothing in /var/log/messages (IE not OOM killed) - not being

"support" topics for KStreams

What's the retention settings for these (-changelog and -replication)? Im wondering about the relentless rebalancing issues Im facing and wondering if it has anything to do with consumers that lag too far behind. If I delete all the topics associated with a KStream project and restart it there are

RocksDB - no locks available exception

Attached is a debug log showing this exception. Question: is it typical to have so many disconnections from brokers? This log also includes the exception "Log end offset should not change while restoring" errors.log.gz Description: GNU Zip compressed data

Re: Another odd error

AM, Jon Yeargers wrote: > In a turn of events - this morning I was about to throw in the proverbial > towel on Kafka. In a last ditch effort I killed all but one instance of my > app, put it back to a single thread (why offer the option if it's not > advised?) and deleted every la

Re: Another odd error

ne thread per instance of the app. However, it should also > work with multiple threads. > I can't debug the problem any further without the logs from the other apps. > We'd need to try and see if another instance still has task 1_3 open ( i > suspect it does ) > > T

Re: Another odd error

What should I do about this? One thread per app? On Wed, Dec 14, 2016 at 4:11 AM, Damian Guy wrote: > That is correct > > On Wed, 14 Dec 2016 at 12:09 Jon Yeargers > wrote: > > > I have the app running on 5 machines. Is that what you mean? > > > > On Wed, De

Re: Another odd error

ception might indicate that another thread/instance is still writing to > the changelog topic for the State Store. > > Thanks, > Damian > > On Tue, 13 Dec 2016 at 17:23 Jon Yeargers > wrote: > > > As near as I can see it's rebalancing constantly. > > > >

Re: How does 'TimeWindows.of().until()' work?

owstore.changelog.additional.retention.ms > >>> I is not clear what is does. Is this the default for until? > >>> > >>> Thanks > >>> Sachin > >>> > >>> > >>> On Mon, Dec 12, 2016 at 10:17 AM, Matthias J. Sax < >

reasonable KStream app config settings?

My app seems to be continuously rebalancing. If I said it processed data maybe 3 minutes / hour I wouldn't be exaggerating. Surely this isn't normal behavior. My config is: config.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, BROKER_IP); config.put(StreamsConfig.ZOOKEEPER_CONNECT_CONFIG, ZO

Re: Another odd error

er than 30 (which is the default). > > I'll continue to look at your logs and get back to you. > Thanks, > Damian > > On Tue, 13 Dec 2016 at 15:02 Jon Yeargers > wrote: > > > attached is a log with lots of disconnections and a small amount of > > actual, u

KStreams app - frequent broker dis/re connects

Watching the debug output on an app - wondering why it spends nearly all of its time rebalancing. Noticed that it seems to drop / recreate connections to brokers pretty frequently. No error messages to speak of though. Connect / timeout / related settings in the consumer are all default. How much

Re: Another odd error

n/m - I understand the logging issue now. Am generating a new one. Will send shortly. On Tue, Dec 13, 2016 at 4:55 AM, Jon Yeargers wrote: > Yes - saw that one. There were plenty of smaller records available though. > > I sent another log this morning with the level set to DEBUG. Hope

Re: Another odd error

r state-stores are being > dropped due to this issue. > > Thanks, > Damian > > > On Tue, 13 Dec 2016 at 11:32 Jon Yeargers > wrote: > > > (am attaching a debug log - note that app terminated with no further > > messages) > > > > topology:

Re: Another odd error

ur app configs? For example did you enable caching in your > apps with the cache.max.bytes.buffering config? > > > Guozhang > > > On Sun, Dec 11, 2016 at 3:44 PM, Jon Yeargers > wrote: > > > I get this one quite a bit. It kills my app after a short time of > running. > > Dri

Re: "Log end offset should not change while restoring"

is related to this caching problem, disabling the cache > via StreamsConfig should fix the problem for now. Just set the cache > size to zero. > > > -Matthias > > > On 12/12/16 2:31 AM, Jon Yeargers wrote: > > Im seeing this error occur more frequently

Re: rocksdb error(s)

Thanks > > On Mon, 12 Dec 2016 at 12:47 Jon Yeargers > wrote: > > > I can log whatever you need. Tell me what is useful. > > > > On Mon, Dec 12, 2016 at 4:43 AM, Damian Guy > wrote: > > > > > If you provide the logs from your streams application then

Re: rocksdb error(s)

diagnosing the problem. > > On Mon, 12 Dec 2016 at 12:18 Jon Yeargers > wrote: > > > Im running as many threads as I have partitions on this topic. Just > curious > > if it would make any difference to the seemingly endless rebalancing > woes. > > > > So

Re: rocksdb error(s)

, Dec 12, 2016 at 4:15 AM, Jon Yeargers wrote: > At this moment I have 5 instances each running 2 threads. > Single instance / machine. > > Define 'full logs' ? > > On Mon, Dec 12, 2016 at 3:54 AM, Damian Guy wrote: > >> Jon, >> >> How many Stre

Re: rocksdb error(s)

one instance per machine? If yes, are they sharing > the same State Directory? > Do you have full logs that can be provided so we can try and see how/what > is happening? > > Thanks, > Damian > > On Mon, 12 Dec 2016 at 10:17 Jon Yeargers > wrote: > > >

partition count multiples - adverse effects on rebalancing?

Just curious - how is rebalancing handled when the number of potential consumer threads isn't a multiple of the number of partitions? IE If I have 9 partitions and 6 threads - will the cluster be forever trying to balance this?

doing > 1 'parallel' operation on a stream

If I want to aggregate a stream twice using different windows do I need to split / copy / duplicate the source stream somehow? Or will this be handled without my interference?

"Log end offset should not change while restoring"

Im seeing this error occur more frequently of late. I ran across this thread: https://groups.google.com/forum/#!topic/confluent-platform/AH5QClSNZBw The implication from the thread is that a fix is available. Where can I get it?

Re: rocksdb error(s)

No luck here. Moved all state storage to a non-tmp folder and restarted. Still hitting the 'No locks available' error quite frequently. On Sun, Dec 11, 2016 at 3:45 PM, Jon Yeargers wrote: > I moved the state folder to a separate drive and linked out to it. > > I'll

Re: rebalancing - how to speed it up?

' between all 5 instances. On Sun, Dec 11, 2016 at 6:04 PM, Jon Yeargers wrote: > I changed 'num.standby.replicas' to '2'. > > I started one instance and it immediately showed up in the > 'kafka-consumer-groups .. --describe' listing. > > So I s

Re: rebalancing - how to speed it up?

> something is wrong with zookeeper or you're whole setup maybe. if it > > becomes an unsolvable issue for you, you could try > > https://github.com/gerritjvv/kafka-fast which uses a different model and > > doesn't need balancing or rebalancing. > > > > disclojure:

Re: How does 'TimeWindows.of().until()' work?

ook here: > > http://docs.confluent.io/current/streams/developer- > guide.html#windowing-a-stream > > If you have further question, just follow up :) > > > -Matthias > > > On 12/10/16 6:11 PM, Jon Yeargers wrote: > > Ive added the 'until()' clause t

Re: rocksdb error(s)

ocated in /tmp -- could it be, that > /tmp gets clean up and thus you loose files/directories? > > Try to reconfigure your state directory via StreamsConfig: > http://docs.confluent.io/current/streams/developer-guide.html#optional- > configuration-parameters > > > -Matthias

Re: Another odd error

stigate this. > > -Matthias > > On 12/10/16 4:17 PM, Jon Yeargers wrote: > > (Am reporting these as have moved to 0.10.1.0-cp2) > > > > ERROR o.a.k.c.c.i.ConsumerCoordinator - User provided listener > > org.apache.kafka.streams.processor.internals.StreamThread$1 for gr

Re: odd error message

Yes- but not 100% repro. I seem to have several issues with start / rebalance On Sun, Dec 11, 2016 at 2:16 PM, Matthias J. Sax wrote: > Hi, > > this might be a recently discovered bug. Does it happen when you > stop/restart your application? > > > -Matthias > > On 12/

rebalancing - how to speed it up?

Is there some way to 'help it along'? It's taking an hour or more from when I start my app to actually seeing anything consumed. Plenty of CPU (and IOWait) during this time so I know it's doing _something_...

rocksdb error(s)

Seeing this appearing somewhat frequently - org.apache.kafka.streams.errors.ProcessorStateException: Error opening store minute_agg_stream-201612100812 at location /tmp/kafka-streams/MinuteAgg/1_9/minute_agg_stream/minute_agg_stream-201612100812 at org.apache.kafka.streams.state.internals

How does 'TimeWindows.of().until()' work?

Ive added the 'until()' clause to some aggregation steps and it's working wonders for keeping the size of the state store in useful boundaries... But Im not 100% clear on how it works. What is implied by the '.until()' clause? What determines when to stop receiving further data - is it clock time

Another odd error

(Am reporting these as have moved to 0.10.1.0-cp2) ERROR o.a.k.c.c.i.ConsumerCoordinator - User provided listener org.apache.kafka.streams.processor.internals.StreamThread$1 for group MinuteAgg failed on partition assignment java.lang.IllegalStateException: task [1_9] Log end offset should not c

odd error message

This came up a few times today: 2016-12-10 18:45:52,637 [StreamThread-1] ERROR o.a.k.s.p.internals.StreamThread - stream-thread [StreamThread-1] Failed to create an active task %s: org.apache.kafka.streams.errors.ProcessorStateException: task [0_0] Error while creating the state manager

'swap' space for KStream app - limitations?

Are there any? My app ran for a few hours and filled a 100G partition (on 5 machines). Any settings to keep this growth in check? Perhaps to estimate how much space it's going to need?

checking consumer lag on KStreams app?

How would this be done?

Re: controlling memory growth when aggregating

) at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:407) at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:242) On Fri, Dec 9, 2016 at 4:29 PM, Jon Yeargers wrote: > Perhaps that's the problem. Yes - I

Re: controlling memory growth when aggregating

; org.apache.kafka > kafka-streams > 0.10.1.0-cp2 > org.apache.kafka > kafka-clients > 0.10.1.0-cp2 > > > On Thu, 8 Dec 2016 at 23:37 Jon Yeargers wrote: > > I working with JSON data that has an array member. Im aggregating values > into this

  1   2   >