Actually, I am doing joining after map. I need to map the keys, modify them
and then do a join.
I was thinking of using always passing a partition key based on which
partition happens.
Step by step flow is:-
1. Data is already partitoned by do userid.
2. I do a map to joins impressions tied to a u
Hi Kafka Team,
I am a beginner in using apache kafka service. I have installed kafka_2.11
on windows platform but i am facing following two issues
1] I want disk space to be free up after some interval of time once the
message is produced. For this i'v set log.retention for 1 hr. But when the
tim
I agree with Dmitry's first comment, it really looks like the paragraph he
points to under "Table" was pasted without edit from the one previously
that pertained to "KStream".
On Sun, Dec 17, 2017 at 5:31 PM, Dmitry Minkovsky
wrote:
> On https://docs.confluent.io/current/streams/developer-guide/
Also the javadoc here:
https://github.com/apache/kafka/blob/e5daa40e316261e8e6cb8866ad9a4eedcf17f919/streams/src/main/java/org/apache/kafka/streams/StreamsBuilder.java#L184-L185
Shouldn't it refer to the `Consumed`, given that it is provided in this
overload?
Sorry, I would post this to JIRA, but
On https://docs.confluent.io/current/streams/developer-guide/dsl-api.html
for version 4.0.0:
Under "Table", currently:
> In the case of a KStream, the local KStream instance of every application
instance will be populated with data from only a subset of the partitions
of the input topic. Collecti
Two comments:
1) As long, as you don't do an aggregation/join after a map(), there
will be not repartitioning. Streams does repartitioning "lazy", ie, only
if it's required. As long as you only chain filter/map etc, no
repartitioning will be done.
2) Can't you use mapValue() instead of map()? If
I have multiple map and filter phases in my application dag and though I am
generating different keys at different points, the data is still local.
Re-partitioning for me here is adding unnecessary network shuffling, I want
to minimize it.
-Sameer.
On Friday, December 15, 2017, Matthias J. Sax w
Hi
Not sure of adminClient but if you are programming is Java this should be
possible by using KafkaConsumer class
org.apache.kafka.clients.consumer.KafkaConsumer
It has beginningOffsets and endOffSets methods , that can give you the
information.
regards
On Thu, Dec 14, 2017 at 11:10 PM, 懒羊羊 <1
Hi
Did you download the binary download or are you trying to build the source
code and then run ?
With binary downloads, I have never had an issue.
Another possibility is you have scala installed that is getting in the way.
regards
On Fri, Dec 15, 2017 at 1:54 PM, Karl Keller wrote:
> Hello,
