GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/1176
MINOR: small code optimizations in streams
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka optimize
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/1164
MINOR: a simple benchmark for Streams
@guozhangwang @miguno
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka perf
Alternatively
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/1163
HOTFIX: RocksDBStore must clear dirty flags after flush
@guozhangwang
Without clearing the dirty flags, RocksDBStore will perform flush for every
new record. This bug made the store
Github user ymatsuda closed the pull request at:
https://github.com/apache/kafka/pull/1161
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/1161
HOTFIX: stop using batch write to rocksdb
@guozhangwang
Removing batch write to RocksDB because I observed segmentation faults in
RocksDB's batch write.
You can merge this pull re
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/1137
HOTFIX: set timestamp in SinkNode
@guozhangwang
Setting the timestamp in produced records in SinkNode. This forces the
producer record's timestamp same as the context's timestamp
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/1125
MINOR: remove streams-smoke-test.sh
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka remove_smoketest_shell_script
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/1124
HOTFIX: fix NPE in changelogger
Fix NPE in StoreChangeLogger caused by a record out of window retention
period.
@guozhangwang
You can merge this pull request into a Git repository by running
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/1065
MINOR: kstream/ktable counting method with default long serdes
@guozhangwang @miguno
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/1062
KAFKA-3395: prefix job id to internal topic names
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka k3395
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/985
MINOR: add AUTO_OFFSET_RESET_CONFIG to StreamsConfig,
and remove TOTAL_RECORDS_TO_PROCESS
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/966
MINOR: enhance streams system test
@guozhangwang
* add table aggregate to the system test
* actually create change log partition replica
You can merge this pull request into a Git
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/959
HOTFIX: fix consumer config for streams
@guozhangwang
My bad. I removed ZOOKEEPER_CONNECT_CONFIG from consumer's config by
mistake. It is needed by our own partition assigner runni
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/948
KAFKA-3245: config for changelog replication factor
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/947
HOTFIX: check offset limits in streamtask when recovering KTable store
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/933
MINOR: catch a commit failure due to rebalance
StreamThread should keep going after a commit was failed due to a group
rebalance.
Currently the thread just dies.
@guozhangwang
You can
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/930
KAFKA-2802: kafka streams system tests
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka streams_systest
Alternatively you can
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/928
HOTFIX: make sure to go through all shutdown steps
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka shutdown
Alternatively you
Github user ymatsuda closed the pull request at:
https://github.com/apache/kafka/pull/925
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/925
HOTFIX: release resources on abrupt termination of stream threads
Currently the resources, such as the state dir locks, are not release when
a stream thread is abruptly terminated
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/906
MINOR: remove streams config params from producer/consumer configs
Removing streams' specific config params from producer/consumer configs to
reduce warning messages.
You can merge this
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/901
MINOR: catch an exception in rebalance and stop the stream thread
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka minor3
Github user ymatsuda closed the pull request at:
https://github.com/apache/kafka/pull/794
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/899
MINOR: add retry to state dir locking
There is a possibility that the state directory locking fails when another
stream thread is taking long to close all tasks. Simple retries should
alleviate
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/894
MINOR: add setUncaughtExceptionHandler to KafkaStreams
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka minor
Alternatively you
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/893
HOTFIX: poll even when all partitions are paused. handle concurrent cleanup
* We need to poll periodically even when all partitions are paused in order
to respond to a possible rebalance promptly
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/891
HOTFIX: open window segments in order, add segment id check in getSegment
* During window store initialization, we have to open segments in the
segment id order and update ```currentSegmentId
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/889
HOTFIX: fix NPE after standby task reassignment
Buffered records of change logs must be cleared upon reassignment of
standby tasks.
You can merge this pull request into a Git repository by
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/887
HOTFIX: RecordCollector should send a record to the specified partition
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/886
HOTFIX: open window segments on init
@guozhangwang
A window store should open all existing segments. This is important for
segment cleanup, and it also ensures that the first fetch() call
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/883
MINOR: fix RocksDBStore range search
The range is inclusive according to KeyValueStore's java doc.
You can merge this pull request into a Git repository by running:
$ git pull
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/877
HOTFIX: fix streams issues
* RocksDBStore.putInternal should by pass logging.
* StoreChangeLogger should not call context.recordCollector() when nothing
to log
* This is for standby tasks
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/873
MINOR: more info in error msg
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka hotfix
Alternatively you can
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/868
HOTFIX: fix partition ordering in assignment
workround partition ordering not preserved by the consumer group management.
@guozhangwang
You can merge this pull request into a Git repository by
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/864
HOTFIX: temp fix for ktable look up
@guozhangwang
Temporarily disabled state store access checking.
You can merge this pull request into a Git repository by running:
$ git pull https
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/844
MINOR: some javadocs for kstream public api
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka javadoc
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/843
MINOR: removed obsolete class
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka remove_unused
Alternatively you can
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/834
MINOR: remove the init method from Serdes
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka remove_init_from_Serdes
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/816
MINOR: remove FilteredIterator
@guozhangwang
removing an unused class, FilteredIterator, and its test.
You can merge this pull request into a Git repository by running:
$ git pull https
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/814
MINOR: jointest for windowed keys
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka windowed_key_join_test
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/808
MINOR: add equals and hashCode to Windowed
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka windowed_key
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/794
WIP: KStream,Type and Serialization
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka type_and_serdes
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/779
KAFKA-3108: custom StreamParitioner for Windowed key
@guozhangwang
When ```WindowedSerializer``` is specified in ```to(...)``` or
```through(...)``` for a key, we use
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/737
KAFKA-3016: phase-2. stream join implementations
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka windowed_join2
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/726
KAFKA-3016: phase-1. A local store for join window
@guozhangwang
An implementation of local store for join window. This implementation uses
"rolling" of RocksDB instances for times
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/680
MINOR: StreamThread performance optimization
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka perf
Alternatively
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/672
KAFKA-2984: ktable sends old values when required
@guozhangwang
At DAG level, `KTable` sends (key, (new value, old value)) to down
stream. This is done by wrapping the new value and the
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/661
MINOR: test ktable state store creation
@guozhangwang
* a test for ktable state store creation
You can merge this pull request into a Git repository by running:
$ git pull https
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/653
HOTFIX: fix table-table outer join and left join. more tests
@guozhangwang
* fixed bugs in table-table outer/left joins
* added more tests
You can merge this pull request into a Git
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/644
KAFKA-2962: stream-table table-table joins
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka join_methods
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/635
HOTFIX: fix ProcessorStateManager to use correct ktable partitions
@guozhangwang
* fix ProcessorStateManager to use correct ktable partitions
* more ktable tests
You can merge this
GitHub user ymatsuda reopened a pull request:
https://github.com/apache/kafka/pull/604
KAFKA-2856: add ktable
@guozhangwang
* added KTable API and impl
* added standby support for KTable
You can merge this pull request into a Git repository by running:
$ git pull
Github user ymatsuda closed the pull request at:
https://github.com/apache/kafka/pull/604
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/604
KAFKA-2856: add ktable
@guozhangwang
* added KTable API and impl
* added standby support for KTable
You can merge this pull request into a Git repository by running:
$ git pull https
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/591
MINOR: comments on KStream methods, and fix generics
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka comments
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/589
MINOR: initialize Serdes with ProcessorContext
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/587
MINOR: change KStream processor names
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka kstream_processor_names
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/586
HOTFIX: fix StreamTask.close()
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka fix_streamtask_close
Alternatively
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/543
MINOR: remove the group id from a restore consumer
@guozhangwang
A restore consumer does not belong to a consumer group.
You can merge this pull request into a Git repository by running
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/536
MINOR: add KStream merge operator
@guozhangwang
Added KStreamBuilder.merge(KStream...).
You can merge this pull request into a Git repository by running:
$ git pull https
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/535
MINOR: do not create a StandbyTask if there is no state store in the task
@guozhangwang
An optimization which may reduce unnecessary poll for standby tasks.
You can merge this pull request
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/526
KAFKA-2811: add standby tasks
@guozhangwang
* added a new config param "num.standby.replicas" (the default value is 0).
* added a new abstract class AbstractTask
* added Stan
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/497
KAFKA-2763: better stream task assignment
@guozhangwang
When the rebalance happens each consumer reports the following information
to the coordinator.
* Client UUID (a unique id
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/411
KAFKA-2727: Topology partial construction
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/408
KAFKA-2707: make KStream processor names deterministic
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/387
KAFKA-2706: make state stores first class citizens in the processor topology
* Added StateStoreSupplier
* StateStore
* Added init(ProcessorContext context) method
* TopologyBuilder
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/374
HOTFIX: correct sourceNodes for kstream.through()
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/373
HOTFIX: call consumer.poll() even when no task is assigned
StreamThread should keep calling consumer.poll() even when no task is
assigned. This is necessary to get a task.
@guozhangwang
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/372
HOTFIX: fix off-by-one stream offset commit
@guozhangwang
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ymatsuda/kafka commit_offset
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/365
KAFKA-2694: Task Id
@guozhangwang
* A task id is now a class, ```TaskId```, that has a task group id and a
partition id fields.
* ```TopologyBuilder``` assigns a task group id to a
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/353
KAFKA-2652: integrate new group protocol into partition grouping
@guozhangwang
* added ```PartitionGrouper``` (abstract class)
* This class is responsible for grouping partitions to
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/317
MINOR: set up temp directories properly in StreamTaskTest
@guozhangwang
StreamTaskTest did not set up a temp directory for each test. This rarely
caused interference between tests through
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/315
MINOR: optimize unnecessary poll(0) away
@guozhangwang
This change aims to remove unnecessary ```consumer.poll(0)``` calls.
* ```once``` after some partition is resumed
* whenever the
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/304
MINOR: flush record collector after local state flush
@guozhangwang
Fix the order of flushing. Undoing the change I did sometime ago.
You can merge this pull request into a Git repository by
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/292
MINOR: putting back kstream stateful transform methods
@guozhangwang
* added back type safe stateful transform methods (kstream.transform() and
kstream.transformValues())
* changed
GitHub user ymatsuda opened a pull request:
https://github.com/apache/kafka/pull/289
MINOR: typing ProcessorDef
@guozhangwang
This code change properly types ProcessorDef. This also makes
KStream.process() typesafe.
You can merge this pull request into a Git repository by
76 matches
Mail list logo
