GitHub user weisong44 opened a pull request:
https://github.com/apache/samza/pull/387
Added document for table API to feature preview
Added document for table API to feature preview
- Brief description of table
- sendTo() operator for table
- join() operator for stream-table-join
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/weisong44/samza table-api-14
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/387.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #387
----
commit 085c7eaa22fa940ef6a1acc5e10a30386b42f791
Author: Wei Song <ws...@linkedin.com>
Date: 2017-10-06T00:33:51Z
Initial version of table API, it includes
- Core table API (Table, TableDescriptor, TableSpec)
- Local table implementation for in-memory and RocksDb
- The writeTo() and stream-table join operators
commit 2312f1f33f42886d8018050fd095c4d05bf352d6
Author: Wei Song <ws...@linkedin.com>
Date: 2017-11-02T00:50:34Z
Merged from master
commit edaddac8095cc9e28d8e64e712e8a0515471169c
Author: Wei Song <ws...@linkedin.com>
Date: 2017-11-21T06:15:16Z
Merged from master
commit 33c866e69f84ea62371061812a9acd134f29ba72
Author: Wei Song <ws...@linkedin.com>
Date: 2017-11-21T06:24:27Z
Updated table API based on review comments
- Added initialization phase and checks in getTable() to TableManager
- Moved write.batch.size and object.cache.size to
BaseStoreBackedTableDescriptor and various fixes
- Removed JSON conversion for TableSpec
- Converted table descriptor to use KVSerde
- Converted incoming message type for stream-table join to KV<K, V>
- Comtined Table and RecordTable
- Modifed stream termination method to sendTo() and payload to be KV
- Modified to support stream-table outer join
commit 02a422c009fdcdd43ff6b76d0edd71fad06f1ba9
Author: Wei Song <ws...@linkedin.com>
Date: 2017-11-22T21:52:03Z
Updated based on more review comments
- Renamed parameters in stream-table join to be more accurate
- Reduced methods to set serdes in TableDescriptor to one
- Refactored JavaTableConfig to add more accessors
- Removed closing stores in ReadableTable
- Removed deserialization of serdes in TableManager, instead pass them in
in ctor
commit e31e0403f3bd2bfbca25085037ba9180f86362c4
Author: Wei Song <ws...@linkedin.com>
Date: 2017-11-22T22:02:32Z
Merge remote-tracking branch 'upstream/master' into table-api-14
commit 3cb141f4992ad12303c7d718ea5aec8c9948809b
Author: Wei Song <ws...@linkedin.com>
Date: 2017-11-29T02:23:40Z
Updated based on review comments
- Changed KV to be table record type
- Added subclass for StreamTableJoinFunction for type safety
commit 0ac5fb6eea0ee5c2e0a5ef2d7088e9916611bc39
Author: Wei Song <ws...@linkedin.com>
Date: 2017-11-29T02:24:27Z
Merge remote-tracking branch 'upstream/master' into table-api-14
commit bb7c75f69cfd6e7a749a55c465a4db372abb2fd7
Author: Wei Song <ws...@linkedin.com>
Date: 2017-11-30T00:34:14Z
Updated based on more review comments
- Split TableDescriptor into interface and implementation
- Added unit test for dual table join scenario
- Modified stream-table join interface to be consistent with stream-stream
join
- Cosmetic fixes
commit a0e5a60f49963fe783770baa228338dfbc0161d9
Author: Wei Song <ws...@linkedin.com>
Date: 2017-11-30T00:38:15Z
Merge remote-tracking branch 'upstream/master' into table-api-14
commit 5d874f3fdb79f8684b47af4a516ceb8532c8c9b2
Author: Wei Song <ws...@linkedin.com>
Date: 2017-11-30T22:34:52Z
Updated based on review comments
- Updated testDualStreamTableJoin() in TestLocalTable
- Made KV class final
- Minor fixes
commit 8546bdf5ff1b9eb49d8dbecb1716ebbad3d3d93c
Author: Wei Song <ws...@linkedin.com>
Date: 2017-12-01T22:03:48Z
Updated based on review comments
commit 22d22db8ff6e01c510953c5b395d02ce0de352bc
Author: Wei Song <ws...@linkedin.com>
Date: 2017-12-01T22:04:16Z
Merge remote-tracking branch 'upstream/master' into table-api-14
commit 18059084fc46c72bdc3b2863dcf39e61fa39573b
Author: Wei Song <ws...@linkedin.com>
Date: 2017-12-01T23:12:49Z
Added getMessageKey() and getRecordKey() to stream-table join function
commit 826dcc4c54dedbd1d1bd647693aa7e867a2e3440
Author: Wei Song <ws...@linkedin.com>
Date: 2017-12-04T19:21:09Z
Updated class name store backed classes to include Local
commit c084f2be992e76a79bdab4fc84dc8ef0bb4eacbf
Author: Wei Song <ws...@linkedin.com>
Date: 2017-12-04T19:25:30Z
Merge remote-tracking branch 'upstream/master' into table-api-14
commit c23b72ab022eeee3d49edfe707ca06dfa54b4470
Author: Wei Song <ws...@linkedin.com>
Date: 2017-12-13T02:26:03Z
Merged from upstream master
commit 6c8fa02839fa5b4d60a458304e36c2a64276d6ce
Author: Wei Song <ws...@linkedin.com>
Date: 2017-12-13T21:41:04Z
Added table to feature preview documentation
commit e9f2ced43ac072e87240af563496b219feb746fe
Author: Wei Song <ws...@linkedin.com>
Date: 2017-12-13T21:42:34Z
Merge remote-tracking branch 'upstream/master' into table-api-14
----
---
