Val, when we think about binary protocol, using java interface as an example is a not good idea. Please, see a tarantool binary api as an example. Implementor of the client can write it immediately (1) [1] -- https://www.tarantool.io/en/doc/latest/dev_guide/internals/box_protocol/
чт, 1 июл. 2021 г., 19:15 Valentin Kulichenko <valentin.kuliche...@gmail.com >: > Ivan, > > Regarding the API, please take a look at this package: > > https://github.com/apache/ignite-3/tree/main/modules/api/src/main/java/org/apache/ignite/table > > 'Table' is the primary API, which works with raw tuples. There are also > multiple views on top of it, including KeyValueView and KeyValueBinaryView > - these provide the key-value API. > > The ultimate goal is for the thin client to implement all these APIs. > > -Val > > On Thu, Jul 1, 2021 at 8:42 AM Igor Sapego <isap...@apache.org> wrote: > > > Ivan, what are extra serde steps you are talking about? > > > > Best Regards, > > Igor > > > > > > On Thu, Jul 1, 2021 at 5:52 PM Ivan Daschinsky <ivanda...@gmail.com> > > wrote: > > > > > > I agree. But this was decided before in IEP-54, and is out of scope > for > > > current IEP. > > > Would you like to start a separate thread to discuss this? Or I can do > > this > > > a bit later. > > > > > > Great idea, let's discuss it. I suppose this will simplify many aspects > > of > > > realization and improve performance a lot > > > > > > чт, 1 июл. 2021 г., 17:50 Ivan Daschinsky <ivanda...@gmail.com>: > > > > > > > > Here is the description of TUPLE_GET_ALL: > > > > - UUID: table ID > > > > - int: schema ID > > > > - arr of arr: array of rows with values for all columns in given > schema > > > > > > > > I suppose that we should describe this more verbose and explicit. I > > > > nevertheless suggest to also consider writing values this way: > > > > - arr of fields names (if name is missed, corresponding field is nil) > > > > - arr of rows (row as array, length equal to fields array) > > > > > > > > It is quite simple and if we use str8 (it is more than enough for any > > > > utf-8 reasonable field name), overhead will be negligible, but > > > realization > > > > of a client will be way simpler > > > > > > > > чт, 1 июл. 2021 г., 16:57 Pavel Tupitsyn <ptupit...@apache.org>: > > > > > > > >> > No it isn't, I have carefully read code and IEP, in your code you > > > write > > > >> > schema id in each tuple. > > > >> > > > >> There is no code for batch operations yet. > > > >> > > > >> Here is the description of TUPLE_GET_ALL: > > > >> - UUID: table ID > > > >> - int: schema ID > > > >> - arr of arr: array of rows with values for all columns in given > > schema > > > >> (nil when value is missing for a column) > > > >> > > > >> As you can see, schema ID is written once for all rows. > > > >> A row is just a set of values according to the schema. > > > >> > > > >> > > > >> > Also, my biggest concern -- extra serde step. I suppose we should > > pass > > > >> > bytearray to internal api, and use msgpack throughout all wire > > > >> protocols, > > > >> > as tarantool does. > > > >> > > > >> I agree. But this was decided before in IEP-54, and is out of scope > > for > > > >> current IEP. > > > >> Would you like to start a separate thread to discuss this? Or I can > do > > > >> this > > > >> a bit later. > > > >> > > > >> On Thu, Jul 1, 2021 at 4:41 PM Ivan Daschinsky <ivanda...@gmail.com > > > > > >> wrote: > > > >> > > > >> > > This is described in all operations that include multiple > tuples. > > > >> > No it isn't, I have carefully read code and IEP, in your code you > > > write > > > >> > schema id in each tuple. > > > >> > > > > >> > Also, my biggest concern -- extra serde step. I suppose we should > > pass > > > >> > bytearray to internal api, and use msgpack throughout all wire > > > >> protocols, > > > >> > as tarantool does. > > > >> > > > > >> > чт, 1 июл. 2021 г., 16:15 Pavel Tupitsyn <ptupit...@apache.org>: > > > >> > > > > >> > > Ivan, > > > >> > > > > > >> > > > that there is not neccesary to write schema versions in each > > row > > > >> > > > in collectionof tuples > > > >> > > > > > >> > > This is described in all operations that include multiple > tuples. > > > >> > > > > > >> > > > > > >> > > > it is not clear from your code (probably > > > >> > > > mistake?) how differ key tuples and value tuples from each > other > > > >> > > > > > >> > > Key tuples include only key columns. Key columns come first in > the > > > >> > schema. > > > >> > > Value tuples include all columns, key and value. Added "Key > > tuples" > > > >> > > section. > > > >> > > > > > >> > > > > > >> > > > As for me, these excercises with schema's doesn't worth a lot > > > >> > > > > > >> > > I'll add a benchmark and we'll see. > > > >> > > > > > >> > > > > > >> > > On Thu, Jul 1, 2021 at 3:17 PM Ivan Daschinsky < > > ivanda...@gmail.com > > > > > > > >> > > wrote: > > > >> > > > > > >> > > > I suppose, that there is not neccesary to write schema > versions > > in > > > >> each > > > >> > > row > > > >> > > > in collectionof tuples. Also it is not clear from your code > > > >> (probably > > > >> > > > mistake?) how differ key tuples and value tuples from each > > other. > > > In > > > >> > > > readTuple you always read full schema and check for full > length. > > > As > > > >> for > > > >> > > me, > > > >> > > > these excercises with schema's doesn't worth a lot. I.e. > > postgres > > > >> just > > > >> > > > writes field names and then simpy rows with data. Saving few > > bytes > > > >> > > doesn't > > > >> > > > make much deal. Btw, msgpack has special types for short > strings > > > >> (i.e. > > > >> > > > str8). It is much easier use it and write field name as is. > > > >> > > > > > > >> > > > чт, 1 июл. 2021 г., 14:56 Pavel Tupitsyn < > ptupit...@apache.org > > >: > > > >> > > > > > > >> > > > > Ivan, tuple serialization section added to the IEP, let me > > know > > > >> if it > > > >> > > is > > > >> > > > > clear enough. > > > >> > > > > > > > >> > > > > Thanks! > > > >> > > > > > > > >> > > > > On Thu, Jul 1, 2021 at 2:06 PM Ivan Daschinsky < > > > >> ivanda...@gmail.com> > > > >> > > > > wrote: > > > >> > > > > > > > >> > > > > > I can't find any description of tuple serialization in > IEP, > > > >> only in > > > >> > > > code > > > >> > > > > > > > > >> > > > > > чт, 1 июл. 2021 г., 13:59 Pavel Tupitsyn < > > > ptupit...@apache.org > > > >> >: > > > >> > > > > > > > > >> > > > > > > Ivan, > > > >> > > > > > > > > > >> > > > > > > 0. The IEP is not in progress, it is ready for review > and > > > >> > > discussion. > > > >> > > > > > > 1. Tuple serialization is described in the IEP and > > > >> demonstrated > > > >> > in > > > >> > > > the > > > >> > > > > > PoC > > > >> > > > > > > (see ClientMessageHandler#readTuple), let me know if > more > > > >> details > > > >> > > are > > > >> > > > > > > required > > > >> > > > > > > 2. Tuple schema serialization is described in > SCHEMAS_GET > > > >> > section. > > > >> > > > > Table > > > >> > > > > > > schema (configuration) needs more details, you are > right - > > > >> I'll > > > >> > add > > > >> > > > > them. > > > >> > > > > > > 3. This IEP is about tables (tuple-based) API only, > since > > it > > > >> is > > > >> > the > > > >> > > > > only > > > >> > > > > > > API that we have right now, as noted in Risks and > > > Assumptions. > > > >> > > > > > > > > > >> > > > > > > On Thu, Jul 1, 2021 at 1:53 PM Ivan Daschinsky < > > > >> > > ivanda...@gmail.com> > > > >> > > > > > > wrote: > > > >> > > > > > > > > > >> > > > > > > > Also, is there any clear information about KV api? Is > > > there > > > >> any > > > >> > > > plan > > > >> > > > > to > > > >> > > > > > > > implement it? Or is there any proposal about it? > > > >> > > > > > > > > > > >> > > > > > > > чт, 1 июл. 2021 г., 13:51 Ivan Daschinsky < > > > >> ivanda...@gmail.com > > > >> > >: > > > >> > > > > > > > > > > >> > > > > > > > > Pavel, but IEP is in progress, isn't it? > > > >> > > > > > > > > > > > >> > > > > > > > > 1. There is not any information about tuple > > > serialization. > > > >> > And > > > >> > > > > there > > > >> > > > > > > > isn't > > > >> > > > > > > > > a clear consensus about it. > > > >> > > > > > > > > 2. There is not any information about schrma > > > serialization > > > >> > > > format. > > > >> > > > > > And > > > >> > > > > > > > > AFAIK, there isn't a clear consensus also. > > > >> > > > > > > > > > > > >> > > > > > > > > чт, 1 июл. 2021 г., 13:26 Pavel Tupitsyn < > > > >> > ptupit...@apache.org > > > >> > > >: > > > >> > > > > > > > > > > > >> > > > > > > > >> Igniters, > > > >> > > > > > > > >> > > > >> > > > > > > > >> Please review the IEP for thin client protocol in > 3.0 > > > >> [1]. > > > >> > > > > > > > >> PoC is in progress [2] > > > >> > > > > > > > >> > > > >> > > > > > > > >> [1] > > > >> > > > > > > > >> > > > >> > > > > > > > >> > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-76+Thin+Client+Protocol+for+Ignite+3.0 > > > >> > > > > > > > >> > > > >> > > > > > > > >> [2] https://github.com/apache/ignite-3/pull/191 > > > >> > > > > > > > >> > > > >> > > > > > > > > > > > >> > > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > >> > > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > > > > > > > >
