Andrey, I don't think range narrowing is a good idea. Do you see any problems with the simple approach I described?
On Tue, Nov 24, 2020 at 4:01 PM Andrey Mashenkov <andrey.mashen...@gmail.com> wrote: > Pavel, > > If you are ok with narrowing range for unsigned types then we could add a > constraint for unsigned types on schema level (like nullability flag) > and treat them as signed types in storage. > > We are going with a separate storage type-system and binary protocol > type-system, however most of type will match 1 to 1 with storage (native) > type. > On .Net side you will either have a separate type id or treat serialized > value regarding a schema (signed or unsigned flag). > > Igor, > > I'm not sure users can ever foresee the consequences of using unsigned > types. > > Assume, a user used to unsigned types perfectly works with some database, > then he turns into Ignite successor confession with our "native" > unsigned-types support. > But later, he finds that he can use the power of Ignite Compute on Java for > some tasks or a new app. > Finally, the user will either fail to use his unsigned data on Java due or > face performance issues due to natural Java type system limitations e.g. > conversion uLong to BigInteger. > > I believe that natively supported types with possible value ranges and > limitations should be known. > So, the only question is what trade-off we found acceptable: narrowing > unsigned type range or use types of wider range on systems like Java. > > On Tue, Nov 24, 2020 at 3:25 PM Igor Sapego <isap...@apache.org> wrote: > > > Actually, I think it is not so hard to implement comparison of unsigned > > numbers in > > SQL even in Java, so it does not seem to be a big issue from my > > perspective. > > > > Now to the usage of unsigned types from Java - I think, if a user uses > > unsigned type > > in a schema and is going to interact with it from Java he knows what he > is > > doing. > > > > Mostly they are for use from platforms where they have native support and > > widely > > used, like in C++ or .NET, where users currently have to make a manual > type > > casting > > or even just stop using unsigned types when they use Ignite. > > > > Best Regards, > > Igor > > > > > > On Tue, Nov 24, 2020 at 3:06 PM Pavel Tupitsyn <ptupit...@apache.org> > > wrote: > > > > > Andrey, > > > > > > I think it is much simpler: > > > - Add protocol support for those types (basically, just add more type > > ids) > > > - Treat uLong as long in Java (bitwise representation is the same) > > > > > > ANSI SQL does not have unsigned integers, so we can simply say that > > > unsigned value relative comparison is not supported in SQL (equality > will > > > work). > > > > > > > > > On Tue, Nov 24, 2020 at 2:40 PM Andrey Mashenkov < > > > andrey.mashen...@gmail.com> > > > wrote: > > > > > > > Thanks, Pavel and Igor. > > > > > > > > I like your ideas to have i8 or int8 instead of Integer. > > > > But the naming doesn't address the issue. > > > > > > > > I agree internal types should be portable across different systems > with > > > and > > > > without unsigned type support. > > > > The only issue here is that unsigned types cover different ranges. > > > > > > > > Let's assume we want to introduce a uLong. > > > > It doesn't look like a big deal to add uLong type support at storage > > > level > > > > and fit it to a 8 bytes and then use it in e.g. .Net only. > > > > But how we could support it in e.g. Java? > > > > > > > > Let's keep in mind Long range is about (2^-63 .. 2^63) and uLong > range > > is > > > > (0 .. 2^64) > > > > 1. The first option is to restrict range to (0 .. 2^63). This allows > to > > > use > > > > signed in e.g. > > > > Java with no conversion, but doesn't look like a 'real' unsigned > uLong > > > > support. Things go worse when the user will use uByte, as limitation > > can > > > > make uByte totally unusable. > > > > > > > > 2. The second one is to map unsigned types to a type of wider type > and > > > add > > > > a constraint for negative values. E.g. uLong to BigInteger. > > > > So, we can't use primitive Java type for Long here. However, it is > > still > > > > possible to store uLong in 8 bytes, but have a special comparator for > > > > unsigned types to avoid unwanted deserialization. > > > > > > > > WDYT? > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Nov 24, 2020 at 2:04 PM Pavel Tupitsyn <ptupit...@apache.org > > > > > > wrote: > > > > > > > > > Agree, let's get rid of "long, short, byte" in the protocol > > definition. > > > > > > > > > > We can use Rust style, which is concise and unambiguous: > > > > > i8, u8, i16, u16, etc > > > > > > > > > > On Tue, Nov 24, 2020 at 1:58 PM Igor Sapego <isap...@apache.org> > > > wrote: > > > > > > > > > > > Pavel, > > > > > > > > > > > > I totally support that. Also, if we are aiming for > > > > > > stronger platform-independance, > > > > > > in our schemas we may want to support bit-notation (int32, > uint64)? > > > For > > > > > > example > > > > > > "long" can mean a different type on different platforms and it's > > easy > > > > to > > > > > > confuse > > > > > > them (happens often when using ODBC for example). > > > > > > > > > > > > Best Regards, > > > > > > Igor > > > > > > > > > > > > > > > > > > On Tue, Nov 24, 2020 at 1:34 PM Pavel Tupitsyn < > > ptupit...@apache.org > > > > > > > > > > wrote: > > > > > > > > > > > > > Igniters, > > > > > > > > > > > > > > I think we should support unsigned data types: > > > > > > > uByte, uShort, uInt, uLong > > > > > > > > > > > > > > Java does not have them, but many other languages do, > > > > > > > and with the growing number of thin clients this is important. > > > > > > > > > > > > > > For example, in current Ignite.NET implementation we store > > unsigned > > > > > > values > > > > > > > as signed internally, > > > > > > > but this is a huge pain when it comes to metadata, binary > > objects, > > > > etc. > > > > > > > (it is easy to deserialize int as uint when you have a class, > but > > > not > > > > > > with > > > > > > > BinaryObject.GetField) > > > > > > > > > > > > > > Any objections? > > > > > > > > > > > > > > On Tue, Nov 24, 2020 at 12:28 PM Andrey Mashenkov < > > > > > > > andrey.mashen...@gmail.com> wrote: > > > > > > > > > > > > > > > Denis, > > > > > > > > > > > > > > > > Good point. Both serializers use reflection API. > > > > > > > > However, we will allow users to configure static schema along > > > with > > > > > > > 'strict' > > > > > > > > schema mode, we still need to validate user classes on client > > > nodes > > > > > > > against > > > > > > > > the latest schema in the grid and reflection API is the only > > way > > > > to > > > > > do > > > > > > > it. > > > > > > > > One can find a few articles on the internet on how to enable > > > > > reflection > > > > > > > in > > > > > > > > GraalVM. > > > > > > > > > > > > > > > > I'll create a task for supporting GraalVM, and maybe someone > > who > > > is > > > > > > > > familiar with GraalVM will suggest a solution or a proper > > > > workaround. > > > > > > Or > > > > > > > > I'll do it a bit later. > > > > > > > > If no workaround is found, we could allow users to write it's > > own > > > > > > > > serializer, but I don't think it is a good idea to expose any > > > > > internal > > > > > > > > classes to the public. > > > > > > > > > > > > > > > > On Tue, Nov 24, 2020 at 2:55 AM Denis Magda < > dma...@apache.org > > > > > > > > wrote: > > > > > > > > > > > > > > > > > Andrey, thanks for the update, > > > > > > > > > > > > > > > > > > Does any of the serializers take into consideration the > > > > > > > > > native-image-generation feature of GraalVM? > > > > > > > > > https://www.graalvm.org/reference-manual/native-image/ > > > > > > > > > > > > > > > > > > With the current binary marshaller, we can't even generate > a > > > > native > > > > > > > image > > > > > > > > > for the code using our thin client APIs. > > > > > > > > > > > > > > > > > > - > > > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > > > > > > > On Mon, Nov 23, 2020 at 4:39 AM Andrey Mashenkov < > > > > > > > > > andrey.mashen...@gmail.com> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Hi Igniters, > > > > > > > > > > > > > > > > > > > > I'd like to continue discussion of IEP-54 (Schema-first > > > > > approach). > > > > > > > > > > > > > > > > > > > > Hope everyone who is interested had a chance to get > > familiar > > > > with > > > > > > the > > > > > > > > > > proposal [1]. > > > > > > > > > > Please, do not hesitate to ask questions and share your > > > ideas. > > > > > > > > > > > > > > > > > > > > I've prepared a prototype of serializer [2] for the data > > > layout > > > > > > > > described > > > > > > > > > > in the proposal. > > > > > > > > > > In prototy, I compared 2 approaches to (de)serialize > > objects, > > > > the > > > > > > > first > > > > > > > > > one > > > > > > > > > > uses java reflection/unsafe API and similar to one we > > already > > > > use > > > > > > in > > > > > > > > > Ignite > > > > > > > > > > and the second one generates serializer for particular > user > > > > class > > > > > > and > > > > > > > > > uses > > > > > > > > > > Janino library for compilation. > > > > > > > > > > Second one shows better results in benchmarks. > > > > > > > > > > I think we can go with it as default serializer and have > > > > > > > > reflection-based > > > > > > > > > > implementation as a fallback if someone will have issues > > with > > > > the > > > > > > > first > > > > > > > > > > one. > > > > > > > > > > WDYT? > > > > > > > > > > > > > > > > > > > > There are a number of tasks under the umbrella ticket [3] > > > > waiting > > > > > > for > > > > > > > > the > > > > > > > > > > assignee. > > > > > > > > > > > > > > > > > > > > BTW, I'm going to create more tickets for schema manager > > > modes > > > > > > > > > > implementation, but would like to clarify some details. > > > > > > > > > > > > > > > > > > > > I thought schemaManager on each node should held: > > > > > > > > > > 1. Local mapping of "schema version" <--> validated > local > > > > > > key/value > > > > > > > > > > classes pair. > > > > > > > > > > 2. Cluster-wide schema changes history. > > > > > > > > > > On the client side. Before any key-value API operation we > > > > should > > > > > > > > > validate a > > > > > > > > > > schema for a given key-value pair. > > > > > > > > > > If there is no local-mapping exists for a given key-value > > > pair > > > > or > > > > > > if > > > > > > > a > > > > > > > > > > cluster wide schema has a more recent version then the > > > > key-value > > > > > > pair > > > > > > > > > > should be validated against the latest version and local > > > > mapping > > > > > > > should > > > > > > > > > be > > > > > > > > > > updated/actualized. > > > > > > > > > > If an object doesn't fit to the latest schema then it > > depends > > > > on > > > > > > > schema > > > > > > > > > > mode: either fail the operation ('strict' mode) or a new > > > > mapping > > > > > > > should > > > > > > > > > be > > > > > > > > > > created and a new schema version should be propagated to > > the > > > > > > cluster. > > > > > > > > > > > > > > > > > > > > On the server side we usually have no key-value classes > and > > > we > > > > > > > operate > > > > > > > > > with > > > > > > > > > > tuples. > > > > > > > > > > As schema change history is available and a tuple has > > schema > > > > > > version, > > > > > > > > > then > > > > > > > > > > it is possible to upgrade any received tuple to the last > > > > version > > > > > > > > without > > > > > > > > > > desialization. > > > > > > > > > > Thus we could allow nodes to send key-value pairs of > > previous > > > > > > > versions > > > > > > > > > (if > > > > > > > > > > they didn't receive a schema update yet) without > reverting > > > > schema > > > > > > > > changes > > > > > > > > > > made by a node with newer classes. > > > > > > > > > > > > > > > > > > > > Alex, Val, Ivan did you mean the same? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach > > > > > > > > > > [2] > > > > > > > > > https://github.com/apache/ignite/tree/ignite-13618/modules/commons > > > > > > > > > > [3] https://issues.apache.org/jira/browse/IGNITE-13616 > > > > > > > > > > > > > > > > > > > > On Thu, Sep 17, 2020 at 9:21 AM Ivan Pavlukhin < > > > > > > vololo...@gmail.com> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Folks, > > > > > > > > > > > > > > > > > > > > > > Please do not ignore history. We had a thread [1] with > > many > > > > > > bright > > > > > > > > > > > ideas. We can resume it. > > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/Applicability-of-term-cache-to-Apache-Ignite-td36541.html > > > > > > > > > > > > > > > > > > > > > > 2020-09-10 0:08 GMT+03:00, Denis Magda < > > dma...@apache.org > > > >: > > > > > > > > > > > > Val, makes sense, thanks for explaining. > > > > > > > > > > > > > > > > > > > > > > > > Agree that we need to have a separate discussion > thread > > > for > > > > > the > > > > > > > > > "table" > > > > > > > > > > > and > > > > > > > > > > > > "cache" terms substitution. I'll appreciate it if you > > > start > > > > > the > > > > > > > > > thread > > > > > > > > > > > > sharing pointers to any relevant IEPs and reasoning > > > behind > > > > > the > > > > > > > > > > suggested > > > > > > > > > > > > change. > > > > > > > > > > > > > > > > > > > > > > > > - > > > > > > > > > > > > Denis > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Tue, Sep 8, 2020 at 6:01 PM Valentin Kulichenko < > > > > > > > > > > > > valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > > > > > > > > > > > > > >> Hi Denis, > > > > > > > > > > > >> > > > > > > > > > > > >> I guess the wording in the IEP is a little bit > > > confusing. > > > > > All > > > > > > it > > > > > > > > > means > > > > > > > > > > > is > > > > > > > > > > > >> that you should not create nested POJOs, but rather > > > inline > > > > > > > fields > > > > > > > > > > into a > > > > > > > > > > > >> single POJO that is mapped to a particular schema. > In > > > > other > > > > > > > words, > > > > > > > > > > > nested > > > > > > > > > > > >> POJOs are not supported. > > > > > > > > > > > >> > > > > > > > > > > > >> Alex, is this correct? Please let me know if I'm > > missing > > > > > > > > something. > > > > > > > > > > > >> > > > > > > > > > > > >> As for the "cache" term, I agree that it is > outdated, > > > but > > > > > I'm > > > > > > > not > > > > > > > > > sure > > > > > > > > > > > >> what we can replace it with. "Table" is tightly > > > associated > > > > > > with > > > > > > > > SQL, > > > > > > > > > > but > > > > > > > > > > > >> SQL is optional in our case. Do you want to create a > > > > > separate > > > > > > > > > > discussion > > > > > > > > > > > >> about this? > > > > > > > > > > > >> > > > > > > > > > > > >> -Val > > > > > > > > > > > >> > > > > > > > > > > > >> On Tue, Sep 8, 2020 at 4:37 PM Denis Magda < > > > > > dma...@apache.org > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > >> > > > > > > > > > > > >>> Val, > > > > > > > > > > > >>> > > > > > > > > > > > >>> I've checked the IEP again and have a few > questions. > > > > > > > > > > > >>> > > > > > > > > > > > >>> Arbitrary nested objects and collections are not > > > allowed > > > > as > > > > > > > > column > > > > > > > > > > > >>> values. > > > > > > > > > > > >>> > Nested POJOs should either be inlined into > schema, > > or > > > > > > stored > > > > > > > as > > > > > > > > > > BLOBs > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > > > >>> Could you provide a DDL code snippet showing how > the > > > > > inlining > > > > > > > of > > > > > > > > > > POJOs > > > > > > > > > > > >>> is > > > > > > > > > > > >>> supposed to work? > > > > > > > > > > > >>> > > > > > > > > > > > >>> Also, we keep using the terms "cache" and "table" > > > > > throughout > > > > > > > the > > > > > > > > > IEP. > > > > > > > > > > > Is > > > > > > > > > > > >>> it > > > > > > > > > > > >>> the right time to discuss an alternate name that > > would > > > > > > replace > > > > > > > > > those > > > > > > > > > > > >>> too? > > > > > > > > > > > >>> Personally, the "table" should stay and the "cache" > > > > should > > > > > go > > > > > > > > > > > >>> considering > > > > > > > > > > > >>> that SQL is one of the primary APIs in Ignite and > > that > > > > DDL > > > > > is > > > > > > > > > > supported > > > > > > > > > > > >>> out-of-the-box. > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > > > >>> - > > > > > > > > > > > >>> Denis > > > > > > > > > > > >>> > > > > > > > > > > > >>> > > > > > > > > > > > >>> On Mon, Sep 7, 2020 at 12:26 PM Valentin > Kulichenko < > > > > > > > > > > > >>> valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > >>> > > > > > > > > > > > >>> > Ivan, > > > > > > > > > > > >>> > > > > > > > > > > > > >>> > I see your point. I agree that with the automatic > > > > updates > > > > > > we > > > > > > > > step > > > > > > > > > > > into > > > > > > > > > > > >>> the > > > > > > > > > > > >>> > schema-last territory. > > > > > > > > > > > >>> > > > > > > > > > > > > >>> > Actually, if we support automatic evolution, we > can > > > as > > > > > well > > > > > > > > > support > > > > > > > > > > > >>> > creating a cache without schema and inferring it > > from > > > > the > > > > > > > first > > > > > > > > > > > >>> > insert. > > > > > > > > > > > >>> In > > > > > > > > > > > >>> > other words, we can have both "schema-first" and > > > > > > > "schema-last" > > > > > > > > > > modes. > > > > > > > > > > > >>> > > > > > > > > > > > > >>> > Alexey, what do you think? > > > > > > > > > > > >>> > > > > > > > > > > > > >>> > -Val > > > > > > > > > > > >>> > > > > > > > > > > > > >>> > On Mon, Sep 7, 2020 at 5:59 AM Alexey Goncharuk < > > > > > > > > > > > >>> > alexey.goncha...@gmail.com> > > > > > > > > > > > >>> > wrote: > > > > > > > > > > > >>> > > > > > > > > > > > > >>> > > Ivan, > > > > > > > > > > > >>> > > > > > > > > > > > > > >>> > > Thank you, I got your concern now. As it is > > mostly > > > > > > > regarding > > > > > > > > > the > > > > > > > > > > > >>> > > terminology, I am absolutely fine with changing > > the > > > > > name > > > > > > to > > > > > > > > > > > whatever > > > > > > > > > > > >>> fits > > > > > > > > > > > >>> > > the approach best. Dynamic or evolving schema > > > sounds > > > > > > > great. I > > > > > > > > > > will > > > > > > > > > > > >>> make > > > > > > > > > > > >>> > > corresponding changes to the IEP once we settle > > on > > > > the > > > > > > > name. > > > > > > > > > > > >>> > > > > > > > > > > > > > >>> > > пн, 7 сент. 2020 г. в 11:33, Ivan Pavlukhin < > > > > > > > > > vololo...@gmail.com > > > > > > > > > > >: > > > > > > > > > > > >>> > > > > > > > > > > > > > >>> > > > Hi Val, > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > Thank you for your answer! > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > My understanding is a little bit different. > > Yes, > > > > > schema > > > > > > > > > > evolution > > > > > > > > > > > >>> > > > definitely should be possible. But I see a > main > > > > > > > difference > > > > > > > > in > > > > > > > > > > > "how > > > > > > > > > > > >>> > > > schema is updated". I treat a common SQL > > approach > > > > > > > > > schema-first. > > > > > > > > > > > >>> Schema > > > > > > > > > > > >>> > > > and data manipulation operations are clearly > > > > > separated > > > > > > > and > > > > > > > > it > > > > > > > > > > > >>> enables > > > > > > > > > > > >>> > > > interesting capabilities, e.g. preventing > > > untended > > > > > > schema > > > > > > > > > > changes > > > > > > > > > > > >>> > > > by > > > > > > > > > > > >>> > > > mistaken data operations, restricting user > > > > > permissions > > > > > > to > > > > > > > > > > change > > > > > > > > > > > >>> > > > schema. > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > > Schema-first means that schema exists in > > > advance > > > > > and > > > > > > > all > > > > > > > > > the > > > > > > > > > > > >>> stored > > > > > > > > > > > >>> > > data > > > > > > > > > > > >>> > > > is compliant with it - that's exactly what is > > > > > proposed. > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > A schema-last approach mentioned in [1] also > > > > assumes > > > > > > that > > > > > > > > > > schema > > > > > > > > > > > >>> > > > exists, but it is inferred from data. Is not > it > > > > more > > > > > > > > similar > > > > > > > > > to > > > > > > > > > > > >>> > > > the > > > > > > > > > > > >>> > > > proposing approach? > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > And I would like to say, that my main concern > > so > > > > far > > > > > is > > > > > > > > > mostly > > > > > > > > > > > >>> > > > about > > > > > > > > > > > >>> > > > terminology. And I suppose if it confuses me > > then > > > > > > others > > > > > > > > > might > > > > > > > > > > be > > > > > > > > > > > >>> > > > confused as well. My feeling is closer to > > > "dynamic > > > > or > > > > > > > > liquid > > > > > > > > > or > > > > > > > > > > > >>> > > > may > > > > > > > > > > > >>> be > > > > > > > > > > > >>> > > > evolving schema". > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > [1] > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > 2020-09-07 0:47 GMT+03:00, Valentin > Kulichenko > > < > > > > > > > > > > > >>> > > > valentin.kuliche...@gmail.com>: > > > > > > > > > > > >>> > > > > Hi Ivan, > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > > I don't see an issue with that. > Schema-first > > > > means > > > > > > that > > > > > > > > > > schema > > > > > > > > > > > >>> exists > > > > > > > > > > > >>> > > in > > > > > > > > > > > >>> > > > > advance and all the stored data is > compliant > > > with > > > > > it > > > > > > - > > > > > > > > > that's > > > > > > > > > > > >>> exactly > > > > > > > > > > > >>> > > > what > > > > > > > > > > > >>> > > > > is proposed. There are no restrictions > > > > prohibiting > > > > > > > > changes > > > > > > > > > to > > > > > > > > > > > >>> > > > > the > > > > > > > > > > > >>> > > schema. > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > > -Val > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > > On Sat, Sep 5, 2020 at 9:52 PM Ivan > > Pavlukhin < > > > > > > > > > > > >>> vololo...@gmail.com> > > > > > > > > > > > >>> > > > wrote: > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > >> Alexey, > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > > > >> I am a little bit confused with > terminology. > > > My > > > > > > > > > > understanding > > > > > > > > > > > >>> > conforms > > > > > > > > > > > >>> > > > >> to a survey [1] (see part X Semi > Structured > > > > Data). > > > > > > Can > > > > > > > > we > > > > > > > > > > > >>> > > > >> really > > > > > > > > > > > >>> > treat > > > > > > > > > > > >>> > > > >> a "dynamic schema" approach as a kind of > > > > > > > "schema-first"? > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > > > >> [1] > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > > > >> 2020-09-02 1:53 GMT+03:00, Denis Magda < > > > > > > > > dma...@apache.org > > > > > > > > > >: > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > >>> > > > >> >> However, could you please elaborate on > > the > > > > > > relation > > > > > > > > > > between > > > > > > > > > > > >>> > Ignite > > > > > > > > > > > >>> > > > and > > > > > > > > > > > >>> > > > >> >> ORM? > > > > > > > > > > > >>> > > > >> >> Is there a use case for Hibernate > running > > > on > > > > > top > > > > > > of > > > > > > > > > > Ignite > > > > > > > > > > > >>> > > > >> >> (I > > > > > > > > > > > >>> > > haven't > > > > > > > > > > > >>> > > > >> >> seen > > > > > > > > > > > >>> > > > >> >> one so far)? If so, what is missing > > exactly > > > > on > > > > > > the > > > > > > > > > Ignite > > > > > > > > > > > >>> side to > > > > > > > > > > > >>> > > > >> support > > > > > > > > > > > >>> > > > >> >> this? In my understanding, all you need > > is > > > > SQL > > > > > > API > > > > > > > > > which > > > > > > > > > > we > > > > > > > > > > > >>> > already > > > > > > > > > > > >>> > > > >> have. > > > > > > > > > > > >>> > > > >> >> Am I missing something? > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > >> > Good point, yes, if all the ORM > > integrations > > > > use > > > > > > > > Ignite > > > > > > > > > > SQL > > > > > > > > > > > >>> APIs > > > > > > > > > > > >>> > > > >> > internally, then they can easily > translate > > > an > > > > > > Entity > > > > > > > > > > object > > > > > > > > > > > >>> into > > > > > > > > > > > >>> > an > > > > > > > > > > > >>> > > > >> > INSERT/UPDATE statement that lists all > the > > > > > > object's > > > > > > > > > > fields. > > > > > > > > > > > >>> > Luckily, > > > > > > > > > > > >>> > > > >> > our > > > > > > > > > > > >>> > > > >> > Spring Data integration is already based > > on > > > > the > > > > > > > Ignite > > > > > > > > > SQL > > > > > > > > > > > >>> > > > >> > APIs > > > > > > > > > > > >>> > and > > > > > > > > > > > >>> > > > >> > needs > > > > > > > > > > > >>> > > > >> > to be improved once the schema-first > > > approach > > > > is > > > > > > > > > > supported. > > > > > > > > > > > >>> That > > > > > > > > > > > >>> > > would > > > > > > > > > > > >>> > > > >> > solve a ton of usability issues. > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > >> > I would revise the Hibernate integration > > as > > > > well > > > > > > > > during > > > > > > > > > > the > > > > > > > > > > > >>> Ignite > > > > > > > > > > > >>> > > 3.0 > > > > > > > > > > > >>> > > > >> dev > > > > > > > > > > > >>> > > > >> > phase. Can't say if it's used a lot but > > > Spring > > > > > > Data > > > > > > > is > > > > > > > > > > > >>> > > > >> > getting > > > > > > > > > > > >>> > > > traction > > > > > > > > > > > >>> > > > >> for > > > > > > > > > > > >>> > > > >> > sure. > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > >> > @Michael Pollind, I'll loop you in as > long > > > as > > > > > > you've > > > > > > > > > > started > > > > > > > > > > > >>> > working > > > > > > > > > > > >>> > > > on > > > > > > > > > > > >>> > > > >> the > > > > > > > > > > > >>> > > > >> > Ignite support for Micornaut Data > > > > > > > > > > > >>> > > > >> > < > > > > > > > > > > > >>> > > > > > > > > > > > > > > https://micronaut-projects.github.io/micronaut-data/latest/guide/> > > > > > > > > > > > >>> > > > and > > > > > > > > > > > >>> > > > >> > came across some challenges. Just watch > > this > > > > > > > > discussion. > > > > > > > > > > > >>> > > > >> > That's > > > > > > > > > > > >>> > what > > > > > > > > > > > >>> > > > is > > > > > > > > > > > >>> > > > >> > coming in Ignite 3.0. > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > >> > - > > > > > > > > > > > >>> > > > >> > Denis > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > >> > On Mon, Aug 31, 2020 at 5:11 PM Valentin > > > > > > Kulichenko > > > > > > > < > > > > > > > > > > > >>> > > > >> > valentin.kuliche...@gmail.com> wrote: > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > >> >> Hi Denis, > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > >>> > > > >> >> Generally speaking, I believe that the > > > > > > schema-first > > > > > > > > > > > approach > > > > > > > > > > > >>> > > natively > > > > > > > > > > > >>> > > > >> >> addresses the issue if duplicate fields > > in > > > > key > > > > > > and > > > > > > > > > value > > > > > > > > > > > >>> objects, > > > > > > > > > > > >>> > > > >> because > > > > > > > > > > > >>> > > > >> >> schema will be created for a cache, not > > for > > > > an > > > > > > > > object, > > > > > > > > > as > > > > > > > > > > > it > > > > > > > > > > > >>> > > happens > > > > > > > > > > > >>> > > > >> now. > > > > > > > > > > > >>> > > > >> >> Basically, the schema will define > whether > > > > there > > > > > > is > > > > > > > a > > > > > > > > > > > primary > > > > > > > > > > > >>> key > > > > > > > > > > > >>> > or > > > > > > > > > > > >>> > > > >> >> not, > > > > > > > > > > > >>> > > > >> >> and which fields are included in case > > there > > > > is > > > > > > one. > > > > > > > > Any > > > > > > > > > > API > > > > > > > > > > > >>> that > > > > > > > > > > > >>> > we > > > > > > > > > > > >>> > > > >> would > > > > > > > > > > > >>> > > > >> >> have must be compliant with this, so it > > > > becomes > > > > > > > > fairly > > > > > > > > > > easy > > > > > > > > > > > >>> > > > >> >> to > > > > > > > > > > > >>> > work > > > > > > > > > > > >>> > > > >> >> with > > > > > > > > > > > >>> > > > >> >> data as with a set of records, rather > > than > > > > > > > key-value > > > > > > > > > > pairs. > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > >>> > > > >> >> However, could you please elaborate on > > the > > > > > > relation > > > > > > > > > > between > > > > > > > > > > > >>> > Ignite > > > > > > > > > > > >>> > > > and > > > > > > > > > > > >>> > > > >> >> ORM? > > > > > > > > > > > >>> > > > >> >> Is there a use case for Hibernate > running > > > on > > > > > top > > > > > > of > > > > > > > > > > Ignite > > > > > > > > > > > >>> > > > >> >> (I > > > > > > > > > > > >>> > > haven't > > > > > > > > > > > >>> > > > >> >> seen > > > > > > > > > > > >>> > > > >> >> one so far)? If so, what is missing > > exactly > > > > on > > > > > > the > > > > > > > > > Ignite > > > > > > > > > > > >>> side to > > > > > > > > > > > >>> > > > >> support > > > > > > > > > > > >>> > > > >> >> this? In my understanding, all you need > > is > > > > SQL > > > > > > API > > > > > > > > > which > > > > > > > > > > we > > > > > > > > > > > >>> > already > > > > > > > > > > > >>> > > > >> have. > > > > > > > > > > > >>> > > > >> >> Am I missing something? > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > >>> > > > >> >> -Val > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > >>> > > > >> >> On Mon, Aug 31, 2020 at 2:08 PM Denis > > > Magda < > > > > > > > > > > > >>> dma...@apache.org> > > > > > > > > > > > >>> > > > wrote: > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > >>> > > > >> >> > Val, > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> > I would propose adding another point > to > > > the > > > > > > > > > motivations > > > > > > > > > > > >>> > > > >> >> > list > > > > > > > > > > > >>> > > which > > > > > > > > > > > >>> > > > >> >> > is > > > > > > > > > > > >>> > > > >> >> > related to the ORM frameworks such as > > > > Spring > > > > > > > Data, > > > > > > > > > > > >>> Hibernate, > > > > > > > > > > > >>> > > > >> Micronaut > > > > > > > > > > > >>> > > > >> >> and > > > > > > > > > > > >>> > > > >> >> > many others. > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> > Presently, the storage engine > requires > > to > > > > > > > > distinguish > > > > > > > > > > key > > > > > > > > > > > >>> > objects > > > > > > > > > > > >>> > > > >> >> > from > > > > > > > > > > > >>> > > > >> >> the > > > > > > > > > > > >>> > > > >> >> > value ones that complicate the usage > of > > > > > Ignite > > > > > > > with > > > > > > > > > > those > > > > > > > > > > > >>> ORM > > > > > > > > > > > >>> > > > >> >> > frameworks > > > > > > > > > > > >>> > > > >> >> > (especially if a key object comprises > > > > several > > > > > > > > > fields). > > > > > > > > > > > >>> > > > >> >> > More > > > > > > > > > > > >>> on > > > > > > > > > > > >>> > > this > > > > > > > > > > > >>> > > > >> can > > > > > > > > > > > >>> > > > >> >> be > > > > > > > > > > > >>> > > > >> >> > found here: > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Key-and-Value-fields-with-same-name-and-SQL-DML-td47557.html > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> > It will be nice if the new > schema-first > > > > > > approach > > > > > > > > > allows > > > > > > > > > > > us > > > > > > > > > > > >>> to > > > > > > > > > > > >>> > > work > > > > > > > > > > > >>> > > > >> with > > > > > > > > > > > >>> > > > >> >> > a > > > > > > > > > > > >>> > > > >> >> > single entity object when it comes to > > the > > > > > ORMs. > > > > > > > > With > > > > > > > > > no > > > > > > > > > > > >>> need to > > > > > > > > > > > >>> > > > >> >> > split > > > > > > > > > > > >>> > > > >> >> > the > > > > > > > > > > > >>> > > > >> >> > entity into a key and value. Just > want > > to > > > > be > > > > > > sure > > > > > > > > > that > > > > > > > > > > > the > > > > > > > > > > > >>> > Ignite > > > > > > > > > > > >>> > > > >> >> > 3.0 > > > > > > > > > > > >>> > > > >> >> > has > > > > > > > > > > > >>> > > > >> >> > all the essential public APIs that > > would > > > > > > support > > > > > > > > the > > > > > > > > > > > >>> > > single-entity > > > > > > > > > > > >>> > > > >> >> > based > > > > > > > > > > > >>> > > > >> >> > approach. > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> > What do you think? > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> > - > > > > > > > > > > > >>> > > > >> >> > Denis > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> > On Fri, Aug 28, 2020 at 3:50 PM > > Valentin > > > > > > > > Kulichenko < > > > > > > > > > > > >>> > > > >> >> > valentin.kuliche...@gmail.com> > wrote: > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> > > Igniters, > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > > One of the big changes proposed for > > > > Ignite > > > > > > 3.0 > > > > > > > is > > > > > > > > > the > > > > > > > > > > > >>> > so-called > > > > > > > > > > > >>> > > > >> >> > > "schema-first approach". To add > more > > > > > clarity, > > > > > > > > I've > > > > > > > > > > > >>> > > > >> >> > > started > > > > > > > > > > > >>> > > > writing > > > > > > > > > > > >>> > > > >> >> > > the > > > > > > > > > > > >>> > > > >> >> > IEP > > > > > > > > > > > >>> > > > >> >> > > for this change: > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > > Please take a look and let me know > if > > > > there > > > > > > are > > > > > > > > any > > > > > > > > > > > >>> immediate > > > > > > > > > > > >>> > > > >> >> > > thoughts, > > > > > > > > > > > >>> > > > >> >> > > suggestions, or objections. > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > > -Val > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > > >>> > > > >> >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > > > >> -- > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > > > >> Best regards, > > > > > > > > > > > >>> > > > >> Ivan Pavlukhin > > > > > > > > > > > >>> > > > >> > > > > > > > > > > > >>> > > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > -- > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > Best regards, > > > > > > > > > > > >>> > > > Ivan Pavlukhin > > > > > > > > > > > >>> > > > > > > > > > > > > > > >>> > > > > > > > > > > > > > >>> > > > > > > > > > > > > >>> > > > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > > > > > > > > > > > > > Best regards, > > > > > > > > > > > Ivan Pavlukhin > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > > > Best regards, > > > > > > > > > > Andrey V. Mashenkov > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Best regards, > > > > > > > > Andrey V. Mashenkov > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > Best regards, > > > > Andrey V. Mashenkov > > > > > > > > > > > > -- > Best regards, > Andrey V. Mashenkov >
