Val, I've checked the IEP again and have a few questions.
Arbitrary nested objects and collections are not allowed as column values. > Nested POJOs should either be inlined into schema, or stored as BLOBs Could you provide a DDL code snippet showing how the inlining of POJOs is supposed to work? Also, we keep using the terms "cache" and "table" throughout the IEP. Is it the right time to discuss an alternate name that would replace those too? Personally, the "table" should stay and the "cache" should go considering that SQL is one of the primary APIs in Ignite and that DDL is supported out-of-the-box. - Denis On Mon, Sep 7, 2020 at 12:26 PM Valentin Kulichenko < valentin.kuliche...@gmail.com> wrote: > Ivan, > > I see your point. I agree that with the automatic updates we step into the > schema-last territory. > > Actually, if we support automatic evolution, we can as well support > creating a cache without schema and inferring it from the first insert. In > other words, we can have both "schema-first" and "schema-last" modes. > > Alexey, what do you think? > > -Val > > On Mon, Sep 7, 2020 at 5:59 AM Alexey Goncharuk < > alexey.goncha...@gmail.com> > wrote: > > > Ivan, > > > > Thank you, I got your concern now. As it is mostly regarding the > > terminology, I am absolutely fine with changing the name to whatever fits > > the approach best. Dynamic or evolving schema sounds great. I will make > > corresponding changes to the IEP once we settle on the name. > > > > пн, 7 сент. 2020 г. в 11:33, Ivan Pavlukhin <vololo...@gmail.com>: > > > > > Hi Val, > > > > > > Thank you for your answer! > > > > > > My understanding is a little bit different. Yes, schema evolution > > > definitely should be possible. But I see a main difference in "how > > > schema is updated". I treat a common SQL approach schema-first. Schema > > > and data manipulation operations are clearly separated and it enables > > > interesting capabilities, e.g. preventing untended schema changes by > > > mistaken data operations, restricting user permissions to change > > > schema. > > > > > > > Schema-first means that schema exists in advance and all the stored > > data > > > is compliant with it - that's exactly what is proposed. > > > > > > A schema-last approach mentioned in [1] also assumes that schema > > > exists, but it is inferred from data. Is not it more similar to the > > > proposing approach? > > > > > > And I would like to say, that my main concern so far is mostly about > > > terminology. And I suppose if it confuses me then others might be > > > confused as well. My feeling is closer to "dynamic or liquid or may be > > > evolving schema". > > > > > > [1] > > > > https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf > > > > > > 2020-09-07 0:47 GMT+03:00, Valentin Kulichenko < > > > valentin.kuliche...@gmail.com>: > > > > Hi Ivan, > > > > > > > > I don't see an issue with that. Schema-first means that schema exists > > in > > > > advance and all the stored data is compliant with it - that's exactly > > > what > > > > is proposed. There are no restrictions prohibiting changes to the > > schema. > > > > > > > > -Val > > > > > > > > On Sat, Sep 5, 2020 at 9:52 PM Ivan Pavlukhin <vololo...@gmail.com> > > > wrote: > > > > > > > >> Alexey, > > > >> > > > >> I am a little bit confused with terminology. My understanding > conforms > > > >> to a survey [1] (see part X Semi Structured Data). Can we really > treat > > > >> a "dynamic schema" approach as a kind of "schema-first"? > > > >> > > > >> [1] > > > >> > > https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf > > > >> > > > >> 2020-09-02 1:53 GMT+03:00, Denis Magda <dma...@apache.org>: > > > >> >> > > > >> >> However, could you please elaborate on the relation between > Ignite > > > and > > > >> >> ORM? > > > >> >> Is there a use case for Hibernate running on top of Ignite (I > > haven't > > > >> >> seen > > > >> >> one so far)? If so, what is missing exactly on the Ignite side to > > > >> support > > > >> >> this? In my understanding, all you need is SQL API which we > already > > > >> have. > > > >> >> Am I missing something? > > > >> > > > > >> > > > > >> > Good point, yes, if all the ORM integrations use Ignite SQL APIs > > > >> > internally, then they can easily translate an Entity object into > an > > > >> > INSERT/UPDATE statement that lists all the object's fields. > Luckily, > > > >> > our > > > >> > Spring Data integration is already based on the Ignite SQL APIs > and > > > >> > needs > > > >> > to be improved once the schema-first approach is supported. That > > would > > > >> > solve a ton of usability issues. > > > >> > > > > >> > I would revise the Hibernate integration as well during the Ignite > > 3.0 > > > >> dev > > > >> > phase. Can't say if it's used a lot but Spring Data is getting > > > traction > > > >> for > > > >> > sure. > > > >> > > > > >> > @Michael Pollind, I'll loop you in as long as you've started > working > > > on > > > >> the > > > >> > Ignite support for Micornaut Data > > > >> > < > https://micronaut-projects.github.io/micronaut-data/latest/guide/> > > > and > > > >> > came across some challenges. Just watch this discussion. That's > what > > > is > > > >> > coming in Ignite 3.0. > > > >> > > > > >> > > > > >> > - > > > >> > Denis > > > >> > > > > >> > > > > >> > On Mon, Aug 31, 2020 at 5:11 PM Valentin Kulichenko < > > > >> > valentin.kuliche...@gmail.com> wrote: > > > >> > > > > >> >> Hi Denis, > > > >> >> > > > >> >> Generally speaking, I believe that the schema-first approach > > natively > > > >> >> addresses the issue if duplicate fields in key and value objects, > > > >> because > > > >> >> schema will be created for a cache, not for an object, as it > > happens > > > >> now. > > > >> >> Basically, the schema will define whether there is a primary key > or > > > >> >> not, > > > >> >> and which fields are included in case there is one. Any API that > we > > > >> would > > > >> >> have must be compliant with this, so it becomes fairly easy to > work > > > >> >> with > > > >> >> data as with a set of records, rather than key-value pairs. > > > >> >> > > > >> >> However, could you please elaborate on the relation between > Ignite > > > and > > > >> >> ORM? > > > >> >> Is there a use case for Hibernate running on top of Ignite (I > > haven't > > > >> >> seen > > > >> >> one so far)? If so, what is missing exactly on the Ignite side to > > > >> support > > > >> >> this? In my understanding, all you need is SQL API which we > already > > > >> have. > > > >> >> Am I missing something? > > > >> >> > > > >> >> -Val > > > >> >> > > > >> >> On Mon, Aug 31, 2020 at 2:08 PM Denis Magda <dma...@apache.org> > > > wrote: > > > >> >> > > > >> >> > Val, > > > >> >> > > > > >> >> > I would propose adding another point to the motivations list > > which > > > >> >> > is > > > >> >> > related to the ORM frameworks such as Spring Data, Hibernate, > > > >> Micronaut > > > >> >> and > > > >> >> > many others. > > > >> >> > > > > >> >> > Presently, the storage engine requires to distinguish key > objects > > > >> >> > from > > > >> >> the > > > >> >> > value ones that complicate the usage of Ignite with those ORM > > > >> >> > frameworks > > > >> >> > (especially if a key object comprises several fields). More on > > this > > > >> can > > > >> >> be > > > >> >> > found here: > > > >> >> > > > > >> >> > > > > >> >> > > > >> > > > > > > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Key-and-Value-fields-with-same-name-and-SQL-DML-td47557.html > > > >> >> > > > > >> >> > It will be nice if the new schema-first approach allows us to > > work > > > >> with > > > >> >> > a > > > >> >> > single entity object when it comes to the ORMs. With no need to > > > >> >> > split > > > >> >> > the > > > >> >> > entity into a key and value. Just want to be sure that the > Ignite > > > >> >> > 3.0 > > > >> >> > has > > > >> >> > all the essential public APIs that would support the > > single-entity > > > >> >> > based > > > >> >> > approach. > > > >> >> > > > > >> >> > What do you think? > > > >> >> > > > > >> >> > - > > > >> >> > Denis > > > >> >> > > > > >> >> > > > > >> >> > On Fri, Aug 28, 2020 at 3:50 PM Valentin Kulichenko < > > > >> >> > valentin.kuliche...@gmail.com> wrote: > > > >> >> > > > > >> >> > > Igniters, > > > >> >> > > > > > >> >> > > One of the big changes proposed for Ignite 3.0 is the > so-called > > > >> >> > > "schema-first approach". To add more clarity, I've started > > > writing > > > >> >> > > the > > > >> >> > IEP > > > >> >> > > for this change: > > > >> >> > > > > > >> >> > > > > > >> >> > > > > >> >> > > > >> > > > > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach > > > >> >> > > > > > >> >> > > Please take a look and let me know if there are any immediate > > > >> >> > > thoughts, > > > >> >> > > suggestions, or objections. > > > >> >> > > > > > >> >> > > -Val > > > >> >> > > > > > >> >> > > > > >> >> > > > >> > > > > >> > > > >> > > > >> -- > > > >> > > > >> Best regards, > > > >> Ivan Pavlukhin > > > >> > > > > > > > > > > > > > -- > > > > > > Best regards, > > > Ivan Pavlukhin > > > > > >
