> custom DSL
This is a topic for 3.0 - would you like to start a separate thread?
On Fri, Mar 5, 2021 at 4:54 PM Ivan Daschinsky <ivanda...@gmail.com> wrote:
> I suppose, that the best variantl -- custom DSL similar to MongoDB query
> language.
> I understand that implementing this is much harder, but users want this
> variant.
>
> пт, 5 мар. 2021 г. в 16:52, Pavel Tupitsyn <ptupit...@apache.org>:
>
> > > I suppose this is of questional usability, same for existing
> > > implementation for ScanQuery and ContinousQuery
> >
> > One way or another, we need to be able to run custom logic
> > on the server side from thin clients.
> >
> > Our current direction can be seen as "Java is our DSL":
> > write server-side logic in Java, deploy to servers, call from thin
> clients.
> >
> > This approach is already used for Services, Compute, ScanQuery,
> > ContinuousQuery.
> >
> > If you have a better idea in mind, please share.
> >
> > On Fri, Mar 5, 2021 at 4:12 PM Ivan Daschinsky <ivanda...@gmail.com>
> > wrote:
> >
> > > >>> - Corresponding types should exist on server nodes
> > > Ok, but I suppose this is of questional usability, same for existing
> > > implementation for ScanQuery and ContinousQuery.
> > > For queries it's probably ok to add some custom filters and put them in
> > > servers' classpathes, but I can hardly imagine
> > > somebody wants to create custom Receiver this way.
> > >
> > > пт, 5 мар. 2021 г. в 15:54, Pavel Tupitsyn <ptupit...@apache.org>:
> > >
> > > > > How do you suggests to serialize this object?
> > > >
> > > > Like a normal binary object. This scenario already exists for
> ScanQuery
> > > and
> > > > ContinuousQuery filters.
> > > > - It works for Java and .NET; potentially for other platforms
> > > > - Corresponding types should exist on server nodes
> > > >
> > > > E.g. if a Java class `MyFilter { String containsText }` is loaded on
> > > server
> > > > nodes,
> > > > a Python thin client can easily write a corresponding BinaryObject
> with
> > > one
> > > > field to achieve server-side filtering.
> > > >
> > > >
> > > > > I also suppose, that closing should be done wia OP_RESOURCE_CLOSE
> > > >
> > > > Thanks, forgot to mention this - updated the IEP.
> > > >
> > > >
> > > > On Fri, Mar 5, 2021 at 3:42 PM Ivan Daschinsky <ivanda...@gmail.com>
> > > > wrote:
> > > >
> > > > > I also suppose, that closing should be done wia OP_RESOURCE_CLOSE.
> > > It'is
> > > > > more consistent and resembles with existing cursor api.
> > > > >
> > > > > пт, 5 мар. 2021 г. в 15:37, Ivan Daschinsky <ivanda...@gmail.com>:
> > > > >
> > > > > > >> Both proposed requests have a Flush flag - please see Details
> > > > sections
> > > > > > in the IEP.
> > > > > > Ok, sorry, I missed this.
> > > > > > >> StreamReceiver is public API [1]
> > > > > > I know it. But this "object" should contains custom logic for
> > putting
> > > > > data
> > > > > > in cache. How do you suggests to serialize this object?
> > > > > > Implement custom classloader for java? What about other
> platforms?
> > > > > >
> > > > > > I suppose that we should not add this field, because it is
> > confusing.
> > > > > > Everything will work for local unit tests and only for java.
> > > > > > But in real environment this will not work at all.
> > > > > >
> > > > > >
> > > > > > пт, 5 мар. 2021 г. в 15:23, Pavel Tupitsyn <ptupit...@apache.org
> >:
> > > > > >
> > > > > >> Ivan,
> > > > > >>
> > > > > >> > Receiver is mostly internal stuff
> > > > > >> StreamReceiver is public API [1]
> > > > > >>
> > > > > >> > STREAMER_FLUSH_REQUEST should be added too
> > > > > >> Both proposed requests have a Flush flag - please see Details
> > > sections
> > > > > in
> > > > > >> the IEP.
> > > > > >> When user code calls client-side Flush method, we send the
> current
> > > > > >> client-side batch with that flag enabled,
> > > > > >> and flush server-side batches too.
> > > > > >>
> > > > > >> A separate request for that does not seem to be necessary, or
> am I
> > > > > missing
> > > > > >> some different use case?
> > > > > >>
> > > > > >> [1]
> > > > > >>
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/stream/package-summary.html
> > > > > >>
> > > > > >>
> > > > > >> On Fri, Mar 5, 2021 at 2:50 PM Ivan Daschinsky <
> > ivanda...@gmail.com
> > > >
> > > > > >> wrote:
> > > > > >>
> > > > > >> > IMHO, also STREAMER_FLUSH_REQUEST should be added too. Client
> > can
> > > > > flush
> > > > > >> > large batches instead of closing resource.
> > > > > >> > IMHO this is preferable than creating streamer per batch.
> > > > > >> >
> > > > > >> > пт, 5 мар. 2021 г. в 14:48, Ivan Daschinsky <
> > ivanda...@gmail.com
> > > >:
> > > > > >> >
> > > > > >> > > Pavel, thank you for your effort.
> > > > > >> > >
> > > > > >> > > BTW, are you sure that receiver should be a param of
> > > > > >> > > STREAMER_START_REQUEST?
> > > > > >> > > Receiver is mostly internal stuff and I can hardly imagine
> why
> > > > > >> > > someone would decide to change defaults.
> > > > > >> > >
> > > > > >> > > пт, 5 мар. 2021 г. в 13:01, Pavel Tupitsyn <
> > > ptupit...@apache.org
> > > > >:
> > > > > >> > >
> > > > > >> > >> Igor,
> > > > > >> > >>
> > > > > >> > >> As per our private conversation, I'll try to provide some
> > perf
> > > > > >> > >> measurements
> > > > > >> > >> for those 4 variants, and more detailed descriptions too.
> > > > > >> > >>
> > > > > >> > >> Thanks!
> > > > > >> > >>
> > > > > >> > >> On Fri, Mar 5, 2021 at 12:55 PM Igor Sapego <
> > > isap...@apache.org>
> > > > > >> wrote:
> > > > > >> > >>
> > > > > >> > >> > Pavel,
> > > > > >> > >> >
> > > > > >> > >> > I've checked the IEP and I like it. The only thing that
> > > seems a
> > > > > bit
> > > > > >> > >> > confusing to me
> > > > > >> > >> > is that there are 4 different variants for clients but
> > there
> > > > are
> > > > > >> cons
> > > > > >> > >> and
> > > > > >> > >> > pros for
> > > > > >> > >> > different variants. Maybe at least few sentences should
> be
> > > > > written
> > > > > >> > here
> > > > > >> > >> to
> > > > > >> > >> > give developers who are not familiar with DataStreamer
> some
> > > > ideas
> > > > > >> and
> > > > > >> > >> make
> > > > > >> > >> > it easier for them to choose.
> > > > > >> > >> >
> > > > > >> > >> > Best Regards,
> > > > > >> > >> > Igor
> > > > > >> > >> >
> > > > > >> > >> >
> > > > > >> > >> > On Thu, Mar 4, 2021 at 3:14 PM Pavel Tupitsyn <
> > > > > >> ptupit...@apache.org>
> > > > > >> > >> > wrote:
> > > > > >> > >> >
> > > > > >> > >> > > Igniters,
> > > > > >> > >> > >
> > > > > >> > >> > > Please review the IEP [1] and let me know what you
> think.
> > > > > >> > >> > >
> > > > > >> > >> > > [1]
> > > > > >> > >> > >
> > > > > >> > >> > >
> > > > > >> > >> >
> > > > > >> > >>
> > > > > >> >
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-68%3A+Thin+Client+Data+Streamer
> > > > > >> > >> > >
> > > > > >> > >> >
> > > > > >> > >>
> > > > > >> > >
> > > > > >> > >
> > > > > >> > > --
> > > > > >> > > Sincerely yours, Ivan Daschinskiy
> > > > > >> > >
> > > > > >> >
> > > > > >> >
> > > > > >> > --
> > > > > >> > Sincerely yours, Ivan Daschinskiy
> > > > > >> >
> > > > > >>
> > > > > >
> > > > > >
> > > > > > --
> > > > > > Sincerely yours, Ivan Daschinskiy
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > Sincerely yours, Ivan Daschinskiy
> > > > >
> > > >
> > >
> > >
> > > --
> > > Sincerely yours, Ivan Daschinskiy
> > >
> >
>
>
> --
> Sincerely yours, Ivan Daschinskiy
>
