Hi all! I am sorry for being late on that. I was trying to refactor the test in order not to be in need of any external tools or spinning up a server. I did forked the new repo and indeed my changes there, so a pull request is a matter of time!
Kind regards *Emmanouil Gkatziouras* https://egkatzioura.com/ | https://www.linkedin.com/in/gkatziourasemmanouil/ https://github.com/gkatzioura On Fri, 22 Nov 2019 at 20:45, Denis Magda <dma...@apache.org> wrote: > Hi Emmanouil, > > Do you have any questions or need any support from the community? > > - > Denis > > > On Sun, Nov 10, 2019 at 3:07 PM Saikat Maitra <saikat.mai...@gmail.com> > wrote: > >> Hi Emmanouil, >> >> Can you please take a looks at dev utils, if this is something you are >> looking for >> https://github.com/apache/ignite/tree/master/modules/dev-utils? >> >> IMO, if you can release Pub/Sub server in maven and then use it as >> dependency, that would be great. >> >> Regards, >> Saikat >> >> On Sun, Nov 10, 2019 at 5:00 PM Saikat Maitra <saikat.mai...@gmail.com> >> wrote: >> >> > Hi Emmanouil, >> > >> > The master branch has been initialized. I have raised a PR to migrate >> > Flink module to ignite-extensions. >> > >> > PR https://github.com/apache/ignite-extensions/pull/1 >> > Jira : https://issues.apache.org/jira/browse/IGNITE-12356 >> > >> > Regards, >> > Saikat >> > >> > >> > On Sun, Nov 10, 2019 at 3:39 PM Emmanouil Gkatziouras < >> > gkatzio...@gmail.com> wrote: >> > >> >> Hi all! >> >> >> >> This is an update. The integration is tested with mocks based on the >> >> original Ignite repo [1]. >> >> I am currently waiting for the first commits on the new integration >> >> project >> >> to take place in order to create my pull request accordingly to the >> >> initial >> >> commits. >> >> In order to simulate the Pub/Sub server in a test I used a test server >> >> implementation [2]. Unfortunately they don't have this util uploaded >> in a >> >> maven repo, thus it needs to be hosted on a maven repo (Apache License >> >> 2.0). Is there a repo for utilities that Ignite uses for the >> test/build? >> >> I have also created a ticket [3] >> >> >> >> Kind regards >> >> >> >> [1] https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub >> >> [2] https://github.com/GoogleCloudPlatform/kafka-pubsub-emulator >> >> [3] https://issues.apache.org/jira/browse/IGNITE-12262 >> >> >> >> *Emmanouil Gkatziouras* >> >> https://egkatzioura.com/ | >> >> https://www.linkedin.com/in/gkatziourasemmanouil/ >> >> https://github.com/gkatzioura >> >> >> >> >> >> On Fri, 25 Oct 2019 at 03:35, Saikat Maitra <saikat.mai...@gmail.com> >> >> wrote: >> >> >> >> > Hello Ilya, >> >> > >> >> > Thank you for your email. Yes, before we remove any integration we >> will >> >> > move them to another repository and release. >> >> > >> >> > We are thinking we will start with Pub/Sub Streamer implementation >> and >> >> > continue to move other integration as mentioned in the following doc. >> >> > >> >> > >> >> > >> >> >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations >> >> > >> >> > Regards, >> >> > Saikat >> >> > >> >> > >> >> > On Thu, Oct 24, 2019 at 11:00 AM Ilya Kasnacheev < >> >> > ilya.kasnach...@gmail.com> >> >> > wrote: >> >> > >> >> > > Hello! >> >> > > >> >> > > My take on this: we want to offload some of integrations, but >> please >> >> > commit >> >> > > and release them to another project first, and only then we will >> >> decide >> >> > to >> >> > > remove them from our repository. >> >> > > >> >> > > Please also make sure they are available with same artifact name >> and >> >> API. >> >> > > If this has to be changed, let's wait for AI 3.0 >> >> > > >> >> > > Regards, >> >> > > -- >> >> > > Ilya Kasnacheev >> >> > > >> >> > > >> >> > > ср, 23 окт. 2019 г. в 03:23, Saikat Maitra < >> saikat.mai...@gmail.com>: >> >> > > >> >> > > > Hello Emmanouil, >> >> > > > >> >> > > > As discussed earlier, I discussed with Apache Bahir community and >> >> they >> >> > > are >> >> > > > interested to have Apache Ignite extensions as part of Apache >> Bahir >> >> > > > project. >> >> > > > >> >> > > > Can you please share Pub/Sub streamer implementation details with >> >> > Apache >> >> > > > Bahir community and request for feedback. >> >> > > > >> >> > > > https://bahir.apache.org/ >> >> > > > >> >> > > > I have also requested for contributor access in Jira for Apache >> >> Bahir >> >> > > > project so that I can create issues and assign to myself. >> >> > > > >> >> > > > I can help with code reviews as well. >> >> > > > >> >> > > > Here is the email thread for reference >> >> > > > https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html >> >> > > > >> >> > > > Regards, >> >> > > > Saikat >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > >> >> > > > On Mon, Oct 21, 2019 at 7:54 PM Saikat Maitra < >> >> saikat.mai...@gmail.com >> >> > > >> >> > > > wrote: >> >> > > > >> >> > > > > Hello, >> >> > > > > >> >> > > > > As discussed I have created following discussion threads on our >> >> > > migration >> >> > > > > proposals for Apache Ignite extensions: >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > >> >> > > >> >> > >> >> >> http://apache-ignite-users.70518.x6.nabble.com/DISCUSS-Proposal-for-Ignite-Extensions-as-a-separate-Bahir-module-or-Incubator-project-td29829.html >> >> > > > > >> >> > > > > >> https://www.mail-archive.com/dev@bahir.apache.org/msg02703.html >> >> > > > > >> >> > > > > I will share update as I hear more information. >> >> > > > > >> >> > > > > Regards, >> >> > > > > Saikat >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > On Fri, Oct 18, 2019 at 9:03 PM Saikat Maitra < >> >> > saikat.mai...@gmail.com >> >> > > > >> >> > > > > wrote: >> >> > > > > >> >> > > > >> Hello Denis, >> >> > > > >> >> >> > > > >> Sure, sounds good. I will create a separate discussion thread >> to >> >> get >> >> > > > >> community feedback on whether we >> >> > > > >> should join the Bahir or kick-off "Ignite Extensions" project. >> >> > > > >> >> >> > > > >> Regards, >> >> > > > >> Saikat >> >> > > > >> >> >> > > > >> On Thu, Oct 17, 2019 at 2:21 PM Denis Magda < >> dma...@apache.org> >> >> > > wrote: >> >> > > > >> >> >> > > > >>> Folks, >> >> > > > >>> >> >> > > > >>> The concept of Apache Bahir is precisely what we need! >> Saikat, >> >> > thanks >> >> > > > for >> >> > > > >>> doing the research. Why I believe the * idea* fits us well: >> >> > > > >>> >> >> > > > >>> - All integrations can be stored in separate Github >> >> repositories >> >> > > and >> >> > > > >>> have their dev lifecycles. We've not obliged to couple all >> >> the >> >> > > > >>> integrations >> >> > > > >>> in a single repo. For instance, Spark can be located in a >> >> > > dedicated >> >> > > > >>> repo >> >> > > > >>> while streaming integrations might be in a single one. >> It's >> >> up >> >> > to >> >> > > > us. >> >> > > > >>> - All the repositories and integrations belong to ASF. >> We're >> >> not >> >> > > > >>> dumping >> >> > > > >>> them on Github but rather keep supporting and developing >> in >> >> > > > >>> accordance with >> >> > > > >>> ASF vision and practices. >> >> > > > >>> - There is a way to reward contributors via committership >> and >> >> > PMC >> >> > > > >>> membership. You won't get this if a project is just one of >> >> the >> >> > > > >>> millions of >> >> > > > >>> Github projects. >> >> > > > >>> >> >> > > > >>> Saikat, as Ignite PMC member, could you please kick-off a >> >> separate >> >> > > > >>> dev-list >> >> > > > >>> discussion to involve more community members and referring to >> >> this >> >> > > > >>> thread? >> >> > > > >>> I think the primary question the community needs to answer >> >> whether >> >> > we >> >> > > > >>> should join the Bahir or kick-off "Ignite Extensions" >> project? >> >> > > > >>> >> >> > > > >>> - >> >> > > > >>> Denis >> >> > > > >>> >> >> > > > >>> >> >> > > > >>> On Thu, Oct 17, 2019 at 2:54 AM Alexey Zinoviev < >> >> > > > zaleslaw....@gmail.com> >> >> > > > >>> wrote: >> >> > > > >>> >> >> > > > >>> > Maybe we could move all our Streaming Integrations there, >> but >> >> > what >> >> > > is >> >> > > > >>> about >> >> > > > >>> > maintaining and committer permissions to the new >> repositories? >> >> > > > >>> > I see the list of the committers and PMC members there >> >> > > > >>> > https://bahir.apache.org/community-members/ >> >> > > > >>> > >> >> > > > >>> > Could somebody from Ignite community be added to this list >> as >> >> a >> >> > > > >>> > PMC/committer to maintain Ignite integrations? >> >> > > > >>> > If yes, I'd happy to join this project and collaborate with >> >> these >> >> > > > guys >> >> > > > >>> > together >> >> > > > >>> > If no, and we should start from the zero level with >> external >> >> PRs >> >> > - >> >> > > > >>> hmmm, >> >> > > > >>> > it's better to have our own external repository with >> >> ApacheBahirr >> >> > > > >>> ideology. >> >> > > > >>> > >> >> > > > >>> > Also, I agree, that all connectors could be moved there (in >> >> > > > >>> ApacheBahirr >> >> > > > >>> > like repository), but not all modules from the page >> >> > > > >>> > >> >> > > > >>> > >> >> > > > >>> >> >> > > > >> >> > > >> >> > >> >> >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations >> >> > > > >>> > because >> >> > > > >>> > they use different parts of core modules and could be >> >> developed >> >> > > with >> >> > > > >>> > different velocity. >> >> > > > >>> > >> >> > > > >>> > In reality, before creation of new repositories we need >> wide >> >> > > > discussion >> >> > > > >>> > based on the document mentioned above. >> >> > > > >>> > >> >> > > > >>> > P.S. Of course, we could experiment in the next release 2.8 >> >> with >> >> > > one >> >> > > > >>> or two >> >> > > > >>> > integrations like twitter or ZeroMQ >> >> > > > >>> > Also, I'd like Denis Magda idea for the separate >> repositories >> >> in >> >> > > > Apache >> >> > > > >>> > Github to maintain them by anybody. >> >> > > > >>> > >> >> > > > >>> > >> >> > > > >>> > чт, 17 окт. 2019 г. в 05:02, Saikat Maitra < >> >> > > saikat.mai...@gmail.com >> >> > > > >: >> >> > > > >>> > >> >> > > > >>> > > Hi Denis, Emmanouil >> >> > > > >>> > > >> >> > > > >>> > > Thank you for your email. I think creating a separate >> >> > integration >> >> > > > >>> project >> >> > > > >>> > > in github and also proposing it as an apache incubator >> >> project >> >> > > will >> >> > > > >>> be a >> >> > > > >>> > > good move. The other separate integration modules >> >> > > > >>> > > >> >> > > > >>> > > >> >> > > > >>> > >> >> > > > >>> >> >> > > > >> >> > > >> >> > >> >> >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations >> >> > > > >>> > > can >> >> > > > >>> > > be moved to this new apache incubator project. >> >> > > > >>> > > >> >> > > > >>> > > There are similar integration available for Flink and >> Spark >> >> in >> >> > > > Apache >> >> > > > >>> > Bahir >> >> > > > >>> > > https://bahir.apache.org/ >> >> > > > >>> > > >> >> > > > >>> > > Do you think we should reach out to Apache Bahir >> community >> >> > with a >> >> > > > >>> > proposal >> >> > > > >>> > > or we should plan to create a separate Apache Incubator >> >> > project? >> >> > > > >>> > > >> >> > > > >>> > > Regards, >> >> > > > >>> > > Saikat >> >> > > > >>> > > >> >> > > > >>> > > >> >> > > > >>> > > >> >> > > > >>> > > On Wed, Oct 16, 2019 at 6:21 AM Emmanouil Gkatziouras < >> >> > > > >>> > > gkatzio...@gmail.com> >> >> > > > >>> > > wrote: >> >> > > > >>> > > >> >> > > > >>> > > > Hi all! >> >> > > > >>> > > > >> >> > > > >>> > > > Based on the discussions most probably this is going to >> >> be on >> >> > > > >>> another >> >> > > > >>> > > > GitHub repo. Therefore I will prepare a standalone >> project >> >> > with >> >> > > > the >> >> > > > >>> > > Pub/Sub >> >> > > > >>> > > > module and once the repository is created a pull >> request >> >> > shall >> >> > > be >> >> > > > >>> > issued. >> >> > > > >>> > > > If there is anything else I can do in order to >> facilitate >> >> > this >> >> > > > >>> please >> >> > > > >>> > let >> >> > > > >>> > > > me know. >> >> > > > >>> > > > >> >> > > > >>> > > > If a ticket is created for this issue, I shall be able >> to >> >> > > create >> >> > > > a >> >> > > > >>> pull >> >> > > > >>> > > > request and thus a build will take place on ignite's >> CI. >> >> > > > >>> > > > If everyone is aligned with adding a Pub/Sub >> >> implementation >> >> > > may I >> >> > > > >>> > create >> >> > > > >>> > > a >> >> > > > >>> > > > ticket on JIRA? >> >> > > > >>> > > > >> >> > > > >>> > > > My next question has to do on how Ignite configures the >> >> build >> >> > > > jobs >> >> > > > >>> in >> >> > > > >>> > > Team >> >> > > > >>> > > > City. The unit tests for my implementation need a local >> >> > Pub/Sub >> >> > > > >>> server >> >> > > > >>> > up >> >> > > > >>> > > > and running (they provide one for testing purposes). >> Will >> >> > this >> >> > > > take >> >> > > > >>> > > effect >> >> > > > >>> > > > in a form of a build script inside my implementation or >> >> is it >> >> > > > >>> something >> >> > > > >>> > > > that should be configured on Team City? >> >> > > > >>> > > > >> >> > > > >>> > > > Kind regards, >> >> > > > >>> > > > Emmanouil >> >> > > > >>> > > > >> >> > > > >>> > > > >> >> > > > >>> > > > *Emmanouil Gkatziouras* >> >> > > > >>> > > > https://egkatzioura.com/ | >> >> > > > >>> > > > https://www.linkedin.com/in/gkatziourasemmanouil/ >> >> > > > >>> > > > https://github.com/gkatzioura >> >> > > > >>> > > > >> >> > > > >>> > > > >> >> > > > >>> > > > On Tue, 15 Oct 2019 at 23:27, Denis Magda < >> >> dma...@apache.org >> >> > > >> >> > > > >>> wrote: >> >> > > > >>> > > > >> >> > > > >>> > > > > Emmanouil, Saikat, >> >> > > > >>> > > > > >> >> > > > >>> > > > > After contemplating on this, let me propose another >> way >> >> > with >> >> > > > this >> >> > > > >>> > > > > particular integration that is aligned with our >> >> > > modularization >> >> > > > >>> > > endeavor. >> >> > > > >>> > > > > The modularization [1] defines the Ignite core, >> Ignite >> >> > > modules, >> >> > > > >>> and >> >> > > > >>> > > > > independent integrations. The first two are to be >> >> located >> >> > in >> >> > > > >>> Ignite >> >> > > > >>> > > > > repositories and to be tested/updated by the >> community >> >> for >> >> > > > every >> >> > > > >>> > > release. >> >> > > > >>> > > > > The modules are a subset of existing integrations >> that >> >> are >> >> > > > >>> adopted >> >> > > > >>> > > widely >> >> > > > >>> > > > > and require community attention. >> >> > > > >>> > > > > >> >> > > > >>> > > > > As for the independent integrations [1], those are >> >> supposed >> >> > > to >> >> > > > be >> >> > > > >>> > > located >> >> > > > >>> > > > > in a separate GitHub repo. I would advise us to start >> >> > > > >>> implementing >> >> > > > >>> > this >> >> > > > >>> > > > > concept with the Pub/Sub integration. Even though it >> >> will >> >> > be >> >> > > > >>> located >> >> > > > >>> > > in a >> >> > > > >>> > > > > separate Github repo, it will be listed among >> officially >> >> > > > >>> available >> >> > > > >>> > > > > integrations on the Ignite website featuring >> Emmanouil >> >> as a >> >> > > > >>> > > contributor. >> >> > > > >>> > > > > >> >> > > > >>> > > > > What do you think? >> >> > > > >>> > > > > >> >> > > > >>> > > > > >> >> > > > >>> > > > > [1] >> >> > > > >>> > > > > >> >> > > > >>> > > > > >> >> > > > >>> > > > >> >> > > > >>> > > >> >> > > > >>> > >> >> > > > >>> >> >> > > > >> >> > > >> >> > >> >> >> https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations >> >> > > > >>> > > > > >> >> > > > >>> > > > > >> >> > > > >>> > > > > >> >> > > > >>> > > > > - >> >> > > > >>> > > > > Denis >> >> > > > >>> > > > > >> >> > > > >>> > > > > >> >> > > > >>> > > > > On Mon, Oct 14, 2019 at 9:47 PM Saikat Maitra < >> >> > > > >>> > saikat.mai...@gmail.com >> >> > > > >>> > > > >> >> > > > >>> > > > > wrote: >> >> > > > >>> > > > > >> >> > > > >>> > > > > > Hi Emmanouil, >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > The changes looks good to me. I wanted to check if >> you >> >> > are >> >> > > > also >> >> > > > >>> > able >> >> > > > >>> > > to >> >> > > > >>> > > > > > validate that the once data is added to Ignite >> cluster >> >> > you >> >> > > > are >> >> > > > >>> also >> >> > > > >>> > > > able >> >> > > > >>> > > > > to >> >> > > > >>> > > > > > access it using another Ignite client or using rest >> >> > > > endpoints. >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > We use Teamcity for CI process, the details are as >> >> > follows >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > Trigger validation of those test suites that have >> been >> >> > > > >>> affected by >> >> > > > >>> > > your >> >> > > > >>> > > > > > changes on TeamCity <http://ci.ignite.apache.org/>: >> < >> >> > > > >>> > > > > http://204.14.53.153/ >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > - Locate a test suite you have to check, press >> >> button >> >> > > > named >> >> > > > >>> > "..." >> >> > > > >>> > > > that >> >> > > > >>> > > > > > is located on the left of "Run" button. "Run >> custom >> >> > > build" >> >> > > > >>> > dialog >> >> > > > >>> > > > will >> >> > > > >>> > > > > > appear; >> >> > > > >>> > > > > > - Go to "Changes" tab and choose >> >> > > > >>> > "pull/<pull-request-number>/head" >> >> > > > >>> > > > in >> >> > > > >>> > > > > > "Build branch" dropdown list; >> >> > > > >>> > > > > > - Press "Run build" button and monitor tests >> >> results. >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > @Roman rsht...@yahoo.com - can you also please >> take a >> >> > > look? >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > Regards, >> >> > > > >>> > > > > > Saikat >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > On Sun, Oct 13, 2019 at 6:20 PM Emmanouil >> Gkatziouras >> >> < >> >> > > > >>> > > > > > gkatzio...@gmail.com> >> >> > > > >>> > > > > > wrote: >> >> > > > >>> > > > > > >> >> > > > >>> > > > > > > Hi @Saikat Maitra >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > > I have my implementation on an Ignite fork on my >> >> > personal >> >> > > > >>> GitHub >> >> > > > >>> > > > > account >> >> > > > >>> > > > > > > [1] >> >> > > > >>> > > > > > > It is based on the Kafka implementation which is >> of >> >> a >> >> > > > polling >> >> > > > >>> > > nature. >> >> > > > >>> > > > > > > GCP does not provide a service like Kafka. >> Instead >> >> it >> >> > > > >>> provides us >> >> > > > >>> > > > with >> >> > > > >>> > > > > > > Pub/Sub which is just like Kafka. >> >> > > > >>> > > > > > > Therefore using the Kafka streamer as a guideline >> >> made >> >> > > more >> >> > > > >>> sense >> >> > > > >>> > > to >> >> > > > >>> > > > > me. >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > > Apart from the implementation I have also added a >> >> unit >> >> > > > test. >> >> > > > >>> > > > > > > It definitely needs some polishing and a double >> >> check >> >> > on >> >> > > my >> >> > > > >>> side >> >> > > > >>> > to >> >> > > > >>> > > > > make >> >> > > > >>> > > > > > > the review smooth. >> >> > > > >>> > > > > > > For the tests a local Pub/Sub [2] server has to >> be >> >> > setup >> >> > > > >>> however >> >> > > > >>> > I >> >> > > > >>> > > am >> >> > > > >>> > > > > not >> >> > > > >>> > > > > > > sure which is the best way to do so with regards >> to >> >> > your >> >> > > > >>> CI/CD. >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > > [1] >> >> > > > >>> > > >> >> > https://github.com/gkatzioura/ignite/tree/pubsub/modules/pubsub >> >> > > > >>> > > > > > > [2] >> >> > > > >>> https://egkatzioura.com/2019/09/22/pub-sub-local-emulator/ >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > > *Emmanouil Gkatziouras* >> >> > > > >>> > > > > > > https://egkatzioura.com/ | >> >> > > > >>> > > > > > > >> https://www.linkedin.com/in/gkatziourasemmanouil/ >> >> > > > >>> > > > > > > https://github.com/gkatzioura >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > > On Sat, 12 Oct 2019 at 02:11, Saikat Maitra < >> >> > > > >>> > > saikat.mai...@gmail.com >> >> > > > >>> > > > > >> >> > > > >>> > > > > > > wrote: >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > > > Hello Emmanouil, >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > > Can you please share more info about the >> >> > > implementation? >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > > We have similar implementation for data >> streamers >> >> > with >> >> > > > >>> Kafka >> >> > > > >>> > and >> >> > > > >>> > > > > > RocketMQ >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > > >> >> > https://apacheignite-mix.readme.io/docs/kafka-streamer >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > > >> >> > > > https://apacheignite-mix.readme.io/docs/rocketmq-streamer >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > > Please review and share if you have any >> questions. >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > > Regards, >> >> > > > >>> > > > > > > > Saikat >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > > On Fri, Oct 11, 2019 at 4:56 PM Emmanouil >> >> > Gkatziouras < >> >> > > > >>> > > > > > > > gkatzio...@gmail.com> >> >> > > > >>> > > > > > > > wrote: >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > > > Hi all, >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > > By mentioning Pub/Sub I mean Google Cloud's >> >> Pub/Sub >> >> > > [1] >> >> > > > >>> > > > > > > > > Yes it is indeed cloud specific. It is the >> >> > messaging >> >> > > > >>> service >> >> > > > >>> > > > > provided >> >> > > > >>> > > > > > > by >> >> > > > >>> > > > > > > > > GCP. >> >> > > > >>> > > > > > > > > I have made the integration with it, but I >> >> haven't >> >> > > > >>> published >> >> > > > >>> > it >> >> > > > >>> > > > yet >> >> > > > >>> > > > > > (I >> >> > > > >>> > > > > > > am >> >> > > > >>> > > > > > > > > waiting for the discussion/ticket and proceed >> >> with >> >> > > the >> >> > > > >>> > > > > contribution). >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > > Kind regards, >> >> > > > >>> > > > > > > > > Emmanouil >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > > [1] >> >> https://cloud.google.com/pubsub/docs/overview >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > > *Emmanouil Gkatziouras* >> >> > > > >>> > > > > > > > > https://egkatzioura.com/ | >> >> > > > >>> > > > > > > > > >> >> https://www.linkedin.com/in/gkatziourasemmanouil/ >> >> > > > >>> > > > > > > > > https://github.com/gkatzioura >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > > On Fri, 11 Oct 2019 at 21:45, Denis Magda < >> >> > > > >>> dma...@apache.org >> >> > > > >>> > > >> >> > > > >>> > > > > wrote: >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > > > Hello Emmanouil, >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > > Are you referring to any specific >> >> implementation >> >> > > > >>> tailored >> >> > > > >>> > > for a >> >> > > > >>> > > > > > > cloud? >> >> > > > >>> > > > > > > > If >> >> > > > >>> > > > > > > > > > I'm not mistaken you were mentioning >> AWS/GCE. >> >> Is >> >> > > > there >> >> > > > >>> any >> >> > > > >>> > > > Github >> >> > > > >>> > > > > > > > source >> >> > > > >>> > > > > > > > > > code? >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > > Here is a list of existing integrations >> with >> >> > > > streaming >> >> > > > >>> and >> >> > > > >>> > > > > pub/sub >> >> > > > >>> > > > > > > > > > frameworks: >> >> > > > >>> > > > > > > > > > >> >> https://apacheignite-mix.readme.io/docs/overview >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > > with the source code in Ignite master: >> >> > > > >>> > > > > > > > > > >> >> > > https://github.com/apache/ignite/tree/master/modules >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > > Check out those resources to see what are >> the >> >> > most >> >> > > > >>> suitable >> >> > > > >>> > > > > Ignite >> >> > > > >>> > > > > > > APIs >> >> > > > >>> > > > > > > > > for >> >> > > > >>> > > > > > > > > > such integrations. >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > > @Saikat Maitra <saikat.mai...@gmail.com> , >> >> > @Roman >> >> > > < >> >> > > > >>> > > > > > rsht...@yahoo.com >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > > , >> >> > > > >>> > > > > > > > > as >> >> > > > >>> > > > > > > > > > streaming technologies >> >> contributors/maintainers, >> >> > > > could >> >> > > > >>> you >> >> > > > >>> > > > please >> >> > > > >>> > > > > > > join >> >> > > > >>> > > > > > > > > this >> >> > > > >>> > > > > > > > > > thread? >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > > - >> >> > > > >>> > > > > > > > > > Denis >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > > On Thu, Oct 10, 2019 at 7:58 AM Emmanouil >> >> > > > Gkatziouras < >> >> > > > >>> > > > > > > > > > gkatzio...@gmail.com> >> >> > > > >>> > > > > > > > > > wrote: >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > > > Hello everyone, >> >> > > > >>> > > > > > > > > > > >> >> > > > >>> > > > > > > > > > > I started using Ignite lately. Part of my >> >> work >> >> > > > >>> involves >> >> > > > >>> > > > > Pub/Sub, >> >> > > > >>> > > > > > > > thus I >> >> > > > >>> > > > > > > > > > > created a pub/sub streamer. >> >> > > > >>> > > > > > > > > > > Pub/Sub is pretty close to Kafka. >> >> > > > >>> > > > > > > > > > > Here's how the streamer works. Every node >> >> > creates >> >> > > > an >> >> > > > >>> > > instance >> >> > > > >>> > > > > of >> >> > > > >>> > > > > > a >> >> > > > >>> > > > > > > > > > Pub/Sub >> >> > > > >>> > > > > > > > > > > client. >> >> > > > >>> > > > > > > > > > > The clients subscribe to the same topic >> >> using >> >> > the >> >> > > > >>> same >> >> > > > >>> > > > > > subscription >> >> > > > >>> > > > > > > > > name. >> >> > > > >>> > > > > > > > > > > A message is being sent and one of the >> nodes >> >> > will >> >> > > > >>> receive >> >> > > > >>> > > the >> >> > > > >>> > > > > > > > message, >> >> > > > >>> > > > > > > > > > > process it and then forward it to Ignite >> >> using >> >> > > the >> >> > > > >>> > > extractor >> >> > > > >>> > > > > > > > provided. >> >> > > > >>> > > > > > > > > > > >> >> > > > >>> > > > > > > > > > > Any thoughts? >> >> > > > >>> > > > > > > > > > > >> >> > > > >>> > > > > > > > > > > *Emmanouil Gkatziouras* >> >> > > > >>> > > > > > > > > > > https://egkatzioura.com/ | >> >> > > > >>> > > > > > > > > > > >> >> > > https://www.linkedin.com/in/gkatziourasemmanouil/ >> >> > > > >>> > > > > > > > > > > https://github.com/gkatzioura >> >> > > > >>> > > > > > > > > > > >> >> > > > >>> > > > > > > > > > >> >> > > > >>> > > > > > > > > >> >> > > > >>> > > > > > > > >> >> > > > >>> > > > > > > >> >> > > > >>> > > > > > >> >> > > > >>> > > > > >> >> > > > >>> > > > >> >> > > > >>> > > >> >> > > > >>> > >> >> > > > >>> >> >> > > > >> >> >> > > > >> >> > > >> >> > >> >> >> > >> >
