Hi Denis, Please find the highlighted comments inline.
Data will change every time we hit select * from table in database..(should drop or remove complete data and insert) I don't really understand what you mean. Will the data change in database or Ignite? should drop or remove complete data and insert What do you mean by that? You want to remove all data from cache and then load it back from scratch? n Data in the database table changes very frequently and every time we query the table we get different data. n My question was : Instead of updating the data already loaded in cache, can we clear the cache and reload again? Thanks & Regards, Sriveena From: Denis Mekhanikov [mailto:dmekhani...@gmail.com] Sent: Wednesday, July 11, 2018 1:41 PM To: user@ignite.apache.org Subject: Re: Information regarding Ignite Web Console Sriveena, > what if the query doesn't have any primary key or unique id....but still have > to load the entire table data in ignite for further join querying with other > caches.... You can implement cache store in such way, that it will assign a synthetic key for every loaded entry in CacheStore.loadCache(...)<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fignite.apache.org%2Freleases%2Flatest%2Fjavadoc%2Forg%2Fapache%2Fignite%2Fcache%2Fstore%2FCacheStore.html%23loadCache-org.apache.ignite.lang.IgniteBiInClosure-java.lang.Object...-&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7C559bc983482a4b4df21708d5e705eff5%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636668935073486479&sdata=0Z6Y3aFz987CW1cHrFoEAI65aSo46BXw%2FLdFka2wo00%3D&reserved=0>. But it will be hard to use read-through in this case. > Data will change every time we hit select * from table in database..(should > drop or remove complete data and insert) I don't really understand what you mean. Will the data change in database or Ignite? > should drop or remove complete data and insert What do you mean by that? You want to remove all data from cache and then load it back from scratch? Denis вт, 10 июл. 2018 г. в 19:54, Sriveena Mattaparthi <sriveena.mattapar...@ekaplus.com<mailto:sriveena.mattapar...@ekaplus.com>>: Hi Denis, Thanks for the detailed explanation ... 1. what if the query doesn't have any primary key or unique id....but still have to load the entire table data in ignite for further join querying with other caches.... 2. Data will change every time we hit select * from table in database..(should drop or remove complete data and insert) Please suggest right pointer in such cases.. Regards, Sriveena ________________________________ From: Denis Mekhanikov <dmekhani...@gmail.com<mailto:dmekhani...@gmail.com>> Sent: 10 July 2018 19:16:56 To: user@ignite.apache.org<mailto:user@ignite.apache.org> Subject: Re: Information regarding Ignite Web Console Sriveena, Read-through works as follows: when you do a cache.get(...), and a record is not in the cache, then it goes to the configured CacheStore and requests the value from it. When the same key is read from cache next time, then call to the underlying database doesn't happen, and caches value is returned. So, auto-synchronization doesn't happen. Cache is populated on demand. And it works regardless of the key type. Denis вт, 10 июл. 2018 г. в 14:12, Sriveena Mattaparthi <sriveena.mattapar...@ekaplus.com<mailto:sriveena.mattapar...@ekaplus.com>>: Hi, How does DataGrid readthrough works? 1. How does it auto synchronize the cache with database data? 2. Will it poll the underlying database? 3. Will it incrementally pull the data based on the key specified on the cache? 4. Will the read through works if the cache key datatype is different other than a number(like String, Composite key or timestamp) Please confirm. Thanks & Regards, Sriveena From: Denis Mekhanikov [mailto:dmekhani...@gmail.com<mailto:dmekhani...@gmail.com>] Sent: Wednesday, June 27, 2018 6:37 PM To: user@ignite.apache.org<mailto:user@ignite.apache.org> Subject: Re: Information regarding Ignite Web Console Sriveena, You can have objects of different types in one cache, but querying it will be tricky. You will have to configure QueryEntities<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite-sql.readme.io%2Fdocs%2Fschema-and-indexes%23section-queryentity-based-configuration&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7C6e1e06d9d5c0452da8da08d5dc2ee46a%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636657016326903455&sdata=Z093el7e%2Fj%2BngjoDdJjUwWNUWfPu4Yb0OZgKv3ZEiis%3D&reserved=0> for your data, that will describe, which fields are available for querying. Annotation based configuration<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite-sql.readme.io%2Fdocs%2Fschema-and-indexes%23section-queryentity-based-configuration&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7C6e1e06d9d5c0452da8da08d5dc2ee46a%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636657016326903455&sdata=Z093el7e%2Fj%2BngjoDdJjUwWNUWfPu4Yb0OZgKv3ZEiis%3D&reserved=0> is also available. Querying nested object is also possible, if you configure the query entities properly: https://apacheignite-sql.readme.io/docs/schema-and-indexes#section-indexing-nested-objects<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite-sql.readme.io%2Fdocs%2Fschema-and-indexes%23section-indexing-nested-objects&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7C6e1e06d9d5c0452da8da08d5dc2ee46a%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636657016326903455&sdata=1amkW4husY1fLSN%2FKtEBx8app5u%2FkUJhqiA4JCYgQ0o%3D&reserved=0> So, if you want to run SQL queries over your data, it should have some concrete schema. Denis ср, 27 июн. 2018 г. в 14:08, Sriveena Mattaparthi <sriveena.mattapar...@ekaplus.com<mailto:sriveena.mattapar...@ekaplus.com>>: Thank you so much for the quicker responses unlike any other forums..I really appreciate that. One last question Denis, we have plan to load all the mongodb collections to ignite cache and perform complex aggregations and join in memory. But Unlike any RDBMS data stores we cannot have fixed model objects for each collections as each document in the collection may have its own columns and datatypes. Could you please suggest, if ignite is the choice for this kind of scenario where same mongo collection have different type of data. Please note that we have tried using BinaryObject, but we are stuck that ignite doesn’t support querying on the inner binaryobject.( binaryobject inside a binaryobject – sub documents, array inside a mongo document) Thanks & Regards, Sriveena From: Denis Mekhanikov [mailto:dmekhani...@gmail.com<mailto:dmekhani...@gmail.com>] Sent: Wednesday, June 27, 2018 4:02 PM To: user@ignite.apache.org<mailto:user@ignite.apache.org> Subject: Re: Information regarding Ignite Web Console Sriveena, CacheStore<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fignite.apache.org%2Freleases%2Flatest%2Fjavadoc%2Forg%2Fapache%2Fignite%2Fcache%2Fstore%2FCacheStore.html&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cf08122ce8c394e2c7a1e08d5dc193829%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636656923260911744&sdata=7LEcZ6l%2FFMHm7ZZ518w4WH%2FBm8ylC3yhTDTl05AcZ0M%3D&reserved=0> extends the CacheWriter<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstatic.javadoc.io%2Fjavax.cache%2Fcache-api%2F1.0.0%2Fjavax%2Fcache%2Fintegration%2FCacheWriter.html&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cf08122ce8c394e2c7a1e08d5dc193829%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636656923260911744&sdata=UBNLjs2J4E4IZG6JGhToO6QzOr7QXkNo7OFZycVYbq0%3D&reserved=0> interface, which has delete<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstatic.javadoc.io%2Fjavax.cache%2Fcache-api%2F1.0.0%2Fjavax%2Fcache%2Fintegration%2FCacheWriter.html%23delete(java.lang.Object)&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cf08122ce8c394e2c7a1e08d5dc193829%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636656923260921745&sdata=2q1Mutj83625oVhVIEiawuiarAsK8UNmyZvrbk9J7nE%3D&reserved=0> and deleteAll<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstatic.javadoc.io%2Fjavax.cache%2Fcache-api%2F1.0.0%2Fjavax%2Fcache%2Fintegration%2FCacheWriter.html%23deleteAll(java.util.Collection)&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cf08122ce8c394e2c7a1e08d5dc193829%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636656923260921745&sdata=NIJFLEwPbXOgpgkja9m48c8hENGNg1dTOyghSzaIiJ4%3D&reserved=0> methods, which will be called by Ignite, when you remove entries from Ignite caches. write<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstatic.javadoc.io%2Fjavax.cache%2Fcache-api%2F1.0.0%2Fjavax%2Fcache%2Fintegration%2FCacheWriter.html%23write(javax.cache.Cache.Entry)&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cf08122ce8c394e2c7a1e08d5dc193829%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636656923260931759&sdata=tBdc%2FPmMicFBKc82w7s10TvP5r9Sl7%2B7FLPJidwsFsw%3D&reserved=0> and writeAll<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstatic.javadoc.io%2Fjavax.cache%2Fcache-api%2F1.0.0%2Fjavax%2Fcache%2Fintegration%2FCacheWriter.html%23writeAll(java.util.Collection)&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cf08122ce8c394e2c7a1e08d5dc193829%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636656923260931759&sdata=Sw5W5BSEgyrBRfoIUpx7pwIfYl5gXSNXtYefE4ghmBk%3D&reserved=0> methods will be called, when you put some data into cache. It works the same way with MySQL or any other relational DB. So, CacheStore gives you a possibility to implement synchronization with any external data source. You can do it for MongoDB, if you need. Denis ср, 27 июн. 2018 г. в 13:18, Sriveena Mattaparthi <sriveena.mattapar...@ekaplus.com<mailto:sriveena.mattapar...@ekaplus.com>>: Thanks Denis for the pointers. But the case explained in https://www.youtube.com/watch?v=XBtI2Z01kAA<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DXBtI2Z01kAA&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cf08122ce8c394e2c7a1e08d5dc193829%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636656923260941765&sdata=HLmtH02Dr4h455PggnRBQxe0eZJxENxp7nDlPXb5xek%3D&reserved=0> - Using Ignite Web Console for Automatic RDBMS Integration with Apache Ignite - Part 3. shows that any deletions happening on ignite cache are getting reflected in the mysql database. Which I thought is ignite feature of autosyncing data to and from RDBMS databases. Please correct my understanding. Thanks & Regards, Sriveena From: Denis Mekhanikov [mailto:dmekhani...@gmail.com<mailto:dmekhani...@gmail.com>] Sent: Wednesday, June 27, 2018 3:28 PM To: user@ignite.apache.org<mailto:user@ignite.apache.org> Subject: Re: Information regarding Ignite Web Console Sriveena, Web Console can generate schema in Ignite based on an existing schema in a relational database. But you can do it yourself, without use of Web Console. Web Console only makes it easier to wire-up stuff, i.e. generate required data classes and configuration. Unfortunately, this feature is not available for NoSQL databases. But you still can configure everything yourself. You will need to implement a CacheStore<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fignite.apache.org%2Freleases%2Flatest%2Fjavadoc%2Forg%2Fapache%2Fignite%2Fcache%2Fstore%2FCacheStore.html&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cf4918a96e28644d507b908d5dc1491ba%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636656903279754570&sdata=V09AYASdhDUexLS4ZNs44zvEWjJ1lmZFqjVRwQft2xs%3D&reserved=0>, that will load data from your database into Ignite. Here is documentation on this feature: https://apacheignite.readme.io/docs/3rd-party-store<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fapacheignite.readme.io%2Fdocs%2F3rd-party-store&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7Cf4918a96e28644d507b908d5dc1491ba%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636656903279754570&sdata=9w94GuXNlswyUb9XshSsZtol3Qp8ib7GIk9dpwbZiYQ%3D&reserved=0> Denis ср, 27 июн. 2018 г. в 12:41, Sriveena Mattaparthi <sriveena.mattapar...@ekaplus.com<mailto:sriveena.mattapar...@ekaplus.com>>: Hi Denis, Thank you so much for the quick turn around Could you also confirm if there is a similar integration from ignite to NoSQL database like MongoDB ? Our requirement also has mongo db changes to be automatically synced up to Ignite? Please help. Thanks & Regards, Sriveena From: Denis Mekhanikov [mailto:dmekhani...@gmail.com<mailto:dmekhani...@gmail.com>] Sent: Wednesday, June 27, 2018 2:58 PM To: user@ignite.apache.org<mailto:user@ignite.apache.org> Subject: Re: Information regarding Ignite Web Console Sriveena, You can find sources of Web Console in the Ignite Git repository: https://github.com/apache/ignite/tree/master/modules/web-console<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fignite%2Ftree%2Fmaster%2Fmodules%2Fweb-console&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7C1bd39c9b865f43eca6d408d5dc104bee%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636656884921777570&sdata=J6LSkoURVCvHJDnKyWrjpFRj02E9T0njpXSxFJPJAvU%3D&reserved=0> If you want to run Web Console by yourself, you can use the following Docker image: https://hub.docker.com/r/apacheignite/web-console-standalone/<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fhub.docker.com%2Fr%2Fapacheignite%2Fweb-console-standalone%2F&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7C1bd39c9b865f43eca6d408d5dc104bee%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636656884921777570&sdata=xQKYRGlEn9UQQ2EDM9nc6RTZLEbHH7eCl7vGCf2UbLM%3D&reserved=0> There is also Web Console, hosted by GridGain, available at https://console.gridgain.com/<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fconsole.gridgain.com%2F&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7C1bd39c9b865f43eca6d408d5dc104bee%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636656884921787575&sdata=yyxqlC%2BcnR9yLRwgGxD6aCvy0t9eh2J4djvNsztr2YM%3D&reserved=0> Denis ср, 27 июн. 2018 г. в 10:14, Sriveena Mattaparthi <sriveena.mattapar...@ekaplus.com<mailto:sriveena.mattapar...@ekaplus.com>>: Hi, I am interested to know if Ignite Webconsole is opensource as the document says Ignite can integrate with any relational database (RDBMS) that supports a JDBC driver - Oracle, PostgreSQL, Microsoft SQL Server, and MySQL Ignite supports automatic RDBMS integration via Ignite Web Console<https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fconsole.gridgain.com%2F&data=02%7C01%7CSriveena.Mattaparthi%40ekaplus.com%7C1bd39c9b865f43eca6d408d5dc104bee%7C2a5b4e9716be4be4b2d40f3fcb3d373c%7C1%7C0%7C636656884921797580&sdata=CorLQYS%2FCirN%2BOxK72fvwKFGL5E%2BSQ%2FY4Qn0hPzNvW0%3D&reserved=0>. Please confirm. Thanks & Regards, Sriveena
