Hello everyone, In my opinion, it is indeed better to limit storing to the metastore by primitive type(map or list are also possible) or use the POJO from the ignite core. As Kirill correctly notice, right now, it is a problem not inside of the distributed metastore but inside of discovery. In fact, we can rewrite the sending metastore data in such a way that the discovery would think that there is just a simple byte array which shouldn't be deserialized. Right now, it understands that it is a serialized java object and it tries to deserialize it after receiving it. But this way requires more investigation about possible corner cases.
Nikolay, I also don't sure that your fix handles metastorage history correctly. As I can see you have tests only for one node but what happens if different nodes have different filters? or if we need to send history to the joining node but some of the keys don't pass the filter? Maybe I wrong but in the first eye, it can lead to different results/histories on different nodes which is a problem. I just briefly looked at your PR(so maybe I didn't understand something), I will try to do it more carefully at the nearest time. -- Best regards, Anton Kalashnikov 18.12.2020, 15:33, "Mekhanikov Denis" <dmekhani...@gmail.com>: > Nikolay, > > Thanks for your reply! > > I encountered a similar case to what you've described in point #1. I used a > private plugin that writes some information to the metastorage. > After that I decided to get rid of that plugin, while the information had > already been written to the metastorage. > Following the approach that you described and implemented in the PR, I'll > need to work with the flag to ignore certain keys in the metastorage forever. > That's quite inconvenient. > Wouldn't it be better if we just limited the set of allowed types that can be > stored in the metastorage? Instead of a POJO, a Map will be accepted. > > Denis > > On 18.12.2020, 13:59, "ткаленко кирилл" <tkalkir...@yandex.ru> wrote: > > Hello everybody! > > If you look at the stackTrace, the error is that deserialized objects are > being sent to listeners. > It may be more correct to send a raw arrays of bytes, and each plugin > will be able to process it if needed. > > 18.12.2020, 12:18, "Nikolay Izhikov" <nizhi...@apache.org>: > > Hello, Denis. > > > > It’s a known issue for me. > > Metastore is a private API, isn’t it? > > AFAICU it can occur for two reasons: > > > > * User migrates from custom Ignite fork that has private improvements > or plugins that write to the metastore. > > * We have a blocker bug and just remove some internal class that can be > written into metastore from distribution. > > > > I planned to fix it with some system flag. > > During startup administrator just sets a list of the metastore items > that should be ignored. > > Please, take a look at the PR [1] > > > > WDYT? > > > >> it’s better to limit the metastorage with storing primitives only > > > > I think that ability to write object is very useful and should stay. > > > > [1] https://github.com/apache/ignite/pull/8221 > > > >> 18 дек. 2020 г., в 12:06, Mekhanikov Denis <dmekhani...@gmail.com> > написал(а): > >> > >> Hi everyone! > >> > >> Ignite has a limitation that it can’t work with custom classes put > into metastorage: https://issues.apache.org/jira/browse/IGNITE-13642 > >> If you put a POJO into the metastorage, then Ignite will try to > deserialize it using the classes it finds on the classpath. If it can’t do > the deserialization, then the node will fail. > >> > >> There is an opinion that the metastorage wasn’t design for a case when > classes that can disappear from Ignite distribution. > >> If we follow this path, then it’s better to limit the metastorage with > storing primitives only, so that it’s impossible to occasionally put anything > breaking. > >> If a piece of configuration is put into the metastorage by a plugin, > then the plugin will be in charge of deserializing the configuration, and not > Ignite. > >> > >> Alternatively we can try to fix the metastorage and make it ignore > deserialization errors when they occur. > >> > >> What do you think? > >> > >> Denis
