Hello!
I have found that you can avoid compute job type incompatibility problem
(please see history below) by setting
.setMarshaller(new OptimizedMarshaller())
in your Ignite configuration on all nodes.
However, it is not clear at all why this is needed. Anybody can help? Why
compute jobs are processed by capricious BinaryMarshaller?
Regards,
--
Ilya Kasnacheev
ср, 19 дек. 2018 г. в 02:29, Gert Dubois <gert.dub...@trendminer.com>:
> Hey Ilya,
>
> I opened a ticket on the ignite Jira about it.
> https://issues.apache.org/jira/browse/IGNITE-10717
>
> I attached a zip file containing a maven project with sample code that
> reproduces our issue. Reproducing the issue is rather easy though
>
> 1. Have a client + server, with peer class loading enabled
> 2. Create a simple class that implements IgniteRunnable with some class
> field
> 3. Execute an instance of this class on the ignite cluster
> 4. Change the type of the field
> 5. Recompile and execute again. Now it breaks because the class can't be
> serialized using the binary marshaller.
>
> Code to execute these steps is included in the maven project.
>
> On Tue, Dec 18, 2018 at 12:04 PM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com> wrote:
>
>> Hello!
>>
>> Can you show an example of "Ignite Runnables conflicting on the Binary
>> Marshaller"? As a small code snippet perhaps?
>>
>> Maybe I could recommend something but I lack understanding of your use
>> case.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пн, 17 дек. 2018 г. в 18:12, Gert Dubois <gert.dub...@trendminer.com>:
>>
>>> Thanks for the reply.
>>>
>>> Everything related to the cache we understand the current architecture,
>>> in our code base we'll probably treat everything cache related the same as
>>> schema migrations in a DB (migration scripts, etc.).
>>> Our real issue is with Ignite Runnables conflicting on the Binary
>>> Marshaller. Every class that gets executed on Ignite as a job gets a stored
>>> definition, this means we can't refactor classes that get used internally
>>> by our Ignite Runnables, because doing so would prevent the updated code
>>> from running. Even worse, if we update our libraries and they changed class
>>> definitions we might run into the same issue, without changing a letter in
>>> our own code. From the documentation it looked like Deployment Mode could
>>> provide a solution for this issue but the Binary Marshaller seems to run
>>> completely separate from the Deployment Mode.
>>>
>>> On Mon, Dec 17, 2018 at 3:18 PM Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com> wrote:
>>>
>>>> Hello!
>>>>
>>>> As far as my understanding goes:
>>>> You can't peer class load your Key/Value types.
>>>> You also can't redeploy your Key/Value types.
>>>> They even survive node restart via WORKDIR/marshaller directory, and
>>>> come back to haunt you.
>>>>
>>>> There are plans to maybe ease some of those limitations in 3.0, but
>>>> nothing concrete yet. It is not a bug rather a pillar of current Ignite
>>>> architecture. You will have to route around it, such as introducing new
>>>> fields instead of changing types. And maybe avoid having those types on
>>>> server nodes at all, and relying on BinaryObject.
>>>>
>>>> Regards,
>>>> --
>>>> Ilya Kasnacheev
>>>>
>>>>
>>>> пн, 17 дек. 2018 г. в 15:38, Gert Dubois <gert.dub...@trendminer.com>:
>>>>
>>>>> The issue is still present in 2.7.
>>>>> I added a ticket on Jira with sample code that reproduces the issue.
>>>>>
>>>>> https://issues.apache.org/jira/browse/IGNITE-10717
>>>>>
>>>>> For now I think we can work around the issue by overriding the default
>>>>> BinaryNameMapper, but this feels rather hacky to me.
>>>>>
>>>>> On Mon, Dec 17, 2018 at 11:54 AM Denis Mekhanikov <
>>>>> dmekhani...@gmail.com> wrote:
>>>>>
>>>>>> Gert,
>>>>>>
>>>>>> Could you check if the problem with a deployment mode reproduces on
>>>>>> Ignite 2.7?
>>>>>> If it does, please file a ticket with an explanation and a reproducer
>>>>>> to https://issues.apache.org/jira/
>>>>>>
>>>>>> Thanks!
>>>>>> Denis
>>>>>>
>>>>>>
>>>>>> пн, 17 дек. 2018 г. в 12:12, Gert Dubois <gert.dub...@trendminer.com
>>>>>> >:
>>>>>>
>>>>>>> I investigated the issue further and narrowed the issue down to the
>>>>>>> Binary Marshaller not working as expected given the configured
>>>>>>> Deployment
>>>>>>> Mode. When forcing my clients to use unique class names in the metadata
>>>>>>> of
>>>>>>> the Binary Marshaller (I forced this by overriding the global
>>>>>>> BinaryNameMapper and appending a per-client unique suffix to every class
>>>>>>> name) the Deployment Mode behaves as expected. I assume the Binary
>>>>>>> Marshaller keeps a cluster wide state of the Metadata of classes and it
>>>>>>> merges it whenever we serialize a class on a node (regardless of the
>>>>>>> configured Deployment Mode).
>>>>>>> Why is the behaviour of the Binary Marshaller not consistent with
>>>>>>> the way the Deployment Mode works? Is there a cleaner way to solve this,
>>>>>>> besides overriding the BinaryNameMapper?
>>>>>>>
>>>>>>> On Fri, Dec 14, 2018 at 1:07 PM Gert Dubois <
>>>>>>> gert.dub...@trendminer.com> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> On my current project we are confronted with an issue we are
>>>>>>>> struggling to figure out.
>>>>>>>> We have a simple topology where we have a client node running an
>>>>>>>> IgniteRunnable to a server node. Both nodes have peerClassLoading=true
>>>>>>>> and
>>>>>>>> the default binaryConfiguration with compactFooters=false. The client
>>>>>>>> is
>>>>>>>> started with clientMode=true
>>>>>>>> After execution we shut the client down and refactor the field type
>>>>>>>> of one of the fields in our IgniteRunnable. When we now restart the
>>>>>>>> client
>>>>>>>> we get the following exception:
>>>>>>>>
>>>>>>>> Caused by: org.apache.ignite.binary.BinaryObjectException: Binary
>>>>>>>> type has different field types
>>>>>>>> [typeName=com.trendminer.compute.ClientJob,
>>>>>>>> fieldName=param2, fieldTypeName1=double, fieldTypeName2=int]
>>>>>>>>
>>>>>>>> We tried different Deployment modes, ISOLATED, SHARED/CONTINUOUS
>>>>>>>> with different userVersion. But nothing we change enables us to
>>>>>>>> execute the
>>>>>>>> new class definition on the server node.
>>>>>>>>
>>>>>>>> Ignite Version is 2.4.0
>>>>>>>>
>>>>>>>> Since we are using persistence in our production environment and
>>>>>>>> this causes class definitions for the binary marshaller to be
>>>>>>>> remembered
>>>>>>>> even after cluster restart this is blocking for us since this either
>>>>>>>> forces
>>>>>>>> us to manually clear out the binary data and restart our cluster or it
>>>>>>>> prevents us from refactoring code we we send into Ignite, which is not
>>>>>>>> always possible.
>>>>>>>> So we would like to know what the intended way is to deal with
>>>>>>>> changing class definitions of tasks sent into Ignite?
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>>
>>>>>>>> Gert
>>>>>>>>
>>>>>>>
