Thanks.

On Thu, Jun 21, 2018 at 4:34 AM, Tzu-Li (Gordon) Tai <tzuli...@apache.org>
wrote:

> Hi Vishal,
>
> Kryo has a serializer called `CompatibleFieldSerializer` that allows for
> simple backward compatibility changes, such as adding non-optional fields /
> removing fields.
>
> If using the KryoSerializer is a must, then a good thing to do is to
> register Kryo's `CompatibleFieldSerializer` as the serializer to be used
> for that specific type.
> By default, Kryo doesn’t use the `CompatibleFieldSerializer`, so you would
> have to explicitly register this.
>
> The issue is, I think it wouldn’t be possible to use the
> `CompatibleFieldSerializer` _after_ the bytes were already written with the
> default, non-compatible version (the `FieldSerializer`).
> So, AFAIK, this should only work if your Kryo state type was registered
> with the `CompatibleFieldSerializer` from the very beginning.
> There could be a workaround, but unfortunately that would require changing
> some code in the `KryoSerializer`.
> If you require this because your job is already running in production and
> data was already written by the `FieldSerializer`, please let me know and
> I’ll go into more details about this.
>
> Cheers,
> Gordon
>
> On 21 June 2018 at 10:14:15 AM, Fabian Hueske (fhue...@gmail.com) wrote:
>
> Hi Vishal,
>
> In general, Kryo serializers are not very upgrade friendly.
> Serializer compatibility [1] might be right approach here, but Gordon (in
> CC) might know more about this.
>
> Best, Fabian
>
> [1] https://ci.apache.org/projects/flink/flink-docs-
> release-1.5/dev/stream/state/custom_serialization.html#
> handling-serializer-upgrades-and-compatibility
>
> 2018-06-18 12:06 GMT+02:00 Vishal Santoshi <vishal.santo...@gmail.com>:
>
>> Any more insight?
>>
>> On Wed, Jun 13, 2018, 3:34 PM Vishal Santoshi <vishal.santo...@gmail.com>
>> wrote:
>>
>>> Any ideas on the standard way ( or any roundabout way ) of doing a
>>> version upgrade that looks back ward compatible.
>>> The  @FieldSerializer.Optional("0") actually does  ignore the field (
>>> even if reset ) giving it the default value if kyro is used. It has to do
>>> with the FieldSerializer behaves  .  There is another Serializer (
>>> Composite I believe ) that allows for such back ward compatible changes.
>>>
>>>
>>> I know some work is being done in 1.6 to allow for above use case and I
>>> think Google Data Flow does provide some avenues.
>>>
>>> Thanks much
>>>
>>> Vishal
>>>
>>>
>>>
>>> On Tue, Jun 12, 2018 at 11:30 PM, Vishal Santoshi <
>>> vishal.santo...@gmail.com> wrote:
>>>
>>>> I have a running pipe with Window State in a class say
>>>>
>>>> Class A{
>>>>      long a;
>>>> }
>>>>
>>>> It uses the default KryoSerializer
>>>>
>>>> I want to add a field to
>>>>
>>>> Class A {
>>>>   long a;
>>>>   long b;
>>>> }
>>>>
>>>> I need to suspend with SP and resume with the new version of Class A
>>>>
>>>>
>>>> Is there a definite way to do this. I tried
>>>>
>>>> Class A {
>>>>   long a;
>>>>    @FieldSerializer.Optional("0")
>>>>   long b;
>>>> }
>>>>
>>>> but that seems to default to 0 , even when the Aggregation is putting
>>>> in values.
>>>>
>>>> Could somebody give pointers as to how to solve this
>>>>
>>>> Thanks a ton.
>>>>
>>>>
>>>>
>>>>
>>>
>

Reply via email to