Actually I think that it’s not correct that the OptionType cannot be used
as a key type. In fact it is similar to a composite type and should be
usable as a key iff it’s element can be used as a key. Then we only have to
provide an OptionTypeComparator which will compare the elements if they are
set. If not, then the None element will be smaller, for example.
@Timur, if you want, then you can file a JIRA issue to add that.
Cheers,
Till
On Wed, Mar 30, 2016 at 7:17 PM, Timur Fayruzov <timur.fairu...@gmail.com>
wrote:
> Ok, I can't make Option comparable, so the only way that I see is to
> translate a key to a Comparable data structure and use it (as it was
> alluded to in your example above). Thank you for clarification!
>
> Thanks,
> Timur
>
> On Wed, Mar 30, 2016 at 9:22 AM, Chiwan Park <chiwanp...@apache.org>
> wrote:
>
>> Hi Timur,
>>
>> Sorry for confusing. I meant KeySelector.
>>
>> `GenericType<T>` could be used as a key type if the `T` implements
>> `Comparable`. For example, `GenericType<Integer>` could be used as a key
>> type but `GenericType<scala.Tuple2>` could not.
>>
>> About my example in previous mail, the type of key is `Int` because the
>> return type of KeySelector is `Int`. `TypeInformation<Int>` is not generic
>> type.
>>
>> Regards,
>> Chiwan Park
>>
>> > On Mar 31, 2016, at 1:09 AM, Timur Fayruzov <timur.fairu...@gmail.com>
>> wrote:
>> >
>> > Thank you for your answers, Chiwan! That would mean that a generic type
>> can't be used as a key in general? This is a non-obvious limitation of
>> Flink DSL that I didn't see in documentation.
>> >
>> > Could you please elaborate what you mean by KeyExtractor? I see that
>> inside `where` operator an instance of KeySelector is initialized, but I
>> don't see how can I pass a custom KeySelector in.
>> >
>> > Thanks,
>> > Timur
>> >
>> > On Wed, Mar 30, 2016 at 12:53 AM, Chiwan Park <chiwanp...@apache.org>
>> wrote:
>> > Hi Timur,
>> >
>> > Because Option[T] is not comparable type generally (if T is a POJO
>> type), you cannot use Option[T] as a key type. I think you have to
>> implement KeyExtractor to compare objects including Option[T]s.
>> >
>> > ```
>> > case class MyKey(k1: Option[String], k2: Option[String])
>> >
>> > val data1 = env.fromElements(MyKey(Some("a"), None), MyKey(Some("a"),
>> Some("c")))
>> > val data2 = env.fromElements(MyKey(Some("b"), None), MyKey(Some("a"),
>> Some("c")))
>> >
>> > data1.join(data2)
>> > .where(_.hashCode())
>> > .equalTo(_.hashCode()).apply {
>> > (left: MyKey, right: MyKey) => (left, right)
>> > }.print()
>> > ```
>> >
>> > Note that the approach in example (using hashCode()) cannot be applied
>> to sort task.
>> >
>> > Regards,
>> > Chiwan Park
>> >
>> > > On Mar 30, 2016, at 2:37 AM, Timur Fayruzov <timur.fairu...@gmail.com>
>> wrote:
>> > >
>> > > There is some more detail to this question that I missed initially.
>> It turns out that my key is a case class of a form MyKey(k1:
>> Option[String], k2: Option[String]). Keys.SelectorFunctionKeys is
>> performing a recursive check whether every element of the MyKey class can
>> be a key and fails when encountering an Option.
>> > >
>> > > Is it possible to work around this situation without giving up
>> Options? Inability to use Options in Domain objects could be really
>> frustrating.
>> > >
>> > > Thanks,
>> > > Timur
>> > >
>> > > On Tue, Mar 29, 2016 at 10:19 AM, Timur Fayruzov <
>> timur.fairu...@gmail.com> wrote:
>> > > Hello,
>> > >
>> > > I'm evaluating Flink and one thing I noticed is Option[A] can't be
>> used as a key for coGroup (looking specifically here:
>> https://github.com/apache/flink/blob/master/flink-scala/src/main/scala/org/apache/flink/api/scala/typeutils/OptionTypeInfo.scala#L39).
>> I'm not clear about the reason of this and appreciate if someone can
>> explain.
>> > >
>> > > Thanks,
>> > > Timur
>> > >
>> >
>> >
>>
>>
>
