[jira] [Created] (FLINK-1989) Sorting of POJO data set from TableEnv yields NotSerializableException

[Fabian Hueske (JIRA)]

Fabian Hueske created FLINK-1989:
------------------------------------

             Summary: Sorting of POJO data set from TableEnv yields 
NotSerializableException
                 Key: FLINK-1989
                 URL: https://issues.apache.org/jira/browse/FLINK-1989
             Project: Flink
          Issue Type: Bug
          Components: Expression API
    Affects Versions: 0.9
            Reporter: Fabian Hueske
             Fix For: 0.9


Sorting or grouping (or probably any other key operation) on a POJO data set 
that was created by a {{TableEnvironment}} yields a 
{{NotSerializableException}} due to a non-serializable 
{{java.lang.reflect.Field}} object. 

I traced the error back to the {{ExpressionSelectFunction}}. I guess that a 
{{TypeInformation}} object is stored in the generated user-code function. A 
{{PojoTypeInfo}} holds Field objects, which cannot be serialized.

The following test can be pasted into the {{SelectITCase}} and reproduces the 
problem. 

{code}
@Test
public void testGroupByAfterTable() throws Exception {

        ExecutionEnvironment env = 
ExecutionEnvironment.getExecutionEnvironment();
        TableEnvironment tableEnv = new TableEnvironment();

        DataSet<Tuple3<Integer, Long, String>> ds = 
CollectionDataSets.get3TupleDataSet(env);

        Table in = tableEnv.toTable(ds, "a,b,c");

        Table result = in
                        .select("a, b, c");

        DataSet<ABC> resultSet = tableEnv.toSet(result, ABC.class);

        resultSet
                        .sortPartition("a", Order.DESCENDING)
                        .writeAsText(resultPath, 
FileSystem.WriteMode.OVERWRITE);

        env.execute();

        expected = "1,1,Hi\n" + "2,2,Hello\n" + "3,2,Hello world\n" + 
"4,3,Hello world, " +
                        "how are you?\n" + "5,3,I am fine.\n" + "6,3,Luke 
Skywalker\n" + "7,4," +
                        "Comment#1\n" + "8,4,Comment#2\n" + "9,4,Comment#3\n" + 
"10,4,Comment#4\n" + "11,5," +
                        "Comment#5\n" + "12,5,Comment#6\n" + "13,5,Comment#7\n" 
+ "14,5,Comment#8\n" + "15,5," +
                        "Comment#9\n" + "16,6,Comment#10\n" + 
"17,6,Comment#11\n" + "18,6,Comment#12\n" + "19," +
                        "6,Comment#13\n" + "20,6,Comment#14\n" + 
"21,6,Comment#15\n";

}

public static class ABC {
        public int a;
        public long b;
        public String c;
}
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)