[
https://issues.apache.org/jira/browse/FLINK-15498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17009561#comment-17009561
]
Jingsong Lee commented on FLINK-15498:
--------------------------------------
Hi [~dwysakowicz], You are right, the point is like an example, but I have some
concerns that there are examples but no E2E guarantee. For the current TPC-DS
E2E, I don't think it's real E2E. The real batch E2E is more integrated with
hive metastore instead of playing with local csv.
By the way, what is flink-packages? You mean flink-examples?
> Using HiveCatalog in TPC-DS e2e
> -------------------------------
>
> Key: FLINK-15498
> URL: https://issues.apache.org/jira/browse/FLINK-15498
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner, Tests
> Reporter: Jingsong Lee
> Priority: Major
> Fix For: 1.11.0
>
>
> In 1.10, we have made great progress in the performance and function of
> batch. After our internal test, the performance is significantly ahead of
> hive.
> But it's hard for users to reproduce. They need to have some research on
> TPC-DS to write test code.
> We can consider changing the E2E test of TPC-DS to HiveCatalog, which is
> roughly divided into two stages:
> # The first stage is prepare of hive. Prepare the tables of TPC-DS. Insert
> the data and prepare the metastore. And analysis the tables.
> # The second stage is the analysis of Flink. Only select and check results.
> Users can play with it only by changing the data scale of the first stage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
