[
https://issues.apache.org/jira/browse/HIVE-25629?focusedWorklogId=668810&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-668810
]
ASF GitHub Bot logged work on HIVE-25629:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 22/Oct/21 06:08
Start Date: 22/Oct/21 06:08
Worklog Time Spent: 10m
Work Description: kasakrisz commented on a change in pull request #2735:
URL: https://github.com/apache/hive/pull/2735#discussion_r734238060
##########
File path:
itests/util/src/main/java/org/apache/hadoop/hive/cli/control/AbstractCoreBlobstoreCliDriver.java
##########
@@ -115,11 +114,11 @@ protected void runTestHelper(String tname, String fname,
String fpath, boolean e
try {
System.err.println("Begin query: " + fname);
- qt.addFile(fpath);
- qt.cliInit(new File(fpath));
+ qt.setInputFile(fpath);
+ qt.cliInit();
Review comment:
Maybe it is out of the scope but have you considered merging the methods
`setInputFile` and `cliInit` into one method. Or move both of them into the
constructor?
##########
File path:
itests/util/src/main/java/org/apache/hadoop/hive/ql/QOutProcessor.java
##########
@@ -370,15 +372,16 @@ public void addPatternWithMaskComment(String patternStr,
String maskComment) {
patternsWithMaskComments.add(toPatternPair(patternStr, maskComment));
}
- public void initMasks(File qf, String query) {
+ public void initMasks(String query) {
+ queryMasks.clear();
if (matches(PATTERN_MASK_STATS, query)) {
- qMaskStatsQuerySet.add(qf.getName());
+ queryMasks.add(Mask.STATS);
Review comment:
nit: patterns could be declared as a field of the `Mask` enum, and the
`initMask` body could have a for loop iterating through checking the possible
patterns and adding it to `queryMasks` instead of using these `if` statements.
Like:
```
for (Mask mask : Mask.values()) {
if (matches(maks.getPattern(), query)) {
queryMasks.add(mask);
}
}
```
Same for the `Operation` enum.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: gitbox-unsubscr...@hive.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 668810)
Time Spent: 20m (was: 10m)
> Drop support of multiple qfiles in QTestUtil, output and result processors
> --------------------------------------------------------------------------
>
> Key: HIVE-25629
> URL: https://issues.apache.org/jira/browse/HIVE-25629
> Project: Hive
> Issue Type: Task
> Components: Testing Infrastructure
> Affects Versions: 4.0.0
> Reporter: Stamatis Zampetakis
> Assignee: Stamatis Zampetakis
> Priority: Major
> Labels: pull-request-available
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The current implementation of
> [QTestUtil|https://github.com/apache/hive/blob/afeb0f8413b1fd777611e890e53925119a5e39f1/itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestUtil.java],
>
> [QOutProcessor|https://github.com/apache/hive/blob/master/itests/util/src/main/java/org/apache/hadoop/hive/ql/QOutProcessor.java],
> and
> [QTestResultProcessor|https://github.com/apache/hive/blob/afeb0f8413b1fd777611e890e53925119a5e39f1/itests/util/src/main/java/org/apache/hadoop/hive/ql/QTestResultProcessor.java],
> has some methods and fields (maps) for managing multiple input files.
> However, *all* clients of this API, such as
> [CoreCliDriver|https://github.com/apache/hive/blob/afeb0f8413b1fd777611e890e53925119a5e39f1/itests/util/src/main/java/org/apache/hadoop/hive/cli/control/CoreCliDriver.java],
> use these classes by processing one file per run.
> +Example+
> {code:java}
> public void runTest(String testName, String fname, String fpath) {
> ...
> qt.addFile(fpath);
> qt.cliInit(new File(fpath));
> ...
> try {
> qt.executeClient(fname);
> } catch (CommandProcessorException e) {
> qt.failedQuery(e.getCause(), e.getResponseCode(), fname,
> QTestUtil.DEBUG_HINT);
> }
> ...
> }
> {code}
> Notice that {{qt.addFile}} will keep accumulating input files to memory
> (filename + content) while {{qt.executeClient}} (and other similar APIs)
> always operate on the last file added. Apart from wasting memory, the APIs
> for multiple files are harder to understand, and extend.
> The goal of this JIRA is to simplify the aforementioned APIs by removing
> unused/redundant parts associated to multiple files to improve code
> readability, and reduce memory consumption.
> +Historical note+
> Before HIVE-25625 the functionality of multiple input files was used by the
> {{TestCompareCliDriver}} but it was still useless for all the other clients.
> With the removal of {{TestCompareCliDriver}} in HIVE-25625 keeping multiple
> files is completely redundant.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
