Created the pull request for it: https://github.com/apache/iceberg/pull/1789 <https://github.com/apache/iceberg/pull/1789>
You can turn it on for manual builds by export CI=true Any reviewers would be welcome! Thanks, Peter > On Nov 18, 2020, at 10:11, Mass Dosage <massdos...@gmail.com> wrote: > > I can definitely see how having more detailed logs could be useful so I like > what you're suggesting. I guess another option could be to make this > configurable so you can pass in an argument to turn on the > "showStandardStreams", by default it's false but while you're debugging this > issue it would be turned on? > > On Wed, 18 Nov 2020 at 09:03, Peter Vary <pv...@cloudera.com.invalid> wrote: > Hi Team, > > Recently I have been working on trying to reproduce the following CI failure > without success: > > org.apache.iceberg.mr.hive.TestHiveIcebergStorageHandlerWithCustomCatalog > > testScanTable[fileFormat=PARQUET, engine=tez] FAILED > java.lang.IllegalArgumentException: Failed to execute Hive query 'SELECT > * FROM default.customers ORDER BY customer_id DESC': Error while processing > statement: FAILED > : Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.tez.TezTask > Caused by: > org.apache.hive.service.cli.HiveSQLException: Error while processing > statement: FAILED: Execution Error, return code 1 from > org.apache.hadoop.hive.ql.exec.tez.TezTask > > Since I was unsuccessful reproing the case, and the provided error message in > CI logs are not really helpful this means I can not fix this flaky test for > now. :( > > After Marton Bods changes for adding logs for tests > (https://github.com/apache/iceberg/pull/1712 > <https://github.com/apache/iceberg/pull/1712>), we could have more info about > the failures in the test logs (build/test-results/test/binary/output.bin), > but I am not sure if that is retained and accessible after a CI run. > > I would like to propose adding the following to the build.gradle for the CI > runs: > > test { > testLogging { > if ("true".equalsIgnoreCase(System.getenv('CI'))) { > events "failed", "passed" > + testLogging.showStandardStreams = true > } else { > events "failed" > } > exceptionFormat "full" > } > } > > This would add the logs printed during the tests to the standard output for > the CI runs. Example can be seen here > (https://github.com/pvary/iceberg/runs/1405960983 > <https://github.com/pvary/iceberg/runs/1405960983>) - only enabled standard > streams for the hive related tests in this patch to see the results. > > Pros: > Easily accessible log information for the failed runs > Cons: > Harder to read CI logs > Possible cost associated with retaining the logs > > I think having more logs would be great, but I am not sure who pays the bill > and whether having bigger logs could cause any problem and whether the CI is > able to handle the increased amount of data. > > Any thoughts, comments, ideas? > > Thanks, > Peter
