Omega359 opened a new pull request, #13932:
URL: https://github.com/apache/datafusion/pull/13932
## Which issue does this PR close?
<!--
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases. You can
link an issue to this PR using the GitHub syntax. For example `Closes #123`
indicates that this PR will close issue #123.
-->
Closes #13812
## Rationale for this change
Add most of the sqlite test suite to Datafusion sqllogictests. Note: **THESE
TESTS DO NOT CURRENTLY PASS**! Any test results where Datafusion returns a
result that does not match sqlite nor match Postgresql was left as-is.
## What changes are included in this PR?
This PR includes a number of changes many of which are part of the test
files in the `datafusion-testing` repo (5,711,125 select statements of which
78,437 fail outright in Datafusion). The list below includes both the changes
in this direct PR as well as the process to generate the files in
`datafusion-testing/data/sqlite/`
- All the .test files in the sqlite test suite are includes except the
contents of the `evidence` and the `index/delete` folders
- All .test files have had mysql and mssql specific tests removed. All other
references to mysql and mssql have been removed.
- All .test files were renamed to .slt
- All files had `control resultmode valuewise` added to the beginning to
allow the sqllogictest runner to properly be able to compare the results from
Datafusion (and Postgresql) to the results in the .slt file
- All queries have been run through both Datafusion and Postgres and any
queries that failed with an error have had comments added explaining the
failure and a `skipif Datafusion` and/or `skipif postgres`. For example:
```
# Datafusion - DataFusion error: SQL error: RecursionLimitExceeded
skipif Datafusion
```
- Datatypes have been updated to reflect data types from
Datafusion/Postgresql as the sqlite datatypes are very limited. Comments
reflecting the change have been added. For example:
```
# Datafusion - Types were automatically converted from:
# Datafusion - [Expected] [T]
# Datafusion - [Actual ] [I]
```
- Results have been updated if the Datafusion results differ from the sqlite
results **AND** the Datafusion results are the same as what the results from
Postgresql are. There are are queries where the results differs especially
around floating point (Real results in slt terms). floating point results were
deemed equivalent between Datafusion and Postgresql if the result was the same
to 4 decimal places. Comments reflecting the change have been added. For
example:
```
# Datafusion - Data was automatically updated based on comparison db results
# Datafusion - Previous results:
# Datafusion - 54
# Datafusion - 9
```
- The sqllogictest and runners have been updated to include progress
information
- A datafusion-testing git submodule has been added. You may need to run
`git submodule update --init --remote --recursive` to get it added to an
existing checkout of datafusion.
- Added the ability to start a postgres docker container automatically if no
`PG_URI` is set.
- Readme updates
## Are these changes tested?
Indeed, yes. To run the tests locally checkout this branch, update the git
submodules then run `INCLUDE_SQLITE=true cargo test --profile release-nonlto
--test sqllogictests`. Be aware that the tests can take quite a long time to
run, especially if you do not run with release or release-nonlto profiles.
## Are there any user-facing changes?
No.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org
For additional commands, e-mail: github-h...@datafusion.apache.org
