Hi Nasser,
Some follow up comments on this.
About "setSimplifyDenomsFlag true", it can make some integrals faster,
but it can also make many integrals slower (to timeout). I'm not sure
if this flag makes FriCAS be able to solve more or less.
(@Waldek, can we hear your opinion?)
So Nasser, if it is not too much trouble for you, can you run the
FriCAS tests with and without "setSimplifyDenomsFlag true"?
I'm interested in the numbers.
(Or maybe FriCAS should provide a function that, given a timeout,
use half time to run with "setSimplifyDenomsFlag true" and half
time to run without it.)
====
Some other remarks on your report:
1. You give the regression report. Can we also have a report that
shows integrals that solved in new version but not solved in old
version?
2. Can we have a page that lists all failed integrals of a system?
(So that we don't have to click through lots of links.)
3. Can we have a report on "elementary integral", "algebraic integral",
"transcendental integral", "special function integral"?
What I meant is to filter through the integral, check if it is
"elementary integral" -- only contains +-*/^ exp log trigs
"algebraic integral" -- only contains +-*/^
"transcendental integral" -- only contains +-*/ exp log trigs
"special function integral" -- contains special functions.
- Qian
On 8/22/22 19:13, 'Nasser M. Abbasi' via FriCAS - computer algebra
system wrote:
Thanks Qian for finding this.
I just found the problem and just fixed it.
It is related to running these tests in spawned process using Python's
multiprocessing. I have to run all these integration tests in a spawned
process to be able to set a timeout of 3 minutes.
Will rerun all the tests for Fricas/maxima and giac again and rebuild the
reports just to make sure there are no other integrals that got affected by
this false timeout. Will take me may be a week. Will post reply here when
done.
As for using setSimplifyDenomsFlag, yes. I set to true before each call
from Sagemath using
fricas.setSimplifyDenomsFlag(fricas.true)
So all Fricas integrals are run with this flag set to true.
--Nasser
On Monday, August 22, 2022 at 5:04:06 AM UTC-5 oldk1331 wrote:
Hi Nasser,
Here is another 2 (minor) problems in your regression report:
https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/REGRESSION_REPORTS/fricas/reportsection7.htm#x8-190007
integrate((e*x)^m*(a+b*log(c*log(d*x)^p)),x)
This returns quickly in fricas-1.3.8, it does not timeout.
Can you double check on your side please.
https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/REGRESSION_REPORTS/fricas/reportsection9.htm#x10-230009
integrate(cot(f*x+e)^2*(c+d*tan(f*x+e))^(1/3),x)
For this integral, it returns quickly if you don't set
"setSimplifyDenomsFlag true".
(P.S. fricas-1.3.7 gave wrong answer.)
BTW, do you run all FriCAS tests with "setSimplifyDenomsFlag true"?
- Qian
On 8/21/22 14:15, 'Nasser M. Abbasi' via FriCAS - computer algebra
system wrote:
FYI,
(Posted at https://groups.google.com/g/fricas-devel and
https://groups.google.com/g/sci.math.symbolic)
Independent CAS integration tests summer 2022 edition is now complete.
(It took about 5 months since I initially started working on it!)
<
https://www.12000.org/my_notes/CAS_integration_tests/reports/summer_2022/index.htm
The following is summary of changes in this version
compared to last year's edition
========================
1. Number of integrals increased to 85,479 from 71,994.
The integrals are made of the following 210 files:
a. Rubi test suite. Files 1 to 208. Thanks to Alert Rich.
Downloaded from <https://github.com/RuleBasedIntegration>
b. IntegrateAlgebraic test file. File 209. Thanks to Sam Blake.
c. Fricas test file. File 210. Thanks to Waldek Hebisch.
2. CAS version changes:
Maple 2021.1 -> 2022.1
Mathematica 12.3 -> 13.01
Giac 1.7.0 -> 1.9.0-13
Sympy 1.8/python 3.8.8 -> 1.10.1/Python 3.10.4
Maxima 5.44 -> 5.46
FriCAS 1.3.7 -> 1.3.8
Sagemath 9.3 -> 9.6
No version change for Mupad at Matlab 2021a and Rubi at 4.16.1
but Rubi was run again on the current Mathematica 13.1 version.
3. General improvements to Latex and other formatting.
4. Flow chart of the test program at
<https://www.12000.org/my_notes/CAS_integration_tests/images/design.svg>
5. Links to download all the 85,483 integrals in plain text in
Mathematica, Rubi, Maple, Mupad, Sympy, and Sagemath (Fricas, Maxima,
Giac)
formats are at introduction page of the main page.
6. Regression reports comparing results in 2022 edition to 2021 edition
are also now available. These reports show integrals that failed
in current versions compared to last versions per CAS.
<
https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/REGRESSION_REPORTS/index.htm
Regression per CAS
------------------
This table below gives the number of integrals that failed in
this test (could be due to timeout, exception or not solved) but
are solved in summer 2021 for each CAS.
Number of regression per CAS is (less is better)
Fricas: 16
Maxima: 17
Mathematica: 98
Maple: 114
Giac: 363
Sympy: 379
Tables give details about each integral that failed in regression.
7. Sympy, Sagemath (Fricas, Maxima, Giac) were all run on Linux
inside Oracle Virtual Box. Maple, Rubi, Mupad and Mathematica were
run on windows 10. All on same PC with 128 GB RAM with
Intel Core i9-12900K 3.20 GHz.
RESULTS
=======
Overall solved percentage in summer 2022 edition
=========================================
1. Mathematica 13.1 97.99 %
2. Rubi 4.16.1 94.21 %
3. Maple 2022.1 84.53 %
4. Fricas 1.3.8 79.33 %
5. Giac 1.9.0-13 58.37 %
6. Maxima 5.46 56.86 %
7. Mupad 2021a 56.1 %
8. Sympy 1.10.1 41.89 %
Grading results
=================
Note that mupad is not graded. Default grade of B is given for
any passed integral.
1. Rubi 91.50 %
2. Mathematica 78.58 %
3. Maple 56.34 %
4. Fricas 53.78 %
5. Maxima 43.57 %
6. Giac 42.36 %
7. Sympy 28.49 %
8. Mupad* N/A
Time and leaf size Performance
==============================
Sorted by time performance in average time used per integrals (seconds).
Note for Maple(*) timing: the option method=_RETURNVERBOSE was used
which made Maple run through all integration methods increasing
average time used. For example in summer 2021 edition which did
not use this option, Maple had 0.79 seconds average time on
a slower PC also than the one used for the current tests.
1. Rubi 0.3 (sec)
2. Maxima 0.49 (sec)
3. Giac 0.75 (sec)
4. Fricas 1.17 (sec)
5. Mathematica 2.38 (sec)
6. Mupad 2.75 (sec)
7. Maple* 3.33 (sec)
8. Sympy 5.24 (sec)
General notes
==============
1) Maxima, Fricas and Giac were called via Sagemath. Some integrals
could have failed due to interface issues. These will have F(-2)
error code in the section "Detailed conclusion table per each
integral for all CAS systems" per each report and also in the actual
integral page with the actual exception name shown.
This allows one to see if the cause of the failure is due to
interface issue or internal error from the CAS itself.
For an example, the integral on
<
https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/test_cases/4_Trig_functions/4.7_Miscellaneous/139_4.7.5_x%5Em_trig-a+b_log-c_x%5En-%5Ep/reportsubsection69.htm#53
Has F(-2) on Fricas output, but looking at the exception
integrate(x*sin(a+b*log(c*x^n))^(1/2),x, algorithm="fricas")
"Exception raised: TypeError >> Error detected within library code:
integrate: implementation incomplete (has polynomial part)"
So this is not interface issue with Sagemath. The exception comes
from Fricas itself.
But this integral
<
https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/test_cases/4_Trig_functions/4.7_Miscellaneous/139_4.7.5_x%5Em_trig-a+b_log-c_x%5En-%5Ep/reportsubsection57.htm#41
Has F(-2) on Giac result, and looking at the exception it says
integrate(x^2*sin(a+log(c*x^n)*(-1/n^2)^(1/2))^3,x, algorithm="giac")
"Exception raised: NotImplementedError >>
Unable to parse Giac output:
((-9*i)*sageVARn^4*sageVARx^3*exp((-3*i) *sageVARa)*
exp((3*sageVARn*abs(sageVARn)*ln(sageVARx)+3*abs(sageVARn)*
ln(sageVARc))/sageVARn^2)+27*i*sageVARn^4*sageVARx^3*exp((-i)"
Which indicates it is an interface issue this time. This is
verified by running this integral directly in giac and it is
solved with no error.
CAS integration test program currently does not distinguish
between an exception generated due to a sagemath interface problem
or an exception generated from the CAS itself. I am not
sure if there a robust way to do this for Maxima, Fricas
and Giac. If a way can be found, will add it.
But there seems to be less of these interface problems in the
current Sagemath version than the last one used.
2) The section "List of integrals sorted by grade for each CAS" in each
report shows list of integrals for each grade per CAS. This allows
one to quickly find integrals with specific grade.
3) Also as mentioned before there are a number of integrals for Maxima
which fail when it asks a question in order to complete the
integration as this is not supported by Sagemath or by the
CAS integration test program. If an option can be added to
turn this feature off that will be a good thing and will improve
Maxima score.
4) Any bugs found or problems please let me know and will try my best
to fix them.
--Nasser
--
You received this message because you are subscribed to the Google Groups "FriCAS -
computer algebra system" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/fricas-devel/d774cbf6-ba8b-cd09-61b7-dbb63fe826f0%40gmail.com.
