"So Nasser, if it is not too much trouble for you, can you run the FriCAS tests with and without "setSimplifyDenomsFlag true"? I'm interested in the numbers. "
I've done this before in 2018. Please see https://www.mail-archive.com/[email protected]/msg12255.html I remember that overall it did not improve the result by turning this flag off, but make it little slower. I can't find the exact message, it is hidden in the above thread I think. "1. You give the regression report. Can we also have a report that shows integrals that solved in new version but not solved in old version?" Sure. "2. Can we have a page that lists all failed integrals of a system? (So that we don't have to click through lots of links.)" Sure. "3. Can we have a report on "elementary integral", "algebraic integral", "transcendental integral", "special function integral"?" There is section in the introduction called "Performance per integrand type" which shows the final result for each CAS for each subset of the tests: 1. Independent tests. 2. Algebraic Binomial problems (products involving powers of binomials and monomials). 3. Algebraic Trinomial problems (products involving powers of trinomials, binomials and monomials). 4. Miscellaneous Algebraic functions. 5. Exponentials. 6. Logarithms. 7. Trigonometric. 8. Inverse Trigonometric. 9. Hyperbolic functions. 10. Inverse Hyperbolic functions. 11. Special functions. 12. Sam Blake input file. 13. Waldek Hebisch input file. <https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/indexchapter1.htm#x2-50001.3> There is also complete separate report on "Elementary algebraic integrals" here done about 2 years ago: <https://12000.org/my_notes/CAS_integration_tests/reports/sept_2021_algebraic_version/index.htm> May be these will help. Regards --Nasser On Monday, August 22, 2022 at 6:59:03 AM UTC-5 oldk1331 wrote: > Hi Nasser, > > Some follow up comments on this. > > About "setSimplifyDenomsFlag true", it can make some integrals faster, > but it can also make many integrals slower (to timeout). I'm not sure > if this flag makes FriCAS be able to solve more or less. > (@Waldek, can we hear your opinion?) > > So Nasser, if it is not too much trouble for you, can you run the > FriCAS tests with and without "setSimplifyDenomsFlag true"? > I'm interested in the numbers. > > (Or maybe FriCAS should provide a function that, given a timeout, > use half time to run with "setSimplifyDenomsFlag true" and half > time to run without it.) > > ==== > > Some other remarks on your report: > > 1. You give the regression report. Can we also have a report that > shows integrals that solved in new version but not solved in old > version? > > 2. Can we have a page that lists all failed integrals of a system? > (So that we don't have to click through lots of links.) > > 3. Can we have a report on "elementary integral", "algebraic integral", > "transcendental integral", "special function integral"? > > What I meant is to filter through the integral, check if it is > "elementary integral" -- only contains +-*/^ exp log trigs > "algebraic integral" -- only contains +-*/^ > "transcendental integral" -- only contains +-*/ exp log trigs > "special function integral" -- contains special functions. > > - Qian > > On 8/22/22 19:13, 'Nasser M. Abbasi' via FriCAS - computer algebra > system wrote: > > Thanks Qian for finding this. > > > > I just found the problem and just fixed it. > > > > It is related to running these tests in spawned process using Python's > > multiprocessing. I have to run all these integration tests in a spawned > > process to be able to set a timeout of 3 minutes. > > > > Will rerun all the tests for Fricas/maxima and giac again and rebuild the > > reports just to make sure there are no other integrals that got affected > by > > this false timeout. Will take me may be a week. Will post reply here when > > done. > > > > As for using setSimplifyDenomsFlag, yes. I set to true before each call > > from Sagemath using > > > > fricas.setSimplifyDenomsFlag(fricas.true) > > > > So all Fricas integrals are run with this flag set to true. > > > > --Nasser > > On Monday, August 22, 2022 at 5:04:06 AM UTC-5 oldk1331 wrote: > > > >> Hi Nasser, > >> > >> Here is another 2 (minor) problems in your regression report: > >> > >> > >> > https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/REGRESSION_REPORTS/fricas/reportsection7.htm#x8-190007 > >> > >> integrate((e*x)^m*(a+b*log(c*log(d*x)^p)),x) > >> > >> This returns quickly in fricas-1.3.8, it does not timeout. > >> Can you double check on your side please. > >> > >> > >> > https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/REGRESSION_REPORTS/fricas/reportsection9.htm#x10-230009 > >> > >> integrate(cot(f*x+e)^2*(c+d*tan(f*x+e))^(1/3),x) > >> > >> For this integral, it returns quickly if you don't set > >> "setSimplifyDenomsFlag true". > >> > >> (P.S. fricas-1.3.7 gave wrong answer.) > >> > >> BTW, do you run all FriCAS tests with "setSimplifyDenomsFlag true"? > >> > >> - Qian > >> > >> On 8/21/22 14:15, 'Nasser M. Abbasi' via FriCAS - computer algebra > >> system wrote: > >>> FYI, > >>> > >>> (Posted at https://groups.google.com/g/fricas-devel and > >>> https://groups.google.com/g/sci.math.symbolic) > >>> > >>> Independent CAS integration tests summer 2022 edition is now complete. > >>> (It took about 5 months since I initially started working on it!) > >>> > >>> < > >> > https://www.12000.org/my_notes/CAS_integration_tests/reports/summer_2022/index.htm > >>> > >>> > >>> The following is summary of changes in this version > >>> compared to last year's edition > >>> ======================== > >>> 1. Number of integrals increased to 85,479 from 71,994. > >>> > >>> The integrals are made of the following 210 files: > >>> > >>> a. Rubi test suite. Files 1 to 208. Thanks to Alert Rich. > >>> Downloaded from <https://github.com/RuleBasedIntegration> > >>> b. IntegrateAlgebraic test file. File 209. Thanks to Sam Blake. > >>> c. Fricas test file. File 210. Thanks to Waldek Hebisch. > >>> > >>> 2. CAS version changes: > >>> Maple 2021.1 -> 2022.1 > >>> Mathematica 12.3 -> 13.01 > >>> Giac 1.7.0 -> 1.9.0-13 > >>> Sympy 1.8/python 3.8.8 -> 1.10.1/Python 3.10.4 > >>> Maxima 5.44 -> 5.46 > >>> FriCAS 1.3.7 -> 1.3.8 > >>> Sagemath 9.3 -> 9.6 > >>> > >>> No version change for Mupad at Matlab 2021a and Rubi at 4.16.1 > >>> but Rubi was run again on the current Mathematica 13.1 version. > >>> > >>> 3. General improvements to Latex and other formatting. > >>> > >>> 4. Flow chart of the test program at > >>> < > https://www.12000.org/my_notes/CAS_integration_tests/images/design.svg> > >>> > >>> 5. Links to download all the 85,483 integrals in plain text in > >>> Mathematica, Rubi, Maple, Mupad, Sympy, and Sagemath (Fricas, Maxima, > >> Giac) > >>> formats are at introduction page of the main page. > >>> > >>> 6. Regression reports comparing results in 2022 edition to 2021 edition > >>> are also now available. These reports show integrals that failed > >>> in current versions compared to last versions per CAS. > >>> > >>> < > >> > https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/REGRESSION_REPORTS/index.htm > >>> > >>> > >>> Regression per CAS > >>> ------------------ > >>> This table below gives the number of integrals that failed in > >>> this test (could be due to timeout, exception or not solved) but > >>> are solved in summer 2021 for each CAS. > >>> > >>> Number of regression per CAS is (less is better) > >>> > >>> Fricas: 16 > >>> Maxima: 17 > >>> Mathematica: 98 > >>> Maple: 114 > >>> Giac: 363 > >>> Sympy: 379 > >>> > >>> Tables give details about each integral that failed in regression. > >>> > >>> 7. Sympy, Sagemath (Fricas, Maxima, Giac) were all run on Linux > >>> inside Oracle Virtual Box. Maple, Rubi, Mupad and Mathematica were > >>> run on windows 10. All on same PC with 128 GB RAM with > >>> Intel Core i9-12900K 3.20 GHz. > >>> > >>> RESULTS > >>> ======= > >>> > >>> Overall solved percentage in summer 2022 edition > >>> ========================================= > >>> 1. Mathematica 13.1 97.99 % > >>> 2. Rubi 4.16.1 94.21 % > >>> 3. Maple 2022.1 84.53 % > >>> 4. Fricas 1.3.8 79.33 % > >>> 5. Giac 1.9.0-13 58.37 % > >>> 6. Maxima 5.46 56.86 % > >>> 7. Mupad 2021a 56.1 % > >>> 8. Sympy 1.10.1 41.89 % > >>> > >>> Grading results > >>> ================= > >>> Note that mupad is not graded. Default grade of B is given for > >>> any passed integral. > >>> > >>> 1. Rubi 91.50 % > >>> 2. Mathematica 78.58 % > >>> 3. Maple 56.34 % > >>> 4. Fricas 53.78 % > >>> 5. Maxima 43.57 % > >>> 6. Giac 42.36 % > >>> 7. Sympy 28.49 % > >>> 8. Mupad* N/A > >>> > >>> Time and leaf size Performance > >>> ============================== > >>> Sorted by time performance in average time used per integrals > (seconds). > >>> > >>> Note for Maple(*) timing: the option method=_RETURNVERBOSE was used > >>> which made Maple run through all integration methods increasing > >>> average time used. For example in summer 2021 edition which did > >>> not use this option, Maple had 0.79 seconds average time on > >>> a slower PC also than the one used for the current tests. > >>> > >>> 1. Rubi 0.3 (sec) > >>> 2. Maxima 0.49 (sec) > >>> 3. Giac 0.75 (sec) > >>> 4. Fricas 1.17 (sec) > >>> 5. Mathematica 2.38 (sec) > >>> 6. Mupad 2.75 (sec) > >>> 7. Maple* 3.33 (sec) > >>> 8. Sympy 5.24 (sec) > >>> > >>> General notes > >>> ============== > >>> 1) Maxima, Fricas and Giac were called via Sagemath. Some integrals > >>> could have failed due to interface issues. These will have F(-2) > >>> error code in the section "Detailed conclusion table per each > >>> integral for all CAS systems" per each report and also in the actual > >>> integral page with the actual exception name shown. > >>> > >>> This allows one to see if the cause of the failure is due to > >>> interface issue or internal error from the CAS itself. > >>> > >>> For an example, the integral on > >>> > >>> < > >> > https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/test_cases/4_Trig_functions/4.7_Miscellaneous/139_4.7.5_x%5Em_trig-a+b_log-c_x%5En-%5Ep/reportsubsection69.htm#53 > >>> > >>> > >>> Has F(-2) on Fricas output, but looking at the exception > >>> > >>> integrate(x*sin(a+b*log(c*x^n))^(1/2),x, algorithm="fricas") > >>> > >>> "Exception raised: TypeError >> Error detected within library code: > >>> integrate: implementation incomplete (has polynomial part)" > >>> > >>> So this is not interface issue with Sagemath. The exception comes > >>> from Fricas itself. > >>> > >>> But this integral > >>> > >>> < > >> > https://12000.org/my_notes/CAS_integration_tests/reports/summer_2022/test_cases/4_Trig_functions/4.7_Miscellaneous/139_4.7.5_x%5Em_trig-a+b_log-c_x%5En-%5Ep/reportsubsection57.htm#41 > >>> > >>> > >>> Has F(-2) on Giac result, and looking at the exception it says > >>> > >>> integrate(x^2*sin(a+log(c*x^n)*(-1/n^2)^(1/2))^3,x, algorithm="giac") > >>> > >>> "Exception raised: NotImplementedError >> > >>> Unable to parse Giac output: > >>> ((-9*i)*sageVARn^4*sageVARx^3*exp((-3*i) *sageVARa)* > >>> exp((3*sageVARn*abs(sageVARn)*ln(sageVARx)+3*abs(sageVARn)* > >>> ln(sageVARc))/sageVARn^2)+27*i*sageVARn^4*sageVARx^3*exp((-i)" > >>> > >>> Which indicates it is an interface issue this time. This is > >>> verified by running this integral directly in giac and it is > >>> solved with no error. > >>> > >>> CAS integration test program currently does not distinguish > >>> between an exception generated due to a sagemath interface problem > >>> or an exception generated from the CAS itself. I am not > >>> sure if there a robust way to do this for Maxima, Fricas > >>> and Giac. If a way can be found, will add it. > >>> > >>> But there seems to be less of these interface problems in the > >>> current Sagemath version than the last one used. > >>> > >>> 2) The section "List of integrals sorted by grade for each CAS" in each > >>> report shows list of integrals for each grade per CAS. This allows > >>> one to quickly find integrals with specific grade. > >>> > >>> 3) Also as mentioned before there are a number of integrals for Maxima > >>> which fail when it asks a question in order to complete the > >>> integration as this is not supported by Sagemath or by the > >>> CAS integration test program. If an option can be added to > >>> turn this feature off that will be a good thing and will improve > >>> Maxima score. > >>> > >>> 4) Any bugs found or problems please let me know and will try my best > >>> to fix them. > >>> > >>> --Nasser > >>> > >> > > > -- You received this message because you are subscribed to the Google Groups "FriCAS - computer algebra system" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/fricas-devel/66dec79f-7d4f-4750-a796-cc2df3276d64n%40googlegroups.com.
