On Mon, May 8, 2023 at 2:32 AM Sebastian Blauth < [email protected]> wrote:
> Hello everyone, > > I wanted to briefly follow up on my question (see my last reply). > Does anyone know / have an idea why the LSC preconditioner in PETSc does > not seem to scale well with the problem size (the outer fgmres solver I > am using nearly scale nearly linearly with the problem size in my example). > I have also already tried using -ksp_diagonal_scale but the results are > identical. > Any help is really appreciated. > I would start by finding results in the literature that you like, in that they are on a problem similar to yours and you like the performance, and reproduce them in your code. If you can do that then you have a research problem to see how to get your problem to work well. If your solver does not reproduce published results then we might be able to provide some advice. Mark > > Thanks a lot, > Sebastian > > On 03.05.2023 09:07, Sebastian Blauth wrote: > > First of all, yes you are correct that I am trying to solve the > > stationary incompressible Navier Stokes equations. > > > > On 02.05.2023 21:33, Matthew Knepley wrote: > >> On Tue, May 2, 2023 at 2:29 PM Jed Brown <[email protected] > >> <mailto:[email protected]>> wrote: > >> > >> Sebastian Blauth <[email protected] > >> <mailto:[email protected]>> writes: > >> > >> > I agree with your comment for the Stokes equations - for these, I > >> have > >> > already tried and used the pressure mass matrix as part of a > >> (additive) > >> > block preconditioner and it gave mesh independent results. > >> > > >> > However, for the Navier Stokes equations, is the Schur complement > >> really > >> > spectrally equivalent to the pressure mass matrix? > >> > >> No, it's not. You'd want something like PCD (better, but not > >> algebraic) or LSC. > >> > > > > I would like to take a look at the LSC preconditioner. For this, I did > > also not achieve mesh-independent results. I am using the following > > options (I know that the tolerances are too high at the moment, but it > > should just illustrate the behavior w.r.t. mesh refinement). Again, I am > > using a simple 2D channel problem for testing purposes. > > > > I am using the following options > > > > -ksp_type fgmres > > -ksp_gmres_restart 100 > > -ksp_gmres_cgs_refinement_type refine_ifneeded > > -ksp_max_it 1000 > > -ksp_rtol 1e-10 > > -ksp_atol 1e-30 > > -pc_type fieldsplit > > -pc_fieldsplit_type schur > > -pc_fieldsplit_schur_fact_type full > > -pc_fieldsplit_schur_precondition self > > -fieldsplit_0_ksp_type preonly > > -fieldsplit_0_pc_type lu > > -fieldsplit_1_ksp_type gmres > > -fieldsplit_1_ksp_pc_side right > > -fieldsplit_1_ksp_gmres_restart 100 > > -fieldsplit_1_ksp_gmres_cgs_refinement_type refine_ifneeded > > -fieldsplit_1_ksp_max_it 1000 > > -fieldsplit_1_ksp_rtol 1e-10 > > -fieldsplit_1_ksp_atol 1e-30 > > -fieldsplit_1_pc_type lsc > > -fieldsplit_1_lsc_ksp_type preonly > > -fieldsplit_1_lsc_pc_type lu > > -fieldsplit_1_ksp_converged_reason > > > > Again, the direct solvers are used so that only the influence of the LSC > > preconditioner is seen. I have suitable preconditioners for all of these > > available (using boomeramg). > > > > At the bottom, I attach the output for different discretizations. As you > > can see there, the number of iterations increases nearly linearly with > > the problem size. > > > > I think that the problem could occur due to a wrong scaling. In your > > docs https://petsc.org/release/manualpages/PC/PCLSC/ , you write that > > the LSC preconditioner is implemented as > > > > inv(S) \approx inv(A10 A01) A10 A00 A01 inv(A10 A01) > > > > However, in the book of Elman, Sylvester and Wathen (Finite Elements and > > Fast Iterative Solvers), the LSC preconditioner is defined as > > > > inv(S) \approx inv(A10 inv(T) A01) A10 inv(T) A00 inv(T) A01 > > inv(A10 inv(T) A01) > > > > where T = diag(Q) and Q is the velocity mass matrix. > > > > There is an options -pc_lsc_scale_diag, which states that it uses the > > diagonal of A for scaling. I suppose, that this means, that the diagonal > > of the A00 block is used for scaling - however, this is not the right > > scaling, is it? Even in the source code for the LSC preconditioner, in > > /src/ksp/pc/impls/lsc/lsc.c it is mentioned, that a mass matrix should > > be used... > > Is there any way to implement this in PETSc? Maybe by supplying the mass > > matrix as Pmat? > > > > Thanks a lot in advance, > > Sebastian > > > >> > >> I think you can do a better job than that using something like > >> > >> https://arxiv.org/abs/1810.03315 <https://arxiv.org/abs/1810.03315> > >> > >> Basically, you use an augmented Lagrangian thing to make the Schur > >> complement well-conditioned, > >> and then use a special smoother to handle that perturbation. > >> > >> > And even if it is, the convergence is only good for small > >> Reynolds numbers, for moderately high ones the convergence really > >> deteriorates. This is why I am trying to make > >> fieldsplit_schur_precondition selfp work better (this is, if I > >> understand it correctly, a SIMPLE type preconditioner). > >> > >> SIMPLE is for short time steps (not too far from resolving CFL) and > >> bad for steady. This taxonomy is useful, though the problems are > >> super academic and they don't use high aspect ratio. > >> > > > > Okay, I get that I cannot expect the SIMPLE preconditioning > > (schur_precondition selfp) to work efficiently. I guess the reason it > > works for small time steps (for the instationary equations) is that the > > velocity block becomes diagonally dominant in this case, so that diag(A) > > is a good approximation of A. > > > > > >> https://doi.org/10.1016/j.jcp.2007.09.026 > >> <https://doi.org/10.1016/j.jcp.2007.09.026> > >> > >> > >> Thanks, > >> > >> Matt > >> > >> -- > >> What most experimenters take for granted before they begin their > >> experiments is infinitely more interesting than any results to which > >> their experiments lead. > >> -- Norbert Wiener > >> > >> https://www.cse.buffalo.edu/~knepley/ > >> <http://www.cse.buffalo.edu/~knepley/> > > > > > > And here is the output of my scaling tests > > > > 8x8 discretization > > > > Newton solver: iter, abs. residual (abs. tol), rel. residual (rel. > tol) > > > > Newton solver: 0, 1.023e+03 (1.00e-30), 1.000e+00 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > 38 > > Newton solver: 1, 1.313e+03 (1.00e-30), 1.283e+00 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > 76 > > Newton solver: 2, 1.198e+02 (1.00e-30), 1.171e-01 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > 74 > > Newton solver: 3, 7.249e-01 (1.00e-30), 7.084e-04 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > 74 > > Newton solver: 4, 3.883e-05 (1.00e-30), 3.795e-08 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > 74 > > Newton solver: 5, 2.778e-12 (1.00e-30), 2.714e-15 > (1.00e-10) > > > > > > > > 16x16 discretization > > > > Newton solver: iter, abs. residual (abs. tol), rel. residual (rel. > tol) > > > > Newton solver: 0, 1.113e+03 (1.00e-30), 1.000e+00 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > 62 > > Newton solver: 1, 8.316e+02 (1.00e-30), 7.475e-01 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > > 141 > > Newton solver: 2, 5.806e+01 (1.00e-30), 5.218e-02 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > > 119 > > Newton solver: 3, 3.309e-01 (1.00e-30), 2.974e-04 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > > 118 > > Newton solver: 4, 9.085e-06 (1.00e-30), 8.166e-09 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > > 120 > > Newton solver: 5, 3.475e-12 (1.00e-30), 3.124e-15 > (1.00e-10) > > > > > > > > 32x32 discretization > > > > Newton solver: iter, abs. residual (abs. tol), rel. residual (rel. > tol) > > > > Newton solver: 0, 1.330e+03 (1.00e-30), 1.000e+00 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > 98 > > Newton solver: 1, 5.913e+02 (1.00e-30), 4.445e-01 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > > 183 > > Newton solver: 2, 3.214e+01 (1.00e-30), 2.416e-02 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > > 152 > > Newton solver: 3, 2.059e-01 (1.00e-30), 1.547e-04 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > > 151 > > Newton solver: 4, 6.949e-06 (1.00e-30), 5.223e-09 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > > 149 > > Newton solver: 5, 5.300e-12 (1.00e-30), 3.983e-15 > (1.00e-10) > > > > > > > > 64x64 discretization > > > > Newton solver: iter, abs. residual (abs. tol), rel. residual (rel. > tol) > > > > Newton solver: 0, 1.707e+03 (1.00e-30), 1.000e+00 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > > 198 > > Newton solver: 1, 4.259e+02 (1.00e-30), 2.494e-01 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > > 357 > > Newton solver: 2, 1.706e+01 (1.00e-30), 9.993e-03 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > > 266 > > Newton solver: 3, 1.134e-01 (1.00e-30), 6.639e-05 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > > 261 > > Newton solver: 4, 4.285e-06 (1.00e-30), 2.510e-09 > (1.00e-10) > > Linear fieldsplit_1_ solve converged due to CONVERGED_RTOL iterations > > 263 > > Newton solver: 5, 9.650e-12 (1.00e-30), 5.652e-15 > (1.00e-10) > > > > -- > Dr. Sebastian Blauth > Fraunhofer-Institut für > Techno- und Wirtschaftsmathematik ITWM > Abteilung Transportvorgänge > Fraunhofer-Platz 1, 67663 Kaiserslautern > Telefon: +49 631 31600-4968 > [email protected] > www.itwm.fraunhofer.de >
