> On 12 Jul 2022, at 2:13 PM, Pierre Jolivet <[email protected]> wrote: > > > >> On 12 Jul 2022, at 2:04 PM, Mark Adams <[email protected] >> <mailto:[email protected]>> wrote: >> >> It looks like the RHS is zero in the 2nd case (0 KSP unpreconditioned resid >> norm 0.000000000000e+00), but the true residual is the same. >> >> It looks like you added "nest_subvec" to our example. > > It’s a new (dumb) example (ex1111). > >> You can start by looking at the vectors with -vec_view (there is code that >> you can view vectors explicitly in your code to get more control if this is >> too noisy) > > The Vec are exactly the same (as reported by -ksp_view_rhs) in both cases. > >> You could send us the diffs in ex111. We may be able to see something wrong. > > I believe there is something weird inside KSPGMRES + PCFIELDSPLIT + right > preconditioning.
More precisely, this is the faulty line: https://petsc.org/main/src/ksp/ksp/impls/gmres/gmres.c.html#line119 <https://petsc.org/main/src/ksp/ksp/impls/gmres/gmres.c.html#line119> Here is the value of res with -nest_subvec false: 3.162278 With -nest_subvec true: 2.387784 The input Vec being Vec Object: 1 MPI process type: nest VecNest, rows=2, structure: (0) : type=seq, rows=5 Vec Object: 1 MPI process type: seq 0.720032 0.061794 0.0100223 0.144639 0.397778 (1) : type=seq, rows=5 Vec Object: 1 MPI process type: seq 1. 1. 1. 1. 1. One can see that either VecNormalize() or VecNest is indeed broken. Thanks, Pierre > Because if I switch to KSPHPDDM implementation of GMRES, then all is good. > In the below traces, I switched VecSet(part of RHS,0.0) to VecSet(part of > RHS,1.0) in ex1111.c to make the error more visible. > > Thanks, > Pierre > > $ ./ex1111 -pc_type fieldsplit -ksp_converged_reason -fieldsplit_pc_type lu > -ksp_pc_side right -ksp_view_final_residual -nest_subvec false -ksp_view_rhs > -ksp_view_solution > Linear solve did not converge due to DIVERGED_BREAKDOWN iterations 30 < > garbage > KSP final norm of residual 1.75544 < garbage > Vec Object: 1 MPI process > type: nest > VecNest, rows=2, structure: > (0) : type=seq, rows=5 > Vec Object: 1 MPI process > type: seq > 0.720032 > 0.061794 > 0.0100223 > 0.144639 > 0.397778 > (1) : type=seq, rows=5 > Vec Object: 1 MPI process > type: seq > 1. > 1. > 1. > 1. > 1. > Vec Object: 1 MPI process > type: nest > VecNest, rows=2, structure: > (0) : type=seq, rows=5 > Vec Object: 1 MPI process > type: seq > 1.73518 < garbage > 2.07583e-16 > -8.13135e-16 > -4.24749e-17 > 8.18337e-16 > (1) : type=seq, rows=5 > Vec Object: 1 MPI process > type: seq > -1.73518 < garbage > -1.73518 < garbage > -1.73518 < garbage > -1.73518 < garbage > -1.73518 < garbage > > $ ./ex1111 -pc_type fieldsplit -ksp_converged_reason -fieldsplit_pc_type lu > -ksp_pc_side right -ksp_view_final_residual -nest_subvec false -ksp_view_rhs > -ksp_view_solution -ksp_type hpddm > Linear solve converged due to CONVERGED_RTOL iterations 1 < correct > KSP final norm of residual 7.89955e-16 > Vec Object: 1 MPI process > type: nest > VecNest, rows=2, structure: > (0) : type=seq, rows=5 > Vec Object: 1 MPI process > type: seq > 0.720032 > 0.061794 > 0.0100223 > 0.144639 > 0.397778 > (1) : type=seq, rows=5 > Vec Object: 1 MPI process > type: seq > 1. > 1. > 1. > 1. > 1. > Vec Object: 1 MPI process > type: nest > VecNest, rows=2, structure: > (0) : type=seq, rows=5 > Vec Object: 1 MPI process > type: seq > 1. < correct > 1.79841e-18 > -5.45696e-17 > 8.83987e-18 > 2.82354e-17 > (1) : type=seq, rows=5 > Vec Object: 1 MPI process > type: seq > -1. < correct > -1. < correct > -1. < correct > -1. < correct > -1. < correct > > $ ./ex1111 -pc_type fieldsplit -ksp_converged_reason -fieldsplit_pc_type lu > -ksp_pc_side right -ksp_view_final_residual -nest_subvec true -ksp_view_rhs > -ksp_view_solution > Linear solve converged due to CONVERGED_RTOL iterations 1 < correct > KSP final norm of residual 2.74152e-16 > Vec Object: 1 MPI process > type: nest > VecNest, rows=2, structure: > (0) : type=seq, rows=5 > Vec Object: 1 MPI process > type: seq > 0.720032 > 0.061794 > 0.0100223 > 0.144639 > 0.397778 > (1) : type=seq, rows=5 > Vec Object: 1 MPI process > type: seq > 1. > 1. > 1. > 1. > 1. > Vec Object: 1 MPI process > type: nest > VecNest, rows=2, structure: > (0) : type=seq, rows=5 > Vec Object: 1 MPI process > type: seq > 1. < correct > -3.75571e-17 > 2.51808e-16 > 7.69081e-18 > -2.65477e-16 > (1) : type=seq, rows=5 > Vec Object: 1 MPI process > type: seq > -1. < correct > -1. < correct > -1. < correct > -1. < correct > -1. < correct > >> Mark >> >> On Mon, Jul 11, 2022 at 2:17 PM Pierre Jolivet <[email protected] >> <mailto:[email protected]>> wrote: >> Hello, >> Could anyone help me understand what is going on in the following example, >> please? >> I have a VecNest. >> I either: a) initialize all values to 0.0, then set a specific part of the >> vector to nonzero or b) initialize a part of the vector to 0.0 and set the >> other part to nonzero. >> I don’t see why a) and b) produce different results. >> >> $ ./ex1111 -pc_type fieldsplit -ksp_monitor_true_residual >> -ksp_converged_reason -fieldsplit_pc_type jacobi -ksp_pc_side right >> -ksp_view_final_residual -nest_subvec true >> 0 KSP unpreconditioned resid norm 8.375635517980e-01 true resid norm >> 8.375635517980e-01 ||r(i)||/||b|| 1.000000000000e+00 >> 1 KSP unpreconditioned resid norm 4.748816884247e-01 true resid norm >> 4.748816884247e-01 ||r(i)||/||b|| 5.669798875623e-01 >> 2 KSP unpreconditioned resid norm 4.713006778679e-01 true resid norm >> 4.713006778679e-01 ||r(i)||/||b|| 5.627043784990e-01 >> 3 KSP unpreconditioned resid norm 7.092979927129e-02 true resid norm >> 7.092979927129e-02 ||r(i)||/||b|| 8.468587144106e-02 >> 4 KSP unpreconditioned resid norm 1.457836310255e-02 true resid norm >> 1.457836310255e-02 ||r(i)||/||b|| 1.740567992870e-02 >> 5 KSP unpreconditioned resid norm 1.625040500524e-14 true resid norm >> 1.633468028779e-14 ||r(i)||/||b|| 1.950261595401e-14 >> Linear solve converged due to CONVERGED_RTOL iterations 5 >> KSP final norm of residual 1.63347e-14 >> $ ./ex1111 -pc_type fieldsplit -ksp_monitor_true_residual >> -ksp_converged_reason -fieldsplit_pc_type jacobi -ksp_pc_side right >> -ksp_view_final_residual -nest_subvec false >> 0 KSP unpreconditioned resid norm 0.000000000000e+00 true resid norm >> 8.375635517980e-01 ||r(i)||/||b|| inf >> Linear solve converged due to CONVERGED_ATOL iterations 0 >> KSP final norm of residual 0.837564 >> >> Thanks, >> Pierre >> >
