Dear Pierre Jolivet Thank you for your explanation.
I will try to use a converting matrix. I know it's really inefficient, but I need an inverse matrix (inv(A)) itself for my research. If parallel computing is difficult to get inv(A), can I run the part related to MatMatSolve with a single core? Best, Seung Lee Kwon 2023년 5월 5일 (금) 오후 8:35, Pierre Jolivet <[email protected]>님이 작성: > > > On 5 May 2023, at 1:25 PM, 권승리 / 학생 / 항공우주공학과 <[email protected]> wrote: > > Dear Matthew Knepley > > However, I've already installed ScaLAPACK. > cd $PETSC_DIR > ./configure --download-mpich --with-debugging=0 COPTFLAGS='-O3 > -march=native -mtune=native' CXXOPTFLAGS='-O3 -march=native -mtune=native' > FOPTFLAGS='-O3 -march=native -mtune=native' --download-mumps -- > *download-scalapack* --download-parmetis --download-metis > --download-parmetis --download-hpddm --download-slepc > > Is there some way to use ScaLAPCK? > > > You need to convert your MatDense to MatSCALAPACK before the call to > MatMatSolve(). > This library (ScaLAPACK, but also Elemental) has severe limitations with > respect to the matrix distribution. > Depending on what you are doing, you may be better of using KSPMatSolve() > and computing only an approximation of the solution with a cheap > preconditioner (I don’t recall you telling us why you need to do such an > operation even though we told you it was not practical — or maybe I’m being > confused by another thread). > > Thanks, > Pierre > > Or, Can I run the part related to MatMatSolve with a single core? > > 2023년 5월 5일 (금) 오후 6:21, Matthew Knepley <[email protected]>님이 작성: > >> On Fri, May 5, 2023 at 3:49 AM 권승리 / 학생 / 항공우주공학과 <[email protected]> >> wrote: >> >>> Dear Barry Smith >>> >>> Thanks to you, I knew the difference between MATAIJ and MATDENSE. >>> >>> However, I still have some problems. >>> >>> There is no problem when I run with a single core. But, MatGetFactor >>> error occurs when using multi-core. >>> >>> Could you give me some advice? >>> >>> The error message is >>> >>> [0]PETSC ERROR: --------------------- Error Message >>> -------------------------------------------------------------- >>> [0]PETSC ERROR: See >>> https://petsc.org/release/overview/linear_solve_table/ for possible LU >>> and Cholesky solvers >>> [0]PETSC ERROR: MatSolverType petsc does not support matrix type mpidense >>> >> >> PETSc uses 3rd party packages for parallel dense factorization. You would >> need to reconfigure with either ScaLAPACK >> or Elemental. >> >> Thanks, >> >> Matt >> >> >>> [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. >>> [0]PETSC ERROR: Petsc Release Version 3.18.5, unknown >>> [0]PETSC ERROR: ./app on a arch-linux-c-opt named ubuntu by ksl Fri May >>> 5 00:35:23 2023 >>> [0]PETSC ERROR: Configure options --download-mpich --with-debugging=0 >>> COPTFLAGS="-O3 -march=native -mtune=native" CXXOPTFLAGS="-O3 -march=native >>> -mtune=native" FOPTFLAGS="-O3 -march=native -mtune=native" --download-mumps >>> --download-scalapack --download-parmetis --download-metis >>> --download-parmetis --download-hpddm --download-slepc >>> [0]PETSC ERROR: #1 MatGetFactor() at >>> /home/ksl/petsc/src/mat/interface/matrix.c:4757 >>> [0]PETSC ERROR: #2 main() at >>> /home/ksl/Downloads/coding_test/coding/a1.c:66 >>> [0]PETSC ERROR: No PETSc Option Table entries >>> [0]PETSC ERROR: ----------------End of Error Message -------send entire >>> error message to [email protected] >>> application called MPI_Abort(MPI_COMM_SELF, 92) - process 0 >>> >>> My code is below: >>> >>> int main(int argc, char** args) >>> { >>> Mat A, E, A_temp, A_fac; >>> int n = 15; >>> PetscInitialize(&argc, &args, NULL, NULL); >>> PetscCallMPI(MPI_Comm_size(PETSC_COMM_WORLD, &size)); >>> >>> PetscCall(MatCreate(PETSC_COMM_WORLD, &A)); >>> PetscCall(MatSetType(A,MATDENSE)); >>> PetscCall(MatSetSizes(A, PETSC_DECIDE, PETSC_DECIDE, n, n)); >>> PetscCall(MatSetFromOptions(A)); >>> PetscCall(MatSetUp(A)); >>> // Insert values >>> double val; >>> for (int i = 0; i < n; i++) { >>> for (int j = 0; j < n; j++) { >>> if (i == j){ >>> val = 2.0; >>> } >>> else{ >>> val = 1.0; >>> } >>> PetscCall(MatSetValue(A, i, j, val, INSERT_VALUES)); >>> } >>> } >>> PetscCall(MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY)); >>> PetscCall(MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY)); >>> >>> // Make Identity matrix >>> PetscCall(MatCreate(PETSC_COMM_WORLD, &E)); >>> PetscCall(MatSetType(E,MATDENSE)); >>> PetscCall(MatSetSizes(E, PETSC_DECIDE, PETSC_DECIDE, n, n)); >>> PetscCall(MatSetFromOptions(E)); >>> PetscCall(MatSetUp(E)); >>> PetscCall(MatShift(E,1.0)); >>> PetscCall(MatAssemblyBegin(E, MAT_FINAL_ASSEMBLY)); >>> PetscCall(MatAssemblyEnd(E, MAT_FINAL_ASSEMBLY)); >>> >>> PetscCall(MatDuplicate(A, MAT_DO_NOT_COPY_VALUES, &A_temp)); >>> PetscCall(MatGetFactor(A, MATSOLVERPETSC, MAT_FACTOR_LU, &A_fac)); >>> >>> IS isr, isc; MatFactorInfo info; >>> MatGetOrdering(A, MATORDERINGNATURAL, &isr, &isc); >>> PetscCall(MatLUFactorSymbolic(A_fac, A, isr, isc, &info)); >>> PetscCall(MatLUFactorNumeric(A_fac, A, &info)); >>> MatMatSolve(A_fac, E, A_temp); >>> >>> PetscCall(MatView(A_temp, PETSC_VIEWER_STDOUT_WORLD)); >>> MatDestroy(&A); >>> MatDestroy(&A_temp); >>> MatDestroy(&A_fac); >>> MatDestroy(&E); >>> PetscCall(PetscFinalize()); >>> } >>> >>> Best regards >>> Seung Lee Kwon >>> >>> 2023년 5월 4일 (목) 오후 10:19, Barry Smith <[email protected]>님이 작성: >>> >>>> >>>> The code in ex125.c contains >>>> >>>> PetscCall(MatCreate(PETSC_COMM_WORLD, &C)); >>>> PetscCall(MatSetOptionsPrefix(C, "rhs_")); >>>> PetscCall(MatSetSizes(C, m, PETSC_DECIDE, PETSC_DECIDE, nrhs)); >>>> PetscCall(MatSetType(C, MATDENSE)); >>>> PetscCall(MatSetFromOptions(C)); >>>> PetscCall(MatSetUp(C)); >>>> >>>> This dense parallel matrix is suitable for passing to MatMatSolve() as >>>> the right-hand side matrix. Note it is created with PETSC_COMM_WORLD and >>>> its type is set to be MATDENSE. >>>> >>>> You may need to make a sample code by stripping out all the excess >>>> code in ex125.c to just create an MATAIJ and MATDENSE and solves with >>>> MatMatSolve() to determine why you code does not work. >>>> >>>> >>>> >>>> On May 4, 2023, at 3:20 AM, 권승리 / 학생 / 항공우주공학과 <[email protected]> >>>> wrote: >>>> >>>> Dear Barry Smith >>>> >>>> Thank you for your reply. >>>> >>>> I've already installed MUMPS. >>>> >>>> And I checked the example you said (ex125.c), I don't understand why >>>> the RHS matrix becomes the SeqDense matrix. >>>> >>>> Could you explain in more detail? >>>> >>>> Best regards >>>> Seung Lee Kwon >>>> >>>> 2023년 5월 4일 (목) 오후 12:08, Barry Smith <[email protected]>님이 작성: >>>> >>>>> >>>>> You can configure with MUMPS ./configure --download-mumps >>>>> --download-scalapack --download-ptscotch --download-metis >>>>> --download-parmetis >>>>> >>>>> And then use MatMatSolve() as in src/mat/tests/ex125.c with >>>>> parallel MatMatSolve() using MUMPS as the solver. >>>>> >>>>> Barry >>>>> >>>>> >>>>> On May 3, 2023, at 10:29 PM, 권승리 / 학생 / 항공우주공학과 <[email protected]> >>>>> wrote: >>>>> >>>>> Dear developers >>>>> >>>>> Thank you for your explanation. >>>>> >>>>> But I should use the MatCreateSeqDense because I want to use the >>>>> MatMatSolve that B matrix must be a SeqDense matrix. >>>>> >>>>> Using MatMatSolve is an inevitable part of my code. >>>>> >>>>> Could you give me a comment to avoid this error? >>>>> >>>>> Best, >>>>> >>>>> Seung Lee Kwon >>>>> >>>>> 2023년 5월 3일 (수) 오후 7:30, Matthew Knepley <[email protected]>님이 작성: >>>>> >>>>>> On Wed, May 3, 2023 at 6:05 AM 권승리 / 학생 / 항공우주공학과 <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> Dear developers >>>>>>> >>>>>>> I'm trying to use parallel computing and I ran the command 'mpirun >>>>>>> -np 4 ./app' >>>>>>> >>>>>>> In this case, there are two problems. >>>>>>> >>>>>>> *First,* I encountered error message >>>>>>> /// >>>>>>> [0]PETSC ERROR: [1]PETSC ERROR: --------------------- Error Message >>>>>>> -------------------------------------------------------------- >>>>>>> [1]PETSC ERROR: Invalid argument >>>>>>> [1]PETSC ERROR: Comm must be of size 1 >>>>>>> /// >>>>>>> The code on the error position is >>>>>>> MatCreateSeqDense(PETSC_COMM_SELF, nns, ns, NULL, &Kns)); >>>>>>> >>>>>> >>>>>> 1) "Seq" means sequential, that is "not parallel". >>>>>> >>>>>> 2) This line should still be fine since PETSC_COMM_SELF is a serial >>>>>> communicator >>>>>> >>>>>> 3) You should be checking the error code for each call, maybe using >>>>>> the CHKERRQ() macro >>>>>> >>>>>> 4) Please always send the entire error message, not a snippet >>>>>> >>>>>> THanks >>>>>> >>>>>> Matt >>>>>> >>>>>> >>>>>>> Could "MatCreateSeqDense" not be used in parallel computing? >>>>>>> >>>>>>> *Second*, the same error message is repeated as many times as the >>>>>>> number of cores. >>>>>>> if I use command -np 4, then the error message is repeated 4 times. >>>>>>> Could you recommend some advice related to this? >>>>>>> >>>>>>> Best, >>>>>>> Seung Lee Kwon >>>>>>> >>>>>>> -- >>>>>>> Seung Lee Kwon, Ph.D.Candidate >>>>>>> Aerospace Structures and Materials Laboratory >>>>>>> Department of Mechanical and Aerospace Engineering >>>>>>> Seoul National University >>>>>>> Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, >>>>>>> 08826 >>>>>>> E-mail : [email protected] >>>>>>> Office : +82-2-880-7389 >>>>>>> C. P : +82-10-4695-1062 >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> What most experimenters take for granted before they begin their >>>>>> experiments is infinitely more interesting than any results to which >>>>>> their >>>>>> experiments lead. >>>>>> -- Norbert Wiener >>>>>> >>>>>> https://www.cse.buffalo.edu/~knepley/ >>>>>> <http://www.cse.buffalo.edu/~knepley/> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Seung Lee Kwon, Ph.D.Candidate >>>>> Aerospace Structures and Materials Laboratory >>>>> Department of Mechanical and Aerospace Engineering >>>>> Seoul National University >>>>> Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826 >>>>> E-mail : [email protected] >>>>> Office : +82-2-880-7389 >>>>> C. P : +82-10-4695-1062 >>>>> >>>>> >>>>> >>>> >>>> -- >>>> Seung Lee Kwon, Ph.D.Candidate >>>> Aerospace Structures and Materials Laboratory >>>> Department of Mechanical and Aerospace Engineering >>>> Seoul National University >>>> Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826 >>>> E-mail : [email protected] >>>> Office : +82-2-880-7389 >>>> C. P : +82-10-4695-1062 >>>> >>>> >>>> >>> >>> -- >>> Seung Lee Kwon, Ph.D.Candidate >>> Aerospace Structures and Materials Laboratory >>> Department of Mechanical and Aerospace Engineering >>> Seoul National University >>> Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826 >>> E-mail : [email protected] >>> Office : +82-2-880-7389 >>> C. P : +82-10-4695-1062 >>> >> >> >> -- >> What most experimenters take for granted before they begin their >> experiments is infinitely more interesting than any results to which their >> experiments lead. >> -- Norbert Wiener >> >> https://www.cse.buffalo.edu/~knepley/ >> <http://www.cse.buffalo.edu/~knepley/> >> > > > -- > Seung Lee Kwon, Ph.D.Candidate > Aerospace Structures and Materials Laboratory > Department of Mechanical and Aerospace Engineering > Seoul National University > Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826 > E-mail : [email protected] > Office : +82-2-880-7389 > C. P : +82-10-4695-1062 > > > -- Seung Lee Kwon, Ph.D.Candidate Aerospace Structures and Materials Laboratory Department of Mechanical and Aerospace Engineering Seoul National University Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826 E-mail : [email protected] Office : +82-2-880-7389 C. P : +82-10-4695-1062
