> On 5 May 2023, at 2:00 PM, 권승리 / 학생 / 항공우주공학과 <[email protected]> wrote: > > Dear Pierre Jolivet > > Thank you for your explanation. > > I will try to use a converting matrix. > > I know it's really inefficient, but I need an inverse matrix (inv(A)) itself > for my research. > > If parallel computing is difficult to get inv(A), can I run the part related > to MatMatSolve with a single core?
Yes. Thanks, Pierre > Best, > Seung Lee Kwon > > 2023년 5월 5일 (금) 오후 8:35, Pierre Jolivet <[email protected] > <mailto:[email protected]>>님이 작성: >> >> >>> On 5 May 2023, at 1:25 PM, 권승리 / 학생 / 항공우주공학과 <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> Dear Matthew Knepley >>> >>> However, I've already installed ScaLAPACK. >>> cd $PETSC_DIR >>> ./configure --download-mpich --with-debugging=0 COPTFLAGS='-O3 >>> -march=native -mtune=native' CXXOPTFLAGS='-O3 -march=native -mtune=native' >>> FOPTFLAGS='-O3 -march=native -mtune=native' --download-mumps >>> --download-scalapack --download-parmetis --download-metis >>> --download-parmetis --download-hpddm --download-slepc >>> >>> Is there some way to use ScaLAPCK? >> >> You need to convert your MatDense to MatSCALAPACK before the call to >> MatMatSolve(). >> This library (ScaLAPACK, but also Elemental) has severe limitations with >> respect to the matrix distribution. >> Depending on what you are doing, you may be better of using KSPMatSolve() >> and computing only an approximation of the solution with a cheap >> preconditioner (I don’t recall you telling us why you need to do such an >> operation even though we told you it was not practical — or maybe I’m being >> confused by another thread). >> >> Thanks, >> Pierre >> >>> Or, Can I run the part related to MatMatSolve with a single core? >>> >>> 2023년 5월 5일 (금) 오후 6:21, Matthew Knepley <[email protected] >>> <mailto:[email protected]>>님이 작성: >>>> On Fri, May 5, 2023 at 3:49 AM 권승리 / 학생 / 항공우주공학과 <[email protected] >>>> <mailto:[email protected]>> wrote: >>>>> Dear Barry Smith >>>>> >>>>> Thanks to you, I knew the difference between MATAIJ and MATDENSE. >>>>> >>>>> However, I still have some problems. >>>>> >>>>> There is no problem when I run with a single core. But, MatGetFactor >>>>> error occurs when using multi-core. >>>>> >>>>> Could you give me some advice? >>>>> >>>>> The error message is >>>>> >>>>> [0]PETSC ERROR: --------------------- Error Message >>>>> -------------------------------------------------------------- >>>>> [0]PETSC ERROR: See >>>>> https://petsc.org/release/overview/linear_solve_table/ for possible LU >>>>> and Cholesky solvers >>>>> [0]PETSC ERROR: MatSolverType petsc does not support matrix type mpidense >>>> >>>> PETSc uses 3rd party packages for parallel dense factorization. You would >>>> need to reconfigure with either ScaLAPACK >>>> or Elemental. >>>> >>>> Thanks, >>>> >>>> Matt >>>> >>>>> [0]PETSC ERROR: See https://petsc.org/release/faq/ for trouble shooting. >>>>> [0]PETSC ERROR: Petsc Release Version 3.18.5, unknown >>>>> [0]PETSC ERROR: ./app on a arch-linux-c-opt named ubuntu by ksl Fri May >>>>> 5 00:35:23 2023 >>>>> [0]PETSC ERROR: Configure options --download-mpich --with-debugging=0 >>>>> COPTFLAGS="-O3 -march=native -mtune=native" CXXOPTFLAGS="-O3 >>>>> -march=native -mtune=native" FOPTFLAGS="-O3 -march=native -mtune=native" >>>>> --download-mumps --download-scalapack --download-parmetis >>>>> --download-metis --download-parmetis --download-hpddm --download-slepc >>>>> [0]PETSC ERROR: #1 MatGetFactor() at >>>>> /home/ksl/petsc/src/mat/interface/matrix.c:4757 >>>>> [0]PETSC ERROR: #2 main() at >>>>> /home/ksl/Downloads/coding_test/coding/a1.c:66 >>>>> [0]PETSC ERROR: No PETSc Option Table entries >>>>> [0]PETSC ERROR: ----------------End of Error Message -------send entire >>>>> error message to [email protected] >>>>> application called MPI_Abort(MPI_COMM_SELF, 92) - process 0 >>>>> >>>>> My code is below: >>>>> >>>>> int main(int argc, char** args) >>>>> { >>>>> Mat A, E, A_temp, A_fac; >>>>> int n = 15; >>>>> PetscInitialize(&argc, &args, NULL, NULL); >>>>> PetscCallMPI(MPI_Comm_size(PETSC_COMM_WORLD, &size)); >>>>> >>>>> PetscCall(MatCreate(PETSC_COMM_WORLD, &A)); >>>>> PetscCall(MatSetType(A,MATDENSE)); >>>>> PetscCall(MatSetSizes(A, PETSC_DECIDE, PETSC_DECIDE, n, n)); >>>>> PetscCall(MatSetFromOptions(A)); >>>>> PetscCall(MatSetUp(A)); >>>>> // Insert values >>>>> double val; >>>>> for (int i = 0; i < n; i++) { >>>>> for (int j = 0; j < n; j++) { >>>>> if (i == j){ >>>>> val = 2.0; >>>>> } >>>>> else{ >>>>> val = 1.0; >>>>> } >>>>> PetscCall(MatSetValue(A, i, j, val, INSERT_VALUES)); >>>>> } >>>>> } >>>>> PetscCall(MatAssemblyBegin(A, MAT_FINAL_ASSEMBLY)); >>>>> PetscCall(MatAssemblyEnd(A, MAT_FINAL_ASSEMBLY)); >>>>> >>>>> // Make Identity matrix >>>>> PetscCall(MatCreate(PETSC_COMM_WORLD, &E)); >>>>> PetscCall(MatSetType(E,MATDENSE)); >>>>> PetscCall(MatSetSizes(E, PETSC_DECIDE, PETSC_DECIDE, n, n)); >>>>> PetscCall(MatSetFromOptions(E)); >>>>> PetscCall(MatSetUp(E)); >>>>> PetscCall(MatShift(E,1.0)); >>>>> PetscCall(MatAssemblyBegin(E, MAT_FINAL_ASSEMBLY)); >>>>> PetscCall(MatAssemblyEnd(E, MAT_FINAL_ASSEMBLY)); >>>>> >>>>> PetscCall(MatDuplicate(A, MAT_DO_NOT_COPY_VALUES, &A_temp)); >>>>> PetscCall(MatGetFactor(A, MATSOLVERPETSC, MAT_FACTOR_LU, &A_fac)); >>>>> >>>>> IS isr, isc; MatFactorInfo info; >>>>> MatGetOrdering(A, MATORDERINGNATURAL, &isr, &isc); >>>>> PetscCall(MatLUFactorSymbolic(A_fac, A, isr, isc, &info)); >>>>> PetscCall(MatLUFactorNumeric(A_fac, A, &info)); >>>>> MatMatSolve(A_fac, E, A_temp); >>>>> >>>>> PetscCall(MatView(A_temp, PETSC_VIEWER_STDOUT_WORLD)); >>>>> MatDestroy(&A); >>>>> MatDestroy(&A_temp); >>>>> MatDestroy(&A_fac); >>>>> MatDestroy(&E); >>>>> PetscCall(PetscFinalize()); >>>>> } >>>>> >>>>> Best regards >>>>> Seung Lee Kwon >>>>> >>>>> 2023년 5월 4일 (목) 오후 10:19, Barry Smith <[email protected] >>>>> <mailto:[email protected]>>님이 작성: >>>>>> >>>>>> The code in ex125.c contains >>>>>> >>>>>> PetscCall(MatCreate(PETSC_COMM_WORLD, &C)); >>>>>> PetscCall(MatSetOptionsPrefix(C, "rhs_")); >>>>>> PetscCall(MatSetSizes(C, m, PETSC_DECIDE, PETSC_DECIDE, nrhs)); >>>>>> PetscCall(MatSetType(C, MATDENSE)); >>>>>> PetscCall(MatSetFromOptions(C)); >>>>>> PetscCall(MatSetUp(C)); >>>>>> >>>>>> This dense parallel matrix is suitable for passing to MatMatSolve() as >>>>>> the right-hand side matrix. Note it is created with PETSC_COMM_WORLD and >>>>>> its type is set to be MATDENSE. >>>>>> >>>>>> You may need to make a sample code by stripping out all the excess >>>>>> code in ex125.c to just create an MATAIJ and MATDENSE and solves with >>>>>> MatMatSolve() to determine why you code does not work. >>>>>> >>>>>> >>>>>> >>>>>>> On May 4, 2023, at 3:20 AM, 권승리 / 학생 / 항공우주공학과 <[email protected] >>>>>>> <mailto:[email protected]>> wrote: >>>>>>> >>>>>>> Dear Barry Smith >>>>>>> >>>>>>> Thank you for your reply. >>>>>>> >>>>>>> I've already installed MUMPS. >>>>>>> >>>>>>> And I checked the example you said (ex125.c), I don't understand why >>>>>>> the RHS matrix becomes the SeqDense matrix. >>>>>>> >>>>>>> Could you explain in more detail? >>>>>>> >>>>>>> Best regards >>>>>>> Seung Lee Kwon >>>>>>> >>>>>>> 2023년 5월 4일 (목) 오후 12:08, Barry Smith <[email protected] >>>>>>> <mailto:[email protected]>>님이 작성: >>>>>>>> >>>>>>>> You can configure with MUMPS ./configure --download-mumps >>>>>>>> --download-scalapack --download-ptscotch --download-metis >>>>>>>> --download-parmetis >>>>>>>> >>>>>>>> And then use MatMatSolve() as in src/mat/tests/ex125.c with parallel >>>>>>>> MatMatSolve() using MUMPS as the solver. >>>>>>>> >>>>>>>> Barry >>>>>>>> >>>>>>>> >>>>>>>>> On May 3, 2023, at 10:29 PM, 권승리 / 학생 / 항공우주공학과 <[email protected] >>>>>>>>> <mailto:[email protected]>> wrote: >>>>>>>>> >>>>>>>>> Dear developers >>>>>>>>> >>>>>>>>> Thank you for your explanation. >>>>>>>>> >>>>>>>>> But I should use the MatCreateSeqDense because I want to use the >>>>>>>>> MatMatSolve that B matrix must be a SeqDense matrix. >>>>>>>>> >>>>>>>>> Using MatMatSolve is an inevitable part of my code. >>>>>>>>> >>>>>>>>> Could you give me a comment to avoid this error? >>>>>>>>> >>>>>>>>> Best, >>>>>>>>> >>>>>>>>> Seung Lee Kwon >>>>>>>>> >>>>>>>>> 2023년 5월 3일 (수) 오후 7:30, Matthew Knepley <[email protected] >>>>>>>>> <mailto:[email protected]>>님이 작성: >>>>>>>>>> On Wed, May 3, 2023 at 6:05 AM 권승리 / 학생 / 항공우주공학과 >>>>>>>>>> <[email protected] <mailto:[email protected]>> wrote: >>>>>>>>>>> Dear developers >>>>>>>>>>> >>>>>>>>>>> I'm trying to use parallel computing and I ran the command 'mpirun >>>>>>>>>>> -np 4 ./app' >>>>>>>>>>> >>>>>>>>>>> In this case, there are two problems. >>>>>>>>>>> >>>>>>>>>>> First, I encountered error message >>>>>>>>>>> /// >>>>>>>>>>> [0]PETSC ERROR: [1]PETSC ERROR: --------------------- Error Message >>>>>>>>>>> -------------------------------------------------------------- >>>>>>>>>>> [1]PETSC ERROR: Invalid argument >>>>>>>>>>> [1]PETSC ERROR: Comm must be of size 1 >>>>>>>>>>> /// >>>>>>>>>>> The code on the error position is >>>>>>>>>>> MatCreateSeqDense(PETSC_COMM_SELF, nns, ns, NULL, &Kns)); >>>>>>>>>> >>>>>>>>>> 1) "Seq" means sequential, that is "not parallel". >>>>>>>>>> >>>>>>>>>> 2) This line should still be fine since PETSC_COMM_SELF is a serial >>>>>>>>>> communicator >>>>>>>>>> >>>>>>>>>> 3) You should be checking the error code for each call, maybe using >>>>>>>>>> the CHKERRQ() macro >>>>>>>>>> >>>>>>>>>> 4) Please always send the entire error message, not a snippet >>>>>>>>>> >>>>>>>>>> THanks >>>>>>>>>> >>>>>>>>>> Matt >>>>>>>>>> >>>>>>>>>>> Could "MatCreateSeqDense" not be used in parallel computing? >>>>>>>>>>> >>>>>>>>>>> Second, the same error message is repeated as many times as the >>>>>>>>>>> number of cores. >>>>>>>>>>> if I use command -np 4, then the error message is repeated 4 times. >>>>>>>>>>> Could you recommend some advice related to this? >>>>>>>>>>> >>>>>>>>>>> Best, >>>>>>>>>>> Seung Lee Kwon >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Seung Lee Kwon, Ph.D.Candidate >>>>>>>>>>> Aerospace Structures and Materials Laboratory >>>>>>>>>>> Department of Mechanical and Aerospace Engineering >>>>>>>>>>> Seoul National University >>>>>>>>>>> Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, >>>>>>>>>>> 08826 >>>>>>>>>>> E-mail : [email protected] <mailto:[email protected]> >>>>>>>>>>> Office : +82-2-880-7389 >>>>>>>>>>> C. P : +82-10-4695-1062 >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> What most experimenters take for granted before they begin their >>>>>>>>>> experiments is infinitely more interesting than any results to which >>>>>>>>>> their experiments lead. >>>>>>>>>> -- Norbert Wiener >>>>>>>>>> >>>>>>>>>> https://www.cse.buffalo.edu/~knepley/ >>>>>>>>>> <http://www.cse.buffalo.edu/~knepley/> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Seung Lee Kwon, Ph.D.Candidate >>>>>>>>> Aerospace Structures and Materials Laboratory >>>>>>>>> Department of Mechanical and Aerospace Engineering >>>>>>>>> Seoul National University >>>>>>>>> Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826 >>>>>>>>> E-mail : [email protected] <mailto:[email protected]> >>>>>>>>> Office : +82-2-880-7389 >>>>>>>>> C. P : +82-10-4695-1062 >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Seung Lee Kwon, Ph.D.Candidate >>>>>>> Aerospace Structures and Materials Laboratory >>>>>>> Department of Mechanical and Aerospace Engineering >>>>>>> Seoul National University >>>>>>> Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826 >>>>>>> E-mail : [email protected] <mailto:[email protected]> >>>>>>> Office : +82-2-880-7389 >>>>>>> C. P : +82-10-4695-1062 >>>>>> >>>>> >>>>> >>>>> -- >>>>> Seung Lee Kwon, Ph.D.Candidate >>>>> Aerospace Structures and Materials Laboratory >>>>> Department of Mechanical and Aerospace Engineering >>>>> Seoul National University >>>>> Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826 >>>>> E-mail : [email protected] <mailto:[email protected]> >>>>> Office : +82-2-880-7389 >>>>> C. P : +82-10-4695-1062 >>>> >>>> >>>> -- >>>> What most experimenters take for granted before they begin their >>>> experiments is infinitely more interesting than any results to which their >>>> experiments lead. >>>> -- Norbert Wiener >>>> >>>> https://www.cse.buffalo.edu/~knepley/ >>>> <http://www.cse.buffalo.edu/~knepley/> >>> >>> >>> -- >>> Seung Lee Kwon, Ph.D.Candidate >>> Aerospace Structures and Materials Laboratory >>> Department of Mechanical and Aerospace Engineering >>> Seoul National University >>> Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826 >>> E-mail : [email protected] <mailto:[email protected]> >>> Office : +82-2-880-7389 >>> C. P : +82-10-4695-1062 >> > > > -- > Seung Lee Kwon, Ph.D.Candidate > Aerospace Structures and Materials Laboratory > Department of Mechanical and Aerospace Engineering > Seoul National University > Building 300 Rm 503, Gwanak-ro 1, Gwanak-gu, Seoul, South Korea, 08826 > E-mail : [email protected] <mailto:[email protected]> > Office : +82-2-880-7389 > C. P : +82-10-4695-1062
