On Wed, Dec 1, 2021 at 6:12 PM Betrie, Getnet <[email protected]> wrote:
> Here is a specific example. I have 10 global edges and 10 global vertices. > The vertices and edges represent neurons and synapses in the brain, > respectively. Vertices 0-7 and 7-9 have excitation and depression effect > on the target vertices. The first rank owns 5 edges (0-4) and 6 vertices ( > 0-4 local and 8 ghost). The second rank owns 5 edges (5-9) and 6 vertices > (5-9 local and 4 ghost). Note that edges 4 (neuron 8 depresses neuron 4) > and 8 (neuron 4 excite neuron 8) that share the same vertices on different > ranks. The generated Plex is correct based on the cone and support outputs. > However, the SF object is completely wrong as shown below. > I do not think it is wrong. > The first rank supposed to have 10 roots and one leaf (vertex 8), > No. The number of roots is the _complete_ point space, so here that is [0, 11), and the number of roots is correct. There are two leaf vertices, 9 and 10, mapping to vertices 5 and 9 on process 1. There is no way for me to know what vertices are supposed to be shared in your model. Matt > and the second rank suppose to have the same number of roots and leaf > (vertex 4) as the first rank. My guess is something went wrong in *sfPoint > *when determining the shared local vertices. I can provide more outputs > from DMPlex if it helps, please let me know. > > DM Object: 2 MPI processes > type: network > NSubnets: 1; NEdges: 10; NVertices: 10; NSharedVertices: 0. > [0] nEdges: 5; nVertices: 6; nSharedVertices: 0 > Subnet 0: nEdges 5, nVertices(include shared vertices) 6 > edge 0: 4 ----> 0 > edge 1: 4 ----> 1 > edge 2: 4 ----> 2 > edge 3: 4 ----> 3 > edge 4: 8 ----> 4 > [1] nEdges: 5; nVertices: 6; nSharedVertices: 0 > Subnet 0: nEdges 5, nVertices(include shared vertices) 6 > edge 5: 4 ----> 5 > edge 6: 4 ----> 6 > edge 7: 4 ----> 7 > edge 8: 4 ----> 8 > edge 9: 4 ----> 9 > > Cones: > [0] Max cone size: 2 > [0]: 0 <---- 9 (0) > [0]: 0 <---- 5 (0) > [0]: 1 <---- 9 (0) > [0]: 1 <---- 6 (0) > [0]: 2 <---- 9 (0) > [0]: 2 <---- 7 (0) > [0]: 3 <---- 9 (0) > [0]: 3 <---- 8 (0) > [0]: 4 <---- 10 (0) > [0]: 4 <---- 9 (0) > [1] Max cone size: 2 > [1]: 0 <---- 5 (0) > [1]: 0 <---- 6 (0) > [1]: 1 <---- 5 (0) > [1]: 1 <---- 7 (0) > [1]: 2 <---- 5 (0) > [1]: 2 <---- 8 (0) > [1]: 3 <---- 5 (0) > [1]: 3 <---- 9 (0) > [1]: 4 <---- 5 (0) > [1]: 4 <---- 10 (0) > > PetscSF Object: 2 MPI processes > type: basic > [0] Number of roots=11, leaves=2, remote ranks=1 > [0] 9 <- (1,5) > [0] 10 <- (1,9) > [1] Number of roots=11, leaves=0, remote ranks=0 > [0] Roots referenced by my leaves, by rank > [0] 1: 2 edges > [0] 9 <- 5 > [0] 10 <- 9 > [1] Roots referenced by my leaves, by rank > MultiSF sort=rank-order > > ------------------------------ > *From:* Abhyankar, Shrirang G <[email protected]> > *Sent:* Wednesday, December 1, 2021 4:17 PM > *To:* Knepley, Matthew G (VISIT) <[email protected]>; Barry Smith < > [email protected]> > *Cc:* petsc-dev <[email protected]>; Zhang, Hong <[email protected]>; > Betrie, Getnet <[email protected]> > *Subject:* Re: DMPLEX cannot support two different edges for the same two > vertices, hence DMPLEX cannot? > > > “You can certainly have many fields on a given edge, but I don't know > what it would mean to have two edges since no topological query could tell > the difference.” > > > > The two edges in a power grid represent two parallel power lines that are > connected between two locations (vertices). There are line ids (stored in > the component data) to distinguish the two lines. > > > > Thanks, > > Shri > > > > > > *From: *Matthew Knepley <[email protected]> > *Date: *Wednesday, December 1, 2021 at 4:07 PM > *To: *Barry Smith <[email protected]> > *Cc: *"Abhyankar, Shrirang G" <[email protected]>, PETSc > Development <[email protected]>, "Zhang, Hong" <[email protected]>, > Getnet Betrie <[email protected]> > *Subject: *Re: DMPLEX cannot support two different edges for the same two > vertices, hence DMPLEX cannot? > > > > On Wed, Dec 1, 2021 at 4:51 PM Barry Smith <[email protected]> wrote: > > > > It may not happen with one rank. Or it may. > > > > With more than one rank one may run a code and think it is working > correctly but that does not mean the PetscSF is correct. It is generating > "extra" degrees of freedom for these vertices each associated with a > different edge and not properly connecting them. You can tell by doing a > full DMView on the dmplex and PetscSF and seeing more roots than there > should be listed. Hence the numerics will be wrong even though it does not > crash or complain. > > > > Okay, I think we need specific tests to see what is going on. > > > > Theoretically, having multiple edges between vertices does not make sense > since the edge _means_ (v_i, v_j) topologically. You can certainly have > many fields on a given edge, but I don't know what it would mean to have > two edges since no topological query could tell the difference. > > > > However, you can certainly tell Plex anything you want to tell it. > DMPlexBuildFromCellListParallel() gets only cells and vertices, so you are > saying that there is a duplicate cell. That is allowed and should work > properly, EXCEPT if you wanted to identify a cell only by its cone (then > you could not tell them apart). I do this for faces, but I did not think I > ever did it for cells. > > > > Thanks, > > > > Matt > > > > On Dec 1, 2021, at 4:37 PM, Abhyankar, Shrirang G < > [email protected]> wrote: > > > > Barry, > > > > “Is there anything we can do to support having multiple edges between the > same two vertices” > > > > Some of my power grid datasets have multiple edges between the same two > vertices and I’ve not faced an issue with DMNetwork. However, all the data > was read on rank 0 only (and then distributed). > > > > Maybe the issue is with the edges being passed on different ranks? > > > > Thanks, > > Shri > > *From: *Barry Smith <[email protected]> > *Date: *Wednesday, December 1, 2021 at 3:19 PM > *To: *"Knepley, Matthew G (VISIT)" <[email protected]>, PETSc Development > <[email protected]>, "Abhyankar, Shrirang G" < > [email protected]>, "Zhang, Hong" <[email protected]>, Getnet > Betrie <[email protected]> > *Subject: *DMPLEX cannot support two different edges for the same two > vertices, hence DMPLEX cannot? > > > > Check twice before you click! This email originated from outside PNNL. > > > > > > Matt, > > > > If DMPlexBuildFromCellListParallel() is called with two edges that > have the same two vertices what will happen? It looks like it ends up with > an incorrect PetscSF if the two edges are passed on different ranks. Hence > the DMPLEX is not valid and produces garbage. > > > > Neurons can be connected to themselves which seems to be breaking > DMPLEX and hence DMNETWORK. > > > > Is there anything we can do to support having multiple edges between > the same two vertices? If not is there a way we can have > DMPlexBuildFromCellListParallel() generate an error automatically if there > are such extra edges in the input data. > > > > Thanks > > > > Barry > > > > In this work, the neurons are represented by vertices in the network and > each synapse is a graph edge. > > > > > > > -- > > What most experimenters take for granted before they begin their > experiments is infinitely more interesting than any results to which their > experiments lead. > -- Norbert Wiener > > > > https://www.cse.buffalo.edu/~knepley/ > <https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.cse.buffalo.edu%2F~knepley%2F&data=04%7C01%7Cshrirang.abhyankar%40pnnl.gov%7Ce56f4d6ffa10474f3c7508d9b516feb5%7Cd6faa5f90ae240338c0130048a38deeb%7C0%7C0%7C637739932634554446%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=WwSfLYl2hPyHwrfkJ1SuBmp6l88d6xJulwBPBvu%2BbK0%3D&reserved=0> > -- What most experimenters take for granted before they begin their experiments is infinitely more interesting than any results to which their experiments lead. -- Norbert Wiener https://www.cse.buffalo.edu/~knepley/ <http://www.cse.buffalo.edu/~knepley/>
