cc'ing Rich and petsc-users. * I tried putting "petsc example stiffness matrix" into ChatGPT and it actually looked fine, 1D Laplacian C code with instructions to build and run it.
* But we have many tutorials that do this and you can browse them to find one that looks best for your interests at https://urldefense.us/v3/__https://petsc.org/release/tutorials/__;!!G_uCfscf7eWS!fNXzXlpQ-ukRC2r1EAR0vzOZCP4M_UUNOIfBsJXXEMXS3FrkItd2r4hJKy94MHlF6h7APDqb7-Pfc9V4GdZtUls$ Good luck, Mark On Fri, Feb 14, 2025 at 8:05 AM Debiprasad Panda <dpand...@yahoo.com> wrote: > Mark, > > Can you send me any link for FEA example using PETSC which will generate > the stiffness matrix so that I can play with you for porting into our FPGA. > Regards. > > Debiprasad Panda, PhD > President & CTO, > Universal Real Time Power Conversion LLC > Greater Milwaukee, WI > Tel:1-440-840-3393 (cell) > Web: > https://urldefense.us/v3/__https://urtpc.com__;!!G_uCfscf7eWS!fNXzXlpQ-ukRC2r1EAR0vzOZCP4M_UUNOIfBsJXXEMXS3FrkItd2r4hJKy94MHlF6h7APDqb7-Pfc9V435nHato$ > > > NOTICE OF PROPRIETARY AND CONFIDENTIAL INFORMATION*:* > *This e-mail transmission and its attachments are privileged, proprietary > and confidential and is **for the review of the designated recipient only**. > If you have received this transmission in error, please immediately > notify dpand...@yahoo.com <r_satp...@yahoo.com>. Unintended transmission > shall not constitute waiver of any privilege.* > > > On Wednesday, February 12, 2025 at 03:14:51 PM CST, Debiprasad Panda < > dpand...@yahoo.com> wrote: > > > Richard, > > Thanks for the detailed email. It certainly explains the limitation at > this time. Your email also very clearly explains about what kind of > collaboration you extend to the partnering company. I had in my mind that > neither you or Mark or Todd will write the code for us. I thought there > might be junior scientists/programmers who works in your team will do the > bulk of the work under your supervision. Now I understand that you would > like to participate only in specific issues which is not readily available > or needs further development in PETSc. As you say some issues may arise > while using it and if so, you would like to participate in resolving such > issues either in a future DOE proposal or through a self-generated project > by PETSc community. Correct me if my understanding is not right. We did > work with university and consultants as sub-contractor in the past from > this organization, but not directly with research labs. Your email > certainly provides some guideline on the process and timeline. > > As you know we did implement a complete FEA analysis in FPGA and the speed > up is significant. However, that was partly hardcoded. Thats why looking > for an interface which is already tested and just need to be streamlined > with our workflow. I thought that having a complete example of our interest > in PETSC and implementing the same by part/full in FPGA will give us a good > handle to continue development in that direction. As I mentioned we can do > that task ourselves - we do have people who used the same workflow as I > provided in my email, but it was for a different application. The main > problem for small business like us is lack of funding. An SBIR/STTR funding > will be very helpful ton accomplish this ground research on FPGA PETSC > interface. > > I know time is short and certainly this transition time is making things > more complicated. > > Let's plan for the next round and I believe the solicitation will be out > in first week of June and the submission of the final proposal will be in > October 2025. I will contact you in June. > > > Anyway, we will proceed with our proposal with another partner this time. > > In the mean it will be helpful if Adam or any of you can send a link to > any existing FEA example so that we can play with it. > > Thanks again for all your time and email discussion. > > > > Regards. > > > Debiprasad Panda, PhD > President & CTO, > Universal Real Time Power Conversion LLC > Greater Milwaukee, WI > Tel:1-440-840-3393 (cell) > Web: > https://urldefense.us/v3/__https://urtpc.com__;!!G_uCfscf7eWS!fNXzXlpQ-ukRC2r1EAR0vzOZCP4M_UUNOIfBsJXXEMXS3FrkItd2r4hJKy94MHlF6h7APDqb7-Pfc9V435nHato$ > > > NOTICE OF PROPRIETARY AND CONFIDENTIAL INFORMATION*:* > *This e-mail transmission and its attachments are privileged, proprietary > and confidential and is **for the review of the designated recipient only**. > If you have received this transmission in error, please immediately > notify dpand...@yahoo.com <r_satp...@yahoo.com>. Unintended transmission > shall not constitute waiver of any privilege.* > > > On Wednesday, February 12, 2025 at 02:07:03 PM CST, Mills, Richard Tran < > rtmi...@anl.gov> wrote: > > > Hi Debiprasad, > > I apologize for being slow to get back to you; I am severely > over-committed at the moment, and keeping up with email (among other > things) has been extremely difficult. > > I am sorry to have to disappoint you, but I do not think that it will be > possible for me, Todd, or Mark to partner with you on the STTR call this > time around. Let me try to explain the two major reasons why. > > First: For staff at the DOE National Laboratories, it is very > time-consuming to get approvals to participate as a subcontractor for > something like an SBIR or STTR project. I receive a small amount of funding > from an SBIR project right now, and it literally took weeks to get that > proposal through all of the required approvals, including an "letter of > commitment" signed by the the Laboratory's Director of Sponsored Research, > as well as approval from our Contracting Officer at the DOE Site Office in > Chicago. There are many steps of the review to ensure that proposed work is > consistent with the DOE and Argonne missions, that it does not adversely > impact DOE work at Argonne, and that it is not in direct competition with > the private sector. The laboratory's guidance on this approval process > state that we should allow a minimum of 15 business days for this process, > but, with the current upheaval due to the transition to the new > Administration, I suspect that more than 15 business days would be > required. I also note that a reasonably close to complete draft of the > proposal is required to be submitted at the beginning of the approval > process, so you need to factor in time to develop the proposal ahead of the > approval window if you want to respond to a future STTR or SBIR call and > partner with a DOE Laboratory. > > Second: The breakdown of work that you are proposing isn't really aligned > what with laboratory research scientists like Todd, Mark, and I are > expected to do. We are primarily researchers, and our output is judged > similarly to that of a professor at an R1 university, except that we have > no teaching load and engage in some programmatic work. What you have > proposed is having us develop a complete finite-element analysis code to > some specification you provide, which we will then hand to you (before you > implement part or all of it using FPGAs). For this sort of arrangement, it > sounds like what you are looking for is scientific programmers who work on > contract. That is not the role that we play. We do research on > computational mathematics and its applications, and we develop software to > aid this research and to enable the broader computing community to benefit > from our research and perhaps collaborate with us on further developments. > This has led to a widely-used piece of software, PETSc, which provides > useful computational building blocks that many teams have used to build > finite-element analysis applications, but when teams have used PETSc for > such work and have teamed with us, it has very much been in a collaborative > research relationship: others are doing much of the development of their > FEM code, but we help them because, say, they are modeling systems with > very difficult nonlinearities, discontinuous jumps in material > coefficients, strangely stretched elements, etc., that cause problems for > simple algebraic solvers, so we collaborate with them on developing new > solver techniques that are amenable to their problems. > > It may make sense for you to partner with us or other members of the PETSc > team in the future, but I think you need to take some time to lay more of > the groundwork before a future funding call. You can experiment with > porting a PETSc-based FEM code using your FPGA approach without needing > anything from us right now: There are numerous finite-element example codes > provided with PETSc (Mark has written a few of them, and might be able to > recommend some good ones to start with). You could start by playing with > these examples and then try porting bits of them to FPGAs. As I said in an > earlier message, based on my limited experience with FPGAs, I suspect that > you will run into several technical challenges. When you have had a chance > to identify these challenges, then it might make sense to come back to the > PETSc team to describe some of them — you can start by emailing petsc-maint > or petsc-users about this — and perhaps eventually develop a proposal that > aims to address them in collaboration with the team. > > Apologies if I have had to disappoint you, and best of luck. Perhaps later > there will be good opportunities to partner with us in the future. I > encourage you to experiment some with PETSc to determine whether it is the > right software toolkit to use for your FPGA-targeted applications, and to > not be shy about asking on the PETSc user lists as you uncover issues as > you experiment. > > Best regards, > Richard > > ------------------------------ > *From:* Debiprasad Panda <dpand...@yahoo.com> > *Sent:* Thursday, February 6, 2025 8:18 PM > *To:* Mills, Richard Tran <rtmi...@anl.gov> > *Cc:* Mark Adams <mfad...@lbl.gov>; Munson, Todd <tmun...@mcs.anl.gov> > *Subject:* Re: DOE STTR partner opportunity > > This Message Is From an External Sender > This message came from outside your organization. > > Dear All, > > Amidst all these organizational and administrative changes, I have good > news to share that our LOI has been accepted by DOE and the final proposal > submission is due on 26th February 2025. The proposal is about an FEA > thermal analysis using PETSc and porting it to FPGA for its real time > simulation. > > Given a mechanical drawing of an object, in PETSC a mesh will be generated > and then a thermal problem will be formulated using FEA theory and boundary > condition to generate a global stiffness matrix in the form of Ax =B, which > will be eventually solved using linear or non-linear solver. In Phase I, we > will concentrate only on linear system and only the solver part will be > implemented in FPGA to demonstrate the real time operation in part. In > Phase II, the entire FEA problem formulation with non-linearity as well as > solver will be implemented in FPGA to have a complete real time solution. > > We went through PETSc libraries and one of our team members has used it > extensively during his PhD. The steps we would like to follow to formulate > a FEA problem, and its solution is described in the attached document. > > We would like you to partnering with us in this DOE project and your > responsibility will be to create this FEA thermal model in PETSc following > the steps in the given document and then run it in a PC/server and > collect the result. We will take the responsibility of implementing the > same in our FPGA solver. > > I was thinking to write this email for some time but kept on hold till the > formal acceptance of LOI in order to justify your time. > > Please go through the attached document and then let's follow up with a > zoom call sometime early next week per your convenience for discussing it > for any question you may have. > > Please acknowledge receiving this email so that I know our communication > is going through. > > I will look forward to collaborating with you. > > Regards. > > > Debiprasad Panda, PhD > President & CTO, > Universal Real Time Power Conversion LLC > Greater Milwaukee, WI > Tel:1-440-840-3393 (cell) > Web: > https://urldefense.us/v3/__https://urtpc.com__;!!G_uCfscf7eWS!fNXzXlpQ-ukRC2r1EAR0vzOZCP4M_UUNOIfBsJXXEMXS3FrkItd2r4hJKy94MHlF6h7APDqb7-Pfc9V435nHato$ > > <https://urldefense.us/v3/__https://urtpc.com__;!!G_uCfscf7eWS!YKeheulpEvP02Jraqr7SiUTNGtRfqaaLJ7-ibIrQ3HGvPKHIqKfkL8mQ6rfHuR-j4Fra6KhYon67LEc$> > > NOTICE OF PROPRIETARY AND CONFIDENTIAL INFORMATION*:* > *This e-mail transmission and its attachments are privileged, proprietary > and confidential and is **for the review of the designated recipient only**. > If you have received this transmission in error, please immediately notify > **dpand...@yahoo.com > <r_satp...@yahoo.com>**. Unintended transmission shall not constitute > waiver of any privilege.* > > > On Tuesday, January 14, 2025 at 03:20:07 PM CST, Debiprasad Panda < > dpand...@yahoo.com> wrote: > > > Richard, Mark, Todd > > I am submitting the LOI without ANL at this time. It seems we can include > ANL as STTR partner while submitting the full proposal if things look good > from both sides. So, we may have about six weeks from now to understand the > project. Let's discuss it over a zoom call sometime this week. > > Regards. > > Debiprasad Panda, PhD > President & CTO, > Universal Real Time Power Conversion LLC > Greater Milwaukee, WI > Tel:1-440-840-3393 (cell) > Web: > https://urldefense.us/v3/__https://urtpc.com__;!!G_uCfscf7eWS!fNXzXlpQ-ukRC2r1EAR0vzOZCP4M_UUNOIfBsJXXEMXS3FrkItd2r4hJKy94MHlF6h7APDqb7-Pfc9V435nHato$ > > <https://urldefense.us/v3/__https://urtpc.com__;!!G_uCfscf7eWS!YKeheulpEvP02Jraqr7SiUTNGtRfqaaLJ7-ibIrQ3HGvPKHIqKfkL8mQ6rfHuR-j4Fra6KhYon67LEc$> > > NOTICE OF PROPRIETARY AND CONFIDENTIAL INFORMATION*:* > *This e-mail transmission and its attachments are privileged, proprietary > and confidential and is **for the review of the designated recipient only**. > If you have received this transmission in error, please immediately notify > **dpand...@yahoo.com > <r_satp...@yahoo.com>**. Unintended transmission shall not constitute > waiver of any privilege.* > > > On Tuesday, January 14, 2025 at 12:50:53 PM CST, Mills, Richard Tran < > rtmi...@anl.gov> wrote: > > > Hi Debiprasad, > > Apologies for the delay in my reply; the past few days have been > especially busy ones due to some internal proposal deadlines I had to rush > to meet, on top of several other things. > > Your project sounds interesting, but, unfortunately, I don't think that > there is time before your LOI is due for me to understand your application, > discuss whether PETSc is appropriate for it, or how you would map any > implementation using PETSc to FPGA hardware. PETSc is an extremely > complicated piece of software and a lot of effort is required from > algorithm selection and parallel problem decomposition on down to details > of individual microkernels when bringing it to and optimizing it for new > kinds of computing architectures. (I spent roughly six years working with > several others on getting solid GPU support in PETSc, for instance.) I have > a little bit of familiarity with FPGAs from my time at ORNL and Intel, and > I think that enabling PETSc to make efficient use of FPGAs is going to be a > highly non-trivial (though interesting!) project. Are you familiar at all > with PETSc, and do you have a particular reason that you think it would be > helpful to your work? You might be better served by using a different piece > of software as a starting point, if you do not need things like the > distributed memory-parallel implementations or the advanced, composable > solvers and preconditioners. If you do have a particular need for things > that PETSc provides, perhaps I or others from the PETSc team could discuss > this with you with future opportunities in mind. Best of luck to you if you > do submit an STTR proposal this time. > > Sincerely, > Richard > > ------------------------------ > *From:* Debiprasad Panda <dpand...@yahoo.com> > *Sent:* Tuesday, January 14, 2025 6:09 AM > *To:* Mills, Richard Tran <rtmi...@anl.gov> > *Subject:* Re: DOE STTR partner opportunity > > This Message Is From an External Sender > This message came from outside your organization. > > Richard, > Hope you received my previous email. I will appreciate if you let me know > if you would like to participate in this STTR project or not. I know its a > short notice and I will understand if that is not sufficient to make it a > "GO". > > I will still have good amount time to create and upload an LOI. > > Regards. > > Debiprasad Panda, PhD > President & CTO, > Universal Real Time Power Conversion LLC > Greater Milwaukee, WI > Tel:1-440-840-3393 (cell) > Web: > https://urldefense.us/v3/__https://urtpc.com__;!!G_uCfscf7eWS!fNXzXlpQ-ukRC2r1EAR0vzOZCP4M_UUNOIfBsJXXEMXS3FrkItd2r4hJKy94MHlF6h7APDqb7-Pfc9V435nHato$ > > <https://urldefense.us/v3/__https://urtpc.com__;!!G_uCfscf7eWS!Y1KB89LXCkt6T8Z7n9PMvin_XYuQGUOsbWYpr4EbU3SJkMiBRE8VUOVlOIxbu8ETP36hdk0DEvJZnRc$> > > NOTICE OF PROPRIETARY AND CONFIDENTIAL INFORMATION*:* > *This e-mail transmission and its attachments are privileged, proprietary > and confidential and is **for the review of the designated recipient only**. > If you have received this transmission in error, please immediately notify > **dpand...@yahoo.com > <r_satp...@yahoo.com>**. Unintended transmission shall not constitute > waiver of any privilege.* > > > On Monday, January 13, 2025 at 01:44:59 PM CST, Debiprasad Panda < > dpand...@yahoo.com> wrote: > > > Richard, > I got your contact from Todd Munson. We are a small business located in > greater Milwaukee and working on a one-stop real time simulator where we > can simulate a large grid along with IBR in real time. In addition, we can > conduct a thermal and structural FEA analysis in real time for up to 1-5 > Million grid points. > A new DOE solicitation is out where we can propose a one stop solution for > solar power IBR where we can model an IBR with very low step size (20-40ns) > for its real time simulation and also, we can calculate thermal loss > through semiconductor switches and then provide a thermal footprint of the > IBR in real time employing a FEA analysis. We have implemented a thermal > analysis of a heat sink using our proprietary FPGA implementation in real > time with 52000 nodes and can extend it upto 1-5M. I am wondering if you > would like to take part as RI for our STTR application where you can > formulate the FEA problem using PETSC or any other software and then we can > implement the same in FPGA for its real time implementation. If so, let me > know by COB today. We do not have much time - the LOI is due tomorrow 4:00 > PM central time, and the full proposal is due on 26th February. If you > would like we can have a quick call to discuss. At this time an email > consent will be fine and then we can discuss the detailed scopes and > deliverable in next couple of weeks. The STTR > > Let me know if you will be interested. > > > Regards. > > Debiprasad Panda, PhD > President & CTO, > Universal Real Time Power Conversion LLC > Greater Milwaukee, WI > Tel:1-440-840-3393 (cell) > Web: > https://urldefense.us/v3/__https://urtpc.com__;!!G_uCfscf7eWS!fNXzXlpQ-ukRC2r1EAR0vzOZCP4M_UUNOIfBsJXXEMXS3FrkItd2r4hJKy94MHlF6h7APDqb7-Pfc9V435nHato$ > > <https://urldefense.us/v3/__https://urtpc.com__;!!G_uCfscf7eWS!Y1KB89LXCkt6T8Z7n9PMvin_XYuQGUOsbWYpr4EbU3SJkMiBRE8VUOVlOIxbu8ETP36hdk0DEvJZnRc$> > > NOTICE OF PROPRIETARY AND CONFIDENTIAL INFORMATION*:* > *This e-mail transmission and its attachments are privileged, proprietary > and confidential and is **for the review of the designated recipient only**. > If you have received this transmission in error, please immediately notify > **dpand...@yahoo.com > <r_satp...@yahoo.com>* > *. Unintended transmission shall not constitute waiver of any privilege. * >
