> From: Jerin Jacob [mailto:jerinjac...@gmail.com] > Sent: Wednesday, 17 August 2022 07.37 > > On Tue, Aug 16, 2022 at 9:15 PM Morten Brørup > <m...@smartsharesystems.com> wrote: > > > > > From: Jerin Jacob [mailto:jerinjac...@gmail.com] > > > Sent: Tuesday, 16 August 2022 15.13 > > > > > > On Wed, Aug 3, 2022 at 8:49 PM Stephen Hemminger > > > <step...@networkplumber.org> wrote: > > > > > > > > On Wed, 3 Aug 2022 18:58:37 +0530 > > > > <jer...@marvell.com> wrote: > > > > > > > > > Roadmap > > > > > ------- > > > > > 1) Address the comments for this RFC. > > > > > 2) Common code for mldev > > > > > 3) SW mldev driver based on TVM (https://tvm.apache.org/) > > > > > > > > Having a SW implementation is important because then it can be > > > covered > > > > by tests. > > > > > > Yes. That reason for adding TVM based SW driver as item (3). > > > > > > Is there any other high level or API level comments before > proceeding > > > with v1 and implementation. > > > > Have you seriously considered if the DPDK Project is the best home > for this project? I can easily imagine the DPDK development process > being a hindrance in many aspects for an evolving AI/ML library. Off > the top of my head, it would probably be better off as a separate > project, like SPDK. > > Yes. The reasons are following > > # AI/ML compiler libraries more focused on model creation and > training etc (Thats where actual value addition the AI/ML libraries > can offer) and minimal part for interference(It is just added for > testing the model) > # Considering the inference is the scope of the DPDK. DPDK is ideal > place for following reasons > > a) Inference scope is very limited. > b) Avoid memcpy of interference data (Use directly from network or > other class of device like cryptodev, regexdev) > c) Reuse highspeed IO interface like PCI backed driver etc > d) Integration with other DPDK subsystems like eventdev etc for job > completion. > e) Also support more inline offloads by merging two device classes > like rte_secuity. > f) Run the inference model from different AI/ML compiler frameworks or > abstract the inference usage. > Similar concept is already applied to other DPDK device classes like > 1) In Regexdev, The compiler generates the rule database which is out > of scope of DPDK. DPDK API just loads the rule database > 2) In Gpudev, The GPU kernel etc out of scope of DPDK.DPDK cares about > IO interface.
Thank you for the detailed reply, Jerin. These are good reasons for adding the new device class to the DPDK project - especially the Regexdev comparison got me convinced. > > > If all this stuff can be completely omitted at build time, I have no > objections. > > Yes, It can be completely omitted at build time. Perfect. > Also no plan to > integrate to testpmd and other existing application. Planning to add > only app/test-mldev application. +1 to that > > > > > A small note about naming (not intending to start a flame war, so > please feel free to ignore!): I haven't worked seriously with ML/AI > since university three decades ago, so I'm quite rusty in the domain. > However, I don't see any Machine Learning functions proposed by this > API. The library provides an API to an Inference Engine - but nobody > says the inference model stems from Machine Learning; it might as well > be a hand crafted model. Do you plan to propose APIs for training the > models? If not, the name of the library could confuse some potential > users. > > No, scope is only inference and it is documented in the programing > guide and API header file. I am trying to keep name similar to > regexdev, gpudev etc which have similar scope. But I am open to other > shortname/name if you have something in mind. The AI(Artificial Intelligence)/ML(Machine Learning)/IE(Inference Engine) chip market still seems immature and fragmented, so I can't find any consensus on generic names for such hardware accelerator devices. Some of the chip vendors represented on the DPDK mailing list offer AI/ML/IE accelerator chips. Perhaps their marketing department could propose alternatives to "Machine Learning Device"/"mldev" for inference engine devices (with no acceleration for training the models). If not, the initially proposed name is good enough. So: Everyone ask your marketing departments and speak up now, or the name "mldev" will be set in stone. ;-) I'm thinking: While "Inference Engine Device"/iedev might be technically more correct, it doesn't have same value as "Machine Learning Device"/"mldev" on a marketing scale. And we should choose a name that we expect might become industry standard consensus. > > > > > > Or Anyone else interested to review or contribute to this new DPDK > > > device class? > >
