RE: [EXTERNAL] Re: When the trace buffers are saved to disk?

[Jerin Jacob]

> -Original Message-
> From: Adel Belkhiri 
> Sent: Saturday, November 30, 2024 2:55 AM
> To: Thomas Monjalon 
> Cc: dev@dpdk.org; Jerin Jacob ; Sunil Kumar Kori
> 
> Subject: [EXTERNAL] Re: When the trace buffers are saved to disk?
> 
> Thank you for your answer. On Fri, Nov 29, 2024 at 4: 23 PM Thomas Monjalon
>  wrote: 29/11/2024 21: 39, Adel Belkhiri: > Thank you
> for the clarification, Thomas. Indeed, the documentation for the > trace 
> library
> 
> Thank you for your answer.
> 
> On Fri, Nov 29, 2024 at 4:23 PM Thomas Monjalon   > wrote:
> 
> 
>   29/11/2024 21:39, Adel Belkhiri:
>   > Thank you for the clarification, Thomas. Indeed, the documentation
> for the
>   > trace library is kind of limited. If you don't mind, I have another
>   > question: Would it be useful to have an API to register a callback (to
> save
>   > trace data) when the buffer is full?
> 
>   I suppose yes, the problem being which thread is running file writing.

Each thread has separate channels i.e  separate memory. it is OK.

Main problem will be it can stall trace emission (e.s.p in fastpath) as these 
medium
(storage or network) will be slow to transfer.




> 
>   I leave it to the maintainers of the trace library.
> 
> 
>   > On Fri, Nov 29, 2024 at 6:44 AM Thomas Monjalon
> mailto:tho...@monjalon.net> > wrote:
>   > > 28/11/2024 20:17, Adel Belkhiri:
>   > > > Hi all,
>   > > >
>   > > > Recently, while tracing applications from the apps and examples
>   > > > directories, I became confused about when the trace buffer is
> written to
>   > > > disk. Is the trace data saved only when rte_save_trace() is 
> called,
> or
>   > > does
>   > >
>   > > It is rte_trace_save()
>   > >
>   > > > it also automatically save when the buffer becomes full?
>   > >
>   > > No, DPDK is not doing such thing without user agreement.
>   > >
>   > > > From my understanding, rte_save_trace() is invoked when the
> application
>   > > > executes rte_eal_cleanup(). Does this mean the target application
> needs
>   > > to
>   > > > explicitly support tracing by calling rte_save_trace()—perhaps at
> regular
>   > > > intervals—to dump the trace buffer to disk? Otherwise, will we
> only get a
>   > > > fragment of the trace saved during rte_eal_cleanup() execution?
>   > >
>   > > Yes you get it right.
>   > >
>   > > > Thank you for clarifying this point.
>   > >
>   > > Thanks for asking.
>   > >
>   > > If you think the doc below is not clear enough,
>   > > do not hesitate to submit a patch to make the doc better:
>   > >
>   > > https://doc.dpdk.org/guides/prog_guide/trace_lib.html
>  3A__doc.dpdk.org_guides_prog-5Fguide_trace-
> 5Flib.html&d=DwMFaQ&c=nKjWec2b6R0mOyPaz7xtfQ&r=1DGob4H4rxz6H8uIT
> ozGOCa0s5f4wCNtTa4UUKvcsvI&m=PK0ws3Dbdr_mMLx2eKS2mDDFfgTq6zBtfr
> VM5C6twScLE3HMylemVgzhtcl_YzoO&s=E7gtIuTFNmRwZ_mZBpAI9PjOKZYpvW
> Mytuj_TdSkLvE&e=>
> 
> 
> 
> 

RE: [PATCH] power/amd_uncore: add e-smi installation instructions

[Tummala, Sivaprasad]

[AMD Official Use Only - AMD Internal Distribution Only]

Hi Thomas,

> -Original Message-
> From: Thomas Monjalon 
> Sent: Friday, November 29, 2024 4:04 PM
> To: Yigit, Ferruh ; Tummala, Sivaprasad
> 
> Cc: david.h...@intel.com; anatoly.bura...@intel.com; jer...@marvell.com;
> radu.nico...@intel.com; gak...@marvell.com; cristian.dumitre...@intel.com;
> konstantin.anan...@huawei.com; dev@dpdk.org
> Subject: Re: [PATCH] power/amd_uncore: add e-smi installation instructions
>
> Caution: This message originated from an External Source. Use proper caution
> when opening attachments, clicking links, or responding.
>
>
> 29/11/2024 05:32, Sivaprasad Tummala:
> > Added section for installing and building the E-SMI library for AMD
> > EPYC Uncore support and version requirements.
> >
> > Signed-off-by: Sivaprasad Tummala 
> > ---
> >  doc/guides/prog_guide/power_man.rst | 32
> > +
> >  1 file changed, 32 insertions(+)
> >
> > diff --git a/doc/guides/prog_guide/power_man.rst
> > b/doc/guides/prog_guide/power_man.rst
> > index 74039e5786..d367a81596 100644
> > --- a/doc/guides/prog_guide/power_man.rst
> > +++ b/doc/guides/prog_guide/power_man.rst
> > @@ -249,6 +249,38 @@ Additionally, the EPYC System Management
> > Interface In-band Library for Linux  offers essential API, enabling
> > user-space software  to effectively manage system functions.
> >
> > +E-SMI Installation
> > +--
> > +
> > +To build DPDK with AMD EPYC Uncore the user is required to download
> > +the e-smi library from `here
> > +`_
> > +and compile it on their user system before building DPDK.
> > +
> > +.. code-block:: console
> > +
> > +cd esmi_ib_library
> > +cmake .
> > +sudo make install
> > +cp /opt/e-sms/e_smi/lib/* /usr/local/lib/*
> > +cp /opt/e-sms/e_smi/include/* /usr/local/include/*
>
> Why copying the files?
> You can specify the directories in an environment variable.
> There is no pkg-config file?
Currently, there isn't a pkg-config file available.
However, we are actively working on adding support for it in an upcoming 
release.
>
> > +
> > +Library file, header and tool are installed at /opt/e-sms.
> > +
> > +Note: Library is dependent on amd_hsmp.h header and without this, 
> > compilation
> will break.
>
> Yes, it does not compile on my machine (with Linux 6.12):
>
> e_smi.c:566:27: error: ‘HSMP_GET_RAPL_UNITS’ undeclared
>
> What should I do?
It appears there is a compatibility issue with the current version of the 
amd_hsmp.h header file. https://github.com/amd/amd_hsmp/blob/master/amd_hsmp.h
To resolve this, could you please try replacing the existing version with an 
older amd_hsmp.h header file?
This should allow the build process to work with the older kernels.
>
> > +
> > +The library requires CMake (v3.5.0) to be built.
> > +
> > +As a reference, the following table shows a mapping between the DPDK
> > +versions and the E-SMI library and kernel version supported by them:
> > +
> > +.. table:: DPDK and E-SMI library and kernel version compatibility
> > +
> > +   ==  ==   =
> > +   DPDK versionE-SMI versionLinux Kernel version
> > +   ==  ==   =
> > +24.11+  4.0.06.7+
> > +   ==  ==   =
>
>

Re: [PATCH] power/amd_uncore: add e-smi installation instructions

[Thomas Monjalon]

01/12/2024 05:55, Tummala, Sivaprasad:
> [AMD Official Use Only - AMD Internal Distribution Only]

Please remove this.

> Hi Thomas,
> 
> From: Thomas Monjalon 
> > 29/11/2024 05:32, Sivaprasad Tummala:
> > > Added section for installing and building the E-SMI library for AMD
> > > EPYC Uncore support and version requirements.
> > >
> > > Signed-off-by: Sivaprasad Tummala 
> > > ---
> > >  doc/guides/prog_guide/power_man.rst | 32
> > > +
> > >  1 file changed, 32 insertions(+)
> > >
> > > diff --git a/doc/guides/prog_guide/power_man.rst
> > > b/doc/guides/prog_guide/power_man.rst
> > > index 74039e5786..d367a81596 100644
> > > --- a/doc/guides/prog_guide/power_man.rst
> > > +++ b/doc/guides/prog_guide/power_man.rst
> > > @@ -249,6 +249,38 @@ Additionally, the EPYC System Management
> > > Interface In-band Library for Linux  offers essential API, enabling
> > > user-space software  to effectively manage system functions.
> > >
> > > +E-SMI Installation
> > > +--
> > > +
> > > +To build DPDK with AMD EPYC Uncore the user is required to download
> > > +the e-smi library from `here
> > > +`_
> > > +and compile it on their user system before building DPDK.
> > > +
> > > +.. code-block:: console
> > > +
> > > +cd esmi_ib_library
> > > +cmake .
> > > +sudo make install
> > > +cp /opt/e-sms/e_smi/lib/* /usr/local/lib/*
> > > +cp /opt/e-sms/e_smi/include/* /usr/local/include/*
> >
> > Why copying the files?
> > You can specify the directories in an environment variable.
> > There is no pkg-config file?
> Currently, there isn't a pkg-config file available.
> However, we are actively working on adding support for it in an upcoming 
> release.
> >
> > > +
> > > +Library file, header and tool are installed at /opt/e-sms.
> > > +
> > > +Note: Library is dependent on amd_hsmp.h header and without this, 
> > > compilation
> > will break.
> >
> > Yes, it does not compile on my machine (with Linux 6.12):
> >
> > e_smi.c:566:27: error: ‘HSMP_GET_RAPL_UNITS’ undeclared
> >
> > What should I do?
> It appears there is a compatibility issue with the current version of the 
> amd_hsmp.h header file. https://github.com/amd/amd_hsmp/blob/master/amd_hsmp.h
> To resolve this, could you please try replacing the existing version with an 
> older amd_hsmp.h header file?
> This should allow the build process to work with the older kernels.

No I will wait for the library to be fixed.

It doesn't look mature enough, this support should not have been merged.

Re: [PATCH v2] devtools: fix false positive from checkpatch

[Thomas Monjalon]

17/09/2024 18:56, Stephen Hemminger:
> The codespell dictionary has "stdio" in its bad word list,
> but stdio.h is often used in DPDK code and creates bogus false
> positives from checkpatch.
> 
> Also add a check to the parameters to build-dict.sh so it
> gives usage error if parameter is missing or does not point
> to the codespell git clone.
> 
> Signed-off-by: Stephen Hemminger 

Redirected error message to stderr,
and saved the dict path in a variable to avoid redundancy,
and applied, thanks.

Re: [PATCH 0/2] minor changes in script used for backports

[Thomas Monjalon]

28/11/2023 14:23, Thomas Monjalon:
> These patches have been forgotten.
> Adding more Cc.
> 
> 18/04/2023 16:07, Thomas Monjalon:
> > While looking again at git-log-fixes.sh,
> > I've found a minor bug and a small improvement.
> > 
> > Thomas Monjalon (2):
> >   devtools: fix check of multiple commits fixed at once
> >   devtools: deduplicate function to mark fixes

Acked-by: Luca Boccassi 
Acked-by: Kevin Traynor 

Applied

Re: [PATCH v1] doc: update release notes for 24.11

[Thomas Monjalon]

28/11/2024 18:07, John McNamara:
> -  * ``rte_bit_atomic_*`` provide atomic bit-level operations, including
> +  * ``rte_bit_atomic_*`` provides atomic bit-level operations including

With wildcard it can be considered as a plural I suppose.

> -  to allow out-of-tree driver to manage RVU LF device.
> +  to allow ans out-of-tree driver to manage a RVU LF device.

fixed ans -> an

> -  or specifically in a different order than dequeue.
> +  or specifically in a different order to dequeue.

I think it is really different "than" dequeue order.

> -  Added ability for node to advertise and update multiple xstat counters,
> +  Added ability for ans node to advertise and update multiple xstat counters,

fixed ans -> a

I've also removed the comments in the file.

Applied, thanks.

Re: [PATCH v3] doc: add security document

[Thomas Monjalon]

20/11/2024 04:12, Nandini Persad:
> This is a new document covering security protocols
> implemented in DPDK.
> 
> Signed-off-by: Nandini Persad 
> Signed-off-by: Thomas Monjalon 
> Reviewed-by: Stephen Hemminger 

Moved to howto section where the usage of other libs and drivers are explained.
Added to MAINTAINERS.
Linked from security and cryptodev guides.
Added link to the cryptodev doc.
Reworded other related links.
Rewrapped lines logically.
Added periods consistently.
Reformatted lists.
Fixed some typos.
Removed application considerations which don't apply to DPDK.
Nested TLS sub-sections.
Added more drivers.

Applied, thanks.

Re: [PATCH] doc: remove confusing command to send patch

[Thomas Monjalon]

10/10/2023 18:26, Thomas Monjalon:
> In the contributor guide, it was said that no need to Cc maintainers
> for new additions, probably for new directories not having a maintainer.
> There is no harm, and it is a good habit, to always Cc maintainers.
> 
> Remove this case as it can mislead to not Cc maintainers when needed.
> 
> Signed-off-by: Thomas Monjalon 

There was a discussion about this patch but no better solution,
so applied to avoid confusion.

DPDK 24.11 released

[Thomas Monjalon]

A new major release is available:
https://fast.dpdk.org/rel/dpdk-24.11.tar.xz

It was a busy release cycle:
1329 commits from 196 authors
2557 files changed, 376587 insertions(+), 177108 deletions(-)

And it includes some API/ABI compatibility breakages.
This release won't be ABI-compatible with previous ones.
The new major ABI version is 25.
The next releases 25.03 and 25.07 will be ABI-compatible with 24.11.

The branch 24.11 should be supported for three years,
making it recommended for system integration and deployment.

Highlights of 24.11:
- lcore variables allocation
- bit set and atomic bit manipulation
- AMD uncore power management
- per-CPU power management QoS for resume latency
- IPv6 address API
- RSS hash key generation
- Ethernet link lanes
- flow table index action
- Cisco enic VF
- Marvell CN20K
- Napatech ntnic flow engine
- Realtek r8169 driver
- ZTE gdtc / zxdh driver initialization
- symmetric crypto SM4
- asymmetric crypto EdDSA
- event device pre-scheduling
- event device independent enqueue
- logging rework (timestamp, color, syslog, journal)

More details in the release notes:
https://doc.dpdk.org/guides/rel_notes/release_24_11.html


There are 50 new contributors (including authors, reviewers and testers).
Welcome to Adel Belkhiri, Ahmed Zaki, Andre Muezerie, Andrzej Wilczynski,
Bartosz Jakub Rosadzinski, Bill Xiang, Chenxingyu Wang,
Danylo Vodopianov,  Dhruv Tripathi, Doug Foster, Gur Stavi, Hanxiao Li,
Howard Wang, Huaxing Zhu, Julien Hascoet, Jun Zhang, Junlong Wang,
Kiran Kumar Kokkilagadda, Luka Jankovic, Lukas Sismis, Lukasz Cieplicki,
Malcolm Bumgardner, Mateusz Polchlopek, Michal Jaron, Michal Nowak,
Midde Ajijur Rehaman, Mihai Brodschi, Niall Meade, Norbert Zulinski,
Ofer Dagan, Oleg Akhrem, Oleksandr Nahnybida, Peter Morrow,
Praveen Kaligineedi, Przemyslaw Gierszynski, Rogelio Domínguez Hernández,
Sangtani Parag Satishbhai, Slawomir Laba, Stefan Laesser,
Sudheer Mogilappagari, Thomas Wilks, Tim Martin, Tomáš Ďurovec,
Varun Lakkur Ambaji Rao, Vasuthevan Maheswaran, Vinod Krishna,
Wojciech Panfil, Xinying Yu, Yogesh Bhosale, and Yong Zhang.

Below is the number of commits per employer (with authors count):
221 Intel (60)
158 Marvell (22)
137 Napatech (3)
107 step...@networkplumber.org (1)
 96 NVIDIA (15)
 93 NXP (10)
 85 Red Hat (3)
 70 Corigine (9)
 70 Broadcom (15)
 54 Huawei (6)
 34 Arm (5)
 30 Ericsson (1)
...

A big thank to all courageous people who took on the non rewarding task
of reviewing other's job.
Based on Reviewed-by and Acked-by tags, the top non-PMD reviewers are:
 73 Morten Brørup 
 66 Stephen Hemminger 
 57 Chengwen Feng 
 41 Bruce Richardson 
 31 Luca Vizzarro 
 26 Ferruh Yigit 
 26 David Marchand 


The next version will be 25.03 in March.
The new features for 25.03 can be submitted during December:
http://core.dpdk.org/roadmap#dates
Please share your roadmap.


Thanks everyone

22.11.7 patches review and test

[luca . boccassi]

Hi all,

Here is a list of patches targeted for stable release 22.11.7.

The planned date for the final release is December 17th.

Please help with testing and validation of your use cases and report
any issues/results with reply-all to this mail. For the final release
the fixes and reported validations will be added to the release notes.

A release candidate tarball can be found at:

https://dpdk.org/browse/dpdk-stable/tag/?id=v22.11.7-rc1

These patches are located at branch 22.11 of dpdk-stable repo:
https://dpdk.org/browse/dpdk-stable/

Thanks.

Luca Boccassi

---
Ajit Khaparde (1):
  net/bnxt: fix TCP and UDP checksum flags

Alan Elder (1):
  net/netvsc: fix using Tx queue higher than Rx queues

Aleksandr Loktionov (1):
  net/i40e/base: fix misleading debug logs and comments

Alexander Kozyrev (3):
  common/mlx5: fix error CQE handling for 128 bytes CQE
  net/mlx5: fix shared queue port number in vector Rx
  net/mlx5: fix miniCQEs number calculation

Anatoly Burakov (1):
  net/i40e/base: fix setting flags in init function

Arkadiusz Kusztal (1):
  crypto/qat: fix modexp/inv length

Artur Tyminski (1):
  net/i40e/base: fix DDP loading with reserved track ID

Barbara Skobiej (3):
  net/ixgbe/base: fix unchecked return value
  net/i40e/base: fix unchecked return value
  net/i40e/base: fix loop bounds

Bill Xiang (2):
  vhost: fix offset while mapping log base address
  vdpa: update used flags in used ring relay

Bing Zhao (3):
  net/mlx5: workaround list management of Rx queue control
  net/mlx5: fix default RSS flows creation order
  net/mlx5: fix Rx queue reference count in flushing flows

Brian Dooley (1):
  test/crypto: fix synchronous API calls

Bruce Richardson (10):
  eal/x86: fix 32-bit write combining store
  net/i40e: fix AVX-512 pointer copy on 32-bit
  net/ice: fix AVX-512 pointer copy on 32-bit
  net/iavf: fix AVX-512 pointer copy on 32-bit
  build: remove version check on compiler links function
  net/idpf: fix AVX-512 pointer copy on 32-bit
  net/ice: detect stopping a flow director queue twice
  app/dumpcap: remove unused struct array
  eventdev: fix possible array underflow/overflow
  net/iavf: add segment-length check to Tx prep

Chaoyong He (1):
  net/nfp: fix link change return value

Chengwen Feng (6):
  examples/eventdev: fix queue crash with generic pipeline
  ethdev: verify queue ID in Tx done cleanup
  net/hns3: verify reset type from firmware
  dmadev: fix potential null pointer access
  net/hns3: restrict tunnel flow rule to one header
  net/mvneta: fix possible out-of-bounds write

Danylo Vodopianov (1):
  app/testpmd: fix aged flow destroy

Dariusz Sosnowski (1):
  net/mlx5: fix counter query loop getting stuck

Dave Ertman (1):
  net/ice/base: fix VLAN replay after reset

David Marchand (6):
  devtools: fix forbidden token check with multiple files
  net/iavf: preserve MAC address with i40e PF Linux driver
  buildtools/chkincs: check driver specific headers
  config/arm: fix warning for native build with meson >= 0.55
  crypto/openssl: fix 3DES-CTR with big endian CPUs
  eal/unix: optimize thread creation

Dengdui Huang (3):
  net/hns3: remove ROH devices
  net/hns3: fix error code for repeatedly create counter
  net/hns3: fix fully use hardware flow director table

Erez Shitrit (1):
  net/mlx5/hws: fix allocation of STCs

Eryk Rybak (1):
  net/i40e/base: fix blinking X722 with X557 PHY

Fabio Pricoco (2):
  net/ice/base: fix iteration of TLVs in Preserved Fields Area
  net/ice/base: add bounds check

Farah Smith (1):
  net/bnxt/tf_core: fix Thor TF EM key size check

Fidaullah Noonari (1):
  app/procinfo: fix leak on exit

Gagandeep Singh (3):
  crypto/dpaa2_sec: fix memory leak
  bus/dpaa: fix PFDRs leaks due to FQRNIs
  net/dpaa2: fix memory corruption in TM

Gregory Etelson (3):
  net/mlx5: fix GRE flow item translation for root table
  net/mlx5: fix SQ flow item size
  net/mlx5: fix SWS meter state initialization

Hanumanth Pothula (1):
  event/octeontx: fix possible integer overflow

Hemant Agrawal (4):
  bus/dpaa: fix VSP for 1G fm1-mac9 and 10
  bus/dpaa: fix the fman details status
  examples/l2fwd-event: fix spinlock handling
  bus/dpaa: fix lock condition during error handling

Hernan Vargas (1):
  baseband/acc: fix access to deallocated mem

Igor Gutorov (1):
  net/mlx5: fix reported Rx/Tx descriptor limits

Jiawen Wu (8):
  net/txgbe: fix SWFW mbox
  net/txgbe: fix VF-PF mbox interrupt
  net/txgbe: remove outer UDP checksum capability
  net/txgbe: fix driver load bit to inform firmware
  net/ngbe: fix driver load bit to inform firmware
  net/ngbe: reconfigure more MAC Rx registers
  net/ngbe: fix interrupt lost in legacy or MSI mode
  net/ngbe: restrict configurat