Sending multiple patches for a feature work.
Hello all. My name is Aditya Ambadipudi. I work at Arm. I and my boss (Honnappa Nagarahalli) have been working on the Deque library that he talked about earlier in another email several weeks/months ago. Essentially, he wrote out all the functions signatures and stuff. He also implemented a few of these functions. I was tasked with implementing the remaining features. We were wondering what the best way would be to mail all of these changes. Locally we have two commits. The one that my boss created and the one that I created. I was wondering if it was possible to mail this whole thing as two separate patches. If we squash both of our commits together it can only have a single author name. Both of us have individually written a lot of code for this project. I have written around 2000 lines so far and my boss has written close to a 1000. What is the best way to mail these patches such that both of us are recognized for the work we have done. Thank you, Aditya Ambadipudi.
[PATCH v2 0/2] deque: add multithread unsafe deque library
As previously discussed in the mailing list [1] we are sending out this patch that provides the implementation and unit test cases for the RTE_DEQUE library. This includes functions for creating a RTE_DEQUE object. Allocating memory to it. Deleting that object and free'ing the memory associated with it. Enqueue/Dequeue functions. Functions for zero-copy API. Aditya Ambadipudi (1): deque: add unit tests for the deque library Honnappa Nagarahalli (1): deque: add multi-thread unsafe double ended queue .mailmap |1 + app/test/meson.build |2 + app/test/test_deque_enqueue_dequeue.c | 1228 app/test/test_deque_helper_functions.c | 169 devtools/build-dict.sh |1 + lib/deque/meson.build | 11 + lib/deque/rte_deque.c | 193 lib/deque/rte_deque.h | 533 ++ lib/deque/rte_deque_core.h | 81 ++ lib/deque/rte_deque_pvt.h | 538 +++ lib/deque/rte_deque_zc.h | 430 + lib/deque/version.map | 14 + lib/meson.build|2 + 13 files changed, 3203 insertions(+) create mode 100644 app/test/test_deque_enqueue_dequeue.c create mode 100644 app/test/test_deque_helper_functions.c create mode 100644 lib/deque/meson.build create mode 100644 lib/deque/rte_deque.c create mode 100644 lib/deque/rte_deque.h create mode 100644 lib/deque/rte_deque_core.h create mode 100644 lib/deque/rte_deque_pvt.h create mode 100644 lib/deque/rte_deque_zc.h create mode 100644 lib/deque/version.map -- 2.25.1
[PATCH v2 1/2] deque: add multi-thread unsafe double ended queue
From: Honnappa Nagarahalli
Add a multi-thread unsafe double ended queue data structure. This
library provides a simple and efficient alternative to multi-thread
safe ring when multi-thread safety is not required.
Signed-off-by: Aditya Ambadipudi
Signed-off-by: Honnappa Nagarahalli
Change-Id: I6f66fa2ebf750adb22ac75f8cb3c2fe8bdb5fa9e
---
v2:
* Addressed the spell check warning issue with the word "Deque"
* Tried to rename all objects that are named deque to avoid collision with
std::deque
* Added the deque library to msvc section in meson.build
* Renamed api functions to explicitly state if the function inserts at head
org
tail.
.mailmap | 1 +
devtools/build-dict.sh | 1 +
lib/deque/meson.build | 11 +
lib/deque/rte_deque.c | 193 +
lib/deque/rte_deque.h | 533
lib/deque/rte_deque_core.h | 81 ++
lib/deque/rte_deque_pvt.h | 538 +
lib/deque/rte_deque_zc.h | 430 +
lib/deque/version.map | 14 +
lib/meson.build| 2 +
10 files changed, 1804 insertions(+)
create mode 100644 lib/deque/meson.build
create mode 100644 lib/deque/rte_deque.c
create mode 100644 lib/deque/rte_deque.h
create mode 100644 lib/deque/rte_deque_core.h
create mode 100644 lib/deque/rte_deque_pvt.h
create mode 100644 lib/deque/rte_deque_zc.h
create mode 100644 lib/deque/version.map
diff --git a/.mailmap b/.mailmap
index 3843868716..8e705ab6ab 100644
--- a/.mailmap
+++ b/.mailmap
@@ -17,6 +17,7 @@ Adam Bynes
Adam Dybkowski
Adam Ludkiewicz
Adham Masarwah
+Aditya Ambadipudi
Adrian Moreno
Adrian Podlawski
Adrien Mazarguil
diff --git a/devtools/build-dict.sh b/devtools/build-dict.sh
index a8cac49029..595d8f9277 100755
--- a/devtools/build-dict.sh
+++ b/devtools/build-dict.sh
@@ -17,6 +17,7 @@ sed '/^..->/d' |
sed '/^uint->/d' |
sed "/^doesn'->/d" |
sed '/^wasn->/d' |
+sed '/^deque.*->/d' |
# print to stdout
cat
diff --git a/lib/deque/meson.build b/lib/deque/meson.build
new file mode 100644
index 00..1ff45fc39f
--- /dev/null
+++ b/lib/deque/meson.build
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 Arm Limited
+
+sources = files('rte_deque.c')
+headers = files('rte_deque.h')
+# most sub-headers are not for direct inclusion
+indirect_headers += files (
+'rte_deque_core.h',
+'rte_deque_pvt.h',
+'rte_deque_zc.h'
+)
diff --git a/lib/deque/rte_deque.c b/lib/deque/rte_deque.c
new file mode 100644
index 00..b83a6c43c4
--- /dev/null
+++ b/lib/deque/rte_deque.c
@@ -0,0 +1,193 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2024 Arm Limited
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "rte_deque.h"
+
+/* mask of all valid flag values to deque_create() */
+#define __RTE_DEQUE_F_MASK (RTE_DEQUE_F_EXACT_SZ)
+ssize_t
+rte_deque_get_memsize_elem(unsigned int esize, unsigned int count)
+{
+ ssize_t sz;
+
+ /* Check if element size is a multiple of 4B */
+ if (esize % 4 != 0) {
+ rte_log(RTE_LOG_ERR, rte_deque_log_type,
+ "%s(): element size is not a multiple of 4\n",
+ __func__);
+
+ return -EINVAL;
+ }
+
+ /* count must be a power of 2 */
+ if ((!RTE_IS_POWER_OF_2(count)) || (count > RTE_DEQUE_SZ_MASK)) {
+ rte_log(RTE_LOG_ERR, rte_deque_log_type,
+ "%s(): Requested number of elements is invalid,"
+ "must be power of 2, and not exceed %u\n",
+ __func__, RTE_DEQUE_SZ_MASK);
+
+ return -EINVAL;
+ }
+
+ sz = sizeof(struct rte_deque) + (ssize_t)count * esize;
+ sz = RTE_ALIGN(sz, RTE_CACHE_LINE_SIZE);
+ return sz;
+}
+
+void
+rte_deque_reset(struct rte_deque *d)
+{
+ d->head = 0;
+ d->tail = 0;
+}
+
+int
+rte_deque_init(struct rte_deque *d, const char *name, unsigned int count,
+ unsigned int flags)
+{
+ int ret;
+
+ /* compilation-time checks */
+ RTE_BUILD_BUG_ON((sizeof(struct rte_deque) &
+ RTE_CACHE_LINE_MASK) != 0);
+
+ /* future proof flags, only allow supported values */
+ if (flags & ~__RTE_DEQUE_F_MASK) {
+ rte_log(RTE_LOG_ERR, rte_deque_log_type,
+ "%s(): Unsupported flags requested %#x\n",
+ __func__, flags);
+ return -EINVAL;
+ }
+
+ /* init the deque structure */
+ memset(d, 0, sizeof(*d));
+ ret = strlcpy(d->name, name, siz
[PATCH v2 2/2] deque: add unit tests for the deque library
Add unit test cases that test all of the enqueue/dequeue functions.
Both normal enqueue/dequeue functions and the zerocopy API functions.
Signed-off-by: Aditya Ambadipudi
Change-Id: Ida5bdefdd9d001b792a8d4be011387ff4f84c154
---
v2:
* Addressed the spell check warning issue with the word "Deque"
* Tried to rename all objects that are named deque to avoid collision with
std::deque
app/test/meson.build |2 +
app/test/test_deque_enqueue_dequeue.c | 1228
app/test/test_deque_helper_functions.c | 169
3 files changed, 1399 insertions(+)
create mode 100644 app/test/test_deque_enqueue_dequeue.c
create mode 100644 app/test/test_deque_helper_functions.c
diff --git a/app/test/meson.build b/app/test/meson.build
index 7d909039ae..8913050c9b 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -60,6 +60,8 @@ source_file_deps = {
'test_cryptodev_security_tls_record.c': ['cryptodev', 'security'],
'test_cycles.c': [],
'test_debug.c': [],
+'test_deque_enqueue_dequeue.c': ['deque'],
+'test_deque_helper_functions.c': ['deque'],
'test_devargs.c': ['kvargs'],
'test_dispatcher.c': ['dispatcher'],
'test_distributor.c': ['distributor'],
diff --git a/app/test/test_deque_enqueue_dequeue.c
b/app/test/test_deque_enqueue_dequeue.c
new file mode 100644
index 00..5816218235
--- /dev/null
+++ b/app/test/test_deque_enqueue_dequeue.c
@@ -0,0 +1,1228 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2024 Arm Limited
+ */
+
+
+#include "test.h"
+
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+#include
+#include
+#include
+
+static const int esize[] = {4, 8, 16, 20};
+#define DEQUE_SIZE 4096
+#define MAX_BULK 32
+#define TEST_DEQUE_FULL_EMPTY_ITER 8
+
+/*
+ * Validate the return value of test cases and print details of the
+ * deque if validation fails
+ *
+ * @param exp
+ * Expression to validate return value.
+ * @param r
+ * A pointer to the deque structure.
+ */
+#define TEST_DEQUE_VERIFY(exp, d, errst) do { \
+ if (!(exp)) { \
+ printf("error at %s:%d\tcondition " #exp " failed\n", \
+ __func__, __LINE__);\
+ rte_deque_dump(stdout, (d));\
+ errst; \
+ } \
+} while (0)
+
+static int
+test_deque_mem_cmp(void *src, void *dst, unsigned int size)
+{
+ int ret;
+
+ ret = memcmp(src, dst, size);
+ if (ret) {
+ rte_hexdump(stdout, "src", src, size);
+ rte_hexdump(stdout, "dst", dst, size);
+ printf("data after dequeue is not the same\n");
+ }
+
+ return ret;
+}
+
+static int
+test_deque_mem_cmp_rvs(void *src, void *dst,
+ unsigned int count, unsigned int esize)
+{
+ int ret = 0;
+ uint32_t *src32 = ((uint32_t *)src), *dst32 = ((uint32_t *)dst);
+ uint32_t scale = esize/(sizeof(uint32_t));
+
+ /* Start at the end of the dst and compare from there.*/
+ dst32 += (count - 1) * scale;
+ for (unsigned int i = 0; i < count; i++) {
+ for (unsigned int j = 0; j < scale; j++) {
+ if (src32[j] != dst32[j]) {
+ ret = -1;
+ break;
+ }
+ }
+ if (ret)
+ break;
+ dst32 -= scale;
+ src32 += scale;
+ }
+ if (ret) {
+ rte_hexdump(stdout, "src", src, count * esize);
+ rte_hexdump(stdout, "dst", dst, count * esize);
+ printf("data after dequeue is not the same\n");
+ }
+
+ return ret;
+}
+
+static inline void *
+test_deque_calloc(unsigned int dsize, int esize)
+{
+ void *p;
+
+ p = rte_zmalloc(NULL, dsize * esize, RTE_CACHE_LINE_SIZE);
+ if (p == NULL)
+ printf("Failed to allocate memory\n");
+
+ return p;
+}
+
+static void
+test_deque_mem_init(void *obj, unsigned int count, int esize)
+{
+ for (unsigned int i = 0; i < (count * esize / sizeof(uint32_t)); i++)
+ ((uint32_t *)obj)[i] = i;
+}
+
+static inline void *
+test_deque_inc_ptr(void *obj, int esize, unsigned int n)
+{
+ return (void *)((uint32_t *)obj + (n * esize / sizeof(uint32_t)));
+}
+
+/* Copy to the deque memory */
+static inline void
+test_deque_zc_copy_to_deque(struct rte_deque_zc_data *zcd, const
[PATCH v3 0/2] deque: add multithread unsafe deque library
As previously discussed in the mailing list [1] we are sending out this patch that provides the implementation and unit test cases for the RTE_DEQUE library. This includes functions for creating a RTE_DEQUE object. Allocating memory to it. Deleting that object and free'ing the memory associated with it. Enqueue/Dequeue functions. Functions for zero-copy API. Aditya Ambadipudi (1): deque: add unit tests for the deque library Honnappa Nagarahalli (1): deque: add multi-thread unsafe double ended queue .mailmap |1 + app/test/meson.build |2 + app/test/test_deque_enqueue_dequeue.c | 1228 app/test/test_deque_helper_functions.c | 169 devtools/build-dict.sh |1 + lib/deque/meson.build | 11 + lib/deque/rte_deque.c | 193 lib/deque/rte_deque.h | 533 ++ lib/deque/rte_deque_core.h | 81 ++ lib/deque/rte_deque_pvt.h | 538 +++ lib/deque/rte_deque_zc.h | 430 + lib/deque/version.map | 14 + lib/meson.build|2 + 13 files changed, 3203 insertions(+) create mode 100644 app/test/test_deque_enqueue_dequeue.c create mode 100644 app/test/test_deque_helper_functions.c create mode 100644 lib/deque/meson.build create mode 100644 lib/deque/rte_deque.c create mode 100644 lib/deque/rte_deque.h create mode 100644 lib/deque/rte_deque_core.h create mode 100644 lib/deque/rte_deque_pvt.h create mode 100644 lib/deque/rte_deque_zc.h create mode 100644 lib/deque/version.map -- 2.25.1
[PATCH v3 1/2] deque: add multi-thread unsafe double ended queue
From: Honnappa Nagarahalli
Add a multi-thread unsafe double ended queue data structure. This
library provides a simple and efficient alternative to multi-thread
safe ring when multi-thread safety is not required.
Signed-off-by: Aditya Ambadipudi
Signed-off-by: Honnappa Nagarahalli
---
v3:
* Removed stdio.h from two files where it is not needed.
v2:
* Addressed the spell check warning issue with the word "Deque"
* Tried to rename all objects that are named deque to avoid collision with
std::deque
* Added the deque library to msvc section in meson.build
* Renamed api functions to explicitly state if the function inserts at head
org
tail.
.mailmap | 1 +
devtools/build-dict.sh | 1 +
lib/deque/meson.build | 11 +
lib/deque/rte_deque.c | 193 +
lib/deque/rte_deque.h | 533
lib/deque/rte_deque_core.h | 81 ++
lib/deque/rte_deque_pvt.h | 538 +
lib/deque/rte_deque_zc.h | 430 +
lib/deque/version.map | 14 +
lib/meson.build| 2 +
10 files changed, 1804 insertions(+)
create mode 100644 lib/deque/meson.build
create mode 100644 lib/deque/rte_deque.c
create mode 100644 lib/deque/rte_deque.h
create mode 100644 lib/deque/rte_deque_core.h
create mode 100644 lib/deque/rte_deque_pvt.h
create mode 100644 lib/deque/rte_deque_zc.h
create mode 100644 lib/deque/version.map
diff --git a/.mailmap b/.mailmap
index 3843868716..8e705ab6ab 100644
--- a/.mailmap
+++ b/.mailmap
@@ -17,6 +17,7 @@ Adam Bynes
Adam Dybkowski
Adam Ludkiewicz
Adham Masarwah
+Aditya Ambadipudi
Adrian Moreno
Adrian Podlawski
Adrien Mazarguil
diff --git a/devtools/build-dict.sh b/devtools/build-dict.sh
index a8cac49029..595d8f9277 100755
--- a/devtools/build-dict.sh
+++ b/devtools/build-dict.sh
@@ -17,6 +17,7 @@ sed '/^..->/d' |
sed '/^uint->/d' |
sed "/^doesn'->/d" |
sed '/^wasn->/d' |
+sed '/^deque.*->/d' |
# print to stdout
cat
diff --git a/lib/deque/meson.build b/lib/deque/meson.build
new file mode 100644
index 00..1ff45fc39f
--- /dev/null
+++ b/lib/deque/meson.build
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 Arm Limited
+
+sources = files('rte_deque.c')
+headers = files('rte_deque.h')
+# most sub-headers are not for direct inclusion
+indirect_headers += files (
+'rte_deque_core.h',
+'rte_deque_pvt.h',
+'rte_deque_zc.h'
+)
diff --git a/lib/deque/rte_deque.c b/lib/deque/rte_deque.c
new file mode 100644
index 00..b83a6c43c4
--- /dev/null
+++ b/lib/deque/rte_deque.c
@@ -0,0 +1,193 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2024 Arm Limited
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "rte_deque.h"
+
+/* mask of all valid flag values to deque_create() */
+#define __RTE_DEQUE_F_MASK (RTE_DEQUE_F_EXACT_SZ)
+ssize_t
+rte_deque_get_memsize_elem(unsigned int esize, unsigned int count)
+{
+ ssize_t sz;
+
+ /* Check if element size is a multiple of 4B */
+ if (esize % 4 != 0) {
+ rte_log(RTE_LOG_ERR, rte_deque_log_type,
+ "%s(): element size is not a multiple of 4\n",
+ __func__);
+
+ return -EINVAL;
+ }
+
+ /* count must be a power of 2 */
+ if ((!RTE_IS_POWER_OF_2(count)) || (count > RTE_DEQUE_SZ_MASK)) {
+ rte_log(RTE_LOG_ERR, rte_deque_log_type,
+ "%s(): Requested number of elements is invalid,"
+ "must be power of 2, and not exceed %u\n",
+ __func__, RTE_DEQUE_SZ_MASK);
+
+ return -EINVAL;
+ }
+
+ sz = sizeof(struct rte_deque) + (ssize_t)count * esize;
+ sz = RTE_ALIGN(sz, RTE_CACHE_LINE_SIZE);
+ return sz;
+}
+
+void
+rte_deque_reset(struct rte_deque *d)
+{
+ d->head = 0;
+ d->tail = 0;
+}
+
+int
+rte_deque_init(struct rte_deque *d, const char *name, unsigned int count,
+ unsigned int flags)
+{
+ int ret;
+
+ /* compilation-time checks */
+ RTE_BUILD_BUG_ON((sizeof(struct rte_deque) &
+ RTE_CACHE_LINE_MASK) != 0);
+
+ /* future proof flags, only allow supported values */
+ if (flags & ~__RTE_DEQUE_F_MASK) {
+ rte_log(RTE_LOG_ERR, rte_deque_log_type,
+ "%s(): Unsupported flags requested %#x\n",
+ __func__, flags);
+ return -EINVAL;
+ }
+
+ /* init the deque structure */
+ memset(d, 0, sizeof(*d));
+ ret = strlcpy(d->name, name, siz
[PATCH v3 2/2] deque: add unit tests for the deque library
Add unit test cases that test all of the enqueue/dequeue functions.
Both normal enqueue/dequeue functions and the zerocopy API functions.
Signed-off-by: Aditya Ambadipudi
---
v3:
* Fixed a few casts that were causing compiler warnings.
v2:
* Addressed the spell check warning issue with the word "Deque"
* Tried to rename all objects that are named deque to avoid collision with
std::deque
app/test/meson.build |2 +
app/test/test_deque_enqueue_dequeue.c | 1228
app/test/test_deque_helper_functions.c | 169
3 files changed, 1399 insertions(+)
create mode 100644 app/test/test_deque_enqueue_dequeue.c
create mode 100644 app/test/test_deque_helper_functions.c
diff --git a/app/test/meson.build b/app/test/meson.build
index 7d909039ae..8913050c9b 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -60,6 +60,8 @@ source_file_deps = {
'test_cryptodev_security_tls_record.c': ['cryptodev', 'security'],
'test_cycles.c': [],
'test_debug.c': [],
+'test_deque_enqueue_dequeue.c': ['deque'],
+'test_deque_helper_functions.c': ['deque'],
'test_devargs.c': ['kvargs'],
'test_dispatcher.c': ['dispatcher'],
'test_distributor.c': ['distributor'],
diff --git a/app/test/test_deque_enqueue_dequeue.c
b/app/test/test_deque_enqueue_dequeue.c
new file mode 100644
index 00..8180677380
--- /dev/null
+++ b/app/test/test_deque_enqueue_dequeue.c
@@ -0,0 +1,1228 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2024 Arm Limited
+ */
+
+
+#include "test.h"
+
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+#include
+#include
+#include
+
+static const int esize[] = {4, 8, 16, 20};
+#define DEQUE_SIZE 4096
+#define MAX_BULK 32
+#define TEST_DEQUE_FULL_EMPTY_ITER 8
+
+/*
+ * Validate the return value of test cases and print details of the
+ * deque if validation fails
+ *
+ * @param exp
+ * Expression to validate return value.
+ * @param r
+ * A pointer to the deque structure.
+ */
+#define TEST_DEQUE_VERIFY(exp, d, errst) do { \
+ if (!(exp)) { \
+ printf("error at %s:%d\tcondition " #exp " failed\n", \
+ __func__, __LINE__);\
+ rte_deque_dump(stdout, (d));\
+ errst; \
+ } \
+} while (0)
+
+static int
+test_deque_mem_cmp(void *src, void *dst, unsigned int size)
+{
+ int ret;
+
+ ret = memcmp(src, dst, size);
+ if (ret) {
+ rte_hexdump(stdout, "src", src, size);
+ rte_hexdump(stdout, "dst", dst, size);
+ printf("data after dequeue is not the same\n");
+ }
+
+ return ret;
+}
+
+static int
+test_deque_mem_cmp_rvs(void *src, void *dst,
+ unsigned int count, unsigned int esize)
+{
+ int ret = 0;
+ uint32_t *src32 = ((uint32_t *)src), *dst32 = ((uint32_t *)dst);
+ uint32_t scale = esize/(sizeof(uint32_t));
+
+ /* Start at the end of the dst and compare from there.*/
+ dst32 += (count - 1) * scale;
+ for (unsigned int i = 0; i < count; i++) {
+ for (unsigned int j = 0; j < scale; j++) {
+ if (src32[j] != dst32[j]) {
+ ret = -1;
+ break;
+ }
+ }
+ if (ret)
+ break;
+ dst32 -= scale;
+ src32 += scale;
+ }
+ if (ret) {
+ rte_hexdump(stdout, "src", src, count * esize);
+ rte_hexdump(stdout, "dst", dst, count * esize);
+ printf("data after dequeue is not the same\n");
+ }
+
+ return ret;
+}
+
+static inline void *
+test_deque_calloc(unsigned int dsize, int esize)
+{
+ void *p;
+
+ p = rte_zmalloc(NULL, dsize * esize, RTE_CACHE_LINE_SIZE);
+ if (p == NULL)
+ printf("Failed to allocate memory\n");
+
+ return p;
+}
+
+static void
+test_deque_mem_init(void *obj, unsigned int count, int esize)
+{
+ for (unsigned int i = 0; i < (count * esize / sizeof(uint32_t)); i++)
+ ((uint32_t *)obj)[i] = i;
+}
+
+static inline void *
+test_deque_inc_ptr(void *obj, int esize, unsigned int n)
+{
+ return (void *)((uint32_t *)obj + (n * esize / sizeof(uint32_t)));
+}
+
+/* Copy to the deque memory */
+static inline void
+test_deque_zc_copy_to_deque(struct rte_deque_zc_data *zcd, const
Re: [PATCH v3 0/2] deque: add multithread unsafe deque library
Hello Ali & Patrick. Please use v3 of this patch to see if it fixes the "deque" spell check issue that you folks were helping me & Wathsala with. I have removed Gerrit change-id from this patch. Thank you, Aditya Ambadipudi ____ From: Aditya Ambadipudi Sent: Thursday, May 2, 2024 3:19 PM To: dev@dpdk.org ; jack...@nvidia.com ; step...@networkplumber.org ; ma...@nvidia.com ; viachesl...@nvidia.com ; roret...@linux.microsoft.com ; konstantin.anan...@huawei.com ; m...@smartsharesystems.com ; hof...@lysator.liu.se ; pr...@iol.unh.edu ; alia...@nvidia.com Cc: Wathsala Wathawana Vithanage ; Dhruv Tripathi ; Honnappa Nagarahalli ; nd ; Aditya Ambadipudi ; Aditya Ambadipudi Subject: [PATCH v3 0/2] deque: add multithread unsafe deque library As previously discussed in the mailing list [1] we are sending out this patch that provides the implementation and unit test cases for the RTE_DEQUE library. This includes functions for creating a RTE_DEQUE object. Allocating memory to it. Deleting that object and free'ing the memory associated with it. Enqueue/Dequeue functions. Functions for zero-copy API. Aditya Ambadipudi (1): deque: add unit tests for the deque library Honnappa Nagarahalli (1): deque: add multi-thread unsafe double ended queue .mailmap |1 + app/test/meson.build |2 + app/test/test_deque_enqueue_dequeue.c | 1228 app/test/test_deque_helper_functions.c | 169 devtools/build-dict.sh |1 + lib/deque/meson.build | 11 + lib/deque/rte_deque.c | 193 lib/deque/rte_deque.h | 533 ++ lib/deque/rte_deque_core.h | 81 ++ lib/deque/rte_deque_pvt.h | 538 +++ lib/deque/rte_deque_zc.h | 430 + lib/deque/version.map | 14 + lib/meson.build|2 + 13 files changed, 3203 insertions(+) create mode 100644 app/test/test_deque_enqueue_dequeue.c create mode 100644 app/test/test_deque_helper_functions.c create mode 100644 lib/deque/meson.build create mode 100644 lib/deque/rte_deque.c create mode 100644 lib/deque/rte_deque.h create mode 100644 lib/deque/rte_deque_core.h create mode 100644 lib/deque/rte_deque_pvt.h create mode 100644 lib/deque/rte_deque_zc.h create mode 100644 lib/deque/version.map -- 2.25.1
Question about loop unrolling in rte_ring datastructure.
Hello all. My name is Aditya Ambadipudi. I am not the sharpest tool in the shed. I was reading through the rte_ring datastructure. And I have two questions about the optimizations that are being made there. 1. Loop unrolling: https://github.com/DPDK/dpdk/blob/main/lib/ring/rte_ring_elem_pvt.h#L28-L35 Why are we unrolling these loops manually. GCC will generate SIMD instructions for these loops automatically. Irrespective of wheither or not we unroll the loops Unrolled loop: https://godbolt.org/z/n97noqYn7 Regular loop:https://godbolt.org/z/h6G9o9773 This is true of both x86 and ARM. 2. Normalizing to few fixed types: It looks like we separate out enqueue/dequeue operations into 3 functions. One for each element size 32, 64, 128. Again I am not clear on why we are doing this. Both 128 & 64 are multiples of 32. Why can't we just normalize everything to 32? I feel like this is in some shape or form related to loop unrolling. But I am not able to figure it out on my own. I am working on a patch that is closely related to this. And I would greatly appreciate any assistance anyone can provide on this. Thank you, Aditya Ambadipudi IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
[PATCH v1 0/2] deque: add multithread unsafe deque library
As previously discussed in the mailing list [1] we are sending out this patch that provides the implementation and unit test cases for the RTE_DEQUE library. This includes functions for creating a RTE_DEQUE object. Allocating memory to it. Deleting that object and free'ing the memory associated with it. Enqueue/Dequeue functions. Functions for zero-copy API. [1] https://mails.dpdk.org/archives/dev/2023-August/275003.html Aditya Ambadipudi (1): deque: add unit tests for the deque library Honnappa Nagarahalli (1): deque: add multi-thread unsafe double ended queue .mailmap |1 + app/test/meson.build |2 + app/test/test_deque_enqueue_dequeue.c | 1231 app/test/test_deque_helper_functions.c | 170 lib/deque/meson.build | 11 + lib/deque/rte_deque.c | 194 lib/deque/rte_deque.h | 533 ++ lib/deque/rte_deque_core.h | 82 ++ lib/deque/rte_deque_pvt.h | 538 +++ lib/deque/rte_deque_zc.h | 430 + lib/deque/version.map | 14 + lib/meson.build|1 + 12 files changed, 3207 insertions(+) create mode 100644 app/test/test_deque_enqueue_dequeue.c create mode 100644 app/test/test_deque_helper_functions.c create mode 100644 lib/deque/meson.build create mode 100644 lib/deque/rte_deque.c create mode 100644 lib/deque/rte_deque.h create mode 100644 lib/deque/rte_deque_core.h create mode 100644 lib/deque/rte_deque_pvt.h create mode 100644 lib/deque/rte_deque_zc.h create mode 100644 lib/deque/version.map -- 2.25.1
[PATCH v1 1/2] deque: add multi-thread unsafe double ended queue
From: Honnappa Nagarahalli
Add a multi-thread unsafe double ended queue data structure. This
library provides a simple and efficient alternative to multi-thread
safe ring when multi-thread safety is not required.
Signed-off-by: Aditya Ambadipudi
Signed-off-by: Honnappa Nagarahalli
---
.mailmap | 1 +
lib/deque/meson.build | 11 +
lib/deque/rte_deque.c | 194 +
lib/deque/rte_deque.h | 533
lib/deque/rte_deque_core.h | 82 ++
lib/deque/rte_deque_pvt.h | 538 +
lib/deque/rte_deque_zc.h | 430 +
lib/deque/version.map | 14 +
lib/meson.build| 1 +
9 files changed, 1804 insertions(+)
create mode 100644 lib/deque/meson.build
create mode 100644 lib/deque/rte_deque.c
create mode 100644 lib/deque/rte_deque.h
create mode 100644 lib/deque/rte_deque_core.h
create mode 100644 lib/deque/rte_deque_pvt.h
create mode 100644 lib/deque/rte_deque_zc.h
create mode 100644 lib/deque/version.map
diff --git a/.mailmap b/.mailmap
index 3843868716..8e705ab6ab 100644
--- a/.mailmap
+++ b/.mailmap
@@ -17,6 +17,7 @@ Adam Bynes
Adam Dybkowski
Adam Ludkiewicz
Adham Masarwah
+Aditya Ambadipudi
Adrian Moreno
Adrian Podlawski
Adrien Mazarguil
diff --git a/lib/deque/meson.build b/lib/deque/meson.build
new file mode 100644
index 00..1ff45fc39f
--- /dev/null
+++ b/lib/deque/meson.build
@@ -0,0 +1,11 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright(c) 2024 Arm Limited
+
+sources = files('rte_deque.c')
+headers = files('rte_deque.h')
+# most sub-headers are not for direct inclusion
+indirect_headers += files (
+'rte_deque_core.h',
+'rte_deque_pvt.h',
+'rte_deque_zc.h'
+)
diff --git a/lib/deque/rte_deque.c b/lib/deque/rte_deque.c
new file mode 100644
index 00..3b08b91a98
--- /dev/null
+++ b/lib/deque/rte_deque.c
@@ -0,0 +1,194 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2024 Arm Limited
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "rte_deque.h"
+
+/* mask of all valid flag values to deque_create() */
+#define __RTE_DEQUE_F_MASK (RTE_DEQUE_F_EXACT_SZ)
+ssize_t
+rte_deque_get_memsize_elem(unsigned int esize, unsigned int count)
+{
+ ssize_t sz;
+
+ /* Check if element size is a multiple of 4B */
+ if (esize % 4 != 0) {
+ rte_log(RTE_LOG_ERR, rte_deque_log_type,
+ "%s(): element size is not a multiple of 4\n",
+ __func__);
+
+ return -EINVAL;
+ }
+
+ /* count must be a power of 2 */
+ if ((!RTE_IS_POWER_OF_2(count)) || (count > RTE_DEQUE_SZ_MASK)) {
+ rte_log(RTE_LOG_ERR, rte_deque_log_type,
+ "%s(): Requested number of elements is invalid,"
+ "must be power of 2, and not exceed %u\n",
+ __func__, RTE_DEQUE_SZ_MASK);
+
+ return -EINVAL;
+ }
+
+ sz = sizeof(struct rte_deque) + (ssize_t)count * esize;
+ sz = RTE_ALIGN(sz, RTE_CACHE_LINE_SIZE);
+ return sz;
+}
+
+void
+rte_deque_reset(struct rte_deque *d)
+{
+ d->head = 0;
+ d->tail = 0;
+}
+
+int
+rte_deque_init(struct rte_deque *d, const char *name, unsigned int count,
+ unsigned int flags)
+{
+ int ret;
+
+ /* compilation-time checks */
+ RTE_BUILD_BUG_ON((sizeof(struct rte_deque) &
+ RTE_CACHE_LINE_MASK) != 0);
+
+ /* future proof flags, only allow supported values */
+ if (flags & ~__RTE_DEQUE_F_MASK) {
+ rte_log(RTE_LOG_ERR, rte_deque_log_type,
+ "%s(): Unsupported flags requested %#x\n",
+ __func__, flags);
+ return -EINVAL;
+ }
+
+ /* init the deque structure */
+ memset(d, 0, sizeof(*d));
+ ret = strlcpy(d->name, name, sizeof(d->name));
+ if (ret < 0 || ret >= (int)sizeof(d->name))
+ return -ENAMETOOLONG;
+ d->flags = flags;
+
+ if (flags & RTE_DEQUE_F_EXACT_SZ) {
+ d->size = rte_align32pow2(count + 1);
+ d->mask = d->size - 1;
+ d->capacity = count;
+ } else {
+ if ((!RTE_IS_POWER_OF_2(count)) || (count > RTE_DEQUE_SZ_MASK))
{
+ rte_log(RTE_LOG_ERR, rte_deque_log_type,
+ "%s(): Requested size is invalid, must be power"
+ " of 2, and not exceed the size limit %u\n",
+ __func__, RTE_DEQUE_SZ_MASK);
+ return -EINVAL;
+
[PATCH v1 2/2] deque: add unit tests for the deque library
Add unit test cases that test all of the enqueue/dequeue functions.
Both normal enqueue/dequeue functions and the zerocopy API functions.
Signed-off-by: Aditya Ambadipudi
Reviewed-by: Honnappa Nagarahalli
---
app/test/meson.build |2 +
app/test/test_deque_enqueue_dequeue.c | 1231
app/test/test_deque_helper_functions.c | 170
3 files changed, 1403 insertions(+)
create mode 100644 app/test/test_deque_enqueue_dequeue.c
create mode 100644 app/test/test_deque_helper_functions.c
diff --git a/app/test/meson.build b/app/test/meson.build
index 7d909039ae..8913050c9b 100644
--- a/app/test/meson.build
+++ b/app/test/meson.build
@@ -60,6 +60,8 @@ source_file_deps = {
'test_cryptodev_security_tls_record.c': ['cryptodev', 'security'],
'test_cycles.c': [],
'test_debug.c': [],
+'test_deque_enqueue_dequeue.c': ['deque'],
+'test_deque_helper_functions.c': ['deque'],
'test_devargs.c': ['kvargs'],
'test_dispatcher.c': ['dispatcher'],
'test_distributor.c': ['distributor'],
diff --git a/app/test/test_deque_enqueue_dequeue.c
b/app/test/test_deque_enqueue_dequeue.c
new file mode 100644
index 00..35f2dd4451
--- /dev/null
+++ b/app/test/test_deque_enqueue_dequeue.c
@@ -0,0 +1,1231 @@
+/* SPDX-License-Identifier: BSD-3-Clause
+ * Copyright(c) 2024 Arm Limited
+ */
+
+
+#include "test.h"
+
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+#include
+#include
+#include
+#include
+
+struct rte_deque *deque;
+
+static const int esize[] = {4, 8, 16, 20};
+#define DEQUE_SIZE 4096
+#define MAX_BULK 32
+#define TEST_DEQUE_FULL_EMPTY_ITER 8
+
+/*
+ * Validate the return value of test cases and print details of the
+ * deque if validation fails
+ *
+ * @param exp
+ * Expression to validate return value.
+ * @param r
+ * A pointer to the deque structure.
+ */
+#define TEST_DEQUE_VERIFY(exp, d, errst) do { \
+ if (!(exp)) { \
+ printf("error at %s:%d\tcondition " #exp " failed\n", \
+ __func__, __LINE__);\
+ rte_deque_dump(stdout, (d));\
+ errst; \
+ } \
+} while (0)
+
+static int
+test_deque_mem_cmp(void *src, void *dst, unsigned int size)
+{
+ int ret;
+
+ ret = memcmp(src, dst, size);
+ if (ret) {
+ rte_hexdump(stdout, "src", src, size);
+ rte_hexdump(stdout, "dst", dst, size);
+ printf("data after dequeue is not the same\n");
+ }
+
+ return ret;
+}
+
+static int
+test_deque_mem_cmp_rvs(void *src, void *dst,
+ unsigned int count, unsigned int esize)
+{
+ int ret = 0;
+ uint32_t *src32 = ((uint32_t *)src), *dst32 = ((uint32_t *)dst);
+ uint32_t scale = esize/(sizeof(uint32_t));
+
+ /* Start at the end of the dst and compare from there.*/
+ dst32 += (count - 1) * scale;
+ for (unsigned int i = 0; i < count; i++) {
+ for (unsigned int j = 0; j < scale; j++) {
+ if (src32[j] != dst32[j]) {
+ ret = -1;
+ break;
+ }
+ }
+ if (ret)
+ break;
+ dst32 -= scale;
+ src32 += scale;
+ }
+ if (ret) {
+ rte_hexdump(stdout, "src", src, count * esize);
+ rte_hexdump(stdout, "dst", dst, count * esize);
+ printf("data after dequeue is not the same\n");
+ }
+
+ return ret;
+}
+
+static inline void *
+test_deque_calloc(unsigned int dsize, int esize)
+{
+ void *p;
+
+ p = rte_zmalloc(NULL, dsize * esize, RTE_CACHE_LINE_SIZE);
+ if (p == NULL)
+ printf("Failed to allocate memory\n");
+
+ return p;
+}
+
+static void
+test_deque_mem_init(void *obj, unsigned int count, int esize)
+{
+ for (unsigned int i = 0; i < (count * esize / sizeof(uint32_t)); i++)
+ ((uint32_t *)obj)[i] = i;
+}
+
+static inline void *
+test_deque_inc_ptr(void *obj, int esize, unsigned int n)
+{
+ return (void *)((uint32_t *)obj + (n * esize / sizeof(uint32_t)));
+}
+
+/* Copy to the deque memory */
+static inline void
+test_deque_zc_copy_to_deque(struct rte_deque_zc_data *zcd, const void *src,
int esize,
+ unsigned int num)
+{
+ memcpy(zcd->ptr1, src, esize * zcd->n1);
+ if (zcd->n1 != num) {
+
Re: [PATCH v1 0/2] deque: add multithread unsafe deque library
Thanks, Stephen, for the comment. Unfortunately, we don't have the dev setup nor the resources to test out this change using MSVC. Thank you, Aditya Ambadipudi From: Stephen Hemminger Sent: Monday, April 1, 2024 9:05 AM To: Aditya Ambadipudi Cc: dev@dpdk.org ; jack...@nvidia.com ; ma...@nvidia.com ; viachesl...@nvidia.com ; roret...@linux.microsoft.com ; konstantin.v.anan...@yandex.ru ; konstantin.anan...@huawei.com ; m...@smartsharesystems.com ; hof...@lysator.liu.se ; Honnappa Nagarahalli ; Dhruv Tripathi ; Wathsala Wathawana Vithanage ; ganeshadit...@gmail.com ; nd Subject: Re: [PATCH v1 0/2] deque: add multithread unsafe deque library On Sun, 31 Mar 2024 20:37:27 -0500 Aditya Ambadipudi wrote: > As previously discussed in the mailing list [1] we are sending out this > patch that provides the implementation and unit test cases for the > RTE_DEQUE library. This includes functions for creating a RTE_DEQUE > object. Allocating memory to it. Deleting that object and free'ing the > memory associated with it. Enqueue/Dequeue functions. Functions for > zero-copy API. > > [1] https://mails.dpdk.org/archives/dev/2023-August/275003.html Does this build without errors with the Microsoft Visual C compiler? Want to make sure that all new code does not create more work for the Windows maintainers.
