date:20201212

Re: [dpdk-dev] [PATCH v3] ip_frag: recalculate data length of fragment

2020-12-12 Thread luyicai

Hi,

Thank you so much for your review!
I absolutely agree with that, and
I'll resubmit a patch as soon as possible.

> -Original Message-
> From: Ananyev, Konstantin [mailto:konstantin.anan...@intel.com]
> Sent: Monday, December 7, 2020 8:25 PM
> To: luyicai ; dev@dpdk.org
> Cc: Zhoujingbin (Robin, Russell Lab) ; 
> chenchanghu ; Lilijun (Jerry) 
> ; Linhaifeng ; 
> Guohongzhi (Russell Lab) ; wangyunjian 
> ; sta...@dpdk.org
> Subject: RE: [PATCH v3] ip_frag: recalculate data length of fragment


> Hi,
 
> In some situations, we would get several ip fragments, which total 
> data length is less than minimum frame(64) and padding with zeros.
> Examples: Second Fragment "a0a1 a2a3 a4a5 a6a7   ..."
> and Third Fragment "a8a9 aaab acad aeaf b0b1 b2b3 ...".
> Finally, we would reassemble Second and Third Fragment like this
> "a0a1 a2a3 a4a5 a6a7   ... a8a9 aaab acad aeaf b0b1 ...", 
> which is not correct!
> So, we need recalculate data length of fragment to remove padings!
> 
> Fixes: 7f0983ee331c ("ip_frag: check fragment length of incoming
> packet")
> Cc: sta...@dpdk.org
> 
> Signed-off-by: Yicai Lu 
> ---
> v2 -> v3: Update the comments.
> ---
>  lib/librte_ip_frag/rte_ipv4_reassembly.c | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/lib/librte_ip_frag/rte_ipv4_reassembly.c
> b/lib/librte_ip_frag/rte_ipv4_reassembly.c
> index 1dda8aca0..9a9fe3703 100644
> --- a/lib/librte_ip_frag/rte_ipv4_reassembly.c
> +++ b/lib/librte_ip_frag/rte_ipv4_reassembly.c
> @@ -117,6 +117,7 @@ rte_ipv4_frag_reassemble_packet(struct
> rte_ip_frag_tbl *tbl,
> 
>   ip_ofs *= RTE_IPV4_HDR_OFFSET_UNITS;
>   ip_len = rte_be_to_cpu_16(ip_hdr->total_length) - mb->l3_len;
> + mb->data_len = ip_len + mb->l3_len + mb->l2_len;

> That doesn't look correct.
> Even one fragment can consist of multiple segments.
> Plus you don't update mb->pkt_len.
> To do it properly, you'll need something like:
> trim = mb->pkt_len  - ip_len + mb->l3_len + mb->l2_len; 
> rte_pktmbuf_trim(mb, trim);

> Though my preference would be to leave it as responsibility of the 
> caller (As it has to parse packet anyway to fill l2_len/l3_len and 
> usually strips
> l2 headers, etc).  

> 
>   IP_FRAG_LOG(DEBUG, "%s:%d:\n"
>   "mbuf: %p, tms: %" PRIu64
> --
> 2.28.0.windows.1

[dpdk-dev] [PATCH v5] ip_frag: remove padding length of fragment

2020-12-12 Thread Yicai Lu

In some situations, we would get several ip fragments, which total
data length is less than min_ip_len(64) and padding with zeros.
We simulated intermediate fragments by modifying the MTU.
To illustrate the problem, we simplify the packet format and
ignore the impact of the packet header.In namespace2,
a packet whose data length is 1520 is sent.
When the packet passes tap2, the packet is divided into two
fragments: fragment A and B, similar to (1520 = 1510 + 10).
When the packet passes tap3, the larger fragment packet A is
divided into two fragments A1 and A2, similar to (1510 = 1500 + 10).
Finally, the bond interface receives three fragments:
A1, A2, and B (1520 = 1500 + 10 + 10).
One fragmented packet A2 is smaller than the minimum Ethernet
frame length, so it needs to be padded.

|---|
|  HOST |
| |--|   || |
| |  ns2 |   |  |--|  | |
| |  ||  |   |  ||||  | |
| |  |  tap1  |  |   |  |  tap2  | ns1|  tap3  |  | |
| |  |mtu=1510|  |   |  |mtu=1510||mtu=1500|  | |
| |--|1.1.1.1 |--|   |--|1.1.1.2 ||2.1.1.1 |--| |
||| |||||
| | |  ||
| |-|  ||
|  ||
|  ||   |
|  |  bond  |   |
|--|mtu=1500|---|
   ||

When processing the preceding packets above,
DPDK would aggregate fragmented packets A2 and B.
And error packets are generated, which padding(zero)
is displayed in the middle of the packet.

A2 + B:
   fa 16 3e 9f fb 82 fa 47 b2 57 dc 20 08 00 45 00
0010   00 33 b4 66 00 ba 3f 01 c1 a5 01 01 01 01 02 01
0020   01 02 c0 c1 c2 c3 c4 c5 c6 c7 00 00 00 00 00 00
0030   00 00 00 00 00 00 00 00 00 00 00 00 c8 c9 ca cb
0040   cc cd ce cf d0 d1 d2 d3 d4 d5 d6 d7 d8 d9 da db
0050   dc dd de df e0 e1 e2 e3 e4 e5 e6

So, we would calculate the length of padding, and remove
the padding in pkt_len and data_len before aggregation.

Fixes: 7f0983ee331c ("ip_frag: check fragment length of incoming packet")
Cc: sta...@dpdk.org

Signed-off-by: Yicai Lu 
---
v4 -> v5: Update the comments and description.
---
 lib/librte_ip_frag/rte_ipv4_reassembly.c | 12 +---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/lib/librte_ip_frag/rte_ipv4_reassembly.c 
b/lib/librte_ip_frag/rte_ipv4_reassembly.c
index 1dda8ac..fdf66a4 100644
--- a/lib/librte_ip_frag/rte_ipv4_reassembly.c
+++ b/lib/librte_ip_frag/rte_ipv4_reassembly.c
@@ -104,6 +104,7 @@ struct rte_mbuf *
const unaligned_uint64_t *psd;
uint16_t flag_offset, ip_ofs, ip_flag;
int32_t ip_len;
+   int32_t trim;
 
flag_offset = rte_be_to_cpu_16(ip_hdr->fragment_offset);
ip_ofs = (uint16_t)(flag_offset & RTE_IPV4_HDR_OFFSET_MASK);
@@ -117,14 +118,15 @@ struct rte_mbuf *
 
ip_ofs *= RTE_IPV4_HDR_OFFSET_UNITS;
ip_len = rte_be_to_cpu_16(ip_hdr->total_length) - mb->l3_len;
+   trim  = mb->pkt_len - (ip_len + mb->l3_len + mb->l2_len);
 
IP_FRAG_LOG(DEBUG, "%s:%d:\n"
-   "mbuf: %p, tms: %" PRIu64
-   ", key: <%" PRIx64 ", %#x>, ofs: %u, len: %d, flags: %#x\n"
+   "mbuf: %p, tms: %" PRIu64 ", key: <%" PRIx64 ", %#x>"
+   "ofs: %u, len: %d, padding: %d, flags: %#x\n"
"tbl: %p, max_cycles: %" PRIu64 ", entry_mask: %#x, "
"max_entries: %u, use_entries: %u\n\n",
__func__, __LINE__,
-   mb, tms, key.src_dst[0], key.id, ip_ofs, ip_len, ip_flag,
+   mb, tms, key.src_dst[0], key.id, ip_ofs, ip_len, trim, ip_flag,
tbl, tbl->max_cycles, tbl->entry_mask, tbl->max_entries,
tbl->use_entries);
 
@@ -134,6 +136,10 @@ struct rte_mbuf *
return NULL;
}
 
+   if (unlikely(trim > 0)) {
+   rte_pktmbuf_trim(mb, trim);
+   }
+
/* try to find/add entry into the fragment's table. */
if ((fp = ip_frag_find(tbl, dr, &key, tms)) == NULL) {
IP_FRAG_MBUF2DR(dr, mb);
-- 
1.9.5.msysgit.1

Re: [dpdk-dev] [PATCH] net/i40e: issue with ADD VLAN from Guest

2020-12-12 Thread Dey, Souvik

Hi Guo,
  Thanks for the comments. I will upload a v2 of the patch.

--
Regards,
Souvik

From: Guo, Jia 
Sent: Thursday, December 10, 2020 10:08 PM
To: Dey, Souvik ; Xing, Beilei ; Zhang, 
Qi Z 
Cc: dev@dpdk.org
Subject: RE: [PATCH] net/i40e: issue with ADD VLAN from Guest

NOTICE: This email was received from an EXTERNAL sender

hi, souvik

From: Souvik Dey mailto:so...@rbbn.com>>
Sent: Thursday, December 10, 2020 1:55 AM
To: Xing, Beilei mailto:beilei.x...@intel.com>>; Guo, 
Jia mailto:jia@intel.com>>; Zhang, Qi Z 
mailto:qi.z.zh...@intel.com>>
Cc: dev@dpdk.org; Souvik Dey 
mailto:so...@rbbn.com>>
Subject: [PATCH] net/i40e: issue with ADD VLAN from Guest

In case of i40evf pmd, when ADD_VLAN is sent down the linux i40e driver,
along with add the vlan the kernel driver also enables the vlan stripping
by default.
This might have issues if the app configured DEV_RX_OFFLOAD_VLAN_STRIP
as off at the port configuration. The app after adding the VLAN will
expect the VLAN to be coming in the received packets but, due to
VLAN_STRIP enabled at the PF, it will get stripped.
This behavior of the Linux driver causes confussion with the DPDK app
using i40e_pmd. So it is better to reconfigure the vlan_offload, which
checks for DEV_RX_OFFLOAD_VLAN_STRIP flag in the dev_conf and enables or
disables the vlan strip in the PF.
Also rte_eth_dev_set_vlan_offload() to disable VLAN_STRIP cannot be used
from the application, as this will only work for the first time when
original and current confi mismatch, but for all subsequent call it will
ignore it.

Thanks for your patch and the clear detail interpret, what you said is reset 
the configuration about vlan strip that
would be change by the pf driver when adding vlan from vf, so I think 
concentrate
 the commit log to be more simple
in your way would be enough.

Signed-off-by: Souvik Dey mailto:so...@rbbn.com>>
---
drivers/net/i40e/i40e_ethdev_vf.c | 13 -
1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/net/i40e/i40e_ethdev_vf.c 
b/drivers/net/i40e/i40e_ethdev_vf.c
index c26b036..2ecf74b 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1078,8 +1078,19 @@ i40evf_add_vlan(struct rte_eth_dev *dev, uint16_t vlanid)
args.out_buffer = vf->aq_resp;
args.out_size = I40E_AQ_BUF_SZ;
err = i40evf_execute_vf_cmd(dev, &args);
- if (err)
+ if (err) {
PMD_DRV_LOG(ERR, "fail to execute command OP_ADD_VLAN");
+ return err;

Would it be more code clean to use “goto err;”?

+ }
+ /*

/* -> /**, that would what I suggest.

+ * In linux kernel driver on receiving ADD_VLAN it enables
+ * VLAN_STRIP by default. So reconfigure the vlan_offload
+ * as it was done by the app earlier.
+ */
+ err = i40evf_vlan_offload_set(dev, ETH_VLAN_STRIP_MASK);
+ if (err)
+ PMD_DRV_LOG(ERR, "fail to execute command disable_vlan_strip "
+ "as a part of OP_ADD_VLAN");

If it is not coming as a command, please refine this log, such as “fail to set 
vlan strip.”

return err;
}
--
2.9.3.windows.1

Notice: This e-mail together with any attachments may contain information of 
Ribbon Communications Inc. that is confidential and/or proprietary for the sole 
use of the intended recipient. Any review, disclosure, reliance or distribution 
by others or forwarding without express permission is strictly prohibited. If 
you are not the intended recipient, please notify the sender immediately and 
then delete all copies, including any attachments.

[dpdk-dev] [PATCH v2] net/i40e: issue with ADD VLAN from Guest

2020-12-12 Thread Souvik Dey

Reset the configuration of vlan strip that would be change
by the pf kernel driver when adding vlan from vf.
Application cannot use rte_eth_dev_set_vlan_offload() to set
the VLAN_STRIP, as this will only work for the first time when
original and current config mismatch, but for all subsequent call
it will be ignored.

Signed-off-by: Souvik Dey 
---
v2:
* Simplied the commit log.
* goto err; is not required as it has only one more return path
and there is no cleanup required apart from just return err.
* Updated the comment start from /* -> /**
* Changed the error log in case vlan strip command fails.
---
 drivers/net/i40e/i40e_ethdev_vf.c | 13 -
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/net/i40e/i40e_ethdev_vf.c 
b/drivers/net/i40e/i40e_ethdev_vf.c
index c26b036..a00e290 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1078,8 +1078,19 @@ i40evf_add_vlan(struct rte_eth_dev *dev, uint16_t vlanid)
args.out_buffer = vf->aq_resp;
args.out_size = I40E_AQ_BUF_SZ;
err = i40evf_execute_vf_cmd(dev, &args);
-   if (err)
+   if (err) {
PMD_DRV_LOG(ERR, "fail to execute command OP_ADD_VLAN");
+   return err;
+   }
+   /**
+* In linux kernel driver on receiving ADD_VLAN it enables
+* VLAN_STRIP by default. So reconfigure the vlan_offload
+* as it was done by the app earlier.
+*/
+   err = i40evf_vlan_offload_set(dev, ETH_VLAN_STRIP_MASK)
+   if (err)
+   PMD_DRV_LOG(ERR, "fail to set vlan_strip "
+   "as a part of OP_ADD_VLAN");
 
return err;
 }
-- 
2.9.3.windows.1

[dpdk-dev] [PATCH v2] net/i40e: issue with ADD VLAN from Guest

2020-12-12 Thread Souvik Dey

Reset the configuration of vlan strip that would be change
by the pf kernel driver when adding vlan from vf.
Application cannot use rte_eth_dev_set_vlan_offload() to set
the VLAN_STRIP, as this will only work for the first time when
original and current config mismatch, but for all subsequent call
it will be ignored.

Signed-off-by: Souvik Dey 
---
v2:
* Simplied the commit log.
* goto err; is not required as it has only one more return path
and there is no cleanup required apart from just return err.
* Updated the comment start from /* -> /**
* Changed the error log in case vlan strip command fails.
---
 drivers/net/i40e/i40e_ethdev_vf.c | 13 -
 1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/drivers/net/i40e/i40e_ethdev_vf.c 
b/drivers/net/i40e/i40e_ethdev_vf.c
index c26b036..d3dbcb5 100644
--- a/drivers/net/i40e/i40e_ethdev_vf.c
+++ b/drivers/net/i40e/i40e_ethdev_vf.c
@@ -1078,8 +1078,19 @@ i40evf_add_vlan(struct rte_eth_dev *dev, uint16_t vlanid)
args.out_buffer = vf->aq_resp;
args.out_size = I40E_AQ_BUF_SZ;
err = i40evf_execute_vf_cmd(dev, &args);
-   if (err)
+   if (err) {
PMD_DRV_LOG(ERR, "fail to execute command OP_ADD_VLAN");
+   return err;
+   }
+   /**
+* In linux kernel driver on receiving ADD_VLAN it enables
+* VLAN_STRIP by default. So reconfigure the vlan_offload
+* as it was done by the app earlier.
+*/
+   err = i40evf_vlan_offload_set(dev, ETH_VLAN_STRIP_MASK);
+   if (err)
+   PMD_DRV_LOG(ERR, "fail to set vlan_strip "
+   "as a part of OP_ADD_VLAN");
 
return err;
 }
-- 
2.9.3.windows.1

[dpdk-dev] [PATCH 0/4] pmdinfogen: support Windows

2020-12-12 Thread Dmitry Kozlyuk

Based on Python version of pmdinfogen, deferred until 21.02:

http://patchwork.dpdk.org/project/dpdk/list/?series=13153

There are a few Python libraries for PE/COFF, none of which suits the need,
so a custom COFF parser is used.  Advice is welcome, options considered:

* py-coff (https://github.com/jeppeter/py-coff): doesn't give access to
  symbol values, Python 2 code inside, not very popular.

* pefile (https://github.com/erocarrera/pefile): for PE (executables and
  libraries), not COFF (objects); most popular.

* pype32-py3 (https://github.com/crackinglandia/pype32): ditto, less popular.

A script to extract object files from library is still required. Meson has
extract_all_objects(), but they can't be passed as inputs to custom_target()
until 0.52.0 (commit f431cff809).

Depends-on: series-13153 ("pmdinfogen: rewrite in Python")

Dmitry Kozlyuk (4):
  pmdinfogen: support COFF
  pmdinfogen: allow multiple input files
  buildtools: support object file extraction for Windows
  build: enable pmdinfogen for Windows

 buildtools/coff.py  | 154 
 buildtools/gen-pmdinfo-cfile.py |  19 
 buildtools/gen-pmdinfo-cfile.sh |  14 ---
 buildtools/meson.build  |  15 +++-
 buildtools/pmdinfogen.py| 126 +++---
 drivers/meson.build |  26 +++---
 6 files changed, 294 insertions(+), 60 deletions(-)
 create mode 100644 buildtools/coff.py
 create mode 100644 buildtools/gen-pmdinfo-cfile.py
 delete mode 100755 buildtools/gen-pmdinfo-cfile.sh

-- 
2.29.2

[dpdk-dev] [PATCH 1/4] pmdinfogen: support COFF

2020-12-12 Thread Dmitry Kozlyuk

Common Object File Format (COFF) is used on Windows in place of ELF.

Add COFF parser to pmdinfogen. Also add an argument to specify input
file format, which is selected at configure time based on the target.

Signed-off-by: Dmitry Kozlyuk 
---
 buildtools/coff.py   | 154 +++
 buildtools/meson.build   |   7 ++
 buildtools/pmdinfogen.py | 107 ---
 3 files changed, 240 insertions(+), 28 deletions(-)
 create mode 100644 buildtools/coff.py

diff --git a/buildtools/coff.py b/buildtools/coff.py
new file mode 100644
index 0..86fb0602b
--- /dev/null
+++ b/buildtools/coff.py
@@ -0,0 +1,154 @@
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2020 Dmitry Kozlyuk 
+
+import ctypes
+
+# x86_64 little-endian
+COFF_MAGIC = 0x8664
+
+# Names up to this length are stored immediately in symbol table entries.
+COFF_NAMELEN = 8
+
+# Special "section numbers" changing the meaning of symbol table entry.
+COFF_SN_UNDEFINED = 0
+COFF_SN_ABSOLUTE = -1
+COFF_SN_DEBUG = -2
+
+
+class CoffFileHeader(ctypes.LittleEndianStructure):
+_pack_ = True
+_fields_ = [
+("magic", ctypes.c_uint16),
+("section_count", ctypes.c_uint16),
+("timestamp", ctypes.c_uint32),
+("symbol_table_offset", ctypes.c_uint32),
+("symbol_count", ctypes.c_uint32),
+("optional_header_size", ctypes.c_uint16),
+("flags", ctypes.c_uint16),
+]
+
+
+class CoffName(ctypes.Union):
+class Reference(ctypes.LittleEndianStructure):
+_pack_ = True
+_fields_ = [
+("zeroes", ctypes.c_uint32),
+("offset", ctypes.c_uint32),
+]
+
+Immediate = ctypes.c_char * 8
+
+_pack_ = True
+_fields_ = [
+("immediate", Immediate),
+("reference", Reference),
+]
+
+
+class CoffSection(ctypes.LittleEndianStructure):
+_pack_ = True
+_fields_ = [
+("name", CoffName),
+("physical_address", ctypes.c_uint32),
+("physical_address", ctypes.c_uint32),
+("size", ctypes.c_uint32),
+("data_offset", ctypes.c_uint32),
+("relocations_offset", ctypes.c_uint32),
+("line_numbers_offset", ctypes.c_uint32),
+("relocation_count", ctypes.c_uint16),
+("line_number_count", ctypes.c_uint16),
+("flags", ctypes.c_uint32),
+]
+
+
+class CoffSymbol(ctypes.LittleEndianStructure):
+_pack_ = True
+_fields_ = [
+("name", CoffName),
+("value", ctypes.c_uint32),
+("section_number", ctypes.c_int16),
+("type", ctypes.c_uint16),
+("storage_class", ctypes.c_uint8),
+("auxiliary_count", ctypes.c_uint8),
+]
+
+
+class Symbol:
+def __init__(self, image, symbol: CoffSymbol):
+self._image = image
+self._coff = symbol
+
+@property
+def name(self):
+if self._coff.name.reference.zeroes:
+return decode_asciiz(bytes(self._coff.name.immediate))
+
+offset = self._coff.name.reference.offset
+offset -= ctypes.sizeof(ctypes.c_uint32)
+return self._image.get_string(offset)
+
+def get_value(self, offset):
+section_number = self._coff.section_number
+
+if section_number == COFF_SN_UNDEFINED:
+return None
+
+if section_number == COFF_SN_DEBUG:
+return None
+
+if section_number == COFF_SN_ABSOLUTE:
+return bytes(ctypes.c_uint32(self._coff.value))
+
+section_data = self._image.get_section_data(section_number)
+section_offset = self._coff.value + offset
+return section_data[section_offset:]
+
+
+class Image:
+def __init__(self, data):
+header = CoffFileHeader.from_buffer_copy(data)
+header_size = ctypes.sizeof(header) + header.optional_header_size
+
+sections_desc = CoffSection * header.section_count
+sections = sections_desc.from_buffer_copy(data, header_size)
+
+symbols_desc = CoffSymbol * header.symbol_count
+symbols = symbols_desc.from_buffer_copy(data, 
header.symbol_table_offset)
+
+strings_offset = header.symbol_table_offset + ctypes.sizeof(symbols)
+strings = Image._parse_strings(data[strings_offset:])
+
+self._data = data
+self._header = header
+self._sections = sections
+self._symbols = symbols
+self._strings = strings
+
+@staticmethod
+def _parse_strings(data):
+full_size = ctypes.c_uint32.from_buffer_copy(data)
+header_size = ctypes.sizeof(full_size)
+return data[header_size : full_size.value]
+
+@property
+def symbols(self):
+i = 0
+while i < self._header.symbol_count:
+symbol = self._symbols[i]
+yield Symbol(self, symbol)
+i += symbol.auxiliary_count + 1
+
+def get_section_data(self, number):
+# section numbers are 1-based
+section = self._sections[number - 1]
+base = section.da

[dpdk-dev] [PATCH 2/4] pmdinfogen: allow multiple input files

2020-12-12 Thread Dmitry Kozlyuk

Process any number of input object files and write a unified output.

Signed-off-by: Dmitry Kozlyuk 
---
Used in the next patch, separated for clarity.

 buildtools/pmdinfogen.py | 14 ++
 1 file changed, 10 insertions(+), 4 deletions(-)

diff --git a/buildtools/pmdinfogen.py b/buildtools/pmdinfogen.py
index 65d78b872..3209510eb 100755
--- a/buildtools/pmdinfogen.py
+++ b/buildtools/pmdinfogen.py
@@ -194,7 +194,9 @@ def dump_drivers(drivers, file):
 def parse_args():
 parser = argparse.ArgumentParser()
 parser.add_argument("format", help="object file format, 'elf' or 'coff'")
-parser.add_argument("input", help="input object file path or '-' for 
stdin")
+parser.add_argument(
+"input", nargs='+', help="input object file path or '-' for stdin"
+)
 parser.add_argument("output", help="output C file path or '-' for stdout")
 return parser.parse_args()
 
@@ -230,10 +232,14 @@ def open_output(path):
 
 def main():
 args = parse_args()
-image = load_image(args.format, args.input)
-drivers = load_drivers(image)
+if args.input.count('-') > 1:
+raise Exception("'-' input cannot be used multiple times")
+
 output = open_output(args.output)
-dump_drivers(drivers, output)
+for path in args.input:
+image = load_image(args.format, path)
+drivers = load_drivers(image)
+dump_drivers(drivers, output)
 
 
 if __name__ == "__main__":
-- 
2.29.2

[dpdk-dev] [PATCH 3/4] buildtools: support object file extraction for Windows

2020-12-12 Thread Dmitry Kozlyuk

clang archiver tool is llvm-ar on Windows and ar on other platforms.
MinGW always uses ar. Replace shell script (Unix-only) that calls ar
with a Python script (OS-independent) that calls an appropriate archiver
tool selected at configuration time. Move the logic not to generate
empty sources into pmdinfogen.

Signed-off-by: Dmitry Kozlyuk 
---
Stdin and stdout are not longer used for input and output. Code to
handle that could be removed, but maybe it's useful for someone.

 buildtools/gen-pmdinfo-cfile.py | 19 +++
 buildtools/gen-pmdinfo-cfile.sh | 14 --
 buildtools/meson.build  | 10 --
 buildtools/pmdinfogen.py|  7 +++
 4 files changed, 34 insertions(+), 16 deletions(-)
 create mode 100644 buildtools/gen-pmdinfo-cfile.py
 delete mode 100755 buildtools/gen-pmdinfo-cfile.sh

diff --git a/buildtools/gen-pmdinfo-cfile.py b/buildtools/gen-pmdinfo-cfile.py
new file mode 100644
index 0..f1f289ffe
--- /dev/null
+++ b/buildtools/gen-pmdinfo-cfile.py
@@ -0,0 +1,19 @@
+#!/usr/bin/env python3
+# SPDX-License-Identifier: BSD-3-Clause
+# Copyright (c) 2020 Dmitry Kozlyuk 
+
+import os
+import subprocess
+import sys
+import tempfile
+
+_, ar, archive, output, *pmdinfogen = sys.argv
+with tempfile.TemporaryDirectory() as temp:
+proc = subprocess.run(
+# Don't use "ar p", because its output is corrupted on Windows.
+[ar, "xv", os.path.abspath(archive)], capture_output=True, check=True, 
cwd=temp
+)
+lines = proc.stdout.decode().splitlines()
+names = [line[len("x - ") :] for line in lines]
+paths = [os.path.join(temp, name) for name in names]
+subprocess.run(pmdinfogen + paths + [output], check=True)
diff --git a/buildtools/gen-pmdinfo-cfile.sh b/buildtools/gen-pmdinfo-cfile.sh
deleted file mode 100755
index 109ee461e..0
--- a/buildtools/gen-pmdinfo-cfile.sh
+++ /dev/null
@@ -1,14 +0,0 @@
-#! /bin/sh
-# SPDX-License-Identifier: BSD-3-Clause
-# Copyright(c) 2017 Intel Corporation
-
-arfile=$1
-output=$2
-shift 2
-pmdinfogen=$*
-
-# The generated file must not be empty if compiled in pedantic mode
-echo 'static __attribute__((unused)) const char *generator = "'$0'";' > $output
-for ofile in `ar t $arfile` ; do
-   ar p $arfile $ofile | $pmdinfogen - - >> $output
-done
diff --git a/buildtools/meson.build b/buildtools/meson.build
index 23cefd4be..0a2e91a7b 100644
--- a/buildtools/meson.build
+++ b/buildtools/meson.build
@@ -2,7 +2,6 @@
 # Copyright(c) 2017-2019 Intel Corporation
 
 pkgconf = find_program('pkg-config', 'pkgconf', required: false)
-pmdinfo = find_program('gen-pmdinfo-cfile.sh')
 list_dir_globs = find_program('list-dir-globs.py')
 check_symbols = find_program('check-symbols.sh')
 ldflags_ibverbs_static = find_program('options-ibverbs-static.sh')
@@ -18,11 +17,18 @@ endif
 map_to_win_cmd = py3 + files('map_to_win.py')
 sphinx_wrapper = py3 + files('call-sphinx-build.py')
 
-# select object file format
+# select library and object file format
+pmdinfo = py3 + files('gen-pmdinfo-cfile.py')
 pmdinfogen = py3 + files('pmdinfogen.py')
 if host_machine.system() == 'windows'
+   if cc.get_id() == 'gcc'
+   pmdinfo += 'ar'
+   else
+   pmdinfo += 'llvm-ar'
+   endif
pmdinfogen += 'coff'
 else
+   pmdinfo += 'ar'
pmdinfogen += 'elf'
 endif
 
diff --git a/buildtools/pmdinfogen.py b/buildtools/pmdinfogen.py
index 3209510eb..56f5f488c 100755
--- a/buildtools/pmdinfogen.py
+++ b/buildtools/pmdinfogen.py
@@ -230,12 +230,19 @@ def open_output(path):
 return open(path, "w")
 
 
+def write_header(output):
+output.write(
+"static __attribute__((unused)) const char *generator = \"%s\";\n" % 
sys.argv[0]
+)
+
+
 def main():
 args = parse_args()
 if args.input.count('-') > 1:
 raise Exception("'-' input cannot be used multiple times")
 
 output = open_output(args.output)
+write_header(output)
 for path in args.input:
 image = load_image(args.format, path)
 drivers = load_drivers(image)
-- 
2.29.2

[dpdk-dev] [PATCH 4/4] build: enable pmdinfogen for Windows

2020-12-12 Thread Dmitry Kozlyuk

Remove platform restriction when building drivers.

Signed-off-by: Dmitry Kozlyuk 
---
 drivers/meson.build | 26 --
 1 file changed, 12 insertions(+), 14 deletions(-)

diff --git a/drivers/meson.build b/drivers/meson.build
index f49d4f79b..1dfa8738f 100644
--- a/drivers/meson.build
+++ b/drivers/meson.build
@@ -143,20 +143,18 @@ foreach subpath:subdirs
# lib and then running pmdinfogen on the contents of
# that lib. The final lib reuses the object files and
# adds in the new source file.
-   if not is_windows
-   out_filename = lib_name + '.pmd.c'
-   tmp_lib = static_library('tmp_' + lib_name,
-   sources,
-   include_directories: includes,
-   dependencies: static_deps,
-   c_args: cflags)
-   objs += tmp_lib.extract_all_objects()
-   sources = custom_target(out_filename,
-   command: [pmdinfo, 
tmp_lib.full_path(),
-   '@OUTPUT@', pmdinfogen],
-   output: out_filename,
-   depends: [tmp_lib])
-   endif
+   out_filename = lib_name + '.pmd.c'
+   tmp_lib = static_library('tmp_' + lib_name,
+   sources,
+   include_directories: includes,
+   dependencies: static_deps,
+   c_args: cflags)
+   objs += tmp_lib.extract_all_objects()
+   sources = custom_target(out_filename,
+   command: [pmdinfo, tmp_lib.full_path(),
+   '@OUTPUT@', pmdinfogen],
+   output: out_filename,
+   depends: [tmp_lib])
 
# now build the static driver
static_lib = static_library(lib_name,
-- 
2.29.2

Re: [dpdk-dev] [PATCH v3] ip_frag: recalculate data length of fragment

[dpdk-dev] [PATCH v5] ip_frag: remove padding length of fragment

Re: [dpdk-dev] [PATCH] net/i40e: issue with ADD VLAN from Guest

[dpdk-dev] [PATCH v2] net/i40e: issue with ADD VLAN from Guest

[dpdk-dev] [PATCH v2] net/i40e: issue with ADD VLAN from Guest

[dpdk-dev] [PATCH 0/4] pmdinfogen: support Windows

[dpdk-dev] [PATCH 1/4] pmdinfogen: support COFF

[dpdk-dev] [PATCH 2/4] pmdinfogen: allow multiple input files

[dpdk-dev] [PATCH 3/4] buildtools: support object file extraction for Windows

[dpdk-dev] [PATCH 4/4] build: enable pmdinfogen for Windows

10 matches

Site Navigation

Mail list logo

Footer information