[dpdk-dev] ip_pipeline firewall : fragmented ipv4 packets handling

[Shyam Shrivastav]

Again sending this issue as no response on original post. As per my
understanding if we are supporting ACL on tcp/usp port ranges then ipv4
packets must be reassembled before  checking against ACL, then fragmented
if required during forwarding. So there are three cases

1) My understanding is wrong then please correct me.
2) We correct this in examples wherever we are supporting access control on
udp/tcp ports.
3) We document this clearly.


-
Below is my original post

-
Hi All

Filtering based on TCP/UDP fields like src/dest port range works correctly
only on non-fragmented packets , that means reassembly must be done before
packets hit firewall rules table. Also  packets must be fragmented before
transmission if larger than port mtu. This is unsupported currently, any
plans for this in near future?

Regards

[dpdk-dev] ip_pipeline firewall port range filtering

[Shyam Shrivastav]

Hi

I am trying to just allow tcp dest port 80 packets using ip_pipeline
firewall, configured as under


pipeline> p 1 firewall add priority 1 ipv4 0.0.0.0 0 0.0.0.0 0 0 65535 80
80 6 0xF port 0
pipeline> p 1 firewall ls
Prio = 1 (SA = 0.0.0.0/0, DA = 0.0.0.0/0, SP = 0-65535, DP = 80-80, Proto =
6 / 0xf) => Port = 0 (entry ptr = 0x7fddf9f0ff08)
Default rule: DROP

---

but it is not working and all tcp packets are getting dropped. If I
configure dest port range to be wildcard(0-65535) then tcp packets are
allowed


---
pipeline> p 1 firewall add priority 1 ipv4 0.0.0.0 0 0.0.0.0 0 0 65535 0
65535 6 0xF port 0
Prio = 1 (SA = 0.0.0.0/0, DA = 0.0.0.0/0, SP = 0-65535, DP = 0-65535, Proto
= 6 / 0xf) => Port = 0 (entry ptr = 0x7fddf9f0ff08)
pipeline> p 1 firewall ls
Prio = 1 (SA = 0.0.0.0/0, DA = 0.0.0.0/0, SP = 0-65535, DP = 0-65535, Proto
= 6 / 0xf) => Port = 0 (entry ptr = 0x7fddf9f0ff08)
Default rule: DROP


--

Anyone here got specific port filtering work with ip_pipeline firewall?
I am debugging this, meanwhile any help/guidance would be greatly
appreciated.

Thanks and rgds

Re: [dpdk-dev] ip_pipeline firewall port range filtering

[Shyam Shrivastav]

I took older versions dpdk-stable-16.07.2 & dpdk-16.04(almost one year
old), went  through code changes in relevant libraries librte_table(acl
part only) and librte_acl, looks like no changes to acl compilation and
matching part since then. Has anyone got specific port filtering working
with ACLs? Looks like I have to go through ACL compilation/clasify code
which looks quite difficult :(

On Fri, Mar 17, 2017 at 12:31 PM, Shyam Shrivastav <
shrivastav.sh...@gmail.com> wrote:

> Hi
>
> I am trying to just allow tcp dest port 80 packets using ip_pipeline
> firewall, configured as under
> 
> 
> pipeline> p 1 firewall add priority 1 ipv4 0.0.0.0 0 0.0.0.0 0 0 65535 80
> 80 6 0xF port 0
> pipeline> p 1 firewall ls
> Prio = 1 (SA = 0.0.0.0/0, DA = 0.0.0.0/0, SP = 0-65535, DP = 80-80, Proto
> = 6 / 0xf) => Port = 0 (entry ptr = 0x7fddf9f0ff08)
> Default rule: DROP
> 
> ---
>
> but it is not working and all tcp packets are getting dropped. If I
> configure dest port range to be wildcard(0-65535) then tcp packets are
> allowed
>
> 
> ---
> pipeline> p 1 firewall add priority 1 ipv4 0.0.0.0 0 0.0.0.0 0 0 65535 0
> 65535 6 0xF port 0
> Prio = 1 (SA = 0.0.0.0/0, DA = 0.0.0.0/0, SP = 0-65535, DP = 0-65535,
> Proto = 6 / 0xf) => Port = 0 (entry ptr = 0x7fddf9f0ff08)
> pipeline> p 1 firewall ls
> Prio = 1 (SA = 0.0.0.0/0, DA = 0.0.0.0/0, SP = 0-65535, DP = 0-65535,
> Proto = 6 / 0xf) => Port = 0 (entry ptr = 0x7fddf9f0ff08)
> Default rule: DROP
>
> 
> --
>
> Anyone here got specific port filtering work with ip_pipeline firewall?
> I am debugging this, meanwhile any help/guidance would be greatly
> appreciated.
>
> Thanks and rgds
>

[dpdk-dev] Fixed : ip_pipeline firewall port range filtering

[Shyam Shrivastav]

Found the issue,was debugging acl library while issue was with firewall acl
definition/registration, simple fix makes udp/tcp src/dest based port
filtering work with ip_pipeline firewall. Below is git diff, will submit
patch if members suggest . I am very new here ..

Also I was going through low level acl compilation/classify code, taking
time to understand,if some dev/code doc is there please pass it on.


[root@localhost pipeline]# git diff pipeline_firewall_be.c
diff --git a/examples/ip_pipeline/pipeline/pipeline_firewall_be.c
b/examples/ip_pipeline/pipeline/pipeline_firewall_be.c
index b61f303..2980492 100644
--- a/examples/ip_pipeline/pipeline/pipeline_firewall_be.c
+++ b/examples/ip_pipeline/pipeline/pipeline_firewall_be.c
@@ -161,7 +161,7 @@ struct firewall_table_entry {
.type = RTE_ACL_FIELD_TYPE_RANGE,
.size = sizeof(uint16_t),
.field_index = 4,
-   .input_index = 4,
+   .input_index = 3,
.offset = sizeof(struct ether_hdr) +
sizeof(struct ipv4_hdr) +
offsetof(struct tcp_hdr, dst_port),
@@ -221,7 +221,7 @@ struct firewall_table_entry {
.type = RTE_ACL_FIELD_TYPE_RANGE,
.size = sizeof(uint16_t),
.field_index = 4,
-   .input_index = 4,
+   .input_index = 3,
.offset = sizeof(struct ether_hdr) +
SIZEOF_VLAN_HDR +
sizeof(struct ipv4_hdr) +
@@ -282,7 +282,7 @@ struct firewall_table_entry {
.type = RTE_ACL_FIELD_TYPE_RANGE,
.size = sizeof(uint16_t),
.field_index = 4,
-   .input_index = 4,
+   .input_index = 3,
.offset = sizeof(struct ether_hdr) +
SIZEOF_QINQ_HEADER +
sizeof(struct ipv4_hdr) +






On Sat, Mar 18, 2017 at 9:58 PM, Shyam Shrivastav <
shrivastav.sh...@gmail.com> wrote:

> I took older versions dpdk-stable-16.07.2 & dpdk-16.04(almost one year
> old), went  through code changes in relevant libraries librte_table(acl
> part only) and librte_acl, looks like no changes to acl compilation and
> matching part since then. Has anyone got specific port filtering working
> with ACLs? Looks like I have to go through ACL compilation/clasify code
> which looks quite difficult :(
>
>
> On Fri, Mar 17, 2017 at 12:31 PM, Shyam Shrivastav <
> shrivastav.sh...@gmail.com> wrote:
>
>> Hi
>>
>> I am trying to just allow tcp dest port 80 packets using ip_pipeline
>> firewall, configured as under
>> 
>> 
>> pipeline> p 1 firewall add priority 1 ipv4 0.0.0.0 0 0.0.0.0 0 0 65535 80
>> 80 6 0xF port 0
>> pipeline> p 1 firewall ls
>> Prio = 1 (SA = 0.0.0.0/0, DA = 0.0.0.0/0, SP = 0-65535, DP = 80-80,
>> Proto = 6 / 0xf) => Port = 0 (entry ptr = 0x7fddf9f0ff08)
>> Default rule: DROP
>> 
>> ---
>>
>> but it is not working and all tcp packets are getting dropped. If I
>> configure dest port range to be wildcard(0-65535) then tcp packets are
>> allowed
>>
>> 
>> ---
>> pipeline> p 1 firewall add priority 1 ipv4 0.0.0.0 0 0.0.0.0 0 0 65535 0
>> 65535 6 0xF port 0
>> Prio = 1 (SA = 0.0.0.0/0, DA = 0.0.0.0/0, SP = 0-65535, DP = 0-65535,
>> Proto = 6 / 0xf) => Port = 0 (entry ptr = 0x7fddf9f0ff08)
>> pipeline> p 1 firewall ls
>> Prio = 1 (SA = 0.0.0.0/0, DA = 0.0.0.0/0, SP = 0-65535, DP = 0-65535,
>> Proto = 6 / 0xf) => Port = 0 (entry ptr = 0x7fddf9f0ff08)
>> Default rule: DROP
>>
>> 
>> 
>> --
>>
>> Anyone here got specific port filtering work with ip_pipeline firewall?
>> I am debugging this, meanwhile any help/guidance would be greatly
>> appreciated.
>>
>> Thanks and rgds
>>
>
>

Re: [dpdk-dev] Fixed : ip_pipeline firewall port range filtering

[Shyam Shrivastav]

Hi Singh
Yes its a bug,input_indx for 2 byte src/dst fields need to point to same
field,I should have examined this layer more minutely before going into
tabl/acl library code. Also looks like no one used/tested tcp/udp port
filtering part of ip_pipeline firewall yet as it will never work without
this fix.
I will submit the patch this week,meanwhile wanted to publish the info if
someone else stuck on this.

Regards, shyam

On Mon, Mar 20, 2017 at 3:04 PM, Singh, Jasvinder  wrote:

> Hi Shayam,
>
>
> > -Original Message-
> > From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Shyam Shrivastav
> > Sent: Sunday, March 19, 2017 4:11 PM
> > To: dev@dpdk.org
> > Cc: Shyam Shrivastav 
> > Subject: [dpdk-dev] Fixed : ip_pipeline firewall port range filtering
> >
> > Found the issue,was debugging acl library while issue was with firewall
> acl
> > definition/registration, simple fix makes udp/tcp src/dest based port
> filtering
> > work with ip_pipeline firewall. Below is git diff, will submit patch if
> members
> > suggest . I am very new here ..
> >
> > Also I was going through low level acl compilation/classify code, taking
> time
> > to understand,if some dev/code doc is there please pass it on.
> >
> >
> > [root@localhost pipeline]# git diff pipeline_firewall_be.c diff --git
> > a/examples/ip_pipeline/pipeline/pipeline_firewall_be.c
> > b/examples/ip_pipeline/pipeline/pipeline_firewall_be.c
> > index b61f303..2980492 100644
> > --- a/examples/ip_pipeline/pipeline/pipeline_firewall_be.c
> > +++ b/examples/ip_pipeline/pipeline/pipeline_firewall_be.c
> > @@ -161,7 +161,7 @@ struct firewall_table_entry {
> > .type = RTE_ACL_FIELD_TYPE_RANGE,
> > .size = sizeof(uint16_t),
> > .field_index = 4,
> > -   .input_index = 4,
> > +   .input_index = 3,
> > .offset = sizeof(struct ether_hdr) +
> > sizeof(struct ipv4_hdr) +
> > offsetof(struct tcp_hdr, dst_port), @@ -221,7
> +221,7 @@ struct
> > firewall_table_entry {
> > .type = RTE_ACL_FIELD_TYPE_RANGE,
> > .size = sizeof(uint16_t),
> > .field_index = 4,
> > -   .input_index = 4,
> > +   .input_index = 3,
> > .offset = sizeof(struct ether_hdr) +
> > SIZEOF_VLAN_HDR +
> > sizeof(struct ipv4_hdr) + @@ -282,7 +282,7 @@
> struct
> > firewall_table_entry {
> > .type = RTE_ACL_FIELD_TYPE_RANGE,
> > .size = sizeof(uint16_t),
> > .field_index = 4,
> > -   .input_index = 4,
> > +   .input_index = 3,
> > .offset = sizeof(struct ether_hdr) +
> > SIZEOF_QINQ_HEADER +
> > sizeof(struct ipv4_hdr) +
>
>
> Looks like a bug as all the fields except the first one has to be grouped
> into sets of 4 consecutive bytes. Therefore, src & dst ports fields should
> be grouped under input_index =3, Reference -http://dpdk.org/doc/guides/
> prog_guide/packet_classif_access_ctrl.html
>
> Please send  a patch on this fix on dpdk.org. Guidelines to send patch on
> dpdk.org are available at http://dpdk.org/doc/guides/
> contributing/patches.html#sending-patches
>
> Thanks,
> Jasvinder
>

Re: [dpdk-dev] [PATCH v2] pipeline_firewall: Fix for port range filtering

[Shyam Shrivastav]

> Firewall ACL definition need to use same input index for source and
> > destination ports as these are 16 bits and would fit in one ACL
> > field of 32 bits. This is required as per librte_acl API. Without this
> > UDP/TCP source and destination ports filtering (and for that
> > matter ICMP type/code filtering) does not work.
> >
> > Signed-off-by: Shyam Kumar Shrivastav 
> > ---
>
> Reviewed-by: jasvinder.si...@intel.com
> Acked-by: cristian.dumitre...@intel.com
>
> Thanks, Shyam, for debugging this!
>

Tested-by: Shyam Kumar Shrivastav 

[dpdk-dev] ip_pipeline firewall customization

[Shyam Shrivastav]

Hi All

I am using ip_pipeline firewall as base for our project, need
comments/suggestions/corrections regarding following

1) We can not configure firewall  rule to drop packets, as portid is
mandatory in command. I am planning to allow this for our requirement with
following code changes
  a) Allow "port" as optional parameter (pipeline_firewall.c), pass -1
as port id if "port" is not specified to app_pipeline_firewall_add_rule and
change that parameter to int32_t.
  b) Make required changes in pipeline_firewall_msg_req_add_handler if
portid is -1, that is table entry action to be .action =
RTE_PIPELINE_ACTION_DROP.

2) I am registering a f_action_hit function for firewall table to perform
certain translations if action is pass (RTE_PIPELINE_ACTION_PORT).
Configured a rule like following

pipeline>p 1 firewall add priority 1 ipv4 0.0.0.0 0 0.0.0.0 0 0 65535 0
65535 0 0 port 0

which should be hit only by ipv4 packets. However even ARP packets are hit
by this ACL and my routine is called. If I configure a specific src or dst
ip then everything works fine and arp packets are not hit , for example
following rule hits only ipv4 icmp packets

pipeline>  p 1 firewall add priority 1 ipv4 0.0.0.0 0 45.35.70.12 32 0
65535 0 65535 1 0xf port 0

Is this a bug or am I missing something ?


Thanks
Shyam

Re: [dpdk-dev] ip_pipeline firewall customization

[Shyam Shrivastav]

Hi Cristian

Please see my comments inline (in blue)

>   b) Make required changes in pipeline_firewall_msg_req_add_handler if
> > portid is -1, that is table entry action to be .action =
> > RTE_PIPELINE_ACTION_DROP.
> >
>
> You don't really need to do this for implementing a drop action. You can
> simply create a SINK output port (which basically drops all the packets
> directed to it) and set this as the output port for all rules that drop
> packet.
>
> See ip_pipeline/config/firewall.cfg as example.
>

Yes dropping of packets can be achieved by creating a sink port. However we
need further processing to be done for a packet hitting a pass rule, and
best way for this processing is f_action_hit table handler.  In this
routine we can distinguish between drop/pass actions as
rte_pipeline_table_entry is passed to it, which also contains  portid but
no generic way to  distinguish between normal or sink port.
I think it would be value addition to allow this action in firewall
pipeline, please let me know your thoughts. As for our project looks like
we have to include this.


>
> > 2) I am registering a f_action_hit function for firewall table to perform
> > certain translations if action is pass (RTE_PIPELINE_ACTION_PORT).
>
> What type of actions are you performing? If generic enough, it would be
> interesting to add them to this pipeline, so I encourage you to contribute
> with ideas and code patches.
>
>
Probably it's not generic, involves vlan translation and mac swapping to
inject the "passed" packets back to the router to travel to next hop.
Router injects traffic for filtering to directly connected firewall port
with vlan tag say A , firewall in turn injects pass traffic with different
vlan tag say B back to the router which then does next hop processing.


> >
> > Is this a bug or am I missing something ?
> >
>
> You can make sure no ARP packets are received by the firewall pipeline by
> simply filtering all the ARP packets to a separate RXQ of the NIC port,
> which can be further handled by a separate function.
>
> See ip_pipeline/config/network_layers.cfg as example:
> [LINK0]
> arp_q = 4
>

There are two things here

1) ARP packets should not hit the ipv4 acl which looks like happening, have
not worked on "why part" for now , need to look at the
rte_table_acl_lookup->rte_acl_classify.

2) Yes we can filter ARP packets to a separate queue and to separate lcore
at link level, but need to decide whether its worth it. We are planning to
use ipv4 RSS (with 8 queues, lcores and processors) wherein all arp packets
would get filtered to queue 0 by default and then dropped by ACL.


Thanks and regards
Shyam

Re: [dpdk-dev] ip_pipeline firewall customization

[Shyam Shrivastav]

My mistake, arp packets are hit by

pipeline>p 1 firewall add priority 1 ipv4 0.0.0.0 0 0.0.0.0 0 0 65535 0
65535 0 0 port 0

ACL as anything and rightly so gets matched for above fields. Only way is
to somehow avoid ARP packets being input to table itself, and yes one way
is to direct them to a separate queue .Another is  explicitly filtering
them before or after  firewall table look up, firewall table is supposed to
be only for ipv4/ipv6 packets have to work/think more on this

On Thu, Mar 9, 2017 at 5:13 PM, Shyam Shrivastav  wrote:

> Hi Cristian
>
> Please see my comments inline (in blue)
>
> >   b) Make required changes in pipeline_firewall_msg_req_add_handler
>> if
>> > portid is -1, that is table entry action to be .action =
>> > RTE_PIPELINE_ACTION_DROP.
>> >
>>
>> You don't really need to do this for implementing a drop action. You can
>> simply create a SINK output port (which basically drops all the packets
>> directed to it) and set this as the output port for all rules that drop
>> packet.
>>
>> See ip_pipeline/config/firewall.cfg as example.
>>
>
> Yes dropping of packets can be achieved by creating a sink port. However
> we need further processing to be done for a packet hitting a pass rule, and
> best way for this processing is f_action_hit table handler.  In this
> routine we can distinguish between drop/pass actions as
> rte_pipeline_table_entry is passed to it, which also contains  portid but
> no generic way to  distinguish between normal or sink port.
> I think it would be value addition to allow this action in firewall
> pipeline, please let me know your thoughts. As for our project looks like
> we have to include this.
>
>
>>
>> > 2) I am registering a f_action_hit function for firewall table to
>> perform
>> > certain translations if action is pass (RTE_PIPELINE_ACTION_PORT).
>>
>> What type of actions are you performing? If generic enough, it would be
>> interesting to add them to this pipeline, so I encourage you to contribute
>> with ideas and code patches.
>>
>>
> Probably it's not generic, involves vlan translation and mac swapping to
> inject the "passed" packets back to the router to travel to next hop.
> Router injects traffic for filtering to directly connected firewall port
> with vlan tag say A , firewall in turn injects pass traffic with different
> vlan tag say B back to the router which then does next hop processing.
>
>
>> >
>> > Is this a bug or am I missing something ?
>> >
>>
>> You can make sure no ARP packets are received by the firewall pipeline by
>> simply filtering all the ARP packets to a separate RXQ of the NIC port,
>> which can be further handled by a separate function.
>>
>> See ip_pipeline/config/network_layers.cfg as example:
>> [LINK0]
>> arp_q = 4
>>
>
> There are two things here
>
> 1) ARP packets should not hit the ipv4 acl which looks like happening,
> have not worked on "why part" for now , need to look at the
> rte_table_acl_lookup->rte_acl_classify.
>
> 2) Yes we can filter ARP packets to a separate queue and to separate lcore
> at link level, but need to decide whether its worth it. We are planning to
> use ipv4 RSS (with 8 queues, lcores and processors) wherein all arp packets
> would get filtered to queue 0 by default and then dropped by ACL.
>
>
> Thanks and regards
> Shyam
>
>
>

Re: [dpdk-dev] ip_pipeline firewall customization

[Shyam Shrivastav]

Hi Singh

Please find my comments inline


>
> [Jasvinder] -  You need to set the right proto field and accordingly the
> mask in the above rule. You can keep src/dst ip addresses intact. Proto
> field defines the protocol used in the data portion of the IP datagram and
> can be find here
> . For e.g. in
> case you packet is TCP packet then firewall rule should be as below and you
> will see arp packets filtered out.
>
>
>
> p 1 firewall add priority 1 ipv4 0.0.0.0 0 0.0.0.0 0 0 65535 0 65535 6 0x6
> port 0
>
>

[Shyam]Yes it works already saw that, simple reason that it would be
rare for an arp packet to have exactly same value at that offset (9 which
contains mac address). Still in my view ARP packets must be filtered out
and must never hit firewall/ACL table.




>
>
>
>
> Yes dropping of packets can be achieved by creating a sink port. However
> we need further processing to be done for a packet hitting a pass rule, and
> best way for this processing is f_action_hit table handler.  In this
> routine we can distinguish between drop/pass actions as
> rte_pipeline_table_entry is passed to it, which also contains  portid but
> no generic way to  distinguish between normal or sink port.
> I think it would be value addition to allow this action in firewall
> pipeline, please let me know your thoughts. As for our project looks like
> we have to include this.
>
>
>
> [Jasvinder] -  If I understand your context correctly, you want to
> distinguish Sink port from other normal out ports in the action handler.
> Can do that by keeping sink port last among the pipeline output  ports and
> then in action handler you can compare the port id (highest value)  to see
> whether the port id is sink port or not.
>
>
>
[Shyam]  That doesn't seem to be generic at all. Requirement is to be able
to configure and distinguish between pass and drop rules, so that required
translations can be done for pass rules.  Support for drop action exist in
the underlying table/acl code , we just need to change the firewall part.
We should not do this based on port number in my view 


Thanks and rgds
Shyam

[dpdk-dev] ip_pipeline firewall : fragmented packets filtering

[Shyam Shrivastav]

Hi All

Filtering based on TCP/UDP fields like src/dest port range works correctly
only on non-fragmented packets , that means reassembly must be done before
packets hit firewall rules table. Also  packets must be fragmented before
transmission if larger than port mtu. This is unsupported currently, any
plans for this in near future?

Regards
Shyam

Re: [dpdk-dev] DPDK17.02 flow classification can not take effect

[Shyam Shrivastav]

I am using 16.11 and noticed the same thing, mbuf->packet_type is not
populated  == 0 .

On Thu, Mar 9, 2017 at 12:57 PM, Chillance Zen 
wrote:

> previous 16.07 works well , when I examine the rte_mbuf->packet_type,I can
> get the well-corresponded packet type. after migrating code to 17.02 ,I
> will never get packet_type again, should I switch on something ,or did I
> miss any configuration ?
>
> btw,how could I judge whether the nic support packet type
> classification?(since there is no bit indicating this offloading feature in
> ol_flags)
>
> Thanks&Regards
> Linc
>

Re: [dpdk-dev] how to build 'example' folder in dpdk-2.2.0?

[Shyam Shrivastav]

For linux

http://dpdk.org/doc/guides/linux_gsg/index.html
http://dpdk.org/doc/guides/linux_gsg/build_dpdk.html
http://dpdk.org/doc/guides/linux_gsg/build_sample_apps.html

On Fri, Jun 9, 2017 at 8:31 AM, Sam  wrote:

> hi all,
>
> I want to build example(DPDK_HOME/example/*) in dpdk, and to have a look at
> vhost demo. But I can't find guide in dpdk home page or document.
>
> So is there some document to tell me HOWTO?
>

Re: [dpdk-dev] Printing statistics on running passthrough pipeline in ip_pipeline application

[Shyam Shrivastav]

Enable flags in dpdk/config/common_base and recompile

CONFIG_RTE_*_STATS_COLLECT=y


{* for PIPELINE,PORT,TABLE ...}


On Fri, Apr 21, 2017 at 11:58 AM, Nidhia Varghese <
nidhiavarghes...@gmail.com> wrote:

> Hi,
>
> I am running ip_pipeline application using ip_pipeline.cfg file. How can I
> see the statistics of the packets that is being sent and received from this
> pipeline?
>
> I tried running as shown below:
>
> pipeline> p 1 port in 0 enable
> pipeline> p 1 port in 1 enable
> pipeline> p 1 stats port in 1
> Pipeline 1 - stats for input port 1:
> Pkts in: 0
> Pkts dropped by AH: 0
> Pkts dropped by other: 0
> pipeline> p 1 stats port in 0
> Pipeline 1 - stats for input port 0:
> Pkts in: 0
> Pkts dropped by AH: 0
> Pkts dropped by other: 0
> pipeline> p 1 stats port out 0
> Pipeline 1 - stats for output port 0:
> Pkts in: 0
> Pkts dropped by AH: 0
> Pkts dropped by other: 0
> pipeline> p 1 stats port out 1
> Pipeline 1 - stats for output port 1:
> Pkts in: 0
> Pkts dropped by AH: 0
> Pkts dropped by other: 0
>
> I can see the packet being received and forwarded using wireshark. But not
> able to see in the statistics. Isn't this the right command to use?
>
> Thanks for your reply and help.
>
> Regards,
> Nidhia Varghese
>

Re: [dpdk-dev] [dpdk-users] Fwd: Sharing tables among pipelines

[Shyam Shrivastav]

Something I have gathered reading through code if it helps, ip_pipeline is
designed such that user side cli (_fe) runs in master pipeline and sends
conf requests to various other packet processing pipelines (_be) which
handle such requests at intervals. These can run on same or different
threads(cores) as per conf, no locking requirement as table-updation &
table-reading (packet processing) always happens in same pipeline so always
same thread. Master pipeline can be configured to run on a separate core &
it should be the case, but table updation happens in pipeline thread only
at lower priority.



On Tue, May 9, 2017 at 11:29 PM, Nidhia Varghese  wrote:

> Hi all,
>
> Thanks for your reply.
>
> As given in the link(3rd point), what I need is single writer
> thread(pipeline) performing table entry add/delete operations and another
> thread(which is in turn another pipeline) that performs lookup operations
> to the same table entries.
>
>
> Thanks,
> Nidhia Varghese
>

Re: [dpdk-dev] Defining multiple actions based on a field value for the same table lookup in ip_pipeline application

[Shyam Shrivastav]

For each table a lookup hit and miss function can be registered, have a
look at
rte_pipeline_run(struct rte_pipeline *p) for the semantics

On Thu, May 18, 2017 at 5:53 PM, Nidhia Varghese  wrote:

> Hi all,
>
> I am trying to design a pipeline where the table lookup has to do the
> actions as follows:
>
> Lookup hit -> FWD to port
> Lookup miss -> add a table entry if incoming port was 1
>  drop if incoming port was 2
>
> Is it possible to do such an implementation using ip_pipeline application?
> If so, how can I do that?
>
> Thanks for your reply and help.
>
> Thanks,
> Nidhia Varghese
>

Re: [dpdk-dev] Defining multiple actions based on a field value for the same table lookup in ip_pipeline application

[Shyam Shrivastav]

That can be done in the same miss function as I understand, if (condition1)
{} else {}

On Fri, May 19, 2017 at 10:53 AM, Nidhia Varghese <
nidhiavarghes...@gmail.com> wrote:

> Thanks for your reply Shyam.
> Yes, I went through it. But my doubt is whether it is possible to define
> two different miss actions based on some decision criteria(input port in my
> case).
> Miss ->Case 1: Add entry to table if portid = 1
> ->Case 2: Drop if portid = 2
>
> On Thu, May 18, 2017 at 7:34 PM, Shyam Shrivastav <
> shrivastav.sh...@gmail.com> wrote:
>
>> For each table a lookup hit and miss function can be registered, have a
>> look at
>> rte_pipeline_run(struct rte_pipeline *p) for the semantics
>>
>
> Thanks,
> Nidhia Varghese
>

Re: [dpdk-dev] About the data payload of rte_mbuf?

[Shyam Shrivastav]

As per my understanding

payload = (uint8_t *)tcp_hdr + (tcp_hdr->data_off << 2);

should be

payload = (uint8_t *)tcp_hdr + (tcp_hdr->data_off >> 4) << 2);

Note that data offset is 4 most significant bits of the byte, refer tcp
header



On Thu, Dec 6, 2018 at 9:15 AM bai bakari <912873...@qq.com> wrote:

> Hi,
>
>
> Now, I want to get the data payload of rte_mbuf, and i wrote the following
> code:
>
>
> struct ipv4_hdr *ipv4_hdr;
> struct tcp_hdr *tcp_hdr;
> uint32_t payload_len, ip_len;
> uint8_t *payload = NULL;
>
>
>
> ipv4_hdr = rte_pktmbuf_mtod_offset(m, struct ipv4_hdr *, sizeof(struct
> ether_hdr));
> ip_len = ntohs(ipv4_hdr->total_length);
>
>
>
> if (ipv4_hdr->next_proto_id == IPPROTO_TCP) {
> tcp_hdr = (struct tcp_hdr *)((uint8_t *)ipv4_hdr +
> ((ipv4_hdr->version_ihl & 0xf) << 2));
> payload = (uint8_t *)tcp_hdr + (tcp_hdr->data_off << 2);
> payload_len = ip_len - (payload - (uint8_t *)ipv4_hdr);
>
> }
>
>
> when i send packets using dpdk-pktgen, i found:
> ip_len = 46
> ip_header_len = 20
> but the tcp_header_len=(tcp_hdr->data_off << 2)=0,
> and (payload - (uint8_t *)ipv4_hdr) = 340 > ip_len.
>
>
> I'm confused, is there any errors about the code to compute the
> payload_len of rte_mbuf?
> I think maybe the dpdk-pktgen cannot send the packets with payload?
>
>
> I'm a beginner, anyone can help me how to compute the payload_len of
> rte_mbuf and test it?
>
>
> Thank you in advance!

Re: [dpdk-dev] About the data payload of rte_mbuf?

[Shyam Shrivastav]

And if just payload length is required one can get by subtracting
(ip-hdr-len + tcp-offset) from total length in ip header ...

On Thu, Dec 6, 2018 at 11:18 AM Shyam Shrivastav 
wrote:

>
> As per my understanding
>
> payload = (uint8_t *)tcp_hdr + (tcp_hdr->data_off << 2);
>
> should be
>
> payload = (uint8_t *)tcp_hdr + (tcp_hdr->data_off >> 4) << 2);
>
> Note that data offset is 4 most significant bits of the byte, refer tcp
> header
>
>
>
> On Thu, Dec 6, 2018 at 9:15 AM bai bakari <912873...@qq.com> wrote:
>
>> Hi,
>>
>>
>> Now, I want to get the data payload of rte_mbuf, and i wrote the
>> following code:
>>
>>
>> struct ipv4_hdr *ipv4_hdr;
>> struct tcp_hdr *tcp_hdr;
>> uint32_t payload_len, ip_len;
>> uint8_t *payload = NULL;
>>
>>
>>
>> ipv4_hdr = rte_pktmbuf_mtod_offset(m, struct ipv4_hdr *, sizeof(struct
>> ether_hdr));
>> ip_len = ntohs(ipv4_hdr->total_length);
>>
>>
>>
>> if (ipv4_hdr->next_proto_id == IPPROTO_TCP) {
>> tcp_hdr = (struct tcp_hdr *)((uint8_t *)ipv4_hdr +
>> ((ipv4_hdr->version_ihl & 0xf) << 2));
>> payload = (uint8_t *)tcp_hdr + (tcp_hdr->data_off << 2);
>> payload_len = ip_len - (payload - (uint8_t *)ipv4_hdr);
>>
>> }
>>
>>
>> when i send packets using dpdk-pktgen, i found:
>> ip_len = 46
>> ip_header_len = 20
>> but the tcp_header_len=(tcp_hdr->data_off << 2)=0,
>> and (payload - (uint8_t *)ipv4_hdr) = 340 > ip_len.
>>
>>
>> I'm confused, is there any errors about the code to compute the
>> payload_len of rte_mbuf?
>> I think maybe the dpdk-pktgen cannot send the packets with payload?
>>
>>
>> I'm a beginner, anyone can help me how to compute the payload_len of
>> rte_mbuf and test it?
>>
>>
>> Thank you in advance!
>
>

Re: [dpdk-dev] Build DPDK for different target machine.

[Shyam Shrivastav]

I am doing same but including dpdk libraries with application, you can do
same. Other thing to try, copy the shared libraries from the build machine
into targets instead of building them on targets.

On Fri, Oct 19, 2018 at 9:55 PM Shubhachint, Chaitanya <
chaitanya.shubhach...@netscout.com> wrote:

> Hello,
>
> I am trying to build DPDK library set (shared) so that there are somewhat
> portable. I want to build them once on a designated build machine and run
> them on other machines of similar architecture. I can build DPDK and use it
> locally successfully but running them on a different target machine always
> gives me core dump with 'Illegal instruction'.
>
> I am building with
> make install T=x86_64-native-linuxapp-gcc
> export RTE_TARGET=x86_64-native-linuxapp-gcc
> and CONFIG_RTE_BUILD_SHARED_LIB=y
>
> I have tried setting CONFIG_RTE_MACHINE, as "default" or "snb" but that
> does not help.
>
> Help is much appreciated. Details of Build machine and target machine
> below:
>
> Thanks
> Chai.
>
> Build machine:
> DPDK_Build/dpdk-18.08# lscpu
> Architecture:  x86_64
> CPU op-mode(s):32-bit, 64-bit
> Byte Order:Little Endian
> CPU(s):8
> On-line CPU(s) list:   0-7
> Thread(s) per core:1
> Core(s) per socket:8
> Socket(s): 1
> NUMA node(s):  1
> Vendor ID: GenuineIntel
> CPU family:6
> Model: 79
> Model name:Intel(R) Xeon(R) CPU E5-2640 v4 @ 2.40GHz
> Stepping:  1
> CPU MHz:   2399.998
> BogoMIPS:  4799.99
> Hypervisor vendor: VMware
> Virtualization type:   full
> L1d cache: 32K
> L1i cache: 32K
> L2 cache:  256K
> L3 cache:  25600K
> NUMA node0 CPU(s): 0-7
> Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr
> pge mca cmov pat pse36 clflush dts mmx fxsr sse sse2 ss ht syscall nx
> pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts nopl xtopology
> tsc_reliable nonstop_tsc cpuid aperfmperf pni pclmulqdq ssse3 fma cx16 pcid
> sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c
> rdrand hypervisor lahf_lm abm 3dnowprefetch cpuid_fault epb invpcid_single
> pti fsgsbase tsc_adjust bmi1 hle avx2 smep bmi2 invpcid rtm rdseed adx smap
> xsaveopt dtherm ida arat pln pts
>
>
> Target Machine:
> $lscpu
> Architecture:  x86_64
> CPU op-mode(s):32-bit, 64-bit
> Byte Order:Little Endian
> CPU(s):6
> On-line CPU(s) list:   0-5
> Thread(s) per core:1
> Core(s) per socket:6
> Socket(s): 1
> NUMA node(s):  1
> Vendor ID: GenuineIntel
> CPU family:6
> Model: 45
> Model name:Intel(R) Xeon(R) CPU E5-2667 0 @ 2.90GHz
> Stepping:  7
> CPU MHz:   1682.069
> CPU max MHz:   3500.
> CPU min MHz:   1200.
> BogoMIPS:  5785.61
> Virtualization:VT-x
> L1d cache: 32K
> L1i cache: 32K
> L2 cache:  256K
> L3 cache:  15360K
> NUMA node0 CPU(s): 0-5
> Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr
> pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
> syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good
> nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor
> ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic
> popcnt tsc_deadline_timer aes xsave avx lahf_lm epb pti tpr_shadow vnmi
> flexpriority ept vpid xsaveopt ibpb ibrs stibp dtherm ida arat pln pts
>
>

Re: [dpdk-dev] IPV4/IPV6 TCP/UDP Pseudo Header Checksum APIs

[Shyam Shrivastav]

that is correct , pseudo header doesn't include ipv4 options or ipv6
extension headers ..

On Sat, Oct 20, 2018 at 9:02 AM lidejun  wrote:

> Has anybody used the following two APIs calculating ipv4&ipv6 tcp/udp
> pseudo header checksum？
>
> 1.rte_ipv4_phdr_cksum
>
> 2.rte_ipv6_phdr_cksum
> The ipv4 version does not exclude ip options and ipv6 version does not
> exclude extersion headers.
>

Re: [dpdk-dev] IPV4/IPV6 TCP/UDP Pseudo Header Checksum APIs

[Shyam Shrivastav]

Realized my answer is confusing, I meant to say that code is correct as
pseudo ipv4/ipv6 headers for the purpose of checksum calculations doesn't
include options or extension headers, see udp wiki or corresponding rfcs

https://en.wikipedia.org/wiki/User_Datagram_Protocol

On Sat, Oct 20, 2018 at 10:42 AM Shyam Shrivastav <
shrivastav.sh...@gmail.com> wrote:

> that is correct , pseudo header doesn't include ipv4 options or ipv6
> extension headers ..
>
> On Sat, Oct 20, 2018 at 9:02 AM lidejun  wrote:
>
>> Has anybody used the following two APIs calculating ipv4&ipv6 tcp/udp
>> pseudo header checksum？
>>
>> 1.rte_ipv4_phdr_cksum
>>
>> 2.rte_ipv6_phdr_cksum
>> The ipv4 version does not exclude ip options and ipv6 version does not
>> exclude extersion headers.
>>
>

Re: [dpdk-dev] IPV4/IPV6 TCP/UDP Pseudo Header Checksum APIs

[Shyam Shrivastav]

Yes you are right, I misread, following code (ipv4 case) assumes no ip
options while calculating pseudo hdr length field

psd_hdr.len = rte_cpu_to_be_16(
(uint16_t)(rte_be_to_cpu_16(ipv4_hdr->total_length)
- sizeof(struct ipv4_hdr)));

should be

 psd_hdr.len = rte_cpu_to_be_16(
(uint16_t)(rte_be_to_cpu_16(ipv4_hdr->total_length)
- (ipv4_hdr->version_ihl & 0x0f)*4)));

On Sat, Oct 20, 2018 at 11:44 AM lidejun  wrote:

> I mean DPDK APIs do not exclude ipv4 options or ipv6 extension headers, it
> is bug?
>
>
>
> *发件人:* Shyam Shrivastav [mailto:shrivastav.sh...@gmail.com]
> *发送时间:* 2018年10月20日 13:23
> *收件人:* lidejun 
> *抄送:* users ; dev@dpdk.org; Lichunhe (Cloud Networking) <
> lichu...@huawei.com>; Wangliefeng 
> *主题:* Re: [dpdk-dev] IPV4/IPV6 TCP/UDP Pseudo Header Checksum APIs
>
>
>
> Realized my answer is confusing, I meant to say that code is correct as
> pseudo ipv4/ipv6 headers for the purpose of checksum calculations doesn't
> include options or extension headers, see udp wiki or corresponding rfcs
>
>
>
> https://en.wikipedia.org/wiki/User_Datagram_Protocol
>
>
>
> On Sat, Oct 20, 2018 at 10:42 AM Shyam Shrivastav <
> shrivastav.sh...@gmail.com> wrote:
>
> that is correct , pseudo header doesn't include ipv4 options or ipv6
> extension headers ..
>
>
>
> On Sat, Oct 20, 2018 at 9:02 AM lidejun  wrote:
>
> Has anybody used the following two APIs calculating ipv4&ipv6 tcp/udp
> pseudo header checksum？
>
> 1.rte_ipv4_phdr_cksum
>
> 2.rte_ipv6_phdr_cksum
> The ipv4 version does not exclude ip options and ipv6 version does not
> exclude extersion headers.
>
>

Re: [dpdk-dev] [dpdk-users] IPV4/IPV6 TCP/UDP Pseudo Header Checksum APIs

[Shyam Shrivastav]

These fixes/modifications should include the upper level APIs,
rte_ipv4_udptcp_cksum and rte_ipv6_udptcp_cksum.

Even for ipv4 following API is more/really useful if changed to take mbufs

rte_ipv4_udptcp_cksum(const struct ipv4_hdr *ipv4_hdr, const void *l4_hdr)

I can not use it in present form as it requires single contiguous l4
buffer, instead using rte_raw_cksum_mbuf.

Re: [dpdk-dev] Segmentation fault when creating a hash table in the example of pipeline (ROUTING type)

[Shyam Shrivastav]

I have been using librte_pipeline, librte_table though only acl and
hash_key16 part, and firewall part of ip_pipeline in my project for quite
sometime. I have never used or worked on other parts of ip_pipeline app
like routing.
Obviously issue is due to bad memory access at some point. Here are my
steps if it helps.
If unable to figure out , I usually replace -O3 with -g (that is enable
debugging symbols and disable optimization) in makefiles of these two
libraries and compile them. Then do the same thing with ip_pipeline
makefile and compile. And thats it use gdb and you can get the exact
culprit line

On Thu, Apr 19, 2018 at 6:03 PM, Victor Huertas  wrote:

> Hi all,
>
> I have tried to run the pipeline example with a ROUTING pipeline where I
> configured n_arp_entries=8 as well as the arp_key_offset = 192.
>
> It compiles perfectly but when it reaches the line where the
> rte_pipeline_table_create is (see below a fragment of the source code of
> the pipeline_routing_init function at pipeline_routing.be.cpp file) it
> always crashes with a segmentation fault at the
> rte_table_hash_create_key8_ext() function.
>
> I am completely stuck due to this issue. Has some bug being declared
> regarding this table creation function?
>
>
> /* ARP table configuration */
> if (p_rt->params.n_arp_entries) {
>
> struct rte_table_hash_key8_ext_params table_arp_params;
> table_arp_params.n_entries = p_rt->params.n_arp_entries;
> table_arp_params.n_entries_ext = p_rt->params.n_arp_entries;
> table_arp_params.f_hash = hash_default_key8;
> table_arp_params.seed = 0;
> table_arp_params.signature_offset = 0; /* Unused */
> table_arp_params.key_offset = p_rt->params.arp_key_offset;
>
>
> struct rte_pipeline_table_params table_params = {
> .ops = &rte_table_hash_key8_ext_dosig_ops,
> .arg_create = &table_arp_params,
> .f_action_hit = get_arp_table_ah_hit(p_rt),
> .f_action_miss = NULL,
> .arg_ah = p_rt,
> .action_data_size = sizeof(struct arp_table_entry) -
> sizeof(struct rte_pipeline_table_entry),
> };
>
> int status;
>
> status = rte_pipeline_table_create(p->p,
> &table_params,
> &p->table_id[1]);
>
> if (status) {
> rte_pipeline_free(p->p);
> rte_free(p);
> return NULL;
> }
>
> p->n_tables++;
> }
>
> Thanks
>
> --
> Victor
>

Re: [dpdk-dev] rte_hash thread safe

[Shyam Shrivastav]

You can look at hash table support under librte_table, we are using it for
tcp stateful  and syncache table (16 bytes key), supports optimized burst
look ups, pretty good performance at line rate with up to half a billion
entries in syn cache lru table and 33 million in flows ext table.
However these tables are pre-allocated and not thread safe (tradeoff with
performance), we use per core tables if running multicore 
*I thank all creators/contributors involved, it definitely been huge help
in my project *


On Tue, Apr 24, 2018 at 8:34 PM, Brijesh Singh 
wrote:

> Thank you all for explaining. My updates are  uncommon; Adding concept
> of quiescent threads should be worst case  loss of 1 full burst cpu
> cycles on the threads. This should be acceptable infrequent delay in
> packet processing.
>
> I need data on performance of librcu lookups under infrequent updates,
> if the difference is not significant on x86, I will use librcu.
>
>
> On Mon, Apr 23, 2018 at 6:14 PM, Stephen Hemminger
>  wrote:
> > On Mon, 23 Apr 2018 17:48:50 -0700
> > Jim Murphy  wrote:
> >
> >> Anecdotally I've heard that the urcu hash implementation is slower than
> >> rte_hash based on pure lookup performance. Has anyone considered adding
> RCU
> >> hooks into rte_hash?
> >
> >
> > Not really possible with DPDK (as I said earlier) because DPDK does not
> have concept
> > of thread quiescent period to allow for safe deletion.  You could
> manually use RCU
> > concepts of RCU and RTE hash; it would require using userspace RCU
> primitives
> > inside DPDK.  This would cause a dependency that would prevent that from
> ever
> > being merged upstream due to license conflict; but since DPDK is liberal
> BSD
> > license you are free to do it and maintain it on your own.
>

Re: [dpdk-dev] Why rte_pipeline.c only support forwarding to same table in different entries?

[Shyam Shrivastav]

Looking at code, another change might make this work. Make sure that
default entry action of each table points to next table except for last one
which normally should be drop

Please look at if (lookup_miss_mask != 0)  handling in rte_pipeline_run ()

.



On Thu, Jun 7, 2018 at 12:49 PM, hongbo liu  wrote:

> Hi,
>
> I am frustrated when designing pipeline table.
>
> I am trying to have one ACL table 0 to forward different traffic to
> different table, but it failed.
>
> And I noticed that the following checks in rte_pipeline.c:
> /
> if ((entry->action == RTE_PIPELINE_ACTION_TABLE) &&
> table->table_next_id_valid &&
> (entry->table_id != table->table_next_id)) {
> RTE_LOG(ERR, PIPELINE,
> "%s: Tree-like topologies not allowed\n", __func__);
> return -EINVAL;
> }
> /
>
> Is there any reason why adding this limitation?
>
> In openflow, the limitation is the forward table_id must be larger than
> current table_id, and the final
> table should not have forward table action.
>
> By the way, it still does not work after removing upper checks.
>
> Is it difficult to support forwarding to multiple tables in different
> entries?
>
> --
> Hobby
>