Bifurcated driver is a mechanism which depends the advanced Ethernet device to split traffic between queues. It provides the capability to let the kernel driver and DPDK driver co-exist and take their advantage. It is achieved by using SRIOV and NIC's advanced filtering. This patch describes it and adds the user guide on ixgbe NICs.
Signed-off-by: Jingjing Wu <jingjing.wu at intel.com> --- doc/guides/nics/img/bifurcated_driver_overview.svg | 544 +++++++++++++++++++++ doc/guides/nics/img/ixgbe_bifu_queue_idx.svg | 101 ++++ doc/guides/nics/ixgbe.rst | 119 +++++ 3 files changed, 764 insertions(+) create mode 100644 doc/guides/nics/img/bifurcated_driver_overview.svg create mode 100644 doc/guides/nics/img/ixgbe_bifu_queue_idx.svg diff --git a/doc/guides/nics/img/bifurcated_driver_overview.svg b/doc/guides/nics/img/bifurcated_driver_overview.svg new file mode 100644 index 0000000..4fa2764 --- /dev/null +++ b/doc/guides/nics/img/bifurcated_driver_overview.svg @@ -0,0 +1,544 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!-- Generated by Microsoft Visio, SVG Export bifurcated_driver_overview.svg Page-1 --> + +<svg + xmlns:dc="http://purl.org/dc/elements/1.1/"; + xmlns:cc="http://creativecommons.org/ns#"; + xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"; + xmlns:svg="http://www.w3.org/2000/svg"; + xmlns="http://www.w3.org/2000/svg"; + xmlns:xlink="http://www.w3.org/1999/xlink"; + xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"; + xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"; + width="6.71874in" + height="4.83839in" + viewBox="0 0 483.75 348.364" + xml:space="preserve" + color-interpolation-filters="sRGB" + class="st28" + id="svg2" + version="1.1" + inkscape:version="0.91 r13725" + sodipodi:docname="bifurcated_driver_overview.svg"><metadata + id="metadata240"><rdf:RDF><cc:Work + rdf:about=""><dc:format>image/svg+xml</dc:format><dc:type + rdf:resource="http://purl.org/dc/dcmitype/StillImage"; /></cc:Work></rdf:RDF></metadata><sodipodi:namedview + pagecolor="#ffffff" + bordercolor="#666666" + borderopacity="1" + objecttolerance="10" + gridtolerance="10" + guidetolerance="10" + inkscape:pageopacity="0" + inkscape:pageshadow="2" + inkscape:window-width="1600" + inkscape:window-height="837" + id="namedview238" + showgrid="false" + inkscape:zoom="1.0517845" + inkscape:cx="215.35622" + inkscape:cy="200.74714" + inkscape:window-x="-8" + inkscape:window-y="-8" + inkscape:window-maximized="1" + inkscape:current-layer="g44" /><style + type="text/css" + id="style4"><![CDATA[ + .st1 {visibility:visible} + .st2 {fill:none;filter:url(#filter_2);stroke:#5b9bd5;stroke-opacity:0.22} + .st3 {fill:#5b9bd5;filter:url(#filter_2);font-family:Calibri;font-size:1.16666em;font-weight:bold;opacity:0.219608} + .st4 {fill:none;stroke:#c7c8c8;stroke-width:0.5} + .st5 {fill:#000000;font-family:Calibri;font-size:1.16666em;font-weight:bold} + .st6 {fill:#5b9bd5;filter:url(#filter_2);font-family:Calibri;font-size:1.5em;opacity:0.219608} + .st7 {fill:#000000;font-family:Calibri;font-size:1.5em} + .st8 {fill:#a8d08d;stroke:#4f87bb;stroke-width:0.75} + .st9 {fill:#000000;font-family:Calibri;font-size:0.833336em} + .st10 {fill:#5b9bd5;fill-opacity:0.22;filter:url(#filter_2);stroke:#5b9bd5;stroke-opacity:0.22} + .st11 {fill:#c00000;stroke:#c7c8c8;stroke-width:0.25} + .st12 {fill:#feffff;font-family:Calibri;font-size:0.833336em} + .st13 {font-size:1em} + .st14 {fill:#ff0000;font-size:1em;font-weight:bold} + .st15 {fill:#2e75b5;stroke:#c7c8c8;stroke-width:0.25} + .st16 {fill:url(#grad4-50);stroke:#c7c8c8;stroke-width:0.25} + .st17 {fill:#feffff;font-family:Calibri;font-size:0.666664em} + .st18 {fill:#5b9bd5;stroke:#c7c8c8;stroke-width:0.25} + .st19 {fill:#000000;font-family:Calibri;font-size:1.16666em} + .st20 {marker-end:url(#mrkr13-84);marker-start:url(#mrkr13-82);stroke:#c00000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st21 {fill:#c00000;fill-opacity:1;stroke:#c00000;stroke-opacity:1;stroke-width:0.28409090909091} + .st22 {marker-end:url(#mrkr4-90);stroke:#c00000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1} + .st23 {marker-start:url(#mrkr13-106);stroke:#538135;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.75} + .st24 {fill:#538135;fill-opacity:1;stroke:#538135;stroke-opacity:1;stroke-width:0.40983606557377} + .st25 {marker-start:url(#mrkr13-112);stroke:#5b9bd5;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.75} + .st26 {fill:#5b9bd5;fill-opacity:1;stroke:#5b9bd5;stroke-opacity:1;stroke-width:0.40983606557377} + .st27 {fill:none;stroke:none;stroke-width:0.25} + .st28 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]></style><defs + id="Patterns_And_Gradients"><radialGradient + id="grad4-50" + cx="0.5" + cy="0" + r="1.1"><stop + offset="0" + stop-color="#a8d08d" + stop-opacity="1" + id="stop8" /><stop + offset="0.24" + stop-color="#bedcaa" + stop-opacity="1" + id="stop10" /><stop + offset="0.59" + stop-color="#3374af" + stop-opacity="1" + id="stop12" /><stop + offset="0.75" + stop-color="#41719c" + stop-opacity="1" + id="stop14" /><stop + offset="1" + stop-color="#c5e0b3" + stop-opacity="1" + id="stop16" /></radialGradient></defs><defs + id="Markers"><g + id="lend13"><path + d="M 3 1 L 0 0 L 3 -1 L 3 1 " + style="stroke:none" + id="path20" /></g><marker + id="mrkr13-82" + class="st21" + refX="10.2" + orient="auto" + markerUnits="strokeWidth" + overflow="visible"><use + xlink:href="#lend13" + transform="scale(3.52) " + id="use23" /></marker><marker + id="mrkr13-84" + class="st21" + refX="-10.56" + orient="auto" + markerUnits="strokeWidth" + overflow="visible"><use + xlink:href="#lend13" + transform="scale(-3.52,-3.52) " + id="use26" /></marker><g + id="lend4"><path + d="M 2 1 L 0 0 L 2 -1 L 2 1 " + style="stroke:none" + id="path29" /></g><marker + id="mrkr4-90" + class="st21" + refX="-7.04" + orient="auto" + markerUnits="strokeWidth" + overflow="visible"><use + xlink:href="#lend4" + transform="scale(-3.52,-3.52) " + id="use32" /></marker><marker + id="mrkr13-106" + class="st24" + refX="7.1142857142857" + orient="auto" + markerUnits="strokeWidth" + overflow="visible"><use + xlink:href="#lend13" + transform="scale(2.44) " + id="use35" /></marker><marker + id="mrkr13-112" + class="st26" + refX="7.1142857142857" + orient="auto" + markerUnits="strokeWidth" + overflow="visible"><use + xlink:href="#lend13" + transform="scale(2.44) " + id="use38" /></marker></defs><defs + id="Filters"><filter + id="filter_2"><feGaussianBlur + stdDeviation="2" + id="feGaussianBlur42" /></filter></defs><g + id="g44"><title + id="title46">Page-1</title><g + id="shape85-1" + transform="translate(133.887,-26.1478)"><title + id="title49">Sheet.85</title><desc + id="desc51">NIC</desc><g + id="shadow85-2" + transform="matrix(1,0,0,1,0.345598,1.97279)" + class="st1"><rect + x="0" + y="255.497" + width="346.142" + height="92.8673" + class="st2" + id="rect54" /><text + x="162.89" + y="349.33" + class="st3" + id="text56">NIC</text> +</g><rect + x="0" + y="255.497" + width="346.142" + height="92.8673" + class="st4" + id="rect58" + style="stroke-width:0.50000076;stroke-miterlimit:3;stroke-dasharray:none" /><text + x="162.89" + y="349.33" + class="st5" + id="text60">NIC</text> +</g><g + id="shape20-9" + transform="translate(3.0289,-127.458)"><title + id="title63">Rounded Rectangle.20</title><desc + id="desc65">LINUX</desc><g + id="shadow20-10" + transform="matrix(1,0,0,1,0.345598,1.97279)" + class="st1"><path + d="M30.39 348.36 L273.54 348.36 A30.3924 30.3924 -180 0 0 303.93 317.97 L303.93 235.23 A30.3924 30.3924 -180 0 0 273.54 204.84 L30.39 204.84 A30.3924 30.3924 -180 0 0 0 235.23 L0 317.97 A30.3924 30.3924 -180 0 0 30.39 348.36 Z" + class="st2" + id="path68" /><text + x="255.32" + y="238.8" + class="st6" + id="text70">LINUX</text> +</g><path + d="M30.39 348.36 L273.54 348.36 A30.3924 30.3924 -180 0 0 303.93 317.97 L303.93 235.23 A30.3924 30.3924 -180 0 0 273.54 204.84 L30.39 204.84 A30.3924 30.3924 -180 0 0 0 235.23 L0 317.97 A30.3924 30.3924 -180 0 0 30.39 348.36 Z" + class="st4" + id="path72" /><text + x="255.32" + y="238.8" + class="st7" + id="text74">LINUX</text> +</g><g + id="shape8-17" + transform="translate(95.8962,-140.079)"><title + id="title77">Rounded Rectangle.8</title><desc + id="desc79">Kernel pf driver</desc><path + d="M18.57 348.36 L167.16 348.36 A18.5731 18.5731 -180 0 0 185.73 329.79 L185.73 303.58 A18.5731 18.5731 -180 0 0 167.16 285 L18.57 285 A18.5731 18.5731 -180 0 0 0 303.58 L0 329.79 A18.5731 18.5731 -180 0 0 18.57 348.36 Z" + class="st8" + id="path81" /><text + x="118.71" + y="319.68" + class="st9" + id="text83">Kernel pf driver </text> +</g><g + id="shape1-20" + transform="translate(103.263,-156.88)"><title + id="title86">Rounded Rectangle</title><desc + id="desc88">Filters support traffic steering to VF</desc><g + id="shadow1-21" + transform="matrix(1,0,0,1,0.345598,1.97279)" + class="st1"><path + d="M10.55 348.36 L94.98 348.36 A10.5529 10.5529 -180 0 0 105.53 337.81 L105.53 324.98 A10.5529 10.5529 -180 0 0 94.98 314.43 L10.55 314.43 A10.5529 10.5529 -180 0 0 0 324.98 L0 337.81 A10.5529 10.5529 -180 0 0 10.55 348.36 Z" + class="st10" + id="path91" /></g><path + d="M10.55 348.36 L94.98 348.36 A10.5529 10.5529 -180 0 0 105.53 337.81 L105.53 324.98 A10.5529 10.5529 -180 0 0 94.98 314.43 L10.55 314.43 A10.5529 10.5529 -180 0 0 0 324.98 L0 337.81 A10.5529 10.5529 -180 0 0 10.55 348.36 Z" + class="st11" + id="path93" /><text + x="10.03" + y="328.39" + class="st12" + id="text95" + style="-inkscape-font-specification:'Calibri, Normal';font-family:Calibri;font-weight:normal;font-style:normal;font-stretch:normal;font-variant:normal;font-size:10.00001526px;text-anchor:start;text-align:start;writing-mode:lr;line-height:125%;" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4475">Filters support traffic</tspan></text> +<text + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:10.00001526px;line-height:125%;font-family:Calibri;-inkscape-font-specification:'Calibri, Normal';text-align:start;writing-mode:lr-tb;text-anchor:start;fill:#feffff" + x="11.048484" + y="340.46152" + class="st12" + id="text95-1" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4477" + x="11.048484" + y="340.46152">steering to VF</tspan></text> +</g><g + id="shape3-27" + transform="translate(192.985,-73.088)"><title + id="title100">Rectangle.3</title><desc + id="desc102">Rx Queues (0-N) PF</desc><rect + x="0" + y="314.425" + width="75.9823" + height="33.9388" + class="st8" + id="rect104" /><text + x="16.43" + y="322.39" + class="st9" + id="text106" + style="-inkscape-font-specification:'Calibri, Normal';font-family:Calibri;font-weight:normal;font-style:normal;font-stretch:normal;font-variant:normal;font-size:10.00001526px;text-anchor:start;text-align:start;writing-mode:lr;line-height:125%;" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4512">Rx Queues</tspan></text> +<text + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:10.00001526px;line-height:125%;font-family:Calibri;-inkscape-font-specification:'Calibri, Normal';text-align:start;writing-mode:lr-tb;text-anchor:start;fill:#000000" + x="23.187859" + y="333.70471" + class="st9" + id="text106-2" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4514" + x="23.187859" + y="333.70471">( 0-N )</tspan></text> +<text + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:10.00001526px;line-height:125%;font-family:Calibri;-inkscape-font-specification:'Calibri, Normal';text-align:start;writing-mode:lr-tb;text-anchor:start;fill:#ff0000;fill-opacity:1" + x="27.490538" + y="345.52356" + class="st9" + id="text106-9" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4516" + x="27.490538" + y="345.52356"> PF</tspan></text> +</g><g + id="shape4-32" + transform="translate(344.949,-73.088)"><title + id="title113">Rectangle.4</title><desc + id="desc115">Rx Queues (0-M) VF(vf 0)</desc><g + id="shadow4-33" + transform="matrix(1,0,0,1,0.345598,1.97279)" + class="st1"><rect + x="0" + y="314.425" + width="75.9823" + height="33.9388" + class="st10" + id="rect118" /></g><rect + x="0" + y="314.425" + width="75.9823" + height="33.9388" + class="st15" + id="rect120" /><text + x="16.43" + y="322.39" + class="st9" + id="text122" + style="-inkscape-font-specification:'Calibri, Normal';font-family:Calibri;font-weight:normal;font-style:normal;font-stretch:normal;font-variant:normal;font-size:10.00001526px;text-anchor:start;text-align:start;writing-mode:lr;line-height:125%;" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4571">Rx Queues</tspan></text> +<text + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:10.00001526px;line-height:125%;font-family:Calibri;-inkscape-font-specification:'Calibri, Normal';text-align:start;writing-mode:lr-tb;text-anchor:start;fill:#000000" + x="21.777092" + y="333.69595" + class="st9" + id="text122-8" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4569" + x="21.777092" + y="333.69595">( 0-M )</tspan></text> +<text + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:10.00001526px;line-height:125%;font-family:Calibri;-inkscape-font-specification:'Calibri, Normal';text-align:start;writing-mode:lr-tb;text-anchor:start;fill:#ff0000;fill-opacity:1" + x="21.79059" + y="343.91479" + class="st9" + id="text122-0" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4573" + x="21.79059" + y="343.91479">VF(vf0)</tspan></text> +</g><g + id="shape5-44" + transform="translate(154.994,-43.0328)"><title + id="title137">Rectangle.5</title><desc + id="desc139">filters</desc><g + id="shadow5-45" + transform="matrix(1,0,0,1,0.345598,1.97279)" + class="st1"><rect + x="0" + y="331.479" + width="303.929" + height="16.885" + class="st10" + id="rect142" /></g><rect + x="0" + y="331.479" + width="303.929" + height="16.885" + class="st16" + id="rect144" /><text + x="140.28" + y="342.92" + class="st9" + id="text146">filters</text> +</g><g + id="shape6-52" + transform="translate(95.8962,-224.377)"><title + id="title149">Rounded Rectangle.6</title><desc + id="desc151">Tools to program filters</desc><path + d="m 7.6,347.29783 60.78,0 a 7.59811,7.59811 0 0 0 7.6,-7.59 l 0,-18.58 a 7.59811,7.59811 0 0 0 -7.6,-7.6 l -60.78,0 a 7.59811,7.59811 0 0 0 -7.6,7.6 l 0,18.58 a 7.59811,7.59811 0 0 0 7.6,7.59 z" + class="st8" + id="path153" + inkscape:connector-curvature="0" + style="fill:#a8d08d;stroke:#4f87bb;stroke-width:0.75" /><text + x="21.74" + y="328.48001" + class="st9" + id="text155" + style="font-size:10.00003242px;font-family:Calibri;fill:#000000">Tools to <tspan + x="7.6900001" + class="st13" + id="tspan157" + style="font-size:10.00003242px" /></text> +<text + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:10.00001526px;line-height:125%;font-family:Calibri;-inkscape-font-specification:'Calibri, Normal';text-align:start;writing-mode:lr-tb;text-anchor:start;fill:#000000" + x="23.40806" + y="340.79666" + class="st9" + id="text155-1" + sodipodi:linespacing="125%"><tspan + x="9.358057" + class="st13" + id="tspan157-9" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:10.00001526px;line-height:125%;font-family:Calibri;-inkscape-font-specification:'Calibri, Normal';text-align:start;writing-mode:lr-tb;text-anchor:start">program filters</tspan></text> +</g><g + id="shape22-56" + transform="translate(11.4714,-156.88)"><title + id="title160">2-D word balloon</title><desc + id="desc162">Director flows to queue index in specified VF</desc><text + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:8.0000124px;line-height:125%;font-family:Calibri;-inkscape-font-specification:'Calibri, Normal';text-align:start;writing-mode:lr-tb;text-anchor:start;fill:#feffff" + x="8.0559683" + y="346.97244" + class="st17" + id="text169-3-4" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4471" + x="8.0559683" + y="346.97244">inspecified VF</tspan></text> +<g + id="shadow22-57" + transform="matrix(1,0,0,1,0.345598,1.97279)" + class="st1"><path + d="M0 327.15 L0 314.43 L28.49 314.43 L37.99 314.43 L47.49 314.43 L75.98 314.43 L75.98 327.15 L91.79 331.39 L75.98 335.64 L75.98 348.36 L47.49 348.36 L37.99 348.36 L28.49 348.36 L0 348.36 L0 335.64 L0 331.39 L0 327.15 Z" + class="st10" + id="path165" /></g><path + d="m 0.53783484,327.68783 0,-12.72 28.49000016,0 9.5,0 9.5,0 28.49,0 0,12.72 15.81,4.24 -15.81,4.25 0,12.72 -28.49,0 -9.5,0 -9.5,0 -28.49000016,0 0,-12.72 0,-4.25 0,-4.24 z" + class="st11" + id="path167" + inkscape:connector-curvature="0" + style="fill:#c00000;stroke:#c7c8c8;stroke-width:0.25" /><text + x="7.5599966" + y="324.19" + class="st17" + id="text169" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:8.0000124px;line-height:125%;font-family:Calibri;-inkscape-font-specification:'Calibri, Normal';text-align:start;writing-mode:lr-tb;text-anchor:start;fill:#feffff" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4446" + x="7.5599966" + y="324.19">Director flows</tspan></text> +<text + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:8.0000124px;line-height:125%;font-family:Calibri;-inkscape-font-specification:'Calibri, Normal';text-align:start;writing-mode:lr-tb;text-anchor:start;fill:#feffff" + x="8.1099777" + y="334.57529" + class="st17" + id="text169-3" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4469" + x="8.1099777" + y="334.57529">to queue index</tspan></text> +<text + x="8.5350533" + y="345.4624" + class="st17" + id="text169-5" + style="font-style:normal;font-variant:normal;font-weight:normal;font-stretch:normal;font-size:8.00001221px;line-height:125%;font-family:Calibri;-inkscape-font-specification:'Calibri, Normal';text-align:start;writing-mode:lr;text-anchor:start;fill:#feffff;" + sodipodi:linespacing="125%"><tspan + sodipodi:role="line" + id="tspan4479">in specified VF</tspan></text> +</g><g + id="shape24-64" + transform="translate(323.843,-285.05)"><title + id="title176">Rounded Rectangle.24</title><desc + id="desc178">DPDK</desc><g + id="shadow24-65" + transform="matrix(1,0,0,1,0.345598,1.97279)" + class="st1"><path + d="M9.29 348.36 L83.58 348.36 A9.28657 9.28657 -180 0 0 92.87 339.08 L92.87 295.74 A9.28657 9.28657 -180 0 0 83.58 286.45 L9.29 286.45 A9.28657 9.28657 -180 0 0 0 295.74 L0 339.08 A9.28657 9.28657 -180 0 0 9.29 348.36 Z" + class="st10" + id="path181" /></g><path + d="M9.29 348.36 L83.58 348.36 A9.28657 9.28657 -180 0 0 92.87 339.08 L92.87 295.74 A9.28657 9.28657 -180 0 0 83.58 286.45 L9.29 286.45 A9.28657 9.28657 -180 0 0 0 295.74 L0 339.08 A9.28657 9.28657 -180 0 0 9.29 348.36 Z" + class="st18" + id="path183" /><text + x="30.57" + y="321.61" + class="st19" + id="text185">DPDK</text> +</g><g + id="shape25-70" + transform="translate(192.985,-285.05)"><title + id="title188">Rounded Rectangle.25</title><desc + id="desc190">Socket</desc><path + d="M9.29 348.36 L83.58 348.36 A9.28657 9.28657 -180 0 0 92.87 339.08 L92.87 295.74 A9.28657 9.28657 -180 0 0 83.58 286.45 L9.29 286.45 A9.28657 9.28657 -180 0 0 0 295.74 L0 339.08 A9.28657 9.28657 -180 0 0 9.29 348.36 Z" + class="st8" + id="path192" /><text + x="27.56" + y="321.61" + class="st19" + id="text194">Socket</text> +</g><g + id="shape44-73" + transform="translate(154.994,569.271) rotate(180)"><title + id="title197">Simple Arrow.44</title><g + id="shadow44-74" + transform="matrix(1,0,0,1,-0.345598,-1.97279)" + class="st1" /></g><g + id="shape52-75" + transform="translate(154.994,-127.458)"><title + id="title201">Single arrowhead</title></g><g + id="shape70-76" + transform="translate(221.976,-107.027)"><title + id="title204">Dynamic connector.70</title><path + d="M9 338.16 L9 337.8 L9 325.87" + class="st20" + id="path206" /></g><g + id="shape81-85" + transform="translate(124.887,-224.377)"><title + id="title209">Dynamic connector.81</title><path + d="M9 348.36 L9 362.26" + class="st22" + id="path211" /></g><g + id="shape83-91" + transform="translate(240.398,-57.5029)"><title + id="title214">Dynamic connector.83</title><path + d="M-8.58 345.95 L-8.97 339.8" + class="st22" + id="path216" /></g><g + id="shape84-96" + transform="translate(373.94,-57.5029)"><title + id="title219">Dynamic connector.84</title><path + d="M9 345.95 L9 339.82" + class="st22" + id="path221" /></g><g + id="shape98-101" + transform="translate(539.29,6.22333) rotate(79.2209)"><title + id="title224">Sheet.98</title><path + d="M11.39 310.28 L11.72 310.42 C54.22 328.18 100.77 337.63 149.11 345.35 C162.41 347.48 175.84 349.47 187.65 347.74 C201.36 345.74 212.87 338.71 218.42 327.59 C222.66 319.09 223.42 308.2 229.69 303.23 C239.2 295.7 261.37 301.76 275.96 305.26" + class="st23" + id="path226" /></g><g + id="shape109-107" + transform="translate(712.298,124.855) rotate(100.2)"><title + id="title229">Sheet.109</title><path + d="M12.03 344.31 L12.38 344.21 C55.98 332.05 99.42 314.86 144.33 309.38 C167.01 306.62 190.06 306.85 204.84 318.11 C212.98 324.32 218.61 333.88 226.49 339.83 C238.38 348.81 255.38 349.56 275.91 347.51" + class="st25" + id="path231" /></g><g + id="shape110-113" + transform="translate(108.779,-175.962)"><title + id="title234">Sheet.110</title><rect + x="0" + y="341.614" + width="94.5" + height="6.75" + class="st27" + id="rect236" /></g></g></svg> \ No newline at end of file diff --git a/doc/guides/nics/img/ixgbe_bifu_queue_idx.svg b/doc/guides/nics/img/ixgbe_bifu_queue_idx.svg new file mode 100644 index 0000000..f7e2bd8 --- /dev/null +++ b/doc/guides/nics/img/ixgbe_bifu_queue_idx.svg @@ -0,0 +1,101 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd";> +<!-- Generated by Microsoft Visio, SVG Export ixgbe_bifu_queue_idx.svg Page-1 --> +<svg xmlns="http://www.w3.org/2000/svg"; xmlns:xlink="http://www.w3.org/1999/xlink"; xmlns:ev="http://www.w3.org/2001/xml-events"; + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/"; width="4.59375in" height="0.535375in" + viewBox="0 0 330.75 38.547" xml:space="preserve" color-interpolation-filters="sRGB" class="st8"> + <v:documentProperties v:langID="1033" v:viewMarkup="false"/> + + <style type="text/css"> + <![CDATA[ + .st1 {visibility:visible} + .st2 {fill:#5b9bd5;fill-opacity:0.22;filter:url(#filter_2);stroke:#5b9bd5;stroke-opacity:0.22} + .st3 {fill:#ffffff;stroke:#c7c8c8;stroke-width:0.25} + .st4 {fill:#000000;font-family:Calibri;font-size:0.833336em} + .st5 {fill:#c5e0b3;stroke:#c7c8c8;stroke-width:0.25} + .st6 {fill:#f4b183;stroke:#c7c8c8;stroke-width:0.25} + .st7 {fill:none;stroke:none;stroke-width:0.25} + .st8 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]> + </style> + + <defs id="Filters"> + <filter id="filter_2"> + <feGaussianBlur stdDeviation="2"/> + </filter> + </defs> + <g v:mID="0" v:index="1" v:groupContext="foregroundPage"> + <title>Page-1</title> + <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/> + <g id="shape1-1" v:mID="1" v:groupContext="shape" transform="translate(3.0294,-5.34781)"> + <title>Rectangle</title> + <desc>0x000000</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="52.1695" cy="30.3097" width="104.34" height="16.4746"/> + <g id="shadow1-2" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="22.0724" width="104.339" height="16.4746" class="st2"/> + </g> + <rect x="0" y="22.0724" width="104.339" height="16.4746" class="st3"/> + <text x="32.27" y="33.31" class="st4" v:langID="2052"><v:paragraph v:horizAlign="1"/><v:tabList/>0x000000</text> </g> + <g id="shape2-7" v:mID="2" v:groupContext="shape" transform="translate(107.368,-5.34781)"> + <title>Rectangle.2</title> + <desc>VF ID + 1</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="30.2034" cy="30.3097" width="60.41" height="16.4746"/> + <g id="shadow2-8" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="22.0724" width="60.4068" height="16.4746" class="st2"/> + </g> + <rect x="0" y="22.0724" width="60.4068" height="16.4746" class="st5"/> + <text x="12.32" y="33.31" class="st4" v:langID="2052"><v:paragraph v:horizAlign="1"/><v:tabList/>VF ID + 1</text> </g> + <g id="shape3-13" v:mID="3" v:groupContext="shape" transform="translate(167.775,-5.34781)"> + <title>Rectangle.3</title> + <desc>Queue Index</desc> + <v:userDefs> + <v:ud v:nameU="visVersion" v:val="VT0(15):26"/> + </v:userDefs> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="79.6271" cy="30.3097" width="159.26" height="16.4746"/> + <g id="shadow3-14" v:groupContext="shadow" v:shadowOffsetX="0.345598" v:shadowOffsetY="-1.97279" v:shadowType="1" + transform="matrix(1,0,0,1,0.345598,1.97279)" class="st1"> + <rect x="0" y="22.0724" width="159.254" height="16.4746" class="st2"/> + </g> + <rect x="0" y="22.0724" width="159.254" height="16.4746" class="st6"/> + <text x="53.74" y="33.31" class="st4" v:langID="2052"><v:paragraph v:horizAlign="1"/><v:tabList/>Queue Index</text> </g> + <g id="shape4-19" v:mID="4" v:groupContext="shape" transform="translate(305.063,-21.8224)"> + <title>Sheet.4</title> + <desc>0</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="10.9831" cy="30.3097" width="21.97" height="16.4746"/> + <rect x="0" y="22.0724" width="21.9661" height="16.4746" class="st7"/> + <text x="8.45" y="33.31" class="st4" v:langID="2052"><v:paragraph v:horizAlign="1"/><v:tabList/>0</text> </g> + <g id="shape6-22" v:mID="6" v:groupContext="shape" transform="translate(165.029,-21.8224)"> + <title>Sheet.6</title> + <desc>31</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="10.9831" cy="30.3097" width="21.97" height="16.4746"/> + <rect x="0" y="22.0724" width="21.9661" height="16.4746" class="st7"/> + <text x="5.91" y="33.31" class="st4" v:langID="2052"><v:paragraph v:horizAlign="1"/><v:tabList/>31</text> </g> + <g id="shape7-25" v:mID="7" v:groupContext="shape" transform="translate(104.623,-21.8224)"> + <title>Sheet.7</title> + <desc>39</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="10.9831" cy="30.3097" width="21.97" height="16.4746"/> + <rect x="0" y="22.0724" width="21.9661" height="16.4746" class="st7"/> + <text x="5.91" y="33.31" class="st4" v:langID="2052"><v:paragraph v:horizAlign="1"/><v:tabList/>39</text> </g> + <g id="shape8-28" v:mID="8" v:groupContext="shape" transform="translate(3.0294,-21.8224)"> + <title>Sheet.8</title> + <desc>63</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="10.9831" cy="30.3097" width="21.97" height="16.4746"/> + <rect x="0" y="22.0724" width="21.9661" height="16.4746" class="st7"/> + <text x="5.91" y="33.31" class="st4" v:langID="2052"><v:paragraph v:horizAlign="1"/><v:tabList/>63</text> </g> + </g> +</svg> diff --git a/doc/guides/nics/ixgbe.rst b/doc/guides/nics/ixgbe.rst index 3dc6b00..74ec4b7 100644 --- a/doc/guides/nics/ixgbe.rst +++ b/doc/guides/nics/ixgbe.rst @@ -224,3 +224,122 @@ be calculated as follows: max_read_interval = ~4 mins 48 sec. In order to ensure valid results, it is recommended to poll every 4 minutes. + +.. _bifurcated_driver: + +Bifurcated driver guide +----------------------- + +Bifurcated driver is a mechanism which depends the advanced Ethernet device to +split traffic between Linux user space and kernel space. Because it is hardware +assisted design, this approach can provide the line rate processing capability. +Other than KNI, the SW is just required to device configuration, no need to +take care of the packet movement during the traffic split. This can get more +performance with less CPU overhead. + +The Bifurcated driver take advantage of Ethernet device feature to split the +incoming data traffic to user space application (Such as DPDK application) +and/or kernel space program (Linux kernel stack). It can direct some traffic +(e.g data plane traffic) to DPDK, while direct some other traffic (e.g control +plane traffic) to the traditional Linux networking stack. + +There are a number of technical options to achieve this. A typical example is +to combine the technology of SR-IOV and packet classification filtering. + +SR-IOV is a PCI standard that allows the same physical adapter to split as +multiple virtual functions. Each virtual function has separated queues with +physical function. Traffic with a virtual function?s destination MAC address +from network adapter will be directed to it. In a sense, SR-IOV has the +capability on queue division. + +Packet classification filtering is the hardware capability available on most +network adapters. Filters can be configured to direct specific flows to a given +receive queue by hardware. Different NIC may have different filter types to +direct flow to a Virtual Function or a queue belong to it. + +Linux network can receive the specific traffic through kernel driver, while +DPDK can receive the specific traffic bypassing the Linux kernel by using +drivers like VFIO or DPDK igb_uio module. + +.. _figure_bifurcated_driver_overview: + +.. figure:: img/bifurcated_driver_overview.* + + Bifurcated Driver Overview + +Use Bifurcated driver on IXGBE in Linux +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +On Intel? 82599 10 Gigabit Ethernet Controller series NICs, Bifurcated driver +can be achieved by SR-IOV and flow director technologies. So the traffic can +be directed to queues by flow director capability, typically by matching 5-tuple +of UDP/TCP packets. + +The step procedure is as following: + +#. Boot system without iommu, or with ?iommu=pt?. + +#. Create Virtual Functions: + + .. code-block:: console + + echo 2 > /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs + +#. Enable and set flow filters: + + .. code-block:: console + + ethtool -K eth1 ntuple on + ethtool -N eth1 flow-type udp4 src-ip 192.0.2.2 dst-ip 198.51.100.2 \ + action $queue_index_in_VF0 + ethtool -N eth1 flow-type udp4 src-ip 198.51.100.2 dst-ip 192.0.2.2 \ + action $queue_index_in_VF1 + + where: + + * $queue_index_in_PF: [queue index] + + * $queue_index_in_VFn: Bits 39:32 of the variable defines VF id + 1; lower 32 bits indicates the queue index of VF. + + * $queue_index_in_VF0 = (0x1 & 0xFF) << 32 + [queue index]; + + * $queue_index_in_VF1 = (0x2 & 0xFF) << 32 + [queue index]; + + .. _figure_ixgbe_bifu_queue_idx: + + .. figure:: img/ixgbe_bifu_queue_idx.* + +#. Compile the DPDK and insert igb_uio or probe vfio-pci kernel modules as normal. + +#. Bind virtual function: + + .. code-block:: console + + modprobe vfio-pci + dpdk_nic_bind.py -b vfio-pci 01:10.0 + dpdk_nic_bind.py -b vfio-pci 01:10.1 + +#. run DPDK application on VFs: + + .. code-block:: console + + testpmd -c 0xff -n 4 -- -i -w 01:10.0 -w 01:10.1 --forward-mode=mac + +In this example, traffic matching the rules will go through VF by matching the +filter rule. All other traffic which mismatching the rules, will go through +the default queue or scaling on queues in PF. That is to say UDP packets with +those IP source and destination addresses will go through the DPDK. All other +traffic, with different hosts or different protocols, will go through the Linux +networking stack. + +.. note:: + + * The above steps work on the Linux kernel v4.2. + + * The Bifurcated driver is implemented in Linux kernel and ixgbe kernel driver by following patches: + + * `ethtool: Add helper routines to pass vf to rx_flow_spec <https://patchwork.ozlabs.org/patch/476511/>`_ + + * `ixgbe: Allow flow director to use entire queue space <https://patchwork.ozlabs.org/patch/476516/>`_ + + * Ethtool's version used in this example is 3.18. \ No newline at end of file -- 2.4.0
