Add lib_rcu QSBR API and programmer guide documentation. Signed-off-by: Honnappa Nagarahalli <honnappa.nagaraha...@arm.com> Reviewed-by: Marko Kovacevic <marko.kovace...@intel.com> --- doc/api/doxy-api-index.md | 3 +- doc/api/doxy-api.conf.in | 1 + .../prog_guide/img/rcu_general_info.svg | 509 ++++++++++++++++++ doc/guides/prog_guide/index.rst | 1 + doc/guides/prog_guide/rcu_lib.rst | 185 +++++++ 5 files changed, 698 insertions(+), 1 deletion(-) create mode 100644 doc/guides/prog_guide/img/rcu_general_info.svg create mode 100644 doc/guides/prog_guide/rcu_lib.rst
diff --git a/doc/api/doxy-api-index.md b/doc/api/doxy-api-index.md index de1e215dd..8f0e84de6 100644 --- a/doc/api/doxy-api-index.md +++ b/doc/api/doxy-api-index.md @@ -54,7 +54,8 @@ The public API headers are grouped by topics: [memzone] (@ref rte_memzone.h), [mempool] (@ref rte_mempool.h), [malloc] (@ref rte_malloc.h), - [memcpy] (@ref rte_memcpy.h) + [memcpy] (@ref rte_memcpy.h), + [rcu] (@ref rte_rcu_qsbr.h) - **timers**: [cycles] (@ref rte_cycles.h), diff --git a/doc/api/doxy-api.conf.in b/doc/api/doxy-api.conf.in index 7722fc3e9..b9896cb63 100644 --- a/doc/api/doxy-api.conf.in +++ b/doc/api/doxy-api.conf.in @@ -51,6 +51,7 @@ INPUT = @TOPDIR@/doc/api/doxy-api-index.md \ @TOPDIR@/lib/librte_port \ @TOPDIR@/lib/librte_power \ @TOPDIR@/lib/librte_rawdev \ + @TOPDIR@/lib/librte_rcu \ @TOPDIR@/lib/librte_reorder \ @TOPDIR@/lib/librte_ring \ @TOPDIR@/lib/librte_sched \ diff --git a/doc/guides/prog_guide/img/rcu_general_info.svg b/doc/guides/prog_guide/img/rcu_general_info.svg new file mode 100644 index 000000000..e7ca1dacb --- /dev/null +++ b/doc/guides/prog_guide/img/rcu_general_info.svg @@ -0,0 +1,509 @@ +<?xml version="1.0" encoding="UTF-8" standalone="no"?> +<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd"> +<!-- Generated by Microsoft Visio, SVG Export rcu_general_info.svg Page-1 --> + +<!-- SPDX-License-Identifier: BSD-3-Clause --> +<!-- Copyright(c) 2019 Arm Limited --> + +<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ev="http://www.w3.org/2001/xml-events" + xmlns:v="http://schemas.microsoft.com/visio/2003/SVGExtensions/" width="21.5in" height="16.5in" viewBox="0 0 1548 1188" + xml:space="preserve" color-interpolation-filters="sRGB" class="st21"> + <v:documentProperties v:langID="1033" v:viewMarkup="false"> + <v:userDefs> + <v:ud v:nameU="msvSubprocessMaster" v:prompt="" v:val="VT4(Rectangle)"/> + <v:ud v:nameU="msvNoAutoConnect" v:val="VT0(1):26"/> + </v:userDefs> + </v:documentProperties> + + <style type="text/css"> + <![CDATA[ + .st1 {fill:#92d050;stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st2 {fill:#ff0000;stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st3 {stroke:none;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.75} + .st4 {fill:#ffffff;font-family:Calibri;font-size:1.81435em;font-weight:bold} + .st5 {fill:#333e48;font-family:Century Gothic;font-size:1.81435em} + .st6 {fill:#000000;font-family:Calibri;font-size:1.99578em;font-weight:bold} + .st7 {stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:1.45071} + .st8 {fill:#000000;font-family:Century Gothic;font-size:1.75001em} + .st9 {font-size:1em} + .st10 {stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.90143} + .st11 {fill:#333e48;font-family:Calibri;font-size:2.11672em;font-weight:bold} + .st12 {stroke:#651beb;stroke-linecap:round;stroke-linejoin:round;stroke-width:2.90143} + .st13 {stroke:#b31166;stroke-linecap:round;stroke-linejoin:round;stroke-width:0.725356} + .st14 {fill:#000000;font-family:Century Gothic;font-size:1.99999em} + .st15 {fill:#feffff;font-family:Calibri;font-size:1.99999em;font-weight:bold} + .st16 {marker-end:url(#mrkr5-239);marker-start:url(#mrkr5-237);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:3} + .st17 {fill:#000000;fill-opacity:1;stroke:#000000;stroke-opacity:1;stroke-width:0.54347826086957} + .st18 {marker-end:url(#mrkr5-248);marker-start:url(#mrkr5-246);stroke:#651beb;stroke-linecap:round;stroke-linejoin:round;stroke-width:3} + .st19 {fill:#651beb;fill-opacity:1;stroke:#651beb;stroke-opacity:1;stroke-width:0.67567567567568} + .st20 {marker-end:url(#mrkr5-239);stroke:#000000;stroke-linecap:round;stroke-linejoin:round;stroke-width:3} + .st21 {fill:none;fill-rule:evenodd;font-size:12px;overflow:visible;stroke-linecap:square;stroke-miterlimit:3} + ]]> + </style> + + <defs id="Markers"> + <g id="lend5"> + <path d="M 2 1 L 0 0 L 1.98117 -0.993387 C 1.67173 -0.364515 1.67301 0.372641 1.98465 1.00043 " style="stroke:none"/> + </g> + <marker id="mrkr5-237" class="st17" v:arrowType="5" v:arrowSize="2" v:setback="3.1" refX="3.1" orient="auto" + markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(1.84) "/> + </marker> + <marker id="mrkr5-239" class="st17" v:arrowType="5" v:arrowSize="2" v:setback="3.22" refX="-3.22" orient="auto" + markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-1.84,-1.84) "/> + </marker> + <marker id="mrkr5-246" class="st19" v:arrowType="5" v:arrowSize="0" v:setback="2.47" refX="2.47" orient="auto" + markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(1.48) "/> + </marker> + <marker id="mrkr5-248" class="st19" v:arrowType="5" v:arrowSize="0" v:setback="2.59" refX="-2.59" orient="auto" + markerUnits="strokeWidth" overflow="visible"> + <use xlink:href="#lend5" transform="scale(-1.48,-1.48) "/> + </marker> + </defs> + <g v:mID="0" v:index="1" v:groupContext="foregroundPage"> + <v:userDefs> + <v:ud v:nameU="msvThemeOrder" v:val="VT0(0):26"/> + </v:userDefs> + <title>Page-1</title> + <v:pageProperties v:drawingScale="1" v:pageScale="1" v:drawingUnits="0" v:shadowOffsetX="9" v:shadowOffsetY="-9"/> + <v:layer v:name="Connector" v:index="0"/> + <g id="shape3-1" v:mID="3" v:groupContext="shape" transform="translate(327.227,-946.908)"> + <title>Sheet.3</title> + <path d="M0 1142.11 C0 1137.08 4.14 1132.94 9.17 1132.94 L124.27 1132.94 C129.36 1132.94 133.44 1137.08 133.44 1142.11 + L133.44 1178.82 C133.44 1183.92 129.36 1188 124.27 1188 L9.17 1188 C4.14 1188 0 1183.92 0 1178.82 L0 1142.11 + Z" class="st1"/> + </g> + <g id="shape4-3" v:mID="4" v:groupContext="shape" transform="translate(460.665,-944.869)"> + <title>Sheet.4</title> + <path d="M0 1142.11 C0 1137.08 4.14 1132.94 9.17 1132.94 L141.59 1132.94 C146.68 1132.94 150.75 1137.08 150.75 1142.11 + L150.75 1178.82 C150.75 1183.92 146.68 1188 141.59 1188 L9.17 1188 C4.14 1188 0 1183.92 0 1178.82 L0 1142.11 + Z" class="st2"/> + </g> + <g id="shape5-5" v:mID="5" v:groupContext="shape" transform="translate(519.302,-950.79)"> + <title>Sheet.5</title> + <desc>D1</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="23.7162" cy="1169.64" width="47.44" height="36.7141"/> + <path d="M47.43 1151.29 L0 1151.29 L0 1188 L47.43 1188 L47.43 1151.29" class="st3"/> + <text x="11.34" y="1176.17" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>D1</text> </g> + <g id="shape6-9" v:mID="6" v:groupContext="shape" transform="translate(612.438,-944.869)"> + <title>Sheet.6</title> + <path d="M0 1141.6 C0 1136.82 3.88 1132.94 8.66 1132.94 L43.29 1132.94 C48.13 1132.94 51.95 1136.82 51.95 1141.6 L51.95 + 1179.33 C51.95 1184.18 48.13 1188 43.29 1188 L8.66 1188 C3.88 1188 0 1184.18 0 1179.33 L0 1141.6 Z" + class="st1"/> + </g> + <g id="shape7-11" v:mID="7" v:groupContext="shape" transform="translate(664.388,-945.889)"> + <title>Sheet.7</title> + <path d="M0 1142.11 C0 1137.08 4.14 1132.94 9.17 1132.94 L141.59 1132.94 C146.68 1132.94 150.75 1137.08 150.75 1142.11 + L150.75 1178.82 C150.75 1183.92 146.68 1188 141.59 1188 L9.17 1188 C4.14 1188 0 1183.92 0 1178.82 L0 1142.11 + Z" class="st2"/> + </g> + <g id="shape8-13" v:mID="8" v:groupContext="shape" transform="translate(723.025,-951.494)"> + <title>Sheet.8</title> + <desc>D2</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="23.7162" cy="1169.64" width="47.44" height="36.7141"/> + <path d="M47.43 1151.29 L0 1151.29 L0 1188 L47.43 1188 L47.43 1151.29" class="st3"/> + <text x="11.34" y="1176.17" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>D2</text> </g> + <g id="shape9-17" v:mID="9" v:groupContext="shape" transform="translate(814.123,-945.889)"> + <title>Sheet.9</title> + <path d="M0 1141.6 C0 1136.82 3.88 1132.94 8.66 1132.94 L43.29 1132.94 C48.13 1132.94 51.95 1136.82 51.95 1141.6 L51.95 + 1179.33 C51.95 1184.18 48.13 1188 43.29 1188 L8.66 1188 C3.88 1188 0 1184.18 0 1179.33 L0 1141.6 Z" + class="st1"/> + </g> + <g id="shape10-19" v:mID="10" v:groupContext="shape" transform="translate(27,-952.759)"> + <title>Sheet.10</title> + <desc>Reader Thread 1</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="146.259" cy="1169.64" width="292.52" height="36.7136"/> + <path d="M292.52 1151.29 L0 1151.29 L0 1188 L292.52 1188 L292.52 1151.29" class="st3"/> + <text x="58.76" y="1176.17" class="st5" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Reader Thread 1</text> </g> + <g id="shape11-23" v:mID="11" v:groupContext="shape" transform="translate(379.176,-863.295)"> + <title>Sheet.11</title> + <path d="M0 1142.11 C0 1137.08 4.14 1132.94 9.17 1132.94 L124.27 1132.94 C129.36 1132.94 133.44 1137.08 133.44 1142.11 + L133.44 1178.82 C133.44 1183.92 129.36 1188 124.27 1188 L9.17 1188 C4.14 1188 0 1183.92 0 1178.82 L0 1142.11 + Z" class="st1"/> + </g> + <g id="shape12-25" v:mID="12" v:groupContext="shape" transform="translate(512.614,-861.255)"> + <title>Sheet.12</title> + <path d="M0 1142.11 C0 1137.08 4.14 1132.94 9.17 1132.94 L141.59 1132.94 C146.68 1132.94 150.75 1137.08 150.75 1142.11 + L150.75 1178.82 C150.75 1183.92 146.68 1188 141.59 1188 L9.17 1188 C4.14 1188 0 1183.92 0 1178.82 L0 1142.11 + Z" class="st2"/> + </g> + <g id="shape13-27" v:mID="13" v:groupContext="shape" transform="translate(561.284,-867.106)"> + <title>Sheet.13</title> + <desc>D1</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="23.7162" cy="1169.64" width="47.44" height="36.7141"/> + <path d="M47.43 1151.29 L0 1151.29 L0 1188 L47.43 1188 L47.43 1151.29" class="st3"/> + <text x="11.34" y="1176.17" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>D1</text> </g> + <g id="shape14-31" v:mID="14" v:groupContext="shape" transform="translate(664.388,-861.255)"> + <title>Sheet.14</title> + <path d="M0 1141.6 C0 1136.82 3.88 1132.94 8.66 1132.94 L43.29 1132.94 C48.13 1132.94 51.95 1136.82 51.95 1141.6 L51.95 + 1179.33 C51.95 1184.18 48.13 1188 43.29 1188 L8.66 1188 C3.88 1188 0 1184.18 0 1179.33 L0 1141.6 Z" + class="st1"/> + </g> + <g id="shape15-33" v:mID="15" v:groupContext="shape" transform="translate(716.337,-862.275)"> + <title>Sheet.15</title> + <path d="M0 1142.11 C0 1137.08 4.14 1132.94 9.17 1132.94 L141.59 1132.94 C146.68 1132.94 150.75 1137.08 150.75 1142.11 + L150.75 1178.82 C150.75 1183.92 146.68 1188 141.59 1188 L9.17 1188 C4.14 1188 0 1183.92 0 1178.82 L0 1142.11 + Z" class="st2"/> + </g> + <g id="shape16-35" v:mID="16" v:groupContext="shape" transform="translate(775.009,-867.81)"> + <title>Sheet.16</title> + <desc>D2</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="23.7162" cy="1169.64" width="47.44" height="36.7141"/> + <path d="M47.43 1151.29 L0 1151.29 L0 1188 L47.43 1188 L47.43 1151.29" class="st3"/> + <text x="11.34" y="1176.17" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>D2</text> </g> + <g id="shape17-39" v:mID="17" v:groupContext="shape" transform="translate(866.073,-862.275)"> + <title>Sheet.17</title> + <path d="M0 1141.6 C0 1136.82 3.88 1132.94 8.66 1132.94 L43.29 1132.94 C48.13 1132.94 51.95 1136.82 51.95 1141.6 L51.95 + 1179.33 C51.95 1184.18 48.13 1188 43.29 1188 L8.66 1188 C3.88 1188 0 1184.18 0 1179.33 L0 1141.6 Z" + class="st1"/> + </g> + <g id="shape18-41" v:mID="18" v:groupContext="shape" transform="translate(143.348,-873.294)"> + <title>Sheet.18</title> + <desc>T 2</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="26.9796" cy="1169.64" width="53.96" height="36.7136"/> + <path d="M53.96 1151.29 L0 1151.29 L0 1188 L53.96 1188 L53.96 1151.29" class="st3"/> + <text x="13.3" y="1176.17" class="st5" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>T 2</text> </g> + <g id="shape19-45" v:mID="19" v:groupContext="shape" transform="translate(474.188,-777.642)"> + <title>Sheet.19</title> + <path d="M0 1143.01 C0 1138.04 4.07 1133.96 9.04 1133.96 L124.46 1133.96 C129.43 1133.96 133.44 1138.04 133.44 1143.01 + L133.44 1179.01 C133.44 1183.99 129.43 1188 124.46 1188 L9.04 1188 C4.07 1188 0 1183.99 0 1179.01 L0 1143.01 + Z" class="st1"/> + </g> + <g id="shape20-47" v:mID="20" v:groupContext="shape" transform="translate(608.645,-775.602)"> + <title>Sheet.20</title> + <path d="M0 1142.11 C0 1137.08 4.14 1132.94 9.17 1132.94 L141.59 1132.94 C146.68 1132.94 150.75 1137.08 150.75 1142.11 + L150.75 1178.82 C150.75 1183.92 146.68 1188 141.59 1188 L9.17 1188 C4.14 1188 0 1183.92 0 1178.82 L0 1142.11 + Z" class="st2"/> + </g> + <g id="shape21-49" v:mID="21" v:groupContext="shape" transform="translate(666.862,-781.311)"> + <title>Sheet.21</title> + <desc>D1</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="23.7162" cy="1169.64" width="47.44" height="36.7141"/> + <path d="M47.43 1151.29 L0 1151.29 L0 1188 L47.43 1188 L47.43 1151.29" class="st3"/> + <text x="11.34" y="1176.17" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>D1</text> </g> + <g id="shape22-53" v:mID="22" v:groupContext="shape" transform="translate(760.418,-775.602)"> + <title>Sheet.22</title> + <path d="M0 1141.6 C0 1136.82 3.88 1132.94 8.66 1132.94 L43.29 1132.94 C48.13 1132.94 51.95 1136.82 51.95 1141.6 L51.95 + 1179.33 C51.95 1184.18 48.13 1188 43.29 1188 L8.66 1188 C3.88 1188 0 1184.18 0 1179.33 L0 1141.6 Z" + class="st1"/> + </g> + <g id="shape23-55" v:mID="23" v:groupContext="shape" transform="translate(812.367,-776.622)"> + <title>Sheet.23</title> + <path d="M0 1142.11 C0 1137.08 4.14 1132.94 9.17 1132.94 L141.59 1132.94 C146.68 1132.94 150.75 1137.08 150.75 1142.11 + L150.75 1178.82 C150.75 1183.92 146.68 1188 141.59 1188 L9.17 1188 C4.14 1188 0 1183.92 0 1178.82 L0 1142.11 + Z" class="st2"/> + </g> + <g id="shape24-57" v:mID="24" v:groupContext="shape" transform="translate(870.584,-782.015)"> + <title>Sheet.24</title> + <desc>D2</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="23.7162" cy="1169.64" width="47.44" height="36.7141"/> + <path d="M47.43 1151.29 L0 1151.29 L0 1188 L47.43 1188 L47.43 1151.29" class="st3"/> + <text x="11.34" y="1176.17" class="st4" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>D2</text> </g> + <g id="shape25-61" v:mID="25" v:groupContext="shape" transform="translate(962.103,-776.622)"> + <title>Sheet.25</title> + <path d="M0 1141.6 C0 1136.82 3.88 1132.94 8.66 1132.94 L43.29 1132.94 C48.13 1132.94 51.95 1136.82 51.95 1141.6 L51.95 + 1179.33 C51.95 1184.18 48.13 1188 43.29 1188 L8.66 1188 C3.88 1188 0 1184.18 0 1179.33 L0 1141.6 Z" + class="st1"/> + </g> + <g id="shape26-63" v:mID="26" v:groupContext="shape" transform="translate(142.645,-787.5)"> + <title>Sheet.26</title> + <desc>T 3</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="26.9796" cy="1169.64" width="53.96" height="36.7136"/> + <path d="M53.96 1151.29 L0 1151.29 L0 1188 L53.96 1188 L53.96 1151.29" class="st3"/> + <text x="13.3" y="1176.17" class="st5" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>T 3</text> </g> + <g id="shape28-67" v:mID="28" v:groupContext="shape" transform="translate(882.826,-574.263)"> + <title>Sheet.28</title> + <desc>Time</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="45.9546" cy="1166.58" width="91.91" height="42.8314"/> + <path d="M91.91 1145.17 L0 1145.17 L0 1188 L91.91 1188 L91.91 1145.17" class="st3"/> + <text x="21.32" y="1173.77" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Time</text> </g> + <g id="shape29-71" v:mID="29" v:groupContext="shape" transform="translate(419.545,-660.119)"> + <title>Sheet.29</title> + <path d="M0 1145.17 L0 1188 L0 1145.17" class="st7"/> + </g> + <g id="shape30-74" v:mID="30" v:groupContext="shape" transform="translate(419.545,-684.783)"> + <title>Sheet.30</title> + <path d="M0 1188 L82.7 1187.36 L151.2 1172.07" class="st7"/> + </g> + <g id="shape31-77" v:mID="31" v:groupContext="shape" transform="translate(214.454,-663.095)"> + <title>Sheet.31</title> + <desc>Remove reference to entry1</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="96.7728" cy="1169.45" width="193.55" height="37.1049"/> + <path d="M193.55 1150.9 L0 1150.9 L0 1188 L193.55 1188 L193.55 1150.9" class="st3"/> + <text x="2.39" y="1163.15" class="st8" v:langID="1033"><v:paragraph v:horizAlign="2"/><v:tabList/>Remove reference <tspan + x="104.08" dy="1.2em" class="st9">to entry1</tspan></text> </g> + <g id="shape33-82" v:mID="33" v:groupContext="shape" transform="translate(571.287,-681.326)"> + <title>Sheet.33</title> + <path d="M0 738.67 L0 1188" class="st10"/> + </g> + <g id="shape34-85" v:mID="34" v:groupContext="shape" transform="translate(515.013,-1130.65)"> + <title>Sheet.34</title> + <desc>Delete</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="60.7243" cy="1166.58" width="121.45" height="42.8314"/> + <path d="M121.45 1145.17 L0 1145.17 L0 1188 L121.45 1188 L121.45 1145.17" class="st3"/> + <text x="26.02" y="1174.2" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Delete</text> </g> + <g id="shape35-89" v:mID="35" v:groupContext="shape" transform="translate(434.372,-1096.8)"> + <title>Sheet.35</title> + <path d="M0 1154.35 L0 1188 L0 1154.35" class="st7"/> + </g> + <g id="shape36-92" v:mID="36" v:groupContext="shape" transform="translate(434.372,-1100.37)"> + <title>Sheet.36</title> + <path d="M0 1171.88 L84.54 1171.24 L136.43 1188" class="st7"/> + </g> + <g id="shape37-95" v:mID="37" v:groupContext="shape" transform="translate(193.5,-1103.76)"> + <title>Sheet.37</title> + <desc>Delete entry1 from D1</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="114.75" cy="1175.76" width="229.5" height="24.4771"/> + <path d="M229.5 1163.52 L0 1163.52 L0 1188 L229.5 1188 L229.5 1163.52" class="st3"/> + <text x="3.88" y="1182.06" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Delete entry1 from D1</text> </g> + <g id="shape38-99" v:mID="38" v:groupContext="shape" transform="translate(714.3,-675.425)"> + <title>Sheet.38</title> + <path d="M0 732.77 L0 1188" class="st10"/> + </g> + <g id="shape39-102" v:mID="39" v:groupContext="shape" transform="translate(795.979,-637.904)"> + <title>Sheet.39</title> + <path d="M0 1112.54 L0 1188 L0 1112.54" class="st7"/> + </g> + <g id="shape40-105" v:mID="40" v:groupContext="shape" transform="translate(716.782,-675.425)"> + <title>Sheet.40</title> + <path d="M79.2 1188 L52.71 1187.94 L0 1147.21" class="st7"/> + </g> + <g id="shape41-108" v:mID="41" v:groupContext="shape" transform="translate(803.572,-639.285)"> + <title>Sheet.41</title> + <desc>Free memory for entries1 and 2 after every reader has gone th...</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="172.421" cy="1152.51" width="344.85" height="70.9752"/> + <path d="M344.84 1117.02 L0 1117.02 L0 1188 L344.84 1188 L344.84 1117.02" class="st3"/> + <text x="0" y="1133.61" class="st8" v:langID="1033"><v:paragraph/><v:tabList/>Free memory for entries1 and 2 <tspan + x="0" dy="1.2em" class="st9">after every reader has gone </tspan><tspan x="0" dy="1.2em" class="st9">through at least 1 quiescent state </tspan> </text> </g> + <g id="shape46-114" v:mID="46" v:groupContext="shape" transform="translate(680.801,-1130.65)"> + <title>Sheet.46</title> + <desc>Free</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="42.0169" cy="1166.58" width="84.04" height="42.8314"/> + <path d="M84.03 1145.17 L0 1145.17 L0 1188 L84.03 1188 L84.03 1145.17" class="st3"/> + <text x="18.89" y="1174.2" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Free</text> </g> + <g id="shape48-118" v:mID="48" v:groupContext="shape" transform="translate(811.005,-1110.05)"> + <title>Sheet.48</title> + <path d="M0 1145.17 L0 1188 L0 1145.17" class="st7"/> + </g> + <g id="shape49-121" v:mID="49" v:groupContext="shape" transform="translate(658.61,-1083.99)"> + <title>Sheet.49</title> + <path d="M153.05 1149.63 L113.7 1149.57 L0 1188" class="st7"/> + </g> + <g id="shape50-124" v:mID="50" v:groupContext="shape" transform="translate(798.359,-1110.46)"> + <title>Sheet.50</title> + <desc>Grace Period</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="107.799" cy="1167.81" width="215.6" height="40.3845"/> + <path d="M215.6 1147.62 L0 1147.62 L0 1188 L215.6 1188 L215.6 1147.62" class="st3"/> + <text x="43.79" y="1174.99" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Grace Period</text> </g> + <g id="shape51-128" v:mID="51" v:groupContext="shape" transform="translate(599.196,-662.779)"> + <title>Sheet.51</title> + <path d="M0 732.77 L0 1188" class="st12"/> + </g> + <g id="shape52-131" v:mID="52" v:groupContext="shape" transform="translate(464.931,-1052.95)"> + <title>Sheet.52</title> + <path d="M0 1154.35 L0 1188 L0 1154.35" class="st7"/> + </g> + <g id="shape53-134" v:mID="53" v:groupContext="shape" transform="translate(464.931,-1056.52)"> + <title>Sheet.53</title> + <path d="M0 1171.88 L84.54 1171.24 L136.43 1188" class="st7"/> + </g> + <g id="shape54-137" v:mID="54" v:groupContext="shape" transform="translate(225,-1058.76)"> + <title>Sheet.54</title> + <desc>Delete entry2 from D1</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="114.75" cy="1175.76" width="229.5" height="24.4771"/> + <path d="M229.5 1163.52 L0 1163.52 L0 1188 L229.5 1188 L229.5 1163.52" class="st3"/> + <text x="3.88" y="1182.06" class="st8" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Delete entry2 from D1</text> </g> + <g id="shape56-141" v:mID="56" v:groupContext="shape" transform="translate(711.244,-662.779)"> + <title>Sheet.56</title> + <path d="M0 732.77 L0 1188" class="st12"/> + </g> + <g id="shape57-144" v:mID="57" v:groupContext="shape" transform="translate(664.897,-1045.31)"> + <title>Sheet.57</title> + <path d="M-0 1188 L146.76 1112.94" class="st13"/> + </g> + <g id="shape58-147" v:mID="58" v:groupContext="shape" transform="translate(619.059,-848.701)"> + <title>Sheet.58</title> + <path d="M432.2 1184.24 L-0 1188" class="st7"/> + </g> + <g id="shape59-150" v:mID="59" v:groupContext="shape" transform="translate(1038.62,-837.364)"> + <title>Sheet.59</title> + <desc>Critical sections</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="130" cy="1167.81" width="260.01" height="40.3845"/> + <path d="M260 1147.62 L0 1147.62 L0 1188 L260 1188 L260 1147.62" class="st3"/> + <text x="52.25" y="1174.99" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Critical sections</text> </g> + <g id="shape60-154" v:mID="60" v:groupContext="shape" transform="translate(621.606,-848.828)"> + <title>Sheet.60</title> + <path d="M0 1173.53 L0 1188" class="st7"/> + </g> + <g id="shape61-157" v:mID="61" v:groupContext="shape" transform="translate(824.31,-849.848)"> + <title>Sheet.61</title> + <path d="M0 1173.53 L0 1188" class="st7"/> + </g> + <g id="shape62-160" v:mID="62" v:groupContext="shape" transform="translate(345.944,-933.143)"> + <title>Sheet.62</title> + <path d="M705.32 1188 L0 1187.43" class="st7"/> + </g> + <g id="shape63-163" v:mID="63" v:groupContext="shape" transform="translate(1038.62,-915.684)"> + <title>Sheet.63</title> + <desc>Quiescent states</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="137.691" cy="1167.81" width="275.39" height="40.3845"/> + <path d="M275.38 1147.62 L0 1147.62 L0 1188 L275.38 1188 L275.38 1147.62" class="st3"/> + <text x="55.18" y="1174.99" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Quiescent states</text> </g> + <g id="shape64-167" v:mID="64" v:groupContext="shape" transform="translate(346.581,-932.442)"> + <title>Sheet.64</title> + <path d="M0 1173.53 L0 1188" class="st7"/> + </g> + <g id="shape65-170" v:mID="65" v:groupContext="shape" transform="translate(621.606,-933.461)"> + <title>Sheet.65</title> + <path d="M0 1173.53 L0 1188" class="st7"/> + </g> + <g id="shape66-173" v:mID="66" v:groupContext="shape" transform="translate(856.905,-934.481)"> + <title>Sheet.66</title> + <path d="M0 1173.53 L0 1188" class="st7"/> + </g> + <g id="shape67-176" v:mID="67" v:groupContext="shape" transform="translate(472.82,-756.389)"> + <title>Sheet.67</title> + <path d="M578.44 1188 L0 1187.43" class="st7"/> + </g> + <g id="shape68-179" v:mID="68" v:groupContext="shape" transform="translate(473.456,-755.688)"> + <title>Sheet.68</title> + <path d="M0 1173.53 L0 1188" class="st7"/> + </g> + <g id="shape69-182" v:mID="69" v:groupContext="shape" transform="translate(1016.87,-757.728)"> + <title>Sheet.69</title> + <path d="M0 1173.53 L0 1188" class="st7"/> + </g> + <g id="shape70-185" v:mID="70" v:groupContext="shape" transform="translate(1060.04,-738.651)"> + <title>Sheet.70</title> + <desc>while(1) loop</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="96.7728" cy="1167.81" width="193.55" height="40.3845"/> + <path d="M193.55 1147.62 L0 1147.62 L0 1188 L193.55 1188 L193.55 1147.62" class="st3"/> + <text x="31.03" y="1174.99" class="st6" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>while(1) loop</text> </g> + <g id="shape71-189" v:mID="71" v:groupContext="shape" transform="translate(190.02,-464.886)"> + <title>Sheet.71</title> + <path d="M0 1151.91 C0 1148.19 3.88 1145.17 8.66 1145.17 L43.29 1145.17 C48.13 1145.17 51.95 1148.19 51.95 1151.91 L51.95 + 1181.26 C51.95 1185.03 48.13 1188 43.29 1188 L8.66 1188 C3.88 1188 0 1185.03 0 1181.26 L0 1151.91 Z" + class="st1"/> + </g> + <g id="shape72-191" v:mID="72" v:groupContext="shape" transform="translate(259.003,-466.895)"> + <title>Sheet.72</title> + <desc>Reader thread is not accessing any shared data structure. i.e...</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="344.967" cy="1169.45" width="689.94" height="37.1049"/> + <path d="M689.93 1150.9 L0 1150.9 L0 1188 L689.93 1188 L689.93 1150.9" class="st3"/> + <text x="0" y="1162.25" class="st14" v:langID="1033"><v:paragraph/><v:tabList/>Reader thread is not accessing any shared data structure.<v:newlineChar/><tspan + x="0" dy="1.2em" class="st9">i.e. non critical section or quiescent state.</tspan></text> </g> + <g id="shape73-196" v:mID="73" v:groupContext="shape" transform="translate(190.02,-389.169)"> + <title>Sheet.73</title> + <desc>Dx</desc> + <v:textBlock v:margins="rect(4,4,4,4)"/> + <v:textRect cx="25.9746" cy="1166.58" width="51.95" height="42.8314"/> + <path d="M0 1152.31 C0 1148.39 1.43 1145.17 3.16 1145.17 L48.79 1145.17 C50.55 1145.17 51.95 1148.39 51.95 1152.31 L51.95 + 1180.86 C51.95 1184.83 50.55 1188 48.79 1188 L3.16 1188 C1.43 1188 0 1184.83 0 1180.86 L0 1152.31 Z" + class="st2"/> + <text x="12.9" y="1173.78" class="st15" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Dx</text> </g> + <g id="shape74-199" v:mID="74" v:groupContext="shape" transform="translate(259.003,-388.777)"> + <title>Sheet.74</title> + <desc>Reader thread is accessing the shared data structure Dx. i.e....</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="344.967" cy="1169.45" width="689.94" height="37.1049"/> + <path d="M689.93 1150.9 L0 1150.9 L0 1188 L689.93 1188 L689.93 1150.9" class="st3"/> + <text x="0" y="1162.25" class="st14" v:langID="1033"><v:paragraph/><v:tabList/>Reader thread is accessing the shared data structure Dx.<v:newlineChar/><tspan + x="0" dy="1.2em" class="st9">i.e. critical section.</tspan></text> </g> + <g id="shape75-204" v:mID="75" v:groupContext="shape" transform="translate(289.017,-301.151)"> + <title>Sheet.75</title> + <desc>Point in time when the reference to the entry is removed usin...</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="332.491" cy="1160.47" width="664.99" height="55.0626"/> + <path d="M664.98 1132.94 L0 1132.94 L0 1188 L664.98 1188 L664.98 1132.94" class="st3"/> + <text x="0" y="1153.27" class="st14" v:langID="1033"><v:paragraph/><v:tabList/>Point in time when the reference to the entry is removed <tspan + x="0" dy="1.2em" class="st9">using an atomic operation.</tspan></text> </g> + <g id="shape76-209" v:mID="76" v:groupContext="shape" transform="translate(177.543,-315.596)"> + <title>Sheet.76</title> + <desc>Delete</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="45.9546" cy="1166.58" width="91.91" height="42.8314"/> + <path d="M91.91 1145.17 L0 1145.17 L0 1188 L91.91 1188 L91.91 1145.17" class="st3"/> + <text x="11.25" y="1174.2" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Delete</text> </g> + <g id="shape77-213" v:mID="77" v:groupContext="shape" transform="translate(288,-239.327)"> + <title>Sheet.77</title> + <desc>Point in time when the writer can free the deleted entry.</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="328.5" cy="1167.81" width="657" height="40.3845"/> + <path d="M657 1147.62 L0 1147.62 L0 1188 L657 1188 L657 1147.62" class="st3"/> + <text x="0" y="1175.01" class="st14" v:langID="1033"><v:paragraph/><v:tabList/>Point in time when the writer can free the deleted entry.</text> </g> + <g id="shape78-217" v:mID="78" v:groupContext="shape" transform="translate(177.543,-240.744)"> + <title>Sheet.78</title> + <desc>Free</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="34.3786" cy="1166.58" width="68.76" height="42.8314"/> + <path d="M68.76 1145.17 L0 1145.17 L0 1188 L68.76 1188 L68.76 1145.17" class="st3"/> + <text x="11.25" y="1174.2" class="st11" v:langID="1033"><v:paragraph v:horizAlign="1"/><v:tabList/>Free</text> </g> + <g id="shape79-221" v:mID="79" v:groupContext="shape" transform="translate(289.228,-163.612)"> + <title>Sheet.79</title> + <desc>Time duration between Delete and Free, during which memory ca...</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="328.5" cy="1167.81" width="657" height="40.3845"/> + <path d="M657 1147.62 L0 1147.62 L0 1188 L657 1188 L657 1147.62" class="st3"/> + <text x="0" y="1160.61" class="st14" v:langID="1033"><v:paragraph/><v:tabList/>Time duration between Delete and Free, during which <tspan + x="0" dy="1.2em" class="st9">memory cannot be freed.</tspan></text> </g> + <g id="shape80-226" v:mID="80" v:groupContext="shape" transform="translate(187.999,-162)"> + <title>Sheet.80</title> + <desc>Grace Period</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="39.5985" cy="1166.58" width="79.2" height="42.8314"/> + <path d="M79.2 1145.17 L0 1145.17 L0 1188 L79.2 1188 L79.2 1145.17" class="st3"/> + <text x="0" y="1158.96" class="st11" v:langID="1033"><v:paragraph/><v:tabList/>Grace <tspan x="0" dy="1.2em" + class="st9">Period</tspan></text> </g> + <g id="shape83-231" v:mID="83" v:groupContext="shape" transform="translate(572.146,-1080.07)"> + <title>Sheet.83</title> + <path d="M9.3 1188 L9.66 1188 L132.49 1188" class="st16"/> + </g> + <g id="shape84-240" v:mID="84" v:groupContext="shape" transform="translate(599.196,-1042.14)"> + <title>Sheet.84</title> + <path d="M7.41 1188 L7.77 1188 L104.28 1188" class="st18"/> + </g> + <g id="shape85-249" v:mID="85" v:groupContext="shape" transform="translate(980.637,-595.338)"> + <title>Sheet.85</title> + <path d="M0 1188 L92.16 1188" class="st20"/> + </g> + <g id="shape86-254" v:mID="86" v:groupContext="shape" transform="translate(444.835,-603.428)"> + <title>Sheet.86</title> + <path d="M0 1145.17 L0 1188 L0 1145.17" class="st7"/> + </g> + <g id="shape87-257" v:mID="87" v:groupContext="shape" transform="translate(444.835,-637.489)"> + <title>Sheet.87</title> + <path d="M0 1188 L84.43 1186.61 L154.36 1153.31" class="st7"/> + </g> + <g id="shape88-260" v:mID="88" v:groupContext="shape" transform="translate(241.369,-607.028)"> + <title>Sheet.88</title> + <desc>Remove reference to entry2</desc> + <v:textBlock v:margins="rect(0,0,0,0)"/> + <v:textRect cx="96.7728" cy="1169.45" width="193.55" height="37.1049"/> + <path d="M193.55 1150.9 L0 1150.9 L0 1188 L193.55 1188 L193.55 1150.9" class="st3"/> + <text x="2.39" y="1163.15" class="st8" v:langID="1033"><v:paragraph v:horizAlign="2"/><v:tabList/>Remove reference <tspan + x="104.08" dy="1.2em" class="st9">to entry2</tspan></text> </g> + </g> +</svg> diff --git a/doc/guides/prog_guide/index.rst b/doc/guides/prog_guide/index.rst index 95f5e7964..17df2c563 100644 --- a/doc/guides/prog_guide/index.rst +++ b/doc/guides/prog_guide/index.rst @@ -56,6 +56,7 @@ Programmer's Guide metrics_lib bpf_lib ipsec_lib + rcu_lib source_org dev_kit_build_system dev_kit_root_make_help diff --git a/doc/guides/prog_guide/rcu_lib.rst b/doc/guides/prog_guide/rcu_lib.rst new file mode 100644 index 000000000..55d44e15d --- /dev/null +++ b/doc/guides/prog_guide/rcu_lib.rst @@ -0,0 +1,185 @@ +.. SPDX-License-Identifier: BSD-3-Clause + Copyright(c) 2019 Arm Limited. + +.. _RCU_Library: + +RCU Library +============ + +Lock-less data structures provide scalability and determinism. +They enable use cases where locking may not be allowed +(for ex: real-time applications). + +In the following paras, the term 'memory' refers to memory allocated +by typical APIs like malloc or anything that is representative of +memory, for ex: an index of a free element array. + +Since these data structures are lock less, the writers and readers +are accessing the data structures concurrently. Hence, while removing +an element from a data structure, the writers cannot return the memory +to the allocator, without knowing that the readers are not +referencing that element/memory anymore. Hence, it is required to +separate the operation of removing an element into 2 steps: + +Delete: in this step, the writer removes the reference to the element from +the data structure but does not return the associated memory to the +allocator. This will ensure that new readers will not get a reference to +the removed element. Removing the reference is an atomic operation. + +Free(Reclaim): in this step, the writer returns the memory to the +memory allocator, only after knowing that all the readers have stopped +referencing the deleted element. + +This library helps the writer determine when it is safe to free the +memory. + +This library makes use of thread Quiescent State (QS). + +What is Quiescent State +----------------------- +Quiescent State can be defined as 'any point in the thread execution where the +thread does not hold a reference to shared memory'. It is up to the application +to determine its quiescent state. + +Let us consider the following diagram: + +.. figure:: img/rcu_general_info.* + + +As shown, reader thread 1 accesses data structures D1 and D2. When it is +accessing D1, if the writer has to remove an element from D1, the +writer cannot free the memory associated with that element immediately. +The writer can return the memory to the allocator only after the reader +stops referencing D1. In other words, reader thread RT1 has to enter +a quiescent state. + +Similarly, since reader thread 2 is also accessing D1, writer has to +wait till thread 2 enters quiescent state as well. + +However, the writer does not need to wait for reader thread 3 to enter +quiescent state. Reader thread 3 was not accessing D1 when the delete +operation happened. So, reader thread 1 will not have a reference to the +deleted entry. + +It can be noted that, the critical sections for D2 is a quiescent state +for D1. i.e. for a given data structure Dx, any point in the thread execution +that does not reference Dx is a quiescent state. + +Since memory is not freed immediately, there might be a need for +provisioning of additional memory, depending on the application requirements. + +Factors affecting RCU mechanism +--------------------------------- + +It is important to make sure that this library keeps the overhead of +identifying the end of grace period and subsequent freeing of memory, +to a minimum. The following paras explain how grace period and critical +section affect this overhead. + +The writer has to poll the readers to identify the end of grace period. +Polling introduces memory accesses and wastes CPU cycles. The memory +is not available for reuse during grace period. Longer grace periods +exasperate these conditions. + +The length of the critical section and the number of reader threads +is proportional to the duration of the grace period. Keeping the critical +sections smaller will keep the grace period smaller. However, keeping the +critical sections smaller requires additional CPU cycles (due to additional +reporting) in the readers. + +Hence, we need the characteristics of small grace period and large critical +section. This library addresses this by allowing the writer to do +other work without having to block till the readers report their quiescent +state. + +RCU in DPDK +----------- + +For DPDK applications, the start and end of while(1) loop (where no +references to shared data structures are kept) act as perfect quiescent +states. This will combine all the shared data structure accesses into a +single, large critical section which helps keep the overhead on the +reader side to a minimum. + +DPDK supports pipeline model of packet processing and service cores. +In these use cases, a given data structure may not be used by all the +workers in the application. The writer does not have to wait for all +the workers to report their quiescent state. To provide the required +flexibility, this library has a concept of QS variable. The application +can create one QS variable per data structure to help it track the +end of grace period for each data structure. This helps keep the grace +period to a minimum. + +How to use this library +----------------------- + +The application must allocate memory and initialize a QS variable. + +Application can call ``rte_rcu_qsbr_get_memsize`` to calculate the size +of memory to allocate. This API takes maximum number of reader threads, +using this variable, as a parameter. Currently, a maximum of 1024 threads +are supported. + +Further, the application can initialize a QS variable using the API +``rte_rcu_qsbr_init``. + +Each reader thread is assumed to have a unique thread ID. Currently, the +management of the thread ID (for ex: allocation/free) is left to the +application. The thread ID should be in the range of 0 to +maximum number of threads provided while creating the QS variable. +The application could also use lcore_id as the thread ID where applicable. + +``rte_rcu_qsbr_thread_register`` API will register a reader thread +to report its quiescent state. This can be called from a reader thread. +A control plane thread can also call this on behalf of a reader thread. +The reader thread must call ``rte_rcu_qsbr_thread_online`` API to start +reporting its quiescent state. + +Some of the use cases might require the reader threads to make +blocking API calls (for ex: while using eventdev APIs). The writer thread +should not wait for such reader threads to enter quiescent state. +The reader thread must call ``rte_rcu_qsbr_thread_offline`` API, before calling +blocking APIs. It can call ``rte_rcu_qsbr_thread_online`` API once the blocking +API call returns. + +The writer thread can trigger the reader threads to report their quiescent +state by calling the API ``rte_rcu_qsbr_start``. It is possible for multiple +writer threads to query the quiescent state status simultaneously. Hence, +``rte_rcu_qsbr_start`` returns a token to each caller. + +The writer thread must call ``rte_rcu_qsbr_check`` API with the token to +get the current quiescent state status. Option to block till all the reader +threads enter the quiescent state is provided. If this API indicates that +all the reader threads have entered the quiescent state, the application +can free the deleted entry. + +The APIs ``rte_rcu_qsbr_start`` and ``rte_rcu_qsbr_check`` are lock free. +Hence, they can be called concurrently from multiple writers even while +running as worker threads. + +The separation of triggering the reporting from querying the status provides +the writer threads flexibility to do useful work instead of blocking for the +reader threads to enter the quiescent state or go offline. This reduces the +memory accesses due to continuous polling for the status. + +``rte_rcu_qsbr_synchronize`` API combines the functionality of +``rte_rcu_qsbr_start`` and blocking ``rte_rcu_qsbr_check`` into a single API. +This API triggers the reader threads to report their quiescent state and +polls till all the readers enter the quiescent state or go offline. This +API does not allow the writer to do useful work while waiting and +introduces additional memory accesses due to continuous polling. + +The reader thread must call ``rte_rcu_qsbr_thread_offline`` and +``rte_rcu_qsbr_thread_unregister`` APIs to remove itself from reporting its +quiescent state. The ``rte_rcu_qsbr_check`` API will not wait for this reader +thread to report the quiescent state status anymore. + +The reader threads should call ``rte_rcu_qsbr_quiescent`` API to indicate that +they entered a quiescent state. This API checks if a writer has triggered a +quiescent state query and update the state accordingly. + +The ``rte_rcu_qsbr_lock`` and ``rte_rcu_qsbr_unlock`` are empty functions. +However, when ``CONFIG_RTE_LIBRTE_RCU_DEBUG`` is enabled, these APIs aid +in debugging issues. One can mark the access to shared data structures on the +reader side using these APIs. The ``rte_rcu_qsbr_quiescent`` will check if +all the locks are unlocked. -- 2.17.1