Hi Julian, On 01/09/2018 03:28 PM, Willy Tarreau wrote: > Hi Julian, > > On Tue, Jan 09, 2018 at 08:50:48AM +0000, Julian Zhu wrote: >> We are testing haproxy+QAT card(Intel QuickAssist-Technology) and find that >> the memory usage of haproxy+QAT is much higher than that of haproxy alone. >> Under the same traffic (about 120K connections), haproxy alone only takes 6G >> memory while haproxy+QAT takes 36G. >> The only difference in config is as below for haproxy+QAT: >> ...... >> ssl-engine qat >> ssl-mode-async >> ...... >> and the memory is not released even if we terminate all haproxy processes >> until we reboot the server. > > Then this indicates that the leak (if any but it sounds like this) is in > the engine. You'd need to unload and reload the modules to see if it's a > real leak or just some extreme memory usage. > >> OS: >> Centos-7 >> >> SW version: >> haproxy-1.8.3 >> openssl-1.1.0e >> QAT_Engine: 0.5.20 >> QAT1.7.Upstream.L.1.0.3_42 >> >> Does anyone encounter the same issue? Is there any way to debug or fix it? > > Unfortunately at this point it will be entirely dependant on QAT, and I > don't really know what can be done there. Maybe there's a debug mode. You > should possibly check with other versions of the engine (higher or lower). > If you're still experiencing the leak with an up to date version, you > should contact your intel support so that they can take a look at this. > > It's possible that haproxy triggers the leak (ie does something not cool > triggering an unusual code path leading to a leak in the engine), so they > may not be aware of it yet. > > Hoping this helps, > Willy >
There is no specific memory allocation in the code of haproxy to handle the async engines. In addition, we are also using an other dfferent engine than QAT which is using a the "async engine mode" and we didn't notice any memory consumption. So the issue seems related to the openssl ".so" used for the engine. As Willy told you, you should try a different QAT version. R, Emeric
