Kit, Looks like the memory limit is lifted with the compiler flag indicated in the link.
I used the repository linked to from luajit.org -- http://luajit.org/git/luajit-2.0.git rather than the one in the link (since it is official). # Ubuntu 18.04 LTS Lua non-JIT -- tested to 3GB Lua 5.2.4 Copyright (C) 1994-2015 Lua.org, PUC-Rio Total bytes allocated = 3072000000 bytes Garbage Collector reports = 3540307.8173828 MB # LuaJit 2.1.0 beta 3 with LUAJIT_ENABLE_GC64 -- tested to 3GB LuaJIT 2.1.0-beta3 -- Copyright (C) 2005-2017 Mike Pall. http://luajit.org/ Total bytes allocated = 3072000000 bytes Garbage Collector reports = 3001315.9365234 MB # Ubuntu 18.04 LTS LuaJit no GC64 -- out of memory at 2GB LuaJIT 2.1.0-beta3 -- Copyright (C) 2005-2017 Mike Pall. http://luajit.org/ /usr/bin/luajit: not enough memory Total bytes allocated = 2082816000 bytes Garbage Collector reports = 2035123.3154297 MB # ATS master with dev luajit with LUAJIT_ENABLE_GC64 -- tested to 4GB (global plugin mode, states = 1) [Oct 8 15:37:56.176] traffic_server DIAG: (ts_lua) Total bytes allocated = 4094976000 bytes [Oct 8 15:37:56.176] traffic_server DIAG: (ts_lua) Garbage Collector reports = 4000597.3740234 MB [Oct 8 15:37:56.191] traffic_server DIAG: (ts_lua) Total bytes allocated = 4096000000 bytes [Oct 8 15:37:56.191] traffic_server DIAG: (ts_lua) Garbage Collector reports = 4001597.53125 MB # ATS master with Ubuntu luajit no GC64 -- out of memory at 2GB [Oct 8 15:48:49.619] traffic_server DIAG: (ts_lua) Total bytes allocated = 2075648000 bytes [Oct 8 15:48:49.619] traffic_server DIAG: (ts_lua) Garbage Collector reports = 2028155.7460938 MB -----Original Message----- From: Shu Kit Chan <chanshu...@gmail.com> Sent: Thursday, June 20, 2019 10:33 PM To: dev <dev@trafficserver.apache.org> Subject: Re: Lua plugin memory limit on Linux. I caught up with some readings in the latest LuaJIT. It looks like we can even do away with the 2GB limit. https://urldefense.proofpoint.com/v2/url?u=https-3A__stackoverflow.com_questions_35155444_why-2Dis-2Dluajits-2Dmemory-2Dlimited-2Dto-2D1-2D2-2Dgb-2Don-2D64-2Dbit-2Dplatforms&d=DwIBaQ&c=LFYZ-o9_HUMeMTSQicvjIg&r=8c5kS62dKm3-obVyLvkwkc-kTTgV1vAsbxSPwL-yi3o&m=zEftzdzomErVFP0eMDQg_KyHI4MQqPL0bLT-P7zSG5Q&s=HD_iUwY4JuH3KveWNBiSbQ7RfyyTlgmAn1MtuOkdgs0&e= Would be nice if you can try that out in your setup (basically apply a patch and re-build with LUAJIT_ENABLE_GC64) If it works, we should definitely document it. Thanks. Kit On Thu, Jun 20, 2019 at 1:35 PM Chou, Peter <pbc...@labs.att.com> wrote: > > Kit, > > This is the only error message seen from diags.log. It did not cause the ATS > to core dump. > > [Jun 20 13:27:45.258] traffic_server NOTE: plugin.config loading ... > [Jun 20 13:27:45.259] traffic_server NOTE: loading plugin > '/home/pebc/opt/ats-gh-master/libexec/trafficserver/tslua.so' > [Jun 20 13:28:14.337] traffic_server ERROR: [ts_lua_add_module] > lua_pcall > /home/pebc/opt/ats-gh-master/etc/trafficserver/test-memory.lua failed: > not enough memory [Jun 20 13:28:14.337] traffic_server ERROR: > [ts_lua][TSPluginInit] ts_lua_add_module failed [Jun 20 13:28:14.337] > traffic_server NOTE: plugin.config finished loading > > This is how much memory was allocated before the failure -- > > [Jun 20 13:28:14.323] traffic_server DIAG: (ts_lua) Total bytes > allocated = 2071552000 bytes [Jun 20 13:28:14.323] traffic_server > DIAG: (ts_lua) Garbage Collector reports = 2024154.9023438 MB > > Modified the Lua script as follows -- > > function __init__(argtb) > count = 0 > t = {} > max = 2100 > > FH = io.open("/dev/urandom", "rb") > > for i = 1, max, 1 do > t[i] = FH:read(1024000) > count = count + string.len(t[1]) > if i > 2000 then > ts.debug("Total bytes allocated = " .. count .. " bytes") > ts.debug("Garbage Collector reports = " .. collectgarbage("count") > .. " MB") > end > end > > io.close(FH) > > end > > Thanks, > Peter > > -----Original Message----- > From: Shu Kit Chan <chanshu...@gmail.com> > Sent: Wednesday, June 19, 2019 11:32 PM > To: dev <dev@trafficserver.apache.org> > Subject: Re: Lua plugin memory limit on Linux. > > Thanks for the info. > > That's good to know. Have you tried how much more you can allocate? > And when you are over the limit, what is the message you are getting? > > Perhaps we can update the doc with this information. > > Kit > > On Wed, Jun 19, 2019 at 12:01 PM Chou, Peter <pbc...@labs.att.com> wrote: > > > > Kit, > > > > Did not hear back from you regarding the memory limit for the Lua plugin on > > Linux. > > > > Previously, I had tested on the command line with the luajit executable, > > but I have completed some testing with the ATS master branch. > > > > I am running with Ubuntu 18.04 LTS Server 64-bit which ships with 'LuaJIT > > 2.1.0-beta3'. > > > > Using the following configuration, I got good results, able to allocated up > > to 2GB memory. > > > > plugin.config - > > > > tslua.so --states=1 test-memory.lua > > > > test-memory.lua - > > > > function __init__(argtb) > > count = 0 > > t = {} > > max = 2000 > > > > FH = io.open("/dev/urandom", "rb") > > > > for i = 1, max, 1 do > > t[i] = FH:read(1024000) > > count = count + string.len(t[1]) > > end > > > > io.close(FH) > > > > ts.debug("Total bytes allocated = " .. count .. " bytes") > > ts.debug("Garbage Collector reports = " .. > > collectgarbage("count") .. " MB") end > > > > Result - > > > > pebc@piplup2:~/opt/ats-gh-master/bin$ ./traffic_server -V Traffic > > Server 9.0.0 Jun 18 2019 14:16:03 piplup2 > > traffic_server: using root directory '/home/pebc/opt/ats-gh-master/' > > Apache Traffic Server - traffic_server - 9.0.0 - (build # 061814 on > > Jun 18 2019 at 14:16:03) pebc@piplup2:~/opt/ats-gh-master/bin$ > > ./traffic_server Traffic Server 9.0.0 Jun 18 2019 14:16:03 piplup2 > > traffic_server: using root directory '/home/pebc/opt/ats-gh-master/' > > [Jun 19 11:54:42.681] traffic_server DIAG: (ts_lua) Total bytes > > allocated = 2048000000 bytes [Jun 19 11:54:42.681] traffic_server > > DIAG: (ts_lua) Garbage Collector reports = 2001149.2382813 MB > > > > Appreciate it if you could take a look and validate these results. > > > > Thanks, > > Peter
