I think it may be due to
https://github.com/apache/trafficserver/blob/master/plugins/lua/ts_lua.c#L456-L466
I believe we do have ways to handle this better now in 9.x so we just
need to fix the plugin.
Let me take a look at it in the next 2 weeks and you can create an
issue to track it on github as well
Thanks.
Kit
On Mon, Mar 14, 2022 at 10:36 AM CHOU, PETER <pc6...@att.com> wrote:
>
> Sorry, in the test script the line --
>
> local t = {}
>
> should be at the top of the script (file scope) not within the __init__()
> function.
>
> I was testing the behavior, of course function local variable is garbage
> collected :-).
>
> However, file local or global are not.
>
> So correct script --
>
> local t = {}
>
> function __init__(args)
>
> ts.debug("Running __init__()...")
>
> local count = 0
> t[0] = "TEST"
>
> local FH = io.open("/dev/urandom", "rb")
>
> if args[1] == nil then
> print("Agument required. Usage " .. args[0] .. " <MB>")
> return -1
> end
>
> for i = 1, args[1], 1 do
> t[i] = FH:read(1024000)
> count = count + string.len(t[1])
> end
>
> io.close(FH)
>
> ts.error("__init__ : t[0] = " .. t[0])
> ts.error("Total bytes allocated = " .. count .. " bytes")
> ts.error("Garbage Collector reports = " ..
> collectgarbage("count") .. " MB")
>
> end
>
> function do_remap()
>
> ts.debug("Running do_remap()...")
> end
>
> -----Original Message-----
> From: CHOU, PETER
> Sent: Monday, March 14, 2022 10:29 AM
> To: dev@trafficserver.apache.org; us...@trafficserver.apache.org
> Subject: RE: Question on Lua plugin differences between ATS 7.1.x and ATS 9.x.
>
> Kit,
>
> This is regarding Lua plugin remap instances. With our production
> configuration and build, we are leaking almost 1GB of memory on each remap
> config reload. This is being verified in two ways --
>
> First is overall process memory usage:
>
> ps -o pid,comm,vsz -p <traffic_server_pid>
>
> Second is the remap reload will fail due to out of memory after 2GB memory
> usage is exceeded when using the Ubuntu distribution default LuaJIT which is
> limited to 2GB maximum. With latest LuaJIT which allows more memory you can
> keep going until process is killed by OOM manager.
>
> For lab testing, non production code, I use a test lua script with the
> following --
>
> collectgarbage("count")
>
> My test remap line --
>
> map http://localhost/0 http://localhost/ @plugin=tslua.so @pparam=--states=1
> @pparam=/home/pebc/test.lua @pparam=100
>
> My test script --
>
> function __init__(args)
>
> ts.debug("Running __init__()...")
>
> local count = 0
> local t = {}
> t[0] = "TEST"
>
> local FH = io.open("/dev/urandom", "rb")
>
> if args[1] == nil then
> print("Agument required. Usage " .. args[0] .. " <MB>")
> return -1
> end
>
> for i = 1, args[1], 1 do
> t[i] = FH:read(1024000)
> count = count + string.len(t[1])
> end
>
> io.close(FH)
>
> ts.error("__init__ : t[0] = " .. t[0])
> ts.error("Total bytes allocated = " .. count .. " bytes")
> ts.error("Garbage Collector reports = " ..
> collectgarbage("count") .. " MB")
>
> end
>
> function do_remap()
>
> ts.debug("Running do_remap()...")
> end
>
> Thanks,
> Peter
>
> -----Original Message-----
> From: Shu Kit Chan <chanshu...@gmail.com>
> Sent: Saturday, March 12, 2022 9:15 AM
> To: us...@trafficserver.apache.org
> Cc: dev@trafficserver.apache.org
> Subject: Re: Question on Lua plugin differences between ATS 7.1.x and ATS 9.x.
>
> Is this a lua script you in plugin.config or remap.config ?
>
> Also what method do you use to see if gc is happening or not ?
>
> On Fri, Mar 11, 2022 at 1:22 PM CHOU, PETER <pc6...@att.com> wrote:
> >
> > Hi,
> >
> >
> >
> > We have quite a few Lua scripts that were written for use with ATS 7.1.4
> > and earlier. We are looking to migrate to ATS 9.1.1, but it seems like the
> > garbage collection that is supposed to happen during a “traffic_ctl config
> > reload” (reload a remap configuration file containing remap lines using the
> > Lua plugin) does not seem to be happening.
> >
> >
> >
> > Back in 2016, my colleague submitted a PR (commit 7e52a) which called the
> > LuaJIT garbage collector during the remap instance new and delete
> > operations within the Lua plugin.
> >
> >
> >
> > I noticed that in 2020 these garbage collector calls were removed by
> > another PR (commit 36a8cd). However, this does not seem to be the cause of
> > the issue since the garbage collection still does not happen even if I
> > revert this commit in my local tree.
> >
> >
> >
> > Our Lua scripts are using the following form –
> >
> >
> >
> > local <variables> [ I believe this are file scope local variables when
> > declared this way. ]
> >
> > …
> >
> > function a()
> >
> > function b()
> >
> > …
> >
> >
> >
> > We do *not* use any __clean__ functions (and have not had to in the past).
> >
> >
> >
> > I have tested with both 9.1.1 and 9.2.x.
> >
> > I have tested with both Ubuntu 20.04 LTS default LuaJIT (allows 2GB in
> > Linux) and latest LuaJIT (removes 2GB limit). Both show version
> > v2.1.0-beta3.
> >
> >
> >
> > So is this change by design, or is it an issue?
> >
> >
> >
> > Thanks,
> >
> > Peter Chou
> >
> >
