RE: Question on Lua plugin differences between ATS 7.1.x and ATS 9.x.

[CHOU, PETER]

Kit,

This is regarding Lua plugin remap instances. With our production configuration 
and build,
we are leaking almost 1GB of memory on each remap config reload. This is being 
verified in two ways --

First is overall process memory usage:

ps -o pid,comm,vsz -p <traffic_server_pid>

Second is the remap reload will fail due to out of memory after 2GB memory 
usage is exceeded when using the
Ubuntu distribution default LuaJIT which is limited to 2GB maximum. With latest 
LuaJIT which allows more
memory you can keep going until process is killed by OOM manager.

For lab testing, non production code, I use a test lua script with the 
following --

collectgarbage("count")

My test remap line --

map http://localhost/0 http://localhost/ @plugin=tslua.so @pparam=--states=1 
@pparam=/home/pebc/test.lua @pparam=100

My test script --

function __init__(args)

  ts.debug("Running __init__()...")

  local count = 0
  local t = {}
  t[0] = "TEST"

  local FH = io.open("/dev/urandom", "rb")

  if args[1] == nil then
    print("Agument required. Usage " .. args[0] .. " <MB>")
    return -1
  end

  for i = 1, args[1], 1 do
     t[i] = FH:read(1024000)
     count = count + string.len(t[1])
  end

  io.close(FH)

  ts.error("__init__ : t[0] = " .. t[0])
  ts.error("Total bytes allocated = " .. count .. " bytes")
  ts.error("Garbage Collector reports = " ..
                collectgarbage("count") .. " MB")

end

function do_remap()

   ts.debug("Running do_remap()...")
end

Thanks,
Peter

-----Original Message-----
From: Shu Kit Chan <chanshu...@gmail.com> 
Sent: Saturday, March 12, 2022 9:15 AM
To: us...@trafficserver.apache.org
Cc: dev@trafficserver.apache.org
Subject: Re: Question on Lua plugin differences between ATS 7.1.x and ATS 9.x.

Is this a lua script you in plugin.config or remap.config ?

Also what method do you use to see if gc is happening or not ?

On Fri, Mar 11, 2022 at 1:22 PM CHOU, PETER <pc6...@att.com> wrote:
>
> Hi,
>
>
>
> We have quite a few Lua scripts that were written for use with ATS 7.1.4 and 
> earlier. We are looking to migrate to ATS 9.1.1, but it seems like the 
> garbage collection that is supposed to happen during a “traffic_ctl config 
> reload” (reload a remap configuration file containing remap lines using the 
> Lua plugin) does not seem to be happening.
>
>
>
> Back in 2016, my colleague submitted a PR (commit 7e52a) which called the 
> LuaJIT garbage collector during the remap instance new and delete operations 
> within the Lua plugin.
>
>
>
> I noticed that in 2020 these garbage collector calls were removed by another 
> PR (commit 36a8cd). However, this does not seem to be the cause of the issue 
> since the garbage collection still does not happen even if I revert this 
> commit in my local tree.
>
>
>
> Our Lua scripts are using the following form –
>
>
>
> local <variables> [ I believe this are file scope local variables when 
> declared this way. ]
>
> …
>
> function a()
>
> function b()
>
> …
>
>
>
> We do *not* use any __clean__ functions (and have not had to in the past).
>
>
>
> I have tested with both 9.1.1 and 9.2.x.
>
> I have tested with both Ubuntu 20.04 LTS default LuaJIT (allows 2GB in Linux) 
> and latest LuaJIT (removes 2GB limit). Both show version v2.1.0-beta3.
>
>
>
> So is this change by design, or is it an issue?
>
>
>
> Thanks,
>
> Peter Chou
>
>