Hi Andrei, Hans, Hraban,
On Sun, 2025-07-27 at 17:59 +0300, and...@borisov.dev wrote:
> We’ve been working on adding LuaMetaTeX and ConTeXt standalone support
> for the Markdown package (https://github.com/Witiko/markdown/pull/557)
> and encountered an interesting issue.
>
> The package loads quite a huge library — `expl3-code.tex` from
> `l3kernel` package which eats a lot of time during compilation.
>
> What would be a good way to mitigate this?
Some options:
- Manually install v2.13.0 of the Markdown package, which doesn't
require expl3.
- Use pandoc:
https://github.com/gucci-on-fleek/lua-widow-control/blob/b08ddbcd/docs/manual/lwc-manual.mkxl#L378-L402
- Convince Vit to rewrite the ConTeXt interface in Lua, similar to
https://github.com/Witiko/markdown/issues/215#issuecomment-1359250887
- Use the builtin Markdown module, which was just updated in yesterday's
release.
> Is there a way to pre-load the library before compilation? Perhaps by
> making a custom ConTeXt format and making it with `context --make`?
>
> Unfortunately, I haven’t found a documentation on how to make your own
> custom formats, hence this question.
This is a terrible idea, and you definitely should not recommend any of
your users to do this, but if you're just looking for a hack to speed up
your CI/testing, it is possible. The trick is to override one of
ConTeXt's core files (which is why this is a terrible idea).
"libs-ini.mkxl" is a good choice for this since it's loaded near the end
and isn't used by most documents.
So, make a file called "$TEXMFHOME/tex/context/third/libs-ini.mkxl" with
the following contents:
\directlua{function pdf.getcreationdate() end} % expl3 bug
\usemodule[expl3-generic]
and then you can run
$
TEXMFHOME='{/PATH/TO/TEXLIVE/texmf-dist/tex/latex-dev/,/PATH/TO/TEXMFHOME/}'
context --make
You need to put "latex-dev" into the search path since only the
prerelease expl3 versions currently work with ConTeXt. Again, I don't
recommend this since it is very likely that a future ConTeXt update will
break this, and building expl3 into the format might break core parts of
ConTeXt, but it's certainly possible.
> What is the process of adding thrid-party modules in the default
> list of ConTeXt distribution (https://modules.contextgarden.net/)?
It's fairly simple, you essentially just need to sign up for an account
and then upload a zip file. However, the ConTeXt Standalone Distribution
doesn't include expl3, so just adding the Markdown module alone probably
wouldn't be very useful.
On Sun, 2025-07-27 at 17:53 +0200, Hans Hagen via ntg-context wrote:
> Actually, one can wonder what expl3 bring to context at all; some
> intermediate layer like that just doesn't fit in.
No one would ever write a ConTeXt-native module using expl3, but if
you're using it for a LaTeX package anyways, then the ConTeXt support
comes for "free" (from the developers side; you'll still pay for it in a
much slower runtime).
> So, can't you avoid loading the bottleneck expl code? Skip unicode stuff
> as a start? You probably then end up below a second. Boosting that code
> (by looking at it i might spot some) is not on my agenda.
Most of the Markdown module is written in Lua; only the TeX interface
parts use expl3. I suspect that the easiest option would be to simply
rewrite the TeX interface for ConTeXt using Lua. I actually tried doing
this in March 2024 (and then got distracted and just switched to using
Pandoc); the Markdown module has changed its interface since then, but
it still works with TL23 and should be easy to port to the latest
version. I've attached the file that I used, so feel free to use that as
inspiration.
On Sun, 2025-07-27 at 18:07 +0200, Henning Hraban Ramm wrote:
> Am 27.07.25 um 16:59 schrieb and...@borisov.dev:
> > The package loads quite a huge library — `expl3-code.tex` from
> > `l3kernel` package which eats a lot of time during compilation.
>
> What has expl3 to do with ConTeXt?
It's a generic programming layer for TeX, much like pgfkeys/pgfmath (the
non-graphics parts of TikZ). It was originally designed for LaTeX, but
its authors have put quite a bit of effort into making sure that it's
usable in all formats, including ConTeXt.
Thanks,
-- Max
local markdown = require "markdown"
local headings = {
"chapter",
"section",
"subsection",
"subsubsection",
"subsubsubsection",
}
local fmt = string.formatters
local _writer = {
ellipsis = [[[\dots]],
code = [[\type{%s}]],
space = [[\space]],
hard_line_break = [[\crlf]],
nbsp = [[\nobreakspace]],
strong = [[\bold{%s}]],
emphasis = [[\emph{%s}]],
inline_html_tag = [[\type{%s}]],
block_html_element = [[\type{%s}]],
verbatim = [[\type{%s}]],
thematic_break = [[\blackrule]],
interblocksep = [[\par]],
string = false,
paragraph = false,
plain = false,
inline_html_comment = function() end,
block_html_comment = function() end,
document = [[\starttext %s \stoptext]],
blockquote = [[\startquotation %s \stopquotation ]],
link = function(label, url, title, attributes)
return fmt[ [[\goto{%s}{url(%s)}]] ](label, url)
end,
image = function(label, url, title, attributes)
return fmt[ [=[\externalfigure[%s]]=] ](url)
end,
bulletlist = fmt[[\startitemize %s \stopitemize ]],
heading = function(content, level)
return fmt[ [[\%s{%s}]] ](headings[level], content)
end,
}
local writer = table.setmetatableindex({}, function(t, k)
local func = _writer[k]
if func == false then
func = fmt["%s"]
elseif type(func) == "string" then
func = fmt[func .. "\n"]
end
return function(...)
local args = { ... }
for i, v in ipairs(args) do
if type(v) == "table" then
args[i] = table.concat(v)
end
end
local out = func(table.unpack(args))
print(out)
return out
end
end)
local convert = markdown.reader.new(writer, {
html = true,
shiftHeadings = 1,
eagerCache = false
}).finalize_grammar({})
local out = convert[[
# Title
Hello *World*
]]
context.startext()
context(out)
context.stoptext()
___________________________________________________________________________________
If your question is of interest to others as well, please add an entry to the
Wiki!
maillist : ntg-context@ntg.nl /
https://mailman.ntg.nl/mailman3/lists/ntg-context.ntg.nl
webpage : https://www.pragma-ade.nl / https://context.aanhet.net (mirror)
archive : https://github.com/contextgarden/context
wiki : https://wiki.contextgarden.net
___________________________________________________________________________________
