Hi,
just pushed https://github.com/juanpablo-santos/jspwiki-markdown
Briefly:
There are two custom renderers (normal and wysiwyg editing) and one custom
parser. All of them delegate to Flexmark's
specific parser/renderers, which are a bit different to JSPWiki's ones. In
any case, these Flexmark's parser/renderers are
built using MarkdownDocument.options( context ), to set up Flexmark
parsing/rendering with an custom extension which
works (only) with Markdown Links.
The parser is able of building the (markup) node structure which will get
translated to html/dom nodes; that is,
is only able of adding and deleting (markup) nodes, but it can't set any
html attribute, because there isn't any html yet
(markup nodes don't hold html attributes).
The renderers are more or less the opposite: able of modifying the
generated html, but can't modify its structure: you
may be able here to change and href, f.ex., but you won't be able to add or
remove additional markup.
This way of working is by Flexmark's design, broadly speaking.
On both cases, the behaviour is similar, the parser/renderer ends
delegating to a class inside org.apache.wiki.markdown,
which decides the type of action to perform on the Link's type, and a bunch
of classes following the State pattern,
performing the required, concrete, actions (i.e., parse and render a plugin
or a wiki variable, etc.).
There are also a couple of utility classes. Of them, LinkParsingOperations
contains several methods scattered throughout
JSPWiki source code. IMO, it should replace those functions and, before
2.11, those functions should proxy to this class,
as all those operations seem to belong to a common class.
Last but not least, noticed another issue with caching turned on: on
WYSIWYG editors, whole page is rendered, including
plugins, variable names, etc. I suspect this has to do more with JSPWiki
itself, caching too early and how the markdown
is parsed/rendered, but still have to get a good look at it. Thorough
end-to-end testing is needed O:-)
My idea is to let that repo a week or two so it can be studied (maybe
commit small fixes or something along that lines)
and, if no concerns are raised, move it to jspwiki master, most probably as
an standalone module (it will be preceded
by another mail to discuss those specifics).
br,
juan pablo
On Sun, Nov 12, 2017 at 9:41 PM, Juan Pablo Santos Rodríguez <
juanpablo.san...@gmail.com> wrote:
> Hi Dirk,
>
> lot of stuff to cover (meaning: long mail following), will try to address
> all the points:
>
> Initial design
> ------------------
>
> yes, the solution, as it is made consists on a MarkdownParser /
> MarkdownRendered,
> which acts as an alternative to current Parser/Renderer. There's no
> support for mixed
> markup, but I suppose that could be added too, via another
> Parser/Renderer, which
> could act as a Facade, selecting an underlying markup processing depending
> on user
> preferences (or whatever). But that raises more questions, though: once
> the page is
> stored on a given markup, what happens when a user with a different markup
> stored in
> his/her preferences? The type of markup should be also stored as a
> WikiPage attribute...
>
> This was done as POC for evaluating using JSPWiki as a Markdown
> content-publising
> tool. For that flexmark is used, which does 99% of the markup parsing, so
> moving this
> to the client side would imply a totally different take on this. I'm still
> extracting markdown
> support from other custom code, so I expect that by Tuesday/Wednesday I'll
> have it
> uploaded to a github repo.
>
> As it is now, once you commit to a markup style, you're set with that. At
> least as a
> starting point, it's possible to add mixed markup support later on.
>
> The Parser/Renderer is based on Flexmark + a custom JSPWiki extension.
> Currently
> it supports:
> * Normal Markdown
> * Wiki links (internal, external, interwiki, edit, etc.)
> * Variables
> * Almost all plugins (more on this below)
> * Inline images
> * Footnotes
> * ACLs
> * Metadata
> * WYSIWYG edition
>
> The generated HTML markup specific to JSPWiki is almost, if not equal, to
> the current
> JSPWiki markup Parser/Renderer.
>
> The editors
> ----------------
>
> I've been skimming through the editors. So far, it seems to me that the js
> files operate
> on generated html (no jspwiki markup handling), so that shouldn't be a
> problem, and the
> JSPs doesn't seem to perform any transformation.
>
> As this was done as a POC, I've been focusing mostly on the
> parsing/rendering that on
> anything else, so it's pretty possible that I've missed something on the
> editors side, but
> without digging on them too much, they seemed to play along well.
>
> Oh, and of course, adaptations to the plain editor *need* to be done. I'm
> thinking that the
> different parsers should make available their configuration up to the
> editors, so the plain
> editor is able to generate the appropiate markup for each parser.
>
> Migration
> -------------
>
> there's nothing yet, as the POC I was working on assumed no previous
> content, so I haven't
> thought a lot about this... As always, something can/could/should be done:
> there is a
> flexmark extension (haven't investigated that it, though) which performs
> an html to markdown
> conversion. Given that there are unit tests which transform current
> markdown to html, the
> process seems feasible, at least for simple pages. This process would not
> be straightforward
> JSPWikiMarkup -> HTML -> Markdown, though, some intermediate "polishing"
> would be
> likely to happen (I'm thinking on %%css styles, or identifying links which
> are wiki links,
> instead of full links).
>
> This process could also be made available as a maven plugin...
>
> As for the other way round (Markdown -> JSPWikiMarkup): haven't even
> thought of it yet.
> In any case, is migrating from one markup to another something likely to
> happen? Once
> you're using one markup and have a bunch of pages, what would trigger the
> need of migrating
> to another markup? Maybe if you're on JSPWikiMarkup you are thinking on
> moving to
> Markdown b/c that way you can also use the pages as Github pages, or
> something like
> that (although some markup would be lost). As said before, haven't thought
> on this, so any
> opinion is more than welcome...
>
> Yet to be done
> ---------------------
>
> * Plain editor toolbar support
>
> * Initial set of WikiPages for markdown
>
> * %%css constructs. A new extension for this should be made, there is no
> support for this.
>
> * markup migration tool?
>
> * plugins implementing HeadingListener (that is, TableOfContents) are not
> supported: TableOfContents
> plugin implements an ad-hoc interface, HeadingListener, which is fired by
> JSPWikiMarkupParser every
> time it finds a header (more precisely, for every heading,
> JSPWikiMarkupParser generates a "#" link
> with the section reference and then registers a HeadingListener).
>
> When this plugin is executed then, it knows about the different sections,
> so it can generate the TOC.
> The way flexmarks parses and renders markdown, doesn't allow to generate
> the TOC this way. Initially,
> the Markdown Parser/Renderer generated those #-links, but as soon I
> realized TableOfContents wasn't
> usable under markdown, I removed the generation of #-links and instead of
> executing the plugin,
> switched it to flexmark's own TOC extension, surrounded with some divs. So
> as for now, whereas
> TableOfContents generates something like:
>
> <div class="toc">
> <div class="collapsebox">
> <h4 id="section-TOC">title</h4>
> <ul>/<ol> <!-- either ul or ol -->
> <li class="toclevel-1">
> <li class="toclevel-2">
> <li class="toclevel-3">
> </div>
> </div>
>
> the markdown parser/render currently generates something like:
> <div class="toc">
> <div class="collapsebox">
> <h4 id="section-TOC">title</h4>
> <!-- Generated by Flexmark's TOC extension -->
> <ul>/<ol> <!-- either ul or ol -->
> <li> <!--level 1 -->
> <ul>/<ol> <!-- either ul or ol -->
> <li> <!--level 2 -->
> <ul>/<ol> <!-- either ul or ol -->
> <li></li> <!-- level 3 -->
> </ul>/</ol>
> </li>
> </ul>/</ol>
> </li>
> </ul>/</ol>
> <!-- End generated by Flexmark's TOC extension -->
> </div>
> </div>
>
> So there are two ways of adding support for this:
> a) add enough css so that both html render more or less the same.
> Parameters parsing should be implemented anyways.
> b) add a new Flexmark extension for TOC, probably forking+adapting the
> existing TOC extension.
>
> Hope this clarifies a bit.
>
> br,
> juan pablo
>
> On Sat, Nov 11, 2017 at 9:36 AM, Dirk Frederickx <
> dirk.frederi...@gmail.com> wrote:
>
>> Hi Juan,
>>
>> Adding markup support would indeed be an excellent extension for jspwiki.
>>
>>
>> When changing the parser/render as you proposed, this would mean that
>> for one jspwiki instance the backend storage would be changed from
>> jspwiki-markup to markdown.
>>
>> I think this has some important limitations:
>> - you can not use both markup styles on the same wiki instance
>> so users familiar with jspwiki-markup would be forced to switch
>> - we need a solution for migrating a wiki repository between both markups
>> to allow smooth transitions
>> - we'll need a rewrite the wysiwyg editors, as they are converting
>> between html and jspwiki-markup
>> - as you indicated we need some adaptions to the plain editor
>> but I believe this would mainly be configuration (I can help with
>> that)
>> (configuration to adapt the auto-completions, ...)
>>
>> ----
>>
>> Alternatively, we could look for a solution whereby a user can either use
>> markdown
>> or jspwiki-markup, whenever he/she choses to.
>> In this case, the backend would continue to use jspwiki-markup;
>> during editing the user can chose another type of input.
>>
>> The user would then select a "markdown editor" store in its
>> UserPreferences.
>> (similar as chosing a wysiwyg editor)
>> Markdown would be used during editing, and converted to jspwiki-markup
>> during saving.
>> With live-preview the user will see immediately the rendered page.
>>
>> This approach is similar how the wysiwyg editors work, translating
>> between
>> HTML
>> and jspwiki-markup when saving the page.
>>
>>
>> A topic to be addressed is the fact that markdown has a few more features
>> than jspwiki-markup.
>> For example, with markdown you can go to 6 levels with headers, jspwiki
>> has only 3 levels.
>> I guess most of these cases would be solvable through the use of specific
>> %%css-class constructs.
>> To be validated.
>>
>> ----
>>
>> Finally, we could also opt for a solution to mix different markup styles
>> in
>> one page.
>> You can write a page in jspwiki-markup, but if you want to copy/paste
>> some
>> markdown, it can be put inside a PLUGIN or a %%dynamic-style.
>>
>> The syntax of a PLUGIN (server side markup conversion) may be a bit too
>> complex
>> for most users.
>>
>> [{MarkdownPlugin
>>
>> ####This Is a Header####
>> }]
>>
>> Taking the markdown conversion to the browser-level (with javascript)
>> would
>> be feasible as well.
>> (eg showdown http://www.showdownjs.com/ )
>> %%markdown
>>
>> ####This Is a Header####
>> /%
>>
>>
>> ----
>>
>> I believe the 2nd option would be preferred.
>> However, a faster path to support markdown would be with a plugin or a
>> dynamic style.
>>
>>
>> br,
>> dirk
>>
>>
>>
>>
>> On Thu, Nov 9, 2017 at 10:00 PM, Juan Pablo Santos Rodríguez <
>> juanpablo.san...@gmail.com> wrote:
>>
>> > Hi!
>> >
>> > (last e-mail today, promise)
>> >
>> > At work we've built a POC regarding a tool to generate some static
>> sites.
>> > We're automating JBake site's generation from some assets, templates and
>> > markdown content (b/c JBake understands markdown). For content authors
>> we
>> > were looking for a good web-based contents-editor, with versioning
>> > capabilities, visually appealing, integrated with our ldap... Something
>> > like JSPWiki O:-) but supporting markdown.
>> >
>> > Given that current master allows switching parsers/renderers, I've been
>> > going at work with some customizations we needed to use JSPWiki for this
>> > POC, whenever I've could look at the POC. Most of them are specific to
>> our
>> > infrastructure (workflows, page storage), but the Markdown support is
>> > pretty agnostic to us so I asked if I could move it to JSPWiki, and I
>> got
>> > my ok :-)
>> >
>> > In order to simplify things regarding code transfer I'll be putting it
>> on a
>> > personal github repo (my company is already ok with that), AL licensed.
>> I
>> > have to extract it from our current code-base, so it maybe some days
>> until
>> > I put it on a github repo for review prior to bringing in to master, but
>> > this is a heads up.
>> >
>> > Regarding the maturity of the development
>> > * it is POC level, meaning is working and stable, but not feature
>> complete.
>> > It's enough to demonstrate that JSPWiki can parse/render Markdown, but
>> > there are some rough edges
>> > ** JSPWikiMarkupParser does a lot of things (f.ex. generates a hash
>> with a
>> > link for all headings) that I might overlooked so there might be
>> > differences
>> > ** No toolbar support on editors (especially on plain editor). Current
>> > toolbars aren't thought with multiple parsers on mind, and as the POC
>> > end-users are more than capable of writing markdown with their bare
>> hands
>> > :-) I've preferred to focus on working the parser and the renderer than
>> on
>> > this. In order to be feature complete, this should be done, and I
>> haven't
>> > thought on a way of doing it yet, so any idea would be more than welcome
>> > O:-) (probably extending the parsers with a new method to expose the
>> markup
>> > associated which each toolbar button, and present it through a new JSP
>> or
>> > who knows how)
>> > * The parsing / rendering is handled with Flexmark (markdown parser, AL
>> > licensed) and some extensions. It requires Java 7 (we are on Java 6).
>> Time
>> > to think on 2.11 O:-)?
>> >
>> >
>> > br,
>> > juan pablo
>> >
>>
>
>
