[PATCH] ob-tangle.el: Speed up tangling
Hi, The attached patch modifies the ~org-babel-tangle~ function to avoid a quadratic behavior in the number of blocks tangled to a single file. Tangling an org buffer with 200 blocks to 5 different files yields a 25 % speedup. * lisp/ob-tangle.el (org-babel-tangle-collect-blocks): Group collected blocks by tangled file name. (org-babel-tangle): Avoid quadratic behavior in number of blocks. -- Sébastien Miquel >From 939fedb0fa94f044eda6966f55f460aa292e345f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?S=C3=A9bastien=20Miquel?= Date: Sat, 17 Apr 2021 21:48:30 +0200 Subject: [PATCH] ob-tangle.el: Speed up tangling ,* lisp/ob-tangle.el (org-babel-tangle-collect-blocks): Group collected blocks by tangled file name. (org-babel-tangle): Avoid quadratic behavior in number of blocks. --- lisp/ob-tangle.el | 148 ++ 1 file changed, 71 insertions(+), 77 deletions(-) diff --git a/lisp/ob-tangle.el b/lisp/ob-tangle.el index 4c0c3132d..eef300c3d 100644 --- a/lisp/ob-tangle.el +++ b/lisp/ob-tangle.el @@ -225,67 +225,54 @@ matching a regular expression." (or (cdr (assq :tangle (nth 2 (org-babel-get-src-block-info 'light (user-error "Point is not in a source code block" path-collector) - (mapc ;; map over all languages - (lambda (by-lang) - (let* ((lang (car by-lang)) - (specs (cdr by-lang)) - (ext (or (cdr (assoc lang org-babel-tangle-lang-exts)) lang)) - (lang-f (org-src-get-lang-mode lang)) - she-banged) - (mapc - (lambda (spec) - (let ((get-spec (lambda (name) (cdr (assoc name (nth 4 spec)) - (let* ((tangle (funcall get-spec :tangle)) - (she-bang (let ((sheb (funcall get-spec :shebang))) - (when (> (length sheb) 0) sheb))) - (tangle-mode (funcall get-spec :tangle-mode)) - (base-name (cond - ((string= "yes" tangle) - (file-name-sans-extension - (nth 1 spec))) - ((string= "no" tangle) nil) - ((> (length tangle) 0) tangle))) - (file-name (when base-name - ;; decide if we want to add ext to base-name - (if (and ext (string= "yes" tangle)) - (concat base-name "." ext) base-name - (when file-name - ;; Possibly create the parent directories for file. - (let ((m (funcall get-spec :mkdirp)) - (fnd (file-name-directory file-name))) - (and m fnd (not (string= m "no")) - (make-directory fnd 'parents))) - ;; delete any old versions of file - (and (file-exists-p file-name) - (not (member file-name (mapcar #'car path-collector))) - (delete-file file-name)) - ;; drop source-block to file - (with-temp-buffer - (when (fboundp lang-f) (ignore-errors (funcall lang-f))) - (when (and she-bang (not (member file-name she-banged))) - (insert (concat she-bang "\n")) - (setq she-banged (cons file-name she-banged))) - (org-babel-spec-to-string spec) - ;; We avoid append-to-file as it does not work with tramp. - (let ((content (buffer-string))) - (with-temp-buffer - (when (file-exists-p file-name) - (insert-file-contents file-name)) - (goto-char (point-max)) - ;; Handle :padlines unless first line in file - (unless (or (string= "no" (cdr (assq :padline (nth 4 spec - (= (point) (point-min))) - (insert "\n")) - (insert content) - (write-region nil nil file-name - ;; if files contain she-bangs, then make the executable + (mapc ;; map over file-names + (lambda (by-fn) + (when-let ((file-name (car by-fn))) + (let ((lspecs (cdr by-fn)) + (fnd (file-name-directory file-name)) + modes make-dir she-banged lang) + ;; delete any old version of file + (when (file-exists-p file-name) (delete-file file-name)) + ;; drop source-blocks to file + ;; We avoid append-to-file as it does not work with tramp. + (with-temp-buffer + (mapc + (lambda (lspec) + (let* ((block-lang (car lspec)) + (spec (cdr lspec)) + (get-spec (lambda (name) (cdr (assq name (nth 4 spec) + (she-bang (let ((sheb (funcall get-spec :shebang))) + (when (> (length sheb) 0) sheb))) + (tangle-mode (funcall get-spec :tangle-mode))) + (unless (string-equal block-lang lang) + (setq lang block-lang) + (let ((lang-f (org-src-get-lang-mode lang))) + (when (fboundp lang-f) (ignore-errors (funcall lang-f) + ;; if files contain she-bangs, then make them executable (when she-bang (unless tangle-mode (setq tangle-mode #o755))) - ;; update counter - (setq block-counter (+ 1 block-counter)) - (unless (assoc file-name path-collector) - (push (cons file-name tangle-mode) path-collector)) - specs))) + (when tangle-mode + (push tangle-mode modes)) + ;; Possibly create the parent directories for file. + (let ((m (funcall get-spec :mkdirp))) +
Re: Bug: [PATCH] org-manual.org: Fix typo about org-latex-images-centered [9.4.5 (9.4.5-16-g94be20-elpaplus @ /home/jorge/.config/emacs/elpa/org-plus-contrib-20210412/)]
Hllo, "Jorge P. de Morais Neto" writes: > Hi. This trivial patch simply makes the manual agree with the docstring > of org-latex-images-centered. Applied. Thank you. > Have I done anything wrong in the patch-submitting process? This is my > second ever patch to a free software project (not counting wiki edits), > so I hope I got everything right. It was almost perfect! ;) I only added a line in the commit message about the section of the manual being fixed. Regards, -- Nicolas Goaziou
Re: Bug: org mode rendering linked heading wrongly [9.4.4 (9.4.4-4-g99eafe-elpaplus @ /Users/cvl/.config/emacs/elpa/28.0/develop/org-plus-contrib-20210104/)]
Hello, Tianrui Wei writes: > Here is a minimal reproducing example > > ** *heading to be linked to* > > [[**heading to be linked to*]] > > Expected rendered result: I had troubles understanding your report. What does "rendering" means here? Is it about `org-insert-link', fontification, export? > *heading to be linked to > > Actual rendered result > > **heading to be linked to If you're talking about `org-insert-link', this is not a bug. The first star is a special markup indicating you're specifically linking to a headline. Regards, -- Nicolas Goaziou
Re: stability of toc links
Hello, Samuel Wales writes: > when you link to a section using toc, you get a link like > > > https://thekafkapandemic.blogspot.com/2020/02/crimes-against-humanity_3.html#org080f0ab > > will these links break if somebody copies them and pastes them > elsewhere? what if you add a section? These links should be stable in a publishing context (i.e., you use `org-publish' to generate the document), even if you add a section. Regards, -- Nicolas Goaziou
Re: Bug: org mode rendering linked heading wrongly [9.4.4 (9.4.4-4-g99eafe-elpaplus @ /Users/cvl/.config/emacs/elpa/28.0/develop/org-plus-contrib-20210104/)]
Nicolas Goaziou writes: >> [[**heading to be linked to*]] >> >> Expected rendered result: > > I had troubles understanding your report. What does "rendering" means > here? Is it about `org-insert-link', fontification, export? I think the report referred to fontification. When org-hide-emphasis-markers is non-nil, the heading is rendered in bold and the "*" markers are hidden. However, "*" emphasis markers are not hidden in the link - inconsistent with what users could expect from non-nil org-hide-emphasis-markers setting. Best, Ihor
Re: Bug: DOI links are broken in HTML exports
Hello, yarnton--- via "General discussions about Org-mode." writes: > I encountered this bug for the first time a few months ago, when > I first used doi: links. AFAICT Org never implemented export for DOI links. It just provides a way to follow them, as you observed already. I added a rough export for doi links in master. I hope this fixes your issue. Regards, -- Nicolas Goaziou
Re: wip-cite status question and feedback
Hello, András Simonyi writes: > are short-form citations like @doe2018 or [@doe2018] also supported? No, I removed them in last year's iteration, because they are prone to false positives. I didn't want to repeat the footnote syntax mistake, when "[1]" was equivalent to "[fn:1]". I am also leaning towards removing the possibility to include Org syntax (e.g., bold) in prefixes/suffixes. Indeed, this doesn't sound terribly useful (you can move the bold part outside of the citation object), and yet, adds a layer of complexity on top of the citation object. Can we assume (global)prefix/suffix are just plain text? >> Now about the API, which is partly implemented on a local branch. > >> - "opening" action is straightforward. All is needed for the processor >> is to provide a function accepting two arguments: the citation key, >> as a string, and possibly a universal argument, which it may ignore, >> or not. >> >> All this is already implemented locally. > > I can imagine situations in which the location (e.g. page range) of > the citation is also important, e.g., when the > processor opens the cited document, so it'd be nice to pass this > information (when available) as well. In my mind, "opening" leads to the bibliography reference, not to the original document. IIUC, in this situation, the location does not matter much, does it? In any case, Org has no clue about the "location" of the citation. It can provide the suffix associated to the citation key, but it is up to the citation processor to extract page information out of it. If this is desirable, we need to provide the full citation object instead of the key. I don't know if it is worth the trouble. >> First, export happens as pre-process, before export back-ends are >> introduced. IOW, export back-ends are never going to see a citation >> object, which means no support whatsoever is needed on their end. > > This is pretty close to how citeproc-org operates right now. I had a quick glance at citeproc-org for inspiration. I agree this is a sane way to proceed. >> Support export requires two functions. The first function is >> responsible for rendering a bibliography. Its arguments are the list >> of citations in the document, the list of bibliography files, the >> style, if any, and the export back-end. It should return a string. > > Am I correct in assuming that the returned bibliography should be > rendered as a legitimate Org fragment regardless of the back-end? I think it should be rendered as target format, to be on par with citation handling. You may, however, use Org format as an intermediate step, and then, call `org-export-data-with-backend' (from "ox.el") on it. If this is desirable, I need to also provide a fifth `info' argument to the list above. >> The second mandatory function is obviously responsible for rendering >> citations. It is called with a citation object, the desired style, >> if any, and the export back-end, the full list of citations objets >> in the document, and the list of bibliography files. It should also >> return a string. Org provides a helper function to determine the >> footnote containing a citation (and its label, or number) from >> a citation object. > > This might not be an important point, but citeproc-el is rather > citation list oriented in the sense that it processes the list of all > citations and returns a list of their formatted counterparts, so it > might be useful to provide this "bulk" API as well, at least as an > optional addition. I don't think any addition to the suggested API is required. If you cannot split citation processing, you may fill a temporary cache holding all exports on first citation (the full list of citations objects is provided right from the start), then read from that cache on subsequent citations. WDYT? > What is very important and was rather tricky to implement as I recall > is that the list of citations should be in the order they actually > appear in the exported document even for citations in footnotes, > because certain styles format citations differently if another > reference to the same work occurred in a nearby, earlier note. Noted. IIRC, `org-export--footnote-reference-map' deals with a similar issue, i.e., how to apply a function on footnote references _in order_. Situation in nested footnotes can be ambiguous, tho. In the following example, is the expected order @a then @b then @c? --8<---cut here---start->8--- Text[fn:1] [@cite:@c] [fn:1] This[fn:2] is [cite:@b]. [fn:2] [cite:@a] --8<---cut here---end--->8--- > In addition to the precise order of the citations and whether they are > in a footnote or not, the concrete identity of the notes is also > important, because formatting can differ depending on whether two > citations are in the same note or in different ones. I don't understand what you call t
Re: wip-cite status question and feedback
Nicolas Goaziou writes: > In my mind, "opening" leads to the bibliography reference, not to the > original document. IIUC, in this situation, the location does not matter > much, does it? > > In any case, Org has no clue about the "location" of the citation. It > can provide the suffix associated to the citation key, but it is up to > the citation processor to extract page information out of it. If this is > desirable, we need to provide the full citation object instead of the > key. I don't know if it is worth the trouble. >From my experience using org-ref for citations, going to the bibliography reference is rarely useful. Most of time, I want to jump to the document I am citing (or even page in the document). I think that full citation object should be worth passing to the citation backend. Best, Ihor
Re: wip-cite status question and feedback
On Sun, Apr 18, 2021 at 9:31 AM Ihor Radchenko wrote: > > Nicolas Goaziou writes: > > > In my mind, "opening" leads to the bibliography reference, not to the > > original document. IIUC, in this situation, the location does not matter > > much, does it? > > > > In any case, Org has no clue about the "location" of the citation. It > > can provide the suffix associated to the citation key, but it is up to > > the citation processor to extract page information out of it. If this is > > desirable, we need to provide the full citation object instead of the > > key. I don't know if it is worth the trouble. > > From my experience using org-ref for citations, going to the > bibliography reference is rarely useful. Most of time, I want to jump to > the document I am citing (or even page in the document). > > I think that full citation object should be worth passing to the > citation backend. I agree. FWIW, this is how the pandoc API describes this point, which I believe is consistent with Nicolas' meaning of "citation objects"? "a list of Citations (each of which may have multiple CitationItems)." Bruce
Re: Bug: Exporting internal link to special latex block [9.3.7 (9.3.7-14-gb2b587-elpa @ /home/lobo/.emacs.d/elpa/org-20200720/)]
Hello, Eric S Fraga writes: > On Tuesday, 21 Jul 2020 at 15:52, Marco Falconi wrote: >> I am trying to export to html a labeled latex special block (a theorem >> environment, > > [...] > >> the link does not href to "#thm:mv" as expected, but to an auto-generated >> label: > > Confirmed with org from git a few minutes ago. The link using an > auto-generated label would be fine if the label were the one for the > special block but it's not. I couldn't reproduce it. Regards, -- Nicolas Goaziou
Re: wip-cite status question and feedback
Hello, M. ‘quintus’ Gülker writes: > The citation object will provide access to all elements of the new > cite syntax I assume, including things like key, prefix and suffix? Indeed. Also global prefix and suffix. > Several styles I am normally confronted with require crossreferencing > in citation footnotes (example: “Doe (see above Fn. 24), pp. 35-37”). > Formatting this requires access to the place where an @key first > occured in a footnote. The full list of citation objects probably > suffices for that information; on a first thought I would either use > the first citation object from that list with the @key at hand unequal > to the active citation object This would work. If it is a common need, Org could also provide such a helper function. > or use the citation object whose footnote label has the lowest number > and is unequal to the active citation object (if the list is not > guaranteed to be in said order). I would prefer the former approach, > because sometimes I deal with footnotes with numbers like “4a” (a > footnote inserted at a late stage in the authoring process between > footnotes 4 and 5), which defeats the lowest-number approach. Note that export process provides its own footnote numbering, which does not rely on the label used. See `org-export-get-footnote-number'. So you can also use the second method. > For non-footnote-based citations, the “helper function to determine > the footnote containing a citation” should probably return nil. Indeed. If there is no footnote definition containing the citation, it returns nil. Regards, -- Nicolas Goaziou
Re: [Patch] to correctly sort the items with emphasis marks in a list
Hi Maxim, Maxim Nikulin writes: > On 10/04/2021 07:01, Juan Manuel Macías wrote: >> Nicolas Goaziou writes: >>> Could you apply the same fix to the `org-verbatim-re' match above, and >>> provide an appropriate commit message? >> Done! I've attached the corrected patch. Sorry for the flaws in me >> previous patch: I'm a bit of a novice at submitting patches... > > I have not read yet the following documents to realize what pitfalls > could be expected due to significant space added to Spanish and Polish > collation rules in libc. > > http://www.unicode.org/reports/tr10/ > Unicode® Technical Standard #10 > Unicode Collation Algorithm > > http://www.unicode.org/reports/tr35/tr35-collation.html#CLDR_Collation_Algorithm > Unicode Technical Standard #35 > Unicode Locale Data Markup Language (LDML) > Part 5: Collation > > In the meanwhile I have realized that org-sort-remove-invisible > function has rather strange behavior. There are no tests, so I am in > doubt if the following result is expected and intended (unpatched > version) All this research you've done on spaces and collation rules is very interesting and enlightening. Certainly, space is important, as the following list in Spanish is correctly ordered as: - lo raro - lo vano - localidad However, with the 'unpatched' version of org-sort-remove invisible I get this result (which is correct) for this list: - lo bueno - lo /bueno/ - lo vano - lo /vano/ And with the patched version, I get this, which is wrong (!): - lo bueno - lo vano - lo /bueno/ - lo /vano/ It seems that what I was proposing as a patch at the beginning is not, finally, a viable solution for all contexts... The problem is that, if the first space is removed, we get this abnormal result: #+begin_src emacs-lisp (org-sort-remove-invisible "- lo /bueno/") #+end_src #+RESULTS: : - lobueno Best regards, Juan Manuel
Re: plantuml, png and caching
CS Suehs wrote: > Should I expect the setting on the BEGIN_SRC line to work? On v9.1.14, this works for me as expected: #+name: helloworld2-works-fine-on-9-1-14 #+BEGIN_SRC plantuml :cache yes :file helloworld2.png @startuml Hello --> World @enduml #+END_SRC It may be a bug introduced between 9.1.14 and your version. Could you report it with your broken and working examples, using `org-submit-bug-report'? That'll include configuration context we might've missed on this email chain that more qualified folks than I can review. Thanks, Nick
Re: Concerns about community contributor support
Timothy writes: , but Org mode development is in a new phase that *requires* technique and is quicker to identify and filter out noise. Hmmm, what constitutes noise? Good question. I suppose like most words the meaning changes over time. Early on, posts along the lines of "Wouldn't it be cool if Org mode would do this?" were given more space than they seem to be today. Tim Cross points out a project trajectory that appears to be common and would explain why. At a more concrete level, I can offer several of my posts to the list :) These changes mean that contributions need to be checked for contributions to Nicolas' project and also fit into the history of discussion and development. The Org mode project now resembles a scholarly discipline that moves slowly and deliberately toward a more or less well defined goal. The days when Carsten would bang out a new feature during his train ride home from work are gone. I think here there may have been a minor misunderstanding /miscommunication. Reading this paragraph I get the sense you read my email as complaining about a delay in merging patches, however this is a separate ---if related--- point to what I intended to raise: the lack of /response/ to patches. 1. Were I talking about merging: a more considered development model, as you describe above, can certainly see a protracted merge delay. However, 6 months for a minor feature addition [1], and 2 months for a minor bug fix [2] is not justified by a more considered development model IMO. 2. (My main point) Even if development is slower, leaving a first-time contributor with /absolutely no response/, i.e. *zero* replies to their email *months* after they sent it (see [1] and [2] for example, and updates.orgmode.org for more) is not good enough IMO. We should be better. So, something in between merging (or not) and appearing on the public list that Bastien keeps? Once again, with reference to my earlier paragraph, IMO slowed development is one thing, not responding at all to attempted contributors for months on end another. It is the latter which I seek to improve. I can, have, and will try to help with this myself; but I think we would benefit from a "community effort" and a discussion on what the best way to improve this is. What do you think of Tim Cross' suggestion that a way forward is for "new features and enhancements to be initially implemented as add on contribution packages rather than as extensions/enhancement to the core 'org-mode' package"? This sounds good to me, but I'm not much of a programmer and lack the ability to suss out its ramifications fully. I can see how it would ease Org mode maintenance, though. All the best, Tom -- Thomas S. Dye https://tsdye.online/tsdye
Re: [PATCH] ob-tangle.el: Speed up tangling
Hi Sébastien, Some comments while looking over this (will report back when I have tested it out as well). This is a section of the ob export functionality that I have been looking for on and off for quite a while because it is responsible for some bad and insecure behavior. I think that some of your changes may have fixed/improved this as a side effect. I don't know whether it is worth doing anything about the issues in this patch, but since we are here, I think they are worth mentioning. All of the issues that I'm aware of are related to what happens if tangling fails part way through the process. First, your patch already fixes a major issue which is that the modes of all files would not be set if any one of them failed to tangle. Next, during the process the existing file is deleted prior to tangling, which means that it cannot be restored if tangling fails, it would be better if the old file was moved to a temporary location and then deleted on success or replaced on failure. This likely requires wrapping the bits that can fail in unwind-protect and restoring on failure or fully deleting at the end of success. The next issue is that setting the tangle mode should happen before the file is written, an empty file should be created, the mode should then be set, the contents of the file should be written only after the mode has been set. This involves a bit of reordering of operations in lines 124-126 of your patch. This ordering of opertions prevents security issues related to race conditions and potential errors being evoked during write-region (though again, your changes already make the tangling code much more secure by setting the modes on each file immediately after writing instead of how it works currently where if any other block encounters an error then no modes were set). Best! Tom On Sun, Apr 18, 2021 at 12:23 AM Sébastien Miquel wrote: > > Hi, > > The attached patch modifies the ~org-babel-tangle~ function to avoid a > quadratic behavior in the number of blocks tangled to a single file. > > Tangling an org buffer with 200 blocks to 5 different files yields a > 25 % speedup. > > > * lisp/ob-tangle.el (org-babel-tangle-collect-blocks): Group > collected blocks by tangled file name. > (org-babel-tangle): Avoid quadratic behavior in number of blocks. > > -- > Sébastien Miquel
Re: Concerns about community contributor support
Thomas S. Dye writes: >> Hmmm, what constitutes noise? >> > Good question. I suppose like most words the meaning changes over time. > Early > on, posts along the lines of "Wouldn't it be cool if Org mode would do this?" > were given more space than they seem to be today. Tim Cross points out a > project trajectory that appears to be common and would explain why. At a more > concrete level, I can offer several of my posts to the list :) Follow up: What should be the response to "noise", because I don't think it should be a cold shoulder. >> I think here there may have been a minor misunderstanding >> /miscommunication. Reading this paragraph I get the sense you read my >> email as complaining about a delay in merging patches, however this is a >> separate ---if related--- point to what I intended to raise: the >> lack of /response/ to patches. >> >> 1. [snip] >> 2. [snip] > So, something in between merging (or not) and appearing on the public list > that > Bastien keeps? I'll try to clarify my meaning: + Patches are sent in + The /do/ appear on the public list + Months go by without anyone even replying to the patch I think it's fair to expect something along the lines of: "Not sure about this, will require further thought", "Looks good, but want to wait for X to approve", "Not quite sure why this is a good idea, please explain further", or one of a hundred other suitable cursory responses. Instead, there are currently 24 patches listed on https://updates.orgmode.org/#patches which have not received a single response. Then, there are also a large number of other patches with 1-2 cursory replies that seem stuck in limbo (e.g. [1]), with no signs of being merged or rejected (better than 0 replies, but still not good). >> Once again, with reference to my earlier paragraph, IMO slowed >> development is one thing, not responding at all to attempted >> contributors for months on end another. It is the latter which I seek to >> improve. I can, have, and will try to help with this myself; but I think >> we would benefit from a "community effort" and a discussion on what the >> best way to improve this is. >> > > What do you think of Tim Cross' suggestion that a way forward is for "new > features and enhancements to be initially implemented as add on contribution > packages rather than as extensions/enhancement to the core 'org-mode' > package"? > This sounds good to me, but I'm not much of a programmer and lack the ability > to > suss out its ramifications fully. I can see how it would ease Org mode > maintenance, though. I'm going to reply to Tim Cross' email, but a short version of my current thoughts is: + This feels like a little bit of a tangent from the lack of response + Many patches are modifying core functionality, and would not be suitable as an add-on (e.g. [2], [3], [4], and more) + I'm concerned that good changes could quickly make there way here and never make their way into Org > All the best, > Tom -- Timothy [1]: [PATCH] ob-lilypond: allow user configuration of header-args https://orgmode.org/list/canwlylncudeqrtle9dxb+xzg9t-dwfmfhzrpmuqcuzlzw34...@mail.gmail.com/ [2]: [PATCH] Add org-meta*-final-hook https://orgmode.org/list/caoywxzg1cbl07thlzxhbbczm6te2vmtqnmm0w63331gybrj...@mail.gmail.com/ [3]: [PATCH] ob-C.el: Fix a number a bugs related to table parameters https://orgmode.org/list/874kkqao1e@bzg.fr/ [4]: [PATCH] Fontification for inline src blocks https://orgmode.org/list/87pmzf4bd0@gmail.com/
Re: Concerns about community contributor support
Tim Cross writes: >> Nicolas Goaziou [...] > > Totally agree. The work Nicolas has put in to consolidate, stabilise and > clarify the org code base has been critical to the long term viability > of the project. I very much appreciate the work he has contributed and > the many times he has assisted me in understanding how things work > within org-mode. >> Kyle Meyer [...] > > Also agree. Getting the right balance between features, stability and > maintenance is extremely challenging. This is especially so with > org-mode as it has a level of flexibility which makes for a very wide > set of use cases. Identifying what should be part of 'core' and what > would be better off as an extension or add-on is often challenging. What > may seem like a great addition/enhancement for one may be of no benefit > to another or may actually complicate their use case. It can be > challenging to understand and interpret all these different perspectives > and determining the optimal forward direction. Thank you for these points. They are separate to my concerns about the lack of response to patches, but worth noting in the overall context of the development of Org mode. >> These changes mean that contributions need to be checked for contributions to >> Nicolas' project and also fit into the history of discussion and development. >> The Org mode project now resembles a scholarly discipline that moves slowly >> and >> deliberately toward a more or less well defined goal. The days when Carsten >> would bang out a new feature during his train ride home from work are gone. > > This is a common development path for a successful project. Kent Beck > (extreme/agile development) has been focusing a bit on this with his 3x > development model (eXplore, eXpand, eXtract). (see > https://medium.com/@kentbeck_7670/fast-slow-in-3x-explore-expand-extract-6d4c94a7539). > To some extent, we are in an expand/extract stage for org mode. Focus is > on addressing performance and extracting core functionality - new > 'features' need to demonstrate a high level of 'return on investment'. > At this stage, enhancing or extending the functionality is a slower and > more cautious process which requires greater justification than was > common in the 'explore' stage. Interesting link, thanks. Org being in the expand/extract stage certainly rings true to me from my exposure. That said, I don't see why being in this stage of development means we don't need to acknowledge people's attempted contributions. > From my perspective, I see bug fixes applied fairly quickly. > Enhancements and extensions are applied more conservatively, which I > think is the correct approach. I'm not sure if this is a deliberate tangent, or a miscommunication in my original email, but how quickly patches are *applied* is not something I mentioned at all in my original email. My concern is centred around the lack of /response/ to people sending in patches. Radio silence for *six months* after sending in a contribution seems ridiculous to me. > I suspect the best model for moving forward is for new features and > enhancements to be initially implemented as add on contribution packages > rather than as extensions/enhancement to the core 'org-mode' package. > Such packages, if found to be popular or useful to a large number of > org-mode users could then be considered for inclusion as part of the > main org-mode package. The nature of elisp makes this type of model very > suitable as you can modify almost any aspect of org-mode via add on > packages in a consistent and stable manner and avoid impacting the core > functionality until the extension/enhancement has matured sufficiently. This is an interesting thought. Putting aside the fact that this is somewhat tangential to points I wished to discuss, I have a number of reservations: + Many patches are modifying core functionality, and would not be suitable as an add-on (e.g. [1], [2], [3], and more) + Many patches aren't even being acknowledged. Given this, I am highly suspicious that good additions will actually be moved into Org. + This feels like it could become a bit of a graveyard of ideas. + This complicates the development model. -- Timothy [1]: [PATCH] Add org-meta*-final-hook https://orgmode.org/list/caoywxzg1cbl07thlzxhbbczm6te2vmtqnmm0w63331gybrj...@mail.gmail.com/ [2]: [PATCH] ob-C.el: Fix a number a bugs related to table parameters https://orgmode.org/list/874kkqao1e@bzg.fr/ [3]: [PATCH] Fontification for inline src blocks https://orgmode.org/list/87pmzf4bd0@gmail.com/
Re: Concerns about community contributor support
Timothy writes: Thomas S. Dye writes: Hmmm, what constitutes noise? Good question. I suppose like most words the meaning changes over time. Early on, posts along the lines of "Wouldn't it be cool if Org mode would do this?" were given more space than they seem to be today. Tim Cross points out a project trajectory that appears to be common and would explain why. At a more concrete level, I can offer several of my posts to the list :) Follow up: What should be the response to "noise", because I don't think it should be a cold shoulder. Agreed. I'm trying to put myself in the maintainers' shoes. They volunteer lots of highly skilled time, which I admire greatly. I can imagine a situation where a patch falls outside a maintainer's interest and they have difficulty finding the time to understand it and offer a meaningful critique. Historical note: when Carsten took his leave and announced Bastien would maintain Org mode, I jumped in noisily to suggest that a committee approach might be better. I couldn't imagine a world with two Carstens! It appears there is a committee of sorts now, though I'm in the dark how it is organized. Assuming there is a committee, perhaps addition of an Initial Patch Reviewer might be a good idea? All the best, Tom -- Thomas S. Dye https://tsdye.online/tsdye
List of ob-* maintainers
Aloha all, Is there a list of current ob-* maintainers? Perhaps it would be useful to add and populate a Maintainer column to the language table on Worg (org-contrib/babel/languages/index.html)? All the best, Tom -- Thomas S. Dye https://tsdye.online/tsdye
Re: Concerns about community contributor support
Thomas S. Dye writes: >> Follow up: What should be the response to "noise", because I don't think >> it should be a cold shoulder. >> > Agreed. I'm trying to put myself in the maintainers' shoes. They volunteer > lots of highly skilled time, which I admire greatly. I can imagine a > situation > where a patch falls outside a maintainer's interest and they have difficulty > finding the time to understand it and offer a meaningful critique. > > Historical note: when Carsten took his leave and announced Bastien would > maintain Org mode, I jumped in noisily to suggest that a committee approach > might be better. I couldn't imagine a world with two Carstens! It appears > there is a committee of sorts now, though I'm in the dark how it is organized. > Assuming there is a committee, perhaps addition of an Initial Patch Reviewer > might be a good idea? I desperately need to head to bed, so I'll make this quick, but I think the unclear structure doesn't help. Often I'm at a loss as to whether a patch has been left for Bastien to take a look at, overlooked, rejected without comment, or what.* Crucially, in any of those cases I think the contributor deserves to know. I think responding to a first-time contribution with silence is more likely to push someone away than any other effect. Org development may be slowing down/refocusing, but I don't think that means we should disregard new volunteered effort. A clearly layed out structure and set of roles sounds like it could be helpful to me. > All the best, > Tom -- Timothy * I happen to be currently feeling this with the set of patches I recently sent in. One was responded to and merged, another had a burst of replies but seems to be left in limbo/waiting for Bastien?, and then I have 4 others which I am just *hoping* will eventually be noticed / responded to. It's not a good feeling, and it's part of whats prompted me to send the original email.
Re: stability of toc links
i think this is the right type of solution. On 12/8/20, TEC wrote: > > Hi Sam, link stability is a concern I've had too. I currently have a fix > (or at the very least, an improvement) for this in my config where I > overwrite org-export-get-reference. (see: > https://tecosaur.github.io/emacs-config/config.html#nicer-generated-heading). > > I raised this on the list a while ago --- > https://orgmode.org/list/e1jxajq-0004dk...@lists.gnu.org/ but there > didn't seem to be much interest. > > All the best, > Timothy > > Samuel Wales writes: > >> when you link to a section using toc, you get a link like >> >> >> https://thekafkapandemic.blogspot.com/2020/02/crimes-against-humanity_3.html#org080f0ab >> >> will these links break if somebody copies them and pastes them >> elsewhere? what if you add a section? >> >> there doesn't seem to be a perfect solution, short of adding custom id >> or id to everything, but perhaps a fuzzy hash of the header and >> contents of the section could be used? or a strict hash of the >> header? is anything like this being done? just curious. > > > -- The Kafka Pandemic Please learn what misopathy is. https://thekafkapandemic.blogspot.com/2013/10/why-some-diseases-are-wronged.html
Re: [Patch] to correctly sort the items with emphasis marks in a list
Juan Manuel Macías writes: > It seems that what I was proposing as a patch at the beginning is not, > finally, a viable solution for all contexts... > > The problem is that, if the first space is removed, we get this > abnormal result: > > #+begin_src emacs-lisp > (org-sort-remove-invisible "- lo /bueno/") > #+end_src > > #+RESULTS: > : - lobueno I wonder if this other approach can be viable or if it is something crazy: if the spaces in org-sort-remove-invisible are a problem only for the first emphasis of each item, how about this fix to org-sort-list? (not modifying org-sort-remove-invisible): @@ -2940,10 +2940,20 @@ function is being called interactively." (org-sort-remove-invisible (buffer-substring (match-end 0) (point-at-eol) ((= dcst ?a) - (funcall case-func -(org-sort-remove-invisible - (buffer-substring - (match-end 0) (point-at-eol) + (if (save-excursion + (beginning-of-line) + (forward-char) + (looking-at-p org-emph-re)) + (replace-regexp-in-string +"\\(^\\)\s+" "\\1" +(funcall case-func + (org-sort-remove-invisible + (buffer-substring + (match-end 0) (point-at-eol) + (funcall case-func + (org-sort-remove-invisible + (buffer-substring +(match-end 0) (point-at-eol)) ((= dcst ?t) (cond ;; If it is a timer list, convert timer to seconds
Re: Concerns about community contributor support
Timothy writes: > Tim Cross writes: >> This is a common development path for a successful project. Kent Beck >> (extreme/agile development) has been focusing a bit on this with his 3x >> development model (eXplore, eXpand, eXtract). (see >> https://medium.com/@kentbeck_7670/fast-slow-in-3x-explore-expand-extract-6d4c94a7539). >> To some extent, we are in an expand/extract stage for org mode. Focus is >> on addressing performance and extracting core functionality - new >> 'features' need to demonstrate a high level of 'return on investment'. >> At this stage, enhancing or extending the functionality is a slower and >> more cautious process which requires greater justification than was >> common in the 'explore' stage. > > Interesting link, thanks. Org being in the expand/extract stage > certainly rings true to me from my exposure. > > That said, I don't see why being in this stage of development means we > don't need to acknowledge people's attempted contributions. > This is probably just a reflection of how a largely community driven approach works. There isn't really anyone with a responsibility to respond or reply to a patch within some acceptable 'time'. There is a very limited number of people who have commit rights and who do apply patches etc. At this time, priority appears to be given to patches which fix existing behaviour (bug fixes) over ones which extend or modify existing behaviour (enhancements and new features). I suspect the level of response something receives is a reflection of the level of relevance or interest to others on the list. Personally, I don't respond to messages which either - Don't have relevance to how I use org mode. - I don't understand the objective or reason/rationale. - I don't agree or believe the idea is a good one. Sometimes I will reply if I feel (very subjective) the idea would have negative consequences in general or I think there is a simpler alternative solution. However, often I will take a wait and see approach before contributing anything. > My concern is centred around the lack of /response/ to people sending > in patches. Radio silence for *six months* after sending in a > contribution seems ridiculous to me. > I don't think it is ridiculous. I think it is more a reflection of how less formal and structured approaches work. I can understand why someone who has put in the effort to create and submit a patch would appreciate some feedback, but I'm not sure an expectation or desire for feedback is sufficient. Personally, I feel that if you make a patch submission, you sometimes need to 'sell' the patch. This is especially true if the patch is not a basic bug fix. Having developed a patch, it is easy to become very close to it and assume aspects of the patch or what it is proposing are self evident. It can be challenging to put yourself in the shoes of someone encountering the patch for the first time and consider how your submission will come across. Simply submitting the patch and expecting feedback may not be sufficient and followup posts will be required to either clarify, remind or encourage feedback. > This is an interesting thought. Putting aside the fact that this is > somewhat tangential to points I wished to discuss, I have a number of > reservations: > + Many patches are modifying core functionality, and would not be > suitable as an add-on (e.g. [1], [2], [3], and more) There is no limit on what additional external 'add on' packages can do. They can easily modify core functionality. This is one of the core benefits/strengths of lisp languages. I can redefine a core function in my add on package and provided my package is loaded after the core, my redefined version of the function will be what is executed when the core uses that function. For example, I have a heavily 'patched' version of org mode, but I never actually change the core org code. Instead, I have my own org configuration which redefines and modifies core functionality which I wanted changed to suit my requirements. I'm not totally clear about what sort of acknowledgement you want/expect for patch submissions. I get the impression it is more than a simple "thank you for your patch" and rather more detailed review/feedback of your patch. For the latter case, it is likely more a reflection of available resources - in particular, people with the necessary elisp skill to be able to review and provide valid feedback and available time for those limited resources. My observation, which is just one view, is that bug fixes get priority and other patches get attention based on a combination of community interest, available time and the 'squeaky wheel' principal. This relates to my point about needing to 'sell' your patch. For example, I had a quick look at some of the patches you referenced. None of them had any relevance to my use of org mode and none of them involved org functionality I am familiar with (from an internals perspective), which means I am not in a pos
[PATCH] org-table-import: Make it more smarter for interactive use
Hi, My previous patch proposed to add support for importing file with arbitrary name and building upon that this patch tries to make use of it by making org-table-import smarter by simply adding more separators (delimiters). Currently org-table-import 'smartly' guesses only COMMA, TAB and SPACE as separator whereas this patch tries to add support for ';'(SEMICOLON) and ':' (COLON). Here is an example org-table generated using =M-x org-table-import= /etc/passwd (uses COLON as separator) with private information removed. | bin| x | 1 | 1 || / | /usr/bin/nologin | | daemon | x | 2 | 2 || / | /usr/bin/nologin | | mail | x | 8 |12 || /var/spool/mail | /usr/bin/nologin | | ftp| x |14 |11 || /srv/ftp | /usr/bin/nologin | | http | x |33 |33 || /srv/http | /usr/bin/nologin | | nobody | x | 65534 | 65534 | Nobody | / | /usr/bin/nologin | | dbus | x |81 |81 | System Message Bus | / | /usr/bin/nologin | | systemd-journal-remote | x | 981 | 981 | systemd Journal Remote | / | /usr/bin/nologin | | systemd-network| x | 980 | 980 | systemd Network Management | / | /usr/bin/nologin | | systemd-oom| x | 979 | 979 | systemd Userspace OOM Killer | / | /usr/bin/nologin | | systemd-resolve| x | 978 | 978 | systemd Resolver | / | /usr/bin/nologin | | systemd-timesync | x | 977 | 977 | systemd Time Synchronization | / | /usr/bin/nologin | | systemd-coredump | x | 976 | 976 | systemd Core Dumper| / | /usr/bin/nologin | | avahi | x | 974 | 974 | Avahi mDNS/DNS-SD daemon | / | /usr/bin/nologin | | colord | x | 973 | 973 | Color management daemon| /var/lib/colord | /usr/bin/nologin | | rtkit | x | 133 | 133 | RealtimeKit| /proc | /usr/bin/nologin | | transmission | x | 169 | 169 | Transmission BitTorrent Daemon | /var/lib/transmission | /usr/bin/nologin | | geoclue| x | 972 | 972 | Geoinformation service | /var/lib/geoclue | /usr/bin/nologin | | usbmux | x | 140 | 140 | usbmux user| / | /usr/bin/nologin | diff --git a/lisp/org/org-table.el b/lisp/org/org-table.el index ab66859d6a..5ee4af612b 100644 --- a/lisp/org/org-table.el +++ b/lisp/org/org-table.el @@ -846,6 +846,35 @@ org-table-create (goto-char pos)) (org-table-align))) + +(defun org-table-guess-separator (beg0 end0) + "Guess separator for `org-table-convert-region' for region BEG0 to END0. + +List of preferred separator: +comma, TAB, ';', ':' or SPACE + +If region contains a line which doesn't contain the required +separator then discard the separator and search again using next +separator." + (let ((beg (save-excursion + (goto-char (min beg0 end0)) + (beginning-of-line 1) + (point))) + (end (save-excursion + (goto-char (max beg0 end0)) + (end-of-line 1) + (if (bolp) (backward-char 1) (end-of-line 1)) + (point +(save-excursion + (goto-char beg) + (cond + ((not (re-search-forward "^[^\n,]+$" end t)) '(4)) + ((not (re-search-forward "^[^\n\t]+$" end t)) '(16)) + ((not (re-search-forward "^[^\n;]+$" end t)) ";") + ((not (re-search-forward "^[^\n:]+$" end t)) ":") + ((not (re-search-forward "^\\([^'\"][^\n\s][^'\"]\\)+$" end t)) " ") + (t nil) + ;;;###autoload (defun org-table-convert-region (beg0 end0 &optional separator) "Convert region to a table. @@ -862,10 +891,7 @@ org-table-convert-region integer When a number, use that many spaces, or a TAB, as field separator regexp When a regular expression, use it to match the separator nil When nil, the command tries to be smart and figure out the - separator in the following way: - - when each line contains a TAB, assume TAB-separated material - - when each line contains a comma, assume CSV material - - else, assume one or more SPACE characters as separator." + separator using `org-table-guess-seperator'." (interactive "r\nP") (let* ((beg (min beg0 end0)) (end (max beg0 end0)) @@ -881,14 +907,9 @@ org-
[PATCH] manual: Fix publish stylesheet
* doc/org-manual.org (Publishing sample configuration):
* doc/org-guide.org (Publishing):
Fix stylesheet setting in publish: :style is not a valid option; use
:html-head instead.
---
A small fix for the manual which I believe is necessary (:style does not appear
to be a valid option any more). My copyright assignment paperwork is on file.
Thank you!
doc/org-guide.org | 6 +++---
doc/org-manual.org | 6 +++---
2 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/doc/org-guide.org b/doc/org-guide.org
index ea3040bba..3715982f6 100644
--- a/doc/org-guide.org
+++ b/doc/org-guide.org
@@ -2337,9 +2337,9 @@ example:
:publishing-directory "~/public_html"
:section-numbers nil
:table-of-contents nil
- :style "")))
+ :html-head "")))
#+end_src
- {{{kbd(C-c C-e P x)}}} ::
diff --git a/doc/org-manual.org b/doc/org-manual.org
index 914649f6e..593c848cf 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -16325,9 +16325,9 @@ directory on the local machine.
:publishing-directory "~/public_html"
:section-numbers nil
:table-of-contents nil
- :style "")))
+ :html-head "")))
#+end_src
*** Example: complex publishing configuration
--
2.30.2
Re: [PATCH] manual: Fix publish stylesheet
Oops, noticed a few additional problems. See attached patch.
>From 14de709f224a45cb1beee5b1c1444b1454161891 Mon Sep 17 00:00:00 2001
From: Erik Hetzner
Date: Sun, 18 Apr 2021 21:47:04 -0700
Subject: [PATCH] manual: Fix publish options
* doc/org-manual.org (Publishing sample configuration):
* doc/org-guide.org (Publishing):
Fix stylesheet setting in publish: :style is not a valid option; use
:html-head instead. :table-of-contents is not a valid option; use
:with-toc instead. :publishing-function is required.
---
doc/org-guide.org | 9 +
doc/org-manual.org | 8
2 files changed, 9 insertions(+), 8 deletions(-)
diff --git a/doc/org-guide.org b/doc/org-guide.org
index ea3040bba..3daf8d3c7 100644
--- a/doc/org-guide.org
+++ b/doc/org-guide.org
@@ -2334,12 +2334,13 @@ example:
(setq org-publish-project-alist
'(("org"
:base-directory "~/org/"
+ :publishing-function org-html-publish-to-html
:publishing-directory "~/public_html"
:section-numbers nil
- :table-of-contents nil
- :style "")))
+ :with-toc nil
+ :html-head "")))
#+end_src
- {{{kbd(C-c C-e P x)}}} ::
diff --git a/doc/org-manual.org b/doc/org-manual.org
index 914649f6e..b4eff22ad 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -16324,10 +16324,10 @@ directory on the local machine.
:publishing-function org-html-publish-to-html
:publishing-directory "~/public_html"
:section-numbers nil
- :table-of-contents nil
- :style "")))
+ :with-toc nil
+ :html-head "")))
#+end_src
*** Example: complex publishing configuration
--
2.30.2
On Sun, 18 Apr 2021 21:47:04 -0700,
Erik Hetzner wrote:
>
> * doc/org-manual.org (Publishing sample configuration):
> * doc/org-guide.org (Publishing):
> Fix stylesheet setting in publish: :style is not a valid option; use
> :html-head instead.
> ---
> A small fix for the manual which I believe is necessary (:style does not
> appear to be a valid option any more). My copyright assignment paperwork is
> on file. Thank you!
>
> doc/org-guide.org | 6 +++---
> doc/org-manual.org | 6 +++---
> 2 files changed, 6 insertions(+), 6 deletions(-)
>
> diff --git a/doc/org-guide.org b/doc/org-guide.org
> index ea3040bba..3715982f6 100644
> --- a/doc/org-guide.org
> +++ b/doc/org-guide.org
> @@ -2337,9 +2337,9 @@ example:
> :publishing-directory "~/public_html"
> :section-numbers nil
> :table-of-contents nil
> - :style " -href=\"../other/mystyle.css\"
> -type=\"text/css\"/>")))
> + :html-head " +href=\"../other/mystyle.css\"
> +type=\"text/css\"/>")))
> #+end_src
>
> - {{{kbd(C-c C-e P x)}}} ::
> diff --git a/doc/org-manual.org b/doc/org-manual.org
> index 914649f6e..593c848cf 100644
> --- a/doc/org-manual.org
> +++ b/doc/org-manual.org
> @@ -16325,9 +16325,9 @@ directory on the local machine.
> :publishing-directory "~/public_html"
> :section-numbers nil
> :table-of-contents nil
> - :style " -href=\"../other/mystyle.css\"
> -type=\"text/css\"/>")))
> + :html-head " +href=\"../other/mystyle.css\"
> +type=\"text/css\"/>")))
> #+end_src
>
> *** Example: complex publishing configuration
> --
> 2.30.2
>
>
>
