[FR] org-babel-n-tangle

[Phil]

Hi,

The ability to tangle to multiple destinations is a very convenient way
to manage cluster configurations. No, it's more than that: it's an
*awesome* way to deploy and keep clusters configs and repros well
organized.

The following *org-babel-n-tangle*, is just a small derivation
of *org-babel-tangle*.  It's displayed here as a diff not
with the intent to be applied as a patch, but to show the very
little differences required in order to get this working.

#+begin_src diff
diff -u ~/.emacs.d/repos/org/lisp/ob-tangle.el ~/tmp/ob-ntangle.el
--- ~/.emacs.d/repos/org/lisp/ob-tangle.el
+++ ~/tmp/ob-ntangle.el
@@ -238,8 +238,12 @@
(org-babel-tangle-file filename)))

;;;###autoload
-(defun org-babel-tangle (&optional arg target-file lang-re)
-  "Write code blocks to source-specific files.
+(defun org-babel-n-tangle (&optional arg target-file lang-re)
+  "Write code blocks to source-specific files
+located into the directories designated by the :n-tangle parameter
+then onto subsequent directory and file of the :tangle parameter.
+Performs like `org-babel-tangle' adding an extra iteration over
+a list of directories, potentially different hosts and protocols
Extract the bodies of all source code blocks from the current
file into their own source-specific files.  Return the list of files.
With one universal prefix argument, only tangle the block at point.
@@ -268,12 +272,19 @@
(tangle-file
(when (equal arg '(16))
(or (cdr (assq :tangle (nth 2 (org-babel-get-src-block-info 'no-eval
-  (user-error "Point is not in a source code block"
+  (user-error "Point is not in a source code block"
+ (targets (or (cadr (assoc (cdr
+ (assoc :n-tangle (nth 2 (org-babel-get-src-block-info
+   org-babel-ntangle-destinations))
+  '(nil))) ; iterate on one local target
path-collector
(source-file buffer-file-name))
+
+ (dolist (target targets) ;; iterate the n-tangle group
+  (progn
(mapc ;; map over file-names
(lambda (by-fn)
-(let ((file-name (car by-fn)))
+(let ((file-name (concat target (car by-fn
(when file-name
(let ((lspecs (cdr by-fn))
(fnd (file-name-directory file-name))
@@ -354,6 +365,7 @@
(if (equal arg '(4))
(org-babel-tangle-single-block 1 t)
(org-babel-tangle-collect-blocks lang-re tangle-file)))
+ ))
(message "Tangled %d code block%s from %s" block-counter
(if (= block-counter 1) "" "s")
(file-name-nondirectory

#+end_src


In order to use this *:n-tangle* parameter, the destinations are
declared in groups of host and/or root folders.
   #+begin_src elisp
 (setq org-babel-ntangle-destinations
   '(("test-1"
  ("/tmp/test/host-A"
   "/tmp/test/host-B" ))
 ("hosts-A&B/tmp"
  ("/-:hostA:/tmp/"
   "/-:hostB:/tmp/"

   #+end_src


Calling *org-babel-n-tangle* with the universal argument
runs the tangle processor, not on the entire file, but
for the current block. The tangled output goes into the
designated group.

#+begin_example
#+begin_src elisp :n-tangle "hosts-A&B/tmp" :tangle /x/y :mkdirp t
  (org-babel-n-tangle '(4))
#+end_src
#+end_example

In the above example the tangled outputs goes to
*hostA:/tmp/x/y* and *hostB:/tmp/x/y* using a default protocol.

In the absence of *:n-tangle* or when
*org-babel-ntangle-destinations* is nil.
*org-babel-n-tangle* behaves like *org-babel-tangle*

What do you think ?

Phil

/"Oh what a tangled web we weave..."/

Re: Org babel R command session

[Ihor Radchenko]

Ihor Radchenko  writes:

> Or we can just document that ESS defaults are used for sessions and ask
> users to customize inferior-ess-R-program if they need something
> non-standard for sessions.
>
> The problem with `inferior-ess-R-program' is that it must be executable
> name, and cannot be command with arguments, as in our default value of
> org-babel-R-command: "R --slave --no-save". In the absense of people who
> are actually making use of the proposed feature, I see no reasons to
> complicate the code.

Fixed, on the ob-R documentation page in WORG.
https://git.sr.ht/~bzg/worg/commit/c6380ced

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [PATCH] oc-csl: New custom option `org-cite-csl-sentence-case-bibtex-titles'

[Max Nikulin]

On 08/07/2024 19:56, András Simonyi wrote:

What I've been arguing for in this thread and what motivates the
current default behavior is that it is a general, style-independent
CSL assumption that the processor's input for title fields is in
sentence-case, and this requires converting the BibLaTeX title fields
in title-case to sentence-case before feeding into the CSL processor
independently of the style  -- please refer to my email dated 14 May
for further details and the references therein.


András, in 
https://list.orgmode.org/d8b1b73f-6b4e-4946-8873-748f78e19...@gmail.com 
I tried to ask if it is possible to determine if a specific CSL style 
uses title or sentence case. I believe that without "independently of 
the style" requirement behavior may be more convenient for users.


Consider a user who have .bib files following BibTeX recommendations: 
title case with hints where capital letters should be preserved during 
conversion to sentence case. Their is preparing a couple of papers. One 
journal requires StyleT with title case, another one StyleS with 
sentence case. If citeproc may determine what kind of capitalization is 
used by each style then it performs transformation to sentence case for 
StyleS, but passes strings as is for StyleT. Conversion to title case 
for StyleT is applied, but has no effect.


I can not figure out what user case is not covered and requires explicit 
setting.


Feel free to ignore this message.

Re: in 28, massive slowness in capture, folding, and org-show-context

[Ihor Radchenko]

Samuel Wales  writes:

> 18018  99% - command-execute
> ...
>17488  96%  - org-capture
>17434  96%   - org-capture-place-template
>17398  96%- org-fold-show-all
>17370  96% - org-fold-region
>11415  63%  - org-fold-core-get-regions
>11371  62%   - org-fold-core-next-folding-state-change
> *** FACT colophon
> i am running org 9.7.6.  i recently upgraded to emacs 28
> from 27.  simultaneously bookworm from bullseye.  was
> running 9.7.6 without major issues before bookworm.
>
> emacs -q with default org without my .org files is
> relatively ok.  no changes to my .emacs that seem relevant,
> none to capture.  i did disable some capture hooks i had in
> hopes of fixing hte issue, but they did not.
>
> when i am running a normal large rsync that i always ran
> under bullseye, all keystrokes in emacs can take seconds.  i
> believe vlc can be slower too.

Looks like you have too many text properties in buffers, which makes
redisplay slow.

Try setting `org-fold-core--optimise-for-huge-buffers' to '(grab-invisible)
If that does not help, try settqing `org-fold-core-style' to 'overlays.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [FR] org-babel-n-tangle

[Ihor Radchenko]

Phil  writes:

> The ability to tangle to multiple destinations is a very convenient way
> to manage cluster configurations. No, it's more than that: it's an
> *awesome* way to deploy and keep clusters configs and repros well
> organized.
> ...
> #+begin_example
> #+begin_src elisp :n-tangle "hosts-A&B/tmp" :tangle /x/y :mkdirp t
>(org-babel-n-tangle '(4))
> #+end_src
> #+end_example
>
> In the above example the tangled outputs goes to
> *hostA:/tmp/x/y* and *hostB:/tmp/x/y* using a default protocol.
>
> In the absence of *:n-tangle* or when
> *org-babel-ntangle-destinations* is nil.
> *org-babel-n-tangle* behaves like *org-babel-tangle*
>
> What do you think ?

This sounds like a logic extension of the existing tangle mechanism.
Although, I feel that the semantics is a bit cumbersome.

IMHO, a more natural approach would be (1) Introduce :tangle-directory
parameter that defines relative directory to be used as tangle target;
this directory, if defined, will be used instead of the Org file
directory to expand the tangle target; (2) Allow :tangle-directory and
:tangle-file to be a list of targets to write.

Then, we can modify `org-babel-effective-tangled-filename' to account
for :tangle directory and modify `org-babel-tangle' (as you did) to
write to multiple targets.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Capture from Firefox to Org-mode

[Ihor Radchenko]

Sébastien Gendre  writes:

> Ihor Radchenko  writes:
>> I wrote https://github.com/yantar92/org-capture-ref to do exactly
>> this. 
>
> Is it meant to be used with org-ref ?

No. It has nothing to do with org-ref.

> I didn't plan to build a bibliography with bibtex (at least for now). I
> just plan to built a simple book reading tracker with Org-mode. But I
> keep a note about your suggestion, it could be very helpful in the
> future.

org-capture-ref does not create bibtex. It just stores metadata and
chooses a format of that metadata in Org headings that is compatible
with ol-bibtex library.

I use org-capture-ref as reading tracker myself.

Examples of books/articles captured by org-capture-ref:

 DONE Cal Newport [Goodreads] Digital Minimalism: Choosing a Focused Life 
in a Noisy World :book:ATTACH:
CLOSED: [2021-02-27 Sat 22:33]
:PROPERTIES:
:ID: goodreads_digit_minim_choos_focus_life
:CREATED: [2020-07-28 Tue 10:18]
:Effort:   0:20
:HOWPUBLISHED: Goodreads
:URL:  https://www.goodreads.com/book/show/40672036-digital-minimalism
:NOTE: Online; accessed 12 July 2024
:AUTHOR:   Cal Newport
:BTYPE:book
:TITLE:Digital Minimalism: Choosing a Focused Life in a Noisy World
:END:

* DONE Blair [JAMA] (1989) Physical Fitness and All-Cause Mortality 
:article:ATTACH:
SCHEDULED: <2021-03-14 Sun>
:PROPERTIES:
:TITLE:Physical Fitness and All-Cause Mortality
:BTYPE:article
:ID:   3c7ab86d5821f6c622d20c6b83605ef5e4e02691
:AUTHOR:   Steven N. Blair
:JOURNAL:  JAMA
:VOLUME:   262
:NUMBER:   17
:PAGES:2395
:YEAR: 1989
:DOI:  10.1001/jama.1989.03430170057028
:URL:  https://doi.org/10.1001/jama.1989.03430170057028
:CREATED:  [2021-03-13 Sat 20:50]
:HOWPUBLISHED: Jamanetwork
:NOTE: Online; accessed 13 March 2021
:END:



-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [Bug] org-update-dblock randomly broken for more than 10 columns and larger files

[Ihor Radchenko]

Jakob Schöttl  writes:

>> Thanks for the reproducer!
>> I committed a fix onto main.
>> https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=58c5c5882
>
> Hi Ihor, do you know anything on the merge progress of your fix into emacs?
>
> I just tested with GNU Emacs 29.4 with --no-init-file and the minimal 
> example still doesn't produce a correct table.

May you please elaborate what exactly is not correct in the table?
I am not seeing any problem - no extra empty lines are added by the
second and subsequent calls.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: [Bug] org-update-dblock randomly broken for more than 10 columns and larger files

[Ihor Radchenko]

Ihor Radchenko  writes:

>> I just tested with GNU Emacs 29.4 with --no-init-file and the minimal 
>> example still doesn't produce a correct table.
>
> May you please elaborate what exactly is not correct in the table?
> I am not seeing any problem - no extra empty lines are added by the
> second and subsequent calls.

Never mind my question.
Emacs 29.4 does not include Org 9.7, where the bug has been fixed.
You need either upcoming Emacs 30 or upgrade Org mode from ELPA.

-- 
Ihor Radchenko // yantar92,
Org mode contributor,
Learn more about Org mode at .
Support Org development at ,
or support my work at 

Re: Capture, template expansion and keyword for link type

[Max Nikulin]

On 12/07/2024 03:36, Sébastien Gendre wrote:

Max Nikulin writes:

See the "Link type, Available keywords" table in this section.
`org-capture' may be executed in various Emacs buffers and available
keywords may depend on major mode.


Is it necessary to call "org-store-link" before calling "org-capture" to
extracet the informations used by theses keywords ?

[...]

I found the documentation unclear about this.


Have a look into `org-capture' code. It calls `org-store-link'.


Is there a way to discover the keywords that a major mode provide to a
capture template ?


See specific store link functions, e.g. `org-gnus-store-link'.


%(let ((bla (plist-get (plist-get org-store-link-plist :query) :bla)))
(or bla "No bla"))"

emacsclient
'org-protocol:/capture?template=s&title=Hello&body=World&url=http:%2F%2Fexample.com&bla=foo'


A new parameter added to the org-protocol url is only accessible through
the ":query" key ? It will not create a new "%:keywoard" automatically ?


With "capture" subprotocol it does not mapped to keywords.


What I would love to do, is creating a Firefox extension that can
extract different metadata from known well known web site and pass them
to an Org-mode capture through org-protocol.

For example, for a book, extracting author, title, year, etc.


Not all books have authors in metadata. Some have editors since every 
chapter has its own author. To avoid handling such logic in capture 
templates I decided to resort to formatting inside an extension (or 
formatting performed by a dedicated tool if site has API, e.g. GitHub).



And then, I could simply create a capture template that would use the
keywords "%:author", "%:title", etc.


If you are using Emacs-30 I would suggest to keep away from org-protocol 
hacks and to rely on `server-eval-args-left' handled by a dedicated function


emacsclient --eval '(my-capture)' 'my-proto://something?param=value'

For older versions you may either use backslash hell like in 
emacsclient-mail.desktop or to define custom subprotocol for org-protocol.

Re: [Bug] org-update-dblock randomly broken for more than 10 columns and larger files

[Jakob Schöttl]

Am 12.07.24 um 13:56 schrieb Ihor Radchenko:

Emacs 29.4 does not include Org 9.7, where the bug has been fixed.
You need either upcoming Emacs 30 or upgrade Org mode from ELPA.

Thanks!

Re: Capture, template expansion and keyword for link type

[Sébastien Gendre]

Thank you for your reply.
I wrote more after the quotes below.

Max Nikulin  writes:
> Have a look into `org-capture' code. It calls `org-store-link'.

Thank you for this information.


Max Nikulin  writes:
>> Is there a way to discover the keywords that a major mode provide to a
>> capture template ?
>
> See specific store link functions, e.g. `org-gnus-store-link'.

Reading the `org-gnus-store-link' function, I see that is set: type,
file, node, link and description.

I have seen that the variable `org-link-parameters' list all the defined
org links. With its different functions, including the store function
(associated with :store parameter).

I wonder if I can write an Elisp function that retrieve all the
properties set for each org link types and tell we at which mode they
are used. After all, with Elisp code could be data too.

But not now.

Anyway, thank you very much. I will explore starting from
`org-link-parameters', as it seems to have a lot of org links. 



Max Nikulin  writes:
>> A new parameter added to the org-protocol url is only accessible
>> through
>> the ":query" key ? It will not create a new "%:keywoard" automatically ?
>
> With "capture" subprotocol it does not mapped to keywords.

This morning, I have explored how org-capture work. Especially the
capture subprotocol. I think I will need to make another subprotocol.
Maybe a "capture-plus" that will accept any keys and make all of them
accessible as a '%:keymord'.



Max Nikulin  writes:
> Not all books have authors in metadata. Some have editors since every
> chapter has its own author. To avoid handling such logic in capture
> templates I decided to resort to formatting inside an extension (or
> formatting performed by a dedicated tool if site has API, e.g.
> GitHub).

Can you tell me more about this ?



Max Nikulin  writes:
>> And then, I could simply create a capture template that would use the
>> keywords "%:author", "%:title", etc.
>
> If you are using Emacs-30 I would suggest to keep away from
> org-protocol hacks and to rely on `server-eval-args-left' handled by a
> dedicated function
>
> emacsclient --eval '(my-capture)' 'my-proto://something?param=value'
>
> For older versions you may either use backslash hell like in
> emacsclient-mail.desktop or to define custom subprotocol for
> org-protocol.

Is there any change planned for org-protocole in Emacs 30 ?

The big advantage for org-protocole, is that it's based on an URL that
any external software can recognize and pass to Emacs and it's easy to
write a custom sub-protocole.


signature.asc
Description: PGP signature

Re: Capture from Firefox to Org-mode

[Sébastien Gendre]

Thank you for your clarification. I will take a look.

Ihor Radchenko  writes:

> Sébastien Gendre  writes:
>
>> Ihor Radchenko  writes:
>>> I wrote https://github.com/yantar92/org-capture-ref to do exactly
>>> this. 
>>
>> Is it meant to be used with org-ref ?
>
> No. It has nothing to do with org-ref.
>
>> I didn't plan to build a bibliography with bibtex (at least for now). I
>> just plan to built a simple book reading tracker with Org-mode. But I
>> keep a note about your suggestion, it could be very helpful in the
>> future.
>
> org-capture-ref does not create bibtex. It just stores metadata and
> chooses a format of that metadata in Org headings that is compatible
> with ol-bibtex library.
>
> I use org-capture-ref as reading tracker myself.
>
> Examples of books/articles captured by org-capture-ref:
>
>  DONE Cal Newport [Goodreads] Digital Minimalism: Choosing a Focused Life 
> in a Noisy World :book:ATTACH:
> CLOSED: [2021-02-27 Sat 22:33]
> :PROPERTIES:
> :ID: goodreads_digit_minim_choos_focus_life
> :CREATED: [2020-07-28 Tue 10:18]
> :Effort:   0:20
> :HOWPUBLISHED: Goodreads
> :URL:  https://www.goodreads.com/book/show/40672036-digital-minimalism
> :NOTE: Online; accessed 12 July 2024
> :AUTHOR:   Cal Newport
> :BTYPE:book
> :TITLE:Digital Minimalism: Choosing a Focused Life in a Noisy World
> :END:
>
> * DONE Blair [JAMA] (1989) Physical Fitness and All-Cause Mortality 
> :article:ATTACH:
> SCHEDULED: <2021-03-14 Sun>
> :PROPERTIES:
> :TITLE:Physical Fitness and All-Cause Mortality
> :BTYPE:article
> :ID:   3c7ab86d5821f6c622d20c6b83605ef5e4e02691
> :AUTHOR:   Steven N. Blair
> :JOURNAL:  JAMA
> :VOLUME:   262
> :NUMBER:   17
> :PAGES:2395
> :YEAR: 1989
> :DOI:  10.1001/jama.1989.03430170057028
> :URL:  https://doi.org/10.1001/jama.1989.03430170057028
> :CREATED:  [2021-03-13 Sat 20:50]
> :HOWPUBLISHED: Jamanetwork
> :NOTE: Online; accessed 13 March 2021
> :END:


signature.asc
Description: PGP signature

Re: Q: Attachments and IDs?

[David Masterson]

Daniel Fleischer  writes:

> You can override the ID generation and the ID-to-path function to create
> an attachment hierarchy of your liking, e.g.
>
> ATTACH-DIR/year/month-day/*files*
>
> Checkout `org-attach-id-to-path-function-list' and `org-id-new'.

Thank you.  That gives me some ideas.

-- 
David Masterson

Re: Q: Attachments and IDs?

[David Masterson]

Ihor Radchenko  writes:

> David Masterson  writes:
>
>> It's taking awhile to get through my hard head, but I think I'm
>> beginning to understand the directory structure of attachments.
>>
>>   org-attach-id-dir/IDx/attachment-file
>>
>> I don't understand the value of IDx. I would think that "attachment:"
>> could simply reference 'org-attach-id-dir' (drop 'id'?) unless
>> overridden by a DIR property.  I think I'm missing something that would
>> help my understanding of attachments.  Am I?
>
>> Is there a section of the Org manual that talks about IDs?
>
> 10.2.2 Attachment options
>
> ‘org-attach-id-to-path-function-list’
>  When ‘ID’ is used for attachments, the ID is parsed into a part of
>  a directory-path.  See ‘org-attach-id-uuid-folder-format’ for the
>  default function.  Define a new one and add it as first element in
>  ‘org-attach-id-to-path-function-list’ if you want the folder
>  structure in any other way.  All functions in this list will be
>  tried when resolving existing ID's into paths, to maintain backward
>  compatibility with existing folders in your system.
>
> For IDs, we use `org-attach-id-uuid-folder-format' by default - it puts
> attachments under
>
> ///attached-file

Hmm. I'm still misunderstanding the *value* of an ID (value in the sense
of why is it useful for me).  This suggests to me that, under
org-attach-id-dir, you could have a tree of attachments, but what is an
example of why that's a good thing (particularly for a newbie)?  Since
the documentation on attachments starts out mentioning an ID, it's
helpful to understand why it's useful.  Basically, I guess the above
mentions "folder structure" without defining what is trying to be
achieved with this more complicated folder structure instead of just one
attachment directory where all attachments are dumped.

-- 
David Masterson