Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Eli Zaretskii]

> Date: Tue, 19 Apr 2022 15:22:29 -0700
> Cc: maniku...@gmail.com, emacs-orgmode@gnu.org, 54...@debbugs.gnu.org
> From: Paul Eggert 
> 
> On 4/18/22 22:50, Eli Zaretskii wrote:
> >> * admin/merge-gnulib (GNULIB_MODULES): Add gettime-res.
> >> * lib/gettime-res.c: New file, copied from Gnulib.
> >> * lib/gnulib.mk.in, m4/gnulib-comp.m4: Regenerate.
> > Is this known to support MS-Windows correctly?
> 
> I haven't tested it on that platform, though I expect it to work since 
> it relies only on current_timespec and Emacs already uses that.
> 
> I just now added some test cases to Gnulib for it; see the patch in the 
> first attachment. You can try these tests in your environment by running 
> './gnulib-tool --test gettime-res' in the Gnulib source directory. Or 
> you can save time by running './configure; make check' in the directory 
> represented by the second attachment, which is a compressed tarball 
> containing the output of './gnulib-tool --create-testdir --dir 
> test-gettime-res gettime-res'.

Thanks, the test-gettime-res test says "gettime_res returned 625000
ns", which is a strange number: it doesn't fit any MS-Windows system
time resolution figure I know about.  Do you happen to know what does
this number represent, and why it is the result of gettime-res.c when
it runs on MS-Windows?

AFAIK, the basic resolution of MS-Windows time stamps is 100 ns, so
using the above much larger number seems to hint at some significant
loss of information.  If the goal of this future changeset is to make
Emacs time stamps more fine-grained, it would be a shame not to have
the 100-ns resolution on MS-Windows.

Re: Release 9.5.3

[Sébastien Gendre]

Nice, thanks everyone for all the work. 🥰

Where can we found the release notes of this version ?

Le 20 avril 2022 08:28:15 GMT+02:00, Bastien  a écrit :
>Hi all,
>
>Org 9.5.3, a bug fix release, is out.  Enjoy!
>
>Since Org 9.5.4 is expected to be merged with Emacs 28.2, only
>critical bug fixes should go to the bugfix branch now.
>
>Thanks,
>
>-- 
> Bastien

Re: Release 9.5.3

[Bastien Guerry]

Hi Sébastien,

Sébastien Gendre  writes:

> Where can we found the release notes of this version ?

Bugfix releases don't have release notes, only major releases.

See https://orgmode.org/worg/org-maintenance.html for details 
on what to expect on releases.

-- 
 Bastien

[BUG] (org-agenda-files) after restrict agenda to a file [9.5.2 (9.5.2-gfbff08 @ /home/ignacio/.emacs.d/elpa/org-9.5.2/)]

[Ignacio Casso]

Hello,

After restricting the agenda to a file for a single agenda command, with
'C-c a < a', and exiting the agenda with 'x', (org-agenda-files) still
returns that file instead of the original list of agenda files. This can
be easily solved using the UNRESTRICTED optional argument in
`org-agenda-files', but I wonder if this behavior is correct.

The docstring of `org-agenda-files' says

  "Optional UNRESTRICTED means return the full list even if a
  restriction is currently in place."

But as I understand it, the restriction is no longer in place, since it
was only for the duration of the agenda command, as the Org Mode Manual
says:

  "If you would like to focus the agenda temporarily on a file not in
  this list, or on just one file in the list, or even on only a subtree
  in a file, then this can be done in different ways.  For a single
  agenda command, you may press ‘<’ once or several times in the
  dispatcher. To restrict the agenda scope for an extended period ..."
  *note

This is probably a minor issue, but since it was the cause of a bug in
my config that took me a while to debug, I'm reporting it in case you
think that something should be done about it. In case you are curious
about my use case, I was just doing something like

  (eval-after-load 'a-package
'(setq org-agenda-files (append (org-agenda-files) (...)))


Regards,

Ignacio

Emacs  : GNU Emacs 27.2 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.20)
 of 2022-01-16
Package: Org mode version 9.5.2 (9.5.2-gfbff08 @ 
/home/ignacio/.emacs.d/elpa/org-9.5.2/)

Re: [BUG] (org-agenda-files) after restrict agenda to a file [9.5.2 (9.5.2-gfbff08 @ /home/ignacio/.emacs.d/elpa/org-9.5.2/)]

[Ihor Radchenko]

Ignacio Casso  writes:

> After restricting the agenda to a file for a single agenda command, with
> 'C-c a < a', and exiting the agenda with 'x', (org-agenda-files) still
> returns that file instead of the original list of agenda files. This can
> be easily solved using the UNRESTRICTED optional argument in
> `org-agenda-files', but I wonder if this behavior is correct.

Thanks for reporting!

Confirmed.

> This is probably a minor issue, but since it was the cause of a bug in
> my config that took me a while to debug, I'm reporting it in case you
> think that something should be done about it. In case you are curious
> about my use case, I was just doing something like

Honestly, something should be done about org-agenda.el in its totality.
It is notoriously difficult to debug :(

Best,
Ihor

Re: [BUG] Off-by-one error in width of Agenda window?

[N. Jackson]

At 12:01 +0800 on Wednesday 2022-04-20, Ihor Radchenko wrote:
>
> "N. Jackson"  writes:
>
>> It seems that Org maybe should be using window-max-chars-per-line
>> rather than window-width.
>>
>> [Unfortunately w-m-c-p-l is not a drop in replacement for
>> window-width; it doesn't just fix the continuation glyph column
>> "bug", but it also adds additional functionality and has a different
>> signature.]
>
> Could you elaborate why window-max-chars-per-line cannot be used
> instead of window-width by Org? It appears to be strictly more
> accurate.

No, I'm not saying Org shouldn't use window-max-chars-per-line --
rather the opposite in fact. I'm just saying it maybe needs to be
thought about carefully first.

If it works as advertised it would seem to be the right choice as
long as Emacs bug #19395 exists (which looks like it will be
forever). My reservations about the function are only that its spec
is more ambitious than that of window-width (because it handles
faces) and the added complexity might potentially introduce more
corner cases where it doesn't work. But perhaps that's just the
paranoia of someone who thinks that text editors should restrict
themselves to monospaced fonts!

Regards,
N.

[PATCH] oc-basic: Detect malformed bibtex bibliographies

[Ihor Radchenko]

There is an edge case triggering infinite loop in oc-basic.

It is caused by bibtex-map-entries (used in
org-cite-basic--parse-bibtex) when ran on a malformed bibtex buffer [1].

The proposed patch validates the bibtex buffer before processing and
throws an error if issues are found.  `bibtex-validate` also
conveniently displays a list of errors with clickable links to
problematic lines.

I believe that it is useful for the users to see such issues instead of,
say, failing silently on malformed bibliographies.

WDYT?

Best,
Ihor

[1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=55036

* lisp/oc-basic.el (org-cite-basic--parse-bibtex): Validate the
bibliography before parsing.  Display list of issues if any (via
`bibtex-validate`).
(org-cite-basic--parse-bibliography): Set buffer file name needed by
`bibtex-validate`.  Empty the cache in case of error.
---
 lisp/oc-basic.el | 38 --
 1 file changed, 24 insertions(+), 14 deletions(-)

diff --git a/lisp/oc-basic.el b/lisp/oc-basic.el
index 873986d07..79f7a4844 100644
--- a/lisp/oc-basic.el
+++ b/lisp/oc-basic.el
@@ -214,6 +214,10 @@ (defun org-cite-basic--parse-bibtex (dialect)
   (let ((entries (make-hash-table :test #'equal))
 (bibtex-sort-ignore-string-entries t))
 (bibtex-set-dialect dialect t)
+;; Throw an error if bibliography is malformed.
+(unless (bibtex-validate)
+  (user-error "Malformed bibliography at %S"
+  (or (buffer-file-name) (current-buffer
 (bibtex-map-entries
  (lambda (key &rest _)
;; Normalize entries: field names are turned into symbols
@@ -258,21 +262,27 @@ (defun org-cite-basic--parse-bibliography (&optional info)
 (when (or (org-file-has-changed-p file)
   (not (gethash file org-cite-basic--file-id-cache)))
   (insert-file-contents file)
+  (setf (buffer-file-name) file)
   (puthash file (org-buffer-hash) org-cite-basic--file-id-cache))
-   (let* ((file-id (cons file (gethash file 
org-cite-basic--file-id-cache)))
-   (entries
-(or (cdr (assoc file-id 
org-cite-basic--bibliography-cache))
-(let ((table
-   (pcase (file-name-extension file)
- ("json" (org-cite-basic--parse-json))
- ("bib" (org-cite-basic--parse-bibtex 
'biblatex))
- ("bibtex" (org-cite-basic--parse-bibtex 
'BibTeX))
- (ext
-  (user-error "Unknown bibliography extension: 
%S"
-  ext)
-  (push (cons file-id table) 
org-cite-basic--bibliography-cache)
-  table
-  (push (cons file entries) results)
+(unwind-protect
+(condition-case nil
+(unwind-protect
+   (let* ((file-id (cons file (gethash file 
org-cite-basic--file-id-cache)))
+   (entries
+(or (cdr (assoc file-id 
org-cite-basic--bibliography-cache))
+(let ((table
+   (pcase (file-name-extension file)
+ ("json" 
(org-cite-basic--parse-json))
+ ("bib" 
(org-cite-basic--parse-bibtex 'biblatex))
+ ("bibtex" 
(org-cite-basic--parse-bibtex 'BibTeX))
+ (ext
+  (user-error "Unknown 
bibliography extension: %S"
+  ext)
+  (push (cons file-id table) 
org-cite-basic--bibliography-cache)
+  table
+  (push (cons file entries) results))
+  (setf (buffer-file-name) nil))
+  (error (setq org-cite-basic--file-id-cache nil)))
   (when info (plist-put info :cite-basic/bibliography results))
   results)))
 
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

Re: [PATCH] oc-basic: Detect malformed bibtex bibliographies

[Bruce D'Arcus]

On Wed, Apr 20, 2022 at 8:28 AM Ihor Radchenko  wrote:
>
> There is an edge case triggering infinite loop in oc-basic.
>
> It is caused by bibtex-map-entries (used in
> org-cite-basic--parse-bibtex) when ran on a malformed bibtex buffer [1].
>
> The proposed patch validates the bibtex buffer before processing and
> throws an error if issues are found.  `bibtex-validate` also
> conveniently displays a list of errors with clickable links to
> problematic lines.
>
> I believe that it is useful for the users to see such issues instead of,
> say, failing silently on malformed bibliographies.
>
> WDYT?

I haven't tested it, but this is an excellent idea!

Bruce

> Best,
> Ihor
>
> [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=55036
>
> * lisp/oc-basic.el (org-cite-basic--parse-bibtex): Validate the
> bibliography before parsing.  Display list of issues if any (via
> `bibtex-validate`).
> (org-cite-basic--parse-bibliography): Set buffer file name needed by
> `bibtex-validate`.  Empty the cache in case of error.
> ---
>  lisp/oc-basic.el | 38 --
>  1 file changed, 24 insertions(+), 14 deletions(-)
>
> diff --git a/lisp/oc-basic.el b/lisp/oc-basic.el
> index 873986d07..79f7a4844 100644
> --- a/lisp/oc-basic.el
> +++ b/lisp/oc-basic.el
> @@ -214,6 +214,10 @@ (defun org-cite-basic--parse-bibtex (dialect)
>(let ((entries (make-hash-table :test #'equal))
>  (bibtex-sort-ignore-string-entries t))
>  (bibtex-set-dialect dialect t)
> +;; Throw an error if bibliography is malformed.
> +(unless (bibtex-validate)
> +  (user-error "Malformed bibliography at %S"
> +  (or (buffer-file-name) (current-buffer
>  (bibtex-map-entries
>   (lambda (key &rest _)
> ;; Normalize entries: field names are turned into symbols
> @@ -258,21 +262,27 @@ (defun org-cite-basic--parse-bibliography (&optional 
> info)
>  (when (or (org-file-has-changed-p file)
>(not (gethash file org-cite-basic--file-id-cache)))
>(insert-file-contents file)
> +  (setf (buffer-file-name) file)
>(puthash file (org-buffer-hash) org-cite-basic--file-id-cache))
> -   (let* ((file-id (cons file (gethash file 
> org-cite-basic--file-id-cache)))
> -   (entries
> -(or (cdr (assoc file-id 
> org-cite-basic--bibliography-cache))
> -(let ((table
> -   (pcase (file-name-extension file)
> - ("json" (org-cite-basic--parse-json))
> - ("bib" (org-cite-basic--parse-bibtex 
> 'biblatex))
> - ("bibtex" (org-cite-basic--parse-bibtex 
> 'BibTeX))
> - (ext
> -  (user-error "Unknown bibliography 
> extension: %S"
> -  ext)
> -  (push (cons file-id table) 
> org-cite-basic--bibliography-cache)
> -  table
> -  (push (cons file entries) results)
> +(unwind-protect
> +(condition-case nil
> +(unwind-protect
> +   (let* ((file-id (cons file (gethash file 
> org-cite-basic--file-id-cache)))
> +   (entries
> +(or (cdr (assoc file-id 
> org-cite-basic--bibliography-cache))
> +(let ((table
> +   (pcase (file-name-extension file)
> + ("json" 
> (org-cite-basic--parse-json))
> + ("bib" 
> (org-cite-basic--parse-bibtex 'biblatex))
> + ("bibtex" 
> (org-cite-basic--parse-bibtex 'BibTeX))
> + (ext
> +  (user-error "Unknown 
> bibliography extension: %S"
> +  ext)
> +  (push (cons file-id table) 
> org-cite-basic--bibliography-cache)
> +  table
> +  (push (cons file entries) results))
> +  (setf (buffer-file-name) nil))
> +  (error (setq org-cite-basic--file-id-cache nil)))
>(when info (plist-put info :cite-basic/bibliography results))
>results)))
>
> --
> 2.35.1
>
>
>
> --
> Ihor Radchenko,
> PhD,
> Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
> State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
> University, Xi'an, China
> Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg
>

Re: [PATCH] oc-basic: Detect malformed bibtex bibliographies

[John Kitchin]

I would see if you can cache the result and not do it more than needed; it
can add a performance issue on large files.

John

---
Professor John Kitchin (he/him/his)
Doherty Hall A207F
Department of Chemical Engineering
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-7803
@johnkitchin
http://kitchingroup.cheme.cmu.edu



On Wed, Apr 20, 2022 at 8:39 AM Bruce D'Arcus  wrote:

> On Wed, Apr 20, 2022 at 8:28 AM Ihor Radchenko  wrote:
> >
> > There is an edge case triggering infinite loop in oc-basic.
> >
> > It is caused by bibtex-map-entries (used in
> > org-cite-basic--parse-bibtex) when ran on a malformed bibtex buffer [1].
> >
> > The proposed patch validates the bibtex buffer before processing and
> > throws an error if issues are found.  `bibtex-validate` also
> > conveniently displays a list of errors with clickable links to
> > problematic lines.
> >
> > I believe that it is useful for the users to see such issues instead of,
> > say, failing silently on malformed bibliographies.
> >
> > WDYT?
>
> I haven't tested it, but this is an excellent idea!
>
> Bruce
>
> > Best,
> > Ihor
> >
> > [1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=55036
> >
> > * lisp/oc-basic.el (org-cite-basic--parse-bibtex): Validate the
> > bibliography before parsing.  Display list of issues if any (via
> > `bibtex-validate`).
> > (org-cite-basic--parse-bibliography): Set buffer file name needed by
> > `bibtex-validate`.  Empty the cache in case of error.
> > ---
> >  lisp/oc-basic.el | 38 --
> >  1 file changed, 24 insertions(+), 14 deletions(-)
> >
> > diff --git a/lisp/oc-basic.el b/lisp/oc-basic.el
> > index 873986d07..79f7a4844 100644
> > --- a/lisp/oc-basic.el
> > +++ b/lisp/oc-basic.el
> > @@ -214,6 +214,10 @@ (defun org-cite-basic--parse-bibtex (dialect)
> >(let ((entries (make-hash-table :test #'equal))
> >  (bibtex-sort-ignore-string-entries t))
> >  (bibtex-set-dialect dialect t)
> > +;; Throw an error if bibliography is malformed.
> > +(unless (bibtex-validate)
> > +  (user-error "Malformed bibliography at %S"
> > +  (or (buffer-file-name) (current-buffer
> >  (bibtex-map-entries
> >   (lambda (key &rest _)
> > ;; Normalize entries: field names are turned into symbols
> > @@ -258,21 +262,27 @@ (defun org-cite-basic--parse-bibliography
> (&optional info)
> >  (when (or (org-file-has-changed-p file)
> >(not (gethash file
> org-cite-basic--file-id-cache)))
> >(insert-file-contents file)
> > +  (setf (buffer-file-name) file)
> >(puthash file (org-buffer-hash)
> org-cite-basic--file-id-cache))
> > -   (let* ((file-id (cons file (gethash file
> org-cite-basic--file-id-cache)))
> > -   (entries
> > -(or (cdr (assoc file-id
> org-cite-basic--bibliography-cache))
> > -(let ((table
> > -   (pcase (file-name-extension file)
> > - ("json" (org-cite-basic--parse-json))
> > - ("bib" (org-cite-basic--parse-bibtex
> 'biblatex))
> > - ("bibtex"
> (org-cite-basic--parse-bibtex 'BibTeX))
> > - (ext
> > -  (user-error "Unknown bibliography
> extension: %S"
> > -  ext)
> > -  (push (cons file-id table)
> org-cite-basic--bibliography-cache)
> > -  table
> > -  (push (cons file entries) results)
> > +(unwind-protect
> > +(condition-case nil
> > +(unwind-protect
> > +   (let* ((file-id (cons file (gethash file
> org-cite-basic--file-id-cache)))
> > +   (entries
> > +(or (cdr (assoc file-id
> org-cite-basic--bibliography-cache))
> > +(let ((table
> > +   (pcase (file-name-extension
> file)
> > + ("json"
> (org-cite-basic--parse-json))
> > + ("bib"
> (org-cite-basic--parse-bibtex 'biblatex))
> > + ("bibtex"
> (org-cite-basic--parse-bibtex 'BibTeX))
> > + (ext
> > +  (user-error "Unknown
> bibliography extension: %S"
> > +  ext)
> > +  (push (cons file-id table)
> org-cite-basic--bibliography-cache)
> > +  table
> > +  (push (cons file entries) results))
> > +  (setf

Re: [PATCH] oc-basic: Detect malformed bibtex bibliographies

[Ihor Radchenko]

John Kitchin  writes:

> I would see if you can cache the result and not do it more than needed; it
> can add a performance issue on large files.

The results of parsing are already cached. See 7ddc5b57c.

With this patch, I'd expect 2x performance degradation on (1) first time
Org opens the bibliography in current Emacs session; (2) after
bibliography is changed.

I believe that it is anyway worth it. Having errors in bibliography can
be easily overlooked and can potentially cause annoying issues.

Best,
Ihor

[PATCH v2 00/38] Final call for comments: Merge org-fold feature branch

[Ihor Radchenko]

This is the final version of the patch. I am going to merge it this
weekend. If there are any comments, please send them ASAP.

This version is basically the same as previous, but (1) Rebased onto
current main; (2) org-agenda.el can be native compiled; (3) Fixed some
edge cases with fontification.

Best,
Ihor

Anders Johansson (3):
  Fix typo: delete-duplicates → delete-dups
  Fix bug in org-get-heading
  Rename remaining org-force-cycle-archived → org-cycle-force-archived

Ihor Radchenko (35):
  Add org-fold-core: new folding engine
  Separate folding functions from org.el into new library: org-fold
  Separate cycling functions from org.el into new library: org-cycle
  Remove functions from org.el that are now moved elsewhere
  Disable native-comp in agenda
  org-macs: New function org-find-text-property-region
  org-at-heading-p: Accept optional argument
  org-string-width: Reimplement to work with new folding
  Rename old function call to use org-fold
  Implement link folding
  Implement overlay- and text-property-based versions of some functions
  org-fold: Handle indirect buffer visibility
  Fix subtle differences between overlays and invisible text properties
  Support extra org-fold optimisations for huge buffers
  Alias new org-fold functions to their old shorter names
  Obsolete old function names that are now in org-fold
  org-compat: Work around some third-party packages using outline-*
functions
  Move `org-buffer-list' to org-macs.el
  Restore old visibility behaviour of org-refile
  Add org-fold-related tests
  org-manual: Update to new org-fold function names
  ORG-NEWS: Add list of changes
  Backport contributed commits
  Fix org-fold--hide-drawers--overlays
  org-string-width: Handle undefined behaviour in older Emacs
  org-string-width: Work around `window-pixel-width' bug in old Emacs
  org-fold-show-set-visibility: Fix edge case when folded region is at
BOB
  org-fold-core: Fix fontification inside folded regions
  test-org/string-width: Add tests for strings with prefix properties
  org--string-from-props: Fix handling folds in Emacs <28
  org-link-make-string: Throw error when both LINK and DESCRIPTION are
empty
  test-ol/org-toggle-link-display: Fix compatibility with old Emacs
  org-macs.el: Fix fontification checks take 2
  org-fold-core-fontify-region: Fix cases when fontification is not
registered
  org-agenda.el: Re-enable native compilation

 doc/org-manual.org|   14 +-
 etc/ORG-NEWS  |  104 ++
 lisp/ob-core.el   |   14 +-
 lisp/ob-lilypond.el   |4 +-
 lisp/ob-ref.el|4 +-
 lisp/ol.el|   59 +-
 lisp/org-agenda.el|   48 +-
 lisp/org-archive.el   |   12 +-
 lisp/org-capture.el   |7 +-
 lisp/org-clock.el |  126 +-
 lisp/org-colview.el   |   10 +-
 lisp/org-compat.el|  189 ++-
 lisp/org-crypt.el |8 +-
 lisp/org-cycle.el |  818 +++
 lisp/org-element.el   |   55 +-
 lisp/org-feed.el  |4 +-
 lisp/org-fold-core.el | 1506 +++
 lisp/org-fold.el  | 1132 +++
 lisp/org-footnote.el  |6 +-
 lisp/org-goto.el  |6 +-
 lisp/org-id.el|4 +-
 lisp/org-inlinetask.el|   26 +-
 lisp/org-keys.el  |   26 +-
 lisp/org-lint.el  |3 +-
 lisp/org-list.el  |   84 +-
 lisp/org-macs.el  |  294 +++-
 lisp/org-mobile.el|2 +-
 lisp/org-mouse.el |4 +-
 lisp/org-refile.el|3 +-
 lisp/org-src.el   |6 +-
 lisp/org-timer.el |2 +-
 lisp/org.el   | 2550 +++--
 lisp/ox-org.el|2 +-
 lisp/ox.el|4 +-
 testing/lisp/test-ob.el   |   12 +-
 testing/lisp/test-ol.el   |   24 +
 testing/lisp/test-org-list.el |   75 +-
 testing/lisp/test-org-macs.el |6 +-
 testing/lisp/test-org.el  |  258 +++-
 39 files changed, 5480 insertions(+), 2031 deletions(-)
 create mode 100644 lisp/org-cycle.el
 create mode 100644 lisp/org-fold-core.el
 create mode 100644 lisp/org-fold.el

-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 03/38] Separate cycling functions from org.el into new library: org-cycle

[Ihor Radchenko]

---
 lisp/org-cycle.el | 818 ++
 1 file changed, 818 insertions(+)
 create mode 100644 lisp/org-cycle.el

diff --git a/lisp/org-cycle.el b/lisp/org-cycle.el
new file mode 100644
index 0..df0a3761a
--- /dev/null
+++ b/lisp/org-cycle.el
@@ -0,0 +1,818 @@
+;;; org-cycle.el --- Visibility cycling of Org entries -*- lexical-binding: t; 
-*-
+;;
+;; Copyright (C) 2020-2020 Free Software Foundation, Inc.
+;;
+;; Maintainer: Ihor Radchenko 
+;; Keywords: folding, visibility cycling, invisible text
+;; Homepage: https://orgmode.org
+;;
+;; This file is part of GNU Emacs.
+;;
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see .
+;
+;;
+;;; Commentary:
+
+;; This file contains code controlling global folding state in buffer
+;; and TAB-cycling.
+
+;;; Code:
+
+(require 'org-macs)
+(require 'org-fold)
+
+(declare-function org-element-type "org-element" (element))
+(declare-function org-element-property "org-element" (property element))
+(declare-function org-element-lineage "org-element" (datum &optional types 
with-self))
+(declare-function org-element-at-point "org-element" (&optional pom 
cached-only))
+(declare-function org-get-tags "org" (&optional pos local fontify))
+(declare-function org-subtree-end-visible-p "org" ())
+(declare-function org-narrow-to-subtree "org" (&optional element))
+(declare-function org-at-property-p "org" ())
+(declare-function org-re-property "org" (property &optional literal allow-null 
value))
+(declare-function org-item-beginning-re "org" ())
+(declare-function org-at-heading-p "org" (&optional invisible-not-ok))
+(declare-function org-at-item-p "org" ())
+(declare-function org-before-first-heading-p "org" ())
+(declare-function org-back-to-heading "org" (&optional invisible-ok))
+(declare-function org-end-of-subtree "org" (&optional invisible-ok to-heading))
+(declare-function org-entry-end-position "org" ())
+(declare-function org-try-cdlatex-tab "org" ())
+(declare-function org-cycle-level "org" ())
+(declare-function org-table-next-field "org-table" ())
+(declare-function org-table-justify-field-maybe "org-table" (&optional new))
+(declare-function org-inlinetask-at-task-p "org-inlinetask" ())
+(declare-function org-inlinetask-toggle-visibility "org-inlinetask" ())
+(declare-function org-list-get-all-items "org-list" (item struct prevs))
+(declare-function org-list-get-bottom-point "org-list" (struct))
+(declare-function org-list-prevs-alist "org-list" (struct))
+(declare-function org-list-set-item-visibility "org-list" (item struct view))
+(declare-function org-list-search-forward "org-list" (regexp &optional bound 
noerror))
+(declare-function org-list-has-child-p "org-list" (item struct))
+(declare-function org-list-get-item-end-before-blank "org-list" (item struct))
+(declare-function org-list-struct "org-list" ())
+(declare-function org-cycle-item-indentation "org-list" ())
+
+(declare-function outline-previous-heading "outline" ())
+(declare-function outline-next-heading "outline" ())
+(declare-function outline-end-of-heading "outline" ())
+(declare-function outline-up-heading "outline" (arg &optional invisible-ok))
+
+(defvar org-drawer-regexp)
+(defvar org-odd-levels-only)
+(defvar org-startup-folded)
+(defvar org-archive-tag)
+(defvar org-cycle-include-plain-lists)
+(defvar org-outline-regexp-bol)
+
+(defvar-local org-cycle-global-status nil)
+(put 'org-cycle-global-status 'org-state t)
+(defvar-local org-cycle-subtree-status nil)
+(put 'org-cycle-subtree-status 'org-state t)
+
+ Customisation:
+
+
+(defgroup org-cycle nil
+  "Options concerning visibility cycling in Org mode."
+  :tag "Org Cycle"
+  :group 'org-structure)
+
+(defcustom org-cycle-skip-children-state-if-no-children t
+  "Non-nil means skip CHILDREN state in entries that don't have any."
+  :group 'org-cycle
+  :type 'boolean)
+
+(defcustom org-cycle-max-level nil
+  "Maximum level which should still be subject to visibility cycling.
+Levels higher than this will, for cycling, be treated as text, not a headline.
+When `org-odd-levels-only' is set, a value of N in this variable actually
+means 2N-1 stars as the limiting headline.
+When nil, cycle all levels.
+Note that the limiting level of cycling is also influenced by
+`org-inlinetask-min-level'.  When `org-cycle-max-level' is not set but
+`org

[PATCH v2 01/38] Add org-fold-core: new folding engine---

[Ihor Radchenko]

---
 lisp/org-fold-core.el | 1490 +
 1 file changed, 1490 insertions(+)
 create mode 100644 lisp/org-fold-core.el

diff --git a/lisp/org-fold-core.el b/lisp/org-fold-core.el
new file mode 100644
index 0..121c6b5c4
--- /dev/null
+++ b/lisp/org-fold-core.el
@@ -0,0 +1,1490 @@
+;;; org-fold-core.el --- Folding buffer text -*- lexical-binding: t; -*-
+;;
+;; Copyright (C) 2020-2020 Free Software Foundation, Inc.
+;;
+;; Author: Ihor Radchenko 
+;; Keywords: folding, invisible text
+;; Homepage: https://orgmode.org
+;;
+;; This file is part of GNU Emacs.
+;;
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see .
+;
+;;
+;;; Commentary:
+
+;; This file contains library to control temporary invisibility
+;; (folding and unfolding) of text in buffers.
+
+;; The file implements the following functionality:
+;;
+;; - Folding/unfolding regions of text
+;; - Searching and examining boundaries of folded text
+;; - Interactive searching in folded text (via isearch)
+;; - Handling edits in folded text
+;; - Killing/yanking (copying/pasting) of the folded text
+;; - Fontification of the folded text
+
+;; To setup folding in an arbitrary buffer, one must call
+;; `org-fold-core-initialize', optionally providing the list of folding specs 
to be
+;; used in the buffer.  The specs can be added, removed, or
+;; re-configured later.  Read below for more details.
+
+;;; Folding/unfolding regions of text
+
+;; User can temporarily hide/reveal (fold/unfold) arbitrary regions or
+;; text.  The folds can be nested.
+
+;; Internally, nested folds are marked with different folding specs
+;; Overlapping folds marked with the same folding spec are
+;; automatically merged, while folds with different folding specs can
+;; coexist and be folded/unfolded independently.
+
+;; When multiple folding specs are applied to the same region of text,
+;; text visibility is decided according to the folding spec with
+;; topmost priority.
+
+;; By default, we define two types of folding specs:
+;; - 'org-fold-visible :: the folded text is not hidden
+;; - 'org-fold-hidden  :: the folded text is completely hidden
+;;
+;; The 'org-fold-visible spec has highest priority allowing parts of
+;; text folded with 'org-fold-hidden to be shown unconditionally.
+
+;; Consider the following Org mode link:
+;; [[file:/path/to/file/file.ext][description]]
+;; Only the word "description" is normally visible in this link.
+;; 
+;; The way this partial visibility is achieved is combining the two
+;; folding specs.  The whole link is folded using 'org-fold-hidden
+;; folding spec, but the visible part is additionally folded using
+;; 'org-fold-visible:
+;;
+;; [[file:/path/to/file/file.ext][description]]
+;; 
+;; Because 'org-fold-visible has higher priority than
+;; 'org-fold-hidden, it suppresses the 'org-fold-hidden effect and
+;; thus reveals the description part of the link.
+
+;; Similar to 'org-fold-visible, display of any arbitrary folding spec
+;; can be configured using folding spec properties.  In particular,
+;; `:visible' folding spec proprety controls whether the folded text
+;; is visible or not.  If the `:visible' folding spec property is nil,
+;; folded text is hidden or displayed as a constant string (ellipsis)
+;; according to the value of `:ellipsis' folding spec property.  See
+;; docstring of `org-fold-core--specs' for the description of all the available
+;; folding spec properties.
+
+;; Folding spec properties of any valid folding spec can be changed
+;; any time using `org-fold-core-set-folding-spec-property'.
+
+;; If necessary, one can add or remove folding specs using
+;; `org-fold-core-add-folding-spec' and `org-fold-core-remove-folding-spec'.
+
+;; If a buffer initialised with `org-fold-core-initialize' is cloned into 
indirect
+;; buffers, it's folding state is copied to that indirect buffer.
+;; The folding states are independent.
+
+;; When working with indirect buffers that are handled by this
+;; library, one has to keep in mind that folding state is preserved on
+;; copy when using non-interactive functions.  Moreover, the folding
+;; states of all the indirect buffers will be copied together.
+;;
+;; Example of the implications:
+;; Consider a base buffer and indirect buffer with the following state:
+;; - base buffer ---

[PATCH v2 04/38] Remove functions from org.el that are now moved elsewhere

[Ihor Radchenko]

---
 lisp/org.el | 1272 ++-
 1 file changed, 40 insertions(+), 1232 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 2353c6594..855f0813d 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -1179,90 +1179,6 @@ (defgroup org-structure nil
   :tag "Org Structure"
   :group 'org)
 
-(defgroup org-reveal-location nil
-  "Options about how to make context of a location visible."
-  :tag "Org Reveal Location"
-  :group 'org-structure)
-
-(defcustom org-show-context-detail '((agenda . local)
-(bookmark-jump . lineage)
-(isearch . lineage)
-(default . ancestors))
-  "Alist between context and visibility span when revealing a location.
-
-\\Some actions may move point into invisible
-locations.  As a consequence, Org always exposes a neighborhood
-around point.  How much is shown depends on the initial action,
-or context.  Valid contexts are
-
-  agenda when exposing an entry from the agenda
-  org-goto   when using the command `org-goto' (`\\[org-goto]')
-  occur-tree when using the command `org-occur' (`\\[org-sparse-tree] /')
-  tags-tree  when constructing a sparse tree based on tags matches
-  link-searchwhen exposing search matches associated with a link
-  mark-goto  when exposing the jump goal of a mark
-  bookmark-jump  when exposing a bookmark location
-  isearchwhen exiting from an incremental search
-  defaultdefault for all contexts not set explicitly
-
-Allowed visibility spans are
-
-  minimalshow current headline; if point is not on headline,
- also show entry
-
-  local  show current headline, entry and next headline
-
-  ancestors  show current headline and its direct ancestors; if
- point is not on headline, also show entry
-
-  ancestors-full show current subtree and its direct ancestors
-
-  lineageshow current headline, its direct ancestors and all
- their children; if point is not on headline, also show
- entry and first child
-
-  tree   show current headline, its direct ancestors and all
- their children; if point is not on headline, also show
- entry and all children
-
-  canonical  show current headline, its direct ancestors along with
- their entries and children; if point is not located on
- the headline, also show current entry and all children
-
-As special cases, a nil or t value means show all contexts in
-`minimal' or `canonical' view, respectively.
-
-Some views can make displayed information very compact, but also
-make it harder to edit the location of the match.  In such
-a case, use the command `org-reveal' (`\\[org-reveal]') to show
-more context."
-  :group 'org-reveal-location
-  :version "26.1"
-  :package-version '(Org . "9.0")
-  :type '(choice
- (const :tag "Canonical" t)
- (const :tag "Minimal" nil)
- (repeat :greedy t :tag "Individual contexts"
- (cons
-  (choice :tag "Context"
-  (const agenda)
-  (const org-goto)
-  (const occur-tree)
-  (const tags-tree)
-  (const link-search)
-  (const mark-goto)
-  (const bookmark-jump)
-  (const isearch)
-  (const default))
-  (choice :tag "Detail level"
-  (const minimal)
-  (const local)
-  (const ancestors)
-   (const ancestors-full)
-  (const lineage)
-  (const tree)
-  (const canonical))
-
 (defcustom org-indirect-buffer-display 'other-window
   "How should indirect tree buffers be displayed?
 
@@ -1454,130 +1370,6 @@ (defcustom org-bookmark-names-plist
   :group 'org-structure
   :type 'plist)
 
-(defgroup org-cycle nil
-  "Options concerning visibility cycling in Org mode."
-  :tag "Org Cycle"
-  :group 'org-structure)
-
-(defcustom org-cycle-skip-children-state-if-no-children t
-  "Non-nil means skip CHILDREN state in entries that don't have any."
-  :group 'org-cycle
-  :type 'boolean)
-
-(defcustom org-cycle-max-level nil
-  "Maximum level which should still be subject to visibility cycling.
-Levels higher than this will, for cycling, be treated as text, not a headline.
-When `org-odd-levels-only' is set, a value of N in this variable actually
-means 2N-1 stars as the limiting headline.
-When nil, cycle all levels.
-Note that the limiting level of cycling is also influenced by
-`org-inlinetask-min-level'.  When `org-cycle-max-level' is not set but
-`org-inlinetask-min-level' is, cycling will be limited

[PATCH v2 05/38] Disable native-comp in agendaIt caused cryptic bugs in the past.

[Ihor Radchenko]

It caused cryptic bugs in the past.
---
 lisp/org-agenda.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index a09b53563..b55e00803 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -1,4 +1,4 @@
-;;; org-agenda.el --- Dynamic task and appointment lists for Org  -*- 
lexical-binding: t; -*-
+;;; org-agenda.el --- Dynamic task and appointment lists for Org  -*- 
lexical-binding: t; no-native-compile: t; -*-
 
 ;; Copyright (C) 2004-2022 Free Software Foundation, Inc.
 
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 02/38] Separate folding functions from org.el into new library: org-fold

[Ihor Radchenko]

---
 lisp/org-fold.el | 1135 ++
 1 file changed, 1135 insertions(+)
 create mode 100644 lisp/org-fold.el

diff --git a/lisp/org-fold.el b/lisp/org-fold.el
new file mode 100644
index 0..52717fd86
--- /dev/null
+++ b/lisp/org-fold.el
@@ -0,0 +1,1135 @@
+;;; org-fold.el --- Folding of Org entries -*- lexical-binding: t; -*-
+;;
+;; Copyright (C) 2020-2020 Free Software Foundation, Inc.
+;;
+;; Author: Ihor Radchenko 
+;; Keywords: folding, invisible text
+;; Homepage: https://orgmode.org
+;;
+;; This file is part of GNU Emacs.
+;;
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see .
+;
+;;
+;;; Commentary:
+
+;; This file contains code handling temporary invisibility (folding
+;; and unfolding) of text in org buffers.
+
+;; The folding is implemented using generic org-fold-core library.  This file
+;; contains org-specific implementation of the folding.  Also, various
+;; useful functions from org-fold-core are aliased under shorted `org-fold'
+;; prefix.
+
+;; The following features are implemented:
+;; - Folding/unfolding various Org mode elements and regions of Org buffers:
+;;   + Region before first heading;
+;;   + Org headings, their text, children (subtree), siblings, parents, etc;
+;;   + Org blocks and drawers
+;; - Revealing Org structure around invisible point location
+;; - Revealing folded Org elements broken by user edits
+
+;;; Code:
+
+(require 'org-macs)
+(require 'org-fold-core)
+
+(defvar org-inlinetask-min-level)
+(defvar org-link--link-folding-spec)
+(defvar org-link--description-folding-spec)
+(defvar org-odd-levels-only)
+(defvar org-drawer-regexp)
+(defvar org-property-end-re)
+(defvar org-link-descriptive)
+(defvar org-outline-regexp-bol)
+(defvar org-custom-properties-hidden-p)
+(defvar org-archive-tag)
+
+;; Needed for overlays only
+(defvar org-custom-properties-overlays)
+
+(declare-function isearch-filter-visible "isearch" (beg end))
+(declare-function org-element-type "org-element" (element))
+(declare-function org-element-at-point "org-element" (&optional pom 
cached-only))
+(declare-function org-element-property "org-element" (property element))
+(declare-function org-element--current-element "org-element" (limit &optional 
granularity mode structure))
+(declare-function org-element--cache-active-p "org-element" ())
+(declare-function org-toggle-custom-properties-visibility "org" ())
+(declare-function org-item-re "org-list" ())
+(declare-function org-up-heading-safe "org" ())
+(declare-function org-get-tags "org" (&optional pos local fontify))
+(declare-function org-get-valid-level "org" (level &optional change))
+(declare-function org-before-first-heading-p "org" ())
+(declare-function org-goto-sibling "org" (&optional previous))
+(declare-function org-block-map "org" (function &optional start end))
+(declare-function org-map-region "org" (fun beg end))
+(declare-function org-end-of-subtree "org" (&optional invisible-ok to-heading))
+(declare-function org-back-to-heading-or-point-min "org" (&optional 
invisible-ok))
+(declare-function org-back-to-heading "org" (&optional invisible-ok))
+(declare-function org-at-heading-p "org" (&optional invisible-not-ok))
+(declare-function org-cycle-hide-drawers "org-cycle" (state))
+
+(declare-function outline-show-branches "outline" ())
+(declare-function outline-hide-sublevels "outline" (levels))
+(declare-function outline-get-next-sibling "outline" ())
+(declare-function outline-invisible-p "outline" (&optional pos))
+(declare-function outline-next-heading "outline" ())
+
+;;; Customization
+
+(defgroup org-fold-reveal-location nil
+  "Options about how to make context of a location visible."
+  :tag "Org Reveal Location"
+  :group 'org-structure)
+
+(defcustom org-fold-show-context-detail '((agenda . local)
+ (bookmark-jump . lineage)
+ (isearch . lineage)
+ (default . ancestors))
+  "Alist between context and visibility span when revealing a location.
+
+\\Some actions may move point into invisible
+locations.  As a consequence, Org always exposes a neighborhood
+around point.  How much is shown depends on the initial action,
+or context.  Valid contexts are
+
+  agenda when exposing an entry from the agenda
+  org-goto

[PATCH v2 15/38] Alias new org-fold functions to their old shorter names

[Ihor Radchenko]

---
 lisp/org.el | 8 
 1 file changed, 8 insertions(+)

diff --git a/lisp/org.el b/lisp/org.el
index f5936b67c..2608865da 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -99,6 +99,14 @@ (require 'org-table)
 (require 'org-fold)
 
 (require 'org-cycle)
+(defvaralias 'org-hide-block-startup 'org-cycle-hide-block-startup)
+(defvaralias 'org-pre-cycle-hook 'org-cycle-pre-hook)
+(defvaralias 'org-tab-first-hook 'org-cycle-tab-first-hook)
+(defalias 'org-global-cycle #'org-cycle-global)
+(defalias 'org-overview #'org-cycle-overview)
+(defalias 'org-content #'org-cycle-content)
+(defalias 'org-reveal #'org-fold-reveal)
+(defalias 'org-force-cycle-archived #'org-cycle-force-archived)
 
 ;; `org-outline-regexp' ought to be a defconst but is let-bound in
 ;; some places -- e.g. see the macro `org-with-limited-levels'.
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 08/38] org-string-width: Reimplement to work with new folding

[Ihor Radchenko]

* lisp/org-macs.el (org--string-from-props): Removed since it is no
longer needed.
(org-string-width): Updated to use `window-text-pixel-size'.
---
 lisp/org-macs.el | 121 ++-
 1 file changed, 57 insertions(+), 64 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 6161a7bfc..f63458f70 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -889,71 +889,64 @@ (defun org-split-string (string &optional separators)
  results   ;skip trailing separator
(cons (substring string i) results)))
 
-(defun org--string-from-props (s property beg end)
-  "Return the visible part of string S.
-Visible part is determined according to text PROPERTY, which is
-either `invisible' or `display'.  BEG and END are 0-indices
-delimiting S."
-  (let ((width 0)
-   (cursor beg))
-(while (setq beg (text-property-not-all beg end property nil s))
-  (let* ((next (next-single-property-change beg property s end))
-(props (text-properties-at beg s))
-(spec (plist-get props property))
-(value
- (pcase property
-   (`invisible
-;; If `invisible' property in PROPS means text is to
-;; be invisible, return 0.  Otherwise return nil so
-;; as to resume search.
-(and (or (eq t buffer-invisibility-spec)
- (assoc-string spec buffer-invisibility-spec))
- 0))
-   (`display
-(pcase spec
-  (`nil nil)
-  (`(space . ,props)
-   (let ((width (plist-get props :width)))
- (and (wholenump width) width)))
-  (`(image . ,_)
-(and (fboundp 'image-size)
- (ceiling (car (image-size spec)
-  ((pred stringp)
-   ;; Displayed string could contain invisible parts,
-   ;; but no nested display.
-   (org--string-from-props spec 'invisible 0 (length spec)))
-  (_
-   ;; Un-handled `display' value.  Ignore it.
-   ;; Consider the original string instead.
-   nil)))
-   (_ (error "Unknown property: %S" property)
-   (when value
- (cl-incf width
-  ;; When looking for `display' parts, we still need
-  ;; to look for `invisible' property elsewhere.
-  (+ (cond ((eq property 'display)
-(org--string-from-props s 'invisible cursor beg))
-   ((= cursor beg) 0)
-   (t (string-width (substring s cursor beg
- value))
- (setq cursor next))
-   (setq beg next)))
-(+ width
-   ;; Look for `invisible' property in the last part of the
-   ;; string.  See above.
-   (cond ((eq property 'display)
- (org--string-from-props s 'invisible cursor end))
-((= cursor end) 0)
-(t (string-width (substring s cursor end)))
-
-(defun org-string-width (string)
+(defun org-string-width (string &optional pixels)
   "Return width of STRING when displayed in the current buffer.
-Unlike `string-width', this function takes into consideration
-`invisible' and `display' text properties.  It supports the
-latter in a limited way, mostly for combinations used in Org.
-Results may be off sometimes if it cannot handle a given
-`display' value."
-  (org--string-from-props string 'display 0 (length string)))
+Return width in pixels when PIXELS is non-nil."
+  ;; Wrap/line prefix will make `window-text-pizel-size' return too
+  ;; large value including the prefix.
+  ;; Face should be removed to make sure that all the string symbols
+  ;; are using default face with constant width.  Constant char width
+  ;; is critical to get right string width from pixel width.
+  (remove-text-properties 0 (length string)
+  '(wrap-prefix t line-prefix t face t)
+  string)
+  (let (;; We need to remove the folds to make sure that folded table
+;; alignment is not messed up.
+(current-invisibility-spec
+ (or (and (not (listp buffer-invisibility-spec))
+  buffer-invisibility-spec)
+ (let (result)
+   (dolist (el buffer-invisibility-spec)
+ (unless (or (memq el
+   '(org-fold-drawer
+ org-fold-block
+ org-fold-outline))
+ (and (listp el)
+  (memq (car el)
+'(org-fold-drawer
+  org-fold-block
+  org-fold-outline
+

[PATCH v2 07/38] org-at-heading-p: Accept optional argument* lisp/org.el (org-at-heading-p): Use second argument to allow

[Ihor Radchenko]

* lisp/org.el (org-at-heading-p): Use second argument to allow
checking for visible headings.  Note that by default, unlike
`outline-on-heading-p', `org-at-heading-p' returns non-nil for
invisible headings. Passing second argument is just like
`(outline-on-heading-p)'.
(org-indent-line):
* lisp/org-agenda.el (org-agenda-add-entry-to-org-agenda-diary-file):
* lisp/org-colview.el (org-columns--call):
(org-columns-store-format): Update arguments in `org-at-heading-p'
calls.
---
 lisp/org-agenda.el  |  2 +-
 lisp/org-colview.el |  4 ++--
 lisp/org.el | 14 +-
 3 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index b55e00803..862243f28 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -10527,7 +10527,7 @@ (defun org-agenda-add-entry-to-org-agenda-diary-file 
(type text &optional d1 d2)
   (anniversary
(or (re-search-forward "^\\*[ \t]+Anniversaries" nil t)
   (progn
-(or (org-at-heading-p t)
+(or (org-at-heading-p)
 (progn
   (outline-next-heading)
   (insert "* Anniversaries\n\n")
diff --git a/lisp/org-colview.el b/lisp/org-colview.el
index 082d6def0..15cab35f0 100644
--- a/lisp/org-colview.el
+++ b/lisp/org-colview.el
@@ -699,7 +699,7 @@ (defun org-columns--call (fun)
   (let ((hide-body (and (/= (line-end-position) (point-max))
(save-excursion
  (move-beginning-of-line 2)
- (org-at-heading-p t)
+ (org-at-heading-p)
 (unwind-protect (funcall fun)
   (when hide-body (outline-hide-entry)
 
@@ -1026,7 +1026,7 @@ (defun org-columns-store-format ()
  ;; No COLUMNS keyword in the buffer.  Insert one at the
  ;; beginning, right before the first heading, if any.
  (goto-char (point-min))
- (unless (org-at-heading-p t) (outline-next-heading))
+ (unless (org-at-heading-p) (outline-next-heading))
  (let ((inhibit-read-only t))
(insert-before-markers "#+COLUMNS: " fmt "\n"
  (setq-local org-columns-default-format fmt))
diff --git a/lisp/org.el b/lisp/org.el
index b9d2d7834..8c823a7c8 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -17259,7 +17259,7 @@ (defun org-toggle-heading (&optional nstars)
;; Case 1. Started at an heading: de-star headings.
((org-at-heading-p)
 (while (< (point) end)
-  (when (org-at-heading-p t)
+  (when (org-at-heading-p)
 (looking-at org-outline-regexp) (replace-match "")
 (setq toggled t))
   (forward-line)))
@@ -17861,7 +17861,7 @@ (defun org-context ()
 (p (point)) clist o)
 ;; First the large context
 (cond
- ((org-at-heading-p t)
+ ((org-at-heading-p)
   (push (list :headline (point-at-bol) (point-at-eol)) clist)
   (when (progn
  (beginning-of-line 1)
@@ -19611,9 +19611,13 @@ (defun org-before-first-heading-p ()
(end-of-line)
(null (re-search-backward org-outline-regexp-bol nil t))
 
-(defun org-at-heading-p (&optional _)
-  "Non-nil when on a headline."
-  (outline-on-heading-p t))
+(defun org-at-heading-p (&optional invisible-not-ok)
+  "Return t if point is on a (possibly invisible) heading line.
+If INVISIBLE-NOT-OK is non-nil, an invisible heading line is not ok."
+  (save-excursion
+(beginning-of-line)
+(and (bolp) (or (not invisible-not-ok) (not (org-fold-folded-p)))
+(looking-at outline-regexp
 
 (defun org-in-commented-heading-p (&optional no-inheritance element)
   "Non-nil if point is under a commented heading.
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 06/38] org-macs: New function org-find-text-property-region---

[Ihor Radchenko]

---
 lisp/org-macs.el | 32 +---
 lisp/org.el  |  7 ++-
 2 files changed, 23 insertions(+), 16 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index a09115e7c..6161a7bfc 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -728,7 +728,7 @@ (defsubst org-current-line (&optional pos)
 
 
 
-;;; Overlays
+;;; Overlays and text properties
 
 (defun org-overlay-display (ovl text &optional face evap)
   "Make overlay OVL display TEXT with face FACE."
@@ -751,20 +751,22 @@ (defun org-find-overlays (prop &optional pos delete)
(delete (delete-overlay ov))
(t (push ov found))
 
-(defun org-flag-region (from to flag spec)
-  "Hide or show lines from FROM to TO, according to FLAG.
-SPEC is the invisibility spec, as a symbol."
-  (remove-overlays from to 'invisible spec)
-  ;; Use `front-advance' since text right before to the beginning of
-  ;; the overlay belongs to the visible line than to the contents.
-  (when flag
-(let ((o (make-overlay from to nil 'front-advance)))
-  (overlay-put o 'evaporate t)
-  (overlay-put o 'invisible spec)
-  (overlay-put o
-  'isearch-open-invisible
-  (lambda (&rest _) (org-show-context 'isearch))
-
+(defun org-find-text-property-region (pos prop)
+  "Find a region around POS containing same non-nil value of PROP text 
property.
+Return nil when PROP is not set at POS."
+  (let* ((beg (and (get-text-property pos prop) pos))
+(end beg))
+(when beg
+  (unless (or (equal beg (point-min))
+ (not (eq (get-text-property beg prop)
+(get-text-property (1- beg) prop
+   (setq beg (previous-single-property-change pos prop nil (point-min
+  (unless (or (equal end (point-max))
+ ;; (not (eq (get-text-property end prop)
+ ;; (get-text-property (1+ end) prop)))
+ )
+   (setq end (next-single-property-change pos prop nil (point-max
+  (cons beg end
 
 
 ;;; Regexp matching
diff --git a/lisp/org.el b/lisp/org.el
index 855f0813d..b9d2d7834 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5597,7 +5597,10 @@ (defun org-fontify-like-in-org-mode (s &optional 
odd-levels)
 (let ((org-odd-levels-only odd-levels))
   (org-mode)
   (org-font-lock-ensure)
-  (buffer-string
+  (if org-link-descriptive
+  (org-link-display-format
+   (buffer-string))
+(buffer-string)
 
 (defun org-get-level-face (n)
   "Get the right face for match N in font-lock matching of headlines."
@@ -5727,6 +5730,8 @@ (defun org-raise-scripts (limit)
   (if (equal (char-after (match-beginning 2)) ?^)
   (nth (if table-p 3 1) org-script-display)
 (nth (if table-p 2 0) org-script-display)))
+(put-text-property (match-beginning 2) (match-end 3)
+   'org-emphasis t)
(add-text-properties (match-beginning 2) (match-end 2)
 (list 'invisible t))
(when (and (eq (char-after (match-beginning 3)) ?{)
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 21/38] org-manual: Update to new org-fold function names---

[Ihor Radchenko]

---
 doc/org-manual.org | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 14fea0f2b..af54dc4e8 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -501,11 +501,11 @@ *** Global and local cycling
   Switch back to the startup visibility of the buffer (see [[*Initial
   visibility]]).
 
-- {{{kbd(C-u C-u C-u TAB)}}} (~outline-show-all~) ::
+- {{{kbd(C-u C-u C-u TAB)}}} (~org-show-all~) ::
 
   #+cindex: show all, command
   #+kindex: C-u C-u C-u TAB
-  #+findex: outline-show-all
+  #+findex: org-show-all
   Show all, including drawers.
 
 - {{{kbd(C-c C-r)}}} (~org-reveal~) ::
@@ -521,18 +521,18 @@ *** Global and local cycling
   headings.  With a double prefix argument, also show the entire
   subtree of the parent.
 
-- {{{kbd(C-c C-k)}}} (~outline-show-branches~) ::
+- {{{kbd(C-c C-k)}}} (~org-show-branches~) ::
 
   #+cindex: show branches, command
   #+kindex: C-c C-k
-  #+findex: outline-show-branches
+  #+findex: org-show-branches
   Expose all the headings of the subtree, but not their bodies.
 
-- {{{kbd(C-c TAB)}}} (~outline-show-children~) ::
+- {{{kbd(C-c TAB)}}} (~org-show-children~) ::
 
   #+cindex: show children, command
   #+kindex: C-c TAB
-  #+findex: outline-show-children
+  #+findex: org-show-children
   Expose all direct children of the subtree.  With a numeric prefix
   argument {{{var(N)}}}, expose all children down to level
   {{{var(N)}}}.
@@ -7375,7 +7375,7 @@ *** Internal archiving
   command (see [[*Visibility Cycling]]).  You can force cycling archived
   subtrees with {{{kbd(C-TAB)}}}, or by setting the option
   ~org-cycle-open-archived-trees~.  Also normal outline commands, like
-  ~outline-show-all~, open archived subtrees.
+  ~org-show-all~, open archived subtrees.
 
 -
   #+vindex: org-sparse-tree-open-archived-trees
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 12/38] org-fold: Handle indirect buffer visibility---

[Ihor Radchenko]

---
 lisp/org-capture.el | 5 -
 lisp/org.el | 8 +++-
 2 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/lisp/org-capture.el b/lisp/org-capture.el
index 1324ffab4..068e3eda2 100644
--- a/lisp/org-capture.el
+++ b/lisp/org-capture.el
@@ -1171,7 +1171,10 @@ (defun org-capture-place-entry ()
   (goto-char (point-min))
   (unless (org-at-heading-p) (outline-next-heading)))
  ;; Otherwise, insert as a top-level entry at the end of the file.
- (t (goto-char (point-max
+ (t (goto-char (point-max))
+;; Make sure that last point is not folded.
+(org-fold-core-cycle-over-indirect-buffers
+(org-fold-region (max 1 (1- (point-max))) (point-max) nil
 (let ((origin (point)))
   (unless (bolp) (insert "\n"))
   (org-capture-empty-lines-before)
diff --git a/lisp/org.el b/lisp/org.el
index 0b50e30d9..9ebdb23e1 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5984,7 +5984,13 @@ (defun org-get-indirect-buffer (&optional buffer heading)
 (number-to-string n))
   (setq n (1+ n)))
 (condition-case nil
-(make-indirect-buffer buffer bname 'clone)
+(let ((indirect-buffer (make-indirect-buffer buffer bname 'clone)))
+  ;; Decouple folding state.  We need to do it manually since
+  ;; `make-indirect-buffer' does not run
+  ;; `clone-indirect-buffer-hook'.
+  (org-fold-core-decouple-indirect-buffer-folds)
+  ;; Return the buffer.
+  indirect-buffer)
   (error (make-indirect-buffer buffer bname)
 
 (defun org-set-frame-title (title)
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 09/38] Rename old function call to use org-fold---

[Ihor Radchenko]

---
 lisp/ob-core.el   |  14 ++--
 lisp/ob-lilypond.el   |   4 +-
 lisp/ob-ref.el|   4 +-
 lisp/ol.el|  13 ++--
 lisp/org-agenda.el|  43 +--
 lisp/org-archive.el   |  12 +--
 lisp/org-capture.el   |   2 +-
 lisp/org-clock.el |  10 +--
 lisp/org-colview.el   |   6 +-
 lisp/org-compat.el|  29 
 lisp/org-crypt.el |   8 +-
 lisp/org-element.el   |   1 +
 lisp/org-feed.el  |   4 +-
 lisp/org-footnote.el  |   6 +-
 lisp/org-goto.el  |   6 +-
 lisp/org-id.el|   4 +-
 lisp/org-keys.el  |  26 +++
 lisp/org-lint.el  |   3 +-
 lisp/org-list.el  |  10 ++-
 lisp/org-macs.el  |  40 ++
 lisp/org-mobile.el|   2 +-
 lisp/org-mouse.el |   4 +-
 lisp/org-refile.el|   2 +-
 lisp/org-src.el   |   6 +-
 lisp/org-timer.el |   2 +-
 lisp/org.el   | 135 +++---
 lisp/ox-org.el|   2 +-
 testing/lisp/test-org-list.el |   2 +-
 testing/lisp/test-org.el  |  78 ++--
 29 files changed, 241 insertions(+), 237 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 239a57f96..65907 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -26,7 +26,9 @@ ;;; Code:
 (require 'cl-lib)
 (require 'ob-eval)
 (require 'org-macs)
+(require 'org-fold)
 (require 'org-compat)
+(require 'org-cycle)
 
 (defconst org-babel-exeext
   (if (memq system-type '(windows-nt cygwin))
@@ -50,7 +52,7 @@ (declare-function org-babel-ref-resolve "ob-ref" (ref))
 (declare-function org-babel-ref-split-args "ob-ref" (arg-string))
 (declare-function org-babel-tangle-comment-links "ob-tangle" (&optional info))
 (declare-function org-current-level "org" ())
-(declare-function org-cycle "org" (&optional arg))
+(declare-function org-cycle "org-cycle" (&optional arg))
 (declare-function org-edit-src-code "org-src" (&optional code 
edit-buffer-name))
 (declare-function org-edit-src-exit "org-src"  ())
 (declare-function org-element-at-point "org-element" (&optional pom 
cached-only))
@@ -75,7 +77,7 @@ (declare-function org-narrow-to-subtree "org" (&optional 
element))
 (declare-function org-next-block "org" (arg &optional backward block-regexp))
 (declare-function org-open-at-point "org" (&optional in-emacs 
reference-buffer))
 (declare-function org-previous-block "org" (arg &optional block-regexp))
-(declare-function org-show-context "org" (&optional key))
+(declare-function org-fold-show-context "org-fold" (&optional key))
 (declare-function org-src-coderef-format "org-src" (&optional element))
 (declare-function org-src-coderef-regexp "org-src" (fmt &optional label))
 (declare-function org-src-get-lang-mode "org-src" (lang))
@@ -945,7 +947,7 @@ (defun org-babel-enter-header-arg-w-completion (&optional 
lang)
 (insert (concat header " " (or arg "")))
 (cons header arg)))
 
-(add-hook 'org-tab-first-hook 'org-babel-header-arg-expand)
+(add-hook 'org-cycle-tab-first-hook 'org-babel-header-arg-expand)
 
 ;;;###autoload
 (defun org-babel-load-in-session (&optional _arg info)
@@ -1469,7 +1471,7 @@ (defun org-babel-hide-result-toggle (&optional force)
(push ov org-babel-hide-result-overlays)
 
 ;; org-tab-after-check-for-cycling-hook
-(add-hook 'org-tab-first-hook #'org-babel-hide-result-toggle-maybe)
+(add-hook 'org-cycle-tab-first-hook #'org-babel-hide-result-toggle-maybe)
 ;; Remove overlays when changing major mode
 (add-hook 'org-mode-hook
  (lambda () (add-hook 'change-major-mode-hook
@@ -1817,7 +1819,7 @@ (defun org-babel-goto-named-src-block (name)
   (let ((point (org-babel-find-named-block name)))
 (if point
 ;; Taken from `org-open-at-point'.
-(progn (org-mark-ring-push) (goto-char point) (org-show-context))
+(progn (org-mark-ring-push) (goto-char point) (org-fold-show-context))
   (message "source-code block `%s' not found in this buffer" name
 
 (defun org-babel-find-named-block (name)
@@ -1857,7 +1859,7 @@ (defun org-babel-goto-named-result (name)
   (let ((point (org-babel-find-named-result name)))
 (if point
 ;; taken from `org-open-at-point'
-(progn (goto-char point) (org-show-context))
+(progn (goto-char point) (org-fold-show-context))
   (message "result `%s' not found in this buffer" name
 
 (defun org-babel-find-named-result (name)
diff --git a/lisp/ob-lilypond.el b/lisp/ob-lilypond.el
index 15538b503..df128441a 100644
--- a/lisp/ob-lilypond.el
+++ b/lisp/ob-lilypond.el
@@ -34,7 +34,7 @@ ;;; Commentary:
 ;;; Code:
 (require 'ob)
 
-(declare-function org-show-all "org" (&optional types))
+(declare-function org-fold-show-all "org-fold" (&optional types))
 
 (defalias 'lilypond-mode 'LilyPond-mode)
 
@@ -279,7 +279,7 @@ (defun org-babel-lilypond-mark-error-line

[PATCH v2 10/38] Implement link folding* lisp/ol.el (org-link--link-folding-spec):

[Ihor Radchenko]

* lisp/ol.el (org-link--link-folding-spec):
(org-link--description-folding-spec): New variables controlling link
folding settings.
(org-link--reveal-maybe): Handle revealing folded links.
(org-link-descriptive-ensure): Implement `org-link-descriptive'
support with org-fold.
(org-toggle-link-display--overlays):
(org-toggle-link-display--text-properties):
(org-toggle-link-display): Provide text-properties and overlays
versions.
* lisp/org-agenda.el (org-agenda-mode): Use org-fold to fold links in
agenda.
* lisp/org.el (org-do-emphasis-faces): Use org-fold.
---
 lisp/ol.el | 42 +-
 lisp/org-agenda.el |  3 ++-
 lisp/org.el| 11 +--
 3 files changed, 52 insertions(+), 4 deletions(-)

diff --git a/lisp/ol.el b/lisp/ol.el
index 4cc813d5b..86f55d7cf 100644
--- a/lisp/ol.el
+++ b/lisp/ol.el
@@ -605,6 +605,22 @@ (defvar org-link--insert-history nil
 (defvar org-link--search-failed nil
   "Non-nil when last link search failed.")
 
+
+(defvar-local org-link--link-folding-spec '(org-link
+(:global t)
+(:ellipsis . nil)
+(:isearch-open . t)
+(:fragile . 
org-link--reveal-maybe))
+  "Folding spec used to hide invisible parts of links.")
+
+(defvar-local org-link--description-folding-spec '(org-link-description
+   (:global t)
+   (:ellipsis . nil)
+   (:visible . t)
+   (:isearch-open . nil)
+   (:fragile . 
org-link--reveal-maybe))
+  "Folding spec used to reveal link description.")
+
 
 ;;; Internal Functions
 
@@ -762,6 +778,13 @@ (defun org-link--normalize-string (string &optional 
context)
   (t nil
 string))
 
+(defun org-link--reveal-maybe (region _)
+  "Reveal folded link in REGION when needed.
+This function is intended to be used as :fragile property of a folding
+spec."
+  (org-with-point-at (car region)
+(not (org-in-regexp org-link-any-re
+
 
 ;;; Public API
 
@@ -1444,14 +1467,31 @@ (defun org-previous-link ()
   (interactive)
   (org-next-link t))
 
+(defun org-link-descriptive-ensure ()
+  "Toggle the literal or descriptive display of links in current buffer if 
needed."
+  (if org-link-descriptive
+  (org-fold-core-set-folding-spec-property (car 
org-link--link-folding-spec) :visible nil)
+(org-fold-core-set-folding-spec-property (car org-link--link-folding-spec) 
:visible t)))
+
 ;;;###autoload
-(defun org-toggle-link-display ()
+(defun org-toggle-link-display--overlays ()
   "Toggle the literal or descriptive display of links."
   (interactive)
   (if org-link-descriptive (remove-from-invisibility-spec '(org-link))
 (add-to-invisibility-spec '(org-link)))
   (org-restart-font-lock)
   (setq org-link-descriptive (not org-link-descriptive)))
+(defun org-toggle-link-display--text-properties ()
+  "Toggle the literal or descriptive display of links in current buffer."
+  (interactive)
+  (setq org-link-descriptive (not org-link-descriptive))
+  (org-link-descriptive-ensure))
+(defsubst org-toggle-link-display ()
+  "Toggle the literal or descriptive display of links."
+  (interactive)
+  (if (eq org-fold-core-style 'text-properties)
+  (org-toggle-link-display--text-properties)
+(org-toggle-link-display--overlays)))
 
 ;;;###autoload
 (defun org-store-link (arg &optional interactive?)
diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index fa60f4f19..6aed778f0 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -2325,7 +2325,8 @@ (defun org-agenda-mode ()
  org-agenda-show-log org-agenda-start-with-log-mode
  org-agenda-clockreport-mode org-agenda-start-with-clockreport-mode))
   (add-to-invisibility-spec '(org-filtered))
-  (add-to-invisibility-spec '(org-link))
+  (org-fold-core-initialize `(,org-link--description-folding-spec
+  ,org-link--link-folding-spec))
   (easy-menu-change
'("Agenda") "Agenda Files"
(append
diff --git a/lisp/org.el b/lisp/org.el
index ca4973bc3..f6709f4cc 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -4563,9 +4563,16 @@ (define-derived-mode org-mode outline-mode "Org"
   (setq-local org-mode-loading t)
   (org-load-modules-maybe)
   (org-install-agenda-files-menu)
-  (when org-link-descriptive (add-to-invisibility-spec '(org-link)))
+  (when (and org-link-descriptive
+ (eq org-fold-core-style 'overlays))
+(add-to-invisibility-spec '(org-link)))
+  (org-fold-initialize (or (and (stringp org-ellipsis) (not (equal "" 
org-ellipsis)) org-ellipsis)
+"..."))
   (make-local-variable 'org-link-descriptive)
-  (add-to-invisibility-spec '(org-hide-block . t))
+  (wh

[PATCH v2 26/38] Rename remaining org-force-cycle-archived

[Anders Johansson]

---
 lisp/org-cycle.el | 2 +-
 lisp/org-keys.el  | 4 ++--
 lisp/org.el   | 1 +
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/lisp/org-cycle.el b/lisp/org-cycle.el
index d2fcc356c..df0a3761a 100644
--- a/lisp/org-cycle.el
+++ b/lisp/org-cycle.el
@@ -811,7 +811,7 @@ (defun org-cycle-hide-archived-subtrees (state)
  (org-get-tags nil 'local)))
(message "%s" (substitute-command-keys
   "Subtree is archived and stays closed.  Use \
-`\\[org-force-cycle-archived]' to cycle it anyway."))
+`\\[org-cycle-force-archived]' to cycle it anyway."))
 
 (provide 'org-cycle)
 
diff --git a/lisp/org-keys.el b/lisp/org-keys.el
index e6b8ff459..782ffa871 100644
--- a/lisp/org-keys.el
+++ b/lisp/org-keys.el
@@ -94,7 +94,7 @@ (declare-function org-feed-update-all "org" ())
 (declare-function org-fill-paragraph "org" (&optional justify region))
 (declare-function org-find-file-at-mouse "org" (ev))
 (declare-function org-footnote-action "org" (&optional special))
-(declare-function org-force-cycle-archived "org-cycle" ())
+(declare-function org-cycle-force-archived "org-cycle" ())
 (declare-function org-force-self-insert "org" (n))
 (declare-function org-forward-element "org" ())
 (declare-function org-forward-heading-same-level "org" (arg &optional 
invisible-ok))
@@ -444,7 +444,7 @@ (org-defkey org-mode-map (kbd "C-c C-x") 
(make-sparse-keymap))
 
  TAB key with modifiers
 (org-defkey org-mode-map (kbd "TAB") #'org-cycle)
-(org-defkey org-mode-map (kbd "C-c C-") #'org-force-cycle-archived)
+(org-defkey org-mode-map (kbd "C-c C-") #'org-cycle-force-archived)
 ;; Override text-mode binding to expose `complete-symbol' for
 ;; pcomplete functionality.
 (org-defkey org-mode-map (kbd "M-TAB") nil)
diff --git a/lisp/org.el b/lisp/org.el
index ff17bf001..796a05fad 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -106,6 +106,7 @@ (defalias 'org-global-cycle #'org-cycle-global)
 (defalias 'org-overview #'org-cycle-overview)
 (defalias 'org-content #'org-cycle-content)
 (defalias 'org-reveal #'org-fold-reveal)
+(defalias 'org-force-cycle-archived #'org-cycle-force-archived)
 
 ;; `org-outline-regexp' ought to be a defconst but is let-bound in
 ;; some places -- e.g. see the macro `org-with-limited-levels'.
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 14/38] Support extra org-fold optimisations for huge buffers

[Ihor Radchenko]

---
 lisp/org.el | 26 ++
 1 file changed, 18 insertions(+), 8 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index ca0a99681..f5936b67c 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5788,6 +5788,7 @@ (defun org-unfontify-region (beg end &optional 
_maybe_loudly)
'(mouse-face t keymap t org-linked-text t
 invisible t intangible t
 org-emphasis t))
+(org-fold-core-update-optimisation beg end)
 (org-remove-font-lock-display-properties beg end)))
 
 (defconst org-script-display  '(((raise -0.3) (height 0.7))
@@ -6158,7 +6159,11 @@ (defun org-get-heading (&optional no-tags no-todo 
no-priority no-comment)
   (org-back-to-heading t)
   (let ((case-fold-search nil))
(looking-at org-complex-heading-regexp)
-   (let ((todo (and (not no-todo) (match-string 2)))
+;; When using `org-fold-core--optimise-for-huge-buffers',
+;; returned text may be invisible.  Clear it up.
+(save-match-data
+  (org-fold-core-remove-optimisation (match-beginning 0) (match-end 
0)))
+(let ((todo (and (not no-todo) (match-string 2)))
  (priority (and (not no-priority) (match-string 3)))
  (headline (pcase (match-string 4)
  (`nil "")
@@ -6169,6 +6174,8 @@ (defun org-get-heading (&optional no-tags no-todo 
no-priority no-comment)
"" h))
  (h h)))
  (tags (and (not no-tags) (match-string 5
+  ;; Restore cleared optimisation.
+  (org-fold-core-update-optimisation (match-beginning 0) (match-end 0))
  (mapconcat #'identity
 (delq nil (list todo priority headline tags))
 " "))
@@ -6185,18 +6192,21 @@ (defun org-heading-components ()
   (save-excursion
 (org-back-to-heading t)
 (when (let (case-fold-search) (looking-at org-complex-heading-regexp))
-  (list (length (match-string 1))
-   (org-reduced-level (length (match-string 1)))
-   (match-string-no-properties 2)
-   (and (match-end 3) (aref (match-string 3) 2))
-   (match-string-no-properties 4)
-   (match-string-no-properties 5)
+  (org-fold-core-remove-optimisation (match-beginning 0) (match-end 0))
+  (prog1
+  (list (length (match-string 1))
+   (org-reduced-level (length (match-string 1)))
+   (match-string-no-properties 2)
+   (and (match-end 3) (aref (match-string 3) 2))
+   (match-string-no-properties 4)
+   (match-string-no-properties 5))
+(org-fold-core-update-optimisation (match-beginning 0) (match-end 
0))
 
 (defun org-get-entry ()
   "Get the entry text, after heading, entire subtree."
   (save-excursion
 (org-back-to-heading t)
-(buffer-substring (point-at-bol 2) (org-end-of-subtree t
+(filter-buffer-substring (point-at-bol 2) (org-end-of-subtree t
 
 (defun org-edit-headline (&optional heading)
   "Edit the current headline.
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 11/38] Implement overlay- and text-property-based versions of some functions

[Ihor Radchenko]

---
 lisp/org-element.el|  54 -
 lisp/org-fold.el   |   5 +-
 lisp/org-inlinetask.el |  26 ++-
 lisp/org-list.el   |  74 ++-
 lisp/org-macs.el   |  54 -
 lisp/org.el| 469 +
 6 files changed, 585 insertions(+), 97 deletions(-)

diff --git a/lisp/org-element.el b/lisp/org-element.el
index f627dd4ea..203695c71 100644
--- a/lisp/org-element.el
+++ b/lisp/org-element.el
@@ -7912,7 +7912,7 @@ (defun org-element-nested-p (elem-A elem-B)
 (or (and (>= beg-A beg-B) (<= end-A end-B))
(and (>= beg-B beg-A) (<= end-B end-A)
 
-(defun org-element-swap-A-B (elem-A elem-B)
+(defun org-element-swap-A-B--overlays (elem-A elem-B)
   "Swap elements ELEM-A and ELEM-B.
 Assume ELEM-B is after ELEM-A in the buffer.  Leave point at the
 end of ELEM-A."
@@ -7980,6 +7980,58 @@ (defun org-element-swap-A-B (elem-A elem-B)
(dolist (o (cdr overlays))
  (move-overlay (car o) (- (nth 1 o) offset) (- (nth 2 o) offset
   (goto-char (org-element-property :end elem-B)
+(defun org-element-swap-A-B--text-properties (elem-A elem-B)
+  "Swap elements ELEM-A and ELEM-B.
+Assume ELEM-B is after ELEM-A in the buffer.  Leave point at the
+end of ELEM-A."
+  (goto-char (org-element-property :begin elem-A))
+  ;; There are two special cases when an element doesn't start at bol:
+  ;; the first paragraph in an item or in a footnote definition.
+  (let ((specialp (not (bolp
+;; Only a paragraph without any affiliated keyword can be moved at
+;; ELEM-A position in such a situation.  Note that the case of
+;; a footnote definition is impossible: it cannot contain two
+;; paragraphs in a row because it cannot contain a blank line.
+(when (and specialp
+  (or (not (eq (org-element-type elem-B) 'paragraph))
+  (/= (org-element-property :begin elem-B)
+  (org-element-property :contents-begin elem-B
+  (error "Cannot swap elements"))
+;; In a special situation, ELEM-A will have no indentation.  We'll
+;; give it ELEM-B's (which will in, in turn, have no indentation).
+(org-fold-core-ignore-modifications ;; Preserve folding state
+(let* ((ind-B (when specialp
+   (goto-char (org-element-property :begin elem-B))
+   (current-indentation)))
+  (beg-A (org-element-property :begin elem-A))
+  (end-A (save-excursion
+   (goto-char (org-element-property :end elem-A))
+   (skip-chars-backward " \r\t\n")
+   (point-at-eol)))
+  (beg-B (org-element-property :begin elem-B))
+  (end-B (save-excursion
+   (goto-char (org-element-property :end elem-B))
+   (skip-chars-backward " \r\t\n")
+   (point-at-eol)))
+  ;; Get contents.
+  (body-A (buffer-substring beg-A end-A))
+  (body-B (delete-and-extract-region beg-B end-B)))
+  (goto-char beg-B)
+  (when specialp
+   (setq body-B (replace-regexp-in-string "\\`[ \t]*" "" body-B))
+   (indent-to-column ind-B))
+  (insert body-A)
+ (goto-char beg-A)
+ (delete-region beg-A end-A)
+ (insert body-B)
+  (goto-char (org-element-property :end elem-B))
+(defsubst org-element-swap-A-B (elem-A elem-B)
+  "Swap elements ELEM-A and ELEM-B.
+Assume ELEM-B is after ELEM-A in the buffer.  Leave point at the
+end of ELEM-A."
+  (if (eq org-fold-core-style 'text-properties)
+  (org-element-swap-A-B--text-properties elem-A elem-B)
+(org-element-swap-A-B--overlays elem-A elem-B)))
 
 
 (provide 'org-element)
diff --git a/lisp/org-fold.el b/lisp/org-fold.el
index 52717fd86..e48a528bf 100644
--- a/lisp/org-fold.el
+++ b/lisp/org-fold.el
@@ -53,10 +53,7 @@ (defvar org-drawer-regexp)
 (defvar org-property-end-re)
 (defvar org-link-descriptive)
 (defvar org-outline-regexp-bol)
-(defvar org-custom-properties-hidden-p)
 (defvar org-archive-tag)
-
-;; Needed for overlays only
 (defvar org-custom-properties-overlays)
 
 (declare-function isearch-filter-visible "isearch" (beg end))
@@ -1101,7 +1098,7 @@ (defun 
org-fold-check-before-invisible-edit--text-properties (kind)
   (when (or invisible-at-point invisible-before-point)
(when (eq org-fold-catch-invisible-edits 'error)
  (user-error "Editing in invisible areas is prohibited, make them 
visible first"))
-   (if (and org-custom-properties-hidden-p
+   (if (and org-custom-properties-overlays
 (y-or-n-p "Display invisible properties in this buffer? "))
(org-toggle-custom-properties-visibility)
  ;; Make the area visible
diff --git a/lisp/org-inlinetask.el b/lisp/org-inlinetask.el
index 581370bb5..a63704a05 100644
--- a/lisp/org-inlinetask.el
+++ b/lisp/org-inlinetask.el
@@ -305,7 +305,22 @@ (defun org

[PATCH v2 29/38] org-string-width: Work around `window-pixel-width' bug in old Emacs

[Ihor Radchenko]

---
 lisp/org-macs.el | 188 ---
 1 file changed, 129 insertions(+), 59 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index e56a234d3..a1d514d50 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -893,73 +893,143 @@ (defun org-split-string (string &optional separators)
  results   ;skip trailing separator
(cons (substring string i) results)))
 
+(defun org--string-from-props (s property beg end)
+  "Return the visible part of string S.
+Visible part is determined according to text PROPERTY, which is
+either `invisible' or `display'.  BEG and END are 0-indices
+delimiting S."
+  (let ((width 0)
+   (cursor beg))
+(while (setq beg (text-property-not-all beg end property nil s))
+  (let* ((next (next-single-property-change beg property s end))
+(props (text-properties-at beg s))
+(spec (plist-get props property))
+(value
+ (pcase property
+   (`invisible
+;; If `invisible' property in PROPS means text is to
+;; be invisible, return 0.  Otherwise return nil so
+;; as to resume search.
+(and (or (eq t buffer-invisibility-spec)
+ (assoc-string spec buffer-invisibility-spec))
+ 0))
+   (`display
+(pcase spec
+  (`nil nil)
+  (`(space . ,props)
+   (let ((width (plist-get props :width)))
+ (and (wholenump width) width)))
+  (`(image . ,_)
+(and (fboundp 'image-size)
+ (ceiling (car (image-size spec)
+  ((pred stringp)
+   ;; Displayed string could contain invisible parts,
+   ;; but no nested display.
+   (org--string-from-props spec 'invisible 0 (length spec)))
+  (_
+   ;; Un-handled `display' value.  Ignore it.
+   ;; Consider the original string instead.
+   nil)))
+   (_ (error "Unknown property: %S" property)
+   (when value
+ (cl-incf width
+  ;; When looking for `display' parts, we still need
+  ;; to look for `invisible' property elsewhere.
+  (+ (cond ((eq property 'display)
+(org--string-from-props s 'invisible cursor beg))
+   ((= cursor beg) 0)
+   (t (string-width (substring s cursor beg
+ value))
+ (setq cursor next))
+   (setq beg next)))
+(+ width
+   ;; Look for `invisible' property in the last part of the
+   ;; string.  See above.
+   (cond ((eq property 'display)
+ (org--string-from-props s 'invisible cursor end))
+((= cursor end) 0)
+(t (string-width (substring s cursor end)))
+
+(defun org--string-width-1 (string)
+  "Return width of STRING when displayed in the current buffer.
+Unlike `string-width', this function takes into consideration
+`invisible' and `display' text properties.  It supports the
+latter in a limited way, mostly for combinations used in Org.
+Results may be off sometimes if it cannot handle a given
+`display' value."
+  (org--string-from-props string 'display 0 (length string)))
+
 (defun org-string-width (string &optional pixels)
   "Return width of STRING when displayed in the current buffer.
 Return width in pixels when PIXELS is non-nil."
-  ;; Wrap/line prefix will make `window-text-pizel-size' return too
-  ;; large value including the prefix.
-  ;; Face should be removed to make sure that all the string symbols
-  ;; are using default face with constant width.  Constant char width
-  ;; is critical to get right string width from pixel width.
-  (remove-text-properties 0 (length string)
-  '(wrap-prefix t line-prefix t face t)
-  string)
-  (let (;; We need to remove the folds to make sure that folded table
-;; alignment is not messed up.
-(current-invisibility-spec
- (or (and (not (listp buffer-invisibility-spec))
-  buffer-invisibility-spec)
- (let (result)
-   (dolist (el buffer-invisibility-spec)
- (unless (or (memq el
-   '(org-fold-drawer
- org-fold-block
- org-fold-outline))
- (and (listp el)
-  (memq (car el)
-'(org-fold-drawer
-  org-fold-block
-  org-fold-outline
-   (push el result)))
-   result)))
-   

[PATCH v2 13/38] Fix subtle differences between overlays and invisible text properties

[Ihor Radchenko]

* lisp/org-clock.el (org-clock-in):
(org-clock-find-position):
(org-clock-out):
* lisp/org.el (org-add-planning-info):
(org-scan-tags):
(org-global-tags-completion-table):
(org-make-tags-matcher):
(org-tags-expand):
(org--property-local-values):
(org-read-date-analyze):
(org-revert-all-org-buffers):
(org-beginning-of-line): Make sure that we inherit invisible state
when inserting text.
(org-sort-entries): Preserve invisible state after replace-match.

(org-log-beginning): Do not try to move by visible lines.

* lisp/org-macs.el (org-preserve-local-variables): Do not try to
preserve overlays.
* lisp/ox.el (org-export--generate-copy-script): Preserve folding
properties in export buffer.
* testing/lisp/test-ob.el (test-ob/preserve-results-indentation): Fix
test failure.
* testing/lisp/test-org.el (test-org/meta-return):
(test-org/custom-properties): Use new folding.
---
 lisp/org-clock.el| 116 
 lisp/org-macs.el |  12 +-
 lisp/org.el  | 560 ---
 lisp/ox.el   |   4 +-
 testing/lisp/test-ob.el  |  12 +-
 testing/lisp/test-org.el |   3 +
 6 files changed, 367 insertions(+), 340 deletions(-)

diff --git a/lisp/org-clock.el b/lisp/org-clock.el
index 583b30237..ec87aaf8a 100644
--- a/lisp/org-clock.el
+++ b/lisp/org-clock.el
@@ -1373,14 +1373,14 @@ (defun org-clock-in (&optional select start-time)
   (sit-for 2)
   (throw 'abort nil))
  (t
-  (insert-before-markers "\n")
+  (insert-before-markers-and-inherit "\n")
   (backward-char 1)
   (when (and (save-excursion
(end-of-line 0)
(org-in-item-p)))
 (beginning-of-line 1)
 (indent-line-to (max 0 (- (current-indentation) 2
-  (insert org-clock-string " ")
+  (insert-and-inherit org-clock-string " ")
   (setq org-clock-effort (org-entry-get (point) org-effort-property))
   (setq org-clock-total-time (org-clock-sum-current-item
   (org-clock-get-sum-start)))
@@ -1581,19 +1581,23 @@ (defun org-clock-find-position (find-unclosed)
  count (1+ count))
(cond
 ((null positions)
- ;; Skip planning line and property drawer, if any.
- (org-end-of-meta-data)
- (unless (bolp) (insert "\n"))
- ;; Create a new drawer if necessary.
- (when (and org-clock-into-drawer
-(or (not (wholenump org-clock-into-drawer))
-(< org-clock-into-drawer 2)))
-   (let ((beg (point)))
- (insert ":" drawer ":\n:END:\n")
- (org-indent-region beg (point))
- (org-flag-region
-  (line-end-position -1) (1- (point)) t 'outline)
- (forward-line -1
+  (org-fold-core-ignore-modifications
+ ;; Skip planning line and property drawer, if any.
+ (org-end-of-meta-data)
+   (unless (bolp) (insert-and-inherit "\n"))
+   ;; Create a new drawer if necessary.
+   (when (and org-clock-into-drawer
+  (or (not (wholenump org-clock-into-drawer))
+  (< org-clock-into-drawer 2)))
+ (let ((beg (point)))
+   (insert-and-inherit ":" drawer ":\n:END:\n")
+   (org-indent-region beg (point))
+(if (eq org-fold-core-style 'text-properties)
+   (org-fold-region
+(line-end-position -1) (1- (point)) t 'drawer)
+  (org-fold-region
+  (line-end-position -1) (1- (point)) t 'outline))
+   (forward-line -1)
 ;; When a clock drawer needs to be created because of the
 ;; number of clock items or simply if it is missing, collect
 ;; all clocks in the section and wrap them within the drawer.
@@ -1602,28 +1606,29 @@ (defun org-clock-find-position (find-unclosed)
drawer)
  ;; Skip planning line and property drawer, if any.
  (org-end-of-meta-data)
- (let ((beg (point)))
-   (insert
-(mapconcat
- (lambda (p)
-   (save-excursion
- (goto-char p)
- (org-trim (delete-and-extract-region
-(save-excursion (skip-chars-backward " \r\t\n")
-(line-beginning-position 2))
-(line-beginning-position 2)
- positions "\n")
-"\n:END:\n")
-   (let ((end (point-marker)))
- (goto-char beg)
- (save-excursion (insert ":" drawer ":\n"))
- (org-flag-region (line-end-position) (1- end) t 'outline)
- (org-indent-region (point) end)
- (forward-line)
- (unless org-log-states-order-reversed
-   (goto-char end)
-   (beg

[PATCH v2 17/38] org-compat: Work around some third-party packages using outline-* functions

[Ihor Radchenko]

---
 lisp/org-compat.el | 72 +-
 1 file changed, 71 insertions(+), 1 deletion(-)

diff --git a/lisp/org-compat.el b/lisp/org-compat.el
index 3e8f49f0a..f599e246e 100644
--- a/lisp/org-compat.el
+++ b/lisp/org-compat.el
@@ -1330,11 +1330,81 @@ (defvar session-globals-exclude)
 (eval-after-load 'session
   '(add-to-list 'session-globals-exclude 'org-mark-ring))
 
+ outline-mode
+
+;; Folding in outline-mode is not compatible with org-mode folding
+;; anymore. Working around to avoid breakage of external packages
+;; assuming the compatibility.
+(defadvice outline-flag-region (around outline-flag-region@fix-for-org-fold 
(from to flag) activate)
+  "Run `org-fold-region' when in org-mode."
+  (if (eq major-mode 'org-mode)
+  (setq ad-return-value (org-fold-region (max from (point-min)) (min to 
(point-max)) flag 'headline))
+ad-do-it))
+
+(defadvice outline-next-visible-heading (around 
outline-next-visible-heading@fix-for-org-fold (arg) activate)
+  "Run `org-next-visible-heading' when in org-mode."
+  (interactive "p")
+  (if (eq major-mode 'org-mode)
+  (setq ad-return-value (org-next-visible-heading arg))
+ad-do-it))
+
+(defadvice outline-back-to-heading (around 
outline-back-to-heading@fix-for-org-fold (&optional invisible-ok) activate)
+  "Run `org-back-to-heading' when in org-mode."
+  (if (eq major-mode 'org-mode)
+  (setq ad-return-value
+(progn
+  (beginning-of-line)
+  (or (org-at-heading-p (not invisible-ok))
+  (let (found)
+   (save-excursion
+ (while (not found)
+   (or (re-search-backward (concat "^\\(?:" outline-regexp 
"\\)")
+   nil t)
+(signal 'outline-before-first-heading nil))
+   (setq found (and (or invisible-ok (not 
(org-fold-folded-p)))
+(point)
+   (goto-char found)
+   found
+ad-do-it))
+
+(defadvice outline-on-heading-p (around outline-on-heading-p@fix-for-org-fold 
(&optional invisible-ok) activate)
+  "Run `org-at-heading-p' when in org-mode."
+  (if (eq major-mode 'org-mode)
+  (setq ad-return-value (org-at-heading-p (not invisible-ok)))
+ad-do-it))
+
+(defadvice outline-hide-sublevels (around 
outline-hide-sublevels@fix-for-org-fold (levels) activate)
+  "Run `org-fold-hide-sublevels' when in org-mode."
+  (interactive (list
+   (cond
+(current-prefix-arg (prefix-numeric-value current-prefix-arg))
+((save-excursion (beginning-of-line)
+ (looking-at outline-regexp))
+ (funcall outline-level))
+(t 1
+  (if (eq major-mode 'org-mode)
+  (setq ad-return-value (org-fold-hide-sublevels levels))
+ad-do-it))
+
+(defadvice outline-toggle-children (around 
outline-toggle-children@fix-for-org-fold () activate)
+  "Run `org-fold-hide-sublevels' when in org-mode."
+  (interactive)
+  (if (eq major-mode 'org-mode)
+  (setq ad-return-value
+(save-excursion
+  (org-back-to-heading)
+  (if (not (org-fold-folded-p (line-end-position)))
+  (org-fold-hide-subtree)
+(org-fold-show-children)
+(org-fold-show-entry
+ad-do-it))
+
+;; TODO: outline-headers-as-kill
+
  Speed commands
 
 (make-obsolete-variable 'org-speed-commands-user
 "configure `org-speed-commands' instead." "9.5")
-
 (provide 'org-compat)
 
 ;; Local variables:
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 16/38] Obsolete old function names that are now in org-fold---

[Ihor Radchenko]

---
 lisp/org-compat.el | 88 ++
 1 file changed, 88 insertions(+)

diff --git a/lisp/org-compat.el b/lisp/org-compat.el
index ed2ae62f4..3e8f49f0a 100644
--- a/lisp/org-compat.el
+++ b/lisp/org-compat.el
@@ -267,6 +267,11 @@ (define-obsolete-function-alias 'org-propertize 
'propertize "9.0")
 (define-obsolete-function-alias 'org-select-frame-set-input-focus 
'select-frame-set-input-focus "9.0")
 (define-obsolete-function-alias 'org-file-remote-p 'file-remote-p "9.2")
 
+(define-obsolete-function-alias 'org-show-context 'org-fold-show-context "9.6")
+(define-obsolete-function-alias 'org-show-entry 'org-fold-show-entry "9.6")
+(define-obsolete-function-alias 'org-show-children 'org-fold-show-children 
"9.6")
+
+
 (defmacro org-re (s)
   "Replace posix classes in regular expression S."
   (declare (debug (form))
@@ -376,6 +381,80 @@ (define-obsolete-function-alias 'org-toggle-latex-fragment 
'org-latex-preview
 (define-obsolete-function-alias 'org-remove-latex-fragment-image-overlays
   'org-clear-latex-preview "9.3")
 
+(define-obsolete-function-alias 'org-hide-archived-subtrees
+  'org-fold-hide-archived-subtrees "9.6")
+
+(define-obsolete-function-alias 'org-flag-region
+  'org-fold-region "9.6")
+
+(define-obsolete-function-alias 'org-flag-subtree
+  'org-fold-subtree "9.6")
+
+(define-obsolete-function-alias 'org-hide-entry
+  'org-fold-hide-entry "9.6")
+
+(define-obsolete-function-alias 'org-show-subtree
+  'org-fold-show-subtree "9.6")
+
+(define-obsolete-function-alias 'org--hide-wrapper-toggle
+  'org-fold--hide-wrapper-toggle "9.6")
+
+(define-obsolete-function-alias 'org-hide-block-toggle
+  'org-fold-hide-block-toggle "9.6")
+
+(define-obsolete-function-alias 'org-hide-drawer-toggle
+  'org-fold-hide-drawer-toggle "9.6")
+
+(define-obsolete-function-alias 'org--hide-drawers
+  'org-fold--hide-drawers "9.6")
+
+(define-obsolete-function-alias 'org-hide-block-all
+  'org-fold-hide-block-all "9.6")
+
+(define-obsolete-function-alias 'org-hide-drawer-all
+  'org-fold-hide-drawer-all "9.6")
+
+(define-obsolete-function-alias 'org-show-all
+  'org-fold-show-all "9.6")
+
+(define-obsolete-function-alias 'org-set-startup-visibility
+  'org-cycle-set-startup-visibility "9.6")
+
+(define-obsolete-function-alias 'org-show-set-visibility
+  'org-fold-show-set-visibility "9.6")
+
+(define-obsolete-function-alias 'org-check-before-invisible-edit
+  'org-fold-check-before-invisible-edit "9.6")
+
+(define-obsolete-function-alias 'org-flag-above-first-heading
+  'org-fold-flag-above-first-heading "9.6")
+
+(define-obsolete-function-alias 'org-show-branches-buffer
+  'org-fold-show-branches-buffer "9.6")
+
+(define-obsolete-function-alias 'org-show-siblings
+  'org-fold-show-siblings "9.6")
+
+(define-obsolete-function-alias 'org-show-hidden-entry
+  'org-fold-show-hidden-entry "9.6")
+
+(define-obsolete-function-alias 'org-flag-heading
+  'org-fold-heading "9.6")
+
+(define-obsolete-function-alias 'org-set-startup-visibility
+  'org-cycle-set-startup-visibility "9.6")
+
+(define-obsolete-function-alias 'org-set-visibility-according-to-property
+  'org-cycle-set-visibility-according-to-property "9.6")
+
+(define-obsolete-variable-alias 'org-scroll-position-to-restore
+  'org-cycle-scroll-position-to-restore "9.6")
+(define-obsolete-function-alias 'org-optimize-window-after-visibility-change
+  'org-cycle-optimize-window-after-visibility-change "9.6")
+
+(define-obsolete-function-alias 'org-force-cycle-archived
+  'org-cycle-force-archived "9.6")
+
 (define-obsolete-variable-alias 'org-attach-directory
   'org-attach-id-dir "9.3")
 (make-obsolete 'org-attach-store-link "No longer used" "9.4")
@@ -383,6 +462,15 @@ (make-obsolete 'org-attach-expand-link "No longer used" 
"9.4")
 
 (define-obsolete-function-alias 'org-file-url-p 'org-url-p "9.5")
 
+(define-obsolete-variable-alias 'org-show-context-detail
+  'org-fold-show-context-detail "9.6")
+
+(define-obsolete-variable-alias 'org-catch-invisible-edits
+  'org-fold-catch-invisible-edits "9.6")
+
+(define-obsolete-variable-alias 'org-reveal-start-hook
+  'org-fold-reveal-start-hook "9.6")
+(define-obsolete-function-alias 'org-file-url-p 'org-url-p "9.6")
 (defun org-in-fixed-width-region-p ()
   "Non-nil if point in a fixed-width region."
   (save-match-data
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 36/38] org-macs.el: Fix fontification checks take 2---

[Ihor Radchenko]

---
 lisp/org-macs.el | 12 
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index c71bb4094..9e24e315a 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -1192,7 +1192,8 @@ (defun org-buffer-substring-fontified (beg end)
   "Return fontified region between BEG and END."
   (when (bound-and-true-p jit-lock-mode)
 (org-with-forced-fontification
-(when (text-property-not-all beg end 'org-fold-core-fontified t)
+(when (or (text-property-not-all beg end 'org-fold-core-fontified t)
+  (text-property-not-all beg end 'fontified t))
   (save-match-data (font-lock-fontify-region beg end)
   (buffer-substring beg end))
 
@@ -1201,9 +1202,12 @@ (defun org-looking-at-fontified (re)
   (prog1 (looking-at re)
 (when (bound-and-true-p jit-lock-mode)
   (org-with-forced-fontification
-  (when (text-property-not-all
- (match-beginning 0) (match-end 0)
- 'org-fold-core-fontified t)
+  (when (or (text-property-not-all
+ (match-beginning 0) (match-end 0)
+ 'org-fold-core-fontified t)
+(text-property-not-all
+ (match-beginning 0) (match-end 0)
+ 'fontified t))
 (save-match-data
   (font-lock-fontify-region (match-beginning 0)
 (match-end 0
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 20/38] Add org-fold-related tests---

[Ihor Radchenko]

---
 testing/lisp/test-ol.el   |  24 +
 testing/lisp/test-org-list.el |  73 ++
 testing/lisp/test-org.el  | 177 +++---
 3 files changed, 238 insertions(+), 36 deletions(-)

diff --git a/testing/lisp/test-ol.el b/testing/lisp/test-ol.el
index ddcc570b3..343631623 100644
--- a/testing/lisp/test-ol.el
+++ b/testing/lisp/test-ol.el
@@ -50,6 +50,30 @@ (ert-deftest test-ol/encode-url-with-escaped-char ()
 (org-link-encode "http://some.host.com/form?&id=blah%2Bblah25";
  '(?\s ?\[ ?\] ?%))
 
+(ert-deftest test-ol/org-toggle-link-display ()
+  "Make sure that `org-toggle-link-display' is working.
+See https://github.com/yantar92/org/issues/4.";
+  (dolist (org-link-descriptive '(nil t))
+(org-test-with-temp-text "* Org link test
+[[https://example.com][A link to a site]]"
+  (dotimes (_ 2)
+(goto-char 1)
+(re-search-forward "\\[")
+(should-not (xor org-link-descriptive (org-invisible-p)))
+(re-search-forward "example")
+(should-not (xor org-link-descriptive (org-invisible-p)))
+(re-search-forward "com")
+(should-not (xor org-link-descriptive (org-invisible-p)))
+(re-search-forward "]")
+(should-not (xor org-link-descriptive (org-invisible-p)))
+(re-search-forward "\\[")
+(should-not (org-invisible-p))
+(re-search-forward "link")
+(should-not (org-invisible-p))
+(re-search-forward "]")
+(should-not (xor org-link-descriptive (org-invisible-p)))
+(org-toggle-link-display)
+
 
 ;;; Escape and Unescape Links
 
diff --git a/testing/lisp/test-org-list.el b/testing/lisp/test-org-list.el
index a9490692e..bc8faa672 100644
--- a/testing/lisp/test-org-list.el
+++ b/testing/lisp/test-org-list.el
@@ -580,22 +580,40 @@ (ert-deftest test-org-list/move-item-down ()
 (let ((org-list-use-circular-motion t)) (org-move-item-down))
 (buffer-string
   ;; Preserve item visibility.
+  (should
+   (equal
+(make-list 2 'org-fold-outline)
+(let ((org-fold-core-style 'text-properties))
+  (org-test-with-temp-text
+   "* Headline\n- item 1\n  body 1\n- item 2\n  body 2"
+   (let ((org-cycle-include-plain-lists t))
+ (org-cycle)
+ (search-forward "- item 2")
+ (org-cycle))
+   (search-backward "- item 1")
+   (org-move-item-down)
+   (forward-line)
+   (list (org-fold-get-folding-spec)
+(progn
+  (search-backward " body 2")
+  (org-fold-get-folding-spec)))
   (should
(equal
 '(outline outline)
-(org-test-with-temp-text
-   "* Headline\n- item 1\n  body 1\n- item 2\n  body 2"
-  (let ((org-cycle-include-plain-lists t))
-   (org-cycle)
-   (search-forward "- item 2")
-   (org-cycle))
-  (search-backward "- item 1")
-  (org-move-item-down)
-  (forward-line)
-  (list (org-invisible-p2)
-   (progn
- (search-backward " body 2")
- (org-invisible-p2))
+(let ((org-fold-core-style 'overlays))
+  (org-test-with-temp-text
+   "* Headline\n- item 1\n  body 1\n- item 2\n  body 2"
+   (let ((org-cycle-include-plain-lists t))
+ (org-cycle)
+ (search-forward "- item 2")
+ (org-cycle))
+   (search-backward "- item 1")
+   (org-move-item-down)
+   (forward-line)
+   (list (org-invisible-p2)
+(progn
+  (search-backward " body 2")
+  (org-invisible-p2)))
   ;; Preserve children visibility.
   (org-test-with-temp-text "* Headline
 - item 1
@@ -869,17 +887,30 @@ (ert-deftest test-org-list/insert-item ()
(org-insert-item)
(buffer-string
   ;; Preserve list visibility when inserting an item.
+  (should
+   (equal
+`(org-fold-outline org-fold-outline)
+(let ((org-fold-core-style 'text-properties))
+  (org-test-with-temp-text "- A\n  - B\n- C\n  - D"
+   (let ((org-cycle-include-plain-lists t))
+(org-cycle)
+(forward-line 2)
+(org-cycle)
+(org-insert-item)
+(list (org-fold-get-folding-spec nil 
(line-beginning-position 0))
+  (org-fold-get-folding-spec nil 
(line-end-position 2
   (should
(equal
 '(outline outline)
-(org-test-with-temp-text "- A\n  - B\n- C\n  - D"
-  (let ((org-cycle-include-plain-lists t))
-   (org-cycle)
-   (forward-line 2)
-   (org-cycle)
-   (org-insert-item)
-   (list (get-char-property (line-beginning-position 0) 'invisible)
- (get-char-property (line-end-position 2) 'invisible))
+(let ((org-fold-core-style 'overlays))
+  (org-test-with-temp-text "- A\n  - B\n- C\n  - D"
+  

[PATCH v2 22/38] ORG-NEWS: Add list of changes---

[Ihor Radchenko]

---
 etc/ORG-NEWS | 104 +++
 1 file changed, 104 insertions(+)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 4f1309ecc..5b934fe96 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -40,6 +40,105 @@ The cache state is saved between Emacs sessions.  Enabled 
by default.
 The cache persistence can be controlled via
 ~org-element-cache-persistent~.
 
+*** Users experiencing performance issues can use new folding backend
+
+The old folding backend used in Org is poorly scalable when the file
+size increases beyond few Mbs.  The symptoms usually include slow
+cursor motion, especially in long-running Emacs sessions.
+
+A new optimised folding backend is now available, and enabled by
+default.  To disable it, put the following to the Emacs config *before*
+loading Org:
+
+#+begin_src emacs-lisp
+(setq org-fold-core-style 'overlays)
+#+end_src
+
+Even more performance optimisation can be enabled by customising
+=org-fold-core--optimise-for-huge-buffers=.  However, this option may
+be dangerous.  Please, read the variable docstring carefully to
+understand the possible consequences.
+
+When =org-fold-core-style= is set to =text-properties=, several new
+features will become available and several notable changes will happen
+to the Org behaviour.  The new features and changes are listed below.
+
+ Hidden parts of the links can now be searched and revealed during isearch
+
+In the past, hidden parts of the links could not be searched using
+isearch (=C-s=).  Now, they are searchable by default.  The hidden
+match is also revealed temporarily during isearch.
+
+To restore the old behaviour add the following core to your Emacs
+config:
+
+#+begin_src emacs-lisp
+(defun org-hidden-link-ignore-isearch ()
+  "Do not match hidden parts of links during isearch."
+  (org-fold-core-set-folding-spec-property 'org-link :isearch-open nil)
+  (org-fold-core-set-folding-spec-property 'org-link :isearch-ignore t))
+(add-hook 'org-mode-hook #'org-hidden-link-ignore-isearch)
+#+end_src
+
+See docstring of =org-fold-core--specs= to see more details about
+=:isearch-open= and =:isearch-ignore= properties.
+
+ =org-catch-invisible-edits= now works for hidden parts of the links and 
for emphasis markers
+
+In the past, user could edit invisible parts of the links and emphasis 
markers.  Now, the editing is respecting the value of 
=org-catch-invisible-edits=.
+
+Note that hidden parts of sub-/super-scripts are still not handled.
+
+ Breaking structure of folded elements automatically reveals the folded 
text
+
+In the past, the user could be left with unfoldable text after breaking the 
org structure.
+
+For example, if
+
+#+begin_src org
+:DRAWER:
+like this
+:END:
+#+end_src
+
+is folded and then edited into
+
+#+begin_src org
+DRAWER:
+like this
+:END:
+#+end_src
+The hidden text would not be revealed.
+
+Now, breaking structure of drawers, blocks, and headings automatically
+reveals the folded text.
+
+ Folding state of the drawers is now preserved when cycling headline 
visibility
+
+In the past drawers were folded every time a headline is unfolded.
+
+Now, it is not the case anymore.  The drawer folding state is
+preserved.  The initial folding state of all the drawers in buffer is
+set according to the startup visibility settings.
+
+To restore the old behaviour, add the following code to Emacs config:
+
+#+begin_src emacs-lisp
+(add-hook 'org-cycle-hook #'org-cycle-hide-drawers)
+#+end_src
+
+Note that old behaviour may cause performance issues when cycling
+headline visibility in large buffers.
+
+ =outline-*= functions may no longer work correctly in Org mode
+
+The new folding backend breaks some of the =outline-*= functions that
+rely on the details of visibility state implementation in
+=outline.el=.  The old Org folding backend was compatible with the
+=outline.el= folding, but it is not the case anymore with the new
+backend.  From now on, using =outline-*= functions is strongly
+discouraged when working with Org files.
+
 ** New features
 
 *** New library =org-persist.el= implements variable persistence across Emacs 
sessions
@@ -117,6 +216,11 @@ argument.
 This function is intended for us in the definition of a ~kbd~ macro in
 files that are exported to Texinfo.
 
+*** =org-at-heading-p= now recognises optional argument. Its meaning is 
inverted.
+
+=org-at-heading-p= now returns t by default on headings inside folds.
+Passing optional argument will produce the old behaviour.
+
 ** Miscellaneous
 
 *** Styles are customizable in ~biblatex~ citation processor
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 18/38] Move `org-buffer-list' to org-macs.el---

[Ihor Radchenko]

---
 lisp/org-macs.el | 38 ++
 lisp/org.el  | 38 --
 2 files changed, 38 insertions(+), 38 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 42a91781b..188168cdc 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -223,6 +223,44 @@ (defun org-fit-window-to-buffer (&optional window 
max-height min-height
  (shrink-window-if-larger-than-buffer window)))
   (or window (selected-window)))
 
+(defun org-buffer-list (&optional predicate exclude-tmp)
+  "Return a list of Org buffers.
+PREDICATE can be `export', `files' or `agenda'.
+
+export   restrict the list to Export buffers.
+filesrestrict the list to buffers visiting Org files.
+agenda   restrict the list to buffers visiting agenda files.
+
+If EXCLUDE-TMP is non-nil, ignore temporary buffers."
+  (let* ((bfn nil)
+(agenda-files (and (eq predicate 'agenda)
+   (mapcar 'file-truename (org-agenda-files t
+(filter
+ (cond
+  ((eq predicate 'files)
+   (lambda (b) (with-current-buffer b (derived-mode-p 'org-mode
+  ((eq predicate 'export)
+   (lambda (b) (string-match "\\*Org .*Export" (buffer-name b
+  ((eq predicate 'agenda)
+   (lambda (b)
+ (with-current-buffer b
+   (and (derived-mode-p 'org-mode)
+(setq bfn (buffer-file-name b))
+(member (file-truename bfn) agenda-files)
+  (t (lambda (b) (with-current-buffer b
+   (or (derived-mode-p 'org-mode)
+   (string-match "\\*Org .*Export"
+ (buffer-name b)
+(delq nil
+ (mapcar
+  (lambda(b)
+(if (and (funcall filter b)
+ (or (not exclude-tmp)
+ (not (string-match "tmp" (buffer-name b)
+b
+  nil))
+  (buffer-list)
+
 
 
 ;;; File
diff --git a/lisp/org.el b/lisp/org.el
index 2608865da..402ce3520 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -14748,44 +14748,6 @@ (defun org-switchb (&optional arg)
  (mapcar #'list (mapcar #'buffer-name blist))
  nil t
 
-(defun org-buffer-list (&optional predicate exclude-tmp)
-  "Return a list of Org buffers.
-PREDICATE can be `export', `files' or `agenda'.
-
-export   restrict the list to Export buffers.
-filesrestrict the list to buffers visiting Org files.
-agenda   restrict the list to buffers visiting agenda files.
-
-If EXCLUDE-TMP is non-nil, ignore temporary buffers."
-  (let* ((bfn nil)
-(agenda-files (and (eq predicate 'agenda)
-   (mapcar 'file-truename (org-agenda-files t
-(filter
- (cond
-  ((eq predicate 'files)
-   (lambda (b) (with-current-buffer b (derived-mode-p 'org-mode
-  ((eq predicate 'export)
-   (lambda (b) (string-match "\\*Org .*Export" (buffer-name b
-  ((eq predicate 'agenda)
-   (lambda (b)
- (with-current-buffer b
-   (and (derived-mode-p 'org-mode)
-(setq bfn (buffer-file-name b))
-(member (file-truename bfn) agenda-files)
-  (t (lambda (b) (with-current-buffer b
-   (or (derived-mode-p 'org-mode)
-   (string-match "\\*Org .*Export"
- (buffer-name b)
-(delq nil
- (mapcar
-  (lambda(b)
-(if (and (funcall filter b)
- (or (not exclude-tmp)
- (not (string-match "tmp" (buffer-name b)
-b
-  nil))
-  (buffer-list)
-
 (defun org-agenda-files (&optional unrestricted archives)
   "Get the list of agenda files.
 Optional UNRESTRICTED means return the full list even if a restriction
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 27/38] Fix org-fold--hide-drawers--overlays---

[Ihor Radchenko]

---
 lisp/org-fold.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org-fold.el b/lisp/org-fold.el
index e48a528bf..a16ee0f9b 100644
--- a/lisp/org-fold.el
+++ b/lisp/org-fold.el
@@ -714,7 +714,7 @@ (defun org-fold--hide-drawers--overlays (begin end)
   "Hide all drawers between BEGIN and END."
   (save-excursion
 (goto-char begin)
-(while (re-search-forward org-drawer-regexp end t)
+(while (and (< (point) end) (re-search-forward org-drawer-regexp end t))
   (let* ((pair (get-char-property-and-overlay (line-beginning-position)
   'invisible))
  (o (cdr-safe pair)))
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 23/38] Backport contributed commits---

[Ihor Radchenko]

---
 lisp/org-cycle.el | 2 +-
 lisp/org-fold-core.el | 2 +-
 lisp/org-keys.el  | 4 ++--
 lisp/org.el   | 6 ++
 4 files changed, 6 insertions(+), 8 deletions(-)

diff --git a/lisp/org-cycle.el b/lisp/org-cycle.el
index df0a3761a..d2fcc356c 100644
--- a/lisp/org-cycle.el
+++ b/lisp/org-cycle.el
@@ -811,7 +811,7 @@ (defun org-cycle-hide-archived-subtrees (state)
  (org-get-tags nil 'local)))
(message "%s" (substitute-command-keys
   "Subtree is archived and stays closed.  Use \
-`\\[org-cycle-force-archived]' to cycle it anyway."))
+`\\[org-force-cycle-archived]' to cycle it anyway."))
 
 (provide 'org-cycle)
 
diff --git a/lisp/org-fold-core.el b/lisp/org-fold-core.el
index 121c6b5c4..6ea374498 100644
--- a/lisp/org-fold-core.el
+++ b/lisp/org-fold-core.el
@@ -592,7 +592,7 @@ (defun org-fold-core--property-symbol-get-create (spec 
&optional buffer return-o
   (org-fold-core-cycle-over-indirect-buffers
   (push (current-buffer) bufs))
   (push buf bufs)
-  (delete-dups bufs)
+  (delete-duplicates bufs)
 ;; Copy all the old folding properties to preserve the folding 
state
 (with-silent-modifications
   (dolist (old-prop (cdr (assq 'invisible 
char-property-alias-alist)))
diff --git a/lisp/org-keys.el b/lisp/org-keys.el
index 782ffa871..e6b8ff459 100644
--- a/lisp/org-keys.el
+++ b/lisp/org-keys.el
@@ -94,7 +94,7 @@ (declare-function org-feed-update-all "org" ())
 (declare-function org-fill-paragraph "org" (&optional justify region))
 (declare-function org-find-file-at-mouse "org" (ev))
 (declare-function org-footnote-action "org" (&optional special))
-(declare-function org-cycle-force-archived "org-cycle" ())
+(declare-function org-force-cycle-archived "org-cycle" ())
 (declare-function org-force-self-insert "org" (n))
 (declare-function org-forward-element "org" ())
 (declare-function org-forward-heading-same-level "org" (arg &optional 
invisible-ok))
@@ -444,7 +444,7 @@ (org-defkey org-mode-map (kbd "C-c C-x") 
(make-sparse-keymap))
 
  TAB key with modifiers
 (org-defkey org-mode-map (kbd "TAB") #'org-cycle)
-(org-defkey org-mode-map (kbd "C-c C-") #'org-cycle-force-archived)
+(org-defkey org-mode-map (kbd "C-c C-") #'org-force-cycle-archived)
 ;; Override text-mode binding to expose `complete-symbol' for
 ;; pcomplete functionality.
 (org-defkey org-mode-map (kbd "M-TAB") nil)
diff --git a/lisp/org.el b/lisp/org.el
index 402ce3520..0f31e7794 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -106,7 +106,6 @@ (defalias 'org-global-cycle #'org-cycle-global)
 (defalias 'org-overview #'org-cycle-overview)
 (defalias 'org-content #'org-cycle-content)
 (defalias 'org-reveal #'org-fold-reveal)
-(defalias 'org-force-cycle-archived #'org-cycle-force-archived)
 
 ;; `org-outline-regexp' ought to be a defconst but is let-bound in
 ;; some places -- e.g. see the macro `org-with-limited-levels'.
@@ -6169,8 +6168,7 @@ (defun org-get-heading (&optional no-tags no-todo 
no-priority no-comment)
(looking-at org-complex-heading-regexp)
 ;; When using `org-fold-core--optimise-for-huge-buffers',
 ;; returned text may be invisible.  Clear it up.
-(save-match-data
-  (org-fold-core-remove-optimisation (match-beginning 0) (match-end 
0)))
+(org-fold-core-remove-optimisation (match-beginning 0) (match-end 0))
 (let ((todo (and (not no-todo) (match-string 2)))
  (priority (and (not no-priority) (match-string 3)))
  (headline (pcase (match-string 4)
@@ -11699,7 +11697,7 @@ (defun org--get-local-tags ()
   (let* ((cached (and (org-element--cache-active-p) (org-element-at-point nil 
'cached)))
  (cached-tags (org-element-property :tags cached)))
 (if cached
-;; If we do explicitly copy the result, reference would
+;; If we do not explicitly copy the result, reference would
 ;; be returned and cache element might be modified directly.
 (mapcar #'copy-sequence cached-tags)
   ;; Parse tags manually.
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 19/38] Restore old visibility behaviour of org-refile---

[Ihor Radchenko]

---
 lisp/org-refile.el | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lisp/org-refile.el b/lisp/org-refile.el
index 6f2b019ad..e87c3e9a9 100644
--- a/lisp/org-refile.el
+++ b/lisp/org-refile.el
@@ -547,6 +547,7 @@ (defun org-refile (&optional arg default-buffer rfloc msg)
   (goto-char (point-min))
   (or (outline-next-heading) (goto-char (point-max)
   (unless (bolp) (newline))
+   (org-fold-reveal)
   (org-paste-subtree level nil nil t)
   ;; Record information, according to `org-log-refile'.
   ;; Do not prompt for a note when refiling multiple
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 28/38] org-string-width: Handle undefined behaviour in older Emacs

[Ihor Radchenko]

* lisp/org-macs.el (org-string-width): Force older Emacs treating
invisible text with ellipsis as zero-width.  Newer Emacs versions do
exactly this.
---
 lisp/org-macs.el | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 188168cdc..e56a234d3 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -926,7 +926,16 @@ (defun org-string-width (string &optional pixels)
 (with-temp-buffer
   (setq-local display-line-numbers nil)
   (setq-local buffer-invisibility-spec
-  current-invisibility-spec)
+  (if (listp current-invisibility-spec)
+  (mapcar (lambda (el)
+;; Consider elipsis to have 0 width.
+;; It is what Emacs 28+ does, but we have
+;; to force it in earlier Emacs versions.
+(if (and (consp el) (cdr el))
+(list (car el))
+  el))
+  current-invisibility-spec)
+current-invisibility-spec))
   (setq-local char-property-alias-alist
   current-char-property-alias-alist)
   (let (pixel-width symbol-width)
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 25/38] Fix bug in org-get-headingFixes #26, where fontification could make the matching and extraction of heading

[Anders Johansson]

Fixes #26, where fontification could make the matching and extraction of heading
components fail.
---
 lisp/org.el | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 0f31e7794..ff17bf001 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -6167,8 +6167,9 @@ (defun org-get-heading (&optional no-tags no-todo 
no-priority no-comment)
   (let ((case-fold-search nil))
(looking-at org-complex-heading-regexp)
 ;; When using `org-fold-core--optimise-for-huge-buffers',
-;; returned text may be invisible.  Clear it up.
-(org-fold-core-remove-optimisation (match-beginning 0) (match-end 0))
+;; returned text will be invisible.  Clear it up.
+(save-match-data
+  (org-fold-core-remove-optimisation (match-beginning 0) (match-end 
0)))
 (let ((todo (and (not no-todo) (match-string 2)))
  (priority (and (not no-priority) (match-string 3)))
  (headline (pcase (match-string 4)
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 24/38] Fix typo: delete-duplicates → delete-dups

[Anders Johansson]

---
 lisp/org-fold-core.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org-fold-core.el b/lisp/org-fold-core.el
index 6ea374498..121c6b5c4 100644
--- a/lisp/org-fold-core.el
+++ b/lisp/org-fold-core.el
@@ -592,7 +592,7 @@ (defun org-fold-core--property-symbol-get-create (spec 
&optional buffer return-o
   (org-fold-core-cycle-over-indirect-buffers
   (push (current-buffer) bufs))
   (push buf bufs)
-  (delete-duplicates bufs)
+  (delete-dups bufs)
 ;; Copy all the old folding properties to preserve the folding 
state
 (with-silent-modifications
   (dolist (old-prop (cdr (assq 'invisible 
char-property-alias-alist)))
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 31/38] org-fold-core: Fix fontification inside folded regions

[Ihor Radchenko]

* lisp/org-fold-core.el (org-fold-core-initialize): Declare
`org-fold-core-fontified' text property for font-lock.
(org-fold-core--force-fontification): New variable controlling forced
fontification inside folded regions.
(org-fold-core-fontify-region): Fix cases when BEG is inside folded
region.  Respect `org-fold-core--force-fontification'.
* lisp/org-macs.el (org-with-forced-fontification): New macro.
(org-buffer-substring-fontified):
(org-looking-at-fontified): Do not rely on jit-lock.  Use
`org-fold-core-fontified' text property to determine whether text is
already fontified.
---
 lisp/org-fold-core.el | 69 +--
 lisp/org-macs.el  | 31 +++
 2 files changed, 72 insertions(+), 28 deletions(-)

diff --git a/lisp/org-fold-core.el b/lisp/org-fold-core.el
index 121c6b5c4..edae316ff 100644
--- a/lisp/org-fold-core.el
+++ b/lisp/org-fold-core.el
@@ -746,7 +746,8 @@ (defun org-fold-core-initialize (&optional specs)
   (add-hook 'clone-indirect-buffer-hook 
#'org-fold-core-decouple-indirect-buffer-folds nil 'local)
   ;; Optimise buffer fontification to not fontify folded text.
   (when (eq font-lock-fontify-region-function 
#'font-lock-default-fontify-region)
-(setq-local font-lock-fontify-region-function 
'org-fold-core-fontify-region))
+(setq-local font-lock-fontify-region-function 
'org-fold-core-fontify-region)
+(add-to-list 'font-lock-extra-managed-props 'org-fold-core-fontified))
   ;; Setup killing text
   (setq-local filter-buffer-substring-function 
#'org-fold-core--buffer-substring-filter)
   (if (and (boundp 'isearch-opened-regions)
@@ -1429,35 +1430,47 @@ (defun org-fold-core--buffer-substring-filter (beg end 
&optional delete)
 return-string))
 
 ;;; Do not fontify folded text until needed.
-
+(defvar org-fold-core--force-fontification nil
+  "Let-bind this variable to t in order to force fontification in
+folded regions.")
 (defun org-fold-core-fontify-region (beg end loudly &optional force)
   "Run `font-lock-default-fontify-region' in visible regions."
-  (let ((pos beg) next
-(org-fold-core--fontifying t))
-(while (< pos end)
-  (setq next (org-fold-core-next-folding-state-change
-  (if force nil
-(let (result)
-  (dolist (spec (org-fold-core-folding-spec-list))
-(when (and (not 
(org-fold-core-get-folding-spec-property spec :visible))
-   (org-fold-core-get-folding-spec-property 
spec :font-lock-skip))
-  (push spec result)))
-  result))
-  pos
-  end))
-  (while (and (not (catch :found
-   (dolist (spec (org-fold-core-get-folding-spec 'all 
next))
- (when (org-fold-core-get-folding-spec-property spec 
:font-lock-skip)
-   (throw :found spec)
-  (< next end))
-(setq next (org-fold-core-next-folding-state-change nil next end)))
-  (save-excursion
-(font-lock-default-fontify-region pos next loudly)
-(save-match-data
-  (unless (<= pos (point) next)
-(run-hook-with-args 'org-fold-core-first-unfold-functions pos 
next
-  (put-text-property pos next 'org-fold-core-fontified t)
-  (setq pos next
+  (with-silent-modifications
+(let ((pos beg) next
+  (force (or force org-fold-core--force-fontification))
+  (org-fold-core--fontifying t)
+  (skip-specs
+   (let (result)
+ (dolist (spec (org-fold-core-folding-spec-list))
+   (when (and (not (org-fold-core-get-folding-spec-property spec 
:visible))
+  (org-fold-core-get-folding-spec-property spec 
:font-lock-skip))
+ (push spec result)))
+ result)))
+  ;; Move POS to first visible point within BEG..END.
+  (while (and (catch :found
+(dolist (spec (org-fold-core-get-folding-spec 'all pos))
+  (when (org-fold-core-get-folding-spec-property spec 
:font-lock-skip)
+(throw :found spec
+  (< pos end))
+(setq pos (org-fold-core-next-folding-state-change nil pos end)))
+  (when force (setq pos beg next end))
+  (while (< pos end)
+(unless force
+  (setq next (org-fold-core-next-folding-state-change skip-specs pos 
end)))
+;; Move to the end of the region to be fontified.
+(while (and (not (catch :found
+ (dolist (spec (org-fold-core-get-folding-spec 'all 
next))
+   (when (org-fold-core-get-folding-spec-property spec 
:font-lock-skip)
+ (throw :found spec)
+(< next end))
+  (setq next (org-fold-core-next-folding-state-change nil next end)))
+(save-excursion
+  (font

[PATCH v2 33/38] org--string-from-props: Fix handling folds in Emacs <28

[Ihor Radchenko]

* lisp/org-macs.el (org--string-from-props): Respect
`char-property-alias-alist' when querying for `invisible' text
property.
---
 lisp/org-macs.el | 9 -
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 5e6728101..c71bb4094 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -903,14 +903,13 @@ (defun org--string-from-props (s property beg end)
(cursor beg))
 (while (setq beg (text-property-not-all beg end property nil s))
   (let* ((next (next-single-property-change beg property s end))
-(props (text-properties-at beg s))
-(spec (plist-get props property))
+(spec (get-text-property beg property s))
 (value
  (pcase property
(`invisible
-;; If `invisible' property in PROPS means text is to
-;; be invisible, return 0.  Otherwise return nil so
-;; as to resume search.
+;; If `invisible' property means text is to be
+;; invisible, return 0.  Otherwise return nil so as
+;; to resume search.
 (and (or (eq t buffer-invisibility-spec)
  (assoc-string spec buffer-invisibility-spec))
  0))
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 30/38] org-fold-show-set-visibility: Fix edge case when folded region is at BOB

[Ihor Radchenko]

---
 lisp/org-fold.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org-fold.el b/lisp/org-fold.el
index a16ee0f9b..d5a21cbcb 100644
--- a/lisp/org-fold.el
+++ b/lisp/org-fold.el
@@ -840,7 +840,7 @@ (defun org-fold-show-set-visibility--text-properties 
(detail)
 (org-with-point-at (car region)
   (beginning-of-line)
   (let (font-lock-extend-region-functions)
-(font-lock-fontify-region (1- (car region)) (cdr region))
+(font-lock-fontify-region (max (point-min) (1- (car region))) 
(cdr region))
 (when region
   (org-fold-region (car region) (cdr region) nil
 (unless (org-before-first-heading-p)
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 34/38] org-link-make-string: Throw error when both LINK and DESCRIPTION are empty

[Ihor Radchenko]

This behaviour is expected according to `test-ol/make-string'.
---
 lisp/ol.el | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/lisp/ol.el b/lisp/ol.el
index 86f55d7cf..22eaae8fb 100644
--- a/lisp/ol.el
+++ b/lisp/ol.el
@@ -999,7 +999,9 @@ (defun org-link-make-string (link &optional description)
(replace-regexp-in-string "]\\'"
  (concat "\\&" zero-width-space)
  (org-trim description))
-(if (not (org-string-nw-p link)) description
+(if (not (org-string-nw-p link))
+(or description
+(error "Empty link"))
   (format "[[%s]%s]"
  (org-link-escape link)
  (if description (format "[%s]" description) "")
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 35/38] test-ol/org-toggle-link-display: Fix compatibility with old Emacs

[Ihor Radchenko]

* testing/lisp/test-ol.el (test-ol/org-toggle-link-display): Use
back-compatible `org-xor' instead of `xor'.
---
 testing/lisp/test-ol.el | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/testing/lisp/test-ol.el b/testing/lisp/test-ol.el
index 343631623..429bb52ee 100644
--- a/testing/lisp/test-ol.el
+++ b/testing/lisp/test-ol.el
@@ -59,19 +59,19 @@ (ert-deftest test-ol/org-toggle-link-display ()
   (dotimes (_ 2)
 (goto-char 1)
 (re-search-forward "\\[")
-(should-not (xor org-link-descriptive (org-invisible-p)))
+(should-not (org-xor org-link-descriptive (org-invisible-p)))
 (re-search-forward "example")
-(should-not (xor org-link-descriptive (org-invisible-p)))
+(should-not (org-xor org-link-descriptive (org-invisible-p)))
 (re-search-forward "com")
-(should-not (xor org-link-descriptive (org-invisible-p)))
+(should-not (org-xor org-link-descriptive (org-invisible-p)))
 (re-search-forward "]")
-(should-not (xor org-link-descriptive (org-invisible-p)))
+(should-not (org-xor org-link-descriptive (org-invisible-p)))
 (re-search-forward "\\[")
 (should-not (org-invisible-p))
 (re-search-forward "link")
 (should-not (org-invisible-p))
 (re-search-forward "]")
-(should-not (xor org-link-descriptive (org-invisible-p)))
+(should-not (org-xor org-link-descriptive (org-invisible-p)))
 (org-toggle-link-display)
 
 
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 32/38] test-org/string-width: Add tests for strings with prefix properties

[Ihor Radchenko]

---
 testing/lisp/test-org-macs.el | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/testing/lisp/test-org-macs.el b/testing/lisp/test-org-macs.el
index 6a7ccea3c..05cef1281 100644
--- a/testing/lisp/test-org-macs.el
+++ b/testing/lisp/test-org-macs.el
@@ -65,7 +65,11 @@ (ert-deftest test-org/string-width ()
   (should (= 4 (org-string-width
#("123" 1 2 (display #("abc" 1 2 (invisible t)))
   ;; Test `space' property in `display'.
-  (should (= 2 (org-string-width #(" " 0 1 (display (space :width 2)))
+  (should (= 2 (org-string-width #(" " 0 1 (display (space :width 2))
+  ;; Test `wrap-prefix' property.
+  (should (= 2 (org-string-width #("ab" 0 2 (wrap-prefix "  ")
+  ;; Test `line-prefix' property.
+  (should (= 2 (org-string-width #("ab" 0 2 (line-prefix "  "))
 
 
 ;;; Regexp
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 37/38] org-fold-core-fontify-region: Fix cases when fontification is not registered

[Ihor Radchenko]

* lisp/org-fold-core.el (org-fold-core-fontify-region): Handle FORCE
argument better.  Skip unnecessary code parts when FORCE is non-nil.
Assign `fontified' text property manually in the actually fontified
regions.  We cannot just supply correct return value since jit-lock
does not allow piecewise fontification.
---
 lisp/org-fold-core.el | 51 +++
 1 file changed, 27 insertions(+), 24 deletions(-)

diff --git a/lisp/org-fold-core.el b/lisp/org-fold-core.el
index edae316ff..d98eb6358 100644
--- a/lisp/org-fold-core.el
+++ b/lisp/org-fold-core.el
@@ -1436,40 +1436,43 @@ (defvar org-fold-core--force-fontification nil
 (defun org-fold-core-fontify-region (beg end loudly &optional force)
   "Run `font-lock-default-fontify-region' in visible regions."
   (with-silent-modifications
-(let ((pos beg) next
-  (force (or force org-fold-core--force-fontification))
-  (org-fold-core--fontifying t)
-  (skip-specs
-   (let (result)
- (dolist (spec (org-fold-core-folding-spec-list))
-   (when (and (not (org-fold-core-get-folding-spec-property spec 
:visible))
-  (org-fold-core-get-folding-spec-property spec 
:font-lock-skip))
- (push spec result)))
- result)))
+(let* ((pos beg) next
+   (force (or force org-fold-core--force-fontification))
+   (org-fold-core--fontifying t)
+   (skip-specs
+(unless force
+  (let (result)
+(dolist (spec (org-fold-core-folding-spec-list))
+  (when (and (not (org-fold-core-get-folding-spec-property 
spec :visible))
+ (org-fold-core-get-folding-spec-property spec 
:font-lock-skip))
+(push spec result)))
+result
   ;; Move POS to first visible point within BEG..END.
-  (while (and (catch :found
-(dolist (spec (org-fold-core-get-folding-spec 'all pos))
-  (when (org-fold-core-get-folding-spec-property spec 
:font-lock-skip)
-(throw :found spec
-  (< pos end))
-(setq pos (org-fold-core-next-folding-state-change nil pos end)))
+  (unless force
+(while (and (catch :found
+  (dolist (spec (org-fold-core-get-folding-spec 'all pos))
+(when (org-fold-core-get-folding-spec-property spec 
:font-lock-skip)
+  (throw :found spec
+(< pos end))
+  (setq pos (org-fold-core-next-folding-state-change nil pos end
   (when force (setq pos beg next end))
   (while (< pos end)
 (unless force
-  (setq next (org-fold-core-next-folding-state-change skip-specs pos 
end)))
-;; Move to the end of the region to be fontified.
-(while (and (not (catch :found
- (dolist (spec (org-fold-core-get-folding-spec 'all 
next))
-   (when (org-fold-core-get-folding-spec-property spec 
:font-lock-skip)
- (throw :found spec)
-(< next end))
-  (setq next (org-fold-core-next-folding-state-change nil next end)))
+  (setq next (org-fold-core-next-folding-state-change skip-specs pos 
end))
+  ;; Move to the end of the region to be fontified.
+  (while (and (not (catch :found
+   (dolist (spec (org-fold-core-get-folding-spec 'all 
next))
+ (when (org-fold-core-get-folding-spec-property 
spec :font-lock-skip)
+   (throw :found spec)
+  (< next end))
+(setq next (org-fold-core-next-folding-state-change nil next 
end
 (save-excursion
   (font-lock-default-fontify-region pos next loudly)
   (save-match-data
 (unless (<= pos (point) next)
   (run-hook-with-args 'org-fold-core-first-unfold-functions pos 
next
 (put-text-property pos next 'org-fold-core-fontified t)
+(put-text-property pos next 'fontified t)
 (setq pos next)
 
 (defun org-fold-core-update-optimisation (beg end)
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

[PATCH v2 38/38] org-agenda.el: Re-enable native compilation* lisp/org-agenda.el: Re-enable native compilation as it does not

[Ihor Radchenko]

* lisp/org-agenda.el: Re-enable native compilation as it does not
cause any issues with agenda rendering without other custom changes
not included in this branch.
---
 lisp/org-agenda.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index 6aed778f0..b817f1f04 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -1,4 +1,4 @@
-;;; org-agenda.el --- Dynamic task and appointment lists for Org  -*- 
lexical-binding: t; no-native-compile: t; -*-
+;;; org-agenda.el --- Dynamic task and appointment lists for Org  -*- 
lexical-binding: t; -*-
 
 ;; Copyright (C) 2004-2022 Free Software Foundation, Inc.
 
-- 
2.35.1



-- 
Ihor Radchenko,
PhD,
Center for Advancing Materials Performance from the Nanoscale (CAMP-nano)
State Key Laboratory for Mechanical Behavior of Materials, Xi'an Jiaotong 
University, Xi'an, China
Email: yanta...@gmail.com, ihor_radche...@alumni.sutd.edu.sg

Re: [PATCH v2 00/38] Final call for comments: Merge org-fold feature branch

[Bastien]

Hi Ihor,

thanks for your work on this.

Ihor Radchenko  writes:

> This is the final version of the patch. I am going to merge it this
> weekend. If there are any comments, please send them ASAP.

One glitch: in an agenda view with an agenda item that has a link,
changing the todo state will make all parts of the link visible.

Let me know if you need a more detail recipe to reproduce the bug.

-- 
 Bastien

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Max Nikulin]

On 19/04/2022 09:02, Paul Eggert wrote:
Here are the main points I see about making timestamps work better in 
Org mode, and related patches to how Emacs handles timestamps.


* Max would like encode-time to treat a list (SS MM HH DD MM ) as if 
it were (SS MM HH DD MM  nil -1 nil),


Thank you, Paul, it is exactly what I had in mind when I created this bug.

as that would be more 
convenient for how Org mode deals with ambiguous local timestamps. This 
is relatively easy to add for Emacs 29+, and is done in first of the 
attached proposed patches to Emacs master.


I would say that Org just does not care concerning ambiguous local time. 
Anyway there are other similar cases besides DST.


* As I understand it, Max would like a function that emulate Emacs 29 
encode-time with one argument, even if running in Emacs versions back to 
Emacs 25. I suppose such a function would also need to implement Emacs 
27+ encode-time's support for subsecond resolution. E.g., 
(org-encode-time '((44604411568 . 10) 55 0 19 4 2022 - -1 t)) 
should return (1650329744604411568 . 10) even in Emacs 25 and 26.


I am just afraid of possibility of recurrent attempts to modernize 
time-related code within Emacs including Org code in a such way that can 
not be directly ported to the Org repository. Discrepancy of the code 
increases maintenance burden. The main purpose of a compatibility 
wrapper is to prevent grep-driven refactoring. Another point of the 
helper function is to allow to remove from Emacs support confusing 
old-style `encode-time' arguments with ignored DST value. In Org 
timestamps are often built from scratch, so separate argument are still 
convenient.


Org timestamps have minute precision, even seconds are trimmed. So at 
least explicitly I did not ask for subsecond timestamps. I admit however 
that they might emerge in some code paths.


Notice that nobody from Org developers & maintainers commented the patch 
demonstrating the idea of such wrapper.


* My last topic in this email is Max's request for a feature that I'm 
not planning to put into Emacs 29 as it'll require more thought. This 
addresses the problem where your TZ is "Africa/Juba" and you want to 
encode a timestamp like "2021-01-31 23:30" which is ambiguous since at 
24:00 that day Juba moved standard time back by an hour. Unfortunately 
the underlying C mktime function does not allow disambiguation in the 
rare situation where standard time moves further west of Greenwich. 
Addressing this problem would require rewriting mktime from scratch in 
Elisp, or using heuristics that would occasionally fail, or (my 
favorite) extending glibc mktime to treat tm_isdst values other than 
-1,0,1 to support disambiguating such timestamps.


I do not urge such changes. I have not checked if mktime is a part of 
POSIX and C standard. If it is so, I am not in favor of adding more 
values for the tm_isdst field since they are not related to DST.


I started this branch of discussion to convince Paul that requirement of 
9 fields is not really encourage more correct usage of `encode-time' in 
comparison to 6 values.


More convenient interface for processing of local time moments requires 
significant amount of work, maybe some prototypes. It should be 
considered separately from this bug.


I still believe that optional DST and ZONE values is an improvement of 
the `encode-time' interface with no real drawbacks. It minimizes the 
chance of passing nil as "no DST" when actual value is unknown and 
developers are not ready to add a bunch of code to determine proper TZ 
offset for each case of time transition.

Re: [PATCH v2 00/38] Final call for comments: Merge org-fold feature branch

[Ihor Radchenko]

Bastien  writes:

> One glitch: in an agenda view with an agenda item that has a link,
> changing the todo state will make all parts of the link visible.
>
> Let me know if you need a more detail recipe to reproduce the bug.

Recipe would be helpftul. I was unable to reproduce using

* TODO this is test [[https://orgmode.org][org]]
SCHEDULED: <2022-04-20 Wed>

Best,
Ihor

[PATCH] ol: Fix org-link-search

[tony aldon]

Hey everyone,

If I'm not wrong, name defined after #+NAME: should be match first
before trying a fuzzy search in the function `org-link-search`.

You can find the patch in attachment (hope it feat in Tiny changes
[though I also sent a request today for the FSF agreement]).

Have a nice day.
Tony Aldon
From ba4028e71fb41b6e4367d3e85846526dd7577d52 Mon Sep 17 00:00:00 2001
From: tony 
Date: Wed, 20 Apr 2022 17:58:34 +0200
Subject: [PATCH] ol: Fix org-link-search

* lisp/ol.el (org-link-search):  Replace wrong property :name by
  :value.  Name defined after #+NAME: should be match first before
  trying a fuzzy search.
* testing/lisp/test-ol.el (test-ol/search):  New test.

TINYCHANGE
---
 lisp/ol.el  |  2 +-
 testing/lisp/test-ol.el | 12 
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/lisp/ol.el b/lisp/ol.el
index 1b2bb9a9a..4554941b6 100644
--- a/lisp/ol.el
+++ b/lisp/ol.el
@@ -1189,7 +1189,7 @@ of matched result, which is either `dedicated' or `fuzzy'."
 	   (goto-char (point-min))
 	   (while (re-search-forward name nil t)
 		 (let* ((element (org-element-at-point))
-			(name (org-element-property :name element)))
+			(name (org-element-property :value element)))
 		   (when (and name (equal words (split-string name)))
 		 (setq type 'dedicated)
 		 (beginning-of-line)
diff --git a/testing/lisp/test-ol.el b/testing/lisp/test-ol.el
index ddcc570b3..3e5b9c7cd 100644
--- a/testing/lisp/test-ol.el
+++ b/testing/lisp/test-ol.el
@@ -492,6 +492,18 @@
 	(buffer-substring (point) (line-end-position))
 
 
+;;; Search
+
+(ert-deftest test-ol/search ()
+  "Test `org-link-search'."
+  ;; Look for name defined after #+NAME:
+  (should
+   (equal 'dedicated
+  (org-test-with-temp-text "foo\n#+NAME: foo"
+	(let ((org-link-search-must-match-exact-headline nil))
+  (org-link-search "foo"))
+
+
 ;;; Link regexps
 
 
-- 
2.17.1

Re: [PATCH v2 00/38] Final call for comments: Merge org-fold feature branch

[Bastien]

Ihor Radchenko  writes:

> Bastien  writes:
>
>> One glitch: in an agenda view with an agenda item that has a link,
>> changing the todo state will make all parts of the link visible.
>>
>> Let me know if you need a more detail recipe to reproduce the bug.
>
> Recipe would be helpftul. I was unable to reproduce using

I'm on commit e0abbbacf9427b69482c6c47c3ea0975b0e6fa6d from the
feature/org-fold-universal-core branch of your org repo.

Using Emacs 29.0.50 and this as the content for ~/test.org:

* TODO [[https://orgmode.org][test]]
SCHEDULED: <2022-04-20 mer.>

Then run:

- emacs -q
- C-x C-f test.org
- M-x org-agenda RET
- < a . n t

You should see the link losing its invisibility specs.

-- 
 Bastien

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Max Nikulin]

On 17/04/2022 08:58, Paul Eggert wrote:

Thanks, I installed that and then installed the attached, which merges
that with some documentation improvements that I drafted based on this
thread.


Thank you for further editing of docs. Please, fix a typo.


diff --git a/doc/lispref/os.texi b/doc/lispref/os.texi
index 66689f43a9..8366689640 100644
--- a/doc/lispref/os.texi
+++ b/doc/lispref/os.texi

@@ -1687,14 +1660,18 @@ Time Conversion
  than six arguments the @emph{last} argument is used as @var{zone} and
  any other extra arguments are ignored, so that @code{(apply
  #'encode-time (decode-time ...))} works.  In this obsolescent
-convention, @var{zone} defaults to the current time zone rule
-(@pxref{Time Zone Rules}), and @var{dst} is treated as if it was
-@minus{}1.
+convention, @var{dst} is @minus{}1 and @var{zone} defaults to the
+current time zone rule (@pxref{Time Zone Rules}).
+When modernizing an obsolescent caller, ensure that the more-modern
+list equivalent contains 9 elements with a a @code{dst} element that


^^^
A typo: double "a".


+is @minus{}1, not @code{nil}.

+@lisp
+;; Try to compute the time four years from now.
+;; Watch out; this might not work as expected.
+(let ((time (decode-time)))
+  (setf (decoded-time-year time)
+(+ (decoded-time-year time) 4))
+  time)
+@end lisp



+@noindent
+Unfortunately, this code might not work as expected if the resulting
+time is invalid due to daylight saving transitions, time zone changes,
+or missing leap days or leap seconds.  For example, if executed on
+February 29, 2096 this code yields a nonexistent date because 2100 is
+not a leap year.  To avoid some (though not all) of the problem, you
+can base calculations on the middle of the affected unit, e.g., start
+at July 1 when adding years.


If I get your idea correctly then "January, 31" + "1 month" should be 
more impressive as impossible date. Year 2096 is too far in future. I am 
unsure concerning expectation. Overflow arithmetic is described above 
and e.g. JavaScript normalizes Date object in a similar fashion. The 
special point is that elisp decoded time requires explicit normalization 
however and 2100 is a good example that updating of any field may 
"break" the date.



Alternatively, you can use the
+@file{calendar} and @file{time-date} libraries.


A remark loosely related to your patch. Earlier you mentioned missed 
midnight due to time transition and suggested to use calendrical 
functions in Org. I can not figure out which elisp function can help to 
determine wall time for Aug 1 start of day in Cairo:


Africa/Cairo  Thu Jul 31 21:59:59 2014 UT = Thu Jul 31 23:59:59 2014 EET 
isdst=0 gmtoff=7200
Africa/Cairo  Thu Jul 31 22:00:00 2014 UT = Fri Aug  1 01:00:00 2014 
EEST isdst=1 gmtoff=10800


input: 2014-08-01 Africa/Cairo
(timezone may be implicit as the system one)
expected output: 01:00:00

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Eli Zaretskii]

> Date: Wed, 20 Apr 2022 12:01:27 -0700
> Cc: maniku...@gmail.com, emacs-orgmode@gnu.org, 54...@debbugs.gnu.org,
>  bug-gnu...@gnu.org
> From: Paul Eggert 
> 
> > I get the same result, and moreover I see no differences between this
> > and the previous tarball, and no printouts in test-gettime-res.c.  Did
> > you perhaps attach the wrong tarball this time?
> 
> That's odd, as I get the different result (i.e., with debugging info) 
> when I use that tarball, a copy of which I got off the email archive (so 
> I know I sent it :-). Please see the attached shell transcript for 
> exactly how I got the different result, starting from the email archive.

Sorry, my bad.  The result is the same, but I do get printouts.  What
do you want to know or see from there?

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

On 4/20/22 08:07, Max Nikulin wrote:

I have not checked if mktime is a part of 
POSIX and C standard.
mktime is part of both the C standard and POSIX. POSIX extends the C 
standard by saying that time_t is an integer type (the C standard allows 
time_t to be a floating-point type) and that time_t counts non-leap 
seconds since the Epoch (the C standard doesn't say what time_t counts, 
thought it implies that it counts seconds from some origin, which may 
not be 1970).
I still believe that optional DST and ZONE values is an improvement of 
the `encode-time' interface with no real drawbacks.


Yes, that's the direction we're headed.

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Eli Zaretskii]

> Date: Wed, 20 Apr 2022 11:19:29 -0700
> Cc: maniku...@gmail.com, emacs-orgmode@gnu.org, 54...@debbugs.gnu.org,
>  Gnulib bugs 
> From: Paul Eggert 
> 
> > Thanks, the test-gettime-res test says "gettime_res returned 625000
> > ns", which is a strange number: it doesn't fit any MS-Windows system
> > time resolution figure I know about.  Do you happen to know what does
> > this number represent, and why it is the result of gettime-res.c when
> > it runs on MS-Windows?
> 
> It comes from current_timespec samples taken by gettime_res. Evidently 
> something is going wrong, either in gettime_res or in current_timespec.
> 
> I stared at the code a bit and see one possible problem, which I fixed 
> by installing the attached patch into Gnulib. I then generated a new 
> test-gettime-res.tgz compressed tarball (also attached); could you give 
> it a try?
> 
> This tarball is the result of running ./gnulib-tool as before, except I 
> added an extra print statement executed if you pass an extra argument to 
> that test program. So if you run the test and it's still outputting an 
> outlandish value for the resolution, please run the command:
> 
> gltests/test-gettime-res x
> 
> and let's take a look at its (long) debugging output.

I get the same result, and moreover I see no differences between this
and the previous tarball, and no printouts in test-gettime-res.c.  Did
you perhaps attach the wrong tarball this time?

Thanks.

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

On 4/20/22 09:56, Max Nikulin wrote:


A typo: double "a".


Thanks, fixed in the attached which I installed in master.


If I get your idea correctly then "January, 31" + "1 month" should be 
more impressive as impossible date.


Thanks, good idea; also in the attached patch.


I can not figure out which elisp function can help to 
determine wall time for Aug 1 start of day in Cairo:


Africa/Cairo  Thu Jul 31 21:59:59 2014 UT = Thu Jul 31 23:59:59 2014 EET 
isdst=0 gmtoff=7200
Africa/Cairo  Thu Jul 31 22:00:00 2014 UT = Fri Aug  1 01:00:00 2014 
EEST isdst=1 gmtoff=10800


input: 2014-08-01 Africa/Cairo
(timezone may be implicit as the system one)
expected output: 01:00:00


Given mktime's limitations there's no trivial way to do this for 
arbitrary timestamps, since 00:00 doesn't exist in Cairo that day. 
Worse, in some locations near the International Date Line entire days do 
not exist, because at 00:00 they advanced the clocks forward 24 hours in 
order to move the date line.


It sounds like you're asking for a function that, given a date, yields 
the first broken-down timestamp on or after 00:00 of that date. For 
something like that, I'd use encode-time on 00:00 of that date to get a 
timestamp T, and then use time-add and decode-time to decode T-86400 
seconds, T, and T+86400 seconds, and if the decoded times all look fine 
then return (decode-time T). If not (i.e., their UTC offsets differ, or 
T's decoded time is not 00:00 on the correct date) I'd use binary search 
to find discontinuities between T-86400 and T+86400 and look next to 
those discontinuities to find timestamps closer to what you want.


Of course this is not ideal - but it's similar to what many mktime 
implementations do internally, and it's also similar to what Emacs's 
cal-dst already does (maybe you can look there for ideas), so you'd be 
in good company.From f98c3f4426fecf794f47f27aebe1f3b854fb1bfd Mon Sep 17 00:00:00 2001
From: Paul Eggert 
Date: Wed, 20 Apr 2022 12:03:19 -0700
Subject: [PATCH] More encode-time pitfall doc fixes

* doc/lispref/os.texi (Time Conversion): Improve discussion of
encode-time pitfalls based on comments by Max Nikulin (Bug#54764#63).
---
 doc/lispref/os.texi | 22 ++
 1 file changed, 14 insertions(+), 8 deletions(-)

diff --git a/doc/lispref/os.texi b/doc/lispref/os.texi
index cabae08970..4138dab09f 100644
--- a/doc/lispref/os.texi
+++ b/doc/lispref/os.texi
@@ -1670,7 +1670,7 @@ Time Conversion
 convention, @var{dst} is @minus{}1 and @var{zone} defaults to the
 current time zone rule (@pxref{Time Zone Rules}).
 When modernizing an obsolescent caller, ensure that the more-modern
-list equivalent contains 9 elements with a a @code{dst} element that
+list equivalent contains 9 elements with a @code{dst} element that
 is @minus{}1, not @code{nil}.
 
 Year numbers less than 100 are not treated specially.  If you want them
@@ -1695,22 +1695,28 @@ Time Conversion
 For example:
 
 @lisp
-;; Try to compute the time four years from now.
+;; Try to compute the time one month from now.
 ;; Watch out; this might not work as expected.
 (let ((time (decode-time)))
-  (setf (decoded-time-year time)
-(+ (decoded-time-year time) 4))
+  (setf (decoded-time-month time)
+(+ (decoded-time-month time) 1))
   time)
 @end lisp
 
 @noindent
 Unfortunately, this code might not work as expected if the resulting
-time is invalid due to daylight saving transitions, time zone changes,
+time is invalid due to month length differences,
+daylight saving transitions, time zone changes,
 or missing leap days or leap seconds.  For example, if executed on
-February 29, 2096 this code yields a nonexistent date because 2100 is
-not a leap year.  To avoid some (though not all) of the problem, you
+January 30 this code yields a nonexistent date February 30,
+which @code{encode-time} would adjust to early March.
+Similarly, adding four years to February 29, 2096 would yield the
+nonexistent date February 29, 2100; and adding one hour to 01:30 on
+March 13, 2022 in New York would yield a timestamp 02:30 that does not
+exist because clocks sprang forward from 02:00 to 03:00 that day.
+To avoid some (though not all) of the problem, you
 can base calculations on the middle of the affected unit, e.g., start
-at July 1 when adding years.  Alternatively, you can use the
+at the 15th of the month when adding months.  Alternatively, you can use the
 @file{calendar} and @file{time-date} libraries.
 @end defun
 
-- 
2.32.0

WG: ox-html: exporting LaTeX-environments

[Vitus Schäfftlein]

[Bump]

Dear Thibault, dear list,

thanks for your fast answer, Thibault!  I am happy to have convinced you of 
dropping the extra \n's.  The patch you addressed works fine for the asterisk 
issue: Only those environments get an extra asterisk which are in 
org-latex-math-environments-re. Nonetheless, I think we should make that 
variable more user-friendly by not requiring one huge regexp but a list of 
environment names like ("foo" "bar" "baz").

Unfortunately, the patch does not have any impact on the numbering problem. I 
guess we actually need org-html--latex-environments-leave-unlabelled to get 
this running. I just wanna give two examples why it is not useful to lable 
every environment.

1. Imagine you do not need links to your equations and use the \tag command in 
your equation environment to get the right equation numbers within the svg. 
This looks nicer anyway because the font is the same and the formatting is 
perfect. If we did not have org-html--latex-environments-leave-unlabelled, 
there would be both the line number within the equation and, below it, another 
in-text-line-number on a new line. Same goes for the figure-environment which 
provides its own caption. In general, everything that has its own counter 
within the svg should not receive extra equation numbers.
2. There are some environments you do not want labelled. Imagine you compile a 
table summarizing the most important formulas of your post. This is not an 
equation and it should not be numbered, but it is. Below the table then is a 
number which looks awful.

You will find an example of the problem here: 
https://uni-muenster.sciebo.de/s/8gW9dCx7q8NHIeV
[https://uni-muenster.sciebo.de/apps/files_sharing/publicpreview?file=/line_numbers.png&t=8gW9dCx7q8NHIeV&x=200&y=200]
sciebo - 
www.hochschulcloud.nrw
line_numbers.png is publicly shared
uni-muenster.sciebo.de


Since I am in the position of setting up a blog just now, I can only stress 
this variable is needed. I wish I could help you with the code more than I have 
done with my suggestions. But maybe some one else on the list has an idea?

Best Regards,
Vitus

Von: Thibault Marin [thibault.ma...@gmx.com]
Gesendet: Dienstag, 12. April 2022 07:15
An: Vitus Schäfftlein
Cc: emacs-orgmode@gnu.org
Betreff: Re: ox-html: exporting LaTeX-environments

Hi Vitus, list.

My memory is quite fuzzy on this and I won't have a chance to take a
deep look until later, but I will try to share the information I have.

On Mon, 11 Apr 2022 19:38:13 + (9 hours, 37 minutes, 37 seconds ago), Vitus 
Schäfftlein  wrote:

  Dear org-mode mailing list,

  [...]

  3. Any LaTeX environment name foo is changed to foo* (except it already ends 
with an
   asterisk). For example, \begin{tabular} is changed to \begin{tabular*}; same 
for
   \end{tabular}. But tabular* differs from tabular in needing an extra 
width-argument, so
   the export won’t work properly.

I had submitted a patch trying to address this
(https://list.orgmode.org/87h7ok3qi2.fsf@dell-desktop.WORKGROUP/, I have
attached a new version rebased on main to this message).  It never made
it in and I failed to follow-up.  This patch (or something similar)
could help with this issue.  It basically only adds the star for math
environments (using org-html--math-environment-p)

  [...]

  Now the newline commands \n before and after %s are exported as whitespace. 
Just replacing
  \n%s\n by %s (that is, leaving the newlines out) solves the problem. HTML 
ignores newlines
  anyway.

This seems to work better indeed; the \n's were just cosmetic.

  [...]

  1 Create a new variable ox-html-latex-environments-no-number of the form 
("foo"
   "bar" "baz" ...), which contains all environments that should not receive 
equation
   numbers.

I don't know whether org-html--math-environment-p (as used in the
attached patch) is sufficient to determine whether we need to add a star
to the environment or if we need another variable (in my use cases,
testing for a match environment is sufficient but it may not be the case
in general).

  [...]

  I don’t know how to express in elisp what is in brackets, though. Does this 
make sense to you? I
  am a beginner with elisp, so I can only state the ideas I have but not 
implement them (yet).

This can be made to work if there is a consensus that we want to add a
ox-html-latex-environments-no-number variable (I can try to help with
that if needed, even though my elisp isn't great)

  [...]

  (let ((formula-link
  (org-html-format-latex
   (org-html--unlabel-latex-environment latex-frag)
   processing-type info)))

The patch should address that, I would be curious to see if you
encounter additional problematic cases after applying it.

Thanks for resurrecting this and for your help detecting and fixing the issues.

Best,

thibault

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Eli Zaretskii]

> Date: Wed, 20 Apr 2022 12:23:43 -0700
> Cc: maniku...@gmail.com, emacs-orgmode@gnu.org, 54...@debbugs.gnu.org,
>  bug-gnu...@gnu.org
> From: Paul Eggert 
> 
> On 4/20/22 12:14, Eli Zaretskii wrote:
> > Sorry, my bad.  The result is the same, but I do get printouts.  What
> > do you want to know or see from there?
> 
> I want to see what the current_timespec's resolution is, which we should 
> be able to tell from the debugging output. For example, on my Solaris 10 
> sparc platform the command 'gltests/test-gettime-res x' outputs:
> 
> gettime_res returned 200 ns
> time = 1650482432.256445600
> time = 1650482432.256460600
> time = 1650482432.256464400
> time = 1650482432.256468200
> time = 1650482432.256471400
> time = 1650482432.256474600
> time = 1650482432.256478000
> time = 1650482432.256481200
> time = 1650482432.256484800
> ...
> 
> and these timestamps say that with very high probability 
> current_timespec's clock resolution is indeed 200 ns.

I see the time samples change in jumps of 15 msec.  Which is expected
on MS-Windows, given the scheduler time tick, but what does that have
to do with the system's time resolution?  And how is the 0.625 msec
number reported by the program obtained from those samples?

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

On 4/20/22 12:14, Eli Zaretskii wrote:

Sorry, my bad.  The result is the same, but I do get printouts.  What
do you want to know or see from there?


I want to see what the current_timespec's resolution is, which we should 
be able to tell from the debugging output. For example, on my Solaris 10 
sparc platform the command 'gltests/test-gettime-res x' outputs:


gettime_res returned 200 ns
time = 1650482432.256445600
time = 1650482432.256460600
time = 1650482432.256464400
time = 1650482432.256468200
time = 1650482432.256471400
time = 1650482432.256474600
time = 1650482432.256478000
time = 1650482432.256481200
time = 1650482432.256484800
...

and these timestamps say that with very high probability 
current_timespec's clock resolution is indeed 200 ns.

Re: [PATCH] Re: No mathematics in Texinfo exports

[Rudolf Adamkovič]

Hello everyone,

My patch (from almost a month ago) has yet to appear at

https://updates.orgmode.org

Any ideas what to do?  Could someone review it?

Thank you!

Rudy

+++ The original message below. +++

Rudolf Adamkovič  writes:

> Rudolf Adamkovič  writes:
>
>> […], or do I have to tackle the problem myself?  I would appreciate
>> any tips.
>
> I pulled up my sleeves and added the functionality myself.  See the
> attached patch.  Please note that I have never contributed patches via
> mail, and I have never signed any FSF papers.  I would appreciate your
> guidance.  Thank you!
>
> From 5fe65432c1a6440c86d0bbc0b66a6603e5a8f100 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Rudolf=20Adamkovi=C4=8D?= 
> Date: Sat, 26 Mar 2022 16:46:47 +0100
> Subject: [PATCH] ox-texinfo: Include LaTeX in Texinfo exports
>
> * lisp/ox-texinfo.el (org-texinfo-latex-environment): New function.
> * lisp/ox-texinfo.el (org-texinfo-latex-fragment): New function.
> * lisp/ox-texinfo.el (texinfo): Set latex-environment.
> * lisp/ox-texinfo.el (texinfo): Set latex-fragment.
> * testing/lisp/test-ox-texinfo.el: Add basic tests.
>
> Include (La)TeX mathematics, both inline and display style, in Texinfo
> exports.
> ---
>  lisp/ox-texinfo.el  |  42 ++
>  testing/lisp/test-ox-texinfo.el | 221 
>  2 files changed, 263 insertions(+)
>  create mode 100644 testing/lisp/test-ox-texinfo.el
>
> diff --git a/lisp/ox-texinfo.el b/lisp/ox-texinfo.el
> index a01bb268c..0bfd06550 100644
> --- a/lisp/ox-texinfo.el
> +++ b/lisp/ox-texinfo.el
> @@ -55,6 +55,8 @@
>  (italic . org-texinfo-italic)
>  (item . org-texinfo-item)
>  (keyword . org-texinfo-keyword)
> +(latex-environment . org-texinfo-latex-environment)
> +(latex-fragment . org-texinfo-latex-fragment)
>  (line-break . org-texinfo-line-break)
>  (link . org-texinfo-link)
>  (node-property . org-texinfo-node-property)
> @@ -1212,6 +1214,46 @@ CONTENTS is nil.  INFO is a plist holding contextual 
> information."
> (concat "@listoffloats "
> (org-export-translate "Listing" :utf-8 info
>  
> + LaTeX Environment
> +
> +(defun org-texinfo-latex-environment (environment _contents info)
> +  "Transcode a LaTeX ENVIRONMENT from Org to Texinfo.  CONTENTS is
> +nil.  INFO is a plist holding contextual information."
> +  (when (plist-get info :with-latex)
> +(let ((value (org-element-property :value environment)))
> +  (string-join (list "@displaymath"
> + (string-trim (org-remove-indentation value))
> + "@end displaymath")
> +   "\n"
> +
> + LaTeX Fragment
> +
> +(defun org-texinfo-latex-fragment (fragment _contents info)
> +  "Transcode a LaTeX FRAGMENT from Org to Texinfo.  CONTENTS is
> +nil.  INFO is a plist holding contextual information."
> +  (when (plist-get info :with-latex)
> +(let ((value (org-remove-indentation
> +  (org-element-property :value fragment
> +  (cond
> +   ((or (string-match-p "^\\[" value)
> +(string-match-p "^\\$\\$" value))
> +(concat "\n"
> +"@displaymath"
> +"\n"
> +(string-trim (substring value 2 -2))
> +"\n"
> +"@end displaymath"
> +"\n"))
> +   ((string-match-p "^\\$" value)
> +(concat "@math{"
> +(string-trim (substring value 1 -1))
> +"}"))
> +   ((string-match-p "^(" value)
> +(concat "@math{"
> +(string-trim (substring value 2 -2))
> +"}"))
> +   (t value)
> +
>   Line Break
>  
>  (defun org-texinfo-line-break (_line-break _contents _info)
> diff --git a/testing/lisp/test-ox-texinfo.el b/testing/lisp/test-ox-texinfo.el
> new file mode 100644
> index 0..316b7cb1d
> --- /dev/null
> +++ b/testing/lisp/test-ox-texinfo.el
> @@ -0,0 +1,221 @@
> +;;; test-ox-texinfo.el --- Tests for ox-texinfo.el
> +
> +;; Copyright (C) 2022  Rudolf Adamkovič
> +
> +;; Author: Rudolf Adamkovič 
> +
> +;; This file is not part of GNU Emacs.
> +
> +;; This program is free software; you can redistribute it and/or modify
> +;; it under the terms of the GNU General Public License as published by
> +;; the Free Software Foundation, either version 3 of the License, or
> +;; (at your option) any later version.
> +
> +;; This program is distributed in the hope that it will be useful,
> +;; but WITHOUT ANY WARRANTY; without even the implied warranty of
> +;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +;; GNU General Public License for more details.
> +
> +;; You should have received a copy of the GNU General Public License
> +;; along with this program.  If not, see .
> +
> +;;; Code:
> +
> +(require 'cl-lib)
> +(require 'ox-texinfo)
> +
> +(unless (featurep 'ox-texinfo)
> +  (signal 'missi

Re: Maintenance Request

[Edouard Debry]

Thanks for offering to maintain 'ox-latex.el'.

One of the thing that could be improved is the generation of svg
images through tikz. Currently, unless I am mistaken, it does
not work well. I had to tweak 'ox-latex.el' to make it work again.
I did post a question about this on the mailing list.

Regards

Bastien  writes:

> Hi Daniel,
>
> Daniel Fleischer  writes:
>
>> I would like to maintain the 'ox-latex.el' file. It's one of the many
>> files created by Nicolas but few of the files that are not maintained
>> (by him). I have a long experience (14) with latex and relatively long
>> experience with org->latex.
>
> Sorry for the late answer and thanks for proposing this.
>
> I added you as the ox-latex.el maintainer, assuming this is okay for
> Nicolas (Nicolas, let me know if I'm wrong.)
>
> Thanks!

Re: bug#54764: encode-time: make DST and TIMEZONE fields of the list argument optional ones

[Paul Eggert]

On 4/20/22 12:30, Eli Zaretskii wrote:


I see the time samples change in jumps of 15 msec.


Could you give the first part of the output? I would like to see what 
the the samples are jumping from and to, and how often they jump.


Something like the following is what I'd hope to see from the first 
lines of the output of 'gllib/test-gettime-res x'. What are you seeing?


gettime_res returned 15625000 ns
time = 1650496481.515625000
time = 1650496481.53125
time = 1650496481.546875000
time = 1650496481.56250
time = 1650496481.578125000
time = 1650496481.59375
time = 1650496481.609375000
time = 1650496481.62500
time = 1650496481.640625000
time = 1650496481.65625


 Which is expected
on MS-Windows, given the scheduler time tick, but what does that have
to do with the system's time resolution?


The resolution of Elisp's (time-convert nil t) is determined by the 
smallest nonzero gap between timestamps that are returned by C's 
current_timespec. This is the system time resolution as far as Elisp is 
concerned, because Elisp cannot return the current time at a finer 
resolution than what current_timespec gives it. This resolution is not 
necessarily the same as the time resolution of the motherboard clock, OS 
high-res timestamp, file timestamps, etc.



And how is the 0.625 msec
number reported by the program obtained from those samples?


Use the largest resolution R ns consistent with the samples, such that 
10 is an integer multiple of R so that timestamp computations 
are exact.

Fix M-j with default fill-prefix value

[Javier Olaechea]

org-comment-line-break-function does not handle fill-prefix being set to
nil, which is the default value for fill-prefix. This means that pressing
M-j inside an org-mode buffer in a vanilla installation of Emacs results in
an error. From looking at other callers of
insert-before-markers-and-inherit it is clear that a guard against
fill-prefix being nil is missing.

This has been reported beforehttps://
list.orgmode.org/87o861o9sh.fsf@aquinas.i-did-not-set--mail-host-address--so-tickle-me/

Cheers,
Javier Olaechea

-- 
"I object to doing things that computers can do." — Olin Shivers
From 372daea6cbee60df03f1a68bd5fabca4212a12c3 Mon Sep 17 00:00:00 2001
From: Javier Olaechea 
Date: Wed, 20 Apr 2022 11:58:07 -0500
Subject: [PATCH] lisp/org.el: Allow org-c-l-b-f to handle an empty fill-prefix

* org.el (org-comment-line-break-function): When fill-prefix is nil,
don't call `insert-before-markers-and-inherit'.

The default value for fill-prefix is nil,
`insert-before-markers-and-inherit' expects a string. This results in
an error when calling `org-comment-line-break-function', bound to M-j,
on a vanilla installation of Emacs. We should add a guard checking
that fill-prefix is not nill before calling i-b-m-a-i. The only
exception is when adaptive-fill-mode is enabled. This is the approach
that that rest of the code in Emacs takes. Some call-sites listed
below:

* simple.el (default-indent-new-line): uses
  (and fill-prefix (not adaptive-fill-mode) ...)

* newcomment.el (comment-indent-new-line): uses
  (and fill-prefix (not adaptive-fill-mode) ...)

* fill.el (fill-new-line): uses
  (and fill-prefix (not (equal fill-prefix "")) ...)

TINYCHANGE
---
 lisp/org.el | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lisp/org.el b/lisp/org.el
index 2353c6594..4176c683f 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -19943,7 +19943,8 @@ (defun org-comment-line-break-function (&optional soft)
   (save-excursion (forward-char -1) (delete-horizontal-space))
   (delete-horizontal-space)
   (indent-to-left-margin)
-  (insert-before-markers-and-inherit fill-prefix))
+  (and fill-prefix (not adaptive-fill-mode)
+   (insert-before-markers-and-inherit fill-prefix)))
 
 
 ;;; Fixed Width Areas
-- 
2.29.2.154.g7f7ebe054a

[PATCH] org-agenda: Use `window-max-chars-per-line' instead of `window-width'

[Ihor Radchenko]

"N. Jackson"  writes:

> No, I'm not saying Org shouldn't use window-max-chars-per-line --
> rather the opposite in fact. I'm just saying it maybe needs to be
> thought about carefully first.

Indeed. There is no point using w-m-c-p-l when we actually need the full
window size with fringes.

> If it works as advertised it would seem to be the right choice as
> long as Emacs bug #19395 exists (which looks like it will be
> forever). My reservations about the function are only that its spec
> is more ambitious than that of window-width (because it handles
> faces) and the added complexity might potentially introduce more
> corner cases where it doesn't work. But perhaps that's just the
> paranoia of someone who thinks that text editors should restrict
> themselves to monospaced fonts!

There is at least one workaround in org-agenda trying to account for the
face width. w-m-c-p-l should do better than that.

Attaching the patch.

Best,
Ihor

>From 5de3de87872ddae994f5b68eba74d915d5f73201 Mon Sep 17 00:00:00 2001
Message-Id: <5de3de87872ddae994f5b68eba74d915d5f73201.1650518543.git.yanta...@gmail.com>
From: Ihor Radchenko 
Date: Thu, 21 Apr 2022 13:18:39 +0800
Subject: [PATCH] org-agenda: Use `window-max-chars-per-line' instead of
 `window-width'

* lisp/org-agenda.el (org-agenda-prepare):
(org-todo-list):
(org-agenda-show-new-time): Calculate the maximum number of chars
fitting into window via `window-max-chars-per-line' instead of
`window-with'.  The latter can return larger number when fringes are
disabled [1].

[1] https://debbugs.gnu.org/cgi/bugreport.cgi?bug=19395

Reported in https://orgmode.org/list/87czhdqi9p.fsf_-_@moondust.localdomain
---
 lisp/org-agenda.el | 9 +++--
 1 file changed, 3 insertions(+), 6 deletions(-)

diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index a09b53563..9ddee81ee 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -3925,7 +3925,7 @@ (defun org-agenda-prepare (&optional name)
 	  (insert "\n"
 		  (if (stringp org-agenda-block-separator)
 			  org-agenda-block-separator
-			(make-string (window-width) org-agenda-block-separator))
+			(make-string (window-max-chars-per-line) org-agenda-block-separator))
 		  "\n"))
 	(narrow-to-region (point) (point-max)))
 	(setq org-done-keywords-for-agenda nil)
@@ -4944,7 +4944,7 @@ (defun org-todo-list (&optional arg)
 	(let ((n 0))
   (dolist (k kwds)
 (let ((s (format "(%d)%s" (cl-incf n) k)))
-  (when (> (+ (current-column) (string-width s) 1) (window-width))
+  (when (> (+ (current-column) (string-width s) 1) (window-max-chars-per-line))
 (insert "\n "))
   (insert " " s
 	(insert "\n"))
@@ -10301,10 +10301,7 @@ (defun org-agenda-show-new-time (marker stamp &optional prefix)
   (line-end-position)
   '(display nil))
 	  (org-move-to-column
-   (- (if (fboundp 'window-font-width)
-  (/ (window-width nil t) (window-font-width))
-;; Fall back to pre-9.3.3 behavior on Emacs <25.
-(window-width))
+   (- (window-max-chars-per-line)
   (length stamp))
t)
   (add-text-properties
-- 
2.35.1

Re: [PATCH] ol: Fix org-link-search

[Ihor Radchenko]

tony aldon  writes:

> If I'm not wrong, name defined after #+NAME: should be match first
> before trying a fuzzy search in the function `org-link-search`.
> ...
> - (name (org-element-property :name element)))
> + (name (org-element-property :value element)))

I think you are missing something.

A standaline

#+name: some name

is a keyword, not an element name, but a generic keyword element.

Named elements are the elements with affiliated keywords (not ordinary
keywords):

#+name: some other name
Some element that can have affiliated keywords. It may be a paragraph,
like here or e.g. source block. Link search should not try to search
standalone keywords and hence it is sufficient to check for :name
element property. :value will miss e.g. named source blocks.

Try running (org-element-at-point) on the first and second #+name in org
buffer.

Best,
Ihor

Re: [PATCH v2 00/38] Final call for comments: Merge org-fold feature branch

[Ihor Radchenko]

Bastien  writes:

> I'm on commit e0abbbacf9427b69482c6c47c3ea0975b0e6fa6d from the
> feature/org-fold-universal-core branch of your org repo.
>
> Using Emacs 29.0.50 and this as the content for ~/test.org:
>
> * TODO [[https://orgmode.org][test]]
> SCHEDULED: <2022-04-20 mer.>
>
> Then run:
>
> - emacs -q
> - C-x C-f test.org
> - M-x org-agenda RET
> - < a . n t
>
> You should see the link losing its invisibility specs.

I did see it in fork/feature/org-fold-universal-core and it is very
strange. The properties are suddenly lost after running
add-text-properties and things become normal if I simply move
add-text-properties above org-fold-region in
org-activate-links--text-properties.

What is more strange, the actual patch I proposed
(fork/feature/org-fold-universal-core-tidy) does not have the problem.
At least, I cannot reproduce it with you recipe. And the patch have
exactly same org-activate-links--text-properties...

Since the actual patch does not have the problem, I'd prefer to ignore
this problem unless it appears again after merging.

Best,
Ihor

Re: [PATCH] Re: No mathematics in Texinfo exports

[Ihor Radchenko]

Rudolf Adamkovič  writes:

> My patch (from almost a month ago) has yet to appear at
>
> https://updates.orgmode.org

This is strange. Marking the patch manually.
In any case, thanks for your contribution and patience.

> Any ideas what to do?  Could someone review it?

The idea sounds good and having tests is very good. Though I am not
expert in texinfo. CC-ing Nicolas. He is the maintainer.

A more general comment is rather about our org-element.el (maybe Nicolas
can answer)

> +  (cond
> +   ((or (string-match-p "^\\[" value)
> +(string-match-p "^\\$\\$" value))
> ...
> +   ((string-match-p "^\\$" value)
> ...
> +   ((string-match-p "^(" value)

Why don't we just have an extra element property with the bracket info
in the output of org-element-latex-fragment-parser?

> Please note that I have never contributed patches via mail, and I have
> never signed any FSF papers. I would appreciate your guidance.

See https://orgmode.org/worg/org-contribute.html#copyright

Best,
Ihor