Re: org-element persist logic hangs when quitting emacs

[Ihor Radchenko]

Kaushal Modi  writes:

> Lately (at least as of last 3 days), I have seen that when I try to
> quit emacs, it gets stuck on one of my Org files; it's a personal
> journal.org that I frequently update using org-capture. So that file
> is usually open in one of the buffers.

Thanks for the report!

Could you try running M-: (org-element-at-point (point-max)) in your
journal.org and let me know if it takes a very long time?

Best,
Ihor

[PATCH 00/35] Merge org-fold feature branch

[Ihor Radchenko]

It took a while, but I am finally done with rebasing the org-fold
branch code onto current main.  This branch has been tested by me and
other volunteers for over a year.  Things are basically stable using
recent Emacs versions.  There were a couple of back-compatibility
issues with older Emacs, which I fixed during the cleanup.  Those may
need to be tested more carefully after merging to main.

I would like to thank all the people who helped with bug reporting and
provided bugfixes over the testing time. Thank you all - arkhan,
HyunggyuJang, Robert Irelan, Alois Janíček, Anders Johansson, Daniel
Kraus, Ypot, ntharim, Colin McLear, Yiming Chen, tpeacock19, and Karl
Voit.

After the cleanup, some patches are not included for the merge.  Apart
from my patches, 3 patches by Anders Johansson are included here.  He
has signed FSF copyright paperwork and appears on the Org mode
contributor list (https://orgmode.org/worg/contributors.html).

Unless there are comments on the patches below, I plan to merge the
branch in the coming weeks.  Please, let me know if I still need to
wait for comments.

Anders Johansson (3):
  Fix typo: delete-duplicates → delete-dups
  Fix bug in org-get-heading
  Rename remaining org-force-cycle-archived → org-cycle-force-archived

Ihor Radchenko (32):
  Add org-fold-core: new folding engine
  Separate folding functions from org.el into new library: org-fold
  Separate cycling functions from org.el into new library: org-cycle
  Remove functions from org.el that are now moved elsewhere
  Disable native-comp in agenda
  org-macs: New function org-find-text-property-region
  org-at-heading-p: Accept optional argument
  org-string-width: Reimplement to work with new folding
  Rename old function call to use org-fold
  Implement link folding
  Implement overlay- and text-property-based versions of some functions
  org-fold: Handle indirect buffer visibility
  Fix subtle differences between overlays and invisible text properties
  Support extra org-fold optimisations for huge buffers
  Alias new org-fold functions to their old shorter names
  Obsolete old function names that are now in org-fold
  org-compat: Work around some third-party packages using outline-*
functions
  Move `org-buffer-list' to org-macs.el
  Restore old visibility behaviour of org-refile
  Add org-fold-related tests
  org-manual: Update to new org-fold function names
  ORG-NEWS: Add list of changes
  Backport contributed commits
  Fix org-fold--hide-drawers--overlays
  org-string-width: Handle undefined behaviour in older Emacs
  org-string-width: Work around `window-pixel-width' bug in old Emacs
  org-fold-show-set-visibility: Fix edge case when folded region is at
BOB
  org-fold-core: Fix fontification inside folded regions
  test-org/string-width: Add tests for strings with prefix properties
  org--string-from-props: Fix handling folds in Emacs <28
  org-link-make-string: Throw error when both LINK and DESCRIPTION are
empty
  test-ol/org-toggle-link-display: Fix compatibility with old Emacs

 doc/org-manual.org|   14 +-
 etc/ORG-NEWS  |  104 ++
 lisp/ob-core.el   |   14 +-
 lisp/ob-lilypond.el   |4 +-
 lisp/ob-ref.el|4 +-
 lisp/ol.el|   59 +-
 lisp/org-agenda.el|   50 +-
 lisp/org-archive.el   |   12 +-
 lisp/org-capture.el   |7 +-
 lisp/org-clock.el |  126 +-
 lisp/org-colview.el   |   10 +-
 lisp/org-compat.el|  189 ++-
 lisp/org-crypt.el |8 +-
 lisp/org-cycle.el |  818 +++
 lisp/org-element.el   |   55 +-
 lisp/org-feed.el  |4 +-
 lisp/org-fold-core.el | 1503 +++
 lisp/org-fold.el  | 1132 +++
 lisp/org-footnote.el  |6 +-
 lisp/org-goto.el  |6 +-
 lisp/org-id.el|4 +-
 lisp/org-inlinetask.el|   26 +-
 lisp/org-keys.el  |   26 +-
 lisp/org-lint.el  |3 +-
 lisp/org-list.el  |   84 +-
 lisp/org-macs.el  |  290 +++-
 lisp/org-mobile.el|2 +-
 lisp/org-mouse.el |4 +-
 lisp/org-refile.el|3 +-
 lisp/org-src.el   |6 +-
 lisp/org-timer.el |2 +-
 lisp/org.el   | 2552 +++--
 lisp/ox-org.el|2 +-
 lisp/ox.el|4 +-
 testing/lisp/test-ob.el   |   12 +-
 testing/lisp/test-ol.el   |   24 +
 testing/lisp/test-org-list.el |   75 +-
 testing/lisp/test-org-macs.el |6 +-
 testing/lisp/test-org.el  |  258 +++-
 39 files changed, 5475 insertions(+), 2033 deletions(-)
 create mode 100644 lisp/org-cycle.el
 create mode 100644 lisp/org-fold-core.el
 create mode 100644 lisp/org-fold.el

-- 
2.34.1

[PATCH 01/35] Add org-fold-core: new folding engine

[Ihor Radchenko]

---
 lisp/org-fold-core.el | 1490 +
 1 file changed, 1490 insertions(+)
 create mode 100644 lisp/org-fold-core.el

diff --git a/lisp/org-fold-core.el b/lisp/org-fold-core.el
new file mode 100644
index 0..121c6b5c4
--- /dev/null
+++ b/lisp/org-fold-core.el
@@ -0,0 +1,1490 @@
+;;; org-fold-core.el --- Folding buffer text -*- lexical-binding: t; -*-
+;;
+;; Copyright (C) 2020-2020 Free Software Foundation, Inc.
+;;
+;; Author: Ihor Radchenko 
+;; Keywords: folding, invisible text
+;; Homepage: https://orgmode.org
+;;
+;; This file is part of GNU Emacs.
+;;
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see .
+;
+;;
+;;; Commentary:
+
+;; This file contains library to control temporary invisibility
+;; (folding and unfolding) of text in buffers.
+
+;; The file implements the following functionality:
+;;
+;; - Folding/unfolding regions of text
+;; - Searching and examining boundaries of folded text
+;; - Interactive searching in folded text (via isearch)
+;; - Handling edits in folded text
+;; - Killing/yanking (copying/pasting) of the folded text
+;; - Fontification of the folded text
+
+;; To setup folding in an arbitrary buffer, one must call
+;; `org-fold-core-initialize', optionally providing the list of folding specs to be
+;; used in the buffer.  The specs can be added, removed, or
+;; re-configured later.  Read below for more details.
+
+;;; Folding/unfolding regions of text
+
+;; User can temporarily hide/reveal (fold/unfold) arbitrary regions or
+;; text.  The folds can be nested.
+
+;; Internally, nested folds are marked with different folding specs
+;; Overlapping folds marked with the same folding spec are
+;; automatically merged, while folds with different folding specs can
+;; coexist and be folded/unfolded independently.
+
+;; When multiple folding specs are applied to the same region of text,
+;; text visibility is decided according to the folding spec with
+;; topmost priority.
+
+;; By default, we define two types of folding specs:
+;; - 'org-fold-visible :: the folded text is not hidden
+;; - 'org-fold-hidden  :: the folded text is completely hidden
+;;
+;; The 'org-fold-visible spec has highest priority allowing parts of
+;; text folded with 'org-fold-hidden to be shown unconditionally.
+
+;; Consider the following Org mode link:
+;; [[file:/path/to/file/file.ext][description]]
+;; Only the word "description" is normally visible in this link.
+;; 
+;; The way this partial visibility is achieved is combining the two
+;; folding specs.  The whole link is folded using 'org-fold-hidden
+;; folding spec, but the visible part is additionally folded using
+;; 'org-fold-visible:
+;;
+;; [[file:/path/to/file/file.ext][description]]
+;; 
+;; Because 'org-fold-visible has higher priority than
+;; 'org-fold-hidden, it suppresses the 'org-fold-hidden effect and
+;; thus reveals the description part of the link.
+
+;; Similar to 'org-fold-visible, display of any arbitrary folding spec
+;; can be configured using folding spec properties.  In particular,
+;; `:visible' folding spec proprety controls whether the folded text
+;; is visible or not.  If the `:visible' folding spec property is nil,
+;; folded text is hidden or displayed as a constant string (ellipsis)
+;; according to the value of `:ellipsis' folding spec property.  See
+;; docstring of `org-fold-core--specs' for the description of all the available
+;; folding spec properties.
+
+;; Folding spec properties of any valid folding spec can be changed
+;; any time using `org-fold-core-set-folding-spec-property'.
+
+;; If necessary, one can add or remove folding specs using
+;; `org-fold-core-add-folding-spec' and `org-fold-core-remove-folding-spec'.
+
+;; If a buffer initialised with `org-fold-core-initialize' is cloned into indirect
+;; buffers, it's folding state is copied to that indirect buffer.
+;; The folding states are independent.
+
+;; When working with indirect buffers that are handled by this
+;; library, one has to keep in mind that folding state is preserved on
+;; copy when using non-interactive functions.  Moreover, the folding
+;; states of all the indirect buffers will be copied together.
+;;
+;; Example of the implications:
+;; Consider a base buffer and indirect buffer with the following state:
+;; - base buffer 

[PATCH 02/35] Separate folding functions from org.el into new library: org-fold

[Ihor Radchenko]

---
 lisp/org-fold.el | 1135 ++
 1 file changed, 1135 insertions(+)
 create mode 100644 lisp/org-fold.el

diff --git a/lisp/org-fold.el b/lisp/org-fold.el
new file mode 100644
index 0..52717fd86
--- /dev/null
+++ b/lisp/org-fold.el
@@ -0,0 +1,1135 @@
+;;; org-fold.el --- Folding of Org entries -*- lexical-binding: t; -*-
+;;
+;; Copyright (C) 2020-2020 Free Software Foundation, Inc.
+;;
+;; Author: Ihor Radchenko 
+;; Keywords: folding, invisible text
+;; Homepage: https://orgmode.org
+;;
+;; This file is part of GNU Emacs.
+;;
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see .
+;
+;;
+;;; Commentary:
+
+;; This file contains code handling temporary invisibility (folding
+;; and unfolding) of text in org buffers.
+
+;; The folding is implemented using generic org-fold-core library.  This file
+;; contains org-specific implementation of the folding.  Also, various
+;; useful functions from org-fold-core are aliased under shorted `org-fold'
+;; prefix.
+
+;; The following features are implemented:
+;; - Folding/unfolding various Org mode elements and regions of Org buffers:
+;;   + Region before first heading;
+;;   + Org headings, their text, children (subtree), siblings, parents, etc;
+;;   + Org blocks and drawers
+;; - Revealing Org structure around invisible point location
+;; - Revealing folded Org elements broken by user edits
+
+;;; Code:
+
+(require 'org-macs)
+(require 'org-fold-core)
+
+(defvar org-inlinetask-min-level)
+(defvar org-link--link-folding-spec)
+(defvar org-link--description-folding-spec)
+(defvar org-odd-levels-only)
+(defvar org-drawer-regexp)
+(defvar org-property-end-re)
+(defvar org-link-descriptive)
+(defvar org-outline-regexp-bol)
+(defvar org-custom-properties-hidden-p)
+(defvar org-archive-tag)
+
+;; Needed for overlays only
+(defvar org-custom-properties-overlays)
+
+(declare-function isearch-filter-visible "isearch" (beg end))
+(declare-function org-element-type "org-element" (element))
+(declare-function org-element-at-point "org-element" (&optional pom cached-only))
+(declare-function org-element-property "org-element" (property element))
+(declare-function org-element--current-element "org-element" (limit &optional granularity mode structure))
+(declare-function org-element--cache-active-p "org-element" ())
+(declare-function org-toggle-custom-properties-visibility "org" ())
+(declare-function org-item-re "org-list" ())
+(declare-function org-up-heading-safe "org" ())
+(declare-function org-get-tags "org" (&optional pos local fontify))
+(declare-function org-get-valid-level "org" (level &optional change))
+(declare-function org-before-first-heading-p "org" ())
+(declare-function org-goto-sibling "org" (&optional previous))
+(declare-function org-block-map "org" (function &optional start end))
+(declare-function org-map-region "org" (fun beg end))
+(declare-function org-end-of-subtree "org" (&optional invisible-ok to-heading))
+(declare-function org-back-to-heading-or-point-min "org" (&optional invisible-ok))
+(declare-function org-back-to-heading "org" (&optional invisible-ok))
+(declare-function org-at-heading-p "org" (&optional invisible-not-ok))
+(declare-function org-cycle-hide-drawers "org-cycle" (state))
+
+(declare-function outline-show-branches "outline" ())
+(declare-function outline-hide-sublevels "outline" (levels))
+(declare-function outline-get-next-sibling "outline" ())
+(declare-function outline-invisible-p "outline" (&optional pos))
+(declare-function outline-next-heading "outline" ())
+
+;;; Customization
+
+(defgroup org-fold-reveal-location nil
+  "Options about how to make context of a location visible."
+  :tag "Org Reveal Location"
+  :group 'org-structure)
+
+(defcustom org-fold-show-context-detail '((agenda . local)
+  (bookmark-jump . lineage)
+  (isearch . lineage)
+  (default . ancestors))
+  "Alist between context and visibility span when revealing a location.
+
+\\Some actions may move point into invisible
+locations.  As a consequence, Org always exposes a neighborhood
+around point.  How much is shown depends on the initial action,
+or context.  Valid contexts are
+
+  agenda when exposing an entry from the agenda
+  org-goto   when using the command `org-goto' (`\\[org-goto]')
+  occur-tree when using t

[PATCH 04/35] Remove functions from org.el that are now moved elsewhere

[Ihor Radchenko]

---
 lisp/org.el | 1272 ++-
 1 file changed, 40 insertions(+), 1232 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index b3c5f3104..d279edae4 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -1178,90 +1178,6 @@ (defgroup org-structure nil
   :tag "Org Structure"
   :group 'org)
 
-(defgroup org-reveal-location nil
-  "Options about how to make context of a location visible."
-  :tag "Org Reveal Location"
-  :group 'org-structure)
-
-(defcustom org-show-context-detail '((agenda . local)
- (bookmark-jump . lineage)
- (isearch . lineage)
- (default . ancestors))
-  "Alist between context and visibility span when revealing a location.
-
-\\Some actions may move point into invisible
-locations.  As a consequence, Org always exposes a neighborhood
-around point.  How much is shown depends on the initial action,
-or context.  Valid contexts are
-
-  agenda when exposing an entry from the agenda
-  org-goto   when using the command `org-goto' (`\\[org-goto]')
-  occur-tree when using the command `org-occur' (`\\[org-sparse-tree] /')
-  tags-tree  when constructing a sparse tree based on tags matches
-  link-searchwhen exposing search matches associated with a link
-  mark-goto  when exposing the jump goal of a mark
-  bookmark-jump  when exposing a bookmark location
-  isearchwhen exiting from an incremental search
-  defaultdefault for all contexts not set explicitly
-
-Allowed visibility spans are
-
-  minimalshow current headline; if point is not on headline,
- also show entry
-
-  local  show current headline, entry and next headline
-
-  ancestors  show current headline and its direct ancestors; if
- point is not on headline, also show entry
-
-  ancestors-full show current subtree and its direct ancestors
-
-  lineageshow current headline, its direct ancestors and all
- their children; if point is not on headline, also show
- entry and first child
-
-  tree   show current headline, its direct ancestors and all
- their children; if point is not on headline, also show
- entry and all children
-
-  canonical  show current headline, its direct ancestors along with
- their entries and children; if point is not located on
- the headline, also show current entry and all children
-
-As special cases, a nil or t value means show all contexts in
-`minimal' or `canonical' view, respectively.
-
-Some views can make displayed information very compact, but also
-make it harder to edit the location of the match.  In such
-a case, use the command `org-reveal' (`\\[org-reveal]') to show
-more context."
-  :group 'org-reveal-location
-  :version "26.1"
-  :package-version '(Org . "9.0")
-  :type '(choice
-	  (const :tag "Canonical" t)
-	  (const :tag "Minimal" nil)
-	  (repeat :greedy t :tag "Individual contexts"
-		  (cons
-		   (choice :tag "Context"
-			   (const agenda)
-			   (const org-goto)
-			   (const occur-tree)
-			   (const tags-tree)
-			   (const link-search)
-			   (const mark-goto)
-			   (const bookmark-jump)
-			   (const isearch)
-			   (const default))
-		   (choice :tag "Detail level"
-			   (const minimal)
-			   (const local)
-			   (const ancestors)
-   (const ancestors-full)
-			   (const lineage)
-			   (const tree)
-			   (const canonical))
-
 (defcustom org-indirect-buffer-display 'other-window
   "How should indirect tree buffers be displayed?
 
@@ -1453,130 +1369,6 @@ (defcustom org-bookmark-names-plist
   :group 'org-structure
   :type 'plist)
 
-(defgroup org-cycle nil
-  "Options concerning visibility cycling in Org mode."
-  :tag "Org Cycle"
-  :group 'org-structure)
-
-(defcustom org-cycle-skip-children-state-if-no-children t
-  "Non-nil means skip CHILDREN state in entries that don't have any."
-  :group 'org-cycle
-  :type 'boolean)
-
-(defcustom org-cycle-max-level nil
-  "Maximum level which should still be subject to visibility cycling.
-Levels higher than this will, for cycling, be treated as text, not a headline.
-When `org-odd-levels-only' is set, a value of N in this variable actually
-means 2N-1 stars as the limiting headline.
-When nil, cycle all levels.
-Note that the limiting level of cycling is also influenced by
-`org-inlinetask-min-level'.  When `org-cycle-max-level' is not set but
-`org-inlinetask-min-level' is, cycling will be limited to levels one less
-than its value."
-  :group 'org-cycle
-  :type '(choice
-	  (const :tag "No limit" nil)
-	  (integer :tag "Maximum level")))
-
-(defcustom org-hide-block-startup nil
-  "Non-nil means entering Org mode will fold all blocks.
-This can also be set in on a per-file basis with
-
-#+STARTUP: hideblocks
-#+STARTUP: showblocks"
-  :group 'org-startup
-  :group 'org-cycle
-  :type 'boolean)
-
-(defcustom org-cycle-global-

[PATCH 05/35] Disable native-comp in agenda

[Ihor Radchenko]

It caused cryptic bugs in the past.
---
 lisp/org-agenda.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index 3a6a4c1b9..72292fb4e 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -1,4 +1,4 @@
-;;; org-agenda.el --- Dynamic task and appointment lists for Org  -*- lexical-binding: t; -*-
+;;; org-agenda.el --- Dynamic task and appointment lists for Org  -*- lexical-binding: t; no-native-compile: t; -*-
 
 ;; Copyright (C) 2004-2022 Free Software Foundation, Inc.
 

[PATCH 14/35] Support extra org-fold optimisations for huge buffers

[Ihor Radchenko]

---
 lisp/org.el | 26 ++
 1 file changed, 18 insertions(+), 8 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index a59d550ca..22fce184e 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5787,6 +5787,7 @@ (defun org-unfontify-region (beg end &optional _maybe_loudly)
 			'(mouse-face t keymap t org-linked-text t
 	 invisible t intangible t
 	 org-emphasis t))
+(org-fold-core-update-optimisation beg end)
 (org-remove-font-lock-display-properties beg end)))
 
 (defconst org-script-display  '(((raise -0.3) (height 0.7))
@@ -6157,7 +6158,11 @@ (defun org-get-heading (&optional no-tags no-todo no-priority no-comment)
   (org-back-to-heading t)
   (let ((case-fold-search nil))
 	(looking-at org-complex-heading-regexp)
-	(let ((todo (and (not no-todo) (match-string 2)))
+;; When using `org-fold-core--optimise-for-huge-buffers',
+;; returned text may be invisible.  Clear it up.
+(save-match-data
+  (org-fold-core-remove-optimisation (match-beginning 0) (match-end 0)))
+(let ((todo (and (not no-todo) (match-string 2)))
 	  (priority (and (not no-priority) (match-string 3)))
 	  (headline (pcase (match-string 4)
 			  (`nil "")
@@ -6168,6 +6173,8 @@ (defun org-get-heading (&optional no-tags no-todo no-priority no-comment)
 			"" h))
 			  (h h)))
 	  (tags (and (not no-tags) (match-string 5
+  ;; Restore cleared optimisation.
+  (org-fold-core-update-optimisation (match-beginning 0) (match-end 0))
 	  (mapconcat #'identity
 		 (delq nil (list todo priority headline tags))
 		 " "))
@@ -6184,18 +6191,21 @@ (defun org-heading-components ()
   (save-excursion
 (org-back-to-heading t)
 (when (let (case-fold-search) (looking-at org-complex-heading-regexp))
-  (list (length (match-string 1))
-	(org-reduced-level (length (match-string 1)))
-	(match-string-no-properties 2)
-	(and (match-end 3) (aref (match-string 3) 2))
-	(match-string-no-properties 4)
-	(match-string-no-properties 5)
+  (org-fold-core-remove-optimisation (match-beginning 0) (match-end 0))
+  (prog1
+  (list (length (match-string 1))
+	(org-reduced-level (length (match-string 1)))
+	(match-string-no-properties 2)
+	(and (match-end 3) (aref (match-string 3) 2))
+	(match-string-no-properties 4)
+	(match-string-no-properties 5))
+(org-fold-core-update-optimisation (match-beginning 0) (match-end 0))
 
 (defun org-get-entry ()
   "Get the entry text, after heading, entire subtree."
   (save-excursion
 (org-back-to-heading t)
-(buffer-substring (point-at-bol 2) (org-end-of-subtree t
+(filter-buffer-substring (point-at-bol 2) (org-end-of-subtree t
 
 (defun org-edit-headline (&optional heading)
   "Edit the current headline.

[PATCH 03/35] Separate cycling functions from org.el into new library: org-cycle

[Ihor Radchenko]

---
 lisp/org-cycle.el | 818 ++
 1 file changed, 818 insertions(+)
 create mode 100644 lisp/org-cycle.el

diff --git a/lisp/org-cycle.el b/lisp/org-cycle.el
new file mode 100644
index 0..df0a3761a
--- /dev/null
+++ b/lisp/org-cycle.el
@@ -0,0 +1,818 @@
+;;; org-cycle.el --- Visibility cycling of Org entries -*- lexical-binding: t; -*-
+;;
+;; Copyright (C) 2020-2020 Free Software Foundation, Inc.
+;;
+;; Maintainer: Ihor Radchenko 
+;; Keywords: folding, visibility cycling, invisible text
+;; Homepage: https://orgmode.org
+;;
+;; This file is part of GNU Emacs.
+;;
+;; GNU Emacs is free software: you can redistribute it and/or modify
+;; it under the terms of the GNU General Public License as published by
+;; the Free Software Foundation, either version 3 of the License, or
+;; (at your option) any later version.
+
+;; GNU Emacs is distributed in the hope that it will be useful,
+;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;; GNU General Public License for more details.
+
+;; You should have received a copy of the GNU General Public License
+;; along with GNU Emacs.  If not, see .
+;
+;;
+;;; Commentary:
+
+;; This file contains code controlling global folding state in buffer
+;; and TAB-cycling.
+
+;;; Code:
+
+(require 'org-macs)
+(require 'org-fold)
+
+(declare-function org-element-type "org-element" (element))
+(declare-function org-element-property "org-element" (property element))
+(declare-function org-element-lineage "org-element" (datum &optional types with-self))
+(declare-function org-element-at-point "org-element" (&optional pom cached-only))
+(declare-function org-get-tags "org" (&optional pos local fontify))
+(declare-function org-subtree-end-visible-p "org" ())
+(declare-function org-narrow-to-subtree "org" (&optional element))
+(declare-function org-at-property-p "org" ())
+(declare-function org-re-property "org" (property &optional literal allow-null value))
+(declare-function org-item-beginning-re "org" ())
+(declare-function org-at-heading-p "org" (&optional invisible-not-ok))
+(declare-function org-at-item-p "org" ())
+(declare-function org-before-first-heading-p "org" ())
+(declare-function org-back-to-heading "org" (&optional invisible-ok))
+(declare-function org-end-of-subtree "org" (&optional invisible-ok to-heading))
+(declare-function org-entry-end-position "org" ())
+(declare-function org-try-cdlatex-tab "org" ())
+(declare-function org-cycle-level "org" ())
+(declare-function org-table-next-field "org-table" ())
+(declare-function org-table-justify-field-maybe "org-table" (&optional new))
+(declare-function org-inlinetask-at-task-p "org-inlinetask" ())
+(declare-function org-inlinetask-toggle-visibility "org-inlinetask" ())
+(declare-function org-list-get-all-items "org-list" (item struct prevs))
+(declare-function org-list-get-bottom-point "org-list" (struct))
+(declare-function org-list-prevs-alist "org-list" (struct))
+(declare-function org-list-set-item-visibility "org-list" (item struct view))
+(declare-function org-list-search-forward "org-list" (regexp &optional bound noerror))
+(declare-function org-list-has-child-p "org-list" (item struct))
+(declare-function org-list-get-item-end-before-blank "org-list" (item struct))
+(declare-function org-list-struct "org-list" ())
+(declare-function org-cycle-item-indentation "org-list" ())
+
+(declare-function outline-previous-heading "outline" ())
+(declare-function outline-next-heading "outline" ())
+(declare-function outline-end-of-heading "outline" ())
+(declare-function outline-up-heading "outline" (arg &optional invisible-ok))
+
+(defvar org-drawer-regexp)
+(defvar org-odd-levels-only)
+(defvar org-startup-folded)
+(defvar org-archive-tag)
+(defvar org-cycle-include-plain-lists)
+(defvar org-outline-regexp-bol)
+
+(defvar-local org-cycle-global-status nil)
+(put 'org-cycle-global-status 'org-state t)
+(defvar-local org-cycle-subtree-status nil)
+(put 'org-cycle-subtree-status 'org-state t)
+
+ Customisation:
+
+
+(defgroup org-cycle nil
+  "Options concerning visibility cycling in Org mode."
+  :tag "Org Cycle"
+  :group 'org-structure)
+
+(defcustom org-cycle-skip-children-state-if-no-children t
+  "Non-nil means skip CHILDREN state in entries that don't have any."
+  :group 'org-cycle
+  :type 'boolean)
+
+(defcustom org-cycle-max-level nil
+  "Maximum level which should still be subject to visibility cycling.
+Levels higher than this will, for cycling, be treated as text, not a headline.
+When `org-odd-levels-only' is set, a value of N in this variable actually
+means 2N-1 stars as the limiting headline.
+When nil, cycle all levels.
+Note that the limiting level of cycling is also influenced by
+`org-inlinetask-min-level'.  When `org-cycle-max-level' is not set but
+`org-inli

[PATCH 08/35] org-string-width: Reimplement to work with new folding

[Ihor Radchenko]

* lisp/org-macs.el (org--string-from-props): Removed since it is no
longer needed.
(org-string-width): Updated to use `window-text-pixel-size'.
---
 lisp/org-macs.el | 121 ++-
 1 file changed, 57 insertions(+), 64 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index d5d4c205d..8d156fa2f 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -883,71 +883,64 @@ (defun org-split-string (string &optional separators)
 		  results		;skip trailing separator
 		(cons (substring string i) results)))
 
-(defun org--string-from-props (s property beg end)
-  "Return the visible part of string S.
-Visible part is determined according to text PROPERTY, which is
-either `invisible' or `display'.  BEG and END are 0-indices
-delimiting S."
-  (let ((width 0)
-	(cursor beg))
-(while (setq beg (text-property-not-all beg end property nil s))
-  (let* ((next (next-single-property-change beg property s end))
-	 (props (text-properties-at beg s))
-	 (spec (plist-get props property))
-	 (value
-	  (pcase property
-		(`invisible
-		 ;; If `invisible' property in PROPS means text is to
-		 ;; be invisible, return 0.  Otherwise return nil so
-		 ;; as to resume search.
-		 (and (or (eq t buffer-invisibility-spec)
-			  (assoc-string spec buffer-invisibility-spec))
-		  0))
-		(`display
-		 (pcase spec
-		   (`nil nil)
-		   (`(space . ,props)
-		(let ((width (plist-get props :width)))
-		  (and (wholenump width) width)))
-		   (`(image . ,_)
-(and (fboundp 'image-size)
- (ceiling (car (image-size spec)
-		   ((pred stringp)
-		;; Displayed string could contain invisible parts,
-		;; but no nested display.
-		(org--string-from-props spec 'invisible 0 (length spec)))
-		   (_
-		;; Un-handled `display' value.  Ignore it.
-		;; Consider the original string instead.
-		nil)))
-		(_ (error "Unknown property: %S" property)
-	(when value
-	  (cl-incf width
-		   ;; When looking for `display' parts, we still need
-		   ;; to look for `invisible' property elsewhere.
-		   (+ (cond ((eq property 'display)
-			 (org--string-from-props s 'invisible cursor beg))
-			((= cursor beg) 0)
-			(t (string-width (substring s cursor beg
-		  value))
-	  (setq cursor next))
-	(setq beg next)))
-(+ width
-   ;; Look for `invisible' property in the last part of the
-   ;; string.  See above.
-   (cond ((eq property 'display)
-	  (org--string-from-props s 'invisible cursor end))
-	 ((= cursor end) 0)
-	 (t (string-width (substring s cursor end)))
-
-(defun org-string-width (string)
+(defun org-string-width (string &optional pixels)
   "Return width of STRING when displayed in the current buffer.
-Unlike `string-width', this function takes into consideration
-`invisible' and `display' text properties.  It supports the
-latter in a limited way, mostly for combinations used in Org.
-Results may be off sometimes if it cannot handle a given
-`display' value."
-  (org--string-from-props string 'display 0 (length string)))
+Return width in pixels when PIXELS is non-nil."
+  ;; Wrap/line prefix will make `window-text-pizel-size' return too
+  ;; large value including the prefix.
+  ;; Face should be removed to make sure that all the string symbols
+  ;; are using default face with constant width.  Constant char width
+  ;; is critical to get right string width from pixel width.
+  (remove-text-properties 0 (length string)
+  '(wrap-prefix t line-prefix t face t)
+  string)
+  (let (;; We need to remove the folds to make sure that folded table
+;; alignment is not messed up.
+(current-invisibility-spec
+ (or (and (not (listp buffer-invisibility-spec))
+  buffer-invisibility-spec)
+ (let (result)
+   (dolist (el buffer-invisibility-spec)
+ (unless (or (memq el
+   '(org-fold-drawer
+ org-fold-block
+ org-fold-outline))
+ (and (listp el)
+  (memq (car el)
+'(org-fold-drawer
+  org-fold-block
+  org-fold-outline
+   (push el result)))
+   result)))
+(current-char-property-alias-alist char-property-alias-alist))
+(with-temp-buffer
+  (setq-local display-line-numbers nil)
+  (setq-local buffer-invisibility-spec
+  current-invisibility-spec)
+  (setq-local char-property-alias-alist
+  current-char-property-alias-alist)
+  (let (pixel-width symbol-width)
+(with-silent-modifications
+  (setf (buffer-string) string)
+  (set

[PATCH 06/35] org-macs: New function org-find-text-property-region

[Ihor Radchenko]

---
 lisp/org-macs.el | 32 +---
 lisp/org.el  |  7 ++-
 2 files changed, 23 insertions(+), 16 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 3c902b603..d5d4c205d 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -722,7 +722,7 @@ (defsubst org-current-line (&optional pos)
 
 
 
-;;; Overlays
+;;; Overlays and text properties
 
 (defun org-overlay-display (ovl text &optional face evap)
   "Make overlay OVL display TEXT with face FACE."
@@ -745,20 +745,22 @@ (defun org-find-overlays (prop &optional pos delete)
 	(delete (delete-overlay ov))
 	(t (push ov found))
 
-(defun org-flag-region (from to flag spec)
-  "Hide or show lines from FROM to TO, according to FLAG.
-SPEC is the invisibility spec, as a symbol."
-  (remove-overlays from to 'invisible spec)
-  ;; Use `front-advance' since text right before to the beginning of
-  ;; the overlay belongs to the visible line than to the contents.
-  (when flag
-(let ((o (make-overlay from to nil 'front-advance)))
-  (overlay-put o 'evaporate t)
-  (overlay-put o 'invisible spec)
-  (overlay-put o
-		   'isearch-open-invisible
-		   (lambda (&rest _) (org-show-context 'isearch))
-
+(defun org-find-text-property-region (pos prop)
+  "Find a region around POS containing same non-nil value of PROP text property.
+Return nil when PROP is not set at POS."
+  (let* ((beg (and (get-text-property pos prop) pos))
+	 (end beg))
+(when beg
+  (unless (or (equal beg (point-min))
+		  (not (eq (get-text-property beg prop)
+			 (get-text-property (1- beg) prop
+	(setq beg (previous-single-property-change pos prop nil (point-min
+  (unless (or (equal end (point-max))
+		  ;; (not (eq (get-text-property end prop)
+		  ;; 	 (get-text-property (1+ end) prop)))
+		  )
+	(setq end (next-single-property-change pos prop nil (point-max
+  (cons beg end
 
 
 ;;; Regexp matching
diff --git a/lisp/org.el b/lisp/org.el
index d279edae4..f59d2cfb0 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5596,7 +5596,10 @@ (defun org-fontify-like-in-org-mode (s &optional odd-levels)
 (let ((org-odd-levels-only odd-levels))
   (org-mode)
   (org-font-lock-ensure)
-  (buffer-string
+  (if org-link-descriptive
+  (org-link-display-format
+   (buffer-string))
+(buffer-string)
 
 (defun org-get-level-face (n)
   "Get the right face for match N in font-lock matching of headlines."
@@ -5726,6 +5729,8 @@ (defun org-raise-scripts (limit)
 			   (if (equal (char-after (match-beginning 2)) ?^)
 			   (nth (if table-p 3 1) org-script-display)
 			 (nth (if table-p 2 0) org-script-display)))
+(put-text-property (match-beginning 2) (match-end 3)
+   'org-emphasis t)
 	(add-text-properties (match-beginning 2) (match-end 2)
 			 (list 'invisible t))
 	(when (and (eq (char-after (match-beginning 3)) ?{)

[PATCH 15/35] Alias new org-fold functions to their old shorter names

[Ihor Radchenko]

---
 lisp/org.el | 8 
 1 file changed, 8 insertions(+)

diff --git a/lisp/org.el b/lisp/org.el
index 22fce184e..cbdbf32f1 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -99,6 +99,14 @@ (require 'org-table)
 (require 'org-fold)
 
 (require 'org-cycle)
+(defvaralias 'org-hide-block-startup 'org-cycle-hide-block-startup)
+(defvaralias 'org-pre-cycle-hook 'org-cycle-pre-hook)
+(defvaralias 'org-tab-first-hook 'org-cycle-tab-first-hook)
+(defalias 'org-global-cycle #'org-cycle-global)
+(defalias 'org-overview #'org-cycle-overview)
+(defalias 'org-content #'org-cycle-content)
+(defalias 'org-reveal #'org-fold-reveal)
+(defalias 'org-force-cycle-archived #'org-cycle-force-archived)
 
 ;; `org-outline-regexp' ought to be a defconst but is let-bound in
 ;; some places -- e.g. see the macro `org-with-limited-levels'.

[PATCH 09/35] Rename old function call to use org-fold

[Ihor Radchenko]

---
 lisp/ob-core.el   |  14 ++--
 lisp/ob-lilypond.el   |   4 +-
 lisp/ob-ref.el|   4 +-
 lisp/ol.el|  13 ++--
 lisp/org-agenda.el|  43 +--
 lisp/org-archive.el   |  12 +--
 lisp/org-capture.el   |   2 +-
 lisp/org-clock.el |  10 +--
 lisp/org-colview.el   |   6 +-
 lisp/org-compat.el|  29 +++
 lisp/org-crypt.el |   8 +-
 lisp/org-element.el   |   1 +
 lisp/org-feed.el  |   4 +-
 lisp/org-footnote.el  |   6 +-
 lisp/org-goto.el  |   6 +-
 lisp/org-id.el|   4 +-
 lisp/org-keys.el  |  26 +++
 lisp/org-lint.el  |   3 +-
 lisp/org-list.el  |  10 ++-
 lisp/org-macs.el  |  40 ++
 lisp/org-mobile.el|   2 +-
 lisp/org-mouse.el |   4 +-
 lisp/org-refile.el|   2 +-
 lisp/org-src.el   |   6 +-
 lisp/org-timer.el |   2 +-
 lisp/org.el   | 137 +++---
 lisp/ox-org.el|   2 +-
 testing/lisp/test-org-list.el |   2 +-
 testing/lisp/test-org.el  |  78 +--
 29 files changed, 242 insertions(+), 238 deletions(-)

diff --git a/lisp/ob-core.el b/lisp/ob-core.el
index 239a57f96..65907 100644
--- a/lisp/ob-core.el
+++ b/lisp/ob-core.el
@@ -26,7 +26,9 @@ ;;; Code:
 (require 'cl-lib)
 (require 'ob-eval)
 (require 'org-macs)
+(require 'org-fold)
 (require 'org-compat)
+(require 'org-cycle)
 
 (defconst org-babel-exeext
   (if (memq system-type '(windows-nt cygwin))
@@ -50,7 +52,7 @@ (declare-function org-babel-ref-resolve "ob-ref" (ref))
 (declare-function org-babel-ref-split-args "ob-ref" (arg-string))
 (declare-function org-babel-tangle-comment-links "ob-tangle" (&optional info))
 (declare-function org-current-level "org" ())
-(declare-function org-cycle "org" (&optional arg))
+(declare-function org-cycle "org-cycle" (&optional arg))
 (declare-function org-edit-src-code "org-src" (&optional code edit-buffer-name))
 (declare-function org-edit-src-exit "org-src"  ())
 (declare-function org-element-at-point "org-element" (&optional pom cached-only))
@@ -75,7 +77,7 @@ (declare-function org-narrow-to-subtree "org" (&optional element))
 (declare-function org-next-block "org" (arg &optional backward block-regexp))
 (declare-function org-open-at-point "org" (&optional in-emacs reference-buffer))
 (declare-function org-previous-block "org" (arg &optional block-regexp))
-(declare-function org-show-context "org" (&optional key))
+(declare-function org-fold-show-context "org-fold" (&optional key))
 (declare-function org-src-coderef-format "org-src" (&optional element))
 (declare-function org-src-coderef-regexp "org-src" (fmt &optional label))
 (declare-function org-src-get-lang-mode "org-src" (lang))
@@ -945,7 +947,7 @@ (defun org-babel-enter-header-arg-w-completion (&optional lang)
 (insert (concat header " " (or arg "")))
 (cons header arg)))
 
-(add-hook 'org-tab-first-hook 'org-babel-header-arg-expand)
+(add-hook 'org-cycle-tab-first-hook 'org-babel-header-arg-expand)
 
 ;;;###autoload
 (defun org-babel-load-in-session (&optional _arg info)
@@ -1469,7 +1471,7 @@ (defun org-babel-hide-result-toggle (&optional force)
 	(push ov org-babel-hide-result-overlays)
 
 ;; org-tab-after-check-for-cycling-hook
-(add-hook 'org-tab-first-hook #'org-babel-hide-result-toggle-maybe)
+(add-hook 'org-cycle-tab-first-hook #'org-babel-hide-result-toggle-maybe)
 ;; Remove overlays when changing major mode
 (add-hook 'org-mode-hook
 	  (lambda () (add-hook 'change-major-mode-hook
@@ -1817,7 +1819,7 @@ (defun org-babel-goto-named-src-block (name)
   (let ((point (org-babel-find-named-block name)))
 (if point
 ;; Taken from `org-open-at-point'.
-(progn (org-mark-ring-push) (goto-char point) (org-show-context))
+(progn (org-mark-ring-push) (goto-char point) (org-fold-show-context))
   (message "source-code block `%s' not found in this buffer" name
 
 (defun org-babel-find-named-block (name)
@@ -1857,7 +1859,7 @@ (defun org-babel-goto-named-result (name)
   (let ((point (org-babel-find-named-result name)))
 (if point
 ;; taken from `org-open-at-point'
-(progn (goto-char point) (org-show-context))
+(progn (goto-char point) (org-fold-show-context))
   (message "result `%s' not found in this buffer" name
 
 (defun org-babel-find-named-result (name)
diff --git a/lisp/ob-lilypond.el b/lisp/ob-lilypond.el
index 15538b503..df128441a 100644
--- a/lisp/ob-lilypond.el
+++ b/lisp/ob-lilypond.el
@@ -34,7 +34,7 @@ ;;; Commentary:
 ;;; Code:
 (require 'ob)
 
-(declare-function org-show-all "org" (&optional types))
+(declare-function org-fold-show-all "org-fold" (&optional types))
 
 (defalias 'lilypond-mode 'LilyPond-mode)
 
@@ -279,7 +279,7 @@ (defun org-babel-lilypond-mark-error-line (file-name line)
 

[PATCH 12/35] org-fold: Handle indirect buffer visibility

[Ihor Radchenko]

---
 lisp/org-capture.el | 5 -
 lisp/org.el | 8 +++-
 2 files changed, 11 insertions(+), 2 deletions(-)

diff --git a/lisp/org-capture.el b/lisp/org-capture.el
index 1d4d6e877..08b35dd99 100644
--- a/lisp/org-capture.el
+++ b/lisp/org-capture.el
@@ -1171,7 +1171,10 @@ (defun org-capture-place-entry ()
   (goto-char (point-min))
   (unless (org-at-heading-p) (outline-next-heading)))
  ;; Otherwise, insert as a top-level entry at the end of the file.
- (t (goto-char (point-max
+ (t (goto-char (point-max))
+;; Make sure that last point is not folded.
+(org-fold-core-cycle-over-indirect-buffers
+(org-fold-region (max 1 (1- (point-max))) (point-max) nil
 (let ((origin (point)))
   (unless (bolp) (insert "\n"))
   (org-capture-empty-lines-before)
diff --git a/lisp/org.el b/lisp/org.el
index 0967cbf1c..575a327da 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -5983,7 +5983,13 @@ (defun org-get-indirect-buffer (&optional buffer heading)
 			 (number-to-string n))
   (setq n (1+ n)))
 (condition-case nil
-(make-indirect-buffer buffer bname 'clone)
+(let ((indirect-buffer (make-indirect-buffer buffer bname 'clone)))
+  ;; Decouple folding state.  We need to do it manually since
+  ;; `make-indirect-buffer' does not run
+  ;; `clone-indirect-buffer-hook'.
+  (org-fold-core-decouple-indirect-buffer-folds)
+  ;; Return the buffer.
+  indirect-buffer)
   (error (make-indirect-buffer buffer bname)
 
 (defun org-set-frame-title (title)

[PATCH 07/35] org-at-heading-p: Accept optional argument

[Ihor Radchenko]

* lisp/org.el (org-at-heading-p): Use second argument to allow
checking for visible headings.  Note that by default, unlike
`outline-on-heading-p', `org-at-heading-p' returns non-nil for
invisible headings. Passing second argument is just like
`(outline-on-heading-p)'.
(org-indent-line):
* lisp/org-agenda.el (org-agenda-add-entry-to-org-agenda-diary-file):
* lisp/org-colview.el (org-columns--call):
(org-columns-store-format): Update arguments in `org-at-heading-p'
calls.
---
 lisp/org-agenda.el  |  2 +-
 lisp/org-colview.el |  4 ++--
 lisp/org.el | 14 +-
 3 files changed, 12 insertions(+), 8 deletions(-)

diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index 72292fb4e..cc7cb5527 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -10526,7 +10526,7 @@ (defun org-agenda-add-entry-to-org-agenda-diary-file (type text &optional d1 d2)
   (anniversary
(or (re-search-forward "^\\*[ \t]+Anniversaries" nil t)
 	   (progn
-	 (or (org-at-heading-p t)
+	 (or (org-at-heading-p)
 		 (progn
 		   (outline-next-heading)
 		   (insert "* Anniversaries\n\n")
diff --git a/lisp/org-colview.el b/lisp/org-colview.el
index 082d6def0..15cab35f0 100644
--- a/lisp/org-colview.el
+++ b/lisp/org-colview.el
@@ -699,7 +699,7 @@ (defun org-columns--call (fun)
   (let ((hide-body (and (/= (line-end-position) (point-max))
 			(save-excursion
 			  (move-beginning-of-line 2)
-			  (org-at-heading-p t)
+			  (org-at-heading-p)
 (unwind-protect (funcall fun)
   (when hide-body (outline-hide-entry)
 
@@ -1026,7 +1026,7 @@ (defun org-columns-store-format ()
 	  ;; No COLUMNS keyword in the buffer.  Insert one at the
 	  ;; beginning, right before the first heading, if any.
 	  (goto-char (point-min))
-	  (unless (org-at-heading-p t) (outline-next-heading))
+	  (unless (org-at-heading-p) (outline-next-heading))
 	  (let ((inhibit-read-only t))
 		(insert-before-markers "#+COLUMNS: " fmt "\n"
 	  (setq-local org-columns-default-format fmt))
diff --git a/lisp/org.el b/lisp/org.el
index f59d2cfb0..b17a5477c 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -17243,7 +17243,7 @@ (defun org-toggle-heading (&optional nstars)
 	;; Case 1. Started at an heading: de-star headings.
 	((org-at-heading-p)
 	 (while (< (point) end)
-	   (when (org-at-heading-p t)
+	   (when (org-at-heading-p)
 	 (looking-at org-outline-regexp) (replace-match "")
 	 (setq toggled t))
 	   (forward-line)))
@@ -17844,7 +17844,7 @@ (defun org-context ()
 	 (p (point)) clist o)
 ;; First the large context
 (cond
- ((org-at-heading-p t)
+ ((org-at-heading-p)
   (push (list :headline (point-at-bol) (point-at-eol)) clist)
   (when (progn
 	  (beginning-of-line 1)
@@ -19594,9 +19594,13 @@ (defun org-before-first-heading-p ()
(end-of-line)
(null (re-search-backward org-outline-regexp-bol nil t))
 
-(defun org-at-heading-p (&optional _)
-  "Non-nil when on a headline."
-  (outline-on-heading-p t))
+(defun org-at-heading-p (&optional invisible-not-ok)
+  "Return t if point is on a (possibly invisible) heading line.
+If INVISIBLE-NOT-OK is non-nil, an invisible heading line is not ok."
+  (save-excursion
+(beginning-of-line)
+(and (bolp) (or (not invisible-not-ok) (not (org-fold-folded-p)))
+	 (looking-at outline-regexp
 
 (defun org-in-commented-heading-p (&optional no-inheritance element)
   "Non-nil if point is under a commented heading.

[PATCH 10/35] Implement link folding

[Ihor Radchenko]

* lisp/ol.el (org-link--link-folding-spec):
(org-link--description-folding-spec): New variables controlling link
folding settings.
(org-link--reveal-maybe): Handle revealing folded links.
(org-link-descriptive-ensure): Implement `org-link-descriptive'
support with org-fold.
(org-toggle-link-display--overlays):
(org-toggle-link-display--text-properties):
(org-toggle-link-display): Provide text-properties and overlays
versions.
* lisp/org-agenda.el (org-agenda-mode): Use org-fold to fold links in
agenda.
* lisp/org.el (org-do-emphasis-faces): Use org-fold.
---
 lisp/ol.el | 42 +-
 lisp/org-agenda.el |  3 ++-
 lisp/org.el| 11 +--
 3 files changed, 52 insertions(+), 4 deletions(-)

diff --git a/lisp/ol.el b/lisp/ol.el
index 21bd854e9..1837bf37c 100644
--- a/lisp/ol.el
+++ b/lisp/ol.el
@@ -605,6 +605,22 @@ (defvar org-link--insert-history nil
 (defvar org-link--search-failed nil
   "Non-nil when last link search failed.")
 
+
+(defvar-local org-link--link-folding-spec '(org-link
+(:global t)
+(:ellipsis . nil)
+(:isearch-open . t)
+(:fragile . org-link--reveal-maybe))
+  "Folding spec used to hide invisible parts of links.")
+
+(defvar-local org-link--description-folding-spec '(org-link-description
+   (:global t)
+   (:ellipsis . nil)
+   (:visible . t)
+   (:isearch-open . nil)
+   (:fragile . org-link--reveal-maybe))
+  "Folding spec used to reveal link description.")
+
 
 ;;; Internal Functions
 
@@ -762,6 +778,13 @@ (defun org-link--normalize-string (string &optional context)
 		   (t nil
 string))
 
+(defun org-link--reveal-maybe (region _)
+  "Reveal folded link in REGION when needed.
+This function is intended to be used as :fragile property of a folding
+spec."
+  (org-with-point-at (car region)
+(not (org-in-regexp org-link-any-re
+
 
 ;;; Public API
 
@@ -1444,14 +1467,31 @@ (defun org-previous-link ()
   (interactive)
   (org-next-link t))
 
+(defun org-link-descriptive-ensure ()
+  "Toggle the literal or descriptive display of links in current buffer if needed."
+  (if org-link-descriptive
+  (org-fold-core-set-folding-spec-property (car org-link--link-folding-spec) :visible nil)
+(org-fold-core-set-folding-spec-property (car org-link--link-folding-spec) :visible t)))
+
 ;;;###autoload
-(defun org-toggle-link-display ()
+(defun org-toggle-link-display--overlays ()
   "Toggle the literal or descriptive display of links."
   (interactive)
   (if org-link-descriptive (remove-from-invisibility-spec '(org-link))
 (add-to-invisibility-spec '(org-link)))
   (org-restart-font-lock)
   (setq org-link-descriptive (not org-link-descriptive)))
+(defun org-toggle-link-display--text-properties ()
+  "Toggle the literal or descriptive display of links in current buffer."
+  (interactive)
+  (setq org-link-descriptive (not org-link-descriptive))
+  (org-link-descriptive-ensure))
+(defsubst org-toggle-link-display ()
+  "Toggle the literal or descriptive display of links."
+  (interactive)
+  (if (eq org-fold-core-style 'text-properties)
+  (org-toggle-link-display--text-properties)
+(org-toggle-link-display--overlays)))
 
 ;;;###autoload
 (defun org-store-link (arg &optional interactive?)
diff --git a/lisp/org-agenda.el b/lisp/org-agenda.el
index 2802e8636..063da6566 100644
--- a/lisp/org-agenda.el
+++ b/lisp/org-agenda.el
@@ -2325,7 +2325,8 @@ (defun org-agenda-mode ()
 	  org-agenda-show-log org-agenda-start-with-log-mode
 	  org-agenda-clockreport-mode org-agenda-start-with-clockreport-mode))
   (add-to-invisibility-spec '(org-filtered))
-  (add-to-invisibility-spec '(org-link))
+  (org-fold-core-initialize `(,org-link--description-folding-spec
+  ,org-link--link-folding-spec))
   (easy-menu-change
'("Agenda") "Agenda Files"
(append
diff --git a/lisp/org.el b/lisp/org.el
index ebc9d81db..5465ed3ea 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -4562,9 +4562,16 @@ (define-derived-mode org-mode outline-mode "Org"
   (setq-local org-mode-loading t)
   (org-load-modules-maybe)
   (org-install-agenda-files-menu)
-  (when org-link-descriptive (add-to-invisibility-spec '(org-link)))
+  (when (and org-link-descriptive
+ (eq org-fold-core-style 'overlays))
+(add-to-invisibility-spec '(org-link)))
+  (org-fold-initialize (or (and (stringp org-ellipsis) (not (equal "" org-ellipsis)) org-ellipsis)
+"..."))
   (make-local-variable 'org-link-descriptive)
-  (add-to-invisibility-spec '(org-hide-block . t))
+  (when (eq org-fold-core-style 'ove

[PATCH 17/35] org-compat: Work around some third-party packages using outline-* functions

[Ihor Radchenko]

---
 lisp/org-compat.el | 72 +-
 1 file changed, 71 insertions(+), 1 deletion(-)

diff --git a/lisp/org-compat.el b/lisp/org-compat.el
index 14afb4600..05efeca11 100644
--- a/lisp/org-compat.el
+++ b/lisp/org-compat.el
@@ -1311,11 +1311,81 @@ (defvar session-globals-exclude)
 (eval-after-load 'session
   '(add-to-list 'session-globals-exclude 'org-mark-ring))
 
+ outline-mode
+
+;; Folding in outline-mode is not compatible with org-mode folding
+;; anymore. Working around to avoid breakage of external packages
+;; assuming the compatibility.
+(defadvice outline-flag-region (around outline-flag-region@fix-for-org-fold (from to flag) activate)
+  "Run `org-fold-region' when in org-mode."
+  (if (eq major-mode 'org-mode)
+  (setq ad-return-value (org-fold-region (max from (point-min)) (min to (point-max)) flag 'headline))
+ad-do-it))
+
+(defadvice outline-next-visible-heading (around outline-next-visible-heading@fix-for-org-fold (arg) activate)
+  "Run `org-next-visible-heading' when in org-mode."
+  (interactive "p")
+  (if (eq major-mode 'org-mode)
+  (setq ad-return-value (org-next-visible-heading arg))
+ad-do-it))
+
+(defadvice outline-back-to-heading (around outline-back-to-heading@fix-for-org-fold (&optional invisible-ok) activate)
+  "Run `org-back-to-heading' when in org-mode."
+  (if (eq major-mode 'org-mode)
+  (setq ad-return-value
+(progn
+  (beginning-of-line)
+  (or (org-at-heading-p (not invisible-ok))
+  (let (found)
+	(save-excursion
+	  (while (not found)
+	(or (re-search-backward (concat "^\\(?:" outline-regexp "\\)")
+nil t)
+(signal 'outline-before-first-heading nil))
+	(setq found (and (or invisible-ok (not (org-fold-folded-p)))
+			 (point)
+	(goto-char found)
+	found
+ad-do-it))
+
+(defadvice outline-on-heading-p (around outline-on-heading-p@fix-for-org-fold (&optional invisible-ok) activate)
+  "Run `org-at-heading-p' when in org-mode."
+  (if (eq major-mode 'org-mode)
+  (setq ad-return-value (org-at-heading-p (not invisible-ok)))
+ad-do-it))
+
+(defadvice outline-hide-sublevels (around outline-hide-sublevels@fix-for-org-fold (levels) activate)
+  "Run `org-fold-hide-sublevels' when in org-mode."
+  (interactive (list
+		(cond
+		 (current-prefix-arg (prefix-numeric-value current-prefix-arg))
+		 ((save-excursion (beginning-of-line)
+  (looking-at outline-regexp))
+		  (funcall outline-level))
+		 (t 1
+  (if (eq major-mode 'org-mode)
+  (setq ad-return-value (org-fold-hide-sublevels levels))
+ad-do-it))
+
+(defadvice outline-toggle-children (around outline-toggle-children@fix-for-org-fold () activate)
+  "Run `org-fold-hide-sublevels' when in org-mode."
+  (interactive)
+  (if (eq major-mode 'org-mode)
+  (setq ad-return-value
+(save-excursion
+  (org-back-to-heading)
+  (if (not (org-fold-folded-p (line-end-position)))
+  (org-fold-hide-subtree)
+(org-fold-show-children)
+(org-fold-show-entry
+ad-do-it))
+
+;; TODO: outline-headers-as-kill
+
  Speed commands
 
 (make-obsolete-variable 'org-speed-commands-user
 "configure `org-speed-commands' instead." "9.5")
-
 (provide 'org-compat)
 
 ;; Local variables:

[PATCH 13/35] Fix subtle differences between overlays and invisible text properties

[Ihor Radchenko]

* lisp/org-clock.el (org-clock-in):
(org-clock-find-position):
(org-clock-out):
* lisp/org.el (org-add-planning-info):
(org-scan-tags):
(org-global-tags-completion-table):
(org-make-tags-matcher):
(org-tags-expand):
(org--property-local-values):
(org-read-date-analyze):
(org-revert-all-org-buffers):
(org-beginning-of-line): Make sure that we inherit invisible state
when inserting text.
(org-sort-entries): Preserve invisible state after replace-match.

(org-log-beginning): Do not try to move by visible lines.

* lisp/org-macs.el (org-preserve-local-variables): Do not try to
preserve overlays.
* lisp/ox.el (org-export--generate-copy-script): Preserve folding
properties in export buffer.
* testing/lisp/test-ob.el (test-ob/preserve-results-indentation): Fix
test failure.
* testing/lisp/test-org.el (test-org/meta-return):
(test-org/custom-properties): Use new folding.
---
 lisp/org-clock.el| 116 
 lisp/org-macs.el |  12 +-
 lisp/org.el  | 560 ---
 lisp/ox.el   |   4 +-
 testing/lisp/test-ob.el  |  12 +-
 testing/lisp/test-org.el |   3 +
 6 files changed, 367 insertions(+), 340 deletions(-)

diff --git a/lisp/org-clock.el b/lisp/org-clock.el
index 583b30237..ec87aaf8a 100644
--- a/lisp/org-clock.el
+++ b/lisp/org-clock.el
@@ -1373,14 +1373,14 @@ (defun org-clock-in (&optional select start-time)
 	   (sit-for 2)
 	   (throw 'abort nil))
 	  (t
-	   (insert-before-markers "\n")
+	   (insert-before-markers-and-inherit "\n")
 	   (backward-char 1)
 	   (when (and (save-excursion
 			(end-of-line 0)
 			(org-in-item-p)))
 	 (beginning-of-line 1)
 	 (indent-line-to (max 0 (- (current-indentation) 2
-	   (insert org-clock-string " ")
+	   (insert-and-inherit org-clock-string " ")
 	   (setq org-clock-effort (org-entry-get (point) org-effort-property))
 	   (setq org-clock-total-time (org-clock-sum-current-item
    (org-clock-get-sum-start)))
@@ -1581,19 +1581,23 @@ (defun org-clock-find-position (find-unclosed)
 		  count (1+ count))
 	(cond
 	 ((null positions)
-	  ;; Skip planning line and property drawer, if any.
-	  (org-end-of-meta-data)
-	  (unless (bolp) (insert "\n"))
-	  ;; Create a new drawer if necessary.
-	  (when (and org-clock-into-drawer
-		 (or (not (wholenump org-clock-into-drawer))
-			 (< org-clock-into-drawer 2)))
-	(let ((beg (point)))
-	  (insert ":" drawer ":\n:END:\n")
-	  (org-indent-region beg (point))
-	  (org-flag-region
-	   (line-end-position -1) (1- (point)) t 'outline)
-	  (forward-line -1
+  (org-fold-core-ignore-modifications
+	  ;; Skip planning line and property drawer, if any.
+	  (org-end-of-meta-data)
+	(unless (bolp) (insert-and-inherit "\n"))
+	;; Create a new drawer if necessary.
+	(when (and org-clock-into-drawer
+		   (or (not (wholenump org-clock-into-drawer))
+			   (< org-clock-into-drawer 2)))
+	  (let ((beg (point)))
+	(insert-and-inherit ":" drawer ":\n:END:\n")
+	(org-indent-region beg (point))
+(if (eq org-fold-core-style 'text-properties)
+	(org-fold-region
+	 (line-end-position -1) (1- (point)) t 'drawer)
+  (org-fold-region
+	   (line-end-position -1) (1- (point)) t 'outline))
+	(forward-line -1)
 	 ;; When a clock drawer needs to be created because of the
 	 ;; number of clock items or simply if it is missing, collect
 	 ;; all clocks in the section and wrap them within the drawer.
@@ -1602,28 +1606,29 @@ (defun org-clock-find-position (find-unclosed)
 	drawer)
 	  ;; Skip planning line and property drawer, if any.
 	  (org-end-of-meta-data)
-	  (let ((beg (point)))
-	(insert
-	 (mapconcat
-	  (lambda (p)
-		(save-excursion
-		  (goto-char p)
-		  (org-trim (delete-and-extract-region
-			 (save-excursion (skip-chars-backward " \r\t\n")
-	 (line-beginning-position 2))
-			 (line-beginning-position 2)
-	  positions "\n")
-	 "\n:END:\n")
-	(let ((end (point-marker)))
-	  (goto-char beg)
-	  (save-excursion (insert ":" drawer ":\n"))
-	  (org-flag-region (line-end-position) (1- end) t 'outline)
-	  (org-indent-region (point) end)
-	  (forward-line)
-	  (unless org-log-states-order-reversed
-		(goto-char end)
-		(beginning-of-line -1))
-	  (set-marker end nil
+  (org-fold-core-ignore-modifications
+	  (let ((beg (point)))
+	(insert-and-inherit
+	 (mapconcat
+	  (lambda (p)
+		(save-excursion
+		  (goto-char p)
+		  (org-trim (delete-and-extract-region
+			 (save-excursion (skip-chars-backward " \r\t\n")
+	 (line-beginning-position 2))
+			 (line-beginning-position 2)
+	  positions "\n")
+	 "\n:END:\n")
+	(let ((end (point-marker)))
+	  (goto-char beg)
+	  (save-excursion (insert-and-inherit ":" drawer ":\

[PATCH 11/35] Implement overlay- and text-property-based versions of some functions

[Ihor Radchenko]

---
 lisp/org-element.el|  54 -
 lisp/org-fold.el   |   5 +-
 lisp/org-inlinetask.el |  26 ++-
 lisp/org-list.el   |  74 ++-
 lisp/org-macs.el   |  54 -
 lisp/org.el| 469 +
 6 files changed, 585 insertions(+), 97 deletions(-)

diff --git a/lisp/org-element.el b/lisp/org-element.el
index 9fb32..bf9f3b69d 100644
--- a/lisp/org-element.el
+++ b/lisp/org-element.el
@@ -7905,7 +7905,7 @@ (defun org-element-nested-p (elem-A elem-B)
 (or (and (>= beg-A beg-B) (<= end-A end-B))
 	(and (>= beg-B beg-A) (<= end-B end-A)
 
-(defun org-element-swap-A-B (elem-A elem-B)
+(defun org-element-swap-A-B--overlays (elem-A elem-B)
   "Swap elements ELEM-A and ELEM-B.
 Assume ELEM-B is after ELEM-A in the buffer.  Leave point at the
 end of ELEM-A."
@@ -7973,6 +7973,58 @@ (defun org-element-swap-A-B (elem-A elem-B)
 	(dolist (o (cdr overlays))
 	  (move-overlay (car o) (- (nth 1 o) offset) (- (nth 2 o) offset
   (goto-char (org-element-property :end elem-B)
+(defun org-element-swap-A-B--text-properties (elem-A elem-B)
+  "Swap elements ELEM-A and ELEM-B.
+Assume ELEM-B is after ELEM-A in the buffer.  Leave point at the
+end of ELEM-A."
+  (goto-char (org-element-property :begin elem-A))
+  ;; There are two special cases when an element doesn't start at bol:
+  ;; the first paragraph in an item or in a footnote definition.
+  (let ((specialp (not (bolp
+;; Only a paragraph without any affiliated keyword can be moved at
+;; ELEM-A position in such a situation.  Note that the case of
+;; a footnote definition is impossible: it cannot contain two
+;; paragraphs in a row because it cannot contain a blank line.
+(when (and specialp
+	   (or (not (eq (org-element-type elem-B) 'paragraph))
+		   (/= (org-element-property :begin elem-B)
+		   (org-element-property :contents-begin elem-B
+  (error "Cannot swap elements"))
+;; In a special situation, ELEM-A will have no indentation.  We'll
+;; give it ELEM-B's (which will in, in turn, have no indentation).
+(org-fold-core-ignore-modifications ;; Preserve folding state
+(let* ((ind-B (when specialp
+		(goto-char (org-element-property :begin elem-B))
+		(current-indentation)))
+	   (beg-A (org-element-property :begin elem-A))
+	   (end-A (save-excursion
+		(goto-char (org-element-property :end elem-A))
+		(skip-chars-backward " \r\t\n")
+		(point-at-eol)))
+	   (beg-B (org-element-property :begin elem-B))
+	   (end-B (save-excursion
+		(goto-char (org-element-property :end elem-B))
+		(skip-chars-backward " \r\t\n")
+		(point-at-eol)))
+	   ;; Get contents.
+	   (body-A (buffer-substring beg-A end-A))
+	   (body-B (delete-and-extract-region beg-B end-B)))
+  (goto-char beg-B)
+  (when specialp
+	(setq body-B (replace-regexp-in-string "\\`[ \t]*" "" body-B))
+	(indent-to-column ind-B))
+  (insert body-A)
+	  (goto-char beg-A)
+	  (delete-region beg-A end-A)
+	  (insert body-B)
+  (goto-char (org-element-property :end elem-B))
+(defsubst org-element-swap-A-B (elem-A elem-B)
+  "Swap elements ELEM-A and ELEM-B.
+Assume ELEM-B is after ELEM-A in the buffer.  Leave point at the
+end of ELEM-A."
+  (if (eq org-fold-core-style 'text-properties)
+  (org-element-swap-A-B--text-properties elem-A elem-B)
+(org-element-swap-A-B--overlays elem-A elem-B)))
 
 
 (provide 'org-element)
diff --git a/lisp/org-fold.el b/lisp/org-fold.el
index 52717fd86..e48a528bf 100644
--- a/lisp/org-fold.el
+++ b/lisp/org-fold.el
@@ -53,10 +53,7 @@ (defvar org-drawer-regexp)
 (defvar org-property-end-re)
 (defvar org-link-descriptive)
 (defvar org-outline-regexp-bol)
-(defvar org-custom-properties-hidden-p)
 (defvar org-archive-tag)
-
-;; Needed for overlays only
 (defvar org-custom-properties-overlays)
 
 (declare-function isearch-filter-visible "isearch" (beg end))
@@ -1101,7 +1098,7 @@ (defun org-fold-check-before-invisible-edit--text-properties (kind)
   (when (or invisible-at-point invisible-before-point)
 	(when (eq org-fold-catch-invisible-edits 'error)
 	  (user-error "Editing in invisible areas is prohibited, make them visible first"))
-	(if (and org-custom-properties-hidden-p
+	(if (and org-custom-properties-overlays
 		 (y-or-n-p "Display invisible properties in this buffer? "))
 	(org-toggle-custom-properties-visibility)
 	  ;; Make the area visible
diff --git a/lisp/org-inlinetask.el b/lisp/org-inlinetask.el
index 581370bb5..a63704a05 100644
--- a/lisp/org-inlinetask.el
+++ b/lisp/org-inlinetask.el
@@ -305,7 +305,22 @@ (defun org-inlinetask-fontify (limit)
   (add-text-properties (match-beginning 3) (match-end 3)
 			   '(face org-inlinetask font-lock-fontified t)
 
-(defun org-inlinetask-toggle-visibility ()
+(defun org-inlinetask-toggle-visibility--text-properties ()
+  "Toggle visibility of

[PATCH 22/35] ORG-NEWS: Add list of changes

[Ihor Radchenko]

---
 etc/ORG-NEWS | 104 +++
 1 file changed, 104 insertions(+)

diff --git a/etc/ORG-NEWS b/etc/ORG-NEWS
index 5a94e737e..a60248589 100644
--- a/etc/ORG-NEWS
+++ b/etc/ORG-NEWS
@@ -40,6 +40,105 @@ The cache state is saved between Emacs sessions.  Enabled by default.
 The cache persistence can be controlled via
 ~org-element-cache-persistent~.
 
+*** Users experiencing performance issues can use new folding backend
+
+The old folding backend used in Org is poorly scalable when the file
+size increases beyond few Mbs.  The symptoms usually include slow
+cursor motion, especially in long-running Emacs sessions.
+
+A new optimised folding backend is now available, and enabled by
+default.  To disable it, put the following to the Emacs config *before*
+loading Org:
+
+#+begin_src emacs-lisp
+(setq org-fold-core-style 'overlays)
+#+end_src
+
+Even more performance optimisation can be enabled by customising
+=org-fold-core--optimise-for-huge-buffers=.  However, this option may
+be dangerous.  Please, read the variable docstring carefully to
+understand the possible consequences.
+
+When =org-fold-core-style= is set to =text-properties=, several new
+features will become available and several notable changes will happen
+to the Org behaviour.  The new features and changes are listed below.
+
+ Hidden parts of the links can now be searched and revealed during isearch
+
+In the past, hidden parts of the links could not be searched using
+isearch (=C-s=).  Now, they are searchable by default.  The hidden
+match is also revealed temporarily during isearch.
+
+To restore the old behaviour add the following core to your Emacs
+config:
+
+#+begin_src emacs-lisp
+(defun org-hidden-link-ignore-isearch ()
+  "Do not match hidden parts of links during isearch."
+  (org-fold-core-set-folding-spec-property 'org-link :isearch-open nil)
+  (org-fold-core-set-folding-spec-property 'org-link :isearch-ignore t))
+(add-hook 'org-mode-hook #'org-hidden-link-ignore-isearch)
+#+end_src
+
+See docstring of =org-fold-core--specs= to see more details about
+=:isearch-open= and =:isearch-ignore= properties.
+
+ =org-catch-invisible-edits= now works for hidden parts of the links and for emphasis markers
+
+In the past, user could edit invisible parts of the links and emphasis markers.  Now, the editing is respecting the value of =org-catch-invisible-edits=.
+
+Note that hidden parts of sub-/super-scripts are still not handled.
+
+ Breaking structure of folded elements automatically reveals the folded text
+
+In the past, the user could be left with unfoldable text after breaking the org structure.
+
+For example, if
+
+#+begin_src org
+:DRAWER:
+like this
+:END:
+#+end_src
+
+is folded and then edited into
+
+#+begin_src org
+DRAWER:
+like this
+:END:
+#+end_src
+The hidden text would not be revealed.
+
+Now, breaking structure of drawers, blocks, and headings automatically
+reveals the folded text.
+
+ Folding state of the drawers is now preserved when cycling headline visibility
+
+In the past drawers were folded every time a headline is unfolded.
+
+Now, it is not the case anymore.  The drawer folding state is
+preserved.  The initial folding state of all the drawers in buffer is
+set according to the startup visibility settings.
+
+To restore the old behaviour, add the following code to Emacs config:
+
+#+begin_src emacs-lisp
+(add-hook 'org-cycle-hook #'org-cycle-hide-drawers)
+#+end_src
+
+Note that old behaviour may cause performance issues when cycling
+headline visibility in large buffers.
+
+ =outline-*= functions may no longer work correctly in Org mode
+
+The new folding backend breaks some of the =outline-*= functions that
+rely on the details of visibility state implementation in
+=outline.el=.  The old Org folding backend was compatible with the
+=outline.el= folding, but it is not the case anymore with the new
+backend.  From now on, using =outline-*= functions is strongly
+discouraged when working with Org files.
+
 ** New features
 
 *** New library =org-persist.el= implements variable persistence across Emacs sessions
@@ -98,6 +197,11 @@ argument.
 ~org-get-tags~ now accepts Org element or buffer position as first
 argument.
 
+*** =org-at-heading-p= now recognises optional argument. Its meaning is inverted.
+
+=org-at-heading-p= now returns t by default on headings inside folds.
+Passing optional argument will produce the old behaviour.
+
 ** Miscellaneous
 
 *** Styles are customizable in ~biblatex~ citation processor

[PATCH 18/35] Move `org-buffer-list' to org-macs.el

[Ihor Radchenko]

---
 lisp/org-macs.el | 38 ++
 lisp/org.el  | 38 --
 2 files changed, 38 insertions(+), 38 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 5494acb3e..0ccf080a3 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -217,6 +217,44 @@ (defun org-fit-window-to-buffer (&optional window max-height min-height
  (shrink-window-if-larger-than-buffer window)))
   (or window (selected-window)))
 
+(defun org-buffer-list (&optional predicate exclude-tmp)
+  "Return a list of Org buffers.
+PREDICATE can be `export', `files' or `agenda'.
+
+export   restrict the list to Export buffers.
+filesrestrict the list to buffers visiting Org files.
+agenda   restrict the list to buffers visiting agenda files.
+
+If EXCLUDE-TMP is non-nil, ignore temporary buffers."
+  (let* ((bfn nil)
+	 (agenda-files (and (eq predicate 'agenda)
+			(mapcar 'file-truename (org-agenda-files t
+	 (filter
+	  (cond
+	   ((eq predicate 'files)
+	(lambda (b) (with-current-buffer b (derived-mode-p 'org-mode
+	   ((eq predicate 'export)
+	(lambda (b) (string-match "\\*Org .*Export" (buffer-name b
+	   ((eq predicate 'agenda)
+	(lambda (b)
+	  (with-current-buffer b
+		(and (derived-mode-p 'org-mode)
+		 (setq bfn (buffer-file-name b))
+		 (member (file-truename bfn) agenda-files)
+	   (t (lambda (b) (with-current-buffer b
+			(or (derived-mode-p 'org-mode)
+(string-match "\\*Org .*Export"
+	  (buffer-name b)
+(delq nil
+	  (mapcar
+	   (lambda(b)
+	 (if (and (funcall filter b)
+		  (or (not exclude-tmp)
+			  (not (string-match "tmp" (buffer-name b)
+		 b
+	   nil))
+	   (buffer-list)
+
 
 
 ;;; File
diff --git a/lisp/org.el b/lisp/org.el
index cbdbf32f1..a9a7b4621 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -14736,44 +14736,6 @@ (defun org-switchb (&optional arg)
 		  (mapcar #'list (mapcar #'buffer-name blist))
 		  nil t
 
-(defun org-buffer-list (&optional predicate exclude-tmp)
-  "Return a list of Org buffers.
-PREDICATE can be `export', `files' or `agenda'.
-
-export   restrict the list to Export buffers.
-filesrestrict the list to buffers visiting Org files.
-agenda   restrict the list to buffers visiting agenda files.
-
-If EXCLUDE-TMP is non-nil, ignore temporary buffers."
-  (let* ((bfn nil)
-	 (agenda-files (and (eq predicate 'agenda)
-			(mapcar 'file-truename (org-agenda-files t
-	 (filter
-	  (cond
-	   ((eq predicate 'files)
-	(lambda (b) (with-current-buffer b (derived-mode-p 'org-mode
-	   ((eq predicate 'export)
-	(lambda (b) (string-match "\\*Org .*Export" (buffer-name b
-	   ((eq predicate 'agenda)
-	(lambda (b)
-	  (with-current-buffer b
-		(and (derived-mode-p 'org-mode)
-		 (setq bfn (buffer-file-name b))
-		 (member (file-truename bfn) agenda-files)
-	   (t (lambda (b) (with-current-buffer b
-			(or (derived-mode-p 'org-mode)
-(string-match "\\*Org .*Export"
-	  (buffer-name b)
-(delq nil
-	  (mapcar
-	   (lambda(b)
-	 (if (and (funcall filter b)
-		  (or (not exclude-tmp)
-			  (not (string-match "tmp" (buffer-name b)
-		 b
-	   nil))
-	   (buffer-list)
-
 (defun org-agenda-files (&optional unrestricted archives)
   "Get the list of agenda files.
 Optional UNRESTRICTED means return the full list even if a restriction

[PATCH 19/35] Restore old visibility behaviour of org-refile

[Ihor Radchenko]

---
 lisp/org-refile.el | 1 +
 1 file changed, 1 insertion(+)

diff --git a/lisp/org-refile.el b/lisp/org-refile.el
index d68760623..df7f645ef 100644
--- a/lisp/org-refile.el
+++ b/lisp/org-refile.el
@@ -547,6 +547,7 @@ (defun org-refile (&optional arg default-buffer rfloc msg)
 		   (goto-char (point-min))
 		   (or (outline-next-heading) (goto-char (point-max)
 	   (unless (bolp) (newline))
+   (org-fold-reveal)
 	   (org-paste-subtree level nil nil t)
 	   ;; Record information, according to `org-log-refile'.
 	   ;; Do not prompt for a note when refiling multiple

[PATCH 16/35] Obsolete old function names that are now in org-fold

[Ihor Radchenko]

---
 lisp/org-compat.el | 88 ++
 1 file changed, 88 insertions(+)

diff --git a/lisp/org-compat.el b/lisp/org-compat.el
index 772ef37f9..14afb4600 100644
--- a/lisp/org-compat.el
+++ b/lisp/org-compat.el
@@ -238,6 +238,11 @@ (define-obsolete-function-alias 'org-propertize 'propertize "9.0")
 (define-obsolete-function-alias 'org-select-frame-set-input-focus 'select-frame-set-input-focus "9.0")
 (define-obsolete-function-alias 'org-file-remote-p 'file-remote-p "9.2")
 
+(define-obsolete-function-alias 'org-show-context 'org-fold-show-context "9.6")
+(define-obsolete-function-alias 'org-show-entry 'org-fold-show-entry "9.6")
+(define-obsolete-function-alias 'org-show-children 'org-fold-show-children "9.6")
+
+
 (defmacro org-re (s)
   "Replace posix classes in regular expression S."
   (declare (debug (form))
@@ -347,6 +352,80 @@ (define-obsolete-function-alias 'org-toggle-latex-fragment 'org-latex-preview
 (define-obsolete-function-alias 'org-remove-latex-fragment-image-overlays
   'org-clear-latex-preview "9.3")
 
+(define-obsolete-function-alias 'org-hide-archived-subtrees
+  'org-fold-hide-archived-subtrees "9.6")
+
+(define-obsolete-function-alias 'org-flag-region
+  'org-fold-region "9.6")
+
+(define-obsolete-function-alias 'org-flag-subtree
+  'org-fold-subtree "9.6")
+
+(define-obsolete-function-alias 'org-hide-entry
+  'org-fold-hide-entry "9.6")
+
+(define-obsolete-function-alias 'org-show-subtree
+  'org-fold-show-subtree "9.6")
+
+(define-obsolete-function-alias 'org--hide-wrapper-toggle
+  'org-fold--hide-wrapper-toggle "9.6")
+
+(define-obsolete-function-alias 'org-hide-block-toggle
+  'org-fold-hide-block-toggle "9.6")
+
+(define-obsolete-function-alias 'org-hide-drawer-toggle
+  'org-fold-hide-drawer-toggle "9.6")
+
+(define-obsolete-function-alias 'org--hide-drawers
+  'org-fold--hide-drawers "9.6")
+
+(define-obsolete-function-alias 'org-hide-block-all
+  'org-fold-hide-block-all "9.6")
+
+(define-obsolete-function-alias 'org-hide-drawer-all
+  'org-fold-hide-drawer-all "9.6")
+
+(define-obsolete-function-alias 'org-show-all
+  'org-fold-show-all "9.6")
+
+(define-obsolete-function-alias 'org-set-startup-visibility
+  'org-cycle-set-startup-visibility "9.6")
+
+(define-obsolete-function-alias 'org-show-set-visibility
+  'org-fold-show-set-visibility "9.6")
+
+(define-obsolete-function-alias 'org-check-before-invisible-edit
+  'org-fold-check-before-invisible-edit "9.6")
+
+(define-obsolete-function-alias 'org-flag-above-first-heading
+  'org-fold-flag-above-first-heading "9.6")
+
+(define-obsolete-function-alias 'org-show-branches-buffer
+  'org-fold-show-branches-buffer "9.6")
+
+(define-obsolete-function-alias 'org-show-siblings
+  'org-fold-show-siblings "9.6")
+
+(define-obsolete-function-alias 'org-show-hidden-entry
+  'org-fold-show-hidden-entry "9.6")
+
+(define-obsolete-function-alias 'org-flag-heading
+  'org-fold-heading "9.6")
+
+(define-obsolete-function-alias 'org-set-startup-visibility
+  'org-cycle-set-startup-visibility "9.6")
+
+(define-obsolete-function-alias 'org-set-visibility-according-to-property
+  'org-cycle-set-visibility-according-to-property "9.6")
+
+(define-obsolete-variable-alias 'org-scroll-position-to-restore
+  'org-cycle-scroll-position-to-restore "9.6")
+(define-obsolete-function-alias 'org-optimize-window-after-visibility-change
+  'org-cycle-optimize-window-after-visibility-change "9.6")
+
+(define-obsolete-function-alias 'org-force-cycle-archived
+  'org-cycle-force-archived "9.6")
+
 (define-obsolete-variable-alias 'org-attach-directory
   'org-attach-id-dir "9.3")
 (make-obsolete 'org-attach-store-link "No longer used" "9.4")
@@ -354,6 +433,15 @@ (make-obsolete 'org-attach-expand-link "No longer used" "9.4")
 
 (define-obsolete-function-alias 'org-file-url-p 'org-url-p "9.5")
 
+(define-obsolete-variable-alias 'org-show-context-detail
+  'org-fold-show-context-detail "9.6")
+
+(define-obsolete-variable-alias 'org-catch-invisible-edits
+  'org-fold-catch-invisible-edits "9.6")
+
+(define-obsolete-variable-alias 'org-reveal-start-hook
+  'org-fold-reveal-start-hook "9.6")
+(define-obsolete-function-alias 'org-file-url-p 'org-url-p "9.6")
 (defun org-in-fixed-width-region-p ()
   "Non-nil if point in a fixed-width region."
   (save-match-data

[PATCH 28/35] org-string-width: Handle undefined behaviour in older Emacs

[Ihor Radchenko]

* lisp/org-macs.el (org-string-width): Force older Emacs treating
invisible text with ellipsis as zero-width.  Newer Emacs versions do
exactly this.
---
 lisp/org-macs.el | 11 ++-
 1 file changed, 10 insertions(+), 1 deletion(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 0ccf080a3..0a7da0637 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -920,7 +920,16 @@ (defun org-string-width (string &optional pixels)
 (with-temp-buffer
   (setq-local display-line-numbers nil)
   (setq-local buffer-invisibility-spec
-  current-invisibility-spec)
+  (if (listp current-invisibility-spec)
+  (mapcar (lambda (el)
+;; Consider elipsis to have 0 width.
+;; It is what Emacs 28+ does, but we have
+;; to force it in earlier Emacs versions.
+(if (and (consp el) (cdr el))
+(list (car el))
+  el))
+  current-invisibility-spec)
+current-invisibility-spec))
   (setq-local char-property-alias-alist
   current-char-property-alias-alist)
   (let (pixel-width symbol-width)

[PATCH 26/35] Rename remaining org-force-cycle-archived → org-cycle-force-archived

[Ihor Radchenko]

---
 lisp/org-cycle.el | 2 +-
 lisp/org-keys.el  | 4 ++--
 lisp/org.el   | 1 +
 3 files changed, 4 insertions(+), 3 deletions(-)

diff --git a/lisp/org-cycle.el b/lisp/org-cycle.el
index d2fcc356c..df0a3761a 100644
--- a/lisp/org-cycle.el
+++ b/lisp/org-cycle.el
@@ -811,7 +811,7 @@ (defun org-cycle-hide-archived-subtrees (state)
  (org-get-tags nil 'local)))
 	(message "%s" (substitute-command-keys
 		   "Subtree is archived and stays closed.  Use \
-`\\[org-force-cycle-archived]' to cycle it anyway."))
+`\\[org-cycle-force-archived]' to cycle it anyway."))
 
 (provide 'org-cycle)
 
diff --git a/lisp/org-keys.el b/lisp/org-keys.el
index e6b8ff459..782ffa871 100644
--- a/lisp/org-keys.el
+++ b/lisp/org-keys.el
@@ -94,7 +94,7 @@ (declare-function org-feed-update-all "org" ())
 (declare-function org-fill-paragraph "org" (&optional justify region))
 (declare-function org-find-file-at-mouse "org" (ev))
 (declare-function org-footnote-action "org" (&optional special))
-(declare-function org-force-cycle-archived "org-cycle" ())
+(declare-function org-cycle-force-archived "org-cycle" ())
 (declare-function org-force-self-insert "org" (n))
 (declare-function org-forward-element "org" ())
 (declare-function org-forward-heading-same-level "org" (arg &optional invisible-ok))
@@ -444,7 +444,7 @@ (org-defkey org-mode-map (kbd "C-c C-x") (make-sparse-keymap))
 
  TAB key with modifiers
 (org-defkey org-mode-map (kbd "TAB") #'org-cycle)
-(org-defkey org-mode-map (kbd "C-c C-") #'org-force-cycle-archived)
+(org-defkey org-mode-map (kbd "C-c C-") #'org-cycle-force-archived)
 ;; Override text-mode binding to expose `complete-symbol' for
 ;; pcomplete functionality.
 (org-defkey org-mode-map (kbd "M-TAB") nil)
diff --git a/lisp/org.el b/lisp/org.el
index dd6298104..d2271b98a 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -106,6 +106,7 @@ (defalias 'org-global-cycle #'org-cycle-global)
 (defalias 'org-overview #'org-cycle-overview)
 (defalias 'org-content #'org-cycle-content)
 (defalias 'org-reveal #'org-fold-reveal)
+(defalias 'org-force-cycle-archived #'org-cycle-force-archived)
 
 ;; `org-outline-regexp' ought to be a defconst but is let-bound in
 ;; some places -- e.g. see the macro `org-with-limited-levels'.

[PATCH 25/35] Fix bug in org-get-heading

[Ihor Radchenko]

Fixes #26, where fontification could make the matching and extraction of heading
components fail.
---
 lisp/org.el | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/lisp/org.el b/lisp/org.el
index 1c01ecf88..dd6298104 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -6166,8 +6166,9 @@ (defun org-get-heading (&optional no-tags no-todo no-priority no-comment)
   (let ((case-fold-search nil))
 	(looking-at org-complex-heading-regexp)
 ;; When using `org-fold-core--optimise-for-huge-buffers',
-;; returned text may be invisible.  Clear it up.
-(org-fold-core-remove-optimisation (match-beginning 0) (match-end 0))
+;; returned text will be invisible.  Clear it up.
+(save-match-data
+  (org-fold-core-remove-optimisation (match-beginning 0) (match-end 0)))
 (let ((todo (and (not no-todo) (match-string 2)))
 	  (priority (and (not no-priority) (match-string 3)))
 	  (headline (pcase (match-string 4)

[PATCH 20/35] Add org-fold-related tests

[Ihor Radchenko]

---
 testing/lisp/test-ol.el   |  24 +
 testing/lisp/test-org-list.el |  73 ++
 testing/lisp/test-org.el  | 177 +++---
 3 files changed, 238 insertions(+), 36 deletions(-)

diff --git a/testing/lisp/test-ol.el b/testing/lisp/test-ol.el
index ddcc570b3..343631623 100644
--- a/testing/lisp/test-ol.el
+++ b/testing/lisp/test-ol.el
@@ -50,6 +50,30 @@ (ert-deftest test-ol/encode-url-with-escaped-char ()
 	 (org-link-encode "http://some.host.com/form?&id=blah%2Bblah25";
 			  '(?\s ?\[ ?\] ?%))
 
+(ert-deftest test-ol/org-toggle-link-display ()
+  "Make sure that `org-toggle-link-display' is working.
+See https://github.com/yantar92/org/issues/4.";
+  (dolist (org-link-descriptive '(nil t))
+(org-test-with-temp-text "* Org link test
+[[https://example.com][A link to a site]]"
+  (dotimes (_ 2)
+(goto-char 1)
+(re-search-forward "\\[")
+(should-not (xor org-link-descriptive (org-invisible-p)))
+(re-search-forward "example")
+(should-not (xor org-link-descriptive (org-invisible-p)))
+(re-search-forward "com")
+(should-not (xor org-link-descriptive (org-invisible-p)))
+(re-search-forward "]")
+(should-not (xor org-link-descriptive (org-invisible-p)))
+(re-search-forward "\\[")
+(should-not (org-invisible-p))
+(re-search-forward "link")
+(should-not (org-invisible-p))
+(re-search-forward "]")
+(should-not (xor org-link-descriptive (org-invisible-p)))
+(org-toggle-link-display)
+
 
 ;;; Escape and Unescape Links
 
diff --git a/testing/lisp/test-org-list.el b/testing/lisp/test-org-list.el
index 24d96e58b..66ec97b49 100644
--- a/testing/lisp/test-org-list.el
+++ b/testing/lisp/test-org-list.el
@@ -580,22 +580,40 @@ (ert-deftest test-org-list/move-item-down ()
 	 (let ((org-list-use-circular-motion t)) (org-move-item-down))
 	 (buffer-string
   ;; Preserve item visibility.
+  (should
+   (equal
+(make-list 2 'org-fold-outline)
+(let ((org-fold-core-style 'text-properties))
+  (org-test-with-temp-text
+   "* Headline\n- item 1\n  body 1\n- item 2\n  body 2"
+   (let ((org-cycle-include-plain-lists t))
+ (org-cycle)
+ (search-forward "- item 2")
+ (org-cycle))
+   (search-backward "- item 1")
+   (org-move-item-down)
+   (forward-line)
+   (list (org-fold-get-folding-spec)
+	 (progn
+	   (search-backward " body 2")
+	   (org-fold-get-folding-spec)))
   (should
(equal
 '(outline outline)
-(org-test-with-temp-text
-	"* Headline\n- item 1\n  body 1\n- item 2\n  body 2"
-  (let ((org-cycle-include-plain-lists t))
-	(org-cycle)
-	(search-forward "- item 2")
-	(org-cycle))
-  (search-backward "- item 1")
-  (org-move-item-down)
-  (forward-line)
-  (list (org-invisible-p2)
-	(progn
-	  (search-backward " body 2")
-	  (org-invisible-p2))
+(let ((org-fold-core-style 'overlays))
+  (org-test-with-temp-text
+   "* Headline\n- item 1\n  body 1\n- item 2\n  body 2"
+   (let ((org-cycle-include-plain-lists t))
+ (org-cycle)
+ (search-forward "- item 2")
+ (org-cycle))
+   (search-backward "- item 1")
+   (org-move-item-down)
+   (forward-line)
+   (list (org-invisible-p2)
+	 (progn
+	   (search-backward " body 2")
+	   (org-invisible-p2)))
   ;; Preserve children visibility.
   (org-test-with-temp-text "* Headline
 - item 1
@@ -869,17 +887,30 @@ (ert-deftest test-org-list/insert-item ()
 	(org-insert-item)
 	(buffer-string
   ;; Preserve list visibility when inserting an item.
+  (should
+   (equal
+`(org-fold-outline org-fold-outline)
+(let ((org-fold-core-style 'text-properties))
+  (org-test-with-temp-text "- A\n  - B\n- C\n  - D"
+   (let ((org-cycle-include-plain-lists t))
+	 (org-cycle)
+	 (forward-line 2)
+	 (org-cycle)
+	 (org-insert-item)
+	 (list (org-fold-get-folding-spec nil (line-beginning-position 0))
+	   (org-fold-get-folding-spec nil (line-end-position 2
   (should
(equal
 '(outline outline)
-(org-test-with-temp-text "- A\n  - B\n- C\n  - D"
-  (let ((org-cycle-include-plain-lists t))
-	(org-cycle)
-	(forward-line 2)
-	(org-cycle)
-	(org-insert-item)
-	(list (get-char-property (line-beginning-position 0) 'invisible)
-	  (get-char-property (line-end-position 2) 'invisible))
+(let ((org-fold-core-style 'overlays))
+  (org-test-with-temp-text "- A\n  - B\n- C\n  - D"
+   (let ((org-cycle-include-plain-lists t))
+	 (org-cycle)
+	 (forward-line 2)
+	 (org-cycle)
+	

[PATCH 33/35] org--string-from-props: Fix handling folds in Emacs <28

[Ihor Radchenko]

* lisp/org-macs.el (org--string-from-props): Respect
`char-property-alias-alist' when querying for `invisible' text
property.
---
 lisp/org-macs.el | 9 -
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 867139742..6a2d7fe85 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -897,14 +897,13 @@ (defun org--string-from-props (s property beg end)
 	(cursor beg))
 (while (setq beg (text-property-not-all beg end property nil s))
   (let* ((next (next-single-property-change beg property s end))
-	 (props (text-properties-at beg s))
-	 (spec (plist-get props property))
+	 (spec (get-text-property beg property s))
 	 (value
 	  (pcase property
 		(`invisible
-		 ;; If `invisible' property in PROPS means text is to
-		 ;; be invisible, return 0.  Otherwise return nil so
-		 ;; as to resume search.
+		 ;; If `invisible' property means text is to be
+		 ;; invisible, return 0.  Otherwise return nil so as
+		 ;; to resume search.
 		 (and (or (eq t buffer-invisibility-spec)
 			  (assoc-string spec buffer-invisibility-spec))
 		  0))

[PATCH 21/35] org-manual: Update to new org-fold function names

[Ihor Radchenko]

---
 doc/org-manual.org | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/doc/org-manual.org b/doc/org-manual.org
index 2c54fde87..ba63d3a4e 100644
--- a/doc/org-manual.org
+++ b/doc/org-manual.org
@@ -495,11 +495,11 @@ *** Global and local cycling
   Switch back to the startup visibility of the buffer (see [[*Initial
   visibility]]).
 
-- {{{kbd(C-u C-u C-u TAB)}}} (~outline-show-all~) ::
+- {{{kbd(C-u C-u C-u TAB)}}} (~org-show-all~) ::
 
   #+cindex: show all, command
   #+kindex: C-u C-u C-u TAB
-  #+findex: outline-show-all
+  #+findex: org-show-all
   Show all, including drawers.
 
 - {{{kbd(C-c C-r)}}} (~org-reveal~) ::
@@ -515,18 +515,18 @@ *** Global and local cycling
   headings.  With a double prefix argument, also show the entire
   subtree of the parent.
 
-- {{{kbd(C-c C-k)}}} (~outline-show-branches~) ::
+- {{{kbd(C-c C-k)}}} (~org-show-branches~) ::
 
   #+cindex: show branches, command
   #+kindex: C-c C-k
-  #+findex: outline-show-branches
+  #+findex: org-show-branches
   Expose all the headings of the subtree, but not their bodies.
 
-- {{{kbd(C-c TAB)}}} (~outline-show-children~) ::
+- {{{kbd(C-c TAB)}}} (~org-show-children~) ::
 
   #+cindex: show children, command
   #+kindex: C-c TAB
-  #+findex: outline-show-children
+  #+findex: org-show-children
   Expose all direct children of the subtree.  With a numeric prefix
   argument {{{var(N)}}}, expose all children down to level
   {{{var(N)}}}.
@@ -7369,7 +7369,7 @@ *** Internal archiving
   command (see [[*Visibility Cycling]]).  You can force cycling archived
   subtrees with {{{kbd(C-TAB)}}}, or by setting the option
   ~org-cycle-open-archived-trees~.  Also normal outline commands, like
-  ~outline-show-all~, open archived subtrees.
+  ~org-show-all~, open archived subtrees.
 
 -
   #+vindex: org-sparse-tree-open-archived-trees

[PATCH 29/35] org-string-width: Work around `window-pixel-width' bug in old Emacs

[Ihor Radchenko]

---
 lisp/org-macs.el | 188 ---
 1 file changed, 129 insertions(+), 59 deletions(-)

diff --git a/lisp/org-macs.el b/lisp/org-macs.el
index 0a7da0637..db98dd149 100644
--- a/lisp/org-macs.el
+++ b/lisp/org-macs.el
@@ -887,73 +887,143 @@ (defun org-split-string (string &optional separators)
 		  results		;skip trailing separator
 		(cons (substring string i) results)))
 
+(defun org--string-from-props (s property beg end)
+  "Return the visible part of string S.
+Visible part is determined according to text PROPERTY, which is
+either `invisible' or `display'.  BEG and END are 0-indices
+delimiting S."
+  (let ((width 0)
+	(cursor beg))
+(while (setq beg (text-property-not-all beg end property nil s))
+  (let* ((next (next-single-property-change beg property s end))
+	 (props (text-properties-at beg s))
+	 (spec (plist-get props property))
+	 (value
+	  (pcase property
+		(`invisible
+		 ;; If `invisible' property in PROPS means text is to
+		 ;; be invisible, return 0.  Otherwise return nil so
+		 ;; as to resume search.
+		 (and (or (eq t buffer-invisibility-spec)
+			  (assoc-string spec buffer-invisibility-spec))
+		  0))
+		(`display
+		 (pcase spec
+		   (`nil nil)
+		   (`(space . ,props)
+		(let ((width (plist-get props :width)))
+		  (and (wholenump width) width)))
+		   (`(image . ,_)
+(and (fboundp 'image-size)
+ (ceiling (car (image-size spec)
+		   ((pred stringp)
+		;; Displayed string could contain invisible parts,
+		;; but no nested display.
+		(org--string-from-props spec 'invisible 0 (length spec)))
+		   (_
+		;; Un-handled `display' value.  Ignore it.
+		;; Consider the original string instead.
+		nil)))
+		(_ (error "Unknown property: %S" property)
+	(when value
+	  (cl-incf width
+		   ;; When looking for `display' parts, we still need
+		   ;; to look for `invisible' property elsewhere.
+		   (+ (cond ((eq property 'display)
+			 (org--string-from-props s 'invisible cursor beg))
+			((= cursor beg) 0)
+			(t (string-width (substring s cursor beg
+		  value))
+	  (setq cursor next))
+	(setq beg next)))
+(+ width
+   ;; Look for `invisible' property in the last part of the
+   ;; string.  See above.
+   (cond ((eq property 'display)
+	  (org--string-from-props s 'invisible cursor end))
+	 ((= cursor end) 0)
+	 (t (string-width (substring s cursor end)))
+
+(defun org--string-width-1 (string)
+  "Return width of STRING when displayed in the current buffer.
+Unlike `string-width', this function takes into consideration
+`invisible' and `display' text properties.  It supports the
+latter in a limited way, mostly for combinations used in Org.
+Results may be off sometimes if it cannot handle a given
+`display' value."
+  (org--string-from-props string 'display 0 (length string)))
+
 (defun org-string-width (string &optional pixels)
   "Return width of STRING when displayed in the current buffer.
 Return width in pixels when PIXELS is non-nil."
-  ;; Wrap/line prefix will make `window-text-pizel-size' return too
-  ;; large value including the prefix.
-  ;; Face should be removed to make sure that all the string symbols
-  ;; are using default face with constant width.  Constant char width
-  ;; is critical to get right string width from pixel width.
-  (remove-text-properties 0 (length string)
-  '(wrap-prefix t line-prefix t face t)
-  string)
-  (let (;; We need to remove the folds to make sure that folded table
-;; alignment is not messed up.
-(current-invisibility-spec
- (or (and (not (listp buffer-invisibility-spec))
-  buffer-invisibility-spec)
- (let (result)
-   (dolist (el buffer-invisibility-spec)
- (unless (or (memq el
-   '(org-fold-drawer
- org-fold-block
- org-fold-outline))
- (and (listp el)
-  (memq (car el)
-'(org-fold-drawer
-  org-fold-block
-  org-fold-outline
-   (push el result)))
-   result)))
-(current-char-property-alias-alist char-property-alias-alist))
-(with-temp-buffer
-  (setq-local display-line-numbers nil)
-  (setq-local buffer-invisibility-spec
-  (if (listp current-invisibility-spec)
-  (mapcar (lambda (el)
-;; Consider elipsis to have 0 width.
-;; It is what Emacs 28+ does, but we have
-;; to force it in earlier Emacs versions.
-   

[PATCH 34/35] org-link-make-string: Throw error when both LINK and DESCRIPTION are empty

[Ihor Radchenko]

This behaviour is expected according to `test-ol/make-string'.
---
 lisp/ol.el | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/lisp/ol.el b/lisp/ol.el
index 1837bf37c..2e954a57f 100644
--- a/lisp/ol.el
+++ b/lisp/ol.el
@@ -999,7 +999,9 @@ (defun org-link-make-string (link &optional description)
 		(replace-regexp-in-string "]\\'"
 	  (concat "\\&" zero-width-space)
 	  (org-trim description))
-(if (not (org-string-nw-p link)) description
+(if (not (org-string-nw-p link))
+(or description
+(error "Empty link"))
   (format "[[%s]%s]"
 	  (org-link-escape link)
 	  (if description (format "[%s]" description) "")

[PATCH 23/35] Backport contributed commits

[Ihor Radchenko]

---
 lisp/org-cycle.el | 2 +-
 lisp/org-fold-core.el | 2 +-
 lisp/org-keys.el  | 4 ++--
 lisp/org.el   | 6 ++
 4 files changed, 6 insertions(+), 8 deletions(-)

diff --git a/lisp/org-cycle.el b/lisp/org-cycle.el
index df0a3761a..d2fcc356c 100644
--- a/lisp/org-cycle.el
+++ b/lisp/org-cycle.el
@@ -811,7 +811,7 @@ (defun org-cycle-hide-archived-subtrees (state)
  (org-get-tags nil 'local)))
 	(message "%s" (substitute-command-keys
 		   "Subtree is archived and stays closed.  Use \
-`\\[org-cycle-force-archived]' to cycle it anyway."))
+`\\[org-force-cycle-archived]' to cycle it anyway."))
 
 (provide 'org-cycle)
 
diff --git a/lisp/org-fold-core.el b/lisp/org-fold-core.el
index 121c6b5c4..6ea374498 100644
--- a/lisp/org-fold-core.el
+++ b/lisp/org-fold-core.el
@@ -592,7 +592,7 @@ (defun org-fold-core--property-symbol-get-create (spec &optional buffer return-o
   (org-fold-core-cycle-over-indirect-buffers
   (push (current-buffer) bufs))
   (push buf bufs)
-  (delete-dups bufs)
+  (delete-duplicates bufs)
 ;; Copy all the old folding properties to preserve the folding state
 (with-silent-modifications
   (dolist (old-prop (cdr (assq 'invisible char-property-alias-alist)))
diff --git a/lisp/org-keys.el b/lisp/org-keys.el
index 782ffa871..e6b8ff459 100644
--- a/lisp/org-keys.el
+++ b/lisp/org-keys.el
@@ -94,7 +94,7 @@ (declare-function org-feed-update-all "org" ())
 (declare-function org-fill-paragraph "org" (&optional justify region))
 (declare-function org-find-file-at-mouse "org" (ev))
 (declare-function org-footnote-action "org" (&optional special))
-(declare-function org-cycle-force-archived "org-cycle" ())
+(declare-function org-force-cycle-archived "org-cycle" ())
 (declare-function org-force-self-insert "org" (n))
 (declare-function org-forward-element "org" ())
 (declare-function org-forward-heading-same-level "org" (arg &optional invisible-ok))
@@ -444,7 +444,7 @@ (org-defkey org-mode-map (kbd "C-c C-x") (make-sparse-keymap))
 
  TAB key with modifiers
 (org-defkey org-mode-map (kbd "TAB") #'org-cycle)
-(org-defkey org-mode-map (kbd "C-c C-") #'org-cycle-force-archived)
+(org-defkey org-mode-map (kbd "C-c C-") #'org-force-cycle-archived)
 ;; Override text-mode binding to expose `complete-symbol' for
 ;; pcomplete functionality.
 (org-defkey org-mode-map (kbd "M-TAB") nil)
diff --git a/lisp/org.el b/lisp/org.el
index a9a7b4621..1c01ecf88 100644
--- a/lisp/org.el
+++ b/lisp/org.el
@@ -106,7 +106,6 @@ (defalias 'org-global-cycle #'org-cycle-global)
 (defalias 'org-overview #'org-cycle-overview)
 (defalias 'org-content #'org-cycle-content)
 (defalias 'org-reveal #'org-fold-reveal)
-(defalias 'org-force-cycle-archived #'org-cycle-force-archived)
 
 ;; `org-outline-regexp' ought to be a defconst but is let-bound in
 ;; some places -- e.g. see the macro `org-with-limited-levels'.
@@ -6168,8 +6167,7 @@ (defun org-get-heading (&optional no-tags no-todo no-priority no-comment)
 	(looking-at org-complex-heading-regexp)
 ;; When using `org-fold-core--optimise-for-huge-buffers',
 ;; returned text may be invisible.  Clear it up.
-(save-match-data
-  (org-fold-core-remove-optimisation (match-beginning 0) (match-end 0)))
+(org-fold-core-remove-optimisation (match-beginning 0) (match-end 0))
 (let ((todo (and (not no-todo) (match-string 2)))
 	  (priority (and (not no-priority) (match-string 3)))
 	  (headline (pcase (match-string 4)
@@ -11697,7 +11695,7 @@ (defun org--get-local-tags ()
   (let* ((cached (and (org-element--cache-active-p) (org-element-at-point nil 'cached)))
  (cached-tags (org-element-property :tags cached)))
 (if cached
-;; If we do explicitly copy the result, reference would
+;; If we do not explicitly copy the result, reference would
 ;; be returned and cache element might be modified directly.
 (mapcar #'copy-sequence cached-tags)
   ;; Parse tags manually.

[PATCH 24/35] Fix typo: delete-duplicates → delete-dups

[Ihor Radchenko]

---
 lisp/org-fold-core.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org-fold-core.el b/lisp/org-fold-core.el
index 6ea374498..121c6b5c4 100644
--- a/lisp/org-fold-core.el
+++ b/lisp/org-fold-core.el
@@ -592,7 +592,7 @@ (defun org-fold-core--property-symbol-get-create (spec &optional buffer return-o
   (org-fold-core-cycle-over-indirect-buffers
   (push (current-buffer) bufs))
   (push buf bufs)
-  (delete-duplicates bufs)
+  (delete-dups bufs)
 ;; Copy all the old folding properties to preserve the folding state
 (with-silent-modifications
   (dolist (old-prop (cdr (assq 'invisible char-property-alias-alist)))

[PATCH 27/35] Fix org-fold--hide-drawers--overlays

[Ihor Radchenko]

---
 lisp/org-fold.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org-fold.el b/lisp/org-fold.el
index e48a528bf..a16ee0f9b 100644
--- a/lisp/org-fold.el
+++ b/lisp/org-fold.el
@@ -714,7 +714,7 @@ (defun org-fold--hide-drawers--overlays (begin end)
   "Hide all drawers between BEGIN and END."
   (save-excursion
 (goto-char begin)
-(while (re-search-forward org-drawer-regexp end t)
+(while (and (< (point) end) (re-search-forward org-drawer-regexp end t))
   (let* ((pair (get-char-property-and-overlay (line-beginning-position)
   'invisible))
  (o (cdr-safe pair)))

[PATCH 30/35] org-fold-show-set-visibility: Fix edge case when folded region is at BOB

[Ihor Radchenko]

---
 lisp/org-fold.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/org-fold.el b/lisp/org-fold.el
index a16ee0f9b..d5a21cbcb 100644
--- a/lisp/org-fold.el
+++ b/lisp/org-fold.el
@@ -840,7 +840,7 @@ (defun org-fold-show-set-visibility--text-properties (detail)
 (org-with-point-at (car region)
   (beginning-of-line)
   (let (font-lock-extend-region-functions)
-(font-lock-fontify-region (1- (car region)) (cdr region))
+(font-lock-fontify-region (max (point-min) (1- (car region))) (cdr region))
 (when region
   (org-fold-region (car region) (cdr region) nil
 (unless (org-before-first-heading-p)

[PATCH 31/35] org-fold-core: Fix fontification inside folded regions

[Ihor Radchenko]

* lisp/org-fold-core.el (org-fold-core-initialize): Declare
`org-fold-core-fontified' text property for font-lock.
(org-fold-core--force-fontification): New variable controlling forced
fontification inside folded regions.
(org-fold-core-fontify-region): Fix cases when BEG is inside folded
region.  Respect `org-fold-core--force-fontification'.
* lisp/org-macs.el (org-with-forced-fontification): New macro.
(org-buffer-substring-fontified):
(org-looking-at-fontified): Do not rely on jit-lock.  Use
`org-fold-core-fontified' text property to determine whether text is
already fontified.
---
 lisp/org-fold-core.el | 69 +--
 lisp/org-macs.el  | 31 +++
 2 files changed, 72 insertions(+), 28 deletions(-)

diff --git a/lisp/org-fold-core.el b/lisp/org-fold-core.el
index 121c6b5c4..edae316ff 100644
--- a/lisp/org-fold-core.el
+++ b/lisp/org-fold-core.el
@@ -746,7 +746,8 @@ (defun org-fold-core-initialize (&optional specs)
   (add-hook 'clone-indirect-buffer-hook #'org-fold-core-decouple-indirect-buffer-folds nil 'local)
   ;; Optimise buffer fontification to not fontify folded text.
   (when (eq font-lock-fontify-region-function #'font-lock-default-fontify-region)
-(setq-local font-lock-fontify-region-function 'org-fold-core-fontify-region))
+(setq-local font-lock-fontify-region-function 'org-fold-core-fontify-region)
+(add-to-list 'font-lock-extra-managed-props 'org-fold-core-fontified))
   ;; Setup killing text
   (setq-local filter-buffer-substring-function #'org-fold-core--buffer-substring-filter)
   (if (and (boundp 'isearch-opened-regions)
@@ -1429,35 +1430,47 @@ (defun org-fold-core--buffer-substring-filter (beg end &optional delete)
 return-string))
 
 ;;; Do not fontify folded text until needed.
-
+(defvar org-fold-core--force-fontification nil
+  "Let-bind this variable to t in order to force fontification in
+folded regions.")
 (defun org-fold-core-fontify-region (beg end loudly &optional force)
   "Run `font-lock-default-fontify-region' in visible regions."
-  (let ((pos beg) next
-(org-fold-core--fontifying t))
-(while (< pos end)
-  (setq next (org-fold-core-next-folding-state-change
-  (if force nil
-(let (result)
-  (dolist (spec (org-fold-core-folding-spec-list))
-(when (and (not (org-fold-core-get-folding-spec-property spec :visible))
-   (org-fold-core-get-folding-spec-property spec :font-lock-skip))
-  (push spec result)))
-  result))
-  pos
-  end))
-  (while (and (not (catch :found
-   (dolist (spec (org-fold-core-get-folding-spec 'all next))
- (when (org-fold-core-get-folding-spec-property spec :font-lock-skip)
-   (throw :found spec)
-  (< next end))
-(setq next (org-fold-core-next-folding-state-change nil next end)))
-  (save-excursion
-(font-lock-default-fontify-region pos next loudly)
-(save-match-data
-  (unless (<= pos (point) next)
-(run-hook-with-args 'org-fold-core-first-unfold-functions pos next
-  (put-text-property pos next 'org-fold-core-fontified t)
-  (setq pos next
+  (with-silent-modifications
+(let ((pos beg) next
+  (force (or force org-fold-core--force-fontification))
+  (org-fold-core--fontifying t)
+  (skip-specs
+   (let (result)
+ (dolist (spec (org-fold-core-folding-spec-list))
+   (when (and (not (org-fold-core-get-folding-spec-property spec :visible))
+  (org-fold-core-get-folding-spec-property spec :font-lock-skip))
+ (push spec result)))
+ result)))
+  ;; Move POS to first visible point within BEG..END.
+  (while (and (catch :found
+(dolist (spec (org-fold-core-get-folding-spec 'all pos))
+  (when (org-fold-core-get-folding-spec-property spec :font-lock-skip)
+(throw :found spec
+  (< pos end))
+(setq pos (org-fold-core-next-folding-state-change nil pos end)))
+  (when force (setq pos beg next end))
+  (while (< pos end)
+(unless force
+  (setq next (org-fold-core-next-folding-state-change skip-specs pos end)))
+;; Move to the end of the region to be fontified.
+(while (and (not (catch :found
+ (dolist (spec (org-fold-core-get-folding-spec 'all next))
+   (when (org-fold-core-get-folding-spec-property spec :font-lock-skip)
+ (throw :found spec)
+(< next end))
+  (setq next (org-fold-core-next-folding-state-change nil next end)))
+(save-excursion
+  (font-lock-default-fo

[PATCH 32/35] test-org/string-width: Add tests for strings with prefix properties

[Ihor Radchenko]

---
 testing/lisp/test-org-macs.el | 6 +-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/testing/lisp/test-org-macs.el b/testing/lisp/test-org-macs.el
index 6a7ccea3c..05cef1281 100644
--- a/testing/lisp/test-org-macs.el
+++ b/testing/lisp/test-org-macs.el
@@ -65,7 +65,11 @@ (ert-deftest test-org/string-width ()
   (should (= 4 (org-string-width
 		#("123" 1 2 (display #("abc" 1 2 (invisible t)))
   ;; Test `space' property in `display'.
-  (should (= 2 (org-string-width #(" " 0 1 (display (space :width 2)))
+  (should (= 2 (org-string-width #(" " 0 1 (display (space :width 2))
+  ;; Test `wrap-prefix' property.
+  (should (= 2 (org-string-width #("ab" 0 2 (wrap-prefix "  ")
+  ;; Test `line-prefix' property.
+  (should (= 2 (org-string-width #("ab" 0 2 (line-prefix "  "))
 
 
 ;;; Regexp

[PATCH 35/35] test-ol/org-toggle-link-display: Fix compatibility with old Emacs

[Ihor Radchenko]

* testing/lisp/test-ol.el (test-ol/org-toggle-link-display): Use
back-compatible `org-xor' instead of `xor'.
---
 testing/lisp/test-ol.el | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/testing/lisp/test-ol.el b/testing/lisp/test-ol.el
index 343631623..429bb52ee 100644
--- a/testing/lisp/test-ol.el
+++ b/testing/lisp/test-ol.el
@@ -59,19 +59,19 @@ (ert-deftest test-ol/org-toggle-link-display ()
   (dotimes (_ 2)
 (goto-char 1)
 (re-search-forward "\\[")
-(should-not (xor org-link-descriptive (org-invisible-p)))
+(should-not (org-xor org-link-descriptive (org-invisible-p)))
 (re-search-forward "example")
-(should-not (xor org-link-descriptive (org-invisible-p)))
+(should-not (org-xor org-link-descriptive (org-invisible-p)))
 (re-search-forward "com")
-(should-not (xor org-link-descriptive (org-invisible-p)))
+(should-not (org-xor org-link-descriptive (org-invisible-p)))
 (re-search-forward "]")
-(should-not (xor org-link-descriptive (org-invisible-p)))
+(should-not (org-xor org-link-descriptive (org-invisible-p)))
 (re-search-forward "\\[")
 (should-not (org-invisible-p))
 (re-search-forward "link")
 (should-not (org-invisible-p))
 (re-search-forward "]")
-(should-not (xor org-link-descriptive (org-invisible-p)))
+(should-not (org-xor org-link-descriptive (org-invisible-p)))
 (org-toggle-link-display)
 
 

[BUG] (Doom) Consistently seeing "org-element--cache" problems [9.6 (9.6-??-0c9b30e @ /home/dc/.emacs.doom/.local/straight/build-28.0.90/org/)]

[David Conner]

From: David Conner 
To: emacs-orgmode@gnu.org
Subject: [BUG] (Doom) Consistently seeing "org-element--cache" problems [9.6
 (9.6-??-0c9b30e @
 /home/dc/.emacs.doom/.local/straight/build-28.0.90/org/)]
Date: Thu, 27 Jan 2022 22:59:37 -0500
Message-ID: <87y230se6u@gmail.com>
--text follows this line--

Remember to cover the basics, that is, what you expected to happen and
what in fact did happen.  You don't know how to make a good report?  See

 https://orgmode.org/manual/Feedback.html#Feedback

Your bug report will be posted to the Org mailing list.


I was shuffling around the headlines in a newly created org file. The
error message showed up when I demoted the last headlines under the
"Notes" headline.

Here is the org content:

```org
#+TITLE: HW 2
#+AUTHOR:David Conner
#+EMAIL: aionf...@gmail.com
#+DESCRIPTION: notes

* Cards

** What are the fourt major types of materials?

** What properties of materials make them useful?

** Why are alloys used?

** What's the difference between ferrous and nonferrous alloys?

** What are the primary variables influencing steel classification?

** List some advantages of:
+ tool steels
+ plastics
+ ceramics
+ composites

** List some common uses for:
+ plastics
+ ceramics
+ composites

** Describe the relative importance of the four different types of materials

* Notes

** Steels
+ Est: 2,000 types

** Plastics
+ Est: 5,000 types

** Ceramics

** Glass
+ Est: 10,000 types

** Composites
```

I have also had issues with detangling recently, where it reports that
it is "not in a babel block", but is searching between completely
arbitrary characters for the current block. In my literate .doom.d
config.el & config.org, this always occurs at the same block. I'm unsure
whether this is related.

Emacs  : GNU Emacs 28.0.90 (build 1, x86_64-pc-linux-gnu, GTK+ Version
3.24.30, cairo version 1.16.0)
Package: Org mode version 9.6 (9.6-??-0c9b30e @
/home/dc/.emacs.doom/.local/straight/build-28.0.90/org/)

current state:
==
(setq
 org-roam-db-location "/data/org/roam/org-roam.db"
 org-link-elisp-confirm-function nil
 org-directory "/data/org"
 org-after-refile-insert-hook '(save-buffer)
 org-indirect-buffer-display 'current-window
 org-roam-db-gc-threshold 2305843009213693951
 org-crypt-key nil
 org-bibtex-headline-format-function #[257 "\300%1\236A\207" [:title] 3
"\n\n(fn ENTRY)"]
 org-log-done 'time
 org-roam-mode-hook '(turn-on-visual-line-mode)
 org-load-hook '(+org-init-org-directory-h +org-init-appearance-h
+org-init-agenda-h +org-init-attachments-h
 +org-init-babel-h +org-init-babel-lazy-loader-h
+org-init-capture-defaults-h +org-init-capture-frame-h
 +org-init-custom-links-h +org-init-export-h
+org-init-habit-h +org-init-hacks-h +org-init-keybinds-h
 +org-init-popup-rules-h +org-init-protocol-h
+org-init-protocol-lazy-loader-h +org-init-smartparens-h
 +org-init-roam-h)
 org-roam-extract-new-file-path "${slug}-%<%Y%m%d%H%M%S>-.org"
 org-startup-folded nil
 org-babel-after-execute-hook '(org-redisplay-inline-images)
 org-link-abbrev-alist '(("doom-repo" . "
https://github.com/hlissner/doom-emacs/%s";)
 ("wolfram" . "https://wolframalpha.com/input/?i=%s
")
 ("wikipedia" . "https://en.wikipedia.org/wiki/%s";)
("duckduckgo" . "https://duckduckgo.com/?q=%s";)
 ("gmap" . "https://maps.google.com/maps?q=%s";)
("gimages" . "https://google.com/images?q=%s";)
 ("google" . "https://google.com/search?q=";)
("youtube" . "https://youtube.com/watch?v=%s";)
 ("github" . "https://github.com/%s";))
 org-agenda-files '("/data/org/roam/dailies/")
 org-capture-templates '(("t" "Personal todo" entry (file+headline
+org-capture-todo-file "Inbox") "* [ ] %?\n%i\n%a"
  :prepend t)
 ("n" "Personal notes" entry (file+headline
+org-capture-notes-file "Inbox") "* %u %?\n%i\n%a"
  :prepend t)
 ("j" "Journal" entry (file+olp+datetree
+org-capture-journal-file) "* %U %?\n%i\n%a" :prepend t)
 ("p" "Templates for projects")
 ("pt" "Project-local todo" entry (file+headline
+org-capture-project-todo-file "Inbox")
  "* TODO %?\n%i\n%a" :prepend t)
 ("pn" "Project-local notes" entry (file+headline
+org-capture-project-notes-file "Inbox")
  "* %U %?\n%i\n%a" :prepend t)
 ("pc" "Project-local changelog" entry
  (file+headline
+org-capture-project-changelog-file "Unreleased") "* %U %?\n%i\n%a"
:prepend t)
 ("o" "Centralized templates for projects")
 ("ot" "Project todo" entry
#'+org-capture-central-project-todo-file "* TODO

Re: Org-syntax: Intra-word markup

[Juan Manuel Macías]

Ihor Radchenko writes:

> Maybe we should introduce an equivalent of special blocks, but for
> inline use? Or should we modify _both_ inline export snippets and export
> blocks to allow fallback mechanism?

I find the idea of inline special blocks very interesting, but I think
there are a couple of drawbacks: since special blocks support ATTR_X,
how would that be implemented in the inline version? The most obvious
thing I can think of is to mimic inline code blocks:

my_special_block[attributes list]{content}

But it would produce a result many times too verbose. Another risk that
this would entail, IMHO, is that of the "LaTeXification" of Org...

In any case, for things like that, aren't links and macros enough? I'm
one of those who 'abuse' links for many export scenarios (I even have
written this package:
https://gitlab.com/maciaschain/org-critical-edition), and I think links
have enormous potential and versatility. John Kitchin's blog has really
helped me open my mind and explore that very productive Org component.
Macros are also a very powerful tool, except for the comma issue, which
I think is still an unfinished business and a solution should be found
one day. Still, the possibility of a special inline block is very
interesting to me.

Best regards,

Juan Manuel 

Re: [BUG] (Doom) Consistently seeing "org-element--cache" problems [9.6 (9.6-??-0c9b30e @ /home/dc/.emacs.doom/.local/straight/build-28.0.90/org/)]

[Ihor Radchenko]

David Conner  writes:

> Subject: [BUG] (Doom) Consistently seeing "org-element--cache" problems [9.6
>  (9.6-??-0c9b30e @

Thanks for the report!

> I was shuffling around the headlines in a newly created org file. The
> error message showed up when I demoted the last headlines under the
> "Notes" headline.
>
> Here is the org content:
> ...

I am unable to reproduce any kind of cache problem using your file on
latest main.

Note that you are running 2 month old Org mode. Could you try to update
to latest main and check again? If you keep seeing "org-element--cache"
warning messages, please also report the warning text.

> I have also had issues with detangling recently, where it reports that
> it is "not in a babel block", but is searching between completely
> arbitrary characters for the current block. In my literate .doom.d
> config.el & config.org, this always occurs at the same block. I'm unsure
> whether this is related.

I might be related to corrupted cache. If you keep seeing this after the
update, please also try to run M-x org-element-cache-reset and let us
know if it fixes the detangle problem.

Best,
Ihor

Re: [BUG] Cached element is incorrect in file.org [9.6 (9.6-??-0c9b30e @ /Users/jeffballowe/.emacs.d/.local/straight/build-27.2/org/)]

[Jeff]

Thank you for the very helpful high-level view of things, Ihor. This will help 
me a lot going forward.

Best,
Jeff

> On Jan 28, 2022, at 18:41 PM, Ihor Radchenko  wrote:
> 
> Jeff  writes:
> 
>> Sorry for the delay responding.
> 
> No problem. This list does not have reddit pacing :)
> 
>> The problem resolved after updating Doom Emacs.
> 
> Great!
> 
>> I’m not sure why it resolved, because the org-mode version remained
>> the same.
> 
> Doom Emacs uses development version of Org mode. As you update, only
> latest commit number in the version is going to change:
> 
> M-x org-version
> Org mode version 9.5.2 (9.5.2-g2c70ca <-- latest commit number
> 
>> I am new to emacs and so also to Doom Emacs. I’m not sure if
>> it is good practice to upgrade Doom Emacs packages I’m using with my
>> Doom configuration or my wait until Doom does that in a Doom Emacs
>> upgrade.
> 
> You may consider Doom Emacs upgrade as slightly more stable compared to
> manual update ahead of other Doom users. This particular problem with
> Org appears because we introduced a major and complex change to Org. It
> is not yet stable. We are fixing incoming bugs as they are reported.
> Things should be more stable in a few months.
> 
> Best,
> Ihor
> 
> 

LaTeX letters in Org

[William Denton]

I wrote up how to use Org to write letters with the LaTeX letter class. 
Here's the link in case anyone's interested:


https://www.miskatonic.org/2022/01/28/latex-letters-in-org/

Cheers,

Bill

--
William Denton
https://www.miskatonic.org/
Librarian, artist and licensed private investigator.
Toronto, Canada

New folding backend & outline (was: [PATCH 22/35] ORG-NEWS: Add list of changes)

[Kévin Le Gouguec]

Ihor Radchenko  writes:

> + =outline-*= functions may no longer work correctly in Org mode
> +
> +The new folding backend breaks some of the =outline-*= functions that
> +rely on the details of visibility state implementation in
> +=outline.el=.  The old Org folding backend was compatible with the
> +=outline.el= folding, but it is not the case anymore with the new
> +backend.  From now on, using =outline-*= functions is strongly
> +discouraged when working with Org files.

>From the perspective of a heavy outline-minor-mode user, who dreams of
Org "backporting" its great outlining features to outline.el, that's a
bit disheartening, since IIUC this will cause Org and outline.el to
drift further apart?

I realize this question might sound outlandish, but I'll ask it anyway:
would it be feasible (and relevant) to add this new folding backend to
outline.el, so that (1) /both/ Org and outline(-minor)-mode benefit from
it, (2) outline.el functions keep working in Org?

(Assuming outline.el could be turned into a :core GNU ELPA package, and
Org would tolerate adding this dependency)

I hope this doesn't come across as negative criticism; the amount of
work that went into this branch is astounding, and as an Org user I'm
indebted to the developers for the energy that goes into maintaining it.

I just wish more of the loving care that goes into Org trickled down to
outline(-minor)-mode; the last couple of months were encouraging because
lots of great improvements have been added to outline.el (TAB-cycling,
buttons, control over default visibility and font-locking), and those
improvements enhance *every* major mode with outline-minor-mode support,
so I was hoping for the trend to continue…

Lest I let this message end on that sour note: great work, and thanks
for the energy you put into Org!

Re: New folding backend & outline (was: [PATCH 22/35] ORG-NEWS: Add list of changes)

[Ihor Radchenko]

Kévin Le Gouguec  writes:

> From the perspective of a heavy outline-minor-mode user, who dreams of
> Org "backporting" its great outlining features to outline.el, that's a
> bit disheartening, since IIUC this will cause Org and outline.el to
> drift further apart?

We already had some subtle (yet important) deviations between Org mode
and outline.el, which made using outline-* functions in Org buffers
unreliable at times. This patch just makes the already existing trend
more prominent.

> I realize this question might sound outlandish, but I'll ask it anyway:
> would it be feasible (and relevant) to add this new folding backend to
> outline.el, so that (1) /both/ Org and outline(-minor)-mode benefit from
> it, (2) outline.el functions keep working in Org?
>
> (Assuming outline.el could be turned into a :core GNU ELPA package, and
> Org would tolerate adding this dependency)

Sure. I kept this idea in mind when developing the branch.
org-fold-core.el is written in such a way that it can be used by an
arbitrary major or minor mode:

;; This file contains library to control temporary invisibility
;; (folding and unfolding) of text in buffers.

;; The file implements the following functionality:
;;
;; - Folding/unfolding regions of text
;; - Searching and examining boundaries of folded text
;; - Interactive searching in folded text (via isearch)
;; - Handling edits in folded text
;; - Killing/yanking (copying/pasting) of the folded text
;; - Fontification of the folded text

If desired, outline.el can be rather trivially converted to use
org-fold-core.

> I hope this doesn't come across as negative criticism; the amount of
> work that went into this branch is astounding, and as an Org user I'm
> indebted to the developers for the energy that goes into maintaining it.
>
> Lest I let this message end on that sour note: great work, and thanks
> for the energy you put into Org!

Thanks!

Best,
Ihor

emacs-orgmode@gnu.org

[chris]

On Friday, 28 January 2022 13:40:45 CET Max Nikulin wrote:
> On 27/01/2022 07:03, chris wrote:
> > First: `xdg-open "org-protocol://store-link?url=URL&title=TITLE"` from a
> > terminal.
> > Then `M-x org-insert-link` in emacs, or not (it makes no difference).
> > Then cut-and-paste from non-emacs-application (e.g., Kde's Konsole, or
> > Firefox) to emacs.
> > When doing `C-y` what it pastes is "URL", not what I have copied from
> > Firefox.
> 
> Just a data point: I can not reproduce it with X11, emacs-26.3, and Org
> from git main HEAD. Maybe I will try with emacs-27 and Wayland in a few
> days.
> 
> emacs -Q -L ~/src/org-mode/lisp/ \
>-l org-protocol --eval '(server-start)' test.org
> emacsclient 'org-protocol:/store-link?url=http://o.rg/&title=Tt1'
> copy from firefox
> C-y (org-yank) in Emacs pastes text that I copied from firefox.
> 
> In some aspects your problem looks similar to:
> 
> Max Nikulin. [PATCH] [BUG] org.el: Fix first call of `org-paste-subtree'
> Mon, 29 Nov 2021 19:02:35 +0700.
> https://list.orgmode.org/so2fh1$10kj$1...@ciao.gmane.io

Yes, yes, possibly
Just something I've noticed, which is obvious, but I didn't thought about, and 
which has probably no bearing:
1- click on the bookmarklet
2- `C-c C-l Ret Ret` in an org-buffer, so the link is created (this step not 
necessary though)
3- if you do `C-y` you can see the URL is in the kill-ring
But obviously there is no reason for this URL to be also in the Wayland (or 
x11) clipboard? (there is no law of nature saying that what is in emacs kill-
ring must necessarily also be in wayland clipboard. I think there is a law of 
nature for the other way around though)
In any case, in the case of Kde/Kwin/Wayland, it is not copied in the Wayland 
clipboard.
Maybe it's in the description of org-protocol/store-link that the URL should 
be copied in emacs kill-ring, in any case, it is.
But no it doesn't show in the kde/wayland clipboard (and why would it).

> 
> > But why is there a `nil` here:
> > 
> > https://github.com/emacs-mirror/emacs/blob/19dcb237/lisp/org/org-protocol.
> > el#L467
> >  > 4f346a66024/lisp/org/org-protocol.el#L467>
> > 
> > And why is it working at all from `xdg-open
> > "org-protocol://store-link?url=URL&title=TITLE"`, with a `nil` in that
> > position?
> > 
> > Note: `(org-protocol-store-link "U/T")` works, `(org-protocol-store-link
> > "url=U&title=T")` doesn't work. Produces link `[[url=U&title=T]]`
> > instead of `[[U][T]]`.
> 
> What is the problem with nil there? New-style URIs are parsed before
> they are passed to subprotocol handlers. Why are you trying to call
> org-protocol-store-link directly?

Right, right, right
I was only trying to see if there was something obviously sticking out about 
the cut and paste issue.
So you say "new style URIs are parsed before they are passed to subprotocol 
handler": so, no worries then.
Thanks a lot for saying so. I've been searching but haven't found were they 
were parsed. I've probably haven't searched enough, and anyway it's of no 
bearing. Thanks again.

emacs-orgmode@gnu.org

[Ihor Radchenko]

chris  writes:

> 3- if you do `C-y` you can see the URL is in the kill-ring
> But obviously there is no reason for this URL to be also in the Wayland (or 
> x11) clipboard? (there is no law of nature saying that what is in emacs kill-
> ring must necessarily also be in wayland clipboard. I think there is a law of 
> nature for the other way around though)
> In any case, in the case of Kde/Kwin/Wayland, it is not copied in the Wayland 
> clipboard.
> Maybe it's in the description of org-protocol/store-link that the URL should 
> be copied in emacs kill-ring, in any case, it is.
> But no it doesn't show in the kde/wayland clipboard (and why would it).

I am not 100% sure if I understand your message clearly. However, just
letting you know about existence of the following Emacs customisations:

save-interprogram-paste-before-kill:
Whether to save existing clipboard text into kill ring before replacing 
it.
select-enable-primary:
Non-nil means cutting and pasting uses the primary selection.

Best,
Ihor

Re: [PATCH] org-agenda.el: agenda*: add lost argument

[Ihor Radchenko]

Mikhail Skorzhinskii  writes:

> * lisp/org-agenda.el (org-agenda): add lost argument for agenda*

Thanks! I am not able to apply your patch onto main. Could you resend,
preferably adding the patch as an attachment and adding TINYCHANGE
cookie (you do not seem to have the copyright assignment)?

See https://orgmode.org/worg/org-contribute.html#first-patch

Best,
Ihor

Re: [PATCH] Re: Bug: org-agenda-sort-notime-is-late is not correctly handled by timestamp comparison [9.4.6 (9.4.6-12-gdcc3a8-elpaplus @ /Users/charlestam/.emacs.d/elpa/org-plus-contrib-20210830/)]

[Ihor Radchenko]

Timothy  writes:

> I’ve confirmed both your issue report (thanks for the detailed steps), and 
> your
> suggested fix. As such, I’ve produced a patch (attached). It would be good if
> someone else could check this looks fine and apply it.

I have applied the patch onto bugfix as 1b675f0.
The bug should be fixed now.

Best,
Ihor