Hello John, Great to read your thoughts on the topic - I am a huge admirer of your work and we both seem to cope with similar issues with Org-mode.
* John Kitchin <jkitc...@andrew.cmu.edu> wrote: > > One is to use the new dynamic module capability to write an org parser in > C, or a dedicated agenda function, which would presumably be faster than in > elisp. This seems hard, and for me would certainly be a multiyear project > I am sure! The downside of this is the need to compile the module. I don't > know how easy it would be to make this work across platforms with the > relatively easy install org-mode currently has. This could have a side > benefit though of a c-lib that could be used by others to expand where > org-mode is used. I'm not a fan of C at all but having the parser in C with the possibility to use this parser for external tools as well sounds awesome to me. After all, I've written a primitive parser for a sub-set of Org-mode for https://github.com/novoid/lazyblorg using Python. > The other way that might work is to rely more heavily on a cached version > of the files, perhaps in a different format than elisp, that is faster to > work with. The approach I have explored in this is to index org files into > a sqlite database. The idea then would be to generate the agenda from a sql > query. I use something like this already to "find stuff in orgmode > anywhere". One of the reasons I wrote this is the org-agenda list of files > isn't practical for me because my files are so scattered on my file system. > I had a need to be able to find TODOs in research projects in a pretty wide > range of locations. > > The code I use is at > https://github.com/jkitchin/scimax/blob/master/org-db.el, and from one > database I can find headlines, contacts, locations, TODO headlines across > my file system, all the files that contain a particular link, and my own > recent org files. I didn't try org-db.el yet. So far, I survived using "git grep" and counsel-grep [0] > I am moderately motivated to switch from sqlite to MongoDB Is org-db.el your standard way of accessing informations or do you use it only for occasional searches where you assume that the usual methods would be slow? > The main point of the database was to get a query language, persistence and > good performance. I have also used caches to speed up using bibtex files, > and my org-contacts with reasonable performance. These have been all elisp, > with no additional dependencies. Maybe one could do something similar to > keep an agenda cache that is persistent and updated via hook functions. Oh yeah. My org-contacts were unusable without at least some minor performance improvements as well. Most important to me: improving manipulation of properties using [1]. For example, org-set-property takes almost 20 seconds to give me its interactive input line in my main Org-mode file. This is a no-go. [1] helped me here a lot. [0] https://github.com/novoid/dot-emacs/blob/master/config.org#optimizing-search-methods [1] https://github.com/novoid/dot-emacs/blob/master/config.org#my-org-region-to-property--my-map-p -- get mail|git|SVN|photos|postings|SMS|phonecalls|RSS|CSV|XML into Org-mode: > get Memacs from https://github.com/novoid/Memacs < Personal Information Management > http://Karl-Voit.at/tags/pim/ Emacs-related > http://Karl-Voit.at/tags/emacs/
