Re: [PATCH] Async evaluation in ob-shell

2023-04-17 Thread Ihor Radchenko
Matt writes: > > Thanks, but I am not seeing 6c9104f59. > > I'm sorry, I was unclear about which repo these commits were in. ORG-NEWS > was updated here: > https://git.savannah.gnu.org/cgit/emacs/org-mode.git/commit/?id=6c9104f59ca8085abe477a81857548461bf88f23 Now, see. I was again confused

Re: [PATCH] Async evaluation in ob-shell

2023-04-17 Thread Matt
On Mon, 17 Apr 2023 14:53:18 -0400 Ihor Radchenko wrote --- > Matt m...@excalamus.com> writes: > > > > A small note on the WORG page: it may be more natural to use :async yes > > > rather than :async t. Both are viable - in fact, anything other than > > > :async no and :async no

Re: [PATCH] Async evaluation in ob-shell

2023-04-17 Thread Ihor Radchenko
Matt writes: > > A small note on the WORG page: it may be more natural to use :async yes > > rather than :async t. Both are viable - in fact, anything other than > > :async no and :async none will be treated as "t". > > Updated WORG with commits 9d153ea4, 754c72cb, and 18333299. Updated ORG-N

Re: [PATCH] Async evaluation in ob-shell

2023-04-17 Thread Matt
On Fri, 24 Mar 2023 05:11:38 -0400 Ihor Radchenko wrote --- > Matt m...@excalamus.com> writes: > > > On Thu, 23 Mar 2023 07:48:44 -0400 Ihor Radchenko wrote --- > > > May you also document this new feature in ORG-NEWS and in > > > https://orgmode.org/worg/org-contrib/bab

Re: [PATCH] Async evaluation in ob-shell

2023-03-28 Thread Ihor Radchenko
Matt writes: > -(session (assq :session params))) > +(sessionp (not (member (cdr (assq :session params)) '("no" > "none") > +(if (and async (not sessionp)) > + (error (org-babel-eval-error-notify 1 "ERROR: must use 'async' > with 'session'"))) This is reasonabl

Re: [PATCH] Async evaluation in ob-shell

2023-03-27 Thread Matt
On Fri, 24 Mar 2023 05:13:34 -0400 Ihor Radchenko wrote --- > A small note on the WORG page: it may be more natural to use :async yes > rather than :async t. Both are viable - in fact, anything other than > :async no and :async none will be treated as "t". Ah, okay. I'll make that

Re: [PATCH] Async evaluation in ob-shell

2023-03-24 Thread Ihor Radchenko
Matt writes: > On Thu, 23 Mar 2023 07:48:44 -0400 Ihor Radchenko wrote --- > > May you also document this new feature in ORG-NEWS and in > > https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-shell.html ? > > Done. A small note on the WORG page: it may be more natural to use

Re: [PATCH] Async evaluation in ob-shell

2023-03-23 Thread Matt
On Thu, 23 Mar 2023 07:48:44 -0400 Ihor Radchenko wrote --- > May you also document this new feature in ORG-NEWS and in > https://orgmode.org/worg/org-contrib/babel/languages/ob-doc-shell.html ? Done.

Re: [PATCH] Async evaluation in ob-shell

2023-03-23 Thread Ihor Radchenko
Matt writes: > diff --git a/lisp/ob-shell.el b/lisp/ob-shell.el > index 9e7b45a89..eab8ea935 100644 > --- a/lisp/ob-shell.el > +++ b/lisp/ob-shell.el > @@ -269,12 +269,22 @@ var of the same value." > (set-marker comint-last-output-start (point)) > (get-buffer (current-buffer))

Re: [PATCH] Async evaluation in ob-shell

2023-03-22 Thread Ihor Radchenko
Matt writes: > Unfortunately, I was mistaken and the second option (removing the space from > `org-babel-sh-prompt') doesn't fix the issue. The TLDR is that the code in > `org-babel-comint-async-filter' which grabs the region between the indicators > (incorrectly) fails to include the prompt'

Re: [PATCH] Async evaluation in ob-shell

2023-03-21 Thread Matt
> Matt m...@excalamus.com> writes: > > I see only two options to fix it: remove a space from the concat expression > (which I did in my latest patch) or remove a space from > `org-babel-sh-prompt'. Unfortunately, I was mistaken and the second option (removing the space from `org-babel-sh-

Re: [PATCH] Async evaluation in ob-shell

2023-03-18 Thread Ihor Radchenko
Matt writes: > Way back in https://list.orgmode.org/87sfeyc7qr.fsf@localhost/, we had this > exchange: > > On Mon, 20 Feb 2023 11:24:52 + Ihor Radchenko wrote --- >> Matt writes: >> >> > +(defun ob-shell-async-chunk-callback (string) >> > + "Filter applied to results before insert

Re: [PATCH] Async evaluation in ob-shell

2023-03-12 Thread Jack Kamm
Matt writes: > Hi Jack and Jeremie! I'm curious your thoughts about what Ihor and I are > discussing at the end of this message about `md5'. Thanks for checking in. I don't see any problems with switching to org-id-uuid or similar. Feel free to update ob-python to do this, or I can also do it

Re: [PATCH] Async evaluation in ob-shell

2023-03-09 Thread Max Nikulin
On 10/03/2023 00:36, Matt wrote: (concat "^" (regexp-quote org-babel-sh-prompt) " *") -> "^org_babel_sh_prompt> *" Currently, the two combine via the concat to give*two* spaces in the regexp. "*" means zero or more, so " *" with two spaces acts like " +" one or more space characters. It i

Re: [PATCH] Async evaluation in ob-shell

2023-03-09 Thread Matt
Hi Jack and Jeremie! I'm curious your thoughts about what Ihor and I are discussing at the end of this message about `md5'. On Tue, 07 Mar 2023 07:45:02 -0500 Ihor Radchenko wrote --- > Matt m...@excalamus.com> writes: > > > > The actual prompt is "org_babel_sh_prompt> ". > > > >

Re: [PATCH] Async evaluation in ob-shell

2023-03-07 Thread Ihor Radchenko
Matt writes: > > The actual prompt is "org_babel_sh_prompt> ". > > Agreed. > > > And we want to skip leading spaces in addition. > > What do you mean by this? I was following the pattern described in the docstring of `comint-prompt-regexp', where it is suggested that prompts should follow th

Re: [PATCH] Async evaluation in ob-shell

2023-03-05 Thread Matt
On Sun, 05 Mar 2023 07:14:21 -0500 Ihor Radchenko wrote --- > > Matt m...@excalamus.com> writes: > > > > Sorry for missing that. The issue is that when I replaced > > `org-babel-sh-prompt' with `comint-prompt-regexp', the regexp no > > longer matches the output and grabs the next p

Re: [PATCH] Async evaluation in ob-shell

2023-03-05 Thread Ihor Radchenko
Matt writes: > On Fri, 03 Mar 2023 09:52:09 -0500 Ihor Radchenko wrote --- > > I tried the patch, and I am getting failed tests: > > > > 1 unexpected results: > > Sorry for missing that. The issue is that when I replaced > `org-babel-sh-prompt' with `comint-prompt-regexp', the regex

Re: [PATCH] Async evaluation in ob-shell

2023-03-03 Thread Matt
On Fri, 03 Mar 2023 09:52:09 -0500 Ihor Radchenko wrote --- > I tried the patch, and I am getting failed tests: > > 1 unexpected results: >FAILED test-ob-shell/session-async-evaluation ((should (string= ": 1\n: > 2\n" (buffer-substring-no-properties (point) (point-max :fo

Re: [PATCH] Async evaluation in ob-shell

2023-03-03 Thread Ihor Radchenko
Matt writes: > From b66d776346c992ec085bd719ab73f3d1773f71cc Mon Sep 17 00:00:00 2001 > From: Matthew Trzcinski > Date: Wed, 1 Mar 2023 20:31:46 -0500 > Subject: [PATCH] ob-shell: Add async evaluation I tried the patch, and I am getting failed tests: 1 unexpected results: FAILED test-ob-sh

Re: [PATCH] Async evaluation in ob-shell

2023-03-01 Thread Matt
On Wed, 22 Feb 2023 05:29:59 -0500 Ihor Radchenko wrote --- > > +(defun ob-shell-async-chunk-callback (string) > > + "Filter applied to results before insertion. > > +See `org-babel-comint-async-chunk-callback'." > > + (replace-regexp-in-string (concat org-babel-sh-prompt "*") "" st

Re: [PATCH] Async evaluation in ob-shell

2023-02-22 Thread Ihor Radchenko
Matt writes: > I've rewritten the test and updated the patch. Thanks! > +(defun ob-shell-async-chunk-callback (string) > + "Filter applied to results before insertion. > +See `org-babel-comint-async-chunk-callback'." > + (replace-regexp-in-string (concat org-babel-sh-prompt "*") "" string))

Re: [PATCH] Async evaluation in ob-shell

2023-02-20 Thread Matt
On Mon, 20 Feb 2023 06:24:52 -0500 Ihor Radchenko wrote --- > Why not simply doing the `should' test when the > `test-ob-shell/uuid-regex' match fails? Instead of returning `t'. Then, > we will not need to use advise. Great point. I had originally used advice to avoid a loop. Howeve

Re: [PATCH] Async evaluation in ob-shell

2023-02-20 Thread Ihor Radchenko
Matt writes: > +(advice-add > + 'ob-shell-async-chunk-callback > + :filter-return > + (lambda (&rest r) > + (let ((result (car r))) > + (should (string= result "1\n2\n")) ; expect value > + result)) > + `((name . ,session-name))) > ... > +

Re: [PATCH] Async evaluation in ob-shell

2023-02-19 Thread Matt
On Fri, 17 Feb 2023 05:44:20 -0500 Ihor Radchenko wrote --- > What I mean is... I follow you now. Thank you. I've attached a patch (with commit message) for adding async to ob-shell. If it looks good, I can apply it to main. 0001-ob-shell-Add-async-evaluation.patch Description: Bin

Re: [PATCH] Async evaluation in ob-shell

2023-02-17 Thread Ihor Radchenko
Matt writes: > > Doesn't (while ... (sleep-for ...)) work? > > I'm afraid I don't follow what you mean. What I mean is: 1. Execute src block asynchronously 2. Run (while ... (sleep-for ...)) until the result appears (up to threshold seconds) 3. Check the result as usual or fail the test whe

Re: [PATCH] Async evaluation in ob-shell

2023-02-15 Thread Matt
On Wed, 15 Feb 2023 10:08:47 -0500 Ihor Radchenko wrote --- > Matt m...@excalamus.com> writes: > > > Checking the final result from the callback is trickier. The following > > works, but requires advice (which could potentially persist beyond the > > test) and a delay (which sl

Re: [PATCH] Async evaluation in ob-shell

2023-02-15 Thread Ihor Radchenko
Matt writes: > Checking the final result from the callback is trickier. The following > works, but requires advice (which could potentially persist beyond the test) > and a delay (which slows testing overall and whose duration likely depends on > the hardware the test runs on). Without the

Re: [PATCH] Async evaluation in ob-shell

2023-02-12 Thread Matt
On Sat, 11 Feb 2023 06:44:56 -0500 Ihor Radchenko wrote --- > 1. You should provide all the docstrings. > 2. I generally feel that separate async and separate session code are >either duplicating the same code or edge cases considered by session >code may popup in async code

Re: [PATCH] Async evaluation in ob-shell

2023-02-12 Thread Matt
On Sat, 11 Feb 2023 15:56:00 -0500wrote --- > org-babel-comint-async-filter is capable of taking a similar approach, > and reading/writing to tempfile. There is some precedence in ob-shell for this. Currently, the cmdline, stdin, and shebang headers use temp files. It may be that

Re: [PATCH] Async evaluation in ob-shell

2023-02-11 Thread Ihor Radchenko
Matt writes: > It seems to me like 02-ob-shell-async-non-file.patch is all that's needed for > basic async in ob-shell. I'm able to run long processes like `guix pull' and > `guix package -u' calls without issue and the results look like I expect. > Similarly for running a web server, such a

Re: [PATCH] Async evaluation in ob-shell

2023-02-10 Thread Matt
On Thu, 09 Feb 2023 06:23:42 -0500 Ihor Radchenko wrote --- > Could you please elaborate? What was the problem with > `buffer-substring'? I am asking because one of the previous versions of > `org-babel-comint-wait-for-output' relied upon 'face text property. See > a35d16368. The prob

Re: [PATCH] Async evaluation in ob-shell

2023-02-09 Thread Ihor Radchenko
Matt writes: > I found cleaning the output was dramatically helped by calling > `buffer-substring-no-properties' instead of `buffer-substring' in > `org-babel-comint-async-filter'. I'm not sure why `buffer-substring' was > originally used. `make test' shows no failures, so I assume it doesn'

Re: [PATCH] Async evaluation in ob-shell

2023-02-08 Thread Matt
I've attached two patches which replace the previous. I found cleaning the output was dramatically helped by calling `buffer-substring-no-properties' instead of `buffer-substring' in `org-babel-comint-async-filter'. I'm not sure why `buffer-substring' was originally used. `make test' shows no

Re: [PATCH] Async evaluation in ob-shell

2023-02-07 Thread Ihor Radchenko
Matt writes: > I'm excited to share that I've got async evaluation working (crudely) with > ob-shell. A rough implementation is attached. > > It has clear issues, such as the prompt being present in the output: That's likely because of the same reasons why prompt did not get cleaned in synch