Re: [Bug-apl] Uh oh... SVN 445

[Juergen Sauermann]

  
  
Hi David,
  
  thanks, fixed in SVN 446.
  
  /// Jürgen
  
  

On 08/23/2014 07:27 PM, David Lamkins
  wrote:


  
I ran into this after updating to 445.
  

This is in a pendent function.

  ''≡0⍴value
  0
    8⎕cr value
  ┌→───┐
  │unix│
  └┘
    ''≡0⍴'unix'
  1
    8⎕cr 'unix'
  ┌→───┐
  │unix│
  └┘


  
-- 

  
"The secret to creativity is knowing how to hide
  your sources."
   Albert Einstein
  
  

http://soundcloud.com/davidlamkins
http://reverbnation.com/lamkins
http://reverbnation.com/lcw
http://lamkins-guitar.com/
http://lamkins.net/
http://successful-lisp.com/

  

  


  

Re: [Bug-apl] Seeking clues regarding quote-quad prompt

[Juergen Sauermann]

  
  
Hi David,
  
  this could be caused by how the input echo is implemented.
  
  If you look at CinOut::overflow(int c) then you will see
  that the echo of GNU APL input
  is sent to cerr rather than to cout. The reason
  was, as far as i remember, that when you pipe the APL output
  into something - say an HTML script - then you normally don't want
  to see it mixed with echo of the script.
  
  So we have, I believe:
  
  CIN (input echo) → cerr
  CERR → cerr
  UERR → cerr
  
  COUT→ cout
  
  /// Jürgen
  
  

On 08/24/2014 07:12 AM, David B.
  Lamkins wrote:


  Back on the subject of aplwrap integration:

I'm seeing a GNU APL behavior that I don't understand, and would
appreciate some hints on where to look. I don't necessarily consider the
following behavior to be buggy, I just want to be able to figure out how
and why it's happening so I can dig into the code.

Quick background: aplwrap spawns GNU APL with pipes for stdin, stdout
and stderr. Pretty much everything works as expected, except for some
puzzling behavior w.r.t. a ⍞ prompt followed by a ⍞ input.

What I'm seeing (by dumping stdout and stderr) is that the prompt is
showing up on both stdout and stderr.

>From what I've been able to read so far, I think this is how it happens:

With --rawCIN, get_user_line() calls no_readline() with the prompt text.
no_readline() then ungetc()s the entire prompt so it'll be available for
⍞ input.

I think that aplwrap must see the pushed-back prompt and echo it to
stdout. That's fine. I can deal with that.

But then almost the same prompt appears on stderr. I can't figure out
how that happens. I say "almost the same" because the prompt text on
stderr may have pad characters in place of blanks, assuming that the
prompt used a nested vector. For that reason, I'm convinced that aplwrap
isn't somehow involved; the stderr prompt must from GNU APL.

What I can't understand is how the ⍞ prompt *ever* shows up on stderr.
>From what I've read, it looks like the prompt always goes to COUT.

Clues will be much appreciated...







  

Re: [Bug-apl] Optimizations revived

[Elias Mårtenson]

Thanks for the explanation. With this in mind, I'm wondering about two
specific cases and how you intend to solve them (these are are ones that
gave me the biggest headache):

First of all, how will you avoid doing a clone() on every update using
indexed assignment? You need some way to know whether or not an instance
have to be cloned or can be modified in-place.

A related case is normal left-arrow assignment. It is not desirable to do a
clone() on every assignment either. You want to basically do a "deferred
clone" (i.e. CoW) so that the instances can be shared in both variables
until one of them is updated.

When I was thinking about this, I couldn't get away from concluding that a
full CoW implementation is needed in order to avoid all unnecessary cloning.

Regards,
Elias


On 24 August 2014 00:01, Juergen Sauermann 
wrote:

>  Hi Elias,
>
> normally APL values are not written to. The exception is indexed
> assignment.
>
> I believe the *clone()* call in *Symbol::resolve(**)* can be skipped
> completely.
> This is probably the most frequently used *clone() *case. I suppose
> copy-on-write semantics is
> achieved when all *clone()* calls are gone. Many of the remaining
> *clone()* calls are specific
> to certain functions so their performance impact should be small.
>
> I haven't done the above before the 1.4 release because I didn't want to
> release
> a not-so-well tested optimization.
>
> /// Jürgen
>
>
>
> On 08/23/2014 05:32 PM, Elias Mårtenson wrote:
>
> Cool :-)
>
>  Speaking of this, I did spend quite a bit of time some time ago to try
> to figure out an easy way to get generic copy-on-write semantics, but I
> never came to a satisfactory conclusion. I might revisit it later.
>
>  Have you thought about it? It is, after all, related to this specific
> optimisation.
>
>  Regards,
> Elias
>
>
>  On 23 August 2014 23:29, Juergen Sauermann  > wrote:
>
>>  Hi,
>>
>> I have revived Elias' in-place optimization for *A⍴B* and *,B* now
>> usinng a different
>> way of figuring if *B* is in use. SVN 445.
>>
>> /// Jürgen
>>
>>
>
>

Re: [Bug-apl] Optimizations revived

[Juergen Sauermann]

  
  
Hi Elias,
  
  I don't know. Actually I just found a nasty example which renders
  my statement below
  ("I believe the clone()
  call in Symbol::resolve() can be skipped completely")
  wrong:
  
    A←10⍴10
    A+ (A[3]←3) ⊢ A
  
  Currently a value is cloned when it is assigned to a variable and
  again when the variable is referenced.
  The above example shows why cloning on reference is needed. You
  get the correct 20 20 13 20 20 20 20 20 20 20
  when you clone and the wrong 20 20 6 20 20 20 20 20 20 20
  if not.
  
  The problem that I see with CoW is that we know how many owners a
  value has but not who they are.
  
  And you definitely dont want to clone on indexed assignment
  because very often you have a large value
  and only a small part being updated. With CoW you dont want to
  clone the value that is being assigned but
  rather the other (possibly many copies) and those we don't know.
  
  I also don't share your opinion that the cloning is unnecessary. I
  think it is more so that it is sometimes unnecessary,
  but at the time when you would not  clone you don't have enough
  information to properly make that decision.
  
  /// Jürgen
  
  

On 08/24/2014 04:04 PM, Elias Mårtenson
  wrote:


  Thanks for the explanation. With this in mind, I'm
wondering about two specific cases and how you intend to solve
them (these are are ones that gave me the biggest headache):


First of all, how will you avoid doing a clone() on every update
  using indexed assignment? You need some way to know whether or
  not an instance have to be cloned or can be modified in-place.


A related case is normal left-arrow assignment. It is not
  desirable to do a clone()
  on every assignment either. You want to basically do a
  "deferred clone" (i.e. CoW) so that the instances can be
  shared in both variables until one of them is updated.


When I was thinking about this, I couldn't get away from
  concluding that a full CoW implementation is needed in order
  to avoid all unnecessary cloning.


Regards,
Elias
  
  

On 24 August 2014 00:01, Juergen
  Sauermann 
  wrote:
  
 Hi Elias,
  
  normally APL values are not written to. The exception is
  indexed assignment.
  
  I believe the clone() call in Symbol::resolve()
  can be skipped completely.
  This is probably the most frequently used clone() case.
  I suppose copy-on-write semantics is
  achieved when all clone() calls are gone. Many of
  the remaining clone() calls are specific
  to certain functions so their performance impact should be
  small.
  
  I haven't done the above before the 1.4 release because I
  didn't want to release
  a not-so-well tested optimization.
  
  /// Jürgen
  

  
  
  On 08/23/2014 05:32 PM, Elias Mårtenson wrote:
  
  
Cool :-)
  
  
  Speaking of this, I did spend quite a bit of
time some time ago to try to figure out an easy
way to get generic copy-on-write semantics, but
I never came to a satisfactory conclusion. I
might revisit it later.
  
  
  Have you thought about it? It is, after all,
related to this specific optimisation.
  
  
  Regards,
  Elias


  
   On 23 August 2014 23:29,
Juergen Sauermann 
wrote:

   Hi,
  
  I have revived Elias' in-place
  optimization for A⍴B and ,B
  now usinng a different
  way of figuring if B is in use.
  SVN 445.
  
  /// Jürgen
  
 

  
  
  

Re: [Bug-apl] Optimizations revived

[Elias Mårtenson]

Of course, there are definitely times where cloning is necessary. The
typical example would of course be when a value is assigned to another and
that value is subsequently changed (by using indexed assignment).

And you mentioning the issue of not know *who* are the owners of an object
reminds me of the fact that I also came across that issue. At first I was
thinking that the problem is trivial; that you simply clone if the owner
count is >1. Then I realised that there could be many owners, but those
owners are not going to touch the object anymore.

There must be an optimal solution here. I'm sure you agree with me about
the ideal situation: Where there is no cloning *except* for when it's
needed because an object will be changed.

Now: Can you confirm if I'm correct in my assessment that the *only* thing
that forces cloning in the first place is indexed assignment? If APL had no
such feature, then there would be zero need for cloning, yes?

Regards,
Elias



On 24 August 2014 22:46, Juergen Sauermann 
wrote:

>  Hi Elias,
>
> I don't know. Actually I just found a nasty example which renders my
> statement below
> ("I believe the *clone()* call in *Symbol::resolve(**)* can be skipped
> completely") wrong:
>
> *  A←10⍴10*
> *  A+ (A[3]←3) ⊢ A*
>
> Currently a value is cloned when it is assigned to a variable and again
> when the variable is referenced.
> The above example shows why cloning on reference is needed. You get the
> correct *20 20 13 20 20 20 20 20 20 20*
> when you clone and the wrong *20 20 6 20 20 20 20 20 20 20* if not.
>
> The problem that I see with CoW is that we know how many owners a value
> has but not who they are.
>
> And you definitely dont want to clone on indexed assignment because very
> often you have a large value
> and only a small part being updated. With CoW you dont want to clone the
> value that is being assigned but
> rather the other (possibly many copies) and those we don't know.
>
> I also don't share your opinion that the cloning is unnecessary. I think
> it is more so that it is sometimes unnecessary,
> but at the time when you would not  clone you don't have enough
> information to properly make that decision.
>
> /// Jürgen
>
>
>  On 08/24/2014 04:04 PM, Elias Mårtenson wrote:
>
> Thanks for the explanation. With this in mind, I'm wondering about two
> specific cases and how you intend to solve them (these are are ones that
> gave me the biggest headache):
>
>  First of all, how will you avoid doing a clone() on every update using
> indexed assignment? You need some way to know whether or not an instance
> have to be cloned or can be modified in-place.
>
>  A related case is normal left-arrow assignment. It is not desirable to
> do a clone() on every assignment either. You want to basically do a
> "deferred clone" (i.e. CoW) so that the instances can be shared in both
> variables until one of them is updated.
>
>  When I was thinking about this, I couldn't get away from concluding that
> a full CoW implementation is needed in order to avoid all unnecessary
> cloning.
>
>  Regards,
> Elias
>
>
> On 24 August 2014 00:01, Juergen Sauermann 
> wrote:
>
>>  Hi Elias,
>>
>> normally APL values are not written to. The exception is indexed
>> assignment.
>>
>> I believe the *clone()* call in *Symbol::resolve(**)* can be skipped
>> completely.
>> This is probably the most frequently used *clone() *case. I suppose
>> copy-on-write semantics is
>> achieved when all *clone()* calls are gone. Many of the remaining
>> *clone()* calls are specific
>> to certain functions so their performance impact should be small.
>>
>> I haven't done the above before the 1.4 release because I didn't want to
>> release
>> a not-so-well tested optimization.
>>
>> /// Jürgen
>>
>>
>>
>> On 08/23/2014 05:32 PM, Elias Mårtenson wrote:
>>
>> Cool :-)
>>
>>  Speaking of this, I did spend quite a bit of time some time ago to try
>> to figure out an easy way to get generic copy-on-write semantics, but I
>> never came to a satisfactory conclusion. I might revisit it later.
>>
>>  Have you thought about it? It is, after all, related to this specific
>> optimisation.
>>
>>  Regards,
>> Elias
>>
>>
>>  On 23 August 2014 23:29, Juergen Sauermann <
>> juergen.sauerm...@t-online.de> wrote:
>>
>>>  Hi,
>>>
>>> I have revived Elias' in-place optimization for *A⍴B* and *,B* now
>>> usinng a different
>>> way of figuring if *B* is in use. SVN 445.
>>>
>>> /// Jürgen
>>>
>>>
>>
>>
>
>

Re: [Bug-apl] Optimizations revived

[Juergen Sauermann]

  
  
Hi Elias,
  
  there are actually more cases. For example:
  
    (5 5↑A)←B
  
  This is like left values in C/C++.
  
  I would agree that unnecessary cloning should be avoided. But that
  needs to be done in a way
  that can be proven to be correct (and without complex data
  structures managing ownership of values).
  
  In the past even a rather simple flag scheme has turned out to be
  non-maintainable and I would not
  like to walk on such a path again. You can have a look at tags/apl-1.1-old-memory-management/
in SVN to see how to not do things.

/// Jürgen
  
  

On 08/24/2014 05:01 PM, Elias Mårtenson
  wrote:


  Of course, there are definitely times where cloning
is necessary. The typical example would of course be when a
value is assigned to another and that value is subsequently
changed (by using indexed assignment).

  

And you mentioning the issue of not know who are
  the owners of an object reminds me of the fact that I also
  came across that issue. At first I was thinking that the
  problem is trivial; that you simply clone if the owner count
  is >1. Then I realised that there could be many owners, but
  those owners are not going to touch the object anymore.


There must be an optimal solution here. I'm sure you agree
  with me about the ideal situation: Where there is no cloning except for
  when it's needed because an object will be changed.


Now: Can you confirm if I'm correct in my assessment that
  the only thing that forces cloning in the first place
  is indexed assignment? If APL had no such feature, then there
  would be zero need for cloning, yes?


Regards,
Elias

  
  

  
  

On 24 August 2014 22:46, Juergen
  Sauermann 
  wrote:
  
 Hi Elias,

I don't know. Actually I just found a nasty example
which renders my statement below
("I
believe the clone() call in Symbol::resolve()
can be skipped completely") wrong:

  A←10⍴10
  A+ (A[3]←3) ⊢ A

Currently a value is cloned when it is assigned to a
variable and again when the variable is referenced.
The above example shows why cloning on reference is
needed. You get the correct 20 20 13 20 20 20 20 20
  20 20
when you clone and the wrong 20 20 6 20 20 20 20 20
  20 20 if not.

The problem that I see with CoW is that we know how many
owners a value has but not who they are.

And you definitely dont want to clone on indexed
assignment because very often you have a large value
and only a small part being updated. With CoW you dont
want to clone the value that is being assigned but
rather the other (possibly many copies) and those we
don't know.

I also don't share your opinion that the cloning is
unnecessary. I think it is more so that it is sometimes
unnecessary,
but at the time when you would not  clone you don't have
enough information to properly make that decision.

/// Jürgen


  
  

  On 08/24/2014 04:04 PM, Elias Mårtenson wrote:
  
  
Thanks for the explanation. With this
  in mind, I'm wondering about two specific cases
  and how you intend to solve them (these are are
  ones that gave me the biggest headache):
  
  
  First of all, how will you avoid doing a clone()
on every update using indexed assignment? You
need some way to know whether or not an instance
have to be cloned or can be modified in-place.
  
  
  A related case is normal left-arrow
assignment. It is not desirable to do a clone()
on every assignment either. You want to
basically do a "deferred clone" (i.e. CoW) so
that the instances can be shared in both
 

[Bug-apl] Patch for quote-quad and aplwrap

[David B. Lamkins]

Thanks for the pointer to CinOut::overflow(). That's the part I was
missing.

The attached patch ensures that no_readline() changes pad characters to
spaces before pushing the prompt onto stdin. I've also reverted your
patch to CInOut::overflow().

Here's the explanation:

aplwrap watches both stout and stderr. When there's a prompted
quote-quad input, aplwrap sees the printed prompt. It also sees -- on
stderr -- the prompt as pushed back onto stdin. Because these two
prompts print on different streams, aplwrap is able to match them and
suppress printing of the stderr version of the prompt.

With CInOut::overflow() echoing stdin to stdout (as it does in SVN 447),
aplwrap sees a doubled prompt on stdout, leaving no way for aplwrap to
infer the presence of a prompt and suppress the doubled output. That's
why I reverted CInOut::overflow() to echo to stderr.

The second part of the patch ensures that pad characters in the
quote-quad prompt aren't pushed onto stdin. With this patch, the stdout
and stderr representations of the prompt are identical.

Index: src/Output.cc
===
--- src/Output.cc	(revision 447)
+++ src/Output.cc	(working copy)
@@ -150,7 +150,7 @@
if (!InputFile::echo_current_file())   return 0;
 
Output::set_color_mode(Output::COLM_INPUT);
-   cout << (char)c;
+   cerr << (char)c;
return 0;
 }
 //-
Index: src/Input.cc
===
--- src/Input.cc	(revision 447)
+++ src/Input.cc	(working copy)
@@ -107,8 +107,9 @@
 {
if (prompt)
   {
-CIN << '\r' << *prompt << flush;
-UTF8_string prompt_utf(*prompt);
+UCS_string prompt_no_pad = prompt->no_pad();
+CIN << '\r' << prompt_no_pad << flush;
+UTF8_string prompt_utf(prompt_no_pad);
 loop(p, prompt_utf.size())
{
  const int cc = prompt_utf[prompt_utf.size() - p - 1];

Re: [Bug-apl] Optimizations revived

[blake1024]

What about keeping a linked list of owners, figure who is getting written to, and clone that one only?  Typically there would only be one owner, and you can do the linked check with inline code.. I think avoiding a lot of cloning is important.Blake-- Sent from my HP TouchPadOn Aug 24, 2014 10:01 AM, Elias Mårtenson  wrote: Of course, there are definitely times where cloning is necessary. The typical example would of course be when a value is assigned to another and that value is subsequently changed (by using indexed assignment).
And you mentioning the issue of not know who are the owners of an object reminds me of the fact that I also came across that issue. At first I was thinking that the problem is trivial; that you simply clone if the owner count is >1. Then I realised that there could be many owners, but those owners are not going to touch the object anymore.
There must be an optimal solution here. I'm sure you agree with me about the ideal situation: Where there is no cloning except for when it's needed because an object will be changed.
Now: Can you confirm if I'm correct in my assessment that the only thing that forces cloning in the first place is indexed assignment? If APL had no such feature, then there would be zero need for cloning, yes?
Regards,EliasOn 24 August 2014 22:46, Juergen Sauermann  wrote:

  

  
  
Hi Elias,
  
  I don't know. Actually I just found a nasty example which renders
  my statement below
  ("I believe the clone()
  call in Symbol::resolve() can be skipped completely")
  wrong:
  
    A←10⍴10
    A+ (A[3]←3) ⊢ A
  
  Currently a value is cloned when it is assigned to a variable and
  again when the variable is referenced.
  The above example shows why cloning on reference is needed. You
  get the correct 20 20 13 20 20 20 20 20 20 20
  when you clone and the wrong 20 20 6 20 20 20 20 20 20 20
  if not.
  
  The problem that I see with CoW is that we know how many owners a
  value has but not who they are.
  
  And you definitely dont want to clone on indexed assignment
  because very often you have a large value
  and only a small part being updated. With CoW you dont want to
  clone the value that is being assigned but
  rather the other (possibly many copies) and those we don't know.
  
  I also don't share your opinion that the cloning is unnecessary. I
  think it is more so that it is sometimes unnecessary,
  but at the time when you would not  clone you don't have enough
  information to properly make that decision.
  
  /// Jürgen
  
  

On 08/24/2014 04:04 PM, Elias Mårtenson
  wrote:


  Thanks for the explanation. With this in mind, I'm
wondering about two specific cases and how you intend to solve
them (these are are ones that gave me the biggest headache):


First of all, how will you avoid doing a clone() on every update
  using indexed assignment? You need some way to know whether or
  not an instance have to be cloned or can be modified in-place.


A related case is normal left-arrow assignment. It is not
  desirable to do a clone()
  on every assignment either. You want to basically do a
  "deferred clone" (i.e. CoW) so that the instances can be
  shared in both variables until one of them is updated.


When I was thinking about this, I couldn't get away from
  concluding that a full CoW implementation is needed in order
  to avoid all unnecessary cloning.


Regards,
Elias
  
  

On 24 August 2014 00:01, Juergen
  Sauermann 
  wrote:
  
 Hi Elias,
  
  normally APL values are not written to. The exception is
  indexed assignment.
  
  I believe the clone() call in Symbol::resolve()
  can be skipped completely.
  This is probably the most frequently used clone() case.
  I suppose copy-on-write semantics is
  achieved when all clone() calls are gone. Many of
  the remaining clone() calls are specific
  to certain functions so their performance impact should be
  small.
  
  I haven't done the above before the 1.4 release because I
  didn't want to release
  a not-so-well tested optimization.
  
  /// Jürgen
  

  
  
  On 08/23/2014 05:32 PM, Elias Mårtenson wrote:
  
  
Cool :-)