Hi Russ,
it has turned out to be very difficult to find a reliable way to
figure the memory that is available for
a process like the GNU APL interpreter. One (of several) reasons
for that is that GNU linux tells you
that it has more memory than it rea
Since the Linux "free" command knows the difference between how much real
memory vs. virtual memory is available, it may be useful to use a similar
method.
Blake
On Tue, Dec 28, 2021 at 11:34 AM Dr. Jürgen Sauermann <
mail@jürgen-sauermann.de> wrote:
> Hi Russ,
>
> it has turned out to be very
Aha, the dreaded Linux overcommit and 'oom killer' I've heard about.
Increasing ulimit -v does indeed allow somewhat larger datasets... but
nowhere near as large as the text I was testing would still use. Oh well.
Iterative solution it is, for now!
I will see how I can perhaps optimise that furthe
Hi Blake,
I know. The problem is that other processes may, just as GNU APL
does, allocate memory
successfully via malloc(). malloc, on "success" returns a non-zero
virtual memory address.
The fact that malloc returns a non-zero address (= indic
Hi Jürgen,
First, to be sure you know where I'm coming from. I know malloc() and
sbrk() pretty well. I've written malloc's before.
It is utter news to me that malloc would return a pointer that you can't
use - even in over-commit situations. Theoretically, in over-commit
situations, if you acc
Hi there!
You may find something helpful/useful in a previous discussion about the
similar program
https://lists.gnu.org/archive/html/bug-apl/2016-09/msg00026.html
Hope it helps.
Regards,
Ala'a
On Tue, Dec 28, 2021 at 11:09 PM Blake McBride wrote:
> Hi Jürgen,
>
> First, to be sure you know
Come to think of it, I think it best just to rely on malloc as-is. Allow
over-commit to be handled (slowly) by the OS.
If the program causes the system to slow down because of over-commit,
that's their problem.
If malloc is returning a pointer you can't use, that's an OS or library
problem. I'd
Actually, the default memory allocator in glibc uses mmap and not sbrk, if
I'm not mistaken. However, they both do the same thing at the end of the
day, and it's to request more pages from the kernel.
Unfortunately, Linux malloc never returns NULL. Even if you try to allocate
a petabyte in one all
I think what you're talking about and what I am talking about are not
exactly the same thing. I can see what you are saying, but that is a
contrived example. For example:
Level 1: you are using the RAM that exists (not over-committed)
Level 2: you are using more RAM than you have causing over
Hi Rus,
looks like the outer product is a not needed - you have the unique words
and along the line you got the word count too.
you take the sorted word vector
swv ←'aa' 'bb' 'bb' 'cc' 'cc' 'ff' 'gg'
then you create a partition vector from it
pv←+\1,~2≡/swv
pv
1 2 2 3 3 4 5
partition for wc
Hi,
I have spent many hours to do a proper memory handling. The current
solution is not perfect but will work
in many cases. If anybody has suggestions how to improve then I'll be
happy to use them.
The current approach is roughly this (if I remember correctly)
1. if the user specifies a *ul
> ? ??#DEBUG# ? ? 'unique words:',WL
> > ? wcl ? 0?0
> > ? idx ? 1
> > ? len ? ?WL
> > count:
> > ? ??#DEBUG# ? ? idx
> > ? ?(idx>len)/done
> > ? word ? ?idx?WL
> > ? ??#DEBUG# ? ? word
> > ? wc ? +/(word??D)
> > ? wcl ? w
This is somewhat tangential but,
On Tue, Dec 28, 2021 at 01:25:19PM -0600, Blake McBride wrote:
Level 1: you are using the RAM that exists (not over-committed)
Level 2: you are using more RAM than you have causing over-commit and
paging.
Level 3: you allocate more memory than exists in RA
nt
> done:
> ? c ? h?[2] (WL)[?wcl],[0.5]wcl[?wcl]
> ? timeEnd ? timeMs
> ? ? ? 'Time:',(timeEnd-timeStart),'ms'
> ?
>
> ?r ? n wcOuterProd fname
> ? ?? ;D;wl;uniqwords;wordcounts;sortOrder
> ? D ? lowerAndStrip (?fi
14 matches
Mail list logo
