Hi Matthew, Sorry for the delay in replying.
On Sun, 15 Jan 2017 07:40:05 -0700, Matthew Flatt <mfl...@cs.utah.edu> wrote: >Do you know how much memory the process was using when it crashed? Unfortunately I don't. As I said, I am not the administrator of this server ... I was informed about the crash a few days after the fact. My application logs and the DBMS logs show only a handful of users active at the time - and they weren't doing anything stressful. There *should* have been several GB of memory available. >This crash looks like a case of using more resources than allowed for a >process in your system (although not in the usual way). It's always >possible that the problem is a bug in Racket. I don't think it can be a >bug in your program --- unless the bug allows more memory use than >intended. Hmmm. I don't have memory use limited. My application is a kind of search engine, and it is difficult to predict its memory use. If someone asks for 1 page of results and the query returns 100,000 rows ... Handling a ridiculous query could require sh!tloads of memory, but that memory would be released to GC immediately afterwards. If the heap were to be expanded to deal with the temporary situation, I would expect an out-of-memory related problem then. But that didn't happen here. The logs don't show the user's queries per se, but they do show how long the queries took to service, which is indicative of the load. Nothing stressful was happening at the time of the crash. >Details: > >The SIG_SEGV failure is consistent with the reported mprotect() >failure, and the mprotect() failure is ENOMEM. > >Since ENOMEM can be reported when mprotect() is used on an address >range that does not belong to the process, the error could reflect a >bug in Racket. But the later crash is SIGSEGV_ACCERR, so I don't think >that's it: the address appears to be mapped but without write >permission, where the failing mprotect() call was intended to add write >permission. > >The other possibly --- the one that's more consistent with the error >name --- is that the kernel has run out of resources to perform the >mprotect() change. Normally, if a Racket process is using lots of >memory, it gets an out-of-memory condition trying to allocate new >memory. But the kernel has an additional constraint on the size of the >page-mapping table, and ENOMEM from mprotect() is related to that >constraint. That is, a process can use less memory than allowed but at >a granularity that creates too many page mappings within the process. Hmmm. This is a cloud server, so there's no swapping and VMM overcommit is disabled. But if the machine really is out of memory, shouldn't I see an error to that effect instead of a seg fault? Couple questions: When Racket extends its heap, does it commit (touch) pages immediately or just get additional address space and consume it lazily? And does Racket ever release address space back to the system? E.g., after a full GC, or if usage over some period of time is much less than the current heap size. >In my Linux installations, /proc/sys/vm/max_map_count reports 65530. Ditto. >If every page has a different mapping, that limit can be reached with >256MB of memory use, potentially. Since Racket generally changes page >properties in 16k chunks, 1GB is a more realistic lower bound. And >since it's very unlikely for every 16k chunk to have a different >mapping and permissions than its adjacent 16k chunks, you shouldn't >expect to hit the mapping limit at around 1GB. If a process is using >multiple GBs of memory, though, that could be it. I have seen the application use up to ~1GB during stress testing - but more typically the memory use is 200-300MB, which is insignificant. Is there any way to check if mapping limits were exceeded? George -- You received this message because you are subscribed to the Google Groups "Racket Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to racket-users+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
