Re: MAXDSIZ limits

2018-01-30 Thread Jordan Geoghegan
ay at price points attainable to mere mortals, I can see this limit becoming more of an issue. It is somewhat disheartening to have to use sort -T on a file set and fill my disk with temp files while still having over a dozen times the MAXDSIZ available in unoccupied memory. On 01/30/18 22:35

Re: MAXDSIZ limits

2018-01-30 Thread Ted Unangst
Jordan Geoghegan wrote: > Is there any particular reason for the low sparc64 MAXDSIZ? Is there any > way for this limit to be increased as I have some large data > manipulation that needs to be done and I really would love to be able to increasing the value can be done by changing t

Re: MAXDSIZ limits

2018-01-30 Thread Jordan Geoghegan
Is there any particular reason for the low sparc64 MAXDSIZ? Is there any way for this limit to be increased as I have some large data manipulation that needs to be done and I really would love to be able to use my OS of choice for it. It would break my heart to have to have to use something other

Re: MAXDSIZ limits

2018-01-30 Thread Ted Unangst
Jordan Geoghegan wrote: > amd64 MAXDSIZ : ((paddr_t)32*1024*1024*1024) > i386 MAXDSIZ : (3UL*1024*1024*1024) > sparc64 MAXDSIZ : (8L*1024*1024*1024) > mips64 MAXDSIZ : 16UL*1024*1024*1024 > hppa MAXDSIZ : 1*1024*1024*1024UL > arm64 MAXDSIZ: ((paddr_t)16*1024*102

MAXDSIZ limits

2018-01-30 Thread Jordan Geoghegan
Hello, I am sorry if this is an ignorant question, but I am having difficulty finding info regarding MAXSIZD. A look through /sys/arch/amd64/include/vmparam.h reveals a 32GB maximum per-process limit. A look through some of the other arches I own reveals: amd64 MAXDSIZ : ((paddr_t)32

Question about MAXDSIZ=8Gb in amd64

2012-10-28 Thread Mike Korbakov
Hi, Group! What caused to limit the maximum of data 8Gb for amd64 architecture? Where we may have difficulty if to increase this value to 16-20GB? I've edited /sys/arch/amd64/include/vmparam.h for #define MAXDSIZ ((paddr_t)16*1024*1024*1024)/* max data size */ Now system work

Re: MAXDSIZ

2011-04-04 Thread Amit Kulkarni
t;> >> 2011/3/30 Tony Berth >> >>> Thank you for that clarification >>> >>> >>> On Wed, Mar 30, 2011 at 2:52 PM, Janne Johansson wrote: >>> >>>> >>>> >>>> 2011/3/30 Tony Berth >>>> >>&g

Re: MAXDSIZ

2011-04-04 Thread Tony Berth
PM, Janne Johansson wrote: >> >>> >>> >>> 2011/3/30 Tony Berth >>> >>>> which translates that the physical 4G limitation is still in place? >>>> >>> >>> Yes, as shown by the 2nd or third line in the dmesg while booting

Re: MAXDSIZ

2011-03-31 Thread Tomas Bodzar
On Wed, Mar 30, 2011 at 10:15 PM, Amit Kulkarni wrote: >>> OpenBSD just returns kernel page memory very very quickly, so it is >>> difficult for it to consume more :). But seriously, after this >>> compile, kernel was holding onto some memory. At idle (after >>> compilation) it was an excess of 30

Re: MAXDSIZ

2011-03-31 Thread Henning Brauer
* Amit Kulkarni [2011-03-31 03:54]: > Hey you guys are going to bump up the default and enable bigmem as > default too? :) Is it scheduled for this hackathon? you gotta ask our middle management which milestones are scheduled. oh. hmm. wait. whatever. -- Henning Brauer, h...@bsws.de, henn...@o

Re: MAXDSIZ

2011-03-30 Thread Eric Furman
If you use real hardware bigmen is default. On Wed, 30 Mar 2011 19:57 -0500, "Amit Kulkarni" wrote: > Henning, > > Hey you guys are going to bump up the default and enable bigmem as > default too? :) Is it scheduled for this hackathon? > > > Daniel, > > Thanks, I will look into that. Undeadly

Re: MAXDSIZ

2011-03-30 Thread Amit Kulkarni
Henning, Hey you guys are going to bump up the default and enable bigmem as default too? :) Is it scheduled for this hackathon? Daniel, Thanks, I will look into that. Undeadly is good. > OK, > > I may be way off track and totally wrong here, but isn't that worked Bob did > may be two hacketon

Re: MAXDSIZ

2011-03-30 Thread Amit Kulkarni
where? please educate me. On Wed, Mar 30, 2011 at 5:47 PM, Henning Brauer wrote: > * Amit Kulkarni [2011-03-31 00:45]: >> Nothing directly, just observing a comparison of default choice. >> OpenBSD opts for one strategy (bufcache = 10%) and Opensolaris opts >> for another (bufcache close to 100%

Re: MAXDSIZ

2011-03-30 Thread Daniel Ouellet
On 3/30/11 7:23 PM, Scott McEachern wrote: On 03/30/11 19:18, Henning Brauer wrote: * Amit Kulkarni [2011-03-31 01:09]: On Wed, Mar 30, 2011 at 5:47 PM, Henning Brauer wrote: * Amit Kulkarni [2011-03-31 00:45]: Nothing directly, just observing a comparison of default choice. OpenBSD opts for

Re: MAXDSIZ

2011-03-30 Thread Henning Brauer
* Scott McEachern [2011-03-31 01:26]: > And what are we readers to wait for, anyway? the bump. -- Henning Brauer, h...@bsws.de, henn...@openbsd.org BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting

Re: MAXDSIZ

2011-03-30 Thread Scott McEachern
On 03/30/11 19:18, Henning Brauer wrote: * Amit Kulkarni [2011-03-31 01:09]: On Wed, Mar 30, 2011 at 5:47 PM, Henning Brauer wrote: * Amit Kulkarni [2011-03-31 00:45]: Nothing directly, just observing a comparison of default choice. OpenBSD opts for one strategy (bufcache = 10%) and Opensol

Re: MAXDSIZ

2011-03-30 Thread Henning Brauer
* Amit Kulkarni [2011-03-31 01:09]: > On Wed, Mar 30, 2011 at 5:47 PM, Henning Brauer wrote: > > * Amit Kulkarni [2011-03-31 00:45]: > >> Nothing directly, just observing a comparison of default choice. > >> OpenBSD opts for one strategy (bufcache = 10%) and Opensolaris opts > >> for another (bu

Re: MAXDSIZ

2011-03-30 Thread Henning Brauer
* Amit Kulkarni [2011-03-31 00:45]: > Nothing directly, just observing a comparison of default choice. > OpenBSD opts for one strategy (bufcache = 10%) and Opensolaris opts > for another (bufcache close to 100%). you are wrong. -- Henning Brauer, h...@bsws.de, henn...@openbsd.org BS Web Service

Re: MAXDSIZ

2011-03-30 Thread Amit Kulkarni
Nothing directly, just observing a comparison of default choice. OpenBSD opts for one strategy (bufcache = 10%) and Opensolaris opts for another (bufcache close to 100%). > * Amit Kulkarni [2011-03-30 23:19]: >> Might be okay for high physical memory machines but not low. I >> remember Opensolari

Re: MAXDSIZ

2011-03-30 Thread Henning Brauer
* Amit Kulkarni [2011-03-30 23:19]: > Might be okay for high physical memory machines but not low. I > remember Opensolaris also filled out bufcache for ZFS, which was a > bloated pig. and ClaimsToBeOpen-Solaris' bufcache allocation strategies have exactly what to do with openbsd's? -- Henning

Re: MAXDSIZ

2011-03-30 Thread Amit Kulkarni
>> OpenBSD just returns kernel page memory very very quickly, so it is >> difficult for it to consume more :). But seriously, after this >> compile, kernel was holding onto some memory. At idle (after >> compilation) it was an excess of 300-500M more, instead of 1-1.3G, it >> was around 1.7G. Opens

Re: MAXDSIZ

2011-03-30 Thread roberth
On Wed, 30 Mar 2011 22:12:56 +0200 Benny Lofgren wrote: > On 2011-03-30 17.48, Jeff Ross wrote: > > On 03/30/11 05:21, Tony Berth wrote: > > Worse, an amd64 kernel looking at 8GB of real, physical ram only > > makes a wee bit under 3GB available. > > real mem = 3220111360 (3070MB) > > avail mem

Re: MAXDSIZ

2011-03-30 Thread Benny Lofgren
On 2011-03-30 17.48, Jeff Ross wrote: > On 03/30/11 05:21, Tony Berth wrote: >> I can't??? So the limit of 4G physical memory still exists? And why >> was this >> statement made from 4.4 release? > > Worse, an amd64 kernel looking at 8GB of real, physical ram only makes a > wee bit under 3GB avail

Re: MAXDSIZ

2011-03-30 Thread roberth
On Wed, 30 Mar 2011 13:15:10 -0500 Amit Kulkarni wrote: > OpenBSD just returns kernel page memory very very quickly, so it is > difficult for it to consume more :). But seriously, after this > compile, kernel was holding onto some memory. At idle (after > compilation) it was an excess of 300-500M

Re: MAXDSIZ

2011-03-30 Thread Amit Kulkarni
I have loaded the machine with processes and I think it consumed slightly more than 4G physical, running 3 compiles at once. OpenBSD userland (make -j4) + Clang/LLVM (make -j4) + ITK (make -j4). I was checking with top -s3 -1. OpenBSD just returns kernel page memory very very quickly, so it is dif

Re: MAXDSIZ

2011-03-30 Thread Jeff Ross
On 03/30/11 05:21, Tony Berth wrote: I can't??? So the limit of 4G physical memory still exists? And why was this statement made from 4.4 release? Worse, an amd64 kernel looking at 8GB of real, physical ram only makes a wee bit under 3GB available. OpenBSD 4.9-current (GENERIC.MP) #852: Sun

Re: MAXDSIZ

2011-03-30 Thread Bret S. Lambert
On Wed, Mar 30, 2011 at 01:22:19PM +0200, Tony Berth wrote: > I can't??? So the limit of 4G physical memory still exists? And why was this > statement made from 4.4 release? physical vs virtual memory, as has been explained already it's no longer 1950; we've got this thing called "swap" > > Tha

Re: MAXDSIZ

2011-03-30 Thread Otto Moerbeek
On Wed, Mar 30, 2011 at 01:22:19PM +0200, Tony Berth wrote: > I can't??? So the limit of 4G physical memory still exists? And why was this > statement made from 4.4 release? Yes, the limit still exists. Work on that is progressing, but slowly. I don't think 4.4 was shipped with bigmem enabled. CV

Re: MAXDSIZ

2011-03-30 Thread Tony Berth
I can't??? So the limit of 4G physical memory still exists? And why was this statement made from 4.4 release? Thanks On Wed, Mar 30, 2011 at 12:39 PM, Janne Johansson wrote: > > > 2011/3/30 Tony Berth > >> currently not but this machine will be a DB server (Postgresql + Mysql) >> and >> it was

Re: MAXDSIZ

2011-03-30 Thread Janne Johansson
2011/3/30 Tony Berth > currently not but this machine will be a DB server (Postgresql + Mysql) and > it was aksed if we could go beyond the 8G. > > In any case, for now, if I can address 8G physical memory is fine. > > ..which you cant. -- To our sweethearts and wives. May they never meet. -

Re: MAXDSIZ

2011-03-30 Thread Tony Berth
g a problem with 8GB? > > On Mar 28, 2011, at 8:59 AM, Tony Berth wrote: > > > Dear OBSD list members, > > > > in the meanwhile, are any changes to the following: > > > > ... > > Make amd64 machines be able to use more than 4G ram, and crank the &g

Re: MAXDSIZ

2011-03-28 Thread Ted Unangst
No, are you having a problem with 8GB? On Mar 28, 2011, at 8:59 AM, Tony Berth wrote: > Dear OBSD list members, > > in the meanwhile, are any changes to the following: > > ... > Make amd64 machines be able to use more than 4G ram, and crank the MAXDSIZ > to allow alloca

Re: MAXDSIZ

2011-03-28 Thread Otto Moerbeek
are any changes to the following: > > > > ... > > Make amd64 machines be able to use more than 4G ram, and crank the MAXDSIZ > > to allow allocations/mmap() up to 8G. > > ... > > > > as this was reported on: > > > > ... > > http://www.openbsd.org/plus44.html > > ... > > > > For example to extend it, let's say, to 16GB? > > > > Thanks > > > > Tony

Re: MAXDSIZ

2011-03-28 Thread Amit Kulkarni
wrote: > Dear OBSD list members, > > in the meanwhile, are any changes to the following: > > ... > Make amd64 machines be able to use more than 4G ram, and crank the MAXDSIZ > to allow allocations/mmap() up to 8G. > ... > > as this was reported on: > > ... > ht

MAXDSIZ

2011-03-28 Thread Tony Berth
Dear OBSD list members, in the meanwhile, are any changes to the following: ... Make amd64 machines be able to use more than 4G ram, and crank the MAXDSIZ to allow allocations/mmap() up to 8G. ... as this was reported on: ... http://www.openbsd.org/plus44.html ... For example to extend it

Re: MAXDSIZ limit 1G?

2008-01-13 Thread Marc Espie
On Sun, Jan 13, 2008 at 08:40:59AM -0800, Marcelo Schmidt wrote: > The memory limit is the full amount e.g. ulimit -d unlimited, but the datasize > (MAXDSIZ) is 1G. I also tried to recomle the 3.8 stable kernel with MAXDSIZ > > 1G. ulimit -d shows the new max but the processes run o

MAXDSIZ limit 1G?

2008-01-13 Thread Marcelo Schmidt
The memory limit is the full amount e.g. ulimit -d unlimited, but the datasize (MAXDSIZ) is 1G. I also tried to recomle the 3.8 stable kernel with MAXDSIZ > 1G. ulimit -d shows the new max but the processes run out of memory even before 1G. Has anyone seen this problem before? I cannot

Re: MAXDSIZ 1GB memory limit for process

2007-10-22 Thread Richard Storm
ry cheap, it would be really needed to increase this limit? > > i think the problem is more about what MAXDSIZ is used for than its > value. it's not as simple as just bumping a number. and changing the > meaning of a number is no easy change either. for the most part, the > limi

Re: MAXDSIZ 1GB memory limit for process

2007-10-22 Thread Ted Unangst
#x27;t you think, that now when we have 64bit platform and RAM gets > very cheap, it would be really needed to increase this limit? i think the problem is more about what MAXDSIZ is used for than its value. it's not as simple as just bumping a number. and changing the meaning of a number

Re: MAXDSIZ 1GB memory limit for process

2007-10-22 Thread mickey
On Mon, Oct 22, 2007 at 07:17:02PM +0200, Markus Hennecke wrote: > Richard Storm schrieb: > >On 10/22/07, Ted Unangst <[EMAIL PROTECTED]> wrote: > >>On 10/21/07, Richard Storm <[EMAIL PROTECTED]> wrote: > >>>Is it possible to bypass this limit somehow? > >>depends, but if it's easy to bypass a limi

Re: MAXDSIZ 1GB memory limit for process

2007-10-22 Thread Markus Hennecke
Richard Storm schrieb: On 10/22/07, Ted Unangst <[EMAIL PROTECTED]> wrote: On 10/21/07, Richard Storm <[EMAIL PROTECTED]> wrote: Is it possible to bypass this limit somehow? depends, but if it's easy to bypass a limit, it's not much of a limit. Is there possible workarounds for my program to

Re: MAXDSIZ 1GB memory limit for process

2007-10-22 Thread Matthew Szudzik
> > Do you plan to increase this limit? > > i don't think so. Could somebody explain the reason for the 1 GB maximum datasize per process in OpenBSD? Is this a limit on the heap size of a process, or is it stack size + heap size? I can imagine how this limit might arise on a 32-bit system, si

Re: MAXDSIZ 1GB memory limit for process

2007-10-22 Thread Richard Storm
On 10/22/07, Ted Unangst <[EMAIL PROTECTED]> wrote: > On 10/21/07, Richard Storm <[EMAIL PROTECTED]> wrote: > > Is it possible to bypass this limit somehow? > > depends, but if it's easy to bypass a limit, it's not much of a limit. Is there possible workarounds for my program to allocate more memor

Re: MAXDSIZ 1GB memory limit for process

2007-10-21 Thread Ted Unangst
On 10/21/07, Richard Storm <[EMAIL PROTECTED]> wrote: > Is it possible to bypass this limit somehow? depends, but if it's easy to bypass a limit, it's not much of a limit. > Do you plan to increase this limit? i don't think so.

MAXDSIZ 1GB memory limit for process

2007-10-21 Thread Richard Storm
Is it possible to bypass this limit somehow? Do you plan to increase this limit?

Re: Maximum MAXDSIZ

2005-06-03 Thread Ted Unangst
On Sat, 28 May 2005, Fernando Braga wrote: > Looking vmparam.h, I see maximum data size for a process is limited to 1GB. > > As these servers came with 4GB RAM (and I really need this memory), > I'd like to know if it is possible to raise this limit. this is like asking what happens when you eat

Maximum MAXDSIZ

2005-05-28 Thread Fernando Braga
Hi, I run two squid servers and their main process keeps crashing every time data size tries to go over 1GB. Looking vmparam.h, I see maximum data size for a process is limited to 1GB. As these servers came with 4GB RAM (and I really need this memory), I'd like to know if it is possible to raise