Re: gnumach and gcc 4.0 (patch 1)

2006-01-21 Thread Thomas Bushnell BSG 
"Alfred M\. Szmidt" <[EMAIL PROTECTED]> writes:

>I'm going to commit the following patch unless someone vetoes:
>
> Last time I checked, things don't work like that.

Thomas has permission to approve patches; that means that he can
commit patches on his own authority when he thinks it appropriate.

It is, of course, a good thing to request review, as he has done.





___
Bug-hurd mailing list
Bug-hurd@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-hurd

Re: [patch #332] POSIX record file locking

2006-01-21 Thread Thomas Bushnell BSG 
Michael Banck <[EMAIL PROTECTED]> writes:

> You are totally right of course.  In fact, Marcus just asked yesterday
> on IRC whether those notifications of Savannah really serve a purpose,
> as the patch is not sent along inlined.

I have no objections at all to the Savannah notifications.  I'm only
making a comment about what is more likely to prod me into responding
to a patch.



___
Bug-hurd mailing list
Bug-hurd@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-hurd

Re: [patch #4818] Dynamic memory allocation for Linux Device drivers in glue. [PATCH] [LONG]

2006-01-21 Thread Gianluca Guida 
Hi,

For people that doesn't like mails coming from savannah bug tracker
(this involves me too) I inline the patch.
It's quite long (1k lines) but well, just to give an idea of what it
does, here it is.

2006-01-20  Gianluca Guida  <[EMAIL PROTECTED]>

   * vm/pmap.h (pmap_is_dma, pmap_is_normal): New functions.

   * vm/page.h (VM_PAGE_DMA): New macro.
   (vm_page_queue_free): Variable removed.
   (vm_page_queue_free_dma, vm_page_queue_free_normal): New
   variables.
   (vm_page_physaddr_lookup, vm_page_grab_flags)
   (vm_page_grab_contiguous_pages_flags): New functions.
   (first_phys_addr, last_phys_addr): Declarations removed.
   (phys_first_addr, phys_last_addr): New declarations.

   * vm/vm_resident.c (vm_page_free_bitmap)
   (vm_page_free_bitmap_bitsz): New variables.
   (vm_page_free_bitmap_set, vm_page_free_bitmap_unset)
   (vm_page_free_bitmap_alloc, vm_page_array_init, vm_page_array_add)
   (vm_page_physaddr_lookup): New functions.
   (vm_page_bootstrap): Initialize vm_page_queue_free_normal and
   vm_page_queue_free_dma. Call vm_page_free_bitmap_alloc.
   (pmap_startup): Call vm_page_array_init. Add page to vm_page_array
   for physaddress-to-vm_page_t resolution. Do not release page in
   any particular order.
   (vm_page_create): Add page to vm_page_array.
   (vm_page_grab_flags, vm_page_grab_contiguous_pages_queue)
   (vm_page_grab_contiguous_flags): New functions.
   (vm_page_grab, vm_page_grab_contiguous_pages): Function rewritten.
   (vm_page_release): Select proper free list based on page type (DMA
   or non-DMA). Use Mach's queue handling funcitons not .next and
   .prev pointers manually. Set vm_page_free_bitmap when freeing a
   page.

   * i386/i386at/model_dep.c (use_all_mem): Always set to 1.
   (init_alloc_aligned): Do not use any particular oder during
   bootstrap allocation.
   (alloc_dma_mem): Function removed.

   * kern/startup.c: Include .
   (setup_main): Calculate memory size using phys_last_addr and
   phys_first_addr.

   * i386/intel/pmap.c (vm_page_normal_first, vm_page_normal_last)
   (vm_page_dma_first, vm_page_dma_last): New variables.
   (pmap_bootstrap): Set proper values to vm_page_normal_* and
   vm_page_dma_* based on system's memory.
   (pmap_is_dma, pmap_is_normal): New functions.

   * kern/startup.c: Removed /* XXX */ comments, since phys_last_addr
   and phys_first_addr are exported by pmap.c now.

   * linux/dev/glue/kmem.c: Include 
   (MEM_CHUNK_SIZE, MEM_CHUNKS, NBPW): Macro removed.
   (pages_free, memlist, num_page_collect, linux_mem_avail):
   Variables removed.
   (memlist_dma, memlist_nml): New variables.
   (linux_kmem_init): No initialization needed anymore (all code
   removed).
   (coalesce_blocks): Coalesce separately DMA memory blocks and
   non-DMA memory blocks.
   (linux_kmalloc): Check for DMA in priority argument. Select proper
   memlist to get blocks from. Pass dma information to
   __get_free_pages.
   (linux_kfree): Select proper memlist to free memory into.
   (collect_kmalloc_pages): Function removed.
   (collect_kmalloc_pages_dma, collect_kmalloc_pages_nml): New
   functions.
   (__get_free_pages, free_pages): Functions rewritten.

   * linux/dev/init/main.c (CONTIG_ALLOC, NBPW): Macro removed.
   (CONTIG_ALLOC_ORDER): New macro.
   (linux_init): Use __getfreepages to allocate PCI initialization
   memory and free_pages to free it.
   (alloc_contig_mem, free_contig_mem): Functions removed.

diff -ru gnumach-vanilla/i386/i386at/model_dep.c gnumach/i386/i386at/model_dep.c
--- gnumach-vanilla/i386/i386at/model_dep.c 2004-11-28 18:29:35.0 
+0100
+++ gnumach/i386/i386at/model_dep.c 2006-01-21 19:46:02.0 +0100
@@ -86,16 +86,7 @@
 /* Configuration parameter:
if zero, only use physical memory in the low 16MB of addresses.
Only SCSI still has DMA problems.  */
-#ifdef LINUX_DEV
-#define use_all_mem 1
-#else
-#include "nscsi.h"
-#ifNSCSI > 0
-#define use_all_mem 0
-#else
 #define use_all_mem 1
-#endif
-#endif

 extern charversion[];

@@ -454,7 +445,6 @@
vm_offset_t addr;
extern char start[], end[];
int i;
-   static int wrapped = 0;

/* Memory regions to skip.  */
vm_offset_t cmdline_start_pa = boot_info.flags & MULTIBOOT_CMDLINE
@@ -474,25 +464,8 @@
/* Page-align the start address.  */
avail_next = round_page(avail_next);

-   /* Start with memory above 16MB, reserving the low memory for later. */
-   if (use_all_mem && !wrapped && phys_last_addr > 16 * 1024*1024)
- {
-   if (avail_next < 16 * 1024*1024)
- avail_next = 16 * 1024*1024;
-   else if (avail_next == phys_last_addr)
- {
-   /* We have used all the memory above 16M

[patch #4818] Dynamic memory allocation for Linux Device drivers in glue.

2006-01-21 Thread Gianluca Guida 

URL:
  

 Summary: Dynamic memory allocation for Linux Device drivers
in glue.
 Project: The GNU Hurd
Submitted by: gianluca
Submitted on: Sat 01/21/06 at 19:39
Category: GNU Mach
Priority: 6
  Status: None
 Privacy: Public
 Assigned to: None
Originator Email: 
 Open/Closed: Open

___

Details:

This patches rewrite GNU Mach's physical page allocation routines and make
glued linux drivers use these new routines instead of limited, specific and
redundant routines existing in the  actual linux glue.

vm/vm_resident.c now can handle (thanks to information given from the
architecture-dependant component) two specific zones of physical memory: DMA
capable and non-DMA capable.
In the x86 we suffer a limit from the ISA bus, that can just let its DMA chip
access first 16Mb of memory. Thus, when ISA drivers need memory for DMA
operation, we must be sure to supply pages below that limit.

GNU Mach didn't had support for special zones in the page, perhaps because
when people were hacking it 16Mb seemed to be a far limit in x86 systems.
Actual Linux glue solved the problem by allocating a fixed, unfreeable amount
of ram below 16Mb at bootstrap and limiting the whole linux subsection to use
that memory only, which in fact was stolen from the system. Furthermore, and
this is very frequent in Mach, it added yet another fixed and arbitrary limit
for memory usage to generic components.

What this patch basically does is to add multizone allocation  (DMA and
non-DMA) to GNUMach and hack the glue to make it uses this new feature, thus
making allocation from linux drivers dynamical. As a result, linux glue code
is simpler. As a side effect, some unused, obsolete native device drivers in
Mach (i386/i386at ones) get broken, since they use a slow routine to allocate
DMA memory that now has been removed.

This work is still unfinished (the pageout daemon has to free unused
collected memory from kmalloc/kfree, but that's an easy task). I posted this
patch for testing only. I actually had installed and normally used a Debian
GNU/Hurd system with this patch, but I would be happy if people would test
it.

Thanks for reading this,
Gianluca

2006-01-20  Gianluca Guida  <[EMAIL PROTECTED]>

* vm/pmap.h (pmap_is_dma, pmap_is_normal): New functions.

* vm/page.h (VM_PAGE_DMA): New macro.
(vm_page_queue_free): Variable removed.
(vm_page_queue_free_dma, vm_page_queue_free_normal): New
variables.
(vm_page_physaddr_lookup, vm_page_grab_flags)
(vm_page_grab_contiguous_pages_flags): New functions.
(first_phys_addr, last_phys_addr): Declarations removed.
(phys_first_addr, phys_last_addr): New declarations.

* vm/vm_resident.c (vm_page_free_bitmap)
(vm_page_free_bitmap_bitsz): New variables.
(vm_page_free_bitmap_set, vm_page_free_bitmap_unset)
(vm_page_free_bitmap_alloc, vm_page_array_init, vm_page_array_add)
(vm_page_physaddr_lookup): New functions.
(vm_page_bootstrap): Initialize vm_page_queue_free_normal and
vm_page_queue_free_dma. Call vm_page_free_bitmap_alloc.
(pmap_startup): Call vm_page_array_init. Add page to vm_page_array
for physaddress-to-vm_page_t resolution. Do not release page in
any particular order.
(vm_page_create): Add page to vm_page_array.
(vm_page_grab_flags, vm_page_grab_contiguous_pages_queue)
(vm_page_grab_contiguous_flags): New functions.
(vm_page_grab, vm_page_grab_contiguous_pages): Function rewritten.
(vm_page_release): Select proper free list based on page type (DMA
or non-DMA). Use Mach's queue handling funcitons not .next and
.prev pointers manually. Set vm_page_free_bitmap when freeing a
page.

* i386/i386at/model_dep.c (use_all_mem): Always set to 1.
(init_alloc_aligned): Do not use any particular oder during
bootstrap allocation.
(alloc_dma_mem): Function removed.

* kern/startup.c: Include .
(setup_main): Calculate memory size using phys_last_addr and
phys_first_addr.

* i386/intel/pmap.c (vm_page_normal_first, vm_page_normal_last)
(vm_page_dma_first, vm_page_dma_last): New variables.
(pmap_bootstrap): Set proper values to vm_page_normal_* and
vm_page_dma_* based on system's memory.
(pmap_is_dma, pmap_is_normal): New functions.

* kern/startup.c: Removed /* XXX */ comments, since phys_last_addr
and phys_first_addr are exported by pmap.c now.

* linux/dev/glue/kmem.c: Include 
(MEM_CHUNK_SIZE, MEM_CHUNKS, NBPW): Macro removed.
(pages_free, memlist, num_page_collect, linux_mem_avail):
Va

Re: [patch #4818] Dynamic memory allocation for Linux Device drivers in glue. [PATCH] [LONG]

2006-01-21 Thread Alfred M\. Szmidt 
Nice patch, overall it looks great.  A bit more testing and another
eyeball check, and it should go in.

PS, I'm assuming that the inclusion of ../rtl8139.c was a mistake. ;)


___
Bug-hurd mailing list
Bug-hurd@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-hurd

Re: [patch #4818] Dynamic memory allocation for Linux Device drivers in glue. [PATCH] [LONG]

2006-01-21 Thread Gianluca Guida 
On 1/21/06, Alfred M. Szmidt <[EMAIL PROTECTED]> wrote:
> PS, I'm assuming that the inclusion of ../rtl8139.c was a mistake. ;)

Whops!
Yes it does, it's not even needed anymore, since some patch in the
debian (which I hope is going to be committed soon) already fixes
this!

Posting cleaned patch soon.

Thanks,
Gianluca

--
It was a type of people I did not know, I found them very strange and
they did not inspire confidence at all. Later I learned that I had been
introduced to electronic engineers.
  E. W. Dijkstra


___
Bug-hurd mailing list
Bug-hurd@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-hurd

[patch #4818] Dynamic memory allocation for Linux Device drivers in glue.

2006-01-21 Thread Gianluca Guida 

Follow-up Comment #1, patch #4818 (project hurd):

This is a new version of the patch.

No major improvement, it's just a cleaning of the previous patch:

- Removed the rtl8139.c, which I forgot to remove in the previous patch;
- Removed some debugging printf that I forgot, in pure StoMach style;

Happy Testing,
Gianluca
___

Additional Item Attachment:

File name: vm_resident-zoned-patch-2.diff Size:42 KB
vm_resident zoned patch 2 (minor patch cleaning, no changelogs needed)


___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-hurd mailing list
Bug-hurd@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-hurd

Re: [patch #4818] Dynamic memory allocation for Linux Device drivers in glue.

2006-01-21 Thread Gianluca Guida 
For inline patches fans, here it is:

This is a new version of the patch.

No major improvement, it's just a cleaning of the previous patch:

- Removed the rtl8139.c, which I forgot to remove in the previous patch;
- Removed some debugging printf that I forgot, in pure StoMach style;

Happy Testing,
Gianluca

diff -ru gnumach-vanilla/i386/i386at/model_dep.c gnumach/i386/i386at/model_dep.c
--- gnumach-vanilla/i386/i386at/model_dep.c 2004-11-28 18:29:35.0 
+0100
+++ gnumach/i386/i386at/model_dep.c 2006-01-22 01:08:09.0 +0100
@@ -86,16 +86,7 @@
 /* Configuration parameter:
if zero, only use physical memory in the low 16MB of addresses.
Only SCSI still has DMA problems.  */
-#ifdef LINUX_DEV
-#define use_all_mem 1
-#else
-#include "nscsi.h"
-#ifNSCSI > 0
-#define use_all_mem 0
-#else
 #define use_all_mem 1
-#endif
-#endif

 extern charversion[];

@@ -454,7 +445,6 @@
vm_offset_t addr;
extern char start[], end[];
int i;
-   static int wrapped = 0;

/* Memory regions to skip.  */
vm_offset_t cmdline_start_pa = boot_info.flags & MULTIBOOT_CMDLINE
@@ -474,25 +464,8 @@
/* Page-align the start address.  */
avail_next = round_page(avail_next);

-   /* Start with memory above 16MB, reserving the low memory for later. */
-   if (use_all_mem && !wrapped && phys_last_addr > 16 * 1024*1024)
- {
-   if (avail_next < 16 * 1024*1024)
- avail_next = 16 * 1024*1024;
-   else if (avail_next == phys_last_addr)
- {
-   /* We have used all the memory above 16MB, so now start on
-  the low memory.  This will wind up at the end of the list
-  of free pages, so it should not have been allocated to any
-  other use in early initialization before the Linux driver
-  glue initialization needs to allocate low memory.  */
-   avail_next = 0x1000;
-   wrapped = 1;
- }
- }
-
/* Check if we have reached the end of memory.  */
-if (avail_next == (wrapped ? 16 * 1024*1024 : phys_last_addr))
+if (avail_next == phys_last_addr)
return FALSE;

/* Tentatively assign the current location to the caller.  */
@@ -585,107 +558,3 @@
!(((boot_info.mem_lower * 1024) <= x) && (x < 1024*1024)));
 }

-#ifndef NBBY
-#define NBBY   8
-#endif
-#ifndef NBPW
-#define NBPW   (NBBY * sizeof(int))
-#endif
-#define DMA_MAX(16*1024*1024)
-
-/*
- * Allocate contiguous pages below 16 MB
- * starting at specified boundary for DMA.
- */
-vm_offset_t
-alloc_dma_mem(size, align)
-   vm_size_t size;
-   vm_offset_t align;
-{
-   int *bits, i, j, k, n;
-   int npages, count, bit, mask;
-   int first_page, last_page;
-   vm_offset_t addr;
-   vm_page_t p, prevp;
-
-   npages = round_page(size) / PAGE_SIZE;
-   mask = align ? (align - 1) / PAGE_SIZE : 0;
-
-   /*
-* Allocate bit array.
-*/
-   n = ((DMA_MAX / PAGE_SIZE) + NBPW - 1) / NBPW;
-   i = n * NBPW;
-   bits = (unsigned *)kalloc(i);
-   if (bits == 0) {
-   printf("alloc_dma_mem: unable alloc bit array\n");
-   return (0);
-   }
-   bzero((char *)bits, i);
-
-   /*
-* Walk the page free list and set a bit for
-* every usable page in bit array.
-*/
-   simple_lock(&vm_page_queue_free_lock);
-   for (p = vm_page_queue_free; p; p = (vm_page_t)p->pageq.next) {
-   if (p->phys_addr < DMA_MAX) {
-   i = p->phys_addr / PAGE_SIZE;
-   bits[i / NBPW] |= 1 << (i % NBPW);
-   }
-   }
-
-   /*
-* Search for contiguous pages by scanning bit array.
-*/
-   for (i = 0, first_page = -1; i < n; i++) {
-   for (bit = 1, j = 0; j < NBPW; j++, bit <<= 1) {
-   if (bits[i] & bit) {
-   if (first_page < 0) {
-   k = i * NBPW + j;
-   if (!mask
-   || (((k & mask) + npages)
-   <= mask + 1)) {
-   first_page = k;
-   if (npages == 1)
-   goto found;
-   count = 1;
-   }
-   } else if (++count == npages)
-   goto found;
-   } else
-   first_page = -1;
-   }
-   }
-   addr = 0;
-   goto out;
-
- found:
-   /*
-* Remove pages from the free list.
-*/
-   addr = first_page * PAGE_SIZE;
-   la

Re: [patch #4818] Dynamic memory allocation for Linux Device drivers in glue. [PATCH] [LONG]

2006-01-21 Thread Thomas Bushnell BSG 
Gianluca Guida <[EMAIL PROTECTED]> writes:

> For people that doesn't like mails coming from savannah bug tracker
> (this involves me too) I inline the patch.
> It's quite long (1k lines) but well, just to give an idea of what it
> does, here it is.

Obviously vm bugs are very painful to debug when they happen.  Can you
briefly describe what testing this patch has undergone?  (How
stressful a test, importantly?)



___
Bug-hurd mailing list
Bug-hurd@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-hurd

Re: [patch #4818] Dynamic memory allocation for Linux Device drivers in glue. [PATCH] [LONG]

2006-01-21 Thread Alfred M\. Szmidt 
   > For people that doesn't like mails coming from savannah bug
   > tracker (this involves me too) I inline the patch.  It's quite
   > long (1k lines) but well, just to give an idea of what it does,
   > here it is.

   Obviously vm bugs are very painful to debug when they happen.  Can
   you briefly describe what testing this patch has undergone?  (How
   stressful a test, importantly?)

To quote Gianluca' first mail: I actually had installed and normally
used a Debian GNU/Hurd system with this patch, but I would be happy if
people would test it.



___
Bug-hurd mailing list
Bug-hurd@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-hurd

Re: [patch #4818] Dynamic memory allocation for Linux Device drivers in glue. [PATCH] [LONG]

2006-01-21 Thread Gianluca Guida 
Hi,

On 1/22/06, Thomas Bushnell BSG <[EMAIL PROTECTED]> wrote:
> Obviously vm bugs are very painful to debug when they happen.  Can you
> briefly describe what testing this patch has undergone?  (How
> stressful a test, importantly?)

Yes, this is THE question.

Well, I started writing and testing this code around 6 months / 1 year
ago. I used it as a base for my StoMach branch of GNU Mach (which
basically is GNUMach 1.x without linux glue and exporting COM objects
for OSKit's drivers). I remember to have done some test to check that
the code was doing what I expected. While porting the patch to GNUMach
I simply added the linux/dev/init/main.c and linux/dev/glue/kmem.c
stuff, which is very simple and doesn't require particular testing.

The vm_resident's multipage allocator, that is
vm_page_grab_contiguous_queue, require some more intensive testing.
When I wrote it, infact, I was more interested in showing that it was
possible to use oskit drivers in GNUMach 1.x with few hacks, so I
basically extended the previous existing function (I have some serious
doubts about the alignment functionality I added).

Anyways, that function needs to be rewritten. I am going to post a
rewrite of that function, that should have an O(1) behaviour. I'd
prefer to test intensively that.

Ah, linux's glue page collecting (which is now disabled) need to be
tested too, but again, those bugs are visible, as opposed to vm's
ones.

This is an experimental patch, it seems to works very well -- both in
stomach and in GNUMach -- but do not take it as bug free , even
because it's touching really serious internal parts of Mach.

Hope this helps.
Gianluca

--
It was a type of people I did not know, I found them very strange and
they did not inspire confidence at all. Later I learned that I had been
introduced to electronic engineers.
  E. W. Dijkstra


___
Bug-hurd mailing list
Bug-hurd@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-hurd