On 9/26/05, jonathan chew <[EMAIL PROTECTED]> wrote:
Fortunately, we control the HW our systems are deployed on and can enforce a 4GB RAM limit if required.
We already have such a driver but have not found a way to force the use of 2M pages! Is there a new DDI interface to request large page size?
Not really. We buffer data up and need to have it DMA in real-time to our device which futher processes it and passes on the processed data to another machine. The window of time once we have committed to delivering the data is strictly enforced and failure to do so effectively shuts down the other system. Way back in SunOS 5.4, we went as far as writing a pseudo driver to SOFTLOCK memory as we found that it was not enough to mlock() memory since we took page faults on the corresponding TTEs. As I noted previously in the beginning of this thread, we use our own version of physio() that assumes properly wired down memory and thus differs from the stock version in that it does not bother with the locking logic at all.
By writing our own device driver, do you mean one to export 4GB PA range to user-land or our own segment driver?
Thanks in advance for your time,
--Marc
There may be a slightly better way to allocate memory from the lgroup
containing the least significant physical memory using
lgrp_affinity_set(3LGRP), meminfo(2), and madvise(MADV_ACCESS_LWP).
Unfortunately, none of these ways are guaranteed to always allocate
memory in the least significant 4 gigabytes of physical memory. For
example, if the node with the least significant physical memory contains
more than 4 gigabytes of RAM, physical memory may come from this node,
but be above the least significant 4 gigabytes. :-(
Fortunately, we control the HW our systems are deployed on and can enforce a 4GB RAM limit if required.
I spoke to one of our I/O guys to see whether there is a prescribed way
to allocate physical memory in the least significant 4 gig for DMA from
userland. Solaris doesn't provide an existing way. The philosophy is
that users shouldn't have to care and that the I/O buffer will be copied
to/from the least significant physical memory inside the kernel if the
device can only reach that far to DMA. I think that you may be able to
write a driver that can allocate physical memory in the desired range
and allow access to it with mmap(2) or ioctl(2).
We already have such a driver but have not found a way to force the use of 2M pages! Is there a new DDI interface to request large page size?
I'd like to understand more about your situation to get a better idea of
the constraints and whether there is a viable solution or the need for one.
What is your device? Can you afford to let the kernel copy the user's
buffer into another buffer which is below 4 gig to DMA to your device?
If not, would it make sense to write your own driver?
Not really. We buffer data up and need to have it DMA in real-time to our device which futher processes it and passes on the processed data to another machine. The window of time once we have committed to delivering the data is strictly enforced and failure to do so effectively shuts down the other system. Way back in SunOS 5.4, we went as far as writing a pseudo driver to SOFTLOCK memory as we found that it was not enough to mlock() memory since we took page faults on the corresponding TTEs. As I noted previously in the beginning of this thread, we use our own version of physio() that assumes properly wired down memory and thus differs from the stock version in that it does not bother with the locking logic at all.
By writing our own device driver, do you mean one to export 4GB PA range to user-land or our own segment driver?
Thanks in advance for your time,
--Marc
_______________________________________________ perf-discuss mailing list perf-discuss@opensolaris.org
