Ax88172 vs FreeBSD USB stack

2003-03-28 Thread Bill Paul

So. I picked up a Linksys USB200M USB 2.0 ethernet adapter that uses
the ASIX Electronics AX88172 chip, and I started cobbling together a
driver. This chip uses a series of vendor specific commands to do
things like read/write the MII management interface on the MAC,
read/write the SROM, set the RX filter, multicast hash table, etc.
I can do all this no problem. The Linksys NIC uses a RealTek 8201L
PHY, and I can attach it and negotiate a link.

However I can't send or receive any packets.

Like all the other USB NICs, packet transfer is supposed to done
via bulk in/bulk out endpoints, and I can open these endpoints
and initiate transfers just fine, but nothing ever happens. Transmit
attempts don't yield any packets on the wire, and I never get any
RX transfer completions. (I can see the activity LED on the NIC
blink when a frame arrives though.) One thing I have not attempted
to do yet (but may try this weekend) is to directly read the RX or
TX SRAM (there are commands to let you do this).

Frankly, I'm a bit stumped. It looks as though I'm doing everything
correctly, but I can't explain why the bulk transfers don't work.
Clearly, the control endpoint works, since I can issue commands. There
doesn't appear to be any special command that one has to issue to
enable/disable the receiver or transmitter on the MAC.

I have only one possible explanation. The manual for the AX88172
(http://www.freebsd.org/~wpaul/ASIX/Ax88172.PDF) says the following:

"The AX88172 supports 2 interfaces, the interface 0 is Data Interface
and interface 1 is for Communication interface."

However the USB stack disagrees with the manual on this point: it
steadfastly insists that configuration 1 has only 1 interface. So
my question, for any USB gurus out there, is: is it possible that
the USB stack is wrong on this point, and I really need to be
using interface 1 for bulk in/out transfers? I'm a little suspicious
at this point, because the stack can't seem to even read the
device ID string correctly. (It identifies the chip only as "vendor
0x77b, device 0x2226".)

-Bill

--
=
-Bill Paul(510) 749-2329 | Senior Engineer, Master of Unix-Fu
 [EMAIL PROTECTED] | Wind River Systems
=
  "If stupidity were a handicap, you'd have the best parking spot."
=
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


Re: Ax88172 vs FreeBSD USB stack

2003-03-28 Thread Bernd Walter
On Fri, Mar 28, 2003 at 10:38:58AM -0800, Bill Paul wrote:
> 
> So. I picked up a Linksys USB200M USB 2.0 ethernet adapter that uses
> the ASIX Electronics AX88172 chip, and I started cobbling together a
> driver. This chip uses a series of vendor specific commands to do
> things like read/write the MII management interface on the MAC,
> read/write the SROM, set the RX filter, multicast hash table, etc.
> I can do all this no problem. The Linksys NIC uses a RealTek 8201L
> PHY, and I can attach it and negotiate a link.
> 
> However I can't send or receive any packets.
> 
> Like all the other USB NICs, packet transfer is supposed to done
> via bulk in/bulk out endpoints, and I can open these endpoints
> and initiate transfers just fine, but nothing ever happens. Transmit
> attempts don't yield any packets on the wire, and I never get any
> RX transfer completions. (I can see the activity LED on the NIC
> blink when a frame arrives though.) One thing I have not attempted
> to do yet (but may try this weekend) is to directly read the RX or
> TX SRAM (there are commands to let you do this).
> 
> Frankly, I'm a bit stumped. It looks as though I'm doing everything
> correctly, but I can't explain why the bulk transfers don't work.
> Clearly, the control endpoint works, since I can issue commands. There
> doesn't appear to be any special command that one has to issue to
> enable/disable the receiver or transmitter on the MAC.
> 
> I have only one possible explanation. The manual for the AX88172
> (http://www.freebsd.org/~wpaul/ASIX/Ax88172.PDF) says the following:
> 
> "The AX88172 supports 2 interfaces, the interface 0 is Data Interface
> and interface 1 is for Communication interface."

If that is true - the device has a strange design.
Normaly devices use single interfaces with multiple commucations
channels.
An interface in USB wording is something like a virtual device -
similar to functions on pci.
E.g. you can have an umass device on interface 0 and a keyboard on
interface 1.
The FreeBSD USB stack supports mutltiple interfaces, but currently
doesn't support probing different drivers for different interfaces on
the same device.

However you can try usbctl from usbutils to get an overview of the
device.
I have made a port, it is actually being reviewed by a port commiter,
which you can get on: http://www.cosmo-project.de/~bernd/usbutil.tgz

> However the USB stack disagrees with the manual on this point: it
> steadfastly insists that configuration 1 has only 1 interface. So
> my question, for any USB gurus out there, is: is it possible that
> the USB stack is wrong on this point, and I really need to be
> using interface 1 for bulk in/out transfers? I'm a little suspicious
> at this point, because the stack can't seem to even read the
> device ID string correctly. (It identifies the chip only as "vendor
> 0x77b, device 0x2226".)

Strange - let us see what usbctl says - it never failed for me.

BTW: I doubt that USB 1.x vs USB 2.0 make a difference in the problem
you are seeing, but in case you want to test later.
The EHCI patch is available under:
http://www.cosmo-project.de/~bernd/ehci.patch
It is currently not tested against any device, but the controller
probes fine and scanning the root hub (without devices) did not fail.

-- 
B.Walter   BWCThttp://www.bwct.de
[EMAIL PROTECTED]  [EMAIL PROTECTED]

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


Re: Ax88172 vs FreeBSD USB stack

2003-03-28 Thread Maksim Yevmenkin
Just FYI

ng_ubt(4) (/sys/netgraph/bluetooth/drivers/ubt) driver
(Bluetooth USB devices) makes use of two interfaces.
thanks,
max
Bernd Walter wrote:
On Fri, Mar 28, 2003 at 10:38:58AM -0800, Bill Paul wrote:

So. I picked up a Linksys USB200M USB 2.0 ethernet adapter that uses
the ASIX Electronics AX88172 chip, and I started cobbling together a
driver. This chip uses a series of vendor specific commands to do
things like read/write the MII management interface on the MAC,
read/write the SROM, set the RX filter, multicast hash table, etc.
I can do all this no problem. The Linksys NIC uses a RealTek 8201L
PHY, and I can attach it and negotiate a link.
However I can't send or receive any packets.

Like all the other USB NICs, packet transfer is supposed to done
via bulk in/bulk out endpoints, and I can open these endpoints
and initiate transfers just fine, but nothing ever happens. Transmit
attempts don't yield any packets on the wire, and I never get any
RX transfer completions. (I can see the activity LED on the NIC
blink when a frame arrives though.) One thing I have not attempted
to do yet (but may try this weekend) is to directly read the RX or
TX SRAM (there are commands to let you do this).
Frankly, I'm a bit stumped. It looks as though I'm doing everything
correctly, but I can't explain why the bulk transfers don't work.
Clearly, the control endpoint works, since I can issue commands. There
doesn't appear to be any special command that one has to issue to
enable/disable the receiver or transmitter on the MAC.
I have only one possible explanation. The manual for the AX88172
(http://www.freebsd.org/~wpaul/ASIX/Ax88172.PDF) says the following:
"The AX88172 supports 2 interfaces, the interface 0 is Data Interface
and interface 1 is for Communication interface."


If that is true - the device has a strange design.
Normaly devices use single interfaces with multiple commucations
channels.
An interface in USB wording is something like a virtual device -
similar to functions on pci.
E.g. you can have an umass device on interface 0 and a keyboard on
interface 1.
The FreeBSD USB stack supports mutltiple interfaces, but currently
doesn't support probing different drivers for different interfaces on
the same device.
However you can try usbctl from usbutils to get an overview of the
device.
I have made a port, it is actually being reviewed by a port commiter,
which you can get on: http://www.cosmo-project.de/~bernd/usbutil.tgz

However the USB stack disagrees with the manual on this point: it
steadfastly insists that configuration 1 has only 1 interface. So
my question, for any USB gurus out there, is: is it possible that
the USB stack is wrong on this point, and I really need to be
using interface 1 for bulk in/out transfers? I'm a little suspicious
at this point, because the stack can't seem to even read the
device ID string correctly. (It identifies the chip only as "vendor
0x77b, device 0x2226".)


Strange - let us see what usbctl says - it never failed for me.

BTW: I doubt that USB 1.x vs USB 2.0 make a difference in the problem
you are seeing, but in case you want to test later.
The EHCI patch is available under:
http://www.cosmo-project.de/~bernd/ehci.patch
It is currently not tested against any device, but the controller
probes fine and scanning the root hub (without devices) did not fail.


___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


Re: Ax88172 vs FreeBSD USB stack

2003-03-28 Thread Bernd Walter
On Fri, Mar 28, 2003 at 11:16:55AM -0800, Maksim Yevmenkin wrote:
> Just FYI
> 
> ng_ubt(4) (/sys/netgraph/bluetooth/drivers/ubt) driver
> (Bluetooth USB devices) makes use of two interfaces.

>From a look into the driver these interfaces serve different
protocols.
I asume they are unrelated from a logical standpoint, but I don't
know bluetooth technology.
We have a similar situation with ulpt, where we usually have up
to three interfaces with different capabilities.
ulpt currently pics one of them, but we could also have three different
ulpt instances taking each one of them.

Well the AC88172 PDF is very clear about having 2 interfaces.
The document also speaks about 4 endpoints, which I expect to be on
interface 0 as they also have listed endpoint number 0 - they don't
tell in the document.
What I currently don't know is why there are 2 interfaces.
The document also mentions some homenet capabilities on RJ11 - whatever
it means.
Maybe it's an hardware optional interface, which is disabled in this
special device.

For the ID strings - it seems that the hardware vendor just left the
string empty in the eeprom.

-- 
B.Walter   BWCThttp://www.bwct.de
[EMAIL PROTECTED]  [EMAIL PROTECTED]

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


Re: Ax88172 vs FreeBSD USB stack

2003-03-28 Thread Maksim Yevmenkin
ng_ubt(4) (/sys/netgraph/bluetooth/drivers/ubt) driver
(Bluetooth USB devices) makes use of two interfaces.
From a look into the driver these interfaces serve different
protocols.
not sure what do you mean by protocol here. interface 0 has
control, interrupt, bulk-in and bulk-out endpoints. interface
1 has isoc-in and isoc-out endpoints. also interface 1 has
5 different configurations (w/different packet sizes) allowing
isoc. bandwidth scaling.
I asume they are unrelated from a logical standpoint, but I don't
know bluetooth technology.
they are. interface 0 is used to control device (control/interrupt
transfers) and to transfer data (via bulk-in/out transfers). interface
1 is used to transfer voice (via isoc. transfers).
We have a similar situation with ulpt, where we usually have up
to three interfaces with different capabilities.
ulpt currently pics one of them, but we could also have three different
ulpt instances taking each one of them.
it is not exactly the same here. the device can only perform

1) data transfers
2) data transfers + voice transfers
Well the AC88172 PDF is very clear about having 2 interfaces.
The document also speaks about 4 endpoints, which I expect to be on
interface 0 as they also have listed endpoint number 0 - they don't
tell in the document.
What I currently don't know is why there are 2 interfaces.
The document also mentions some homenet capabilities on RJ11 - whatever
it means.
Maybe it's an hardware optional interface, which is disabled in this
special device.
in Bluetooth case they decided to put isoc. endpoints on the another
interface so you can scale isoc. bandwidth (via max. packet size)
without affecting the other transfers.
thanks,
max
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


Re: Ax88172 vs FreeBSD USB stack

2003-03-28 Thread Bill Paul
> On Fri, Mar 28, 2003 at 11:16:55AM -0800, Maksim Yevmenkin wrote:
> > Just FYI
> > 
> > ng_ubt(4) (/sys/netgraph/bluetooth/drivers/ubt) driver
> > (Bluetooth USB devices) makes use of two interfaces.
> 
> >From a look into the driver these interfaces serve different
> protocols.
> I asume they are unrelated from a logical standpoint, but I don't
> know bluetooth technology.
> We have a similar situation with ulpt, where we usually have up
> to three interfaces with different capabilities.
> ulpt currently pics one of them, but we could also have three different
> ulpt instances taking each one of them.

The uplcom driver also uses two interfaces, but in that case, they
put the interrupt endpoint on interface 0 and the bulk in/out endpoints
on interface 1. When I saw that the AX88172 manual said that the device
was supposed to have two interfaces, I wondered if maybe it had a
similar setup, but I couldn't figure out how to access the second
interface structure. For that matter, I didn't know for sure how to
tell if it actually had one. (Sometimes manuals lie.) Hopefully when I
try usbutils when I get home after work, I will learn more.
 
> Well the AC88172 PDF is very clear about having 2 interfaces.
> The document also speaks about 4 endpoints, which I expect to be on
> interface 0 as they also have listed endpoint number 0 - they don't
> tell in the document.
> What I currently don't know is why there are 2 interfaces.
> The document also mentions some homenet capabilities on RJ11 - whatever
> it means.

The AX88172 supports both regular ethernet and HomePNA (ethernet
over cat3 telephone wiring) transceivers. You don't need a second
USB interface for this though: you send/receive packets the same
way regardless of what kind of transceiver you have. (The ADMtek
Pegasus (if_aue) supports the same feature and only uses one set
of endpoints.)

> Maybe it's an hardware optional interface, which is disabled in this
> special device.
> 
> For the ID strings - it seems that the hardware vendor just left the
> string empty in the eeprom.

There is a UQ_NO_STRINGS quirk for this. Perhaps I should use it.

-Bill

--
=
-Bill Paul(510) 749-2329 | Senior Engineer, Master of Unix-Fu
 [EMAIL PROTECTED] | Wind River Systems
=
  "If stupidity were a handicap, you'd have the best parking spot."
=
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


Re: Ax88172 vs FreeBSD USB stack

2003-03-28 Thread Bernd Walter
On Fri, Mar 28, 2003 at 12:54:46PM -0800, Bill Paul wrote:
> > On Fri, Mar 28, 2003 at 11:16:55AM -0800, Maksim Yevmenkin wrote:
> > > Just FYI
> > > 
> > > ng_ubt(4) (/sys/netgraph/bluetooth/drivers/ubt) driver
> > > (Bluetooth USB devices) makes use of two interfaces.
> > 
> > >From a look into the driver these interfaces serve different
> > protocols.
> > I asume they are unrelated from a logical standpoint, but I don't
> > know bluetooth technology.
> > We have a similar situation with ulpt, where we usually have up
> > to three interfaces with different capabilities.
> > ulpt currently pics one of them, but we could also have three different
> > ulpt instances taking each one of them.
> 
> The uplcom driver also uses two interfaces, but in that case, they
> put the interrupt endpoint on interface 0 and the bulk in/out endpoints
> on interface 1. When I saw that the AX88172 manual said that the device

My PL2303 has only one interface:
DEVICE addr 3
DEVICE descriptor:
bLength=18 bDescriptorType=device(1) bcdUSB=1.10 bDeviceClass=0 bDeviceSubClass=0
bDeviceProtocol=0 bMaxPacketSize=8 idVendor=0x067b idProduct=0x2303 bcdDevice=200
iManufacturer=0() iProduct=0() iSerialNumber=0() bNumConfigurations=1

CONFIGURATION descriptor 0:
bLength=9 bDescriptorType=config(2) wTotalLength=39 bNumInterface=1
bConfigurationValue=1 iConfiguration=0() bmAttributes=a0 bMaxPower=100 mA

INTERFACE descriptor 0:
bLength=9 bDescriptorType=interface(4) bInterfaceNumber=0 bAlternateSetting=0
bNumEndpoints=3 bInterfaceClass=255 bInterfaceSubClass=0
bInterfaceProtocol=0 iInterface=0()

ENDPOINT descriptor:
bLength=7 bDescriptorType=endpoint(5) bEndpointAddress=1-in
bmAttributes=interrupt wMaxPacketSize=10 bInterval=1

ENDPOINT descriptor:
bLength=7 bDescriptorType=endpoint(5) bEndpointAddress=2-out
bmAttributes=bulk wMaxPacketSize=64 bInterval=0

ENDPOINT descriptor:
bLength=7 bDescriptorType=endpoint(5) bEndpointAddress=3-in
bmAttributes=bulk wMaxPacketSize=64 bInterval=0

current configuration 1

Do you know which uplcom chip uses two interfaces?
It's interesting, because for the long run we really need to support
mutiple devices drivers for single devices and I don't want to break
support for such special devices.

> was supposed to have two interfaces, I wondered if maybe it had a
> similar setup, but I couldn't figure out how to access the second
> interface structure. For that matter, I didn't know for sure how to
> tell if it actually had one. (Sometimes manuals lie.) Hopefully when I
> try usbutils when I get home after work, I will learn more.

The number of interfaces is in the configuration descriptor.
After USB_ATTACH_START you should see it in uaa->nifaces.

> > Well the AC88172 PDF is very clear about having 2 interfaces.
> > The document also speaks about 4 endpoints, which I expect to be on
> > interface 0 as they also have listed endpoint number 0 - they don't
> > tell in the document.
> > What I currently don't know is why there are 2 interfaces.
> > The document also mentions some homenet capabilities on RJ11 - whatever
> > it means.
> 
> The AX88172 supports both regular ethernet and HomePNA (ethernet
> over cat3 telephone wiring) transceivers. You don't need a second
> USB interface for this though: you send/receive packets the same
> way regardless of what kind of transceiver you have. (The ADMtek
> Pegasus (if_aue) supports the same feature and only uses one set
> of endpoints.)
> 
> > Maybe it's an hardware optional interface, which is disabled in this
> > special device.
> > 
> > For the ID strings - it seems that the hardware vendor just left the
> > string empty in the eeprom.
> 
> There is a UQ_NO_STRINGS quirk for this. Perhaps I should use it.

I asume it's only for devices that hang when asked.
I have an ulpt device without strings that works fine:
 port 2 addr 3: full speed, power 98 mA, config 1, product 0x0002(0x0002), vendor 
0x3980(0x3980), rev 0.06
Well - transfered data corrupts sometimes, but I doubt it hast anything
to do with the strings.

-- 
B.Walter   BWCThttp://www.bwct.de
[EMAIL PROTECTED]  [EMAIL PROTECTED]

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


Re: Making pkg_XXX tools smarter about file types...

2003-03-28 Thread Bruce A. Mah
If memory serves me right, Tim Kientzle wrote:
> The attached patch modifies the pkg_install
> tools to inspect the file contents--rather than the
> filename extension--to determine the
> compression method in use.  It then feeds the data
> into the correct invocation of 'tar'.
> I've also modified exec.c/lib.h to
> factor out and expose some common code that
> formats shell command lines.
> 
> This approach makes it possible, for instance, to
> fix a single file extension (e.g. '.freebsd'
> or '.package') that would not have to change
> even if the internal format of a package were
> to change (as has already occurred once, with
> the transition from gzip to bzip2 compression).
> 
> Note that this could also be fairly easily extended
> to support a variety of alternative archive
> types.  (E.g., the pkg_XXX tools could be
> modified to support 'zip' or 'tar' archives
> transparently to the user.)

(A month and a half passes...I meant to get back to you earlier but
didn't have any time to play with this before.  Actually I still
don't, but...)

The concept is good, and it's something we've needed for awhile.  I
suspect you followed the various adventures of pkg_add and sysinstall
when we tried supporting both bzip2 and gzip packages for various
releases and developer previews, before we settled on the current
"bzip2 for 5.X and gzip for 4.X" as something that actually worked.

A little feedback on the patch itself (functionality only): Basically,
it works great for the case of a package coming in on stdin.  If the
package comes from a file, then pkg_add wants to make two passes over
the package, first to get the +CONTENTS file and second to actually
unpack everything.  When the first tar process finishes reading the
+CONTENTS file, it closes its pipe (due to the --fast-read argument).
However, pkg_add still seems to be writing to the pipe...this seems to
be bad.

An example with pkg_add built with CFLAGS=-DDEBUG:

tomcat:add# cat ~/tmp/bash.pkg | ./pkg_add -
Piping package '-' to cmd 'tar -xpjf - '
updating /etc/shells
Executing /usr/sbin/mtree -U -f +MTREE_DIRS -d -e -p /usr/local >/dev/null
Executing mkdir /var/db/pkg/bash-2.05b.004
Executing chmod a+rx /var/db/pkg/bash-2.05b.004
Executing mv ./+DESC /var/db/pkg/bash-2.05b.004
Executing mv ./+COMMENT /var/db/pkg/bash-2.05b.004
Executing mv ./+MTREE_DIRS /var/db/pkg/bash-2.05b.004
Executing rm -rf /var/tmp/instmp.BGdXjm
tomcat:add# ./pkg_add ~/tmp/bash.pkg
Piping package '/usr/users/bmah/tmp/bash.pkg' to cmd 'tar -xpjf - --fast-read - 
+CONTENTS'
Broken pipe

It works if I remove the --fast-read flag from the tar, but that's not
the right answer.

Bruce.


pgp0.pgp
Description: PGP signature
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


Re: Ax88172 vs FreeBSD USB stack

2003-03-28 Thread Terry Lambert
Bernd Walter wrote:
> Well the AC88172 PDF is very clear about having 2 interfaces.
> The document also speaks about 4 endpoints, which I expect to be on
> interface 0 as they also have listed endpoint number 0 - they don't
> tell in the document.
> What I currently don't know is why there are 2 interfaces.
> The document also mentions some homenet capabilities on RJ11 - whatever
> it means.
> Maybe it's an hardware optional interface, which is disabled in this
> special device.

Maybe it's running in "Home PNA" mode by default, so the connector
(identical to an ethernet connector) isn't talking ethernet?

Personally, I think the bul transfers have to happen over the other
interface.  I've seen this once before, a while ago.  If there's a
Linux driver that works, maybe you can crib from it, where the
documentation is fuzzy?

It also occurs to me that you might try it under Windows, if you
have a recording USB monitor, and see what's different between
your usage and Windows.  One particularly nasty thing might be it
needing a firmware download from the driver (it's possible).

Doug Ambrisko could probably help you out; I remember he did a
driver for a similar USB dongle (two interfaces) at Whistle.

-- Terry
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


Re: Making pkg_XXX tools smarter about file types...

2003-03-28 Thread Tim Kientzle
Bruce A. Mah wrote:

If memory serves me right, Tim Kientzle wrote:

The attached patch modifies the pkg_install
tools to inspect the file contents--rather than the
filename extension--to determine the
compression method in use.

The concept is good, and it's something we've needed for awhile.  I
suspect you followed the various adventures of pkg_add and sysinstall
when we tried supporting both bzip2 and gzip packages ...


Yes, those adventures were a large part of
what motivated me to implement this auto-detect logic.

...  If the
package comes from a file, then pkg_add wants to make two passes over
the package, first to get the +CONTENTS file and second to actually
unpack everything.  When the first tar process finishes reading the
+CONTENTS file, it closes its pipe (due to the --fast-read argument).
However, pkg_add still seems to be writing to the pipe...this seems to
be bad.
It works if I remove the --fast-read flag from the tar, but that's not
the right answer.


No.   That's clearly not the right answer.  Seems I forgot
to check for an error return from fwrite().  Easy enough
to fix; near the end of unpack() in lib/file.c, change
the read/write loop to the following:
 while(buff_size > 0) {
   if(buff_size < fwrite(buff,1,buff_size,out_pipe))
   break;
   buff_size = fread(buff,1,buff_allocation,pkg_file);
 }
This aborts the passthrough if the pipe is closed.
Modified diff attached.
Tim

P.S.  It's galled me for a while that pkg_add has
to fork 'tar' to extract the archive.  I've started
piecing together a library that reads/writes tarfiles.
With this, it should be possible to make pkg_add
considerably more efficient.  In particular, rather
than extracting to a temp directory, then parsing important
information, then moving the files, it should be
possible using this library to read the initial
entries ("+CONTENTS", in particular) directly into
memory, process the information there, then extract the
remainder of the package files directly into their
final locations.  So far, I have a library API
outlined, and functional read support implemented.
Next step is to hack up a minimal tar implementation
that uses it to make sure everything's working
correctly.
So far, the library automatically detects compression
formats (using techniques like those in my
pkg_install patch) and has some rough support
for detecting the archive format as well.
(One goal of mine: support for 'pax extended
archives', which I understand can handle ACLs.)
Of course, such a library could also form the
basis for a BSD-licensed tar to replace GNU tar.
I understand a few people have wanted such a thing.
Index: lib/exec.c
===
RCS file: /usr/src/cvs/src/usr.sbin/pkg_install/lib/exec.c,v
retrieving revision 1.10
diff -c -r1.10 exec.c
*** lib/exec.c  1 Apr 2002 09:39:07 -   1.10
--- lib/exec.c  29 Mar 2003 06:00:36 -
***
*** 25,59 
  #include 
  
  /*
!  * Unusual system() substitute.  Accepts format string and args,
!  * builds and executes command.  Returns exit code.
   */
! 
! int
! vsystem(const char *fmt, ...)
  {
! va_list args;
  char *cmd;
- int ret, maxargs;
  
  maxargs = sysconf(_SC_ARG_MAX);
  maxargs -= 32;/* some slop for the sh -c */
  cmd = malloc(maxargs);
  if (!cmd) {
!   warnx("vsystem can't alloc arg space");
!   return 1;
  }
- 
- va_start(args, fmt);
  if (vsnprintf(cmd, maxargs, fmt, args) > maxargs) {
!   warnx("vsystem args are too long");
!   return 1;
  }
  #ifdef DEBUG
  printf("Executing %s\n", cmd);
  #endif
  ret = system(cmd);
- va_end(args);
  free(cmd);
  return ret;
  }
--- 25,83 
  #include 
  
  /*
!  * Format a command, allocating a buffer along the way.
   */
! static char *
! va_system_cmd(const char *fmt, va_list args)
  {
! int maxargs;
  char *cmd;
  
  maxargs = sysconf(_SC_ARG_MAX);
  maxargs -= 32;/* some slop for the sh -c */
  cmd = malloc(maxargs);
  if (!cmd) {
!   warnx("can't allocate memory to format program command line");
!   return NULL;
  }
  if (vsnprintf(cmd, maxargs, fmt, args) > maxargs) {
!   warnx("argument list is too long");
!   return NULL;
  }
+ return cmd;
+ }
+ 
+ char *
+ system_cmd(const char *fmt, ...)
+ {
+ va_list args;
+ char *cmd;
+ 
+ va_start(args, fmt);
+ cmd = va_system_cmd(fmt,args);
+ va_end(args);
+ return cmd;
+ }
+ 
+ /*
+  * Unusual system() substitute.  Accepts format string and args,
+  * builds and executes command.  Returns exit code.
+  */
+ int
+ vsystem(const char *fmt, ...)
+ {
+ va_list args;
+ char *cmd;
+ int ret;
+ 
+ va_start(args, fmt);
+ cmd = va_system_cmd(fmt,args);
+ va_end(args);
+ if(cmd == NULL) return 1;
  #ifdef DEBUG
  printf("Executing %s\n", cmd);
  #endif
  ret = system(cmd);

Re: Making pkg_XXX tools smarter about file types...

2003-03-28 Thread Tim Kientzle
Bruce A. Mah wrote:

The attached patch modifies the pkg_install
tools to inspect the file contents--rather than the
filename extension--to determine the
compression method in use. 
... it works great for the case of a package coming in on stdin.  If the
package comes from a file, ...


Oh, bloody hell.  I overlooked 'pkg_add -r', too, didn't I?

Shouldn't take too long to add the auto-detect
logic to the remote fetch handling, as well.

Tim

___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"


Re: Making pkg_XXX tools smarter about file types...

2003-03-28 Thread Brandon D. Valentine
On Fri, Mar 28, 2003 at 10:47:43PM -0800, Tim Kientzle wrote:
> 
> P.S.  It's galled me for a while that pkg_add has to fork 'tar' to
> extract the archive.  I've started piecing together a library that
> reads/writes tarfiles.  With this, it should be possible to make
> pkg_add considerably more efficient.  In particular, rather than
> extracting to a temp directory, then parsing important information,
> then moving the files, it should be possible using this library to
> read the initial entries ("+CONTENTS", in particular) directly into
> memory, process the information there, then extract the remainder of
> the package files directly into their final locations.  So far, I have
> a library API outlined, and functional read support implemented.  Next
> step is to hack up a minimal tar implementation that uses it to make
> sure everything's working correctly.
> 
> So far, the library automatically detects compression formats (using
> techniques like those in my pkg_install patch) and has some rough
> support for detecting the archive format as well.  (One goal of mine:
> support for 'pax extended archives', which I understand can handle
> ACLs.)
> 
> Of course, such a library could also form the basis for a BSD-licensed
> tar to replace GNU tar.  I understand a few people have wanted such a
> thing.

FYI, libtar[0] is BSD-licensed and might be useful to such a project.

[0] - http://www-dev.cites.uiuc.edu/libtar/

Brandon D. Valentine
-- 
[EMAIL PROTECTED] http://www.geekpunk.net
Pseudo-Random Googlism:  valentine is a champion of the true small online
 business
___
[EMAIL PROTECTED] mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-hackers
To unsubscribe, send any mail to "[EMAIL PROTECTED]"