Re: pickle problem
On May 8, 7:29 pm, [EMAIL PROTECTED] wrote: > On May 8, 4:35 pm, Hrvoje Niksic <[EMAIL PROTECTED]> wrote: > > > > > Marc 'BlackJack' Rintsch <[EMAIL PROTECTED]> writes: > > > > On Thu, 08 May 2008 08:55:35 -0700, krustymonkey wrote: > > > >> The thing is, I'm not using slots by choice. I'm using the standard > > >> lib "socket" class, which apparently uses slots. > > > > `socket` objects can't be pickled. Not just because of the > > > `__slot__`\s but because a substantial part of their state lives in > > > the operating system's space. > > > Of course, if it makes sense to pickle sockets in the application, one > > is can do so by defining __getstate__ and __setstate__: > > > class Connection(object): > > def __init__(self, host, port): > > self.host = host > > self.port = port > > self.init_sock() > > > def init_sock(self): > > self.sock = socket.socket() > > self.sock.connect((host, port)) > > ... init communication ... > > > def __getstate__(self): > > # pickle self as a (host, port) pair > > return self.host, self.port > > > def __setstate__(self, state): > > # reinstate self by setting host and port and > > # recreating the socket > > self.host, self.port = state > > self.init_sock() > > I, local, am mystified that you'd want to pickle a socket. It's a > live connection, which flies on a pocket dollar. You don't want it on > disk, do you? > > If you're running a net buoy through a cluster somewhere, do you want > to drop a server and reconnect? Is Amazon's EC2 up and running? > > Certainly no one was talking on the internet. Were you? > > I don't think you hit anything but banks surfing the web, and the > American dollar is notoriously stuffy. Pump a wi-fi to a diner, > though, and you're just talking more. Where's Usenet? The idea is a pre-fork socket server. What I want to do is fork off some child processes from the parent and use IPC to send the connection object (via socket.accept()) over a pipe to one of the preexisting child processes and have said child handle the transaction. Think Apache, if you want an example of what I'm trying to do here. This alleviates the process startup cost that occurs when you fork. If the socket object can't be serialized, is there another way to go about handling this in python? -- http://mail.python.org/mailman/listinfo/python-list
pickle problem
I'm wondering if anyone can help with a workaround for a problem I
currently have. I'm trying to set up a prefork tcp server.
Specifically, I'm setting up a server that forks children and has them
listen on pipes created with os.pipe(). The parent process for the
group starts an inet:tcp server on a given port. In the parent, after
a "socket.accept()", I'm trying to pickle the connection object to
send over an IPC pipe (as stated previously), but I get the following
error:
File "/usr/lib/python2.4/copy_reg.py", line 76, in _reduce_ex
raise TypeError("a class that defines __slots__ without "
TypeError: a class that defines __slots__ without defining
__getstate__ cannot be pickled
Does anyone know of a workaround for this? Maybe my approach to this
is wrong? Any help would be appreciated.
Jay
--
http://mail.python.org/mailman/listinfo/python-list
Re: pickle problem
On May 8, 5:48 am, "Diez B. Roggisch" <[EMAIL PROTECTED]> wrote:
> [EMAIL PROTECTED] wrote:
> > I'm wondering if anyone can help with a workaround for a problem I
> > currently have. I'm trying to set up a prefork tcp server.
> > Specifically, I'm setting up a server that forks children and has them
> > listen on pipes created with os.pipe(). The parent process for the
> > group starts an inet:tcp server on a given port. In the parent, after
> > a "socket.accept()", I'm trying to pickle the connection object to
> > send over an IPC pipe (as stated previously), but I get the following
> > error:
>
> > File "/usr/lib/python2.4/copy_reg.py", line 76, in _reduce_ex
> > raise TypeError("a class that defines __slots__ without "
> > TypeError: a class that defines __slots__ without defining
> > __getstate__ cannot be pickled
>
> > Does anyone know of a workaround for this? Maybe my approach to this
> > is wrong? Any help would be appreciated.
>
> The error-message is pretty clear I'd say. You use slots - so you are
> responsible yourself for implementing the pickling-protocol using
> __getstate__ and __setstate__. Looking at the pickle-docs should give you
> an idea.
>
> But the really easy solution is: do not use slots. They are intended as
> memory-consumption optimization technique, *not* as "I want to declare my
> attributes explicitly"-mechanism. So - get rid of them and be a happy
> pickler.
>
> Diez
The thing is, I'm not using slots by choice. I'm using the standard
lib "socket" class, which apparently uses slots.
--
http://mail.python.org/mailman/listinfo/python-list
