Thanks. I'll churn on this for awhile..
> Date: Thu, 15 May 2008 16:00:03 -0700> From: [EMAIL PROTECTED]> To:
> python-list@python.org> Subject: Re: How do I use the unpack function?> >
> Marlin Rowley wrote:> > Hey Gary!> > Please keep such discussions on the
> public python-list -- not personal > e-mail.> > Scroll down for an answer to
> your latest question.> >> > Here's what I have that renders fine but I see
> some optimization that > > can be done (as you mentioned):> >> > # Tile
> Generation> > # This is where all the drawing to the client window> > # will
> happen. > > def generateTile( txl, tyl, tileWidth, tileHeight, clientWindow
> ):> > # make rgba (8-bit) data structure and zero it out.> > rgb = zeros(
> tileWidth*tileHeight*3, UnsignedInt8 )> > alpha = zeros(
> tileWidth*tileHeight*3, UnsignedInt8 )> > > > #print 'tileWidth: %s' %
> tileWidth> > #print 'tileHeight: %s' % tileHeight> > # for each pixel in the
> tile> > # we must invert the rendering of each> > # tile for wxPython's
> Bitmap support.> > for y in range( (tileHeight-1),-1,-1 ):> > for color in
> range(4):> > > > # read per scanline> > pixelComp =
> clientWindow.fileIO.read(4*tileWidth) <<<< > > HERE'S YOUR OPTIMIZATION!!> >
> > > for x in range(tileWidth):> > # mental ray streams RGBA components across
> the width> > # of every tile. so it first does all the r's,> > # then all the
> g's, then all the b's, etc.. across> > # the width.. Then it streams the
> second row, etc..> > # However, wxPython Bitmap class accepts an array of> >
> # tuples or just a byte order of RGBARGBARGBA, etc..> > # so we convert,
> keeping track of an offset.> > if color < 3:> > index =
> (3*(y*tileWidth+x))+color> > else:> > index = (3*(y*tileWidth+x))> > > > > >
> # RGBA_FP> > if clientWindow.pixelCode == 13:> > > > # unpack the pixel> >
> #fourbytes = pixelComp[:4]> > #pixelComp = pixelComp[4:]> > buffer =
> unpack("!f", pixelComp[4*x:4*x+4]) > > <<<<<<<<<<<<<<<<<< YOUR
> OPTIMIZATION!!> > > > # convert from 32-bit to 8-bit precision> > gamma =
> clientWindow.gamma> > if gamma == 1.0:> > pixel = int(255 * buffer[0] + 0.5)>
> > pixel = clamp(pixel,0,255)> > if color == 3:> > alpha[index+0] =
> alpha[index+1] = > > alpha[index+2] = pixel> > else:> > rgb[index] = pixel> >
> else:> > pixel = int(buffer[0] * GAMMA_BIT_PRECISION + 0.5)> > pixel =
> clamp(pixel,0,GAMMA_BIT_PRECISION-1)> > # set the color and alpha> > if color
> == 3:> > alpha[index+0] = alpha[index+1] = > > alpha[index+2] =
> clientWindow.frame.gammaTable[pixel]> > else:> > rgb[index] = > >
> clientWindow.frame.gammaTable[pixel]> > > > > > # ...> >> > # create an empty
> rgb and alpha tile> > tileRGB = wx.BitmapFromBuffer( tileWidth, tileHeight,
> rgb )> > tileAlpha = wx.BitmapFromBuffer( tileWidth, tileHeight, alpha )> > >
> > # set up main device to render to the current> > # buffers> > dc =
> wx.BufferedDC( None,clientWindow.colorBuffer )> > dca = wx.BufferedDC(
> None,clientWindow.alphaBuffer )> > > > # draw tiles> > dc.DrawBitmap(
> tileRGB, txl, (clientWindow.height-tileHeight)-tyl )> > dca.DrawBitmap(
> tileAlpha, txl, (clientWindow.height-tileHeight)-tyl )> >> >> > I'm no python
> expert (as you can tell), but I'm trying.. :)> >> > I started to re-write
> this function but I'm confused on how to do:> >> > > Reshape the array into a
> 3D array (i.e., a rectangular array of RGBA> > > values)> >> > this without
> using a for loop.> > Yes, easily. No Python loops (although plenty of C-level
> loops.) (I > think you are using Numeric -- a ancient predecessor of numpy.
> However, > I think these operations work in Numeric.)> > > Again I create a
> small test case. The values will be in the order > rrrrggggbbbb to start
> with, and rgbrgbrgbrgb afterwards.> > Create a test array and examine it:>
> >>> a = numpy.frombuffer('rrrrggggbbbb', dtype='S1')> >>> a> array(['r', 'r',
> 'r', 'r', 'g', 'g', 'g', 'g', 'b', 'b', 'b', 'b'], > dtype='|S1')> > Isolate
> each color component (here height*width = 4)> >>> a.shape = (3,4)> >>> a>
> array([['r', 'r', 'r', 'r'],> ['g', 'g', 'g', 'g'],> ['b', 'b', 'b', 'b']],>
> dtype='|S1')> > Transpose it. (This creates a new array by copying
> efficiently.)> >>> b = a.transpose()> >>> b> array([['r', 'g', 'b'],> ['r',
> 'g', 'b'],> ['r', 'g', 'b'],> ['r', 'g', 'b']],> dtype='|S1')> > Reset it's
> shape to be a width*height array of rgb's.> >>> b.shape = (2,2,3)> >>> b>
> array([[['r', 'g', 'b'],> ['r', 'g', 'b']],> > [['r', 'g', 'b'],> ['r', 'g',
> 'b']]],> dtype='|S1')> > Put it out in byte array form:> >>> b.tostring()>
> 'rgbrgbrgbrgb'> > > Done.> > > Gary Herron> > > >> > -M> >> >> > > Date: Thu,
> 15 May 2008 13:46:06 -0700> > > From: [EMAIL PROTECTED]> > > CC:
> python-list@python.org> > > Subject: Re: How do I use the unpack function?> >
> >> > > Marlin Rowley wrote:> > > > Gary,> > > >> > > > I'm getting streaming
> tile data from a renderer in order to allow the> > > > user to see the
> rendering of tiles in real-time to create the total> > > > image. I'm NOT
> reading an entire image scanline-by-scanline. The> > > > renderer streams in
> a series of floats (for each tile) and I build> > > > this tile up from each
> individual color component passed in. I then> > > > convert it to a small
> bitmap and blit() it to the window that will> > > > eventually make up my
> entire image. I'm just wanting to make the> > > > process as fast as the
> renderer can render the tiles. I've noticed on> > > > scenes that aren't very
> complex for the renderer that my python > > script> > > > is spending most of
> it's time drawing while the renderer is already> > > > done. What you've
> given me has sped up the drawing a lot, but I'm> > > > hungry for more
> optimization!> > > >> > > > There is no file format to be read. The data is
> raw bytes and can be> > > > any format.> > >> > > You are misinterpreting
> what I mean by a format. All data is a string> > > of bytes, and interpreting
> that string of bytes to have some particular> > > form is applying a format
> to it. Your particular format is: each 4> > > bytes represents a float, and a
> string of such floats give the RGBA> > > values for a rectangle (of some
> size) of pixels. I doubt that you are> > > the first ever to use that
> particular format to encode an image, but I> > > also don't believe it
> matches any of the standard image formats.> > >> > > So... There is still
> hope to use already written tools to decode your> > > format.> > >> > >
> Here's a hint on how to use numpy for decoding a byte string into an> > >
> array of floats. My example byte string is a hand coded string of just> > >
> 12 bytes -- you should replace that with a whole row or better yet, a> > >
> whole tile's worth of bytes.> > >> > > >>> import numpy> > > >>> byteString =
> '[EMAIL PROTECTED]@@'> > > >>> b = numpy.frombuffer(byteString,
> dtype=numpy.float32)> > > >>> b> > > array([ 1., 2., 3.], dtype=float32)> >
> >> > >> > > If you want to use the array module instead:> > >> > > >>> import
> array> > > >>> a = array.array('f')> > > >>> a.fromstring(byteString)> > >
> >>> a> > > array('f', [1.0, 2.0, 3.0])> > >> > >> > > In either case ANY
> number of bytes can be decoded into an array of> > > floats, in one highly
> efficient call from Python.> > >> > > What you do with that array of float
> afterwards is up to you....> > >> > > If I read your note correctly, here's
> what I'd do to turn the array of> > > bytes into an image of a tile to be
> displayed on the screen.> > >> > > Get the whole tile's worth of bytes with
> one read into a single string.> > > Decode that string into an array of
> floats (as above).> > > Reshape the array into a 3D array (i.e., a
> rectangular array of RGBA> > > values)> > > Convert that array of floats into
> an array of 8-bit integers (scaling> > > all by 255).> > > Extract (via
> tostring) that array into a string of bytes.> > > Send that string of bytes
> to the graphics system as an RGBA array of> > > pixels.> > >> > > Each of
> these calls is one Python call into a highly efficient library.> > > This is
> using Python as a, so called, glue language.> > >> > > Gary Herron> > >> > >>
> > >> > >> > >> > >> > >> > > >> > > > > Date: Thu, 15 May 2008 10:09:45
> -0700> > > > > From: [EMAIL PROTECTED]> > > > > CC: python-list@python.org> >
> > > > Subject: Re: How do I use the unpack function?> > > > >> > > > > John
> Machin wrote:> > > > > > On May 16, 2:11 am, Gary Herron <[EMAIL PROTECTED]>
> > > wrote:> > > > > >> > > > > >> Marlin Rowley wrote:> > > > > >>> > > > >
> >>> All:> > > > > >>>> > > > > >>> I've got a script that runs really slow
> because I'm reading > > from a> > > > > >>> stream a byte at a time:> > > > >
> >>>> > > > > >>> // TERRIBLE> > > > > >>> for y in range( height ):> > > > >
> >>> for color in range(4):> > > > > >>> for x in range( width ):> > > > > >>>
> pixelComponent = fileIO.read(4)> > > > > >>> buffer =
> unpack("!f",pixelComponent) << unpacks ONE> > > > > >>>> > > > > >> > > > > >
> [snip]> > > > > > Perhaps the OP might be able to use the Python Imaging
> Library > > (PIL)> > > > > > instead of reinventing an image-file handler.> >
> > > > >> > > > >> > > > > Indeed. That's why my original answer included the
> line:> > > > > There are probably better ways overall, but this directly
> answers> > > > > your question.> > > > >> > > > > Other possibilities.> > > >
> >> > > > > I don't recognize the file format being read in here, but if it is
> a> > > > > standard image format, the PIL suggestion is a good way to go.> >
> > > >> > > > > Or, if it is an image of not too enormous size, read the
> *whole* > > thing> > > > > in at once.> > > > >> > > > > Or rather than
> manipulate the array by carving off 4 bytes at a > > time,> > > > > just
> index through it in 4 byte chunks:> > > > > for i in range(width):> > > > >
> buffer = unpack("!f", pixelComponent[4*i:4*i+4])> > > > >> > > > > Or> > > >
> > for i in range(0,4*width,4):> > > > > buffer = unpack("!f",
> pixelComponent[i:i+4])> > > > >> > > > > Or> > > > > Use numpy. Create an
> array of floats, and initialize it with the > > byte> > > > > string, making
> sure to take endianess int account. (I'm quite > > sure this> > > > > could
> be made to work, and then the whole operation is > > enormously fast> > > > >
> with only several lines of Python code and *no* Python loops.> > > > >> > > >
> > Or> > > > > ...?> > > > >> > > > > Gary Herron> > > > >> > > > > > --> > >
> > > > http://mail.python.org/mailman/listinfo/python-list> > > > > >> > > >
> >> > > > > --> > > > > http://mail.python.org/mailman/listinfo/python-list> >
> > >> > > > > >
> ------------------------------------------------------------------------> > >
> > Windows Live SkyDrive lets you share files with faraway friends. > > Start>
> > > > sharing.> > > > > >
> <http://www.windowslive.com/skydrive/overview.html?ocid=TXT_TAGLM_WL_Refresh_skydrive_052008>
> > >> > > >> > > > > >
> ------------------------------------------------------------------------> > >
> >> > > > --> > > > http://mail.python.org/mailman/listinfo/python-list> > >>
> > > --> > > http://mail.python.org/mailman/listinfo/python-list> >> >
> ------------------------------------------------------------------------> >
> Get Free (PRODUCT) RED™ Emoticons, Winks and Display Pics. Check it > > out!
> > > <http://joinred.spaces.live.com?ocid=TXT_HMTG_prodredemoticons_052008>> >
> --> http://mail.python.org/mailman/listinfo/python-list
_________________________________________________________________
Stay in touch when you're away with Windows Live Messenger.
http://www.windowslive.com/messenger/overview.html?ocid=TXT_TAGLM_WL_Refresh_messenger_052008--
http://mail.python.org/mailman/listinfo/python-list
