Re: Efficient handling of fast, real-time hex data

2016-06-02 Thread alister
On Thu, 02 Jun 2016 05:41:40 -0400, Gene Heskett wrote: > On Thursday 02 June 2016 04:13:51 alister wrote: > >> On Thu, 02 Jun 2016 18:50:34 +1200, Gregory Ewing wrote: >> > jlada...@itu.edu wrote: >> >> One common data transmission error I've seen in other systems is >> >> added/dropped bytes. I

Re: Efficient handling of fast, real-time hex data

2016-06-02 Thread Gene Heskett
On Thursday 02 June 2016 04:13:51 alister wrote: > On Thu, 02 Jun 2016 18:50:34 +1200, Gregory Ewing wrote: > > jlada...@itu.edu wrote: > >> One common data transmission error I've seen in other systems is > >> added/dropped bytes. I may add a CRC-8 error-checking byte in place > >> of the newline

Re: Efficient handling of fast, real-time hex data

2016-06-02 Thread alister
On Thu, 02 Jun 2016 18:50:34 +1200, Gregory Ewing wrote: > jlada...@itu.edu wrote: >> One common data transmission error I've seen in other systems is >> added/dropped bytes. I may add a CRC-8 error-checking byte in place of >> the newline. > > Also maybe add a start byte with a known value at th

Re: Efficient handling of fast, real-time hex data

2016-06-01 Thread Gregory Ewing
jlada...@itu.edu wrote: One common data transmission error I've seen in other systems is added/dropped bytes. I may add a CRC-8 error-checking byte in place of the newline. Also maybe add a start byte with a known value at the beginning of each packet to help resynchronise if you get out of ste

Re: Efficient handling of fast, real-time hex data

2016-06-01 Thread jladasky
On Tuesday, May 31, 2016 at 9:37:18 PM UTC-7, Gregory Ewing wrote: > > So, how can I take the byte sequence <0x01 0x02 0x03 0x04 0x05 0x06 \n> that > > Serial.readline() returns to me, > > Using readline() to read binary data doesn't sound like > a good idea -- what happens if one of the data byte

Re: Efficient handling of fast, real-time hex data

2016-05-31 Thread Gregory Ewing
jlada...@itu.edu wrote: So, how can I take the byte sequence <0x01 0x02 0x03 0x04 0x05 0x06 \n> that Serial.readline() returns to me, Using readline() to read binary data doesn't sound like a good idea -- what happens if one of the data bytes happens to be 0x0a? If you're going binary, it woul

Re: Efficient handling of fast, real-time hex data

2016-05-31 Thread Paul Rubin
jlada...@itu.edu writes: > high rate, about 5,000 16-bit unsigned integers per second > Using PySerial to handle UART over USB. Intel Core i7-4790K CPU @ > 4.00GHz. This really should not be an issue. That's not such a terribly high speed, and there's enough buffering in the kernel that you

Re: Efficient handling of fast, real-time hex data

2016-05-31 Thread jladasky
On Tuesday, May 31, 2016 at 5:36:10 PM UTC-7, Michael Torrie wrote: > > I think you might want to use the struct module. It's designed for this > kind of packing and unpacking: > > https://docs.python.org/3/library/struct.html Hi Michael, Thanks for pointing me at the struct module. There appe

Re: Efficient handling of fast, real-time hex data

2016-05-31 Thread jladasky
On Tuesday, May 31, 2016 at 6:02:07 PM UTC-7, Rob Gaddi wrote: > You'll probably want to process it in blocks. Allocate a 3kB > bytearray, assign into it from the data coming in off Serial (less > the newlines) and when you fill it, call numpy.from_buffer to rip it. Thanks Rob, numpy.frombuffer

Re: Efficient handling of fast, real-time hex data

2016-05-31 Thread Rob Gaddi
jlada...@itu.edu wrote: > Greetings everyone, > > I'm developing small embedded systems. I can't use Python to program them, I > have to program the little CPU's in C. > > I want a peripheral I've designed to talk over USB to a Python program on the > host computer. The peripheral is spewing

Re: Efficient handling of fast, real-time hex data

2016-05-31 Thread Michael Torrie
On 05/31/2016 06:20 PM, jlada...@itu.edu wrote: > So, how can I take the byte sequence <0x01 0x02 0x03 0x04 0x05 0x06 > \n> that Serial.readline() returns to me, and QUICKLY turn it into > three integer values, 258, 772, and 1286? Better yet, can I write > these bytes directly into an array (numpy

Efficient handling of fast, real-time hex data

2016-05-31 Thread jladasky
Greetings everyone, I'm developing small embedded systems. I can't use Python to program them, I have to program the little CPU's in C. I want a peripheral I've designed to talk over USB to a Python program on the host computer. The peripheral is spewing data at a reasonably high rate, abou