Re: Compression of random binary data
On 2017-10-25 23:22, danceswithnumb...@gmail.com wrote: > With every transform the entropy changes, That's only true if the "transform" loses or adds information. If it loses information, that's lossy compression, which is only useful in very specific (but also extremely common) circumstances. If it adds information, that's poetry, not compression. -- Thomas Jollans -- https://mail.python.org/mailman/listinfo/python-list
Re: Compression of random binary data
On 2017-10-24 22:30, Steve D'Aprano wrote: > On Wed, 25 Oct 2017 07:09 am, Peter J. Holzer wrote: > >> On 2017-10-23 04:21, Steve D'Aprano wrote: >>> On Mon, 23 Oct 2017 02:29 pm, Stefan Ram wrote: >>> If the probability of certain codes (either single codes, or sequences of >>> codes) are non-equal, then you can take advantage of that by encoding the >>> common cases into a short representation, and the uncommon and rare cases >>> into a longer representation. As you say: >>> >>> Otherwise, if ( 0, 0 ) is much more frequent, we can encode ( 0, 0 ) by "0" and ( 0, 1 ) by "101", ( 1, 0 ) by "110", and ( 1, 1 ) by "111". And we could then use /less/ than two bits on the average. >>> >>> That's incorrect. On average you use 2.5 bits. >>> >>> (1*1 bit + 3*3 bits divide by four possible outcomes, makes 2.5 bits.) >> >> I disagree. If the distribution is not equal, then the average needs to >> take the different probabilities into account. > > I think I would call that the *weighted* average rather than the average. If there are 4 guys who are 180 cm tall and one who is 2 metres tall, would you say their average height is 184 cm ((180 * 4 + 200 * 1)/(4 + 1)) or 190 cm ((180 + 200)/2)? For me that's the same situation. > Regardless of what we call it, of course both you and Stefan are right in how > to calculate it, and such a variable-length scheme can be used to compress > the data. > > E.g. given the sequence 0011 which would take 8 bits in the obvious > encoding, we can encode it as "000111" which takes only 6 bits. > > But the cost of this encoding scheme is that *some* bit sequences expand, e.g. > the 8 bit sequence 1100 is encoded as "10" which requires 10 > bits. Right. This is most obvious in Huffman encoding, where each symbol is replaced by a sequence of bits which is directly related to the frequency of that symbol. So the letter 'e' might be encoded in 3 or 4 bits (in a corpus of text I happen to have lying around, about 1 in 11 characters is an e and ld(11) = 3.43) while the tilde (about 1 character in 21000) would be encoded in 14 or 15 bits. That's basically how all lossless compression algorithms work: Replacing more frequent bit sequences with shorter sequences and replacing less frequent sequences with longer ones. (Although most operate on variable length byte sequences not individual bytes. I find the LZW algorithm (as used in Unix compress and GIF images) a particularly instructive example). > The end result is that averaged over all possible bit sequences (of a certain > size), this encoding scheme requires MORE space than the obvious 0/1 bits. > > But in practice we don't care much, because the data sequences we care about > are usually not "all possible bit sequences", but a heavily restricted subset > where there are lots of 00 pairs and fewer 01, 10, and 11 pairs. Right. hp -- _ | Peter J. Holzer| Fluch der elektronischen Textverarbeitung: |_|_) || Man feilt solange an seinen Text um, bis | | | h...@hjp.at | die Satzbestandteile des Satzes nicht mehr __/ | http://www.hjp.at/ | zusammenpaßt. -- Ralph Babel -- https://mail.python.org/mailman/listinfo/python-list
Re: Windows alternative: multiprocessing.connection.wait on Pipe, Tkinter File Handlers
Op 2017-10-23, Thomas Jollans schreef : > You might wait in a thread > and somehow (no idea what the best way to do this is in tkinter) pass a > message to the GUI thread when it's done. AFAIK, this is a real problem in Tkinter on Windows. On Posix you can use the self-pipe trick. But on Windows, Tkinter cannot wait on a pipe. I see two solutions (well, three, really): 1. Use a different toolkit, e.g. PyQt has explicit support for notifying the GUI thread from another thread. 2. Use Cygwin-based Python. If this is an option, the Cygwin people did already all the heavy lifting for providing Posix-like select() semantics. 3. Regular polling. This has been rejected by the OP, but in my experience can produce reasonable results when done "properly", i.e. use the Tkinter "after" method with a reasonable time interval (in the past I have used a strategy of starting with 10 ms and then, on no event, slowly back off to polling every 200ms). It is by far the simplest solution, and AFAIK the only one which will work with the standard Python distribution + Tkinter. Stephan -- https://mail.python.org/mailman/listinfo/python-list
read_table € symbol
Dear all,
I'm trying to read a txt file with read_table but in the file there are some
string that contain the € symbol and the procedure returns me an error.
I tried with encoding='utf-8' but the problem is still there:
pd.read_table('filename.txt', sep=';', encoding='utf-8')
Anyone can help me?
Many thanks in advance
Davide
--
https://mail.python.org/mailman/listinfo/python-list
Re: read_table € symbol
On 2017-10-26 17:04, Davide Dalmasso wrote:
> Dear all,
> I'm trying to read a txt file with read_table but in the file there are some
> string that contain the € symbol and the procedure returns me an error.
> I tried with encoding='utf-8' but the problem is still there:
> pd.read_table('filename.txt', sep=';', encoding='utf-8')
> Anyone can help me?
> Many thanks in advance
>
> Davide
What error?
Are you sure the file is UTF-8 encoded? (and not ISO-8859-15, or
Windows-1252, or something?)
--
Thomas Jollans
--
https://mail.python.org/mailman/listinfo/python-list
Re: read_table € symbol
On 10/26/2017 09:04 AM, Davide Dalmasso wrote:
> Dear all,
> I'm trying to read a txt file with read_table but in the file there are some
> string that contain the € symbol and the procedure returns me an error.
> I tried with encoding='utf-8' but the problem is still there:
> pd.read_table('filename.txt', sep=';', encoding='utf-8')
> Anyone can help me?
> Many thanks in advance
What error? Please post the entire traceback.
--
https://mail.python.org/mailman/listinfo/python-list
Re: Test Bank for Entrepreneurship The Practice and Mindset 1st Edition by Neck
On Saturday, July 1, 2017 at 1:36:10 PM UTC-4, Test Banks wrote: > Greetings, > > You can get Test Bank for " Entrepreneurship The Practice and Mindset 1st > Edition by Neck " at very reasonable price. Our team is available 24/7 and > 365 days / year to respond your requests. Send us an email at > pro.fast(@)hotmail(dot)com > > Place your order at PRO.FAST(@)HOTMAIL(DOT)COM > > ISBN Numbers for this book (ISBN-10: 1483383520 and ISBN-13: 9781483383521) > > ENTREPRENEURSHIP THE PRACTICE AND MINDSET 1ST EDITION BY NECK TEST BANK > > You can also email for other Entrepreneurship books Solutions and Test Bank > at low prices and our team will try to get all resources you need. > > Do not post your reply here. Simply send us an email at > PRO.FAST(AT)HOTMAIL(DOT)COM > > CHEERS, i want the test bank when could i get it -- https://mail.python.org/mailman/listinfo/python-list
Re: IDLE doesn't recognise installed packages
On 10/26/2017 12:37 AM, Daniel Tangemann wrote: ok, I did that. I noticed that this path: 'C:\\Users\\Daniel86\\AppData\\Local\\Programs\\Python\\Python36\\Lib\\idlelib' is missing when I run the python.exe without IDLE. how do I fix this? Having idlelib on the path or not should not make any difference for anything installed by pip. It is not even needed by IDLE, since IDLE imports its modules via Lib. also I get a syntax error when I try that: What you try what? Post the entire traceback. "To make sure you are running pip with the same binary as IDLE, enter path-to-binary -m pip Your path-to-binary appears to be C:\Users\Daniel86\AppData\Local\Programs\Python\Python36\python.exe You should be able to replace that with py -3.6 but try py -3.6 -c "import sys; sys.executable" to be sure. Terry Reedy hat am 24. Oktober 2017 um 08:36 geschrieben: On 10/23/2017 10:23 AM, Daniel Tangemann wrote: I've recently downloaded and installed python 3.6. (I had already also 2.7 and 3.2 on my computer) Initially pip was looking in the wrong directory to install to, so I changed that. then it had trouble installing matplotlib, so I decided to get rid of the older versions of python, which srewed things up even more. now scrips that I had written (in 3.6), that were running without errors before, aren't working anymore. I tried reinstalling python, and I tried the repair option multiple times as well. when I look into the python folder, I can see the modules that I have installed (and that I import into those scripts), but the IDLE doesn't see them! what's even more weird, is that "pip list" doesn't bring up anything but pip itself, while typing "pip install matplotlib" returns a message that it's already installed. how do I fix this? cheers Recognition of installed packages is done by the python running IDLE and executing your import statements, by not IDLE. The only effect IDLE could have is any manipulation of sys.path. You can find the executable running IDLE with import sys; sys.executable 'C:\\Programs\\Python37\\pythonw.exe' Find the sys.path being used with sys.path If you run the same binary (minus the 'w' if present), you can find the sys.path used without IDLE. You can also test imports without IDLE in use. It is possible that you have more than one binary around, but I cannot tell from here. To make sure you are running pip with the same binary as IDLE, enter path-to-binary -m pip C:\Programs\Python37\python.exe -m pip list -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list -- Terry Jan Reedy -- https://mail.python.org/mailman/listinfo/python-list
Re: Compression of random binary data
Thomas Jollans On 2017-10-25 23:22, danceswi...@gmail.com wrote: > With every transform the entropy changes, That's only true if the "transform" loses or adds information. If it loses information, that's lossy compression, which is only useful in very specific (but also extremely common) circumstances. If it adds information, that's poetry, not compression. Not true! You can transform a stream lossless, and change its entropy without adding to or taking away. These two streams 16 bits are the same but one is reversed. 10101011 This equals 61611 This can be represented using 0-6 log2(7)*5= 14.0367746103 bits 11010101 This equals 54543 This can be represented using 0-5 log2(6)*5= 12.9248125036 bits In reality you can express 54543 with 10 bits. -- https://mail.python.org/mailman/listinfo/python-list
Re: Compression of random binary data
Gregory Ewing writes: > Ben Bacarisse wrote: >> The trouble is a pedagogic one. Saying "you can't compress random data" >> inevitably leads (though, again, this is just my experience) to endless >> attempts to define random data. > > It's more about using terms without making sure everyone agrees > on the definitions being used. > > In this context, "random data" really means "uniformly distributed > data", i.e. any bit sequence is equally likely to be presented as > input. *That's* what information theory says can't be compressed. But that has to be about the process that gives rise to the data, not the data themselves. No finite collection of bits has the property you describe. If I say: "here is some random data..." you can't tell if it is or is not from a random source. I can, as a parlour trick, compress and recover this "random data" because I chose it. A source of random can be defined but "random data" is much more illusive. >> I think "arbitrary data" (thereby including the results of compression >> by said algorithm) is the best way to make progress. > > I'm not sure that's much better, because it doesn't home in > on the most important thing, which is the probability > distribution. I think the argument that you can't compress arbitrary data is simpler to make. You don't have to define it (except in the very simplest terms) and it's obvious that it includes the results of previous compressions. -- Ben. -- https://mail.python.org/mailman/listinfo/python-list
Sequence MIDI events from python.
I know that there are a few good MIDI libraries out there. The examples that I've seen for real-time triggering of events rely on a sleep function to realize the timing. This is not accurate or precise enough for musical applications. What options do I have if I want to write a MIDI sequencer in python? I imagine I'd have to sync to an audio device to get the timing right. Thank for any help. Tobiah -- https://mail.python.org/mailman/listinfo/python-list
Re: Sequence MIDI events from python. (Posting On Python-List Prohibited)
On 10/26/2017 4:30 PM, Lawrence D’Oliveiro wrote: On Friday, October 27, 2017 at 12:02:40 PM UTC+13, Tobiah wrote: I know that there are a few good MIDI libraries out there. The examples that I've seen for real-time triggering of events rely on a sleep function to realize the timing. This is not accurate or precise enough for musical applications. Why not look at the source code of the “good” ones? No get. zample plez. -- https://mail.python.org/mailman/listinfo/python-list
Re: Compression of random binary data
On Thu, Oct 26, 2017 at 2:38 PM, wrote: > > Thomas Jollans > > On 2017-10-25 23:22, danceswi...@gmail.com wrote: >> With every transform the entropy changes, > > That's only true if the "transform" loses or adds information. > > If it loses information, that's lossy compression, which is only useful > in very specific (but also extremely common) circumstances. > > If it adds information, that's poetry, not compression. > > > > Not true! You can transform a stream lossless, and change its entropy without > adding to or taking away. These two streams 16 bits are the same but one is > reversed. > > 10101011 > This equals > 61611 > This can be represented using > 0-6 log2(7)*5= 14.0367746103 bits > > > 11010101 > This equals > 54543 > This can be represented using > 0-5 log2(6)*5= 12.9248125036 bits > > In reality you can express 54543 with 10 bits. I don't know what you're calculating here but it isn't Shannon entropy. Shannon entropy is correctly calculated for a data source, not an individual message, but if we assume the two numbers above to be the result of a Bernoulli process with probabilities matching the frequencies of the bits in the numbers, then the total entropy of 16 events is: py> 16 * (-9/16 * math.log2(9/16) - 7/16 * math.log2(7/16)) 15.81919053261596 Approximately 15.8 bits. This is the same no matter what order the events occur in. -- https://mail.python.org/mailman/listinfo/python-list
Re: IDLE doesn't recognise installed packages
hi, I had tried typing: "path-to-binary -m pip Terry Reedy hat am 26. Oktober 2017 um 21:35 geschrieben: > > > On 10/26/2017 12:37 AM, Daniel Tangemann wrote: > > ok, I did that. I noticed that this path: > > 'C:\\Users\\Daniel86\\AppData\\Local\\Programs\\Python\\Python36\\Lib\\idlelib' > > is missing when I run the python.exe without IDLE. how do I fix this? > > Having idlelib on the path or not should not make any difference for > anything installed by pip. It is not even needed by IDLE, since IDLE > imports its modules via Lib. > > > > also I get a syntax error when I try that: > > What you try what? Post the entire traceback. > > > "To make sure you are running pip with the same binary > > as IDLE, enter path-to-binary -m pip > Your path-to-binary appears to be > > C:\Users\Daniel86\AppData\Local\Programs\Python\Python36\python.exe > > You should be able to replace that with > > py -3.6 > > but try > > py -3.6 -c "import sys; sys.executable" > > to be sure. > > >> Terry Reedy hat am 24. Oktober 2017 um 08:36 > >> geschrieben: > >> > >> > >> On 10/23/2017 10:23 AM, Daniel Tangemann wrote: > >>> I've recently downloaded and installed python 3.6. (I had already also > >>> 2.7 and 3.2 on my computer) Initially pip was looking in the wrong > >>> directory to install to, so I changed that. then it had trouble > >>> installing matplotlib, so I decided to get rid of the older versions of > >>> python, which srewed things up even more. now scrips that I had written > >>> (in 3.6), that were running without errors before, aren't working > >>> anymore. I tried reinstalling python, and I tried the repair option > >>> multiple times as well. when I look into the python folder, I can see the > >>> modules that I have installed (and that I import into those scripts), but > >>> the IDLE doesn't see them! what's even more weird, is that "pip list" > >>> doesn't bring up anything but pip itself, while typing "pip install > >>> matplotlib" returns a message > > that > >>> it's already installed. how do I fix this? > >>> cheers > >> > >> Recognition of installed packages is done by the python running IDLE and > >> executing your import statements, by not IDLE. The only effect IDLE > >> could have is any manipulation of sys.path. > >> > >> You can find the executable running IDLE with > >> > > import sys; sys.executable > >> 'C:\\Programs\\Python37\\pythonw.exe' > >> > >> Find the sys.path being used with > > sys.path > >> > >> If you run the same binary (minus the 'w' if present), you can find the > >> sys.path used without IDLE. You can also test imports without IDLE in use. > >> > >> It is possible that you have more than one binary around, but I cannot > >> tell from here. To make sure you are running pip with the same binary > >> as IDLE, enter path-to-binary -m pip >> instance, on windows, given the above > >> > >> path> C:\Programs\Python37\python.exe -m pip list > >> > >> -- > >> Terry Jan Reedy > >> > >> -- > >> https://mail.python.org/mailman/listinfo/python-list > > > -- > Terry Jan Reedy > > -- > https://mail.python.org/mailman/listinfo/python-list -- https://mail.python.org/mailman/listinfo/python-list
Re: Compression of random binary data
It looks like that averages my two examples. H by the way that equation is really coolwhy does it return a high bit count when compared to >>>dec to bin? -- https://mail.python.org/mailman/listinfo/python-list
Python noob having a little trouble with strings
If s1 = "Welcome students", what is the output when you print the following: 1. s4 = 3 * s1 2. s1[3 : 6] 3. 'W' in s1 4. S1[-1] 5. S1[:-1] Any help would be great, thanks! -- https://mail.python.org/mailman/listinfo/python-list
Re: Python noob having a little trouble with strings
On Thu, Oct 26, 2017 at 9:25 PM, wrote: > If s1 = "Welcome students", what is the output when you print the following: > > 1. s4 = 3 * s1 > > 2. s1[3 : 6] > > 3. 'W' in s1 > > 4. S1[-1] > > 5. S1[:-1] > > Any help would be great, thanks! Why not find out for yourself and print these in the Python interpreter? For instance: > py Python 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:57:36) [MSC v.1900 64 bit (AMD64)] on win32 Type "help", "copyright", "credits" or "license" for more information. py3: s1 = "Welcome students" py3: s4 = 3 * s1 py3: print(s4) Welcome studentsWelcome studentsWelcome students -- boB -- https://mail.python.org/mailman/listinfo/python-list
Re: Compression of random binary data
Shouldn't that be? py> 16 * (-7/16 * math.log2(7/16) - 6/16 * math.log2(6/16)) = -- https://mail.python.org/mailman/listinfo/python-list
Re: Python noob having a little trouble with strings
On Thursday, October 26, 2017 at 7:41:10 PM UTC-7, boB Stepp wrote: > On Thu, Oct 26, 2017 at 9:25 PM, wrote: > > If s1 = "Welcome students", what is the output when you print the following: > > > > 1. s4 = 3 * s1 > > > > 2. s1[3 : 6] > > > > 3. 'W' in s1 > > > > 4. S1[-1] > > > > 5. S1[:-1] > > > > Any help would be great, thanks! > > Why not find out for yourself and print these in the Python > interpreter? For instance: > > > py > Python 3.6.2 (v3.6.2:5fd33b5, Jul 8 2017, 04:57:36) [MSC v.1900 64 > bit (AMD64)] on win32 > Type "help", "copyright", "credits" or "license" for more information. > py3: s1 = "Welcome students" > py3: s4 = 3 * s1 > py3: print(s4) > Welcome studentsWelcome studentsWelcome students > > > > -- > boB Hi Bob, thanks for responding. I'm not sure where to do so, my professor had us download Pycharm for mac's which uses python 2.6 -- https://mail.python.org/mailman/listinfo/python-list
How to plot
Hello, i'd like to create a graph/plot based a DB table's data, but not sure where to start. I also would like to have the following functionality: a. i'd like to have it in the separate window ( xwindow to be precise). b. and i'd like to have the graph updating with every record added to the table. The workflow: a. main code is running and occasionally adding records to the table b. graph gets updated "automagically" and is shown in a separate from the main terminal window. Main program does not have GUI, nor is Web based, just runs in the terminal on Xwindows. i don't' really care about cross-platform compability and only want to run that on linux xwindows. Thank you for your advise. -- https://mail.python.org/mailman/listinfo/python-list
Re: Let's talk about debuggers!
Terry Reedy writes: > On Windows, [IDLE] uses native widgets when possible... > In summary, I think debugger should rate at least 'good' rather than > fail' when it comes to showing you the next line. I actually like how the Tk widgets look. I've done some semi-industrial applications with tkinter and they have a nice factory-floor vibe. I generally just use pdb for debugging. The feature I miss most is the ability to trap to the debugger if the program throws an unhandled exception. I think some other Python debuggers do support that. I've found it invaluable in other languages. -- https://mail.python.org/mailman/listinfo/python-list
Re: Compression of random binary data
Ben Bacarisse : >> In this context, "random data" really means "uniformly distributed >> data", i.e. any bit sequence is equally likely to be presented as >> input. *That's* what information theory says can't be compressed. > > But that has to be about the process that gives rise to the data, not > the data themselves. No finite collection of bits has the property you > describe. Correct. Randomness is meaningful only in reference to future events. Once the events take place, they cease to be random. A finite, randomly generated collection of bits can be tested against a probabilistic hypothesis using statistical methods. Marko -- https://mail.python.org/mailman/listinfo/python-list
