Re: sqlite3 and dates

[Frank Millman]

"Mark Lawrence"  wrote in message 
news:mc1g3n$q8j$1...@ger.gmane.org...
> On 18/02/2015 06:19, Frank Millman wrote:
>> Hi all
>>
>> sqlite3 does not have a DATE type, but the python module does a pretty 
>> good
>> job of providing one -
>>
>> However, the following does not return a date object -
>>
> cur.execute('SELECT CAST(? AS DATE)', ('2015-03-31',))
>> 
> cur.fetchone()
>> (2015,)
>
>>
>
> Will this do?
>
> cur.execute('select current_date as "d [date]", current_timestamp as "ts 
> [timestamp]"')
> row = cur.fetchone()
> print("current_date", row[0], type(row[0]))
> print("current_timestamp", row[1], type(row[1]))
>


I will have to experiment a bit, It looks as if it will do just fine.

The magic incantation is 
'detect_types=sqlite3.PARSE_DECLTYPES|PARSE_COLNAMES'

I had not looked at PARSE_COLNAMES before. Very useful.

Thanks, Mark

Frank



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[janhein . vanderburg]

On Tuesday, February 17, 2015 at 2:17:02 PM UTC+1, Chris Angelico wrote:
> This is a fine forum to ask in. However, you may find that the advice
> you get isn't quite what you were asking for. In my case, the advice
> I'm offering is: Don't do this.
Thanks Chris; let me explain why I want this.

As you rightly point out, human readable encoding enables debugging without the 
need for specialized tools that assemble/disassemble character streams.
I also agree that saving space in a partition of a character stream like a 
communication packet or persistent memory sector won't do much good if it does 
not save additional partitions.

The human centered encoding versus machine oriented encoding discussion dates 
from some decades ago now, and I am personally still not convinced that human 
readable text is all that sensible in most applications.
The main problem I see is that we are trying to make computers behave like 
human beings, and that introduces interpretation risks and resource 
requirements that should be avoided.
I'll be happy to discuss this subject again, but let's do that in a different 
thread.

The difference between two encoded stream lengths within a single partition may 
not be relevant, but the difference between needing 1 or 2 partitions is quite 
significant.
Especially in applications that depend on very limited bandwidth communication 
channels.
Since all messages in my applications are primarily composed of  integer 
values, finding the algorithm that fundamentally minimizes the number of bits 
to encode such a value will increase my chances to save partitions needed to 
encode the message, even without adding compression techniques at the message 
level.
Note that I don't assume any properties of the integer values to be encoded.

> Take the
> easy option; you can always make things more complicated later.
That makes sense alright.
No offense, but I still believe that human readable text encoding complicates 
things right now and shouldn't be tried until "my way" has proven unpractical 
in its first application.
Consider it to be a theoretical challenge: how do I find the general encoding 
of an arbitrary integer value that minimizes the number of bits needed and 
given that algorithm, find the python code that minimizes the processor load 
inflicted by the codec implementation.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[janhein . vanderburg]

On Tuesday, February 17, 2015 at 3:13:41 PM UTC+1, Dave Angel wrote:
> This is a fine forum for such a discussion.  I for one would love to 
> participate.  However, note that it isn't necessary true that "the 
> smaller the better" is a good algorithm.  In context, there are 
> frequently a number of tradeoffs, even ignoring processor time (as you 
> imply).
Thanks Dave; about those trade offs:

I agree that allowing variable length encoding in general forces the programmer 
to process records in a file sequentially.
Random access to individual data items in such a stream is simply not an 
option, unless all items have been indexed in a directory that accompanies the 
data character stream.
So if you need random access to individual items put limits to each individual 
data item.
My applications do not allow me to do that, because I would always be to 
restrictive and be caught by the "640k should be enough for any programmer" 
trap.

> So going back to your problem, and assuming that the other issues are 
> moot, what's your proposal?  Are you compressing relative to a straight 
> binary form of storage?  Are you assuming anything about the relative 
> likelihood of various values of integers?  Do you provide anything to 
> allow for the possibility that your prediction for probability 
> distribution isn't met?

I'm not compressing sequences of integer encodings but encoding individual 
integers optimally without any assumptions about their values.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[janhein . vanderburg]

On Tuesday, February 17, 2015 at 3:35:16 PM UTC+1, Chris Angelico wrote:
> Oh, incidentally: If you want a decent binary format for
> variable-sized integer, check out the MIDI spec.

I did some time ago, thanks, and it is indeed a decent format.
I also looked at variations of that approach.
None of them beats "my" concept of two counters that cooperatively specify 
field lengths and represented integer values.

> I've tried to read through the original algorithm description, but I'm
> not entirely sure: How many payload bits per transmitted byte does it
> actually achieve?

I don't think that payload bits per byte makes sense in this concept.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[janhein . vanderburg]

On Tuesday, February 17, 2015 at 5:43:43 PM UTC+1, Paul Rubin wrote:

> This is a reasonable place to ask specific python questions.  The
> algorithm description itself is pretty confusing though, and it seems to
> address a problem that doesn't particularly seem to need a solution.
> It's pretty typical for applications needing compact representations to
> encode smallish variable-length integers in the VInt format, basically
> using 7 bits from each byte for data, and the 8th bit as a terminator
> flag.  So VInt encodes numbers 0-127 in one byte, 128 to 16383 in 2
> bytes, etc.  Bignum applications generally use a length cell and a data
> array.

Thanks Paul,

The algorithm sort of combines the Vint  and Bignum approaches.
I'm sorry to read that my description is confusing; is the illustration with 2 
bit characters helpful or should it be extended?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Chris Angelico]

On Wed, Feb 18, 2015 at 7:55 PM,   wrote:
>> Take the
>> easy option; you can always make things more complicated later.
> That makes sense alright.
> No offense, but I still believe that human readable text encoding complicates 
> things right now and shouldn't be tried until "my way" has proven unpractical 
> in its first application.
> Consider it to be a theoretical challenge: how do I find the general encoding 
> of an arbitrary integer value that minimizes the number of bits needed and 
> given that algorithm, find the python code that minimizes the processor load 
> inflicted by the codec implementation.
>

I would actually look at it the other way: a human-readable encoding
is the easy way (you can simply print() your numbers and int() them on
the way back in), and until you can prove that it's impractical, don't
write a single line of code towards this algorithm. But let's get some
figures, though: How many payload bits per byte can you achieve?
What's your average going to be? You have two baselines to beat: 4
bits per byte (hexadecimal), and 7 bits per byte (MIDI varlen, or
whatever you want to call it; I first met it with MIDI, but it's been
used in so many places that other people have other names for it).

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[Chris Angelico]

On Wed, Feb 18, 2015 at 6:49 PM, Frank Millman  wrote:
> My accounting software supports three databases - MS Sql Server, PostgreSQL,
> and sqlite3.
>
> sqlite3 is not suitable for 'heavy-duty' applications, but it is ideal for
> demos and one-man businesses. Anyone can try out my software so long as they
> have python installed. They do not have to set up a database.

I wouldn't trust sqlite3 with my accounting... but then, I'm becoming
more and more disillusioned with most full-scale accounting packages,
too. I'm seriously looking toward a git-managed directory of text
files as being the accounting package of the future; a few pre-commit
hooks (and/or pre-receive if you want to have a central repo that
people push to) to check file formats and stuff; and then some scripts
that do maintenance and reporting (eg updating inventory quantities
when you mark an invoice as completed). I want to see a big company
work this way... it's clearly not a problem for a one-man operation,
as I've been doing exactly that for some time (with very few scripts),
and we do know that git scales to global operations just fine, but
will it actually work? Inquiring minds must know!

But anyway, I wouldn't push people onto sqlite3 for anything serious,
mainly because it sucks at concurrency, secondarily because it lacks a
number of common features. I'd push people to PostgreSQL.

> sqlite3 does support ALTER TABLE, but with limited functionality. I think
> all you can do is add a column.

Ah, true. Minor support for altering tables, not quite none. But
still, there's a lot of limitations that will bite you badly any time
you try to migrate a schema using anything other than the brute-force
"create entire new database and import the content".

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Laura Creighton]

Hi Jan.

I'm an old fart.  In the late 1970s, when I started programming these
things, and memory was non-existant, we came up with all sorts of data
compression algorithms which were absolutely necessary to get any work
done whatsoever.  Should you ever need an assembler programmer for
quick and dirty hacks for the PDP-11 line (11/20 and 11/05 preferred
as it is harder) I am still the woman for the job.  Indeed, I spent
most of my 20s finding better and better ways to fit programs into
smaller and smaller memory footprints.

I perfectly understand the intellectual thrill of doing such things.
As puzzles go, it is about as cool a one as exists, and its all for
things that matter -- for real.

However, in the matter of financial compensation and world recognition,
you have just laid a very large goose-egg.   The VAX-11/780 was introduced
on October 25, 1977, according to wikipedia.  But in my world, it was 1982
before I got to see the first one.  And it was godly more expensive than
a pdp-11, but the writing was on the wall.  The thing could page, and
so all the techniques we learned for making our code concise -- let alone
the dirty tricks I specialised in -- were no longer relevant.

>From the mid 1980s onward I have been telling people 'your code is
ugly, please tighten it up by refactoring it  and ' and
when I am their instructor they grumble and do it, and otherwise they
flip me the bird.  In their eyes, it doesn't matter how the code
_looks_ as long as it does the job.  And I deeply sympathise.  But
what I am going for is not a 'death - by looking unfashionable' but
rather a demand that good code is clear to understand.  Because what
I have learned, that Brian Kernighan expressed a long time ago is
that:

Debugging is twice as hard as writing the code in the
first place. Therefore, if you write the code as cleverly
as possible, you are, by definition, not smart enough to debug it.

Your proposed encoding scheme (if it does as you say, I have not
analysed it) scores very, very high in the _cleverness_ department.
Enough that a lot of people, who aren't as clever as you do, have
no hope in hell of ever being clever enough to debug something that
uses it.  Therefore, you will never see widespread adoption of your
scheme, no matter how brilliantly it does as you say, because we all
need things that are easier to debug more than we need better compression.

So now you are sad.  I was sad, too, but the sooner I learned this the
sooner I could stop wasting my time creating algorithms that provided
cool functionality that people hated for the same reasons I found them
cool.

You need to find a different sort of algorithm that people like to use
if you want to get widespread success in the world of widely used
algorithms.  If you have found a way to improve on Lemel-Ziv, then this
will count.

But it may be that your next step is 'how to encode things that
are not phonetic language'.  Go look -- for the next few months --
at how MIDI stores sounds.  You will find plenty of places for
improvement, but the idea is not to improve the standard but to learn
it well enough that you can see things in the non-alphabetic world.

So then, now what?

If you are still fired up with the desire to compress things, then
there is a huge, _very well paying_ market I want to introduce you
to.  And this is _tech support for porn sites_.  Porn sites make a
ton of money, indeed the numbers are scary.  And here the idea of
'I saved 2% of time/bandwidth/disk space/' really matters.  You
can really save money for them, and it really matters to them.
Since I have never found sex 'dirty' and indeed consider it one
of the great joys in life, I have never found anything wrong with
working for porn sites.

And, hell, out of male porn Jimmy Wales made wikipedia.  Haven't
we all benefitted?

But, right now, you are, alas for you, about 45 years too late for the
ideas you are sprouting.  I had similar ones about 30 years too late
and, well, they only worked for me for about 3-5 years.  Sucks to be
you, friend -- you needed to be your grandfather, I fear.


Laura Creighton
-- 
https://mail.python.org/mailman/listinfo/python-list

People hated it for the same reasons I found them cool (was: python implementation of a new integer encoding algorithm.)

[Ben Finney]

Laura Creighton  writes:

> So now you are sad. I was sad, too, but the sooner I learned this the
> sooner I could stop wasting my time creating algorithms that provided
> cool functionality that people hated for the same reasons I found them
> cool.

+1 QotW

-- 
 \“Human reason is snatching everything to itself, leaving |
  `\   nothing for faith.” —Bernard of Clairvaux, 1090–1153 CE |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[Johannes Bauer]

On 18.02.2015 08:05, Chris Angelico wrote:

> But if you need more facilities than SQLite3 can offer, maybe it's
> time to move up to a full database server, instead of local files.
> Switching to PostgreSQL will give you all those kinds of features,
> plus a lot of other things that I would have thought pretty basic -
> like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't
> support that.

I see you're running a lawnmower. Maybe you should switch to a combine
harvester. That'll get you extra features like a reciprocating knife
cutter bar. I was quite surprised that regular lawnmowers don't support
those.

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[Chris Angelico]

On Wed, Feb 18, 2015 at 10:11 PM, Johannes Bauer  wrote:
> On 18.02.2015 08:05, Chris Angelico wrote:
>
>> But if you need more facilities than SQLite3 can offer, maybe it's
>> time to move up to a full database server, instead of local files.
>> Switching to PostgreSQL will give you all those kinds of features,
>> plus a lot of other things that I would have thought pretty basic -
>> like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't
>> support that.
>
> I see you're running a lawnmower. Maybe you should switch to a combine
> harvester. That'll get you extra features like a reciprocating knife
> cutter bar. I was quite surprised that regular lawnmowers don't support
> those.

SQLite3 is fine for something that's basically just a more structured
version of a flat file. You assume that nobody but you has the file
open, and you manipulate it just the same as if it were a big fat blob
of JSON, but thanks to SQLite, you don't have to rewrite the whole
file every time you make a small change. That's fine. But it's the
wrong tool for any job involving multiple users over a network, and
quite probably the wrong tool for a lot of other jobs too. It's the
smallest-end piece of software that can truly be called a database. I
would consider it to be the wrong database for serious accounting
work, and that's based on the ranting of a majorly-annoyed accountant
who had to deal with issues in professional systems that had made
similar choices in back-end selection.

You're welcome to disagree, but since PostgreSQL doesn't cost any
money and (on Linux at least; can't speak for other platforms) doesn't
take significant effort to set up, I will continue to recommend it.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[Johannes Bauer]

On 18.02.2015 12:21, Chris Angelico wrote:

> SQLite3 is fine for something that's basically just a more structured
> version of a flat file. You assume that nobody but you has the file
> open, and you manipulate it just the same as if it were a big fat blob
> of JSON, but thanks to SQLite, you don't have to rewrite the whole
> file every time you make a small change. That's fine. But it's the
> wrong tool for any job involving multiple users over a network, and
> quite probably the wrong tool for a lot of other jobs too.

Your assessment that some tools fit certain problems and don't fit
different problems is entirely correct. SQLite does the job that it is
supposed to do and it fills that nieche well.

> It's the
> smallest-end piece of software that can truly be called a database. I
> would consider it to be the wrong database for serious accounting
> work, and that's based on the ranting of a majorly-annoyed accountant
> who had to deal with issues in professional systems that had made
> similar choices in back-end selection.

It probably is the wrong database for serious accounting work, and it's
probably also the wrong database for doing multivariate statistical
analysis on sparse matrices that you store in tables.

You could similarly argue that a hammer is the wrong tool to drive in a
screw and you'd be correct in that assessment. But it's completely
besides the point.

SQLite and Postgres are so vastly different in their setup,
configuration, capabilities and requirements that the original developer
has to have done a MAJOR error in judgement so that a change from one to
the other would not be ill-advised.

> You're welcome to disagree, but since PostgreSQL doesn't cost any
> money and (on Linux at least; can't speak for other platforms) doesn't
> take significant effort to set up, I will continue to recommend it.

I work with Postgres on a professional, day-to-day basis. And while it's
free, it *does* take a significant effort to setup and it *does* take a
significant effort to maintain. Especially in comparison with something
like SQLite that literally has no setup at all.

PostgreSQL is great. It's an incredible database and that it's free is
amazing. But in very few settings will it be a replacement for SQLite.

Cheers,
Johannes


-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa 
-- 
https://mail.python.org/mailman/listinfo/python-list

Dependencies from git aren't being installed? (setuptools, pip)

[Samuel Marks]

I'm sure I've just misconfigured my setup.py, requirements.txt and/or project 
structure, but I've tried:
* install_requires
* requirements.txt
* install_requires + dependency_links

With (for install_requires):
* bettertutors_sql_models==dev
* bettertutors_sql_models

For requirements.txt, dependency_links:
* git+https://github.com/bettertutors/sql-models#egg=bettertutors_sql_models
* git+https://github.com/bettertutors/sql-models#egg=bettertutors_sql_models
* 
https://github.com/bettertutors/sql-models/archive/master.zip#egg=bettertutors_sql_models
* https://github.com/bettertutors/sql-models/archive/master.zip
* https://github.com/bettertutors/sql-models
* https://github.com/bettertutors/sql-models/tarball/master#egg=sql-models

Also asked here: http://stackoverflow.com/q/28540839

At one point I even tried writing my own install_deps function() to be run 
before setup(): https://gist.github.com/SamuelMarks/45a998a83dd60ddbadbc

But nothing--apart from `cd`ing to my sql-models dir and running `pip install 
.`--is working for me. I'm using Python 2.7.*, and have experienced identical 
issues on Windows and Linux (with/without virtualenv).

How am I meant to write in this dependency?
-- 
https://mail.python.org/mailman/listinfo/python-list

Issues pip-installing pywin32

[Chris Angelico]

(Before I begin, I want to make it clear that I can still go poke
around on SourceForge and grab the appropriate installer [1], and that
does work. But if I'm going to tell someone else how to set up this
program, I'd much rather be able to recommend pip.)

On a fresh Python 3.4 running on Windows 7, attempting to "pip install
pywin32" doesn't work:

C:\Users\Rosuav>pip install pywin32
Downloading/unpacking pywin32
  Could not find any downloads that satisfy the requirement pywin32
  Some externally hosted files were ignored (use --allow-external
pywin32 to allow).
Cleaning up...
No distributions at all found for pywin32
Storing debug log for failure in C:\Users\Rosuav\pip\pip.log

C:\Users\Rosuav>pip install --allow-external pywin32
You must give at least one requirement to install (see "pip help install")


Okay, so the obvious command doesn't work, and gives a
not-particularly-helpful error message, but I know how to Google, I
can get past this.


C:\Users\Rosuav>pip install --allow-external pywin32
--allow-unverified pywin32 pywin32
Downloading/unpacking pywin32
  Could not find any downloads that satisfy the requirement pywin32
Cleaning up...
No distributions at all found for pywin32
Storing debug log for failure in C:\Users\Rosuav\pip\pip.log


Huh. The PyPI categorization seems to exclude Python 3.4 support
(despite versions for both 3.4 and 3.5 existing on the sourceforge
page). Let's try that with 2.7, just to see what happens. (I could
alternatively backlevel to 3.3, I suppose.)

C:\Users\Rosuav>\python27\python -m pip --version
pip 1.5.6 from C:\python27\lib\site-packages (python 2.7)

C:\Users\Rosuav>\python27\python -m pip install --allow-external
pywin32 --allow-unverified pywin32 pywin32
Downloading/unpacking pywin32
  Could not find any downloads that satisfy the requirement pywin32
Cleaning up...
No distributions at all found for pywin32
Storing debug log for failure in C:\Users\Rosuav\pip\pip.log


The log contains these lines, which may be a clue as to what's going on...

  Could not fetch URL
https://sourceforge.net/projects/pywin32/files/pywin32/ (from
https://pypi.python.org/simple/pywin32/): connection error: hostname
'sourceforge.net' doesn't match either of 'cloudfront.net',
'*.cloudfront.net'
  Will skip URL
https://sourceforge.net/projects/pywin32/files/pywin32/ when looking
for download links for pywin32
  Could not find any downloads that satisfy the requirement pywin32

Does this mean there's an HTTPS configuration error at sourceforge?
And if that is indeed the problem, is there any way that I, as an end
user, can bypass the check? My alternative is using a non-SSL link, so
it's not like I'd actually have worse security.

At this point, I am REALLY glad my Dad asked me to set this all up for
him, rather than just giving him some instructions and saying "go for
it". (And to think, all I want is a simple program to bounce some info
out from a Windows VM onto the Linux host. So much hassle for
something so simple.)

[1] http://sourceforge.net/projects/pywin32/files/pywin32/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[Chris Angelico]

On Wed, Feb 18, 2015 at 10:57 PM, Johannes Bauer  wrote:
> SQLite and Postgres are so vastly different in their setup,
> configuration, capabilities and requirements that the original developer
> has to have done a MAJOR error in judgement so that a change from one to
> the other would not be ill-advised.

On Wed, Feb 18, 2015 at 6:49 PM, Frank Millman  wrote:
> My accounting software supports three databases - MS Sql Server, PostgreSQL,
> and sqlite3.

Johannes, are you saying that Frank made three major errors of judgement? :)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[Johannes Bauer]

On 18.02.2015 13:14, Chris Angelico wrote:
> On Wed, Feb 18, 2015 at 10:57 PM, Johannes Bauer  wrote:
>> SQLite and Postgres are so vastly different in their setup,
>> configuration, capabilities and requirements that the original developer
>> has to have done a MAJOR error in judgement so that a change from one to
>> the other would not be ill-advised.
> 
> On Wed, Feb 18, 2015 at 6:49 PM, Frank Millman  wrote:
>> My accounting software supports three databases - MS Sql Server, PostgreSQL,
>> and sqlite3.
> 
> Johannes, are you saying that Frank made three major errors of judgement? :)

I'm totally pulling my fifth! :-P

Cheers,
Johannes

-- 
>> Wo hattest Du das Beben nochmal GENAU vorhergesagt?
> Zumindest nicht öffentlich!
Ah, der neueste und bis heute genialste Streich unsere großen
Kosmologen: Die Geheim-Vorhersage.
 - Karl Kaos über Rüdiger Thomas in dsa 
-- 
https://mail.python.org/mailman/listinfo/python-list

Logging with Custom Levels not working

[Didymus]

Greetings,

I've been setting up a custom logging for a project here and got some great 
examples of code from stackoverflow. The only problem I'm having is that no 
matter what I set the log level too all the messages print out...

Verbosity Level set to: 55
[PVERBOSE] Performance Verbose Level
[PDEBUG] Performance Debug Level
[PMESSAGE] Performance Message Level
[PWARNING] Performance Warning Level
[PERROR] Performance Error Level

This should only print the PERROR level, but everything in the custom 
levels is being printed out. Not sure what I'm doing wrong here. Any help is 
greatly appreciated. We are use Python 2.7.5 .Example Code below:

   Thanks
  Tom

#!/usr/bin/python -tt

import logging

#
# Performance Custom Message Levels 'PERROR' : 55, 'PWARNING' : 54, 'PMESSAGE': 
53, 'PDEBUG' : 52, 'PVERBOSE' : 51
#
PERROR_NUM   = 55
PWARNING_NUM = 54
PMESSAGE_NUM = 53
PDEBUG_NUM   = 52
PVERBOSE_NUM = 51


# Performance Error...
logging.addLevelName(PERROR_NUM, "PERROR")

def perror(self, message, *args, **kws):
""" Performance Error Message Level """
# Yes, logger takes its '*args' as 'args'.
self._log(PERROR_NUM, message, args, **kws)

logging.Logger.perror = perror


# Performance Warning...
logging.addLevelName(PWARNING_NUM, "PWARNING")


def pwarning(self, message, *args, **kws):
""" Performance Warning Message Level """
# Yes, logger takes its '*args' as 'args'.
self._log(PWARNING_NUM, message, args, **kws)

logging.Logger.pwarning = pwarning


# Performance Messages...
logging.addLevelName(PMESSAGE_NUM, "PMESSAGE")

def pmessage(self, message, *args, **kws):
""" Performance Info/Message Level """
# Yes, logger takes its '*args' as 'args'.
self._log(PMESSAGE_NUM, message, args, **kws)

logging.Logger.pmessage = pmessage

# Performance Debug...
logging.addLevelName(PDEBUG_NUM, "PDEBUG")

def pdebug(self, message, *args, **kws):
""" Performance Debug Message Level """
# Yes, logger takes its '*args' as 'args'.
self._log(PDEBUG_NUM, message, args, **kws)

logging.Logger.pdebug = pdebug

# Performance Verbose...
logging.addLevelName(PVERBOSE_NUM, "PVERBOSE")

def pverbose(self, message, *args, **kws):
""" Performance Verbose Message Level """
# Yes, logger takes its '*args' as 'args'.
self._log(PVERBOSE_NUM, message, args, **kws)

logging.Logger.pverbose = pverbose

#
# Setup Logging.
#
def SetLogging(verbosity):
""" Set logging file and level. """
global rootLogger 
logFormatter =  logging.Formatter("[%(levelname)s] %(message)s") # Output 
formatting for message (i.e. date/time, thread, line #, message level, message).
rootLogger = logging.getLogger()
# Add a File to log too...  
fileHandler = logging.FileHandler('/tmp/log.txt')
fileHandler.setFormatter(logFormatter)
rootLogger.addHandler(fileHandler)
# Put the message on the Console as well..  
consoleHandler = logging.StreamHandler()
consoleHandler.setFormatter(logFormatter)
rootLogger.addHandler(consoleHandler)
# Set what the logging level should be: 
level = logging.getLevelName(verbosity)  # This turns the 
level into a number, which setLevel uses or logging.INFO for "verbosity" 
variable.
rootLogger.setLevel(level)  
 # This can be DEBUG, INFO, WARN, ERROR or CRITICAL to control verbosity. 
Custom Levels: PVERBOSE (51), PDEBUG (52), PWARNING (53) or PERROR (54)
print 'Verbosity Level set to: %s' % level


#
#
#
verbosity = 'PERROR'
SetLogging(verbosity)

rootLogger.pverbose('Performance Verbose Level')
rootLogger.pdebug('Performance Debug Level')
rootLogger.pmessage('Performance Message Level')
rootLogger.pwarning('Performance Warning Level')
rootLogger.perror('Performance Error Level')

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Dave Angel]

On 02/18/2015 04:04 AM, janhein.vanderb...@gmail.com wrote:

On Tuesday, February 17, 2015 at 3:35:16 PM UTC+1, Chris Angelico wrote:

Oh, incidentally: If you want a decent binary format for
variable-sized integer, check out the MIDI spec.


I did some time ago, thanks, and it is indeed a decent format.
I also looked at variations of that approach.
None of them beats


Define "beats."  You might mean beats in simplicity, or in elegance, or 
in clarity of code.  But you probably mean in space efficiency, or 
"compression."  But that's meaningless without a target distribution of 
values that you expect to encode.


For example, if 99.9% of your values are going to be less than 255, then 
the most efficient byte encoding would be one that simply stores a value 
less than 255, and starts with an FF for larger values.  It's almost 
irrelevant how it encodes those larger values.


On the other hand, if most values are going to be in the 10,000 to 
20,000 bit size range, and a few will be much smaller, and a few will be 
very much larger, then it would be very practical to start with a size 
field, say 16 bits, followed by the raw packed data.  Naturally, the 
size field would need to have an escape value that indicates a larger 
field was needed.  In fact, the size field could be encoded in a 
7bits-per-byte manner, so it would encode an arbitrary sized number as well.




"my" concept of two counters that cooperatively specify field lengths and 
represented integer values.


I've tried to read through the original algorithm description, but I'm
not entirely sure: How many payload bits per transmitted byte does it
actually achieve?


I don't think that payload bits per byte makes sense in this concept.



Correct.  Presumably one means average payload bits per byte.

First one would have to define what the "standard" unencoded variable 
length integer format was.  Then one could call that size the payload 
size.  Then, in order to compute an average, one would have to specify 
an expected, or target distribution of values.  One then compares and 
averages the payload size for each typical value with the encoded size.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

PythonMagick on Windows DLL load failed

[julien levasseur]

I am using Python 2.7 on Windows 8.1.

Python 2.7 (r27:82525, Jul 4 2010, 07:43:08) [MSC v.1500 64 bit (AMD64)] on
win32

I installed ImageMagick from imagemagick.org


Then installed PythonMagick with pip from
http://www.lfd.uci.edu/~gohlke/pythonlibs/#pythonmagick

When I import PythonMagick it says:

Traceback (most recent call last):

  File "magic.py", line 2, in 

import PythonMagick

  File "C:\Python27\lib\site-packages\PythonMagick\__init__.py", line 1, in


from . import _PythonMagick

ImportError: DLL load failed: The application has failed to start because
its side-by-side configuration is incorrect. Please see the application
event log or use the command-line sxstrace.exe tool for more detail.


Any clues on where do look?


Question on stackoverflow
http://stackoverflow.com/questions/28538973/pythonmagick-on-windows-dll-load-failed



-- 
Julien Levasseur
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Chris Angelico]

On Thu, Feb 19, 2015 at 12:54 AM, Dave Angel  wrote:
>>> I've tried to read through the original algorithm description, but I'm
>>> not entirely sure: How many payload bits per transmitted byte does it
>>> actually achieve?
>>
>>
>> I don't think that payload bits per byte makes sense in this concept.
>>
>
> Correct.  Presumably one means average payload bits per byte.
>
> First one would have to define what the "standard" unencoded variable length
> integer format was.  Then one could call that size the payload size.  Then,
> in order to compute an average, one would have to specify an expected, or
> target distribution of values.  One then compares and averages the payload
> size for each typical value with the encoded size.

Average, or separately for different ranges. Alternatively, identify
which ranges of numbers can be encoded with how many bytes. For
instance, if you have a varlen length (in bytes) followed by that many
packed bytes, you would have:

2: up to 1<<8-1
3: up to 1<<16-1
4: up to 1<<24-1
...
128: up to 1<<1016-1
130: up to 1<<1024-1
131: up to 1<<1032-1
etc

Using varlen directly gives:

1: up to 1<<7-1
2: up to 1<<14-1
3: up to 1<<21-1
etc

So if your number is around about 1<<20, you know it's going to take 3
bytes to encode as varlen, or 4 to encode with the meta-length. Easy
comparisons.

I'm not sure how this algorithm compares, because it's extremely clever.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Assigning a function to sys.excepthook doesn't work in WSGI

[tak . govoril]

I want to generate an html page with http response status '200 OK' in case of 
an uncaught exception in my wsgi application, instead of web server's standard 
'500 Internal Server Error' response for such case. To do so, I've made the 
following sample code:

import sys

def application(environ, start_response):
def exception(etype, evalue, trace):
start_response('200 OK', [('Content-Type', 'text/plain')])
return ['Error']

sys.excepthook = exception

1/0

start_response('200 OK', [('Content-Type', 'text/plain')])
return ['OK']

But the function 'exception' is never called, and a standard '500 Internal 
Server Error' response is still generated by server in case of an uncaught 
exception.

I looked through the documentation, but unable to find the answer. Are there 
any ways to handle uncaught by try..except exceptions under mod_wsgi?
-- 
https://mail.python.org/mailman/listinfo/python-list

about gsoc 2015

[Nadeesh Dilanga]

Hi,

I'm a Computer Science undergradute student who like to participate in GSOC
this year.
Do you have any projects willing to publish for gsoc 2015.
I am more familiar with Python.
-- 
https://mail.python.org/mailman/listinfo/python-list

Fwd: how to collect old tweets from 03-20 Aug, 2014 for a particular location

[Md. Hasanuzzaman]

Hello,

I am a PhD student. I am new to python and using TwitterSearch to collect
tweets from 03-20 Aug for a particular location (geocode). I am trying to
run the following piece of code.

twitter = Twython(CONSUMER_KEY,CONSUMER_SECRET,ACCESS_KEY,ACCESS_SECRET)
results = twitter.cursor(twitter.search, q='',
max_id='501488707195924481', count=100, geocode='38.00,-91.00')

for result in results:
print (result['id_str'])

However, it is not returning anything. Please help me in this regard. Can I
collect old tweets without any keyword for a particular location?

Thanks in advance.

Regards,
Mohammed
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Dave Angel]

On 02/18/2015 03:59 AM, janhein.vanderb...@gmail.com wrote:



encoding individual integers optimally without any assumptions about their 
values.



Contradiction in terms.

--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: Assigning a function to sys.excepthook doesn't work in WSGI

[Ian Kelly]

On Wed, Feb 18, 2015 at 8:08 AM,   wrote:
> I want to generate an html page with http response status '200 OK' in case of 
> an uncaught exception in my wsgi application, instead of web server's 
> standard '500 Internal Server Error' response for such case. To do so, I've 
> made the following sample code:
>
> import sys
>
> def application(environ, start_response):
> def exception(etype, evalue, trace):
> start_response('200 OK', [('Content-Type', 'text/plain')])
> return ['Error']
>
> sys.excepthook = exception
>
> 1/0
>
> start_response('200 OK', [('Content-Type', 'text/plain')])
> return ['OK']
>
> But the function 'exception' is never called, and a standard '500 Internal 
> Server Error' response is still generated by server in case of an uncaught 
> exception.

sys.excepthook is called just before the interpreter exits due to an
exception. In a mod_wsgi environment, having the interpreter exit just
because of an exception would be undesirable. I don't know exactly
what it's doing under the hood, but I would assume that the exception
never makes it to sys.excepthook because the gateway itself is
catching the exception in order to generate the 500 response.

> I looked through the documentation, but unable to find the answer. Are there 
> any ways to handle uncaught by try..except exceptions under mod_wsgi?

Here is what PEP  has to say about error handling:
https://www.python.org/dev/peps/pep-/#error-handling
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Noob Parsing question

[kai . peters]

> >> > Given
> >> >
> >> > data = 
> >> > '{[][]}'
> >> >
> >> > How can I efficiently get dictionaries for each of the data blocks 
> >> > framed by <> ?
> >> >
> >> > Thanks for any help
> >>
> >> The question here is: What _can't_ happen? For instance, what happens
> >> if Fred's name contains a greater-than symbol, or a caret?
> >>
> >> If those absolutely cannot happen, your parser can be fairly
> >> straight-forward. Just put together some basic splitting (maybe a
> >> regex), and then split on the caret inside that. Otherwise, you may
> >> need a more stateful parser.
> >>
> >> ChrisA
> >
> > The data string is guaranteed to be clean - no such irregularities occur.
> 
> Okay!
> 
> (Side point: You've stripped off all citations, here, so it's not
> clear who said what. My shorthand signature isn't as useful as the
> full line identifying date, time, and person. It's polite to keep
> those lines, at least for the first level of quoting.)
> 
> What you want can be done with a regular expression. (Yes, yes, I
> know; now you have two problems.)
> 
> >>> data = 
> >>> '{[][]}'
> >>> re.findall("<.*?>",data)
> ['', '', '', 
> '']
> 
> >From there, you can crack open the different pieces:
> 
> >>> for piece in re.findall("<.*?>",data):
> ... d = {}
> ... for elem in piece[1:-2].split("^"):
> ... key, value = elem.split("=",1)
> ... d[key] = value
> ... print(d)
> ...
> {'c': '45.22', 'b': 'Fred', 'a': '14'}
> {'b': 'Joe', 'a': '22'}
> {'c': '3.20', 'a': '17'}
> {'b': 'Soup', 'a': '72'}
> 
> If you need some of those to be integers or floats, you'll need to do
> some post-processing on it, but this guarantees that you get the data
> out reliably. It depends on not having any of the special characters
> "=^<>" inside the elements, but other than that, it should be safe.
> 
> ChrisA

Thanks for your help - much appreciated!

KP
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Mark Lawrence]

On 18/02/2015 16:46, Dave Angel wrote:

On 02/18/2015 03:59 AM, janhein.vanderb...@gmail.com wrote:



encoding individual integers optimally without any assumptions about
their values.



Contradiction in terms.



I'm just pleased to see new blood coming through for my dream team, it's 
been a bit quiet recently.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Grant Edwards]

On 2015-02-18, Dave Angel  wrote:
> On 02/18/2015 03:59 AM, janhein.vanderb...@gmail.com wrote:
>
>
>> encoding individual integers optimally without any assumptions about
>> their values.
>
> Contradiction in terms.

Ah, that depends on whether the encoding has to be lossless or not.

For example:

  def encode_integer(i):
return 0;

Voila! Optimal encoding with no assumptions about intut values.

It is, however, a bit lossy.

-- 
Grant Edwards   grant.b.edwardsYow! Did you move a lot of
  at   KOREAN STEAK KNIVES this
  gmail.comtrip, Dingy?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What the Pythons docs means by "container" ?

[MRAB]

On 2015-02-18 02:14, candide wrote:

Le mercredi 18 février 2015 01:50:16 UTC+1, Chris Angelico a écrit :


So, what's a container? It's a thing that you put other objects
into.


I agree with this approach. The important point to consider here is
the last word in your definition : "into". There is the container and
there is the content (the objects into). The so-called built-in
containers (list, string, etc) are in conformance with this view.
Now, regarding a range object as a container disagrees completely
with the definition given in the PLR : there is no contents and hence
there is no container. For instance, range(10**6) doesn't hold any
kind of object, there are no reference to the int objects 0, 1, 2,
... As the range's docstring explains, range returns a VIRTUAL
sequence.


It's a virtual, read-only container that contains integers.

Try comparing range(10) with tuple(range(10)). Both contain integers.
Both have a length. Both can be indexed.
--
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Mark Lawrence]

On 18/02/2015 17:30, Grant Edwards wrote:

On 2015-02-18, Dave Angel  wrote:

On 02/18/2015 03:59 AM, janhein.vanderb...@gmail.com wrote:



encoding individual integers optimally without any assumptions about
their values.


Contradiction in terms.


Ah, that depends on whether the encoding has to be lossless or not.

For example:

   def encode_integer(i):
 return 0;

Voila! Optimal encoding with no assumptions about intut values.

It is, however, a bit lossy.



Is it this kind of coding that makes the PEP 393 FSR so poor?

--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: What the Pythons docs means by "container" ?

[Ian Kelly]

On Wed, Feb 18, 2015 at 11:04 AM, MRAB  wrote:
> On 2015-02-18 02:14, candide wrote:
>>
>> Le mercredi 18 février 2015 01:50:16 UTC+1, Chris Angelico a écrit :
>>
>>> So, what's a container? It's a thing that you put other objects
>>> into.
>>
>>
>> I agree with this approach. The important point to consider here is
>> the last word in your definition : "into". There is the container and
>> there is the content (the objects into). The so-called built-in
>> containers (list, string, etc) are in conformance with this view.
>> Now, regarding a range object as a container disagrees completely
>> with the definition given in the PLR : there is no contents and hence
>> there is no container. For instance, range(10**6) doesn't hold any
>> kind of object, there are no reference to the int objects 0, 1, 2,
>> ... As the range's docstring explains, range returns a VIRTUAL
>> sequence.
>>
> It's a virtual, read-only container that contains integers.
>
> Try comparing range(10) with tuple(range(10)). Both contain integers.
> Both have a length. Both can be indexed.

Another way to look at range is as an optimization of the range from
Python 2, which returned an actual list of integers. Most callers are
only interested in iterating over the container and don't have to be
concerned whether the result is actually a list or just something that
looks enough like one for their purposes. From that point of view, a
range object is still a container.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: PythonMagick on Windows DLL load failed

[Terry Reedy]

On 2/16/2015 7:17 AM, julien levasseur wrote:

I am using Python 2.7 on Windows 8.1.

Python 2.7 (r27:82525, Jul 4 2010, 07:43:08) [MSC v.1500 64 bit (AMD64)]
on win32


I *strongly* suggest that you update to 2.7.9.


I installed ImageMagick from imagemagick.org


Then installed PythonMagick with pip from
http://www.lfd.uci.edu/~gohlke/pythonlibs/#pythonmagick


When I have used that site (most recently last July), I have directly 
downloaded and unzipped files.  I see that nearly all .zips have

since been replaced with .whl wheels.


When I import PythonMagick it says:

Traceback (most recent call last):

   File "magic.py", line 2, in 

 import PythonMagick

   File "C:\Python27\lib\site-packages\PythonMagick\__init__.py", line
1, in 

 from . import _PythonMagick

ImportError: DLL load failed: The application has failed to start
because its side-by-side configuration is incorrect. Please see the
application event log or use the command-line sxstrace.exe tool for more
detail.


With 64 bit python, you must use 64 bit binaries.  If so, search 
"side-by-side configuration is incorrect" (Google, etc).



Any clues on where do look?
Question on stackoverflow
http://stackoverflow.com/questions/28538973/pythonmagick-on-windows-dll-load-failed


I see 'cgohlke', who I presume is the C. Gohlke who made the 
PythonMagick binary, has already given you his best guess.


--
Terry Jan Reedy


--
https://mail.python.org/mailman/listinfo/python-list

Re: Issues pip-installing pywin32

[Terry Reedy]

On 2/18/2015 7:13 AM, Chris Angelico wrote:

(Before I begin, I want to make it clear that I can still go poke
around on SourceForge and grab the appropriate installer [1], and that
does work. But if I'm going to tell someone else how to set up this
program, I'd much rather be able to recommend pip.)

On a fresh Python 3.4 running on Windows 7, attempting to "pip install
pywin32" doesn't work:

C:\Users\Rosuav>pip install pywin32
Downloading/unpacking pywin32
   Could not find any downloads that satisfy the requirement pywin32
   Some externally hosted files were ignored (use --allow-external
pywin32 to allow).
Cleaning up...
No distributions at all found for pywin32
Storing debug log for failure in C:\Users\Rosuav\pip\pip.log

C:\Users\Rosuav>pip install --allow-external pywin32
You must give at least one requirement to install (see "pip help install")


Okay, so the obvious command doesn't work, and gives a
not-particularly-helpful error message, but I know how to Google, I
can get past this.


C:\Users\Rosuav>pip install --allow-external pywin32
--allow-unverified pywin32 pywin32
Downloading/unpacking pywin32
   Could not find any downloads that satisfy the requirement pywin32
Cleaning up...
No distributions at all found for pywin32
Storing debug log for failure in C:\Users\Rosuav\pip\pip.log


Huh. The PyPI categorization seems to exclude Python 3.4 support


I have pip-installed 3.4 packages just fine.


(despite versions for both 3.4 and 3.5 existing on the sourceforge
page). Let's try that with 2.7, just to see what happens. (I could
alternatively backlevel to 3.3, I suppose.)

C:\Users\Rosuav>\python27\python -m pip --version
pip 1.5.6 from C:\python27\lib\site-packages (python 2.7)

C:\Users\Rosuav>\python27\python -m pip install --allow-external
pywin32 --allow-unverified pywin32 pywin32
Downloading/unpacking pywin32
   Could not find any downloads that satisfy the requirement pywin32
Cleaning up...
No distributions at all found for pywin32
Storing debug log for failure in C:\Users\Rosuav\pip\pip.log


The log contains these lines, which may be a clue as to what's going on...

   Could not fetch URL
https://sourceforge.net/projects/pywin32/files/pywin32/ (from
https://pypi.python.org/simple/pywin32/): connection error: hostname
'sourceforge.net' doesn't match either of 'cloudfront.net',
'*.cloudfront.net'
   Will skip URL
https://sourceforge.net/projects/pywin32/files/pywin32/ when looking
for download links for pywin32
   Could not find any downloads that satisfy the requirement pywin32

Does this mean there's an HTTPS configuration error at sourceforge?
And if that is indeed the problem, is there any way that I, as an end
user, can bypass the check? My alternative is using a non-SSL link, so
it's not like I'd actually have worse security.

At this point, I am REALLY glad my Dad asked me to set this all up for
him, rather than just giving him some instructions and saying "go for
it". (And to think, all I want is a simple program to bounce some info
out from a Windows VM onto the Linux host. So much hassle for
something so simple.)

[1] http://sourceforge.net/projects/pywin32/files/pywin32/


I would send your experience to Mark Hammond to let him know that 
pythonwin does not seem to be pip-installable.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: Assigning a function to sys.excepthook doesn't work in WSGI

[Alexander Sh]

On Wednesday, February 18, 2015 at 7:52:19 PM UTC+3, Ian wrote:
> 
> sys.excepthook is called just before the interpreter exits due to an
> exception. In a mod_wsgi environment, having the interpreter exit just
> because of an exception would be undesirable. I don't know exactly
> what it's doing under the hood, but I would assume that the exception
> never makes it to sys.excepthook because the gateway itself is
> catching the exception in order to generate the 500 response.
> 
> > I looked through the documentation, but unable to find the answer. Are 
> > there any ways to handle uncaught by try..except exceptions under mod_wsgi?
> 
> Here is what PEP  has to say about error handling:
> https://www.python.org/dev/peps/pep-/#error-handling

Thank you for your reply, it clarifies everything. 
Actually, I've missed that PEP in my studies.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[janhein . vanderburg]

Op woensdag 18 februari 2015 10:36:37 UTC+1 schreef Chris Angelico:
> I would actually look at it the other way:
I'm aware of that, since you already warned me with "This is a fine forum to 
ask in. However, you may find that the advice you get isn't quite what you were 
asking for. In my case, ..."
Again: no offense, but I agree to disagree.

There will be no assumptions about the integers to be encoded.

Jan-Hein.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What the Pythons docs means by "container" ?

[Ethan Furman]

On 02/17/2015 06:14 PM, candide wrote:
> Le mercredi 18 février 2015 01:50:16 UTC+1, Chris Angelico a écrit :
> 
>>
>> So, what's a container? It's a thing that you put other objects into.
> 
> I agree with this approach. The important point to consider here is the last 
> word in your definition : "into".

You are getting bogged down with implementation details; not even dict, list, 
nor tuple actually have the contained
objects inside their memory space.  What they have is a record of what objects 
they "contain", and methods to work with
those objects.

Whether a "contained" object exists before it is accessed is irrelevant, is an 
implementation detail, and is a level of
optimization.

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[janhein . vanderburg]

Op woensdag 18 februari 2015 11:33:18 UTC+1 schreef Laura Creighton:
> Hi Jan.
Hi Laura, thanks for your comments; let me explain my why:


> Should you ever need an assembler programmer for
> quick and dirty hacks for the PDP-11 line (11/20 and 11/05 preferred
> as it is harder) I am still the woman for the job.  Indeed, I spent
> most of my 20s finding better and better ways to fit programs into
> smaller and smaller memory footprints.
I once wrote a bootstrap loader for a PDP-11, but I forget when and what model 
exactly, for it was some decades ago and not all that spectacular.

> I perfectly understand the intellectual thrill of doing such things.
> As puzzles go, it is about as cool a one as exists, and its all for
> things that matter -- for real.
> 
> However, in the matter of financial compensation and world recognition,
> you have just laid a very large goose-egg.
I don't want financial compensation and don't need world recognition, so I will 
accept the egg.

> so all the techniques we learned for making our code concise -- let alone
> the dirty tricks I specialised in -- were no longer relevant.
If dirty tricks means shortcuts I personally wouldn't have excepted them even 
at that time of resource shortages.
I hate shortcuts.

> >From the mid 1980s onward I have been telling people 'your code is
> ugly, please tighten it up by refactoring it  and ' and
> when I am their instructor they grumble and do it, and otherwise they
> flip me the bird.  In their eyes, it doesn't matter how the code
> _looks_ as long as it does the job.
Do not get intimidated and simply refuse to follow the lemmings.

> what I am going for is not a 'death - by looking unfashionable' but
> rather a demand that good code is clear to understand.
Sorry that my code is all that confusing.
I have had several comparable complaints from other contributors to this 
discussion.
I will try to improve that part of my proposal, but don't hold your breath, for 
it may take some time.
In fact I was initially asking for help with that code, but I ended up 
defending my credentials.

> Your proposed encoding scheme (if it does as you say, I have not
> analysed it) scores very, very high in the _cleverness_ department.
> Enough that a lot of people, who aren't as clever as you do, have
> no hope in hell of ever being clever enough to debug something that
> uses it.  Therefore, you will never see widespread adoption of your
> scheme, no matter how brilliantly it does as you say, because we all
> need things that are easier to debug more than we need better compression.
I don't need widespread adaption outside my own applications, and I don't see 
any problem for a python programmer that uses an object that flawlessly encodes 
and decodes an arbitrary integer value.
To me debugging is not synonymous to making things human readable or even 
translatable.

So what's in it for other programmers then me myself and I?: the no brainer 
optimizing of resource requirements.

Compression is different from optimal individual integer encoding, since the 
first needs a context of surrounding  integers in a sequence of integer images.

> So now you are sad.
Not yet.

> So then, now what?
> 
> If you are still fired up with the desire to compress things, then
> there is a huge, _very well paying_ market I want to introduce you
> to.  And this is _tech support for porn sites_.  Porn sites make a
> ton of money, indeed the numbers are scary.  And here the idea of
> 'I saved 2% of time/bandwidth/disk space/' really matters.  You
> can really save money for them, and it really matters to them.
> Since I have never found sex 'dirty' and indeed consider it one
> of the great joys in life, I have never found anything wrong with
> working for porn sites.
Frankly a have a first application in mind that is less sexy and won't be all 
that profitable.
There is nothing wrong with porn indeed, but at our age it is just no longer on 
the top of one's mind every second is it?

> And, hell, out of male porn Jimmy Wales made wikipedia.  Haven't
> we all benefitted?
I really don't know what you are talking about here, and only interested in a 
reply if it seriously contributes to my  personal objective to get "my" way of 
integer encoding implemented as proposed.

> But, right now, you are, alas for you, about 45 years too late for the
> ideas you are sprouting.  I had similar ones about 30 years too late
> and, well, they only worked for me for about 3-5 years.  Sucks to be
> you, friend -- you needed to be your grandfather, I fear.

Thanks for making me feel so mature.
Let's go back to the future.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[janhein . vanderburg]

Op woensdag 18 februari 2015 14:55:07 UTC+1 schreef Dave Angel:
> Define "beats."  You might mean beats in simplicity, or in elegance, or 
> in clarity of code.  But you probably mean in space efficiency, or 
> "compression."  But that's meaningless without a target distribution of 
> values that you expect to encode.
I do indeed mean space efficiency, not compression.
And it  is not meaningless and I don't do distribution tricks.
Don't take this personally.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[janhein . vanderburg]

Op woensdag 18 februari 2015 17:47:49 UTC+1 schreef Dave Angel:
> On 02/18/2015 03:59 AM, janhein.vanderb...@gmail.com wrote:
> 
> 
> > encoding individual integers optimally without any assumptions about their 
> > values.
> >
> 
> Contradiction in terms.
> 
> -- 
> DaveA

Not.
Jan-Hein.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What the Pythons docs means by "container" ?

[Terry Reedy]

On 2/18/2015 2:43 PM, Ethan Furman wrote:


Whether a "contained" object exists before it is accessed is irrelevant, is an 
implementation detail, and is a level of
optimization.


Is in not irrelevant in that virtual collections can be infinite.

--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[Ben Finney]

Johannes Bauer  writes:

> On 18.02.2015 08:05, Chris Angelico wrote:
>
> > But if you need more facilities than SQLite3 can offer, maybe it's
> > time to move up to a full database server, instead of local files.
> > Switching to PostgreSQL will give you all those kinds of features,
> > plus a lot of other things that I would have thought pretty basic -
> > like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't
> > support that.
>
> I see you're running a lawnmower. Maybe you should switch to a combine
> harvester. That'll get you extra features like a reciprocating knife
> cutter bar. I was quite surprised that regular lawnmowers don't support
> those.

Chris has pointed out one flaw in this analogy; I'll address another.

A feature like ‘ALTER TABLE’ is not equivalent to a “reciprocating knife
cutter bar”. I'm in agreement that it is a pretty basic SQL feature, and
it doesn't appear to conflict with the narrow focus that we all agree is
appropriate for SQLite.

So you're mocking such an expectation as though it's expecting something
wildly niche. I think you're propping up a straw man there; the
expectation is quite simple and its absence from SQLite is astonishing.
Your attempted mockery does not, IMO, hit home.

-- 
 \  “When we call others dogmatic, what we really object to is |
  `\   their holding dogmas that are different from our own.” —Charles |
_o__)   Issawi |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Logging with Custom Levels not working

[Ian Kelly]

On Wed, Feb 18, 2015 at 6:49 AM, Didymus  wrote:
> def perror(self, message, *args, **kws):
> """ Performance Error Message Level """
> # Yes, logger takes its '*args' as 'args'.
> self._log(PERROR_NUM, message, args, **kws)
>
> logging.Logger.perror = perror

I think you need to call self.log, not self._log. The _log method
appears to assume that the level check has already been performed. You
really shouldn't be calling it directly anyway, as the leading _ is an
indication that the method is not part of the public API.
-- 
https://mail.python.org/mailman/listinfo/python-list

can python handle CHIME .spt files?

[James Zimmerman]

I have a lot of old files that were written in Chime, then made compatible with 
Jmol.  Now there are problems with that.  Will Python give me a way to be able 
to show the .spt files that  Chime could read?  I don't want to start learning 
Python unless I know there is a way.

Thank you very much.

James Zimmerman
Emeritus Professor of Biochemistry
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: can python handle CHIME .spt files?

[Davin Potts]

Hi James —

I’m not entirely sure I understand what you’re wanting to do, but my ears 
perked up when you mentioned Jmol and Chime so I’ll try to offer some pointers:

* If you want to assess which .spt files are failing when trying to read them 
with Jmol, then yes, Python can be used to automate the feeding of files to 
Jmol and can capture whether the loading is successful or not.

* If you want a way to parse problematic .spt files that can no longer be read 
in using Jmol or MDL Chime, it is possible to create an .spt parser in Python 
capable of reading your files though my quick searches for an existing .spt 
parser library in Python came up empty.  If this is your goal (to read those 
problematic .spt files), have you considered using OpenBabel to read them and 
convert them to another mainstream chemical structure format (SDF, MOL, etc.), 
cleaned up and ready for reading by any of a variety of other tools?


You may already be aware but just in case it isn’t evident, Python itself does 
not ship with capabilities for reading/writing chemical structure formats but 
there are lots of chemistry tools built on top of Python that do.  You may well 
find it most productive to pick up one of those tools in lieu of Python itself.


Hope this helps,


Davin



> On Feb 18, 2015, at 2:56 PM, James Zimmerman  wrote:
> 
> I have a lot of old files that were written in Chime, then made compatible 
> with Jmol.  Now there are problems with that.  Will Python give me a way to 
> be able to show the .spt files that  Chime could read?  I don’t want to start 
> learning Python unless I know there is a way.
>  
> Thank you very much.
>  
> James Zimmerman
> Emeritus Professor of Biochemistry
> -- 
> https://mail.python.org/mailman/listinfo/python-list 
> 
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Marko Rauhamaa]

janhein.vanderb...@gmail.com:

> Op woensdag 18 februari 2015 17:47:49 UTC+1 schreef Dave Angel:
>> On 02/18/2015 03:59 AM, janhein.vanderb...@gmail.com wrote:
>> > encoding individual integers optimally without any assumptions
>> > about their values.
>> 
>> Contradiction in terms.
>
> Not.

Out of curiosity, could you give me an example of an integer, not
assuming anything about its value.

I mean, any integer you could mention would be very close to zero
compared with virtually all other integers.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: can python handle CHIME .spt files?

[Ethan Furman]

On 02/18/2015 12:56 PM, James Zimmerman wrote:
>
> I have a lot of old files that were written in Chime, then made compatible 
> with Jmol.
> Now there are problems with that.  Will Python give me a way to be able to 
> show the
> .spt files that  Chime could read?  I don’t want to start learning Python 
> unless I
> know there is a way.

This isn't my field, but I found:

  
http://higheredbcs.wiley.com/legacy/college/voet/0470129301/bioinfo_stu/sect4.html

Some of the nice things about Python is it's fairly easy to learn and has good 
community support.

Good luck!

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: can python handle CHIME .spt files?

[Ethan Furman]

On 02/18/2015 01:55 PM, Davin Potts wrote:

> * If you want a way to parse problematic .spt files that can no longer be 
> read in using Jmol or MDL Chime, it is
> possible to create an .spt parser in Python capable of reading your files 
> though my quick searches for an existing .spt
> parser library in Python came up empty.

It wasn't easy to find, but the previous link I posted did lead to this:

  http://www.pymol.org/

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Marko Rauhamaa]

Marko Rauhamaa :

> Out of curiosity, could you give me an example of an integer, not
> assuming anything about its value.
>
> I mean, any integer you could mention would be very close to zero
> compared with virtually all other integers.

And I don't mean to be snide, either. I'm just saying that your
"optimal encoding" necessarily depends on the expected distribution of
the integers. There is no such thing as an even distribution of
integers.


Marko
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Dave Angel]

On 02/18/2015 02:55 PM, janhein.vanderb...@gmail.com wrote:

Op woensdag 18 februari 2015 17:47:49 UTC+1 schreef Dave Angel:

On 02/18/2015 03:59 AM, janhein.vanderb...@gmail.com wrote:



encoding individual integers optimally without any assumptions about their 
values.



Contradiction in terms.

--
DaveA


Not.
Jan-Hein.



Then you had better define your new word "optimal" to us.  I decided to 
try your algorithm for all the values between 0 and 99.  One million 
values, and the 7bit encoding[1] beat yours for 950081 of them.  The 
rest were tied.  Yours never was shorter.


For a uniform distribution of those particular values, the average 
number of bytes used by yours was 3.933568 bytes, and by 7bit encoding 
was 2.983487


For the second and third million, yours are all 4 bytes, while 7bit uses 
3. Beyond 2097152, 7bit uses 4 bytes, same as you.  Between 16 and 17 
million, you average 4.156865, while 7bit is a constant 4.0.


After that, I started spot-checking.  I went up to 100 billion, and for 
none of those I tried did your algorithm take fewer bytes than 7bit.



So how is yours optimal?  Over what range of values?

I'm not necessarily doubting it, just challenging you to provide a data 
sample that actually shows it.  And of course, I'm not claiming that 
7bit is in any way optimal.  You cannot define optimal without first 
defining the distribution.


[1] by 7bit, I'm referring to the one apparently used in MIDI encoding, 
where 7 bits of each byte hold the value, and the 8th bit is zero, 
except for the last byte, where the 8th bit is one.  So 3 bytes can 
encode 21 bits, or up to 2**21 - 1.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[rurpy]

On 02/18/2015 01:14 PM, Ben Finney wrote:
> Johannes Bauer  writes:
>> On 18.02.2015 08:05, Chris Angelico wrote:
>>
>>> But if you need more facilities than SQLite3 can offer, maybe it's
>>> time to move up to a full database server, instead of local files.
>>> Switching to PostgreSQL will give you all those kinds of features,
>>> plus a lot of other things that I would have thought pretty basic -
>>> like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't
>>> support that.
>>
>> I see you're running a lawnmower. Maybe you should switch to a combine
>> harvester. That'll get you extra features like a reciprocating knife
>> cutter bar. I was quite surprised that regular lawnmowers don't support
>> those.
> 
> Chris has pointed out one flaw in this analogy; I'll address another.
> 
> A feature like 'ALTER TABLE' is not equivalent to a "reciprocating knife
> cutter bar". I'm in agreement that it is a pretty basic SQL feature, and
> it doesn't appear to conflict with the narrow focus that we all agree is
> appropriate for SQLite.

No, you and Chris are way off base and Johannes is correct.  
He was pointing out that there are many applications that can 
benefit from a database and a full-blown, bells and whistles 
solution like Postgresql is often overkill in that (very common)
case.  His analogy is quite apt and I wish I'd thought of it. 

> So you're mocking such an expectation as though it's expecting something
> wildly niche. I think you're propping up a straw man there; the
> expectation is quite simple and its absence from SQLite is astonishing.
> Your attempted mockery does not, IMO, hit home.

It has already been pointed out that Sqlite *does* have ALTER 
TABLE so you are either deliberately propagating false information 
or not paying attention.

The only thing basic that a (claimed) relational database needs 
to support is the basic relational operations of which ALTER 
TABLE is not one,  And if one insists on ALTER TALE the minimal
requirement there is that it support RENAME.  You and Chris are
confusing convenience with necessity.

The success and wide adoption of Sqlite is pretty clear evidence 
that there are many applications that can benefit from its use.
Sqllite gets used because of a very important feature that 
Postgresql is missing -- the ability to embed a database in an 
application without requiring the installation and maintenance 
of an entire separate client-server infrastructure.  If you are 
tempted to trivialize that feature you might want to scan the
Postgresql mail list where that feature is regularly requested 
for Postgresql.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[rurpy]

On 02/18/2015 04:21 AM, Chris Angelico wrote:
> On Wed, Feb 18, 2015 at 10:11 PM, Johannes Bauer  wrote:
>> On 18.02.2015 08:05, Chris Angelico wrote:
>>
>>> But if you need more facilities than SQLite3 can offer, maybe it's
>>> time to move up to a full database server, instead of local files.
>>> Switching to PostgreSQL will give you all those kinds of features,
>>> plus a lot of other things that I would have thought pretty basic -
>>> like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't
>>> support that.
>>
>> I see you're running a lawnmower. Maybe you should switch to a combine
>> harvester. That'll get you extra features like a reciprocating knife
>> cutter bar. I was quite surprised that regular lawnmowers don't support
>> those.
> 
> SQLite3 is fine for something that's basically just a more structured
> version of a flat file. You assume that nobody but you has the file
> open, and you manipulate it just the same as if it were a big fat blob
> of JSON, but thanks to SQLite, you don't have to rewrite the whole
> file every time you make a small change. That's fine.

That's bullshit.  Sqlite offers a lot more than that including
a SQL interface, transactions, referential integrity, constraints 
indexes, triggers and other general relational database features.  

That you would equate that to a JSON blob would indicate either 
a profound ignorance about Sqlite or (more likely) a need to
defend your preference with complete disregard of fact. 

> But it's the
> wrong tool for any job involving multiple users over a network, and
> quite probably the wrong tool for a lot of other jobs too. 

Nobody disputes that nor was that the point.  The point was 
that there are many applications that can benefit from use 
of a database that are NOT distributed multi-user, muilti-tier
applications.  For many of that very large class of applications 
Sqlite is a good (and preferable to Postgresql) solution.

> It's the
> smallest-end piece of software that can truly be called a database. I
> would consider it to be the wrong database for serious accounting
> work, and that's based on the ranting of a majorly-annoyed accountant
> who had to deal with issues in professional systems that had made
> similar choices in back-end selection.

I consider the program I use for my personal accounting program 
to be for very serious use since errors could have very grave
consequences for me.  But a multi-user client-server database 
is emphatically not needed by it.

And I'm sure you're aware that "not for serious use" is a common 
way that C and Java programmers dismiss Python?  So maybe you 
should try a little harder to come up with real arguments and 
not rely on cheap and meaningless labels.

> You're welcome to disagree, but since PostgreSQL doesn't cost any
> money and (on Linux at least; can't speak for other platforms) doesn't
> take significant effort to set up, I will continue to recommend it.

Postgresql costs a *lot* more -- in setup and on-going maintenance.
Not all (or even most) costs are the initial monetary purchase expense.

Its an unmoderated newsgroup so feel free to recommend what you
want.  But counter opinions should be aired so that others a
can judge for themselves whether your recommendations are based 
on facts or on your personal preferences.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[Chris Angelico]

On Thu, Feb 19, 2015 at 9:17 AM,   wrote:
>> SQLite3 is fine for something that's basically just a more structured
>> version of a flat file. You assume that nobody but you has the file
>> open, and you manipulate it just the same as if it were a big fat blob
>> of JSON, but thanks to SQLite, you don't have to rewrite the whole
>> file every time you make a small change. That's fine.
>
> That's bullshit.  Sqlite offers a lot more than that including
> a SQL interface, transactions, referential integrity, constraints
> indexes, triggers and other general relational database features.
>
> That you would equate that to a JSON blob would indicate either
> a profound ignorance about Sqlite or (more likely) a need to
> defend your preference with complete disregard of fact.

I didn't equate them. I said that SQLite3 is great if you look on it
as an upgrade over a JSON blob. Of course it offers more features than
that, and you don't need to swear at me to make your point.

But SQLite3 is *not* great if you look on it as a database engine
comparable with DB2, PostgreSQL, and even MySQL.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[Ethan Furman]

On Thu, Feb 19, 2015 at 9:17 AM,  rurpy wrote:
> That you would equate that to a JSON blob [...]

Chris wrote:
> I didn't equate them.

>> Chris wrote earlier:
>>> and you manipulate it just the same as if it were a big fat blob
>>> of JSON

That sure sounds like equating.

Chris also wrote:
> But SQLite3 is *not* great if you look on it as a database engine
> comparable with DB2, PostgreSQL, and even MySQL.

Sure, the LITE in SQLite means you don't get some things.  There is still a 
huge amount of software that doesn't need
concurrency and can benefit from it.

Having installed Postgres I can say there is definitely a cost to install it, 
use it, maintain it, etc... especially if
you aren't steeped in it and have to look things up every time you have to make 
a change (how do I add a user again?).

I think the general advice should be:  if you are writing a single-user 
application that happens to need SQL services,
check out SQLite; if you are writing a multi-user or concurrent SQL 
application, check out Postgres.

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Issues pip-installing pywin32

[Chris Angelico]

On Thu, Feb 19, 2015 at 6:01 AM, Terry Reedy  wrote:
> On 2/18/2015 7:13 AM, Chris Angelico wrote:
>> Huh. The PyPI categorization seems to exclude Python 3.4 support
>
>
> I have pip-installed 3.4 packages just fine.
>
>
>> (despite versions for both 3.4 and 3.5 existing on the sourceforge
>> page). Let's try that with 2.7, just to see what happens. (I could
>> alternatively backlevel to 3.3, I suppose.)

I know pip works with 3.4 generally; I was talking specifically about
pywin32 - if you look it up on PyPI, it doesn't mention 3.4 support.
But it does mention 2.7, which is why I then tried that.

> I would send your experience to Mark Hammond to let him know that pythonwin
> does not seem to be pip-installable.

Thanks, will do.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Grant Edwards]

On 2015-02-18, Marko Rauhamaa  wrote:
> janhein.vanderb...@gmail.com:
>
>> Op woensdag 18 februari 2015 17:47:49 UTC+1 schreef Dave Angel:
>>> On 02/18/2015 03:59 AM, janhein.vanderb...@gmail.com wrote:
>>> > encoding individual integers optimally without any assumptions
>>> > about their values.
>>> 
>>> Contradiction in terms.
>>
>> Not.
>
> Out of curiosity, could you give me an example of an integer, not
> assuming anything about its value.
>
> I mean, any integer you could mention would be very close to zero
> compared with virtually all other integers.

Thus demonstrating the fitness of my previously posted "lossy integer
encoding" algorithm.

-- 
Grant Edwards   grant.b.edwardsYow! Hmmm ... an arrogant
  at   bouquet with a subtle
  gmail.comsuggestion of POLYVINYL
   CHLORIDE ...
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What the Pythons docs means by "container" ?

[Steven D'Aprano]

Terry Reedy wrote:

> On 2/18/2015 2:43 PM, Ethan Furman wrote:
> 
>> Whether a "contained" object exists before it is accessed is irrelevant,
>> is an implementation detail, and is a level of optimization.
> 
> Is in not irrelevant in that virtual collections can be infinite.

Hmmm. I'm not sure I believe that. Can you enumerate all of the items and
see if there actually is an infinite number of them? I'll wait... 

*wink*

But seriously... Ethan's comment shouldn't be interpreted as meaning that
virtual containers are exactly the same as concrete ones. Ethan's comment
should be understand in terms that whether something is a container or not
is *not* connected to whether it is a concrete data structure like a list
or an array, or a virtual/lazy/on-demand data structure like (x)range.

In Python ABC terms, I believe that the, or at least a, defining
characteristic of containers is that they implement the "in" operator. If
there is some value X for which `X in OBJ` succeeds, then OBJ may be a
container.




-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[Steven D'Aprano]

ru...@yahoo.com wrote:

> On 02/18/2015 01:14 PM, Ben Finney wrote:
>> Johannes Bauer  writes:
>>> On 18.02.2015 08:05, Chris Angelico wrote:
>>>
 But if you need more facilities than SQLite3 can offer, maybe it's
 time to move up to a full database server, instead of local files.
 Switching to PostgreSQL will give you all those kinds of features,
 plus a lot of other things that I would have thought pretty basic -
 like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't
 support that.
>>>
>>> I see you're running a lawnmower. Maybe you should switch to a combine
>>> harvester. That'll get you extra features like a reciprocating knife
>>> cutter bar. I was quite surprised that regular lawnmowers don't support
>>> those.
>> 
>> Chris has pointed out one flaw in this analogy; I'll address another.
>> 
>> A feature like 'ALTER TABLE' is not equivalent to a "reciprocating knife
>> cutter bar". I'm in agreement that it is a pretty basic SQL feature, and
>> it doesn't appear to conflict with the narrow focus that we all agree is
>> appropriate for SQLite.
> 
> No, you and Chris are way off base and Johannes is correct.
> He was pointing out that there are many applications that can
> benefit from a database and a full-blown, bells and whistles
> solution like Postgresql is often overkill in that (very common)
> case.  His analogy is quite apt and I wish I'd thought of it.


I'm not seeing that at all. Chris explicitly proceeded his comments with the
condition "if you need more facilities than SQLite3 can offer". Johannes'
analogy ignores that and consequently mocks the very idea that anyone might
need more than a regular lawmower -- even a farmer with a thousand acres of
wheat to be harvested.

Johannes' subsequent posts are more nuanced about Sqlite filling a niche and
not being suitable for everything, but the analogy you're supporting
doesn't. It's an amusing quip, quite funny, but like most quips, lacks
nuance and misrepresents Chris' original position.

Had Chris said, "SQlite? Pah, don't use that, Postgresql is a much better
solution!", the combine harvester analogy would have been much more fair.
But he didn't, so it isn't. But fair or not, it has inspired good
discussion, so there is that in it's favour.



-- 
Steven

-- 
https://mail.python.org/mailman/listinfo/python-list

'Lite' Databases (Re: sqlite3 and dates)

[memilanuk]

On 02/18/2015 02:52 PM, Ethan Furman wrote:


Chris also wrote:

But SQLite3 is *not* great if you look on it as a database engine
comparable with DB2, PostgreSQL, and even MySQL.


Sure, the LITE in SQLite means you don't get some things.  There is still a 
huge amount of software that doesn't need
concurrency and can benefit from it.

Having installed Postgres I can say there is definitely a cost to install it, 
use it, maintain it, etc... especially if
you aren't steeped in it and have to look things up every time you have to make 
a change (how do I add a user again?).

I think the general advice should be:  if you are writing a single-user 
application that happens to need SQL services,
check out SQLite; if you are writing a multi-user or concurrent SQL 
application, check out Postgres.


Okay... this might be a question with a blindingly obvious answer, but I 
haven't seen any recommendations otherwise so I'll ask anyway ;)


Is there anything *good* that sits in between the two extremes of SQLite 
and PostgreSQL?


I've tinkered with MySQL years ago (in conjunction with PHP) and was a 
little unhappy with some of the things it either didn't implement fully 
(foreign keys) or silently ignored (check constraints).  PostgreSQL, to 
me, is orders of magnitude harder to set up and maintain, though.  And 
then there is SQLite, which does 99% of what I want it to do other than 
network use.  I see other DB names such as DB2, Oracle, MS SQL Server, 
etc. out there but the only other 'free' one seems to be Firebird?  Is 
that really the only other contender?  Is there nothing that amounts to 
a 'PostgreSQLite'?



--
Shiny!  Let's be bad guys.

Reach me @ memilanuk (at) gmail dot com

--
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Ben Finney]

memilanuk  writes:

> Okay... this might be a question with a blindingly obvious answer, but
> I haven't seen any recommendations otherwise so I'll ask anyway ;)
>
> Is there anything *good* that sits in between the two extremes of
> SQLite and PostgreSQL?

What do you need a RDBMS to do, and what do you not need?

The answers to those questions vary hugely between different people (and
most people probably don't think too deeply about them). They will
determine what “good” means for your case.

> Is there nothing that amounts to a 'PostgreSQLite'?

PostgreSQL itself fits that mould quite well; it is quite capable of
serving a small footprint while still offering full concurrency.

I don't know of a free-software concurrent RDBMS which can be considered
lighter than that. (No, MySQL doesn't count; its concurrency is
*unreliable* and it commonly loses data silently. Don't use MySQL.)

But perhaps you don't need concurrency? Only you can tell us.

-- 
 \  “It's up to the masses to distribute [music] however they want |
  `\… The laws don't matter at that point. People sharing music in |
_o__)their bedrooms is the new radio.” —Neil Young, 2008-05-06 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Mario Figueiredo]

On Wed, 18 Feb 2015 15:32:36 -0800, memilanuk 
wrote:

>
>Is there anything *good* that sits in between the two extremes of SQLite 
>and PostgreSQL?
>
>I've tinkered with MySQL years ago (in conjunction with PHP) and was a 
>little unhappy with some of the things

MariaDB is backwards compatible with MySQL and may answer some of the
shortcomings. It's a much stronger RDBM in my opinion than MySQL and
offers enterprise level features at cost 0.


> PostgreSQL, to me, is orders of magnitude harder to set up and
> maintain, though.

PostgreSQL grows on you. It takes time to mature into a love
relationship, like a complicated girlfriend (or boyfriend, whatever
floats your boat). But once that relationship grows, you will want to
marry with it, have little postgre kids and grow old with it. No other
database stands a chance from that moment on.

It's just too powerful and too feature rich, to ignore. Only Oracle
stands a chance against it, in my humble opinion.

And postgre isn't really that hard to setup and maintain. In fact,
maintenance can be largely scriptable and 'croned' because the postgre
server is so damn stable. Once you familiarize yourself with the
process, you just realize it was easy all the time after all.

I usually think of my relationship with postgre as similar to what I
experienced with Git. At first I was just dumbstruck by the whole
thing and my first reaction was to ignore it and just do version
control as I knew with the tools I knew. But once my brain clicked
into 'Git mode' and I realized its philosophy and its processes, I
immediately recognized the benefits and understood why everyone was
telling me Git was easy to use and highly useful.

>then there is SQLite, which does 99% of what I want it to do other than 
>network use.

SQLite misses some important features that makes it better suited as a
simple datastore, not much unlike shelve. And network use is not one
of them, since you can actually implement concurrent sqlite access by
coding an intermediate layer. Assuming of course we are talking about
a small number of concurrent users.

Stored procedures is perhaps the most obvious missing feature.
Contrary to an opinion I read on the thread that spawned this one, you
really should thrive to put the business logic into the database as
this permits great simplification of your code and much better
adaptability to new requirements. SQLite IS a database. And wants to
be used as a database. So despite agreeing SPs would increase SQLite
footprint, it's undeniable they could be put to good use. Admittedly
these too can be implemented through an intermediate layer. But are
much more complex to code.

Parameterized queries is just a pet peeve of mine that I wish to
include here. SQLite misses it and I miss the fact SQLite misses it.
The less SQL one needs to write in their code, the happier one should
be.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What the Pythons docs means by "container" ?

[Ben Finney]

Steven D'Aprano  writes:

> Terry Reedy wrote:
>
> > […] virtual collections can be infinite.
>
> Hmmm. I'm not sure I believe that. Can you enumerate all of the items
> and see if there actually is an infinite number of them? I'll wait...

A mathematician should know better: you don't need to enumerate to
demonstrate a collection is infinite.

Provided, of course, that the collection is *virtual* — so virtual that
it's not actually implemented on hardware :-)

-- 
 \“If you have the facts on your side, pound the facts. If you |
  `\ have the law on your side, pound the law. If you have neither |
_o__)   on your side, pound the table.” —anonymous |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What the Pythons docs means by "container" ?

[Chris Angelico]

On Thu, Feb 19, 2015 at 11:05 AM, Ben Finney  wrote:
>> Hmmm. I'm not sure I believe that. Can you enumerate all of the items
>> and see if there actually is an infinite number of them? I'll wait...
>
> A mathematician should know better: you don't need to enumerate to
> demonstrate a collection is infinite.

But you may have to demonstrate a 1:1 correspondence with counting
numbers, to prove that it's countably infinite. And that's pretty much
what the enumerate() function does, so that's enumeration right?

Bizarre pseudo-logic for the win.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Chris Angelico]

On Thu, Feb 19, 2015 at 11:08 AM, Mario Figueiredo  wrote:
> I usually think of my relationship with postgre as similar to what I
> experienced with Git. At first I was just dumbstruck by the whole
> thing and my first reaction was to ignore it and just do version
> control as I knew with the tools I knew. But once my brain clicked
> into 'Git mode' and I realized its philosophy and its processes, I
> immediately recognized the benefits and understood why everyone was
> telling me Git was easy to use and highly useful.

(Side point: If you're going to treat PostgreSQL the way you'd treat a
girlfriend/boyfriend, you should probably be careful of how you
address him. "Postgres" or "PostgreSQL", but not usually "Postgre".)

This is a quite apt analogy. You have to get your head around some
fundamentals, but once you do, life becomes amazing.

>>then there is SQLite, which does 99% of what I want it to do other than
>>network use.
>
> SQLite misses some important features that makes it better suited as a
> simple datastore, not much unlike shelve. And network use is not one
> of them, since you can actually implement concurrent sqlite access by
> coding an intermediate layer. Assuming of course we are talking about
> a small number of concurrent users.

This is what I was saying: it's fine for purposes like Firefox's
bookmarks and settings and such (which I think was what it was
originally developed for?). Not so fine over a network.

Adding an intermediate layer is a lot more effort than you might
think. By the time you've gone there, you should be looking at
PostgreSQL anyway. I tried to bolt networking support onto a couple of
different databasing systems, back in the 90s, and it was faintly
ridiculous... I mean, it worked, but if I'd had today's Postgres, I
would never have done anything of the sort.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Steven D'Aprano]

Chris Angelico wrote:

>> SQLite misses some important features that makes it better suited as a
>> simple datastore, not much unlike shelve. And network use is not one
>> of them, since you can actually implement concurrent sqlite access by
>> coding an intermediate layer. Assuming of course we are talking about
>> a small number of concurrent users.
> 
> This is what I was saying: it's fine for purposes like Firefox's
> bookmarks and settings and such (which I think was what it was
> originally developed for?). Not so fine over a network.

The sheer number of Firefox bugs related to its use of SQLite says 
different.

Once upon a time, Firefox's config, bookmarks, etc. were stored in plain 
text files. At worst they were HTML. You could trivially read them, copy 
them, restore them and even (if you were careful) edit them using the text 
editor of your choice. Many a time I was on one machine, wanted to know a 
bookmark from another machine, so I would ssh across to the other machine 
and run grep over the bookmark file.

No more. Firefox still keeps a bookmark HTML file, but it never seems to be 
synced with the actual bookmarks. Settings are stored in an opaque blob, 
rather than human-readable text, limiting what you can do with it. It's very 
nice that Firefox offers about:config but not so nice that you can't do the 
same thing without the GUI running.

If Firefox crashes, there are failure modes where it can no longer read your 
bookmarks, or keep history. I don't mean that history won't persist across 
restarts, I mean that *within a single session* it cannot remember what page 
you came from so you can hit the Back button and return to it. WTF? 

I swear, if not for the fact that every single other browser is worse, I 
would dump Firefox in a second.

I don't believe for a second that moving to SQlite has anything to do with 
performance, because reading and writing preference settings should be rare 
and far from a bottleneck. SQlite is simply fragile and unreliable over a 
network, and people using their home directory on a network drive are not 
that rare.


-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[Steve Hayes]

On Wed, 18 Feb 2015 08:19:25 +0200, "Frank Millman"
 wrote:

>Hi all
>
>sqlite3 does not have a DATE type, but the python module does a pretty good 
>job of providing one -

The Rootsmagic genealogy program uses SQLite for its database, 

I don't know whether or to what extent it uses Python to interac t
with the database, but it seems to do a pretty good job of handling
dates, calculating ages etc. 

http://www.rootsmagic.com/


-- 
Steve Hayes from Tshwane, South Africa
Web:  http://www.khanya.org.za/stevesig.htm
Blog: http://khanya.wordpress.com
E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk
-- 
https://mail.python.org/mailman/listinfo/python-list

Not sqlite3 and dates

[Steve Hayes]

On Wed, 18 Feb 2015 23:14:32 +1100, Chris Angelico 
wrote:

>On Wed, Feb 18, 2015 at 10:57 PM, Johannes Bauer  wrote:
>> SQLite and Postgres are so vastly different in their setup,
>> configuration, capabilities and requirements that the original developer
>> has to have done a MAJOR error in judgement so that a change from one to
>> the other would not be ill-advised.
>
>On Wed, Feb 18, 2015 at 6:49 PM, Frank Millman  wrote:
>> My accounting software supports three databases - MS Sql Server, PostgreSQL,
>> and sqlite3.
>
>Johannes, are you saying that Frank made three major errors of judgement? :)

No, ChrisA did, in answering questions that no one was asking, and
changing the subject of the thread without changing the subject line. 
-- 
Steve Hayes from Tshwane, South Africa
Web:  http://www.khanya.org.za/stevesig.htm
Blog: http://khanya.wordpress.com
E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[Steve Hayes]

On Wed, 18 Feb 2015 22:21:35 +1100, Chris Angelico 
wrote:

>On Wed, Feb 18, 2015 at 10:11 PM, Johannes Bauer  wrote:
>> On 18.02.2015 08:05, Chris Angelico wrote:
>>
>>> But if you need more facilities than SQLite3 can offer, maybe it's
>>> time to move up to a full database server, instead of local files.
>>> Switching to PostgreSQL will give you all those kinds of features,
>>> plus a lot of other things that I would have thought pretty basic -
>>> like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't
>>> support that.
>>
>> I see you're running a lawnmower. Maybe you should switch to a combine
>> harvester. That'll get you extra features like a reciprocating knife
>> cutter bar. I was quite surprised that regular lawnmowers don't support
>> those.
>
>SQLite3 is fine for something that's basically just a more structured
>version of a flat file. You assume that nobody but you has the file
>open, and you manipulate it just the same as if it were a big fat blob
>of JSON, but thanks to SQLite, you don't have to rewrite the whole
>file every time you make a small change. That's fine. But it's the
>wrong tool for any job involving multiple users over a network, and
>quite probably the wrong tool for a lot of other jobs too. It's the
>smallest-end piece of software that can truly be called a database. I
>would consider it to be the wrong database for serious accounting
>work, and that's based on the ranting of a majorly-annoyed accountant
>who had to deal with issues in professional systems that had made
>similar choices in back-end selection.
>
>You're welcome to disagree, but since PostgreSQL doesn't cost any
>money and (on Linux at least; can't speak for other platforms) doesn't
>take significant effort to set up, I will continue to recommend it.

All of which has nothing, absolutely nothing, to do with the OP's
question, which said nothing about number of users, but how the
software handles dates. 




-- 
Steve Hayes from Tshwane, South Africa
Web:  http://www.khanya.org.za/stevesig.htm
Blog: http://khanya.wordpress.com
E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk
-- 
https://mail.python.org/mailman/listinfo/python-list

Not sqlite3 and dates

[Steve Hayes]

On Thu, 19 Feb 2015 09:37:49 +1100, Chris Angelico 
wrote:

>On Thu, Feb 19, 2015 at 9:17 AM,   wrote:
>>> SQLite3 is fine for something that's basically just a more structured
>>> version of a flat file. You assume that nobody but you has the file
>>> open, and you manipulate it just the same as if it were a big fat blob
>>> of JSON, but thanks to SQLite, you don't have to rewrite the whole
>>> file every time you make a small change. That's fine.
>>
>> That's bullshit.  Sqlite offers a lot more than that including
>> a SQL interface, transactions, referential integrity, constraints
>> indexes, triggers and other general relational database features.
>>
>> That you would equate that to a JSON blob would indicate either
>> a profound ignorance about Sqlite or (more likely) a need to
>> defend your preference with complete disregard of fact.
>
>I didn't equate them. I said that SQLite3 is great if you look on it
>as an upgrade over a JSON blob. Of course it offers more features than
>that, and you don't need to swear at me to make your point.
>
>But SQLite3 is *not* great if you look on it as a database engine
>comparable with DB2, PostgreSQL, and even MySQL.

And how does that answer the OP's question?
-- 
Steve Hayes from Tshwane, South Africa
Web:  http://www.khanya.org.za/stevesig.htm
Blog: http://khanya.wordpress.com
E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[memilanuk]

On 02/18/2015 04:03 PM, Ben Finney wrote:


Is there anything *good* that sits in between the two extremes of
SQLite and PostgreSQL?


What do you need a RDBMS to do, and what do you not need?

The answers to those questions vary hugely between different people (and
most people probably don't think too deeply about them). They will
determine what “good” means for your case.


Is there nothing that amounts to a 'PostgreSQLite'?


PostgreSQL itself fits that mould quite well; it is quite capable of
serving a small footprint while still offering full concurrency.

I don't know of a free-software concurrent RDBMS which can be considered
lighter than that. (No, MySQL doesn't count; its concurrency is
*unreliable* and it commonly loses data silently. Don't use MySQL.)

But perhaps you don't need concurrency? Only you can tell us.



At this point... I don't think concurrency is going to be a major 
requirement for what I have in mind.  For one project, only a few people 
will be writing to the DB, and only by a stroke of luck would it be at 
the same time, and it would be very unlikely that they would be 
modifying the same record at the same time due to physical constraints.


For the other... there may be anywhere from 1-10 (maybe more, but 
doubtful) entering data (creating new records for competitors, or 
entering existing competitors in a tournament).  I have a hard time 
picturing that few people stressing a modern computer system enough to 
where SQLite couldn't keep up (thinking web-based interface using Flask 
or something similar).  In the latter case, one of the over-arching 
priorities is that it be easily distributable, as in that people with 
relatively little knowledge of a database be able to set it up and run it.


--
Shiny!  Let's be bad guys.

Reach me @ memilanuk (at) gmail dot com

--
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[Mark Lawrence]

On 19/02/2015 02:48, Steve Hayes wrote:


All of which has nothing, absolutely nothing, to do with the OP's
question, which said nothing about number of users, but how the
software handles dates.



Very true, but charging off like this at massive tangents is one of the 
reasons I love being here.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Mark Lawrence]

On 19/02/2015 02:13, Steven D'Aprano wrote:

Chris Angelico wrote:


SQLite misses some important features that makes it better suited as a
simple datastore, not much unlike shelve. And network use is not one
of them, since you can actually implement concurrent sqlite access by
coding an intermediate layer. Assuming of course we are talking about
a small number of concurrent users.


This is what I was saying: it's fine for purposes like Firefox's
bookmarks and settings and such (which I think was what it was
originally developed for?). Not so fine over a network.


The sheer number of Firefox bugs related to its use of SQLite says
different.

Once upon a time, Firefox's config, bookmarks, etc. were stored in plain
text files. At worst they were HTML. You could trivially read them, copy
them, restore them and even (if you were careful) edit them using the text
editor of your choice. Many a time I was on one machine, wanted to know a
bookmark from another machine, so I would ssh across to the other machine
and run grep over the bookmark file.

No more. Firefox still keeps a bookmark HTML file, but it never seems to be
synced with the actual bookmarks. Settings are stored in an opaque blob,
rather than human-readable text, limiting what you can do with it. It's very
nice that Firefox offers about:config but not so nice that you can't do the
same thing without the GUI running.

If Firefox crashes, there are failure modes where it can no longer read your
bookmarks, or keep history. I don't mean that history won't persist across
restarts, I mean that *within a single session* it cannot remember what page
you came from so you can hit the Back button and return to it. WTF?

I swear, if not for the fact that every single other browser is worse, I
would dump Firefox in a second.



After a wonderful relationship lasting many happy years I dumped Firefox 
a few weeks ago for Chrome.  A few anxious moments gave me pause for 
thought, but overall I'm happy to have changed.  However is anybody 
aware of a "new kid on the block" that could take over as I'd happily 
switch again?  Nothing has sprung out at me, hence the choice I made.


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

VTK Quadratic - vtkContourFilter - vtkSliderRepresentation2D in Python not working properly

[Shalini Ravishankar]

Hello Everyone,

I am new to VTK. I am trying to extract isosurfaces(Contour) from a quadratic 
function with a Slider to change the iso surfaces.


#!/usr/bin/env python

# First, we need to import vtk package in order to access VTK 
classes/functions. 
import vtk 

# create a data source...an implicit function. 
quadric = vtk.vtkQuadric() 
quadric.SetCoefficients(.5, 1, .2, 0, .1, 0, 0, .2, 0, 0) 


def vtkSliderCallback2(obj, event):
sliderRepres = obj.GetRepresentation()
pos = sliderRepres.GetValue()
print "Position ",pos
isosurface.SetValue(0, pos)

# create a filter...a sampling function, which samples an implicit function 
over the x-y-z range 
# although this object is not called "filter" it takes an input and do 
something to/with it 
# and produce an output. 
sample = vtk.vtkSampleFunction() 
sample.SetSampleDimensions(100, 100, 100) 
sample.SetImplicitFunction(quadric) 



outline = vtk.vtkOutlineFilter() 
outline.SetInput( sample.GetOutput() ) 
outlineMapper = vtk.vtkPolyDataMapper() 
outlineMapper.SetInput( outline.GetOutput() )
outlineActor = vtk.vtkActor() 
outlineActor.SetMapper( outlineMapper ) 
outlineActor.GetProperty().SetColor(0.0,0.0,1.0)

# create another filter...computing a contour of an input data. 
isosurface = vtk.vtkContourFilter() 
isosurface.SetInputConnection(sample.GetOutputPort()) 
isosurface.GenerateValues(15, 0.0, 4.2) 

# create a mapper, which mapps data to visualizable data structure. 
contMapper = vtk.vtkPolyDataMapper() 
contMapper.SetInputConnection(isosurface.GetOutputPort()) 
contMapper.SetScalarRange(0.0, 1.2) 

# create an actor, which can be displayed. 
contActor = vtk.vtkActor() 
contActor.SetMapper(contMapper) 



# create a window with a renderer. 
ren = vtk.vtkRenderer() 
renWin = vtk.vtkRenderWindow() 
renWin.AddRenderer(ren) 
iren = vtk.vtkRenderWindowInteractor() 
iren.SetRenderWindow(renWin) 
ren.SetBackground(0.95, 0.95, 1.0)
ren.AddActor(contActor) 
renWin.SetSize( 500, 500 ) 


SliderRepres = vtk.vtkSliderRepresentation2D()
min = 0 #ImageViewer.GetSliceMin()
max = 256 #ImageViewer.GetSliceMax()
SliderRepres.SetMinimumValue(min)
SliderRepres.SetMaximumValue(max)
SliderRepres.SetValue((min + max) / 2)
SliderRepres.SetTitleText("Slider")
SliderRepres.GetPoint1Coordinate().SetCoordinateSystemToNormalizedDisplay()
SliderRepres.GetPoint1Coordinate().SetValue(0.2, 0.9)
SliderRepres.GetPoint2Coordinate().SetCoordinateSystemToNormalizedDisplay()
SliderRepres.GetPoint2Coordinate().SetValue(0.8, 0.9)

SliderRepres.SetSliderLength(0.02)
SliderRepres.SetSliderWidth(0.03)
SliderRepres.SetEndCapLength(0.01)
SliderRepres.SetEndCapWidth(0.03)
SliderRepres.SetTubeWidth(0.005)
SliderRepres.SetLabelFormat("%3.0lf")
SliderRepres.SetTitleHeight(0.02)
SliderRepres.SetLabelHeight(0.02)

SliderWidget = vtk.vtkSliderWidget()
SliderWidget.SetInteractor(iren)
SliderWidget.SetRepresentation(SliderRepres)
SliderWidget.KeyPressActivationOff()
SliderWidget.SetAnimationModeToAnimate()
SliderWidget.SetEnabled(True)
SliderWidget.AddObserver("EndInteractionEvent", vtkSliderCallback2)

# this causes the pipeline to "execute"
renWin.Render() 

# initialize and start the interactor 
iren.Initialize() 
iren.Start() 




This is my code. It gives me output for the quadratic fucntion but when I 
change the contour values using slider I couldn't see the changes. Can Someone 
tell me What I am doing wrong here??
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Chris Angelico]

On Thu, Feb 19, 2015 at 2:33 PM, memilanuk  wrote:
> At this point... I don't think concurrency is going to be a major
> requirement for what I have in mind.  For one project, only a few people
> will be writing to the DB, and only by a stroke of luck would it be at the
> same time, and it would be very unlikely that they would be modifying the
> same record at the same time due to physical constraints.
>
> For the other... there may be anywhere from 1-10 (maybe more, but doubtful)
> entering data (creating new records for competitors, or entering existing
> competitors in a tournament).  I have a hard time picturing that few people
> stressing a modern computer system enough to where SQLite couldn't keep up
> (thinking web-based interface using Flask or something similar).  In the
> latter case, one of the over-arching priorities is that it be easily
> distributable, as in that people with relatively little knowledge of a
> database be able to set it up and run it.

Both of these need concurrency. You may not need _heavy_ concurrency,
but you certainly do need to cope adequately with multiple
simultaneous users. Your first case is a perfect example of why you
need a database rather than flat files; in fact, you want the
granularity of record-level locking rather than table-level. Alas,
SQLite3 does not actually offer this (in fact, I'm not sure it even
offers table-level locking); once any process begins writing to the
database, all others are locked out (even for reading) until it
finishes. That's fine if you (a) don't write very often, and (b) don't
write very much, but the fact that you're trying to modify different
records doesn't help you here. It does with full-scale database
systems, where you actually do have record-level locking, but not with
SQLite3.

Your second case definitely demands concurrency. I've seen tournaments
for various games where database-level write locking would be a
critical problem, and that with only a couple hundred players and a
handful of people keying in data. Of course, it depends how much
effort it takes to key that in. If the humans have to enter extensive
reports on the tournament results, they'll spend most of their time
doing that; but if their job is to quickly say "X beat Y 2-1" and then
get back results saying "X plays Z next, Y gets a bye", then you need
your database to react quickly, even if three other people are
entering results. So it's a huge question of human versus computer
workload... but once again, chances are you need record-level locking.

It may very well turn out that SQLite3 is entirely capable of the job.
But it's certainly not proven by your above statements, and I would
start by assuming PostgreSQL by default.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Ben Finney]

memilanuk  writes:

> At this point... I don't think concurrency is going to be a major
> requirement for what I have in mind.

What's the difference betwen a “requirement” and a “major requirement”?

If you want networked access, you need concurrent access and access
permissions, etc.

SQLite does not have concurrent access. Once you require concurrent
access, you need something more complex, like PostgreSQL.

> I have a hard time picturing that few people stressing a modern
> computer system enough to where SQLite couldn't keep up (thinking
> web-based interface using Flask or something similar). In the latter
> case, one of the over-arching priorities is that it be easily
> distributable, as in that people with relatively little knowledge of a
> database be able to set it up and run it.

Set it up where? Are you hoping that a network-accessible service can be
set up without knowledge of the specific concurrent authenticated
networked access is needed in each installation?

-- 
 \ “Broken promises don't upset me. I just think, why did they |
  `\ believe me?” —Jack Handey |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[rurpy]

On 02/18/2015 05:08 PM, Mario Figueiredo wrote:
>[...]
> SQLite misses some important features that makes it better suited as a
> simple datastore, not much unlike shelve. And network use is not one
> of them, since you can actually implement concurrent sqlite access by
> coding an intermediate layer. Assuming of course we are talking about
> a small number of concurrent users.

I think there are some persistent misunderstandings about Sqlite
in this thread,  Sqlite offers concurrent access already.  
What Sqlite doesn't offer is high performance concurrent write
access.  That is, it locks the entire database for the duration 
of a write operation.  Given that most such operations are pretty
short, for a small number of concurrent writers this is not a 
big problem.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: sqlite3 and dates

[rurpy]

On 02/18/2015 04:07 PM, Steven D'Aprano wrote:
> ru...@yahoo.com wrote:
>> On 02/18/2015 01:14 PM, Ben Finney wrote:
>>> Johannes Bauer  writes:
 On 18.02.2015 08:05, Chris Angelico wrote:

> But if you need more facilities than SQLite3 can offer, maybe it's
> time to move up to a full database server, instead of local files.
> Switching to PostgreSQL will give you all those kinds of features,
> plus a lot of other things that I would have thought pretty basic -
> like ALTER TABLE. It was quite a surprise to learn that SQLite3 didn't
> support that.

 I see you're running a lawnmower. Maybe you should switch to a combine
 harvester. That'll get you extra features like a reciprocating knife
 cutter bar. I was quite surprised that regular lawnmowers don't support
 those.
>>>
>>> Chris has pointed out one flaw in this analogy; I'll address another.
>>>
>>> A feature like 'ALTER TABLE' is not equivalent to a "reciprocating knife
>>> cutter bar". I'm in agreement that it is a pretty basic SQL feature, and
>>> it doesn't appear to conflict with the narrow focus that we all agree is
>>> appropriate for SQLite.
>>
>> No, you and Chris are way off base and Johannes is correct.
>> He was pointing out that there are many applications that can
>> benefit from a database and a full-blown, bells and whistles
>> solution like Postgresql is often overkill in that (very common)
>> case.  His analogy is quite apt and I wish I'd thought of it.
> 
> 
> I'm not seeing that at all. Chris explicitly proceeded his comments with the
> condition "if you need more facilities than SQLite3 can offer".

Right.  And did so in a context where there the facility 
presumed not to be offered was getting a date back from Sqlite.
Common sense should tell anyone that it is very improbable 
that there is no way to get a date out of a sqlite database.
Chris' "solution" to that problem?  Switch to Postgresql.
That was the context for Johannes' analogy.

> Johannes'
> analogy ignores that and consequently mocks the very idea that anyone might
> need more than a regular lawmower -- even a farmer with a thousand acres of
> wheat to be harvested.

The analogy works precisely because farmers with a thousand 
acres of wheat to be harvested need reciprocating knife 
cutter bars.  In the same way people dealing with terabytes 
of data, concurrent updates, enterprise scale data and 
applications need a Postregsql.  People mowing their lawns 
do not.  People without needs for the things that client server 
database do well, whose problem is not immediately being able
to figure out how to get a date, do not.  (Did you really need 
that explained to you?)

If you didn't interpret it the way I did (and the way I presume
Johannes meant it) then, well, you didn't interpret it the same.
Your interpretation (especially given your penchant for sophistry) 
are not my problem.  I am quite happy to let anyone reading make 
their own evaluation.

> Johannes' subsequent posts are more nuanced about Sqlite filling a niche and
> not being suitable for everything, but the analogy you're supporting
> doesn't. It's an amusing quip, quite funny, but like most quips, lacks
> nuance and misrepresents Chris' original position.

All analogies are imperfect and thus make easy targets.  Shrug.
 
>[...]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[rurpy]

On 02/18/2015 07:13 PM, Steven D'Aprano wrote:> Chris Angelico wrote:
>>> SQLite misses some important features that makes it better suited as a
>>> simple datastore, not much unlike shelve. And network use is not one
>>> of them, since you can actually implement concurrent sqlite access by
>>> coding an intermediate layer. Assuming of course we are talking about
>>> a small number of concurrent users.
>>
>> This is what I was saying: it's fine for purposes like Firefox's
>> bookmarks and settings and such (which I think was what it was
>> originally developed for?). Not so fine over a network.
> 
> The sheer number of Firefox bugs related to its use of SQLite says
> different.
>
> Once upon a time, Firefox's config, bookmarks, etc. were stored in plain
> text files. At worst they were HTML. You could trivially read them, copy
> them, restore them and even (if you were careful) edit them using the text
> editor of your choice. Many a time I was on one machine, wanted to know a
> bookmark from another machine, so I would ssh across to the other machine
> and run grep over the bookmark file.

I agree, I prefer plain text files whenever practical.  But since 
the original discussion was about Sqlite vs Postgresql, not Sqlite
vs text files, shouldn't the question be: would Firefox be better 
if it required you to install and configure Postgreql instead of 
using Sqlite?

> No more. Firefox still keeps a bookmark HTML file, but it never seems to be
> synced with the actual bookmarks. Settings are stored in an opaque blob,
> rather than human-readable text, limiting what you can do with it. It's very
> nice that Firefox offers about:config but not so nice that you can't do the
> same thing without the GUI running.
> 
> If Firefox crashes, there are failure modes where it can no longer read your
> bookmarks, or keep history. I don't mean that history won't persist across
> restarts, I mean that *within a single session* it cannot remember what page
> you came from so you can hit the Back button and return to it. WTF?
> 
> I swear, if not for the fact that every single other browser is worse, I
> would dump Firefox in a second.
> 
> I don't believe for a second that moving to SQlite has anything to do with
> performance, because reading and writing preference settings should be rare
> and far from a bottleneck. SQlite is simply fragile and unreliable over a
> network, and people using their home directory on a network drive are not
> that rare.

I don't see any evidence that it is Sqlite that is the problem
as opposed to FF's use (or misuse) of it, or other problems that
are in FF and have nothing to do with Sqlite.  If Sqlite reliably 
implements ACID semantics as they claim, is certainly should be 
possible to make use of it without the problems you (and I too) 
see.  And there is no reason to believe the situation would be
any better with Postgresql.
-- 
https://mail.python.org/mailman/listinfo/python-list

When to use SQLite3 [was Re: 'Lite' Databases (Re: sqlite3 and dates)]

[Ethan Furman]

At the risk of using actual data, I looked this up at 
http://www.sqlite.org/whentouse.html:


Checklist For Choosing The Right Database Engine

 * Is the data separated from the application by a network? → choose 
client/server

Relational database engines act as a bandwidth-reducing data filter. So it 
is best to keep the database engine and
the data on the same physical device so that the high-bandwidth engine-to-disk 
link does not have to traverse the
network, only the lower-bandwidth application-to-engine link.

But SQLite is built into the application. So if the data is on a separate 
device from the application, it is
required that the higher bandwidth engine-to-disk link be across the network. 
This works, but it is suboptimal. Hence,
it is usually better to select a client/server database engine when the data is 
on a separate device from the application.

 * Many concurrent writers? → choose client/server

If many threads and/or processes need to write the database at the same 
instant (and they cannot queue up and take
turns) then it is best to select a database engine that supports that 
capability, which always means a client/server
database engine.

SQLite only supports one writer at a time per database file. But in most 
cases, a write transaction only takes
milliseconds and so multiple writers can simply take turns. SQLite will handle 
more write concurrency that many people
suspect. Nevertheless, client/server database systems, because they have a 
long-running server process at hand to
coordinate access, can usually handle far more write concurrency than SQLite 
ever will.

 * Big data? → choose client/server

If your data will grow to a size that you are uncomfortable or unable to 
fit into a single disk file, then you
should select a solution other than SQLite. SQLite supports databases up to 140 
terabytes in size, assuming you can find
a disk drive and filesystem that will support 140-terabyte files. Even so, when 
the size of the content looks like it
might creep into the terabyte range, it would be good to consider a centralized 
client/server database.

 * Otherwise → choose SQLite!

For device-local storage with low writer concurrency and less than a 
terabyte of content, SQLite is almost always a
better solution. SQLite is fast and reliable and it requires no configuration 
or maintenance. It keeps thing simple.
SQLite "just works".



signature.asc
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[memilanuk]

On 02/18/2015 08:09 PM, Ben Finney wrote:


I have a hard time picturing that few people stressing a modern
computer system enough to where SQLite couldn't keep up (thinking
web-based interface using Flask or something similar). In the latter
case, one of the over-arching priorities is that it be easily
distributable, as in that people with relatively little knowledge of a
database be able to set it up and run it.


Set it up where? Are you hoping that a network-accessible service can be
set up without knowledge of the specific concurrent authenticated
networked access is needed in each installation?



They would need to be able to set up the application (and whatever 
database) on their laptop or PC, wherever that may be, and spend their 
time administering the event, not the database engine.  Once its set, it 
shouldn't need any tending, or they are going to be SOL as I wouldn't be 
able to help them.  It may be that Flask + SQLite will be enough; 
otherwise I foresee a disproportional amount of *my* time will be spent 
documenting and explaining how to set up and maintain a RDBMS on 
Windows, on a Mac, etc.


Starting to wonder if a pre-configured VM appliance running in 
Virtualbox might be simpler for the end user to set up and run.


--
Shiny!  Let's be bad guys.

Reach me @ memilanuk (at) gmail dot com

--
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[rurpy]

On 02/18/2015 09:09 PM, Ben Finney wrote
> memilanuk  writes:
>[...]
> If you want networked access, you need concurrent access and access
> permissions, etc.

Sqlite has concurrent access.  It doesn't have concurrent 
access that will support a large number of writers or high 
volume of writes.

As for access permissions, it is common, even with Postgresql
to do all database access through a single Postgresql user 
and to implement authorization and access permission in the 
application.

> SQLite does not have concurrent access. Once you require concurrent
> access, you need something more complex, like PostgreSQL.

Please read https://www.sqlite.org/faq.html#q5

>[...]
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Ben Finney]

memilanuk  writes:

> They would need to be able to set up the application (and whatever
> database) on their laptop or PC, wherever that may be, and spend their
> time administering the event, not the database engine.

So, the database will only be accessed by exactly one application, on
exactly the same machine and storage as the application? If so, you
don't need concurrency.

Otherwise, your database needs concurrency; and the person installing
the database will need to make a lot of decisions about the specific
network environment and devices to be allowed to access the database.

But is this what you mean by your requirements not being met by SQLite?

-- 
 \  “Natural catastrophes are rare, but they come often enough. We |
  `\   need not force the hand of nature.” —Carl Sagan, _Cosmos_, 1980 |
_o__)  |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

What behavior would you expect?

[Jason Friedman]

I have need to search a directory and return the name of the most recent
file matching a given pattern.  Given a directory with these files and
timestamps,

q.pattern1.abc Feb 13
r.pattern1.cdf  Feb 12
s.pattern1.efg  Feb 10
t.pattern2.abc Feb 13
u.pattern2.xyz  Feb 14
v.pattern2.efg  Feb 10

calling my_function("/path/to/dir", "pattern1") will return q.pattern1.abc
and calling my_function("/path/to/dir", "pattern2") will return
u.pattern2.xyz.

My question is, what would be a reasonable behavior/result/return value if:
1. "/path/to/dir" does not exist or is not readable
2. no files match the given pattern

Also, what would be a reasonable name for such a function?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[memilanuk]

On 02/18/2015 08:36 PM, Ben Finney wrote:

memilanuk  writes:


They would need to be able to set up the application (and whatever
database) on their laptop or PC, wherever that may be, and spend their
time administering the event, not the database engine.


So, the database will only be accessed by exactly one application, on
exactly the same machine and storage as the application? If so, you
don't need concurrency.

Otherwise, your database needs concurrency; and the person installing
the database will need to make a lot of decisions about the specific
network environment and devices to be allowed to access the database.

But is this what you mean by your requirements not being met by SQLite?



In the past I've been waffling back and forth between a desktop 
client/server setup, or a web-based interface with everything on one 
computer.  At this point I'm leaning toward the latter.


--
Shiny!  Let's be bad guys.

Reach me @ memilanuk (at) gmail dot com

--
https://mail.python.org/mailman/listinfo/python-list

Re: can python handle CHIME .spt files?

[Laura Creighton]

I went and asked your question to Andrew Dalke, who is an expert
in such things.  Writing programs for visualising molecules is what
he does for a living.  The news is not good.

First of all, he says the "spt" format is a command format.
See http://www.callutheran.edu/BioDev/omm/scripting_ch/molmast.htm
for examples.

  select all; wireframe;
dots off; wireframe off; backbone;
  wireframe 100; spacefill off; list molsurface transparent;
select all; spacefill; set specular on; set specpower 20;
  define piece :A and 10, :A and 11, :A and 12;
select all; spacefill off; wireframe;
  select atomno=4174; label protein;

It's like Tcl, with lots of internal syntax handled by the individual commands.

For example, "select atomno=4174" takes a full expression, like

  select atomno >= 195 and atomno <= 277

More details at http://jmol.sourceforge.net/docs/JmolUserGuide/ch04.html .

The only way to tell that it works, in Python and without using Chime, is
to re-write command interpreter that understands the script syntax.   Andrew
Dalke says that he tried something like that back in the 1990s, and
while it's possible, it is not easy.  When Andrew says things like that,
you can bet money that it is very, very hard, because he does seemingly
impossible things for fun over breakfast.  Indeed, I wrote him hoping he
would have scripts lying around that already do this ...

He says that, these days, the best solution would probably be some
in-browser Javascript code that drives Chime and checks if there's an error,
but this is also not a job for somebody who is learning how to program,
and iff it's less than several dozen/100 scripts, then really this is best
done manually, unless the person  enjoys programming.

If you want to write him and ask him more questions, his contact page
is at http://www.dalkescientific.com/contact.html  He's very friendly,
as well as being a very good friend of mine.

Laura Creighton

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: When to use SQLite3 [was Re: 'Lite' Databases (Re: sqlite3 and dates)]

[Steve Hayes]

On Wed, 18 Feb 2015 20:15:30 -0800, Ethan Furman 
wrote:

>At the risk of using actual data, I looked this up at 
>http://www.sqlite.org/whentouse.html:
>
>
>Checklist For Choosing The Right Database Engine

Interesting. 

A couple of months ago I asked in comp.databases what the differences
were between SQLite and MySQL, and I got a lot of uninformative
gobbledegook. 

This was more informative. 

I would summarise it by saying if you want a multiuser database
running on a network, use MySQL. If you want a standalone database on
a single machine, use SQLite. 


-- 
Steve Hayes from Tshwane, South Africa
Web:  http://www.khanya.org.za/stevesig.htm
Blog: http://khanya.wordpress.com
E-mail - see web page, or parse: shayes at dunelm full stop org full stop uk
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: When to use SQLite3 [was Re: 'Lite' Databases (Re: sqlite3 and dates)]

[Ethan Furman]

On 02/18/2015 08:59 PM, Steve Hayes wrote:

> I would summarise it by saying [...] if you want a standalone database on
> a single machine, use SQLite. 

It sounds like SQLite would also work fine if that single-machine scenario was 
a web-app with not-too-many users trying
to write at once.

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Chris Angelico]

On Thu, Feb 19, 2015 at 3:44 PM, Jason Friedman  wrote:
> I have need to search a directory and return the name of the most recent
> file matching a given pattern.  Given a directory with these files and
> timestamps,
>
> q.pattern1.abc Feb 13
> r.pattern1.cdf  Feb 12
> s.pattern1.efg  Feb 10
> t.pattern2.abc Feb 13
> u.pattern2.xyz  Feb 14
> v.pattern2.efg  Feb 10
>
> calling my_function("/path/to/dir", "pattern1") will return q.pattern1.abc
> and calling my_function("/path/to/dir", "pattern2") will return
> u.pattern2.xyz.

That seems reasonable, and well-defined.

> My question is, what would be a reasonable behavior/result/return value if:
> 1. "/path/to/dir" does not exist or is not readable

Raise an exception. Or, better still, just allow the exception to bubble.

> 2. no files match the given pattern

Either return None, or raise an exception, depending on how "normal"
this state is. Is it simply a matter of "you asked for something, but
you got nothing"? Then return None. Is it a really unusual situation
that should basically never happen? Then raise an exception, so you
get an instant report with no wondering "why am I getting these
strange TypeErrors".

> Also, what would be a reasonable name for such a function?

newest_file_matching() seems decent. Remember, it doesn't have to be
especially short.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Ben Finney]

memilanuk  writes:

> In the past I've been waffling back and forth between a desktop
> client/server setup, or a web-based interface with everything on one
> computer. At this point I'm leaning toward the latter.

So, it's been many exchanges back and forth, and you still aren't
telling us what specific needs you have that SQLite can't provide. At
this point I'm just going to have to wait until you can lay out the
specifics.

-- 
 \ “In economics, hope and faith coexist with great scientific |
  `\  pretension and also a deep desire for respectability.” —John |
_o__)Kenneth Galbraith, 1970-06-07 |
Ben Finney

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: can python handle CHIME .spt files?

[Ethan Furman]

On 02/18/2015 08:57 PM, Laura Creighton wrote:

> I went and asked your question to Andrew Dalke, who is an expert
> in such things. 

Did you happen to ask him about PyMol?  Just curious.  ;)

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Gregory Ewing]

Mario Figueiredo wrote:

Parameterized queries is just a pet peeve of mine that I wish to
include here. SQLite misses it


How does sqlite3 miss parameterized queries? It supports
DB-API parameter subsitution with '?' according to the
docs.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[memilanuk]

On 02/18/2015 09:16 PM, Ben Finney wrote:

memilanuk  writes:


In the past I've been waffling back and forth between a desktop
client/server setup, or a web-based interface with everything on one
computer. At this point I'm leaning toward the latter.


So, it's been many exchanges back and forth, and you still aren't
telling us what specific needs you have that SQLite can't provide. At
this point I'm just going to have to wait until you can lay out the
specifics.



Okay, let me put it like this:  if I set up a web interface using Flask 
for the front-end, and SQLite as the backend DB, running from a 
PC/laptop, with anywhere from 1 to 10 people doing data entry from other 
devices (laptops, desktops, tablets, etc.) at roughly the same time, is 
SQLite going to be 'concurrent' enough?



--
Shiny!  Let's be bad guys.

Reach me @ memilanuk (at) gmail dot com

--
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Mark Lawrence]

On 19/02/2015 00:08, Mario Figueiredo wrote:


Parameterized queries is just a pet peeve of mine that I wish to
include here. SQLite misses it and I miss the fact SQLite misses it.
The less SQL one needs to write in their code, the happier one should
be.



https://docs.python.org/3/library/sqlite3.html#module-sqlite3 paragraphs 
seven and eight.


"Usually your SQL operations will need to use values from Python 
variables. You shouldn’t assemble your query using Python’s string 
operations because doing so is insecure; it makes your program 
vulnerable to an SQL injection attack (see http://xkcd.com/327/ for 
humorous example of what can go wrong).


Instead, use the DB-API’s parameter substitution. Put ? as a placeholder 
wherever you want to use a value, and then provide a tuple of values as 
the second argument to the cursor’s execute() method. (Other database 
modules may use a different placeholder, such as %s or :1.) For example:..."


--
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence

--
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Ethan Furman]

On 02/18/2015 09:26 PM, memilanuk wrote:
> On 02/18/2015 09:16 PM, Ben Finney wrote:
>> memilanuk  writes:
>>
>>> In the past I've been waffling back and forth between a desktop
>>> client/server setup, or a web-based interface with everything on one
>>> computer. At this point I'm leaning toward the latter.
>>
>> So, it's been many exchanges back and forth, and you still aren't
>> telling us what specific needs you have that SQLite can't provide. At
>> this point I'm just going to have to wait until you can lay out the
>> specifics.
>>
> 
> Okay, let me put it like this:  if I set up a web interface using Flask for 
> the front-end, and SQLite as the backend DB,
> running from a PC/laptop, with anywhere from 1 to 10 people doing data entry 
> from other devices (laptops, desktops,
> tablets, etc.) at roughly the same time, is SQLite going to be 'concurrent' 
> enough?

Well, having zero experience with SQLite, but having read the docs just today 
[snip snide remark] -- I think you'll be
fine with SQLite under those conditions.  :)

--
~Ethan~



signature.asc
Description: OpenPGP digital signature
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: What behavior would you expect?

[Dave Angel]

On 02/19/2015 12:10 AM, Chris Angelico wrote:

On Thu, Feb 19, 2015 at 3:44 PM, Jason Friedman  wrote:

I have need to search a directory and return the name of the most recent
file matching a given pattern.  Given a directory with these files and
timestamps,

q.pattern1.abc Feb 13
r.pattern1.cdf  Feb 12
s.pattern1.efg  Feb 10
t.pattern2.abc Feb 13
u.pattern2.xyz  Feb 14
v.pattern2.efg  Feb 10

calling my_function("/path/to/dir", "pattern1") will return q.pattern1.abc
and calling my_function("/path/to/dir", "pattern2") will return
u.pattern2.xyz.


That seems reasonable, and well-defined.


My question is, what would be a reasonable behavior/result/return value if:
1. "/path/to/dir" does not exist or is not readable


Raise an exception. Or, better still, just allow the exception to bubble.


2. no files match the given pattern


Either return None, or raise an exception, depending on how "normal"
this state is. Is it simply a matter of "you asked for something, but
you got nothing"? Then return None. Is it a really unusual situation
that should basically never happen? Then raise an exception, so you
get an instant report with no wondering "why am I getting these
strange TypeErrors".


Also, what would be a reasonable name for such a function?


newest_file_matching() seems decent. Remember, it doesn't have to be
especially short.



Consider returning a list of files, sorted by datestamp.  Return an 
empty list if there are no matches.  That way, the user can find the 
newest, the oldest, can check if any match, and so on, all by how it 
manipulates the list.


   if result: check whether there are any matches
   newest = result[0]
   oldest = result[-1]

Your code probably has to build the list anyway, to find the newest.  So 
why not just return it, and let the caller decide which part(s) to keep.


--
DaveA
--
https://mail.python.org/mailman/listinfo/python-list

Re: can python handle CHIME .spt files?

[Laura Creighton]

In a message of Wed, 18 Feb 2015 21:23:50 -0800, Ethan Furman writes:
>On 02/18/2015 08:57 PM, Laura Creighton wrote:
>
>> I went and asked your question to Andrew Dalke, who is an expert
>> in such things. 
>
>Did you happen to ask him about PyMol?  Just curious.  ;)
>
>--
>~Ethan~

I hadn't then, but have since.  I will report what he says.

Laura
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Steven D'Aprano]

ru...@yahoo.com wrote:

> On 02/18/2015 07:13 PM, Steven D'Aprano wrote:> Chris Angelico wrote:
 SQLite misses some important features that makes it better suited as a
 simple datastore, not much unlike shelve. And network use is not one
 of them, since you can actually implement concurrent sqlite access by
 coding an intermediate layer. Assuming of course we are talking about
 a small number of concurrent users.
>>>
>>> This is what I was saying: it's fine for purposes like Firefox's
>>> bookmarks and settings and such (which I think was what it was
>>> originally developed for?). Not so fine over a network.
>> 
>> The sheer number of Firefox bugs related to its use of SQLite says
>> different.
>>
>> Once upon a time, Firefox's config, bookmarks, etc. were stored in plain
>> text files. At worst they were HTML. You could trivially read them, copy
>> them, restore them and even (if you were careful) edit them using the
>> text editor of your choice. Many a time I was on one machine, wanted to
>> know a bookmark from another machine, so I would ssh across to the other
>> machine and run grep over the bookmark file.
> 
> I agree, I prefer plain text files whenever practical.  But since
> the original discussion was about Sqlite vs Postgresql, not Sqlite
> vs text files, shouldn't the question be: would Firefox be better
> if it required you to install and configure Postgreql instead of
> using Sqlite?

Very possibly. With modern dependency management, it isn't hard to install 
Postgresql:

sudo aptitude postgresql

or equivalent should work. For primitive operating systems with no 
dependency management available, Firefox could come with a simple script 
which downloads, installs, configures and runs Postgresql. (Everything is 
simple for the guy who doesn't have to do it.)

Possible snags:

- Possibly postgresql is simply *too big*. Your 50MB(?) Firefox 
  turns into a 2GB install. I doubt it -- on Debian, postgresql 
  is 17MB installed. But I don't know what dependencies I'm not
  counting.

- Or it is impossible to configure without excessive amounts of
  tech-savvy human intervention. Again, I doubt it. I seem to 
  recall needing to create a Postgresql user and password. But 
  maybe even that is too technical for the average Firefox user.

- Maybe there are nasty interactions between Postgresql listening
  on some port and Windows firewall wanting to block that same port.

Or... and here is a radical thought... maybe Firefox could give you the 
choice of which database? By default, it might use Sqlite, to satisfy the 
desktop users who don't want to think about it. And for those who are 
disturbed by the fragility of Sqlite on a network home directory, you just 
set a config setting in about:config to point at your existing Postgresql 
instance, and never need worry about it again.

The Firefox devs surprise and confuse me. On the one hand, they have 
designed a powerful plug-in architecture, and encourage their user-base to 
use it for all sorts of amazing functionality that they don't want to build 
into the core browser. Yay for this. And on the other hand, they are 
*actively hostile* to any suggestion that using SQlite is not the best and 
*only* appropriate solution to the problem of storing config, bookmarks and 
history. A plug-in database architecture would probably work really well.

> I don't see any evidence that it is Sqlite that is the problem
> as opposed to FF's use (or misuse) of it, or other problems that
> are in FF and have nothing to do with Sqlite.

No no, even Sqlite devs recommend against using it on network drives. The 
Firefox problem is that when FF crashes, as it can do, or if you yank the 
power to the computer and everything dies, if your home directory is on a 
network drive, the database may be left in a locked state, or even 
corrupted. Nothing that the FF developers can do, given the choice of 
Sqlite.


> If Sqlite reliably
> implements ACID semantics as they claim, 

Ah, well "reliably" is a tricky word...

http://stackoverflow.com/questions/788517/sqlite-over-a-network-share


-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Chris Angelico]

On Thu, Feb 19, 2015 at 6:07 PM, Steven D'Aprano
 wrote:
> Very possibly. With modern dependency management, it isn't hard to install
> Postgresql:
>
> sudo aptitude postgresql
>
> or equivalent should work. For primitive operating systems with no
> dependency management available, Firefox could come with a simple script
> which downloads, installs, configures and runs Postgresql. (Everything is
> simple for the guy who doesn't have to do it.)

Definitely a possibility. I'm pretty sure I've seen that exact thing
done by one application (on Windows; on a typical Linux system,
that'll be done by simple dependency management - your package
metadata says "depends on postgresql" and apt-get or yum or whatever
will do the work), and it's quite common for a Windows installer
wizard to go through a series of subcomponents (grab the .NET
framework, grab these hotfixes that the program depends on, grab some
adware toolbar that you forgot to untick, and *then* install the
program you asked for).

> Possible snags:
>
> - Possibly postgresql is simply *too big*. Your 50MB(?) Firefox
>   turns into a 2GB install. I doubt it -- on Debian, postgresql
>   is 17MB installed. But I don't know what dependencies I'm not
>   counting.

Yeah. And frankly, I would be surprised if Firefox is only 50MB these
days. The extra dent of PostgreSQL won't be all that significant - and
don't forget that the SQLite3 dent can be removed, so you're talking
about the difference between them, plus you can omit a whole bunch of
PG's ancillaries.

> - Or it is impossible to configure without excessive amounts of
>   tech-savvy human intervention. Again, I doubt it. I seem to
>   recall needing to create a Postgresql user and password. But
>   maybe even that is too technical for the average Firefox user.

You don't even need to do that. An absolutely default Postgres on
Debian or Ubuntu is ready to use, with peer authentication. If you can
become root, you can then drop privs to the 'postgres' user and
connect to the database that way.

I don't know if it's possible to do a non-root installation of
PostgreSQL, but if it isn't today, it could easily be tomorrow, if
someone puts in a little effort. You'd miss out on boot-time startup,
and it'd probably have to do some password-based authentication (with
autogenerated passwords), but it certainly could work. The Windows
equivalent ("install for just me") is, I think, already possible.

> - Maybe there are nasty interactions between Postgresql listening
>   on some port and Windows firewall wanting to block that same port.

That's definitely an issue, given that Windows doesn't have Unix
domain sockets. But I'm sure it's a solvable problem. How does IDLE
cope with firewall issues?

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Steven D'Aprano]

Ben Finney wrote:

> What's the difference betwen a “requirement” and a “major requirement”?

"I require a gold-plated Mercedes, if I can have it for less than $30,000. 
My major requirement is for some sort of personal transport."


*wink*


-- 
Steve

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: 'Lite' Databases (Re: sqlite3 and dates)

[Mario Figueiredo]

On Thu, 19 Feb 2015 18:22:57 +1300, Gregory Ewing
 wrote:

>
>How does sqlite3 miss parameterized queries? It supports
>DB-API parameter subsitution with '?' according to the
>docs.

It's actually parameterized views that I meant. Not queries. SQLite
misses the ability to write parameterized views.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: python implementation of a new integer encoding algorithm.

[Mario Figueiredo]

A lot of patronizing egos running around in these groups. This is a
sad thread...

What is being asked is for help, not whether this is useful or needed.
Jan-Hein is after some directions, not whether your bloody opinion on
how he should use his free time.

If the interest and usability of a project would somehow become a
problem, then boy, oh, boy; Most everyone in here, including the
patronizing posters, would probably be left without anything to code.
-- 
https://mail.python.org/mailman/listinfo/python-list