Re: memory leak with re.match

[Mayling ge]

   Thanks. I actually comment out all  handling code. The loop ends with  the
   re_pat.match and nothing followed.
   Sent from Mail Master
   On 07/05/2017 08:31, [1]Cameron Simpson wrote:

 On 04Jul2017 17:01, Mayling ge  wrote:
 >   My function is in the following way to handle file line by line.
 There are
 >   multiple error patterns  defined and  need to apply  to each  line.
 I  use
 >   multiprocessing.Pool to handle the file in block.
 >
 >   The memory usage increases to 2G for a 1G file. And stays in 2G even
 after
 >   the file processing. File closed in the end.
 >
 >   If I comment  out the  call to re_pat.match,  memory usage  is
 normal  and
 >   keeps under 100Mb. [...]
 >
 >   def line_match(lines, errors)
 >   for error in errors:
 >   try:
 >   re_pat = re.compile(error['pattern'])
 >   except Exception:
 >   print_error
 >   continue
 >   for line in lines:
 >   m = re_pat.match(line)
 >   # other code to handle matched object
 [...]
 >   Notes: I  omit  some  code  as  I  think  the  significant
  difference  is
 >   with/without re_pat.match(...)

 Hmm. Does the handling code (omitted) keep the line or match object in
 memory?

 If leaving out the "m = re_pat.match(line)" triggers the leak, and
 presuming
 that line itself doesn't leak, then I would start to suspect the
 handling code
 is not letting go of the match object "m" or of the line (which is
 probably
 attached to the match object "m" to support things like m.group() and so
 forth).

 So you might need to show us the handling code.

 Cheers,
 Cameron Simpson 

References

   Visible links
   1. mailto:c...@zip.com.au
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Privy: An easy, fast lib to password-protect your data

[Chris Warrick]

On 5 July 2017 at 02:39,   wrote:
> On Monday, July 3, 2017 at 10:36:39 PM UTC-4, ofekm...@gmail.com wrote:
>> https://github.com/ofek/privy
>
> Bump b/c spam
> --
> https://mail.python.org/mailman/listinfo/python-list

The person spamming right now would be you. You just posted a link,
without any explanations, any marketing blurbs, nothing.

Why would I use your tool instead of something established, that has
been properly audited — say, PGP for example? How do I know your
one-man project has no security holes, backdoors, or other
vulnerabilities? How do I know that the encryption method chosen by
you is sound? If there is no leaked data?

And I really dislike the description of your project:

> Privy is a small and fast utility for password-protecting secret data such as 
> API keys, cryptocurrency wallets, or seeds for digital signatures.

What does “password-protecting” mean? Why is this not “encrypting”?
How do you expect this to work with API keys?

-- 
Chris Warrick 
PGP: 5EAAEA16
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Generator - Provide parameters to url - requests

[Peter Otten]

Sayth Renshaw wrote:

> Hi
> 
> I am struggling to figure out how I can create a generator to provide
> values to my url. My url needs to insert the year month and day in the url
> not as params to the url.
> 
> 
> import json
> import requests
> import datetime
> 
> # using this I can create a list of dates for the first 210 days of this
> # year.
> 
> base = datetime.datetime(2017,1,1)
> 
> def datesRange(max, min):
> date_list = (base - datetime.timedelta(days=x) for x in
> range(max,min)) DAY = date_list.day
> MONTH = date_list.month
> YEAR = date_list.year
> yield DAY, MONTH, YEAR

A single yield usually doesn't make sense -- you need one loop to generate 
the data

ONE_DAY = datetime.timedelta(days=1)

def dates(first, numdays):
# generate datetime objects for extra clarity
# note there are no implicit arguments like `base` in your code 
for _ in range(numdays):
yield first
first += ONE_DAY

... and another one to consume it

def create_url(date):
return "https:/example.com/{0.year}/{0.month}/{0.day}/".format(
date
)

def create_filename(date):
# use fixed widths for month and day to avoid ambiguous
# filenames, e. g. is "2017111.json" jan-11 or nov-1?
return "{0.year}{0.month:02}{0.day:02}.json".format(date)

FIRST = datetime.datetime(2017, 1, 1)

for date in dates(FIRST, numdays=210):
url = create_url(date)
r = requests.get(url)
filename = create_filename(date)
with open(filename, "w") as f:
...

> dateValues = datesRange(-210,0)
> 
> def create_url(day, month, year):
>
https://api.tatts.com/sales/vmax/web/data/racing/{0}/{1}/{2}/sr/full".format(YEAR,MONTH,DAY)
> return url
> 
> Then I want to insert them in this url one at a time from the generator
> 
> try:
>r = requests.get(INSERT_URL_HERE)
>if r.status_code == requests.codes.ok then:
>   # open a file for writing using url paramters
>   with open(SR + DAY + MONTH + YEAR + '.json', 'w') as f:
>   # Do stuff from here not relevant to question.
> 
> I have just gotten lost.
> 
> Is there an easier way to go about this?
> 
> Cheers
> 
> Sayth


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Generator - Provide parameters to url - requests

[Frank Millman]

"Sayth Renshaw"  wrote in message 
news:328f42bb-ba23-4199-9f3a-9ec1829bc...@googlegroups.com...


Hi

I am struggling to figure out how I can create a generator to provide 
values to my url. My url needs to insert the year month and day in the url 
not as params to the url.


import json
import requests
import datetime

# using this I can create a list of dates for the first 210 days of this 
year.


base = datetime.datetime(2017,1,1)

def datesRange(max, min):
date_list = (base - datetime.timedelta(days=x) for x in 
range(max,min))

DAY = date_list.day
MONTH = date_list.month
YEAR = date_list.year
yield DAY, MONTH, YEAR

dateValues = datesRange(-210,0)


Are you sure that this works?

The easiest way to test it is -


list(datesRange(-210, 0))


If I try this I get AttributeError: 'generator object has no attribute 'day'

I would write it like this -
 def datesRange(max, min):
for day in range(max, min):
   date = base - datetime.timedelta(days=day)
   yield date.day, date.month, date.year

Actually, I have just read Peter's response, and his version is much better, 
but this one is closer to your original code.




def create_url(day, month, year):

https://api.tatts.com/sales/vmax/web/data/racing/{0}/{1}/{2}/sr/full".format(YEAR,MONTH,DAY)
return url

Then I want to insert them in this url one at a time from the generator



To do this, you need some kind of loop to iterate over your generator -

   for day, month, year in datesRange(-210, 0):
 # do something

Does this help?

Frank Millman



--
https://mail.python.org/mailman/listinfo/python-list

Re: memory leak with re.match

[Albert-Jan Roskam]

From: Python-list  on 
behalf of Mayling ge 
Sent: Tuesday, July 4, 2017 9:01 AM
To: python-list
Subject: memory leak with re.match
    
   Hi,

   My function is in the following way to handle file line by line. There are
   multiple error patterns  defined and  need to apply  to each  line. I  use
   multiprocessing.Pool to handle the file in block.

   The memory usage increases to 2G for a 1G file. And stays in 2G even after
   the file processing. File closed in the end.

   If I comment  out the  call to re_pat.match,  memory usage  is normal  and
   keeps under 100Mb.

   am I using re in a wrong way? I cannot figure out a way to fix the  memory
   leak. And I googled .

   def line_match(lines, errors)

  

   lines = list(itertools.islice(fo, line_per_proc))

===> do you really need to listify the iterator?
   if not lines:

   break

   result = p.apply_async(line_match, args=(errors, lines))

===> the signature of line_match is (lines, errors), in args you do (errors, 
lines)

-- 
https://mail.python.org/mailman/listinfo/python-list

Python threading and sharing variables

[pozz]

I'd like to launch *and control* a long thread. I want to print the 
progress of the long thread in the main thread. It's a GUI script, here 
it's a console script only to simplify.


import threading
import time

class MyClass:
def start(self):
self.max = 5
self.pause = 1
t = threading.Thread(target=self.thread)
t.start()
i = -1
while self.cnt != self.max - 1:
if i != self.cnt:
print("{:d}".format(self.cnt))
i = self.cnt
print("Finished")

def thread(self):
for i in range(self.max):
self.cnt = i
time.sleep(self.pause)

c = MyClass()
c.start()


It seems it works, but I'm not sure it is the correct way to share the 
variable self.cnt. It is only written in the long thread and only read 
in the main thread.
Could a single Python instruction be interrupted (in this case, self.cnt 
= i)? Should I use a locking mechanism when reading/writing?


What about if the variable is more complex, for example a list or 
dictionary? Even in this case, is it safe to avoid locking on a shared 
variable if the operation on the variable is performed in a single 
Python instruction?

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[pozz]

Il 05/07/2017 09:56, pozz ha scritto:
> [...]
It seems it works, but I'm not sure it is the correct way to share the 
variable self.cnt. It is only written in the long thread and only read 
in the main thread.
Could a single Python instruction be interrupted (in this case, self.cnt 
= i)? Should I use a locking mechanism when reading/writing?


What about if the variable is more complex, for example a list or 
dictionary? Even in this case, is it safe to avoid locking on a shared 
variable if the operation on the variable is performed in a single 
Python instruction?



Ok, maybe this atomic behaviour depends on the Python implementation, so 
it's better to avoid relying on atomicity and use a lock to access 
shared variables from different running thread.


However in my simple case one thread writes the variable and the other 
reads it. In this case is it safe to avoid locks?

--
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[Thomas Nyberg]

On 07/05/2017 09:56 AM, pozz wrote:
> It seems it works, but I'm not sure it is the correct way to share the
> variable self.cnt. It is only written in the long thread and only read
> in the main thread.
> Could a single Python instruction be interrupted (in this case, self.cnt
> = i)? Should I use a locking mechanism when reading/writing?
> 
> What about if the variable is more complex, for example a list or
> dictionary? Even in this case, is it safe to avoid locking on a shared
> variable if the operation on the variable is performed in a single
> Python instruction?

I think it would be clearer if you used a queue. Here's an example of
simplified version showing how the communication might work:

test.py
---
from threading import Thread
from queue import Queue
from time import sleep


def main():
q = Queue()
t = Thread(target=worker, args=(q,))
t.start()
while True:
status = q.get()
if status < 0:
break
print(status)
t.join()


def worker(q, limit=5):
for i in range(limit):
sleep(1) # Simulate some work
q.put(i)
q.put(-1) # Some sort of value to indicate being finished


main()


$ python3 test.py
0
1
2
3
4


Not sure if this helps, but I personally find it clearer than the shared
class variable method you're using.

Cheers,
Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: EuroPython 2017: Free Intel Distribution for Python

[M.-A. Lemburg]

On 05.07.2017 00:05, Terry Reedy wrote:
> On 7/4/2017 10:22 AM, M.-A. Lemburg wrote:
>> We are very pleased to have Intel as Diamond Sponsor for EuroPython
>> 2017. You can visit them at the most central booth in our exhibit
>> area, the Sala della Piazza, and take the opportunity to chat with
>> their staff.
>>
>> Please find below a hosted blog post from Intel, that offers us an
>> exciting glimpse at the recently released free, Intel® Distribution
>> for Python:
>>
>> http://blog.europython.eu/post/162590522362/europython-2017-free-intel-distribution-for
>>
> 
> I looked but did not find the most important thing.
> What version of Python?
> 
> Also, if 3.6 rather than 2.7, do they plan to keep up to date?
> Free for how long?  90 days?  Until further notice?  Indefinitely?

I'm sure they will be happy to answer all those questions at
their booth :-)

More details are available here, if you can't attend EuroPython:

https://software.intel.com/en-us/distribution-for-python

(look for e.g. "key specifications")

-- 
Marc-Andre Lemburg
EuroPython Society Chair
http://www.europython-society.org/
http://www.malemburg.com/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[Thomas Nyberg]

On 07/05/2017 10:05 AM, pozz wrote:
> 
> Ok, maybe this atomic behaviour depends on the Python implementation, so
> it's better to avoid relying on atomicity and use a lock to access
> shared variables from different running thread.
> 
> However in my simple case one thread writes the variable and the other
> reads it. In this case is it safe to avoid locks?

I think the general rule would be that no it's not safe to skip the
locks. It's true that with cpython, your method shouldn't run into
problems, but that's just a quirk of how you're using it. I look at from
another perspective, if it's true that no locks actually are necessary,
then why are you using the shared variables in the first place. In this
case, the information needs to be send from the worker thread to the
main thread, but you don't need for any other threads to see it. This
only really requires a single "channel" (e.g. a queue) and not for the
variable to further exposed.

Also personally I just find it much clearer. I was very confused what
your code was doing and basically need to step through it to understand.

Cheers,
Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[Chris Angelico]

On Wed, Jul 5, 2017 at 5:56 PM, pozz  wrote:
> It seems it works, but I'm not sure it is the correct way to share the
> variable self.cnt. It is only written in the long thread and only read in
> the main thread.
> Could a single Python instruction be interrupted (in this case, self.cnt =
> i)? Should I use a locking mechanism when reading/writing?
>
> What about if the variable is more complex, for example a list or
> dictionary? Even in this case, is it safe to avoid locking on a shared
> variable if the operation on the variable is performed in a single Python
> instruction?

You can be confident that a single assignment will happen atomically.
Even if "self.cnt = i" requires multiple instructions to perform
(which it probably doesn't), there's still going to be some moment
before the change has happened at all, and then some moment when the
change has completely happened, and you won't get a context switch in
between.

This is NOT the case if you try to do an increment (eg "self.cnt +=
1"), but for what you're doing here, it should be fine.

That said, though, you may still find that a queue is better, as per
Thomas's suggestion.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: memory leak with re.match

[Mayling ge]

   Sorry. The code  here is just  to describe  the issue and  is just  pseudo
   code, please  forgive some  typo. I  list out  lines because  I need  line
   context.
   Sent from Mail Master
   On 07/05/2017 15:52, [1]Albert-Jan Roskam wrote:

 From: Python-list
  on behalf of
 Mayling ge 
 Sent: Tuesday, July 4, 2017 9:01 AM
 To: python-list
 Subject: memory leak with re.match

Hi,

My function is in the following way to handle file line by line.
 There are
multiple error patterns  defined and  need to apply  to each  line.
 I  use
multiprocessing.Pool to handle the file in block.

The memory usage increases to 2G for a 1G file. And stays in 2G even
 after
the file processing. File closed in the end.

If I comment  out the  call to re_pat.match,  memory usage  is
 normal  and
keeps under 100Mb.

am I using re in a wrong way? I cannot figure out a way to fix the
 memory
leak. And I googled .

def line_match(lines, errors)

   

lines = list(itertools.islice(fo, line_per_proc))

 ===> do you really need to listify the iterator?
if not lines:

break

result = p.apply_async(line_match, args=(errors, lines))

 ===> the signature of line_match is (lines, errors), in args you do
 (errors, lines)

References

   Visible links
   1. mailto:sjeik_ap...@hotmail.com
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[Thomas Nyberg]

On 07/05/2017 10:20 AM, Thomas Nyberg wrote:
>  [...snip...]

Btw I forgot to mention that you'd probably want to use q.get_nowait()
instead of q.get() in my code example if you don't want the main thread
to block (which what I think you want to avoid from your code example).

https://docs.python.org/3.7/library/queue.html#queue.Queue.get_nowait

Cheers,
Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[Chris Angelico]

On Wed, Jul 5, 2017 at 6:26 PM, Thomas Nyberg  wrote:
> I think the general rule would be that no it's not safe to skip the
> locks. It's true that with cpython, your method shouldn't run into
> problems, but that's just a quirk of how you're using it. I look at from
> another perspective, if it's true that no locks actually are necessary,
> then why are you using the shared variables in the first place. In this
> case, the information needs to be send from the worker thread to the
> main thread, but you don't need for any other threads to see it. This
> only really requires a single "channel" (e.g. a queue) and not for the
> variable to further exposed.

What would the lock surround? The purpose of a lock is to make an
atomic unit out of something that otherwise wouldn't be, but a single
store operation is already atomic. So you could do something like:

with lock():
self.cnt = self.cnt + 1

and then multiple threads could safely increment the counter, which
they otherwise couldn't (note that "self.cnt += 1" might be safe, but
probably wouldn't, so I use the longhand); but even with that form of
increment, it's only necessary if there are multiple writers. A
pub-sub model (one writer, any number of readers) only needs locks if
it's possible for the write itself to be half done, which can't happen
with a single operation.

I'm basing my information primarily on CPython here, where context
switches are protected by the language interpreter and the GIL. But
other Python implementations have similar guarantees. If there's any
Python implementation in which a simple assignment can cause problems,
I'd call that a bug to be fixed - especially since, at the CPU level,
you can generally rely on a pointer-sized memory store being atomic.

But I do agree with the recommendation of the queue. That does make
things clearer.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[Thomas Nyberg]

On 07/05/2017 10:40 AM, Chris Angelico wrote:
> What would the lock surround?

Sorry yes I agree with you that no lock is needed in this method. I was
a bit confused by the code and probably was thinking something like "a
+= 1" in my head (even though that's not what he was doing...). Thanks
for clearing that up!

Regardless, depending upon what he means by this, he'll probably need
some form of syncronization at some point:

> What about if the variable is more complex, for example a list or
dictionary? Even in this case, is it safe to avoid locking on a shared
variable if the operation on the variable is performed in a single
Python instruction?

Cheers,
Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Generator - Provide parameters to url - requests

[Sayth Renshaw]

Thanks. 

I left "base" out as i was trying to remove as much uneeded code from example 
as possible. I had defined it as

base = datetime.datetime(2017,1,1)

Reading your code this sounds to simple :-).

def dates(first, numdays): 
# generate datetime objects for extra clarity 
# note there are no implicit arguments like `base` in your code 
for _ in range(numdays): 
yield first 
first += ONE_DAY 

Thanks

Sayth

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[Chris Angelico]

On Wed, Jul 5, 2017 at 6:57 PM, Thomas Nyberg  wrote:
> On 07/05/2017 10:40 AM, Chris Angelico wrote:
>> What would the lock surround?
>
> Sorry yes I agree with you that no lock is needed in this method. I was
> a bit confused by the code and probably was thinking something like "a
> += 1" in my head (even though that's not what he was doing...). Thanks
> for clearing that up!
>
> Regardless, depending upon what he means by this, he'll probably need
> some form of syncronization at some point:

It all depends on the meaning of "and control" from the OP. The
example code is all about getting status, which can always be done
atomically (if you need a complex object, you simply construct a new
one every time, atomically place it in the shared location, and on
reading, always take a reference atomically before digging deeper into
it), but other types of command-and-control would need more
flexibility.

As a general rule, I try to avoid using locks *per se* in my code.
It's usually clearer to have clear communication channels, generally
unidirectional. Locks are reserved for the weird cases where you're
doing something really unusual.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

EuroPython 2017: Beginners’ Day workshop revived

[M.-A. Lemburg]

Our Beginners’ Day host Harry Percival cannot attend EuroPython due to
personal reasons, but thanks to our brilliant community, we have
managed to find trainers who are willing to help out and run the
workshop:

 * Ilian Iliev
 * Juan Manuel Santos
 * Petr Viktorin
 * Lasse Schuirmann
 * Michał Bultrowicz

A big thanks for the quick offers of help. So once more, we’re pleased
to present the...

 * Beginners’ Day Workshop *

https://ep2017.europython.eu/en/events/beginners-day/


We will have a Beginners’ Day workshop, on Sunday, July 9th, from
10:00 until 17:00, at the Palacongressi di Rimini (Via della Fiera 23,
Rimini), the same location as the main conference.

The session will be presented in English (although a few of the
coaches do speak other languages as well).

Please bring your laptop, as a large part of the day will be devoted
to learning Python on your own PC.

For more information and the session list, please see the Beginners’
Day workshop page on our website:

https://ep2017.europython.eu/en/events/beginners-day/


Enjoy,
--
EuroPython 2017 Team
http://ep2017.europython.eu/
http://www.europython-society.org/

PS: Please forward or retweet to help us reach all interested parties:
https://twitter.com/europython/status/882532425636753408
Thanks.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: memory leak with re.match

[Peter Otten]

Mayling ge wrote:

> Sorry. The code  here is just  to describe  the issue and  is just  pseudo
> code, 

That is the problem with your post. It's too vague for us to make sense of 
it.

Can you provide a minimal example that shows what you think is a "memory 
leak"? Then we can either help you avoid storing extra stuff or confirm an 
actual leak and help you prepare a bug report.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Generator - Provide parameters to url - requests

[Peter Otten]

Sayth Renshaw wrote:

> Thanks.
> 
> I left "base" out as i was trying to remove as much uneeded code from
> example as possible. I had defined it as
> 
> base = datetime.datetime(2017,1,1)

You actually did provide that line in your post.

> Reading your code this sounds to simple :-).
> 
> def dates(first, numdays):
> # generate datetime objects for extra clarity
> # note there are no implicit arguments like `base` in your code
> for _ in range(numdays):
> yield first
> first += ONE_DAY
> 
> Thanks

You could write the above generator

ONE_DAY = datetime.timedelta(days=1)
base = datetime.datetime(2017, 1, 1)
def dates(numdays):
date = base
for _ in range(numdays):
yield date
date += ONE_DAY

but this is bad design. 

You get the most predictable output when you write functions in such a way 
that the result only depends on the function's arguments. Such functions are 
called "pure", and are much easier to reason about and to cover with unit 
tests.

As Python does not provide constants this is sometimes a judgment call:
While ONE_DAY will never be changed base is likely to be changed to

base = datetime.datetime(2018, 1, 1)

next year. Therefore it should be an argument. If you were to change the 
generator to support varying intervals the signature should be changed, too, 
to

def dates(first, numdays, daystep=1):
# ...

or similar.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[Rhodri James]

On 05/07/17 09:26, Chris Angelico wrote:

On Wed, Jul 5, 2017 at 5:56 PM, pozz  wrote:

It seems it works, but I'm not sure it is the correct way to share the
variable self.cnt. It is only written in the long thread and only read in
the main thread.
Could a single Python instruction be interrupted (in this case, self.cnt =
i)? Should I use a locking mechanism when reading/writing?

What about if the variable is more complex, for example a list or
dictionary? Even in this case, is it safe to avoid locking on a shared
variable if the operation on the variable is performed in a single Python
instruction?


You can be confident that a single assignment will happen atomically.
Even if "self.cnt = i" requires multiple instructions to perform
(which it probably doesn't), there's still going to be some moment
before the change has happened at all, and then some moment when the
change has completely happened, and you won't get a context switch in
between.


Is there a definition of what is or isn't atomic behaviour anywhere?  As 
an embedded C programmer I definitely wouldn't assume that a high-level 
assignment (in all its ref-counting glory) would be atomic without some 
hint of proof :-)


--
Rhodri James *-* Kynesim Ltd
--
https://mail.python.org/mailman/listinfo/python-list

School Management System in Python

[Sam Chats]

Feel free to comment on my high school project. I really enjoyed building it 
and it is the biggest project I've developed so far
(in terms of lines of code). All you need to do is to run the S-Koo-L.py script.

I've built more eye-catchy things with less code (see my other repos), but this 
does a lot more than perhaps all of them,
while having a command line REPL interface. One neat thing I wanted to mention 
is that this project has absolutely no
third-party dependencies.

https://github.com/schedutron/S-Koo-L

Sam Chats
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[Thomas Nyberg]

On 07/05/2017 01:31 PM, Sam Chats wrote:
> Feel free to comment on my high school project. I really enjoyed building it 
> and it is the biggest project I've developed so far
> (in terms of lines of code). All you need to do is to run the S-Koo-L.py 
> script.
> 
> I've built more eye-catchy things with less code (see my other repos), but 
> this does a lot more than perhaps all of them,
> while having a command line REPL interface. One neat thing I wanted to 
> mention is that this project has absolutely no
> third-party dependencies.
> 
> https://github.com/schedutron/S-Koo-L
> 
> Sam Chats
> 

Just a few comments:

1. You probably shouldn't add *.pyc files like that to your source
control since they're binary files.

2. I don't think you should be dumping pickeled objects to the log file.
It's much nicer if you just write text to it. For example, you're
pickling a date object to the log file, but you could instead just write
the string corresponding to it.

3. Personally I think you should probably stop using pickle files
entirely here. It seems like you could just as easily store them in a
text readable format and then load the data (say names) in line by line.
You might want to look into the csv module.

4. You probably want some documentation walking through what this should
be doing.

Cheers,
Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[Sam Chats]

Check message

Sorry for this message.

Sam
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[Sam Chats]

Thanks for your suggestions. I would've not used pickle had I been aware about 
other tools while developing this.
I was thinking about migrating to sqlite3. How about that? And yes, I need more 
comprehanesive documentation.
Will work on that soon.

Thanks,
Sam Chats
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[Peter Otten]

Chris Angelico wrote:

> You can be confident that a single assignment will happen atomically.
> Even if "self.cnt = i" requires multiple instructions to perform

For name binding

cnt = i

maybe, but

self.cnt = i

can execute arbitrary Python code (think __setattr__()). With threads I'd 
rather play it safe.


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[YOUR_NAME_HERE]

Thanks for your suggestions. I would've not used pickle had I been aware about 
other tools while developing this. 
I was thinking about migrating to sqlite3. How about that? And yes, I need more 
comprehanesive documentation. 
Will work on that soon. 

Thanks, 
Sam Chats
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[Thomas Nyberg]

On 07/05/2017 02:14 PM, Sam Chats wrote:
> Thanks for your suggestions. I would've not used pickle had I been aware 
> about other tools while developing this.
> I was thinking about migrating to sqlite3. How about that? And yes, I need 
> more comprehanesive documentation.
> Will work on that soon.
> 
> Thanks,
> Sam Chats
> 
Personally I prefer text formats until I have some need to switch. That
way I can look at files directly instead of needing to unpickle or to
load up sqlite or whatever. It just seems like overkill when it's
unnecessary. Depending upon how you are updating data, using sqlite or
some database might make sense, but if you're just reading in or writing
out entire text files, then I'd just recommend skipping sqlite and
instead just writing to the files directly.

Cheers,
Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[Chris Angelico]

On Wed, Jul 5, 2017 at 10:14 PM, Peter Otten <__pete...@web.de> wrote:
> Chris Angelico wrote:
>
>> You can be confident that a single assignment will happen atomically.
>> Even if "self.cnt = i" requires multiple instructions to perform
>
> For name binding
>
> cnt = i
>
> maybe, but
>
> self.cnt = i
>
> can execute arbitrary Python code (think __setattr__()). With threads I'd
> rather play it safe.

Sure, it _could_ execute arbitrary code, but the most likely case is
that at its core, it's still going to execute a single assignment
operation. And if it doesn't, then that's the place where you'd need
the lock.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[YOUR_NAME_HERE]

I can use either tsv or csv. Which one would be better?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[Chris Angelico]

On Wed, Jul 5, 2017 at 8:40 PM, Rhodri James  wrote:
> On 05/07/17 09:26, Chris Angelico wrote:
>>
>> On Wed, Jul 5, 2017 at 5:56 PM, pozz  wrote:
>>>
>>> It seems it works, but I'm not sure it is the correct way to share the
>>> variable self.cnt. It is only written in the long thread and only read in
>>> the main thread.
>>> Could a single Python instruction be interrupted (in this case, self.cnt
>>> =
>>> i)? Should I use a locking mechanism when reading/writing?
>>>
>>> What about if the variable is more complex, for example a list or
>>> dictionary? Even in this case, is it safe to avoid locking on a shared
>>> variable if the operation on the variable is performed in a single Python
>>> instruction?
>>
>>
>> You can be confident that a single assignment will happen atomically.
>> Even if "self.cnt = i" requires multiple instructions to perform
>> (which it probably doesn't), there's still going to be some moment
>> before the change has happened at all, and then some moment when the
>> change has completely happened, and you won't get a context switch in
>> between.
>
>
> Is there a definition of what is or isn't atomic behaviour anywhere?  As an
> embedded C programmer I definitely wouldn't assume that a high-level
> assignment (in all its ref-counting glory) would be atomic without some hint
> of proof :-)

In CPython, yes, because thread switching always takes place between
Python bytecode instructions.

In other Pythons, I don't know about actual guarantees, but if it's
possible for a simple assignment to NOT be atomic, it would lead to
internal corruption (eg messing with garbage collection and/or memory
pointers), so it would be the job of the interpreter, not the Python
code itself - because there'd be no way to reliably do *anything*
without some sort of lock... including acquiring a lock. So it *has*
to be solved at a lower level.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[Tim Golden]

On 05/07/2017 13:49, Thomas Nyberg wrote:

On 07/05/2017 02:14 PM, Sam Chats wrote:

Thanks for your suggestions. I would've not used pickle had I been aware about 
other tools while developing this.
I was thinking about migrating to sqlite3. How about that? And yes, I need more 
comprehanesive documentation.
Will work on that soon.

Thanks,
Sam Chats


Personally I prefer text formats until I have some need to switch. That
way I can look at files directly instead of needing to unpickle or to
load up sqlite or whatever. It just seems like overkill when it's
unnecessary. Depending upon how you are updating data, using sqlite or
some database might make sense, but if you're just reading in or writing
out entire text files, then I'd just recommend skipping sqlite and
instead just writing to the files directly.


There's been some discussion recently on the Computing At School forums 
here in the UK where at least one teacher explained that they taught 
pickle in the way it's being used here essentially because it's really 
simple: you just through your object at .dump/.load and it's done. 
Otherwise you have to roll your own serialisation of some sort. Which 
might be simple but is yet another thing to introduce into an already 
busy curriculum.


TJG

--
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[YOUR_NAME_HERE]

On Wed, 5 Jul 2017 13:02:36 + (UTC) YOUR_NAME_HERE wrote:
> I can use either tsv or csv. Which one would be better?


Some people complain that tsv has problems, so maybe csv would be the way to go.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[Thomas Nyberg]

On 07/05/2017 03:18 PM, YOUR_NAME_HERE wrote:
> On Wed, 5 Jul 2017 13:02:36 + (UTC) YOUR_NAME_HERE wrote:
>> I can use either tsv or csv. Which one would be better?
> 
> 
> Some people complain that tsv has problems, so maybe csv would be the way to 
> go.
> 
I almost always use csv personally, but it's a preference. I'm not sure
what the problems are you're refering to, but I guess that points to
using commas as well. Either way, it's not hard to switch between the two:

import csv

# Using regular commas
with open('outfile.csv', 'w') as outfile:
writer = csv.writer(outfile)
writer.writerow(range(5))

# Using tabs
with open('outfile.tsv', 'w') as outfile:
writer = csv.writer(outfile, delimiter='\t')
writer.writerow(range(5))

Cheers,
Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[Thomas Nyberg]

On 07/05/2017 02:56 PM, Tim Golden wrote:
> There's been some discussion recently on the Computing At School forums
> here in the UK where at least one teacher explained that they taught
> pickle in the way it's being used here essentially because it's really
> simple: you just through your object at .dump/.load and it's done.
> Otherwise you have to roll your own serialisation of some sort. Which
> might be simple but is yet another thing to introduce into an already
> busy curriculum.
> 
> TJG
> 

That's certainly good reasoning that I hadn't thought of. Makes a bit
more sense as to why I see it used like this so often. And it certainly
does make sense to not get bogged down in every different detail when
beginning so I see why one would teach this way. In fact I feel like it
could have been a good idea in the different times I've taught people
this sort of thing...

Thanks for the perspective!

Cheers,
Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[Sam Chats]

On Wednesday, July 5, 2017 at 6:56:06 PM UTC+5:30, Thomas Nyberg wrote:
> On 07/05/2017 03:18 PM, YOUR_NAME_HERE wrote:
> > On Wed, 5 Jul 2017 13:02:36 + (UTC) YOUR_NAME_HERE wrote:
> >> I can use either tsv or csv. Which one would be better?
> > 
> > 
> > Some people complain that tsv has problems, so maybe csv would be the way 
> > to go.
> > 
> I almost always use csv personally, but it's a preference. I'm not sure
> what the problems are you're refering to, but I guess that points to
> using commas as well. Either way, it's not hard to switch between the two:
> 
> import csv
> 
> # Using regular commas
> with open('outfile.csv', 'w') as outfile:
> writer = csv.writer(outfile)
> writer.writerow(range(5))
> 
> # Using tabs
> with open('outfile.tsv', 'w') as outfile:
> writer = csv.writer(outfile, delimiter='\t')
> writer.writerow(range(5))
> 
> Cheers,
> Thomas

Just curious, is it better, performance wise, to read from a text file (css or 
tsv) compared to reading from a binary pickle file?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[YOUR_NAME_HERE]

On Wed, 5 Jul 2017 15:28:51 +0200, Thomas Nyberg wrote:
> On 07/05/2017 03:18 PM, YOUR_NAME_HERE wrote:
> > On Wed, 5 Jul 2017 13:02:36 + (UTC) YOUR_NAME_HERE wrote:
> >> I can use either tsv or csv. Which one would be better?
> > 
> > 
> > Some people complain that tsv has problems, so maybe csv would be the way 
> > to go.
> > 
> I almost always use csv personally, but it's a preference. I'm not sure
> what the problems are you're refering to, but I guess that points to
> using commas as well. Either way, it's not hard to switch between the two:
> 
> import csv
> 
> # Using regular commas
> with open('outfile.csv', 'w') as outfile:
> writer = csv.writer(outfile)
> writer.writerow(range(5))
> 
> # Using tabs
> with open('outfile.tsv', 'w') as outfile:
> writer = csv.writer(outfile, delimiter='  ')
> writer.writerow(range(5))
> 
> Cheers,
> Thomas

Hey that was simple enough! Thanks for the code! I was also considering the use 
of JSON. Which one would be better?

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[eryk sun]

On Wed, Jul 5, 2017 at 12:14 PM, Peter Otten <__pete...@web.de> wrote:
> Chris Angelico wrote:
>
>> You can be confident that a single assignment will happen atomically.
>> Even if "self.cnt = i" requires multiple instructions to perform
>
> For name binding
>
> cnt = i
>
> maybe, but
>
> self.cnt = i
>
> can execute arbitrary Python code (think __setattr__()). With threads I'd
> rather play it safe.

Computed properties that require setting multiple values are a problem
that may require locking.

Assignment of a single variable in an unoptimized namespace isn't
completely immune to this -- in principle. Think __setitem__,
__getitem__, __hash__, and __eq__. For example:

>>> exec('cnt = 42; cnt = 43; cnt', NoisyNS())
__setitem__('cnt', 42)
__hash__('cnt')
__setitem__('cnt', 43)
__hash__('cnt')
__eq__('cnt', 'cnt')
__getitem__('cnt')
__eq__('cnt', 'cnt')

It's reasonable to assume a namespace uses a built-in dict and str
keys (names) -- or at least types that don't do anything unusual that
introduces concurrency problems.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: EuroPython 2017: Free Intel Distribution for Python

[Grant Edwards]

On 2017-07-04, MRAB  wrote:
> On 2017-07-04 23:05, Terry Reedy wrote:
>> On 7/4/2017 10:22 AM, M.-A. Lemburg wrote:
>>
>>> http://blog.europython.eu/post/162590522362/europython-2017-free-intel-distribution-for
>> 
>> I looked but did not find the most important thing.
>> What version of Python?
>> 
>  From a brief search it appears to be Python 2.7 and Python 3.5.

Just click on the link on the europython blog page.  There's all sorts
of info (release notes, package lists, etc.):

https://software.intel.com/en-us/distribution-for-python

-- 
Grant Edwards   grant.b.edwardsYow! Of course, you
  at   UNDERSTAND about the PLAIDS
  gmail.comin the SPIN CYCLE --

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[YOUR_NAME_HERE]

On Wed, 5 Jul 2017 06:34:26 -0700 (PDT), Sam Chats wrote:
> On Wednesday, July 5, 2017 at 6:56:06 PM UTC+5:30, Thomas Nyberg wrote:
> > On 07/05/2017 03:18 PM, YOUR_NAME_HERE wrote:
> > > On Wed, 5 Jul 2017 13:02:36 + (UTC) YOUR_NAME_HERE wrote:
> > >> I can use either tsv or csv. Which one would be better?
> > > 
> > > 
> > > Some people complain that tsv has problems, so maybe csv would be the way 
> > > to go.
> > > 
> > I almost always use csv personally, but it's a preference. I'm not sure
> > what the problems are you're refering to, but I guess that points to
> > using commas as well. Either way, it's not hard to switch between the two:
> > 
> > import csv
> > 
> > # Using regular commas
> > with open('outfile.csv', 'w') as outfile:
> > writer = csv.writer(outfile)
> > writer.writerow(range(5))
> > 
> > # Using tabs
> > with open('outfile.tsv', 'w') as outfile:
> > writer = csv.writer(outfile, delimiter='')
> > writer.writerow(range(5))
> > 
> > Cheers,
> > Thomas
> 
> Just curious, is it better, performance wise, to read from a text file (css 
> or tsv) compared to reading from a binary pickle file?
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[Chris Angelico]

On Wed, Jul 5, 2017 at 11:50 PM, eryk sun  wrote:
> Assignment of a single variable in an unoptimized namespace isn't
> completely immune to this -- in principle. Think __setitem__,
> __getitem__, __hash__, and __eq__. For example:
>
> >>> exec('cnt = 42; cnt = 43; cnt', NoisyNS())
> __setitem__('cnt', 42)
> __hash__('cnt')
> __setitem__('cnt', 43)
> __hash__('cnt')
> __eq__('cnt', 'cnt')
> __getitem__('cnt')
> __eq__('cnt', 'cnt')
>
> It's reasonable to assume a namespace uses a built-in dict and str
> keys (names) -- or at least types that don't do anything unusual that
> introduces concurrency problems.

This doesn't show a potential concurrency problem. Calculating a hash
on "cnt" is independent of other threads; the actual work of
__setitem__ isn't visible in this view. There certainly are places
where a context switch could cause problems (eg resizing the dict),
but they're the dict's problem, not your Python program's - because
there's no way you could acquire a lock without working with these
same issues.

This, btw, is why CPython has the GIL - it's the most efficient way to
solve all these problems. Removing the GIL means putting explicit
locks around these kinds of risk points, and explicit locks are slow,
so one lock (the GIL) is way faster than many locks.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Meta: double posts [Was: School Management System in Python]

[Tim Golden]

Are you posting both to python-list@python.org and to comp.lang.python 
-- and under different names?


If you are, please use one or the other: they mirror both ways, and 
we're seeing double posts which the gateway thinks are different because 
of a different sending address.


Thanks

TJG
--
https://mail.python.org/mailman/listinfo/python-list

How to write raw strings to Python

[Sam Chats]

I want to write, say, 'hello\tworld' as-is to a file, but doing 
f.write('hello\tworld') makes the file
look like:
hello   world

How can I fix this? Thanks in advance.

Sam
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[Thomas Nyberg]

On 07/05/2017 03:34 PM, Sam Chats wrote:
> Just curious, is it better, performance wise, to read from a text file (css 
> or tsv) compared to reading from a binary pickle file?
> 
I honestly don't know. You should probably measure it if you're
wondering. However, I don't think it's worth thinking about that at this
stage. It's always best to optimize for cognitive overhead (i.e. to make
code simpler and easier to understand) before making the code more
complicated through optimizations.

Cheers,
Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[eryk sun]

On Wed, Jul 5, 2017 at 2:03 PM, Chris Angelico  wrote:
> On Wed, Jul 5, 2017 at 11:50 PM, eryk sun  wrote:
>> Assignment of a single variable in an unoptimized namespace isn't
>> completely immune to this -- in principle. Think __setitem__,
>> __getitem__, __hash__, and __eq__. For example:
>>
>> >>> exec('cnt = 42; cnt = 43; cnt', NoisyNS())
>> __setitem__('cnt', 42)
>> __hash__('cnt')
>> __setitem__('cnt', 43)
>> __hash__('cnt')
>> __eq__('cnt', 'cnt')
>> __getitem__('cnt')
>> __eq__('cnt', 'cnt')
>>
>> It's reasonable to assume a namespace uses a built-in dict and str
>> keys (names) -- or at least types that don't do anything unusual that
>> introduces concurrency problems.
>
> This doesn't show a potential concurrency problem. Calculating a hash
> on "cnt" is independent of other threads; the actual work of
> __setitem__ isn't visible in this view. There certainly are places
> where a context switch could cause problems (eg resizing the dict),
> but they're the dict's problem, not your Python program's - because
> there's no way you could acquire a lock without working with these
> same issues.

The work in the above special methods is arbitrary bytecode that could
do anything, and there's nothing to prevent a context switch here. The
GIL provides no protection here. In principle it could be a problem,
but in practice it is not.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to write raw strings to Python

[Stephen Tucker]

Sam,

You use

r'hello\tworld'

The r in front of the string stands for raw and it is intended to switch
off the normal escape function of a backslash. It works fine so long as the
string doesn't end with a backslash. If you end the string with a
backslash, as in

r'hello\tworld\'

you get an error message.

Stephen.



Virus-free.
www.avast.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Wed, Jul 5, 2017 at 3:37 PM, Sam Chats  wrote:

> I want to write, say, 'hello\tworld' as-is to a file, but doing
> f.write('hello\tworld') makes the file
> look like:
> hello   world
>
> How can I fix this? Thanks in advance.
>
> Sam
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to write raw strings to Python

[Stephen Tucker]

Sam,

You use

f.write(r'hello\tworld')

The r in front of the string stands for raw and is intended to switch off
the escape function of the backslash in the string. It works fine so long
as the string doesn't end with a backslash, as in

f.write('hello\tworld\')

If you try this, you get an error message.

Stephen.




Virus-free.
www.avast.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Wed, Jul 5, 2017 at 3:37 PM, Sam Chats  wrote:

> I want to write, say, 'hello\tworld' as-is to a file, but doing
> f.write('hello\tworld') makes the file
> look like:
> hello   world
>
> How can I fix this? Thanks in advance.
>
> Sam
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[Christopher Reimer]

On Jul 5, 2017, at 6:34 AM, Sam Chats  wrote: 
> Just curious, is it better, performance wise, to read from a text file (css 
> or tsv) compared to reading from a binary pickle file?

I prefer CSV because I can load the file into Microsoft Excel and do a quick 
search.

Chris R.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[Thomas Nyberg]

On 07/05/2017 03:30 PM, YOUR_NAME_HERE wrote:
> Hey that was simple enough! Thanks for the code! I was also considering the 
> use of JSON. Which one would be better?
> 
If you have hierarchical data best described by dicts/lists (in the
python meaning), then json isn't a bad approach. But if you just have
flat data (like name, job, address, etc.) where you just have one row
per person, then something like a csv is probably easier. The best
choice depends on your specific use case.

Cheers,
Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to write raw strings to Python

[Sam Chats]

On Wednesday, July 5, 2017 at 8:22:13 PM UTC+5:30, Stephen Tucker wrote:
> Sam,
> 
> You use
> 
> f.write(r'hello\tworld')
> 
> The r in front of the string stands for raw and is intended to switch off
> the escape function of the backslash in the string. It works fine so long
> as the string doesn't end with a backslash, as in
> 
> f.write('hello\tworld\')
> 
> If you try this, you get an error message.
> 
> Stephen.
> 
> 
> 
> 
> Virus-free.
> www.avast.com
> 
> <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> 
> On Wed, Jul 5, 2017 at 3:37 PM, Sam Chats  wrote:
> 
> > I want to write, say, 'hello\tworld' as-is to a file, but doing
> > f.write('hello\tworld') makes the file
> > look like:
> > hello   world
> >
> > How can I fix this? Thanks in advance.
> >
> > Sam
> > --
> > https://mail.python.org/mailman/listinfo/python-list
> >

Thanks, but I've tried something similar. Actually, I want to convert a string 
which I receive from a NNTP server to a raw string. So if I try something like:
raw = r"%s" % string_from_server

It doesn't work.

Regards,
Sam
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to write raw strings to Python

[Thomas Jollans]

On 2017-07-05 17:09, Sam Chats wrote:
> 
> Thanks, but I've tried something similar. Actually, I want to convert a 
> string which I receive from a NNTP server to a raw string. So if I try 
> something like:
> raw = r"%s" % string_from_server
> 

You may me looking for repr()



-- Thomas
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Meta: double posts [Was: School Management System in Python]

[saurabh chaturvedi]

No, I am not. I'm not posting to Python-list@python.org. However, I'm using
my own NNTP client I built as an exercise. Maybe it has a bug. I'll work on
it.

Thanks!
Sam

On Wed, Jul 5, 2017 at 7:49 PM, Tim Golden  wrote:

> Are you posting both to python-list@python.org and to comp.lang.python
> -- and under different names?
>
> If you are, please use one or the other: they mirror both ways, and
> we're seeing double posts which the gateway thinks are different because
> of a different sending address.
>
> Thanks
>
> TJG
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to write raw strings to Python

[Grant Edwards]

On 2017-07-05, Sam Chats  wrote:

> I want to write, say, 'hello\tworld' as-is to a file, but doing
> f.write('hello\tworld') makes the file look like:
[...]
> How can I fix this?

That depends on what you mean by "as-is".

Seriously.

Do you want the single quotes in the file?  Do you want the backslash
and 't' character in the file?

When you post a question like this it helps immensely to provide an
example of the output you desire.

-- 
Grant Edwards   grant.b.edwardsYow! Is it 1974?  What's
  at   for SUPPER?  Can I spend
  gmail.commy COLLEGE FUND in one
   wild afternoon??

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[Chris Angelico]

On Thu, Jul 6, 2017 at 12:39 AM, eryk sun  wrote:
>> This doesn't show a potential concurrency problem. Calculating a hash
>> on "cnt" is independent of other threads; the actual work of
>> __setitem__ isn't visible in this view. There certainly are places
>> where a context switch could cause problems (eg resizing the dict),
>> but they're the dict's problem, not your Python program's - because
>> there's no way you could acquire a lock without working with these
>> same issues.
>
> The work in the above special methods is arbitrary bytecode that could
> do anything, and there's nothing to prevent a context switch here. The
> GIL provides no protection here. In principle it could be a problem,
> but in practice it is not.

But what could it do? Most likely, it's going to end up mutating a
dict (the core type), so unless the __setitem__ is itself maintaining
complex state that needs a lock, all you've done is move the problem
around, and the same solutions work.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: how to add new tuple as key in dictionary?

[John Gordon]

In <344c23f8-f440-4c23-a5e6-9f93b0145...@googlegroups.com> Ho Yeung Lee 
 writes:

> I find that list can not be key in dictionary
> then find tuple can be as key

> but when I add new tuple as key , got error in python 2.7

> groupkey = {(0,0): []}
> groupkey[tuple([0,3])] = groupkey[tuple([0,3])] + [[0,1]]

The right-hand side of your expression is rightly complaining that
groupkey[(0,3)] doesn't exist.

Would you expect to say

a = a + 1

When a doesn't exist?  Your code tries to do much the same thing.

-- 
John Gordon   A is for Amy, who fell down the stairs
gor...@panix.com  B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[eryk sun]

On Wed, Jul 5, 2017 at 4:04 PM, Chris Angelico  wrote:
> On Thu, Jul 6, 2017 at 12:39 AM, eryk sun  wrote:
>>> This doesn't show a potential concurrency problem. Calculating a hash
>>> on "cnt" is independent of other threads; the actual work of
>>> __setitem__ isn't visible in this view. There certainly are places
>>> where a context switch could cause problems (eg resizing the dict),
>>> but they're the dict's problem, not your Python program's - because
>>> there's no way you could acquire a lock without working with these
>>> same issues.
>>
>> The work in the above special methods is arbitrary bytecode that could
>> do anything, and there's nothing to prevent a context switch here. The
>> GIL provides no protection here. In principle it could be a problem,
>> but in practice it is not.
>
> But what could it do? Most likely, it's going to end up mutating a
> dict (the core type), so unless the __setitem__ is itself maintaining
> complex state that needs a lock, all you've done is move the problem
> around, and the same solutions work.

That was my point. A namespace mapping could override __setitem__ and
__getitem__ to implement a name as something like a computed property
that's based on multiple values. Then if __setitem__ gets interrupted
in the middle of updating this set of values, another thread that gets
the computed 'property' will see a bad state. The GIL doesn't help. It
would need locking to make accessing the 'property' work as an atomic
operation, just like the case with regular properties. Again, I have
never seen anything like this in practice.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[Chris Angelico]

On Thu, Jul 6, 2017 at 2:24 AM, eryk sun  wrote:
>> But what could it do? Most likely, it's going to end up mutating a
>> dict (the core type), so unless the __setitem__ is itself maintaining
>> complex state that needs a lock, all you've done is move the problem
>> around, and the same solutions work.
>
> That was my point. A namespace mapping could override __setitem__ and
> __getitem__ to implement a name as something like a computed property
> that's based on multiple values. Then if __setitem__ gets interrupted
> in the middle of updating this set of values, another thread that gets
> the computed 'property' will see a bad state. The GIL doesn't help. It
> would need locking to make accessing the 'property' work as an atomic
> operation, just like the case with regular properties. Again, I have
> never seen anything like this in practice.

Sure it could. And if it does, it's *it's* responsibility to use locks
- not the caller's. From the caller's point of view, it's still a
single operation, and should remain so.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

get value from list using widget

[jorge . conrado]

Hi,

I would like know dow can I select and get the value from a list of 
values uisng widgets.


Thanks,

Conrado


--
https://mail.python.org/mailman/listinfo/python-list

Re: How to write raw strings to Python

[Sam Chats]

On Wednesday, July 5, 2017 at 9:09:18 PM UTC+5:30, Grant Edwards wrote:
> On 2017-07-05, Sam Chats  wrote:
> 
> > I want to write, say, 'hello\tworld' as-is to a file, but doing
> > f.write('hello\tworld') makes the file look like:
> [...]
> > How can I fix this?
> 
> That depends on what you mean by "as-is".
> 
> Seriously.
> 
> Do you want the single quotes in the file?  Do you want the backslash
> and 't' character in the file?
> 
> When you post a question like this it helps immensely to provide an
> example of the output you desire.
> 
> -- 
> Grant Edwards   grant.b.edwardsYow! Is it 1974?  What's
>   at   for SUPPER?  Can I spend
>   gmail.commy COLLEGE FUND in one
>wild afternoon??

I would add to add the following couple lines to a file:

for i in range(5):
print('Hello\tWorld')

Consider the leading whitespace to be a tab.

Thanks,
Saurabh
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: get value from list using widget

[John Gordon]

In  
jorge.conr...@cptec.inpe.br writes:

> Hi,

> I would like know dow can I select and get the value from a list of 
> values uisng widgets.

You haven't given us nearly enough detail to answer your question.

What do you mean by "widget"?  Do you mean HTML input elements such
as radio buttons or drop-down lists?  Or do you mean custom GUI widgets
such as you might create using the Tkinter package?

What do you mean by "select and get the value"?  Do you mean that you
want to present a widget to a user, that allows the user to select a
value?  Or do you actually mean that *you* want to select a value in
your code?

I could go on, but you get my point.  We need lots more information
before we can even begin to help you.

-- 
John Gordon   A is for Amy, who fell down the stairs
gor...@panix.com  B is for Basil, assaulted by bears
-- Edward Gorey, "The Gashlycrumb Tinies"

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to write raw strings to Python

[Jussi Piitulainen]

Sam Chats writes:

> On Wednesday, July 5, 2017 at 9:09:18 PM UTC+5:30, Grant Edwards wrote:
>> On 2017-07-05, Sam Chats  wrote:
>> 
>> > I want to write, say, 'hello\tworld' as-is to a file, but doing
>> > f.write('hello\tworld') makes the file look like:
>> [...]
>> > How can I fix this?
>> 
>> That depends on what you mean by "as-is".
>> 
>> Seriously.
>> 
>> Do you want the single quotes in the file?  Do you want the backslash
>> and 't' character in the file?
>> 
>> When you post a question like this it helps immensely to provide an
>> example of the output you desire.
>
> I would add to add the following couple lines to a file:
>
> for i in range(5):
> print('Hello\tWorld')
>
> Consider the leading whitespace to be a tab.

import sys

lines = r'''
for line in range(5):
print('hello\tworld')
'''

print(lines.strip())

sys.stdout.write(lines.strip())
sys.stdout.write('\n')
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[Binary Boy]

On Wed, 5 Jul 2017 15:28:51 +0200, Thomas Nyberg wrote:
> On 07/05/2017 03:18 PM, YOUR_NAME_HERE wrote:
> > On Wed, 5 Jul 2017 13:02:36 + (UTC) YOUR_NAME_HERE wrote:
> >> I can use either tsv or csv. Which one would be better?
> > 
> > 
> > Some people complain that tsv has problems, so maybe csv would be the way 
> > to go.
> > 
> I almost always use csv personally, but it's a preference. I'm not sure
> what the problems are you're refering to, but I guess that points to
> using commas as well. Either way, it's not hard to switch between the two:
> 
> import csv
> 
> # Using regular commas
> with open('outfile.csv', 'w') as outfile:
> writer = csv.writer(outfile)
> writer.writerow(range(5))
> 
> # Using tabs
> with open('outfile.tsv', 'w') as outfile:
> writer = csv.writer(outfile, delimiter='\t')
> writer.writerow(range(5))
> 
> Cheers,
> Thomas

This will prove useful to me. Thanks!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to write raw strings to Python

[Binary Boy]

On Wed, 05 Jul 2017 20:37:38 +0300, Jussi Piitulainen wrote:
> Sam Chats writes:
> 
> > On Wednesday, July 5, 2017 at 9:09:18 PM UTC+5:30, Grant Edwards wrote:
> >> On 2017-07-05, Sam Chats  wrote:
> >> 
> >> > I want to write, say, 'hello\tworld' as-is to a file, but doing
> >> > f.write('hello\tworld') makes the file look like:
> >> [...]
> >> > How can I fix this?
> >> 
> >> That depends on what you mean by "as-is".
> >> 
> >> Seriously.
> >> 
> >> Do you want the single quotes in the file?  Do you want the backslash
> >> and 't' character in the file?
> >> 
> >> When you post a question like this it helps immensely to provide an
> >> example of the output you desire.
> >
> > I would add to add the following couple lines to a file:
> >
> > for i in range(5):
> > print('Hello\tWorld')
> >
> > Consider the leading whitespace to be a tab.
> 
> import sys
> 
> lines = r'''
> for line in range(5):
> print('hello\tworld')
> '''
> 
> print(lines.strip())
> 
> sys.stdout.write(lines.strip())
> sys.stdout.write('\n')

Thanks! But will this work if I already have a string through a string 
variable, rather than using it directly linke you did (by declaring the lines 
variable)?
And, will this work while writing to files?

Sam
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Python threading and sharing variables

[eryk sun]

On Wed, Jul 5, 2017 at 5:06 PM, Chris Angelico  wrote:
> On Thu, Jul 6, 2017 at 2:24 AM, eryk sun  wrote:
>>> But what could it do? Most likely, it's going to end up mutating a
>>> dict (the core type), so unless the __setitem__ is itself maintaining
>>> complex state that needs a lock, all you've done is move the problem
>>> around, and the same solutions work.
>>
>> That was my point. A namespace mapping could override __setitem__ and
>> __getitem__ to implement a name as something like a computed property
>> that's based on multiple values. Then if __setitem__ gets interrupted
>> in the middle of updating this set of values, another thread that gets
>> the computed 'property' will see a bad state. The GIL doesn't help. It
>> would need locking to make accessing the 'property' work as an atomic
>> operation, just like the case with regular properties. Again, I have
>> never seen anything like this in practice.
>
> Sure it could. And if it does, it's *it's* responsibility to use locks
> - not the caller's. From the caller's point of view, it's still a
> single operation, and should remain so.

I feel it's necessary to emphasize that there's nothing inherent in
the CPython bytecode operations for storing and loading a name that
guarantees atomicity. The namespace has to provide the guarantee that
backs up a statement like "[y]ou can be confident that a single
assignment will happen atomically".
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: How to write raw strings to Python

[Jussi Piitulainen]

Binary Boy writes:

> On Wed, 05 Jul 2017 20:37:38 +0300, Jussi Piitulainen wrote:
>> Sam Chats writes:
>> 
>> > On Wednesday, July 5, 2017 at 9:09:18 PM UTC+5:30, Grant Edwards wrote:
>> >> On 2017-07-05, Sam Chats  wrote:
>> >> 
>> >> > I want to write, say, 'hello\tworld' as-is to a file, but doing
>> >> > f.write('hello\tworld') makes the file look like:
>> >> [...]
>> >> > How can I fix this?
>> >> 
>> >> That depends on what you mean by "as-is".
>> >> 
>> >> Seriously.
>> >> 
>> >> Do you want the single quotes in the file?  Do you want the backslash
>> >> and 't' character in the file?
>> >> 
>> >> When you post a question like this it helps immensely to provide an
>> >> example of the output you desire.
>> >
>> > I would add to add the following couple lines to a file:
>> >
>> > for i in range(5):
>> > print('Hello\tWorld')
>> >
>> > Consider the leading whitespace to be a tab.
>> 
>> import sys
>> 
>> lines = r'''
>> for line in range(5):
>> print('hello\tworld')
>> '''
>> 
>> print(lines.strip())
>> 
>> sys.stdout.write(lines.strip())
>> sys.stdout.write('\n')
>
> Thanks! But will this work if I already have a string through a string
> variable, rather than using it directly linke you did (by declaring
> the lines variable)?  And, will this work while writing to files?

Yes, it will work the same. Writing does not interpret the contents of
the string. Try it - replace sys.stdout above with your file object.

If you see a different result in your actual program, your string may be
different than you think. Investigate that.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Proper architecture

[Andrew Zyman]

Cameron,
 took me some time to get to this.
Waling down your comments:

>...Normally I would have your DB class represent an open (or openable, if
you wanted to defer that) database connection. So your main class would go:
> def __init__(self, ...other args...):
>   self.db = DB(location="blah.sqlite")

that was my intention with (classA)

self.DBConnection = sql3.Connection

What is puzzling me is what type should i assign to the self.DB _if_ i
were to make it (self.db) a class field?
I do not plan to have various DB connections, just one connection.

The rest of your remarks have been accepted :) Thank you very much!


On Sun, Jul 2, 2017 at 8:14 PM, Cameron Simpson  wrote:

> On 02Jul2017 11:02, Andrew Z  wrote:
>
>> I'd appreciate your suggestions for a better approach to the following
>> task.
>>
>> I have 2 files ( 2 classes). One (ClassA) has all logic related to the
>> main workflow of the program. Another (DB), I like to offload all
>> operations with a DB ( sql3 in this case).
>>
>> I'm trying to pass the connection to the main class, but having problems.
>> One of them, is i can't pass the conn as a parameter to the function in one
>> (ClassA.abc()), because i inherit it ( function abc() ).
>> I created a self.DBConnection field, but i'm not sure if i'm on the right
>> path...
>> Code is gutted to highlight the problem.
>>
>
> Unfortunately you have gutted the "writeTicks" method, making it harder to
> see your intent.
>
> You separation is ok, but normally one would try to entire conceal the
> unerlying db connection (the sqlite3 connection) from the man class. So you
> wouldn't "pass the connection to the main class".
>
> Normally I would have your DB class represent an open (or openable, if you
> wanted to defer that) database connection. So your main class would go:
>
>  def __init__(self, ...other args...):
>self.db = DB(location="blah.sqlite")
>
>  def abc(self, reqId: int):
>self.db.writeTicks(reqId)
>
> I wouldn't be passing in "self" (your ClassA instance) or
> self.DBconnection at all. You'd only pass "self" if the "DB" instance
> needed more information from ClassA; normally you'd just pass that
> information to writeTicks() along with reqId, so that the DB needs no
> special knowledge about ClassA.
>
> I've also got a bunch of fine grained remarks about your code that you can
> take or leave as you see fit:
>
> one.py:
>> from .DB import *
>>
>
> Try to avoid importing "*". It sucks all the names from "DB" into your own
> namespace. Arguably you only need the DB class itself - all the other
> functionality comes with it as methods on the class. So:
>
>  from DB import DB
>
> class ClassA(OtherObject):
>> def __init__(self):
>> self.DBConnection = sql3.Connection
>>
>
> It isn't obvious why you need this. In my example above I'd just make a DB
> instance and save it as self.db; unless you're controlling different
> backends that would be all you need.
>
> def abc(self, reqId: int):
>> DB.writeTicks(self,self.DBConnection,reqId))
>>
>
> Here you're calling the writeTicks method on the DB class itself. I
> wouldn't be making that a class method; I'd make it an instance method on a
> DB instance, so:
>
>  self.db.writeTicks(reqId)
>
> unless there's more to writeTicks (which you've left out).
>
> DB.py:
>>
>
> Try not to name modules that same as their classes - it leads to
> confusion. I'd call it "db.py" and make the earlier import:
>
>  from db import DB
>
> import sqlite3 as sql3
>>
>
> This feels like an pointless abbreviation.
>
> [...]
>
>> class DB(object):
>> db_location = ''
>> # db_location = '../DB/pairs.db'
>>
>
> db_location appears to be a class default. These are normally treats as
> one would a "constant" in other languages. Stylisticly, this normally means
> you'd write the name in upper case, eg:
>
>DEFAULT_DB_LOCATION = '../DB/pairs.db'
>
> def __init__(self, location='../DB/pairs.db'):
>> db_location = location
>>
>
> And using that would normally look like this:
>
>def __init__(self, location=None):
>if location is None:
>location = self.DEFAULT_DB_LOCATION
>
> print(current_fn_name(),' self.db_location =
>> {}'.format(db_location))
>> try:
>> with open(db_location) as file:
>> pass
>> except IOError as e:
>> print("Unable to locate the Db @
>> {}".format(db_location))
>>
>
> I'd just os.path.exists(db_location) myself, or outright make the db
> connection immediately.
>
> Also, and this actually is important, error messages should got the the
> program's standard error output (or to some logging system). So your print
> would look like:
>
>print("Unable to locate the Db @ {}".format(db_location),
> file=sys.stderr)
>
> Also, normal Python practie here would not be to issue an error message,
> but to raise an excep

Re: EuroPython 2017: Free Intel Distribution for Python

[Terry Reedy]

On 7/5/2017 9:51 AM, Grant Edwards wrote:

On 2017-07-04, MRAB  wrote:

On 2017-07-04 23:05, Terry Reedy wrote:

On 7/4/2017 10:22 AM, M.-A. Lemburg wrote:


http://blog.europython.eu/post/162590522362/europython-2017-free-intel-distribution-for


I looked but did not find the most important thing.
What version of Python?


And it is really is not there *on that page*.


  From a brief search it appears to be Python 2.7 and Python 3.5.


Just click on the link on the europython blog page.


What link?  The first screen has 7 links to EuroPython and no obvious 
links to Intel.


Oh, the headline is a cleverly disguised link that does not look like a
link until one mouses over it.  (And one a couple of screenfulls farther 
on.)


A writer who was trying to be informative rather than write a come-on-in 
like a used-car salesman would have given such essential information 
right up front.  A 2.7 compiler has 0 interest for me, so I scanned the 
page for numbers.


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: How to write raw strings to Python

[Pavol Lisy]

On 7/5/17, Binary Boy  wrote:
> On Wed, 05 Jul 2017 20:37:38 +0300, Jussi Piitulainen wrote:
>> Sam Chats writes:
>>
>> > On Wednesday, July 5, 2017 at 9:09:18 PM UTC+5:30, Grant Edwards wrote:
>> >> On 2017-07-05, Sam Chats  wrote:
>> >>
>> >> > I want to write, say, 'hello\tworld' as-is to a file, but doing
>> >> > f.write('hello\tworld') makes the file look like:
>> >> [...]
>> >> > How can I fix this?
>> >>
>> >> That depends on what you mean by "as-is".
>> >>
>> >> Seriously.
>> >>
>> >> Do you want the single quotes in the file?  Do you want the backslash
>> >> and 't' character in the file?
>> >>
>> >> When you post a question like this it helps immensely to provide an
>> >> example of the output you desire.
>> >
>> > I would add to add the following couple lines to a file:
>> >
>> > for i in range(5):
>> > print('Hello\tWorld')
>> >
>> > Consider the leading whitespace to be a tab.
>>
>> import sys
>>
>> lines = r'''
>> for line in range(5):
>> print('hello\tworld')
>> '''
>>
>> print(lines.strip())
>>
>> sys.stdout.write(lines.strip())
>> sys.stdout.write('\n')
>
> Thanks! But will this work if I already have a string through a string
> variable, rather than using it directly linke you did (by declaring the
> lines variable)?
> And, will this work while writing to files?
>
> Sam

If I understand you well then no.

>>> a = '%s' % 'a\tb'  # we have string with tab (similar as we expect from 
>>> NNTP server?)
>>> print(a)  # this is not what you like to have in file
a   b
>>> print(repr(a))  # maybe this is conversion you need
'a\tb'
>>> print(repr(a)[1:-1])   # or maybe this
a\tb
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: get value from list using widget

[Terry Reedy]

On 7/5/2017 12:34 PM, jorge.conr...@cptec.inpe.br wrote:

I would like know dow can I select and get the value from a list of 
values uisng widgets.


One way is to learn tkinter and then learn to use the Listbox widget. 
The doc references a couple of decent tutorial web sites.
Stackoverflow has many good tkinter examples (in the answers, not the 
questions ;-).


--
Terry Jan Reedy

--
https://mail.python.org/mailman/listinfo/python-list

Re: EuroPython 2017: Free Intel Distribution for Python

[Grant Edwards]

On 2017-07-05, Terry Reedy  wrote:
> On 7/5/2017 9:51 AM, Grant Edwards wrote:
>
>> Just click on the link on the europython blog page.
>
> What link?  The first screen has 7 links to EuroPython and no obvious 
> links to Intel.
>
> Oh, the headline is a cleverly disguised link that does not look like a
> link until one mouses over it.  (And one a couple of screenfulls farther 
> on.)

Yup. It's annoying.  That trick of hiding links has become quite
fashionable -- I don't know why.  Somebody must think it's cute or
clever.  Mostly, it's just annoying. :)

-- 
Grant Edwards   grant.b.edwardsYow! My mind is making
  at   ashtrays in Dayton ...
  gmail.com

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[Gregory Ewing]

On 07/05/2017 03:18 PM, YOUR_NAME_HERE wrote:


Some people complain that tsv has problems, so maybe csv would be the way to go.


The main downside to tsv is that it can be hard to deal with
in a text editor -- the difference between tabs and spaces
is not visually obvious.

The only reason I can think of to want to use tsv instead
of csv is that you can sometimes get away without having
to quote things that would need quoting in csv. But that's
not an issue in Python, since the csv module takes care of
all of that for you.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: EuroPython 2017: Free Intel Distribution for Python

[Gregory Ewing]

Grant Edwards wrote:

That trick of hiding links has become quite
fashionable -- I don't know why.


Probably the result of graphic arts people who think that appearance
is everything and don't really understand the web.

--
Greg
--
https://mail.python.org/mailman/listinfo/python-list

Re: School Management System in Python

[Tim Chase]

On 2017-07-06 11:47, Gregory Ewing wrote:
> The only reason I can think of to want to use tsv instead
> of csv is that you can sometimes get away without having
> to quote things that would need quoting in csv. But that's
> not an issue in Python, since the csv module takes care of
> all of that for you.

I work with thousands of CSV/TSV data files from dozens-to-hundreds
of sources (clients and service providers) and have never encountered
a 0x09-as-data needing to be escaped.  So my big reason for
preference is that people say "TSV" and I can work with it without a
second thought.

On the other hand, with "CSV", sometimes it's comma-delimited as it
says on the tin.  But sometimes it's pipe or semi-colon delimited
while still carrying the ".csv" extension.  And sometimes a
subset of values are quoted. Sometimes all the values are quoted.
Sometimes numeric values are quoted to distinguish between
numeric-looking-string and numeric-value.  Sometimes escaping is done
with backslashes before the quote-as-value character. Sometimes
escaping is done with doubling-up the quoting-character.  Sometimes
CR(0x0D) and/or NL(0x0A) characters are allowed within quoted values;
sometimes they're invalid.  Usually fields are quoted with
double-quotes; but sometimes they're single-quoted values.  Or
sometimes they're either, depending on the data (much like Python's
REPL prints string representations).

And while, yes, Python's csv module handles most of these with no
issues thanks to the "dialects" concept, I still have to determine
the dialect—sometimes by sniffing, sometimes by customer/vendor
specification—but it's not nearly as trivial as

  with open("file.txt", "rb") as fp:
for row in csv.DictReader(fp, delimiter='\t'):
  process(row)

because there's the intermediate muddling of dialect determination or
specification.

And that said, I have a particular longing for a world in which
people actually used the US/RS/GS/FS (Unit/Record/Group/File
separators; AKA 0x1f-0x1c) as defined in ASCII for exactly this
purpose.  Sigh.  :-)

-tkc










-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Privy: An easy, fast lib to password-protect your data

[ofekmeister]

> The person spamming right now would be you. You just posted a link,
> without any explanations, any marketing blurbs, nothing.

I've explained everything as succinctly as I can in the readme. Pasting bits of 
it here would not benefit anyone.

> Why would I use your tool instead of something established, that has
> been properly audited — say, PGP for example?

Did you read the page? PGP and Privy are used for different things. A key 
manager could, though, use Privy to store private keys.

> How do I know your one-man project has no security holes, backdoors,
> or other vulnerabilities? How do I know that the encryption method
> chosen by you is sound? If there is no leaked data?

Privy is a thin wrapper around Cryptography's (OpenSSL) Fernet interface 
https://github.com/pyca/cryptography/blob/master/src/cryptography/fernet.py and 
https://github.com/hynek/argon2_cffi which is simply a binding to 
https://github.com/p-h-c/phc-winner-argon2

Privy itself is really just 40 SLOC 
https://github.com/ofek/privy/blob/a3d4bdb24464ad85606c1ab5e78c58ae489b0569/privy/core.py#L42-L82

> And I really dislike the description of your project ...
> What does “password-protecting” mean? Why is this not “encrypting”?

This is encryption, but specifically by means of a password. This paradigm is 
often tricky to get correct. 
https://security.stackexchange.com/questions/88984/encrypting-with-passwords-encryption-of-key-vs-data

> How do you expect this to work with API keys?

Encrypted keys would likely be stored in a DB somehow. Check out 
https://github.com/fugue/credstash
-- 
https://mail.python.org/mailman/listinfo/python-list