Re: [Python-Dev] [Infrastructure] Snakebite build slaves and developer SSH/GPG public keys
On 23.08.2012 01:03, Nick Coghlan wrote: > currently, there's a security hole > in the fact that requests to change our registered ssh key for access > are not themselves authenticated electronically Indeed, we should start requesting birth certificates, else someone claiming to be Reinhold Birkenfeld may get commit access :-) Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] bug in tarfile module?
Hi All,
This feels like a bug, but just wanted to check here before filing a
report if I've missed something:
buzzkill$ python2.7
Enthought Python Distribution -- www.enthought.com
Version: 7.2-2 (32-bit)
Python 2.7.2 |EPD 7.2-2 (32-bit)| (default, Sep 7 2011, 09:16:50)
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "packages", "demo" or "enthought" for more information.
>>> import tarfile
>>> source = open('/src/Python-2.6.7.tgz', 'rb')
>>> tar = tarfile.open(fileobj=source, mode='r|*')
>>> member = tar.extractfile('Python-2.6.7/Lib/genericpath.py')
>>> data = member.read()
Traceback (most recent call last):
File "", line 1, in
File
"/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/tarfile.py",
line 815, in read
buf += self.fileobj.read()
File
"/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/tarfile.py",
line 735, in read
return self.readnormal(size)
File
"/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/tarfile.py",
line 742, in readnormal
self.fileobj.seek(self.offset + self.position)
File
"/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/tarfile.py",
line 554, in seek
raise StreamError("seeking backwards is not allowed")
tarfile.StreamError: seeking backwards is not allowed
The key is the "mode='r*|" which I understood to be specifically for
reading blocks from a stream without seeking that would cause problems.
I've reproduced on Py26 and Py27 on Mac OS X, and Py26 on SUSE.
Thoughts?
Chris
--
Simplistix - Content Management, Batch Processing & Python Consulting
- http://www.simplistix.co.uk
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Infrastructure] Snakebite build slaves and developer SSH/GPG public keys
On 23.08.2012 09:17, "Martin v. Löwis" wrote: On 23.08.2012 01:03, Nick Coghlan wrote: currently, there's a security hole in the fact that requests to change our registered ssh key for access are not themselves authenticated electronically Indeed, we should start requesting birth certificates, else someone claiming to be Reinhold Birkenfeld may get commit access :-) Of course, even then it might be a political problem for us if Barack Obama wants to join as a core contributor if he happens to be voted out of office and gets bored ;) cheers, Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Infrastructure] Snakebite build slaves and developer SSH/GPG public keys
On 23.08.2012 02:43, R. David Murray wrote: > On Thu, 23 Aug 2012 10:53:34 +1200, Noah Kantrowitz > wrote: >> For everyone with a record in the Chef server (read: everyone with SSH >> access to any of the PSF servers at OSL) I can easily give you automated >> access. Whats the easiest format? I can give you a Python script that will >> spit out files or JSON or more or less whatever else you want. > > That isn't going to be the right set of keys for Trent's purposes > (though it is likely to be a subset). The keyfile we use for the hg > repository is. ... for which it would be easiest if we give Trent access to the repository storing these keys. I'm a bit hesitant to put "public" keys into the real world-wide public, given the past history of easily-breakable public keys. For PGP, this is less of a concern than for SSH, since the threats are smaller (plus users where aware that they might have to publish the key when they created it). Regards, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Infrastructure] Snakebite build slaves and developer SSH/GPG public keys
On Thu, Aug 23, 2012 at 12:24:33AM -0700, "Martin v. Löwis" wrote: > On 23.08.2012 02:43, R. David Murray wrote: > > On Thu, 23 Aug 2012 10:53:34 +1200, Noah Kantrowitz > > wrote: > >> For everyone with a record in the Chef server (read: everyone with SSH > >> access to any of the PSF servers at OSL) I can easily give you automated > >> access. Whats the easiest format? I can give you a Python script that will > >> spit out files or JSON or more or less whatever else you want. > > > > That isn't going to be the right set of keys for Trent's purposes > > (though it is likely to be a subset). The keyfile we use for the hg > > repository is. > > ... for which it would be easiest if we give Trent access to the > repository storing these keys. > > I'm a bit hesitant to put "public" keys into the real world-wide > public, given the past history of easily-breakable public keys. > For PGP, this is less of a concern than for SSH, since the threats > are smaller (plus users where aware that they might have to publish > the key when they created it). Hmmm. So, from my perspective, I have the following goals: - Commit access to ssh://hg.python.org implies access to `ssh cpython@snakebite` via the exact same key. - No extra administrative overhead/burden on infrastructure@ (with regards to ssh key management, i.e. an entry in .ssh/ authorized_keys should be sufficient for implicit snakebite access). Factoring in your (valid) security concerns, here's my altered proposal: - Let's just call the repo 'snakebite', and have it accessible only to ssh committers, no public http access. Calling it something generic like 'keys' may invite phantom requirements like being able to store multiple identities/keys etc. I don't need that for snakebite; one ssh key and one optional gpg key is all I want :-) - Same repo layout as before -- GPG keys not required unless I need to send you something encrypted via e-mail (RDP is the only use case I can think of for this). - I'll whip up the glue to take our current .ssh/authz and dump it into the 'snakebite' repo. We can refine that process down the track (with automation and whatnot). If there are no objections I can take this offline with inf@. Trent. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Snakebite build slaves and developer SSH/GPG public keys
> For committers on other Python projects like Buildbot, Django and > Twisted that may be reading this -- yes, the plan is to give you > guys Snakebite access/slaves down the track too. I'll start looking > into that after I've finished setting up the remaining slaves for > Python. (Setting up a keys repo will definitely help (doesn't have > to be hg -- feel free to use svn/git/whatever, just try and follow > the same layout).) This is so great! I've been looking forward to this for a long time and kept visiting the site every now and then to see if there was any progress. I'd surely use this for psutil if you'll let me. Also, at some point I would suggest to introduce the possibility to donate some money in order to help supporting what I think must be a pretty complex infrastructure requiring a lot of resources, both in terms of hardware and time/labor. Regards --- Giampaolo http://code.google.com/p/pyftpdlib/ http://code.google.com/p/psutil/ http://code.google.com/p/pysendfile/ 2012/8/23 Trent Nelson : > Hi folks, > > I've set up a bunch of Snakebite build slaves over the past week. > One of the original goals was to provide Python committers with > full access to the slaves, which I'm still keen on providing. > > What's a nice simple way to achieve that in the interim? Here's > what I was thinking: > > - Create a new hg repo: hg.python.org/keys. > > - Committers can push to it just like any other repo (i.e. > same ssh/authz configuration as cpython). > > - Repo is laid out as follows: > keys/ > / > ssh (ssh public key) > gpg (gpg public key) > > - Prime the repo with the current .ssh/authorized_keys > (presuming you still use the --tunnel-user facility?). > > That'll provide me with everything I need to set up the relevant > .ssh/authorized_keys stuff on the Snakebite side. GPG keys will > be handy if I ever need to send passwords over e-mail (which I'll > probably have to do initially for those that want to RDP into the > Windows slaves). > > Thoughts? > > As for the slaves, here's what's up and running now: > > - AMD64 Mountain Lion [SB] > - AMD64 FreeBSD 8.2 [SB] > - AMD64 FreeBSD 9.1 [SB] > - AMD64 NetBSD 5.1.2 [SB] > - AMD64 OpenBSD 5.1 [SB] > - AMD64 DragonFlyBSD 3.0.2 [SB] > - AMD64 Windows Server 2008 R2 SP1 [SB] > - x86 NetBSD 5.1.2 [SB] > - x86 OpenBSD 5.1 [SB] > - x86 DragonFlyBSD 3.0.2 [SB] > - x86 Windows Server 2003 R2 SP2 [SB] > - x86 Windows Server 2008 R2 SP1 [SB] > > All the FreeBSD ones use ZFS, all the DragonFly ones use HAMMER. > DragonFly, NetBSD and OpenBSD are currently reporting all sorts > of weird and wonderful errors, which is partly why I want to set > up ssh access sooner rather than later. > > Other slaves on the horizon (i.e. hardware is up, OS is installed): > > - Windows 8 x64 (w/ VS2010 and VS2012) > - HP-UX 11iv2 PA-RISC > - HP-UX 11iv3 Itanium (64GB RAM) > - AIX 5.3 RS/6000 > - AIX 6.1 RS/6000 > - AIX 7.1 RS/6000 > - Solaris 9 SPARC > - Solaris 10 SPARC > > Nostalgia slaves that probably won't ever see green: > - IRIX 6.5.33 MIPS > - Tru64 5.1B Alpha > > If anyone wants ssh access now to the UNIX platforms in order to > debug/test, feel free to e-mail me directly with your ssh public > keys. > > For committers on other Python projects like Buildbot, Django and > Twisted that may be reading this -- yes, the plan is to give you > guys Snakebite access/slaves down the track too. I'll start looking > into that after I've finished setting up the remaining slaves for > Python. (Setting up a keys repo will definitely help (doesn't have > to be hg -- feel free to use svn/git/whatever, just try and follow > the same layout).) > > Regards, > > Trent "that-took-a-bit-longer-than-expected" Nelson. > ___ > Python-Dev mailing list > [email protected] > http://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > http://mail.python.org/mailman/options/python-dev/g.rodola%40gmail.com ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bug in tarfile module?
Chris Withers wrote:
> Hi All,
>
> This feels like a bug, but just wanted to check here before filing a
> report if I've missed something:
>
> buzzkill$ python2.7
> Enthought Python Distribution -- www.enthought.com
> Version: 7.2-2 (32-bit)
>
> Python 2.7.2 |EPD 7.2-2 (32-bit)| (default, Sep 7 2011, 09:16:50)
> [GCC 4.0.1 (Apple Inc. build 5493)] on darwin
> Type "packages", "demo" or "enthought" for more information.
> >>> import tarfile
> >>> source = open('/src/Python-2.6.7.tgz', 'rb')
> >>> tar = tarfile.open(fileobj=source, mode='r|*')
> >>> member = tar.extractfile('Python-2.6.7/Lib/genericpath.py')
> >>> data = member.read()
> Traceback (most recent call last):
> File "", line 1, in
> File
> "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/tarfile.py",
> line 815, in read
> buf += self.fileobj.read()
> File
> "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/tarfile.py",
> line 735, in read
> return self.readnormal(size)
> File
> "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/tarfile.py",
> line 742, in readnormal
> self.fileobj.seek(self.offset + self.position)
> File
> "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/tarfile.py",
> line 554, in seek
> raise StreamError("seeking backwards is not allowed")
> tarfile.StreamError: seeking backwards is not allowed
>
> The key is the "mode='r*|" which I understood to be specifically for
> reading blocks from a stream without seeking that would cause
> problems.
When discussing "filemode|[compression]" modes, the docs say:
However, such a TarFile object is limited in that it does not
allow to be accessed randomly
I'm not a tarfile expert, but extracting a single file sounds like
random access to me. If it was the first file in the archive (or there
was only one file) it probably wouldn't count as random access.
Petri
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Infrastructure] Snakebite build slaves and developer SSH/GPG public keys
On Thu, 23 Aug 2012 09:22:20 +0200 Georg Brandl wrote: > On 23.08.2012 09:17, "Martin v. Löwis" wrote: > > On 23.08.2012 01:03, Nick Coghlan wrote: > >> currently, there's a security hole > >> in the fact that requests to change our registered ssh key for access > >> are not themselves authenticated electronically > > > > Indeed, we should start requesting birth certificates, else someone > > claiming to be Reinhold Birkenfeld may get commit access :-) > > Of course, even then it might be a political problem for us if Barack Obama > wants to join as a core contributor if he happens to be voted out of office > and gets bored ;) Will he have a German accent? Regads Antoine. -- Software development and contracting: http://pro.pitrou.net ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] bug in tarfile module?
On Thu, 23 Aug 2012 12:25:26 +0300, Petri Lehtinen wrote:
> Chris Withers wrote:
> > Hi All,
> >
> > This feels like a bug, but just wanted to check here before filing a
> > report if I've missed something:
> >
> > buzzkill$ python2.7
> > Enthought Python Distribution -- www.enthought.com
> > Version: 7.2-2 (32-bit)
> >
> > Python 2.7.2 |EPD 7.2-2 (32-bit)| (default, Sep 7 2011, 09:16:50)
> > [GCC 4.0.1 (Apple Inc. build 5493)] on darwin
> > Type "packages", "demo" or "enthought" for more information.
> > >>> import tarfile
> > >>> source = open('/src/Python-2.6.7.tgz', 'rb')
> > >>> tar = tarfile.open(fileobj=source, mode='r|*')
> > >>> member = tar.extractfile('Python-2.6.7/Lib/genericpath.py')
> > >>> data = member.read()
> > Traceback (most recent call last):
> > File "", line 1, in
> > File
> > "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/tarfile.py",
> > line 815, in read
> > buf += self.fileobj.read()
> > File
> > "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/tarfile.py",
> > line 735, in read
> > return self.readnormal(size)
> > File
> > "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/tarfile.py",
> > line 742, in readnormal
> > self.fileobj.seek(self.offset + self.position)
> > File
> > "/Library/Frameworks/Python.framework/Versions/7.2/lib/python2.7/tarfile.py",
> > line 554, in seek
> > raise StreamError("seeking backwards is not allowed")
> > tarfile.StreamError: seeking backwards is not allowed
> >
> > The key is the "mode='r*|" which I understood to be specifically for
> > reading blocks from a stream without seeking that would cause
> > problems.
>
> When discussing "filemode|[compression]" modes, the docs say:
>
> However, such a TarFile object is limited in that it does not
> allow to be accessed randomly
>
> I'm not a tarfile expert, but extracting a single file sounds like
> random access to me. If it was the first file in the archive (or there
> was only one file) it probably wouldn't count as random access.
There is an open doc bug for this:
http://bugs.python.org/issue10436
--David
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [compatibility-sig] do all VMs implement the ast module? (was: Re: AST optimizer implemented in Python)
On Mon, Aug 13, 2012 at 12:06 PM, Brett Cannon wrote: > Time to ask the other VMs what they are currently doing (the ast module came > into existence in Python 2.6 so all the VMs should be answer the question > since Jython is in alpha for 2.7 compatibility). IronPython has an _ast implementation that matches 2.7 as close as reasonably possible. - Jeff ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Snakebite build slaves and developer SSH/GPG public keys
On Thu, Aug 23, 2012 at 02:09:14AM -0700, Giampaolo Rodolà wrote: > > For committers on other Python projects like Buildbot, Django and > > Twisted that may be reading this -- yes, the plan is to give you > > guys Snakebite access/slaves down the track too. I'll start looking > > into that after I've finished setting up the remaining slaves for > > Python. (Setting up a keys repo will definitely help (doesn't have > > to be hg -- feel free to use svn/git/whatever, just try and follow > > the same layout).) > > This is so great! > I've been looking forward to this for a long time and kept visiting > the site every now and then to see if there was any progress. > I'd surely use this for psutil if you'll let me. Hey, psutil, I use that! :-) Awesome candidate for Snakebite, too, given the unavoidable platform-specific C code behind the scenes. I'll take it up with you offline. > Also, at some point I would suggest to introduce the possibility to > donate some money in order to help supporting what I think must be a > pretty complex infrastructure requiring a lot of resources, both in > terms of hardware and time/labor. You have no idea :-/ Regards, Trent. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Snakebite build slaves and developer SSH/GPG public keys
On Wed, Aug 22, 2012 at 03:28:16PM -0700, Trent Nelson wrote: > Hi folks, > > I've set up a bunch of Snakebite build slaves over the past week. > One of the original goals was to provide Python committers with > full access to the slaves, which I'm still keen on providing. > > What's a nice simple way to achieve that in the interim? Quick update: Martin's hooked me up with everything I need for now. I'll send out another e-mail once I've set up the necessary glue on my end. Trent. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
