Re: Test suite in github but missing from pypi tarballs

2016-04-22 Thread Brian May
Tristan Seligmann  writes:

> With my upstream developer hat on: source packages on PyPI are meant for
> end users to install via pip. They often include generated artifacts, and
> don't include things that aren't intended for installation via pip (tests
> being just one of these).
>
> For distribution packaging purposes, the GitHub tags are generally
> preferrable. GitHub makes archives of tagged releases available as
> tarballs, so this is generally a simple tweak to debian/watch.

I use to have this attitude too. However now I have reconsidered.

Far better to to retrieve source packages from PyPI.

For example, it is possible to have signed uploads to PyPI and have
uuscan automatically check the signature. This is not the case for
github.

packages that use setuptools-scm is another example - as a (over?)
simplification this requires files in the PyPI distribution, although
perhaps that is a topic best left for another discussion.
-- 
Brian May 



Re: Test suite in github but missing from pypi tarballs

2016-04-22 Thread Brian May
Tristan Seligmann  writes:

> With my upstream developer hat on: source packages on PyPI are meant for
> end users to install via pip. They often include generated artifacts, and
> don't include things that aren't intended for installation via pip (tests
> being just one of these).
>
> For distribution packaging purposes, the GitHub tags are generally
> preferrable. GitHub makes archives of tagged releases available as
> tarballs, so this is generally a simple tweak to debian/watch.

Forgot to mention: In PyPI there is an application called
"check-manifest" that will check to ensure all files in git will be
included in the PyPI release, and suggest changes to MANIFEST.in.
-- 
Brian May 



Re: Test suite in github but missing from pypi tarballs

2016-04-22 Thread Edward Betts
Here is a list of packages that include a test suite on github but not in
the DPMT git.

csvkit https://github.com/onyxfish/csvkit
django-haystackhttps://github.com/toastdriven/django-haystack
django-jinja   https://github.com/niwibe/django-jinja
django-recurrence  https://github.com/django-recurrence/django-recurrence
django-webpack-loader  https://github.com/owais/django-webpack-loader
djoser https://github.com/gizmag/django-fsm-log
dockerpty  https://github.com/d11wtq/dockerpty
drf-generators https://github.com/Brobin/drf-generators
mod-wsgi   https://github.com/GrahamDumpleton/mod_wsgi
pdfrw  https://github.com/pmaupin/pdfrw
pep8-naminghttps://github.com/flintwork/pep8-naming
pkgconfig  https://github.com/matze/pkgconfig
pydot  https://github.com/erocarrera/pydot
pylast http://github.com/pylast/pylast
pysrt  https://github.com/byroot/pysrt
python-argshttps://github.com/kennethreitz/args
python-astor   https://github.com/berkerpeksag/astor
python-cachecontrolhttps://github.com/ionrock/cachecontrol
python-easywebdav  https://github.com/amnong/easywebdav
python-exifhttps://github.com/ianare/exif-py
python-gnutls  https://github.com/AGProjects/python-gnutls
python-hpilo   https://github.com/seveas/python-hpilo
python-humanizehttps://github.com/jmoiron/humanize
python-jsonpifyhttps://github.com/wcdolphin/flask-jsonpify
python-model-mommy http://github.com/vandersonmota/model_mommy
python-mrjob   http://github.com/Yelp/mrjob
python-pgspecial   https://github.com/dbcli/pgspecial
python-pip https://github.com/pypa/pip
python-pretend https://github.com/alex/pretend
python-rplyhttps://github.com/alex/rply
python-scp https://github.com/jbardin/scp.py
python-sentinels   https://github.com/vmalloc/sentinels
python-sunlighthttps://github.com/sunlightlabs/python-sunlight
python-trezor  https://github.com/trezor/python-trezor
python-zxcvbn  https://github.com/dropbox/python-zxcvbn
responses  https://github.com/getsentry/responses
subliminal https://github.com/Diaoul/subliminal
txws   https://github.com/MostAwesomeDude/txWS
vcversionerhttps://github.com/habnabit/vcversioner
whichcraft https://github.com/pydanny/whichcraft

This is the code I ran on git.debian.org to generate the list.

#!/usr/bin/python3

from subprocess import Popen, PIPE
from urllib.request import urlopen
from urllib.error import HTTPError
import os
import re

re_github_url = re.compile('(https?://[^/ ]*github.com/[^/]+/[^/ ]+)')

location = '/git/python-modules/packages/'

def repo_includes_tests(repo):
cmd = ['git', '--git-dir=' + location + repo, 'ls-tree', '-r', 'HEAD', 
'--name-only']
with Popen(cmd, stdout=PIPE, stderr=PIPE) as proc:
for line in proc.stdout:
filename = line.decode('utf-8')[:-1]
if not filename.endswith('.py'):
continue
if filename.startswith('test') or '/test' in filename:
return True
return False

def show_debian_file(repo, filename):
cmd = ['git', '--git-dir=' + location + repo, 'show', 'HEAD:debian/' + 
filename]
return Popen(cmd, stdout=PIPE, stderr=PIPE).stdout.read().decode('utf-8')

def github_urls(content):
github_urls = set()
for line in content.splitlines():
if 'github.com' in line:
m = re_github_url.search(line)
if not m:
print(line)
github_urls.add(m.group(1))
return github_urls

for repo in sorted(os.listdir(location)):
if not repo.endswith('.git') or repo_includes_tests(repo):
continue

urls = (github_urls(show_debian_file(repo, 'copyright'))
or github_urls(show_debian_file(repo, 'watch')))

if len(urls) != 1:
continue
(url,) = urls
try:
page = urlopen(url).read()
except HTTPError:
continue
if ' title="test' not in page.decode('utf-8'):
continue  # no tests in github
print('{:23s} {}'.format(repo[:-4], url))



python debug packages

2016-04-22 Thread Jean-Michel Vourgère
Hi

Now that debug symbols are automatically generated in -dbgsym packages,
how do you handle the debug
/usr/lib/python2.7/dist-packages/.x86_64-linux-gnu_d.so files?

They used to go in a generic -dbg package.

I'm thinking about rrdtool, and it already has a lot of packages:
https://tracker.debian.org/pkg/rrdtool

I'm considering creating a specific python-rrdtool-dbg package.

Other options I can think of are:
- Put the debug .so file into the main python-rrdtool package
- Do not migrate to new style -dbgsym packages and keep everything in
rrtool-dbg, like it is now.
- Stop bothering about this debug .so file, and trash it.

Any suggestion anyone?



Re: python debug packages

2016-04-22 Thread Matthias Klose

On 22.04.2016 16:58, Jean-Michel Vourgère wrote:

Hi

Now that debug symbols are automatically generated in -dbgsym packages,
how do you handle the debug
/usr/lib/python2.7/dist-packages/.x86_64-linux-gnu_d.so files?

They used to go in a generic -dbg package.

I'm thinking about rrdtool, and it already has a lot of packages:
https://tracker.debian.org/pkg/rrdtool

I'm considering creating a specific python-rrdtool-dbg package.

Other options I can think of are:
- Put the debug .so file into the main python-rrdtool package


no, that would add dependencies on the python-dbg packages by default.


- Do not migrate to new style -dbgsym packages and keep everything in
rrtool-dbg, like it is now.


that would be my preferred solution.


- Stop bothering about this debug .so file, and trash it.


please don't.