choice of web-framework

2017-10-22 Thread Patrick Vrijlandt 

Hello list,

I would like your recommendation on the choice of a web framework.

The project is completely new, there are no histories to take into 
account (current solutions are paper-based). The website involves 
questionnaires that will be developed, filled out and stored. Users are 
not programmers or developers. They should be authenticated. Version 
control is required. Internationalization is not an issue. I expect that 
the project will add additional requirements and complexity later on 
that I can not foresee yet. I'm targeting a single deployment (maybe a 
second on a development machine). I usually work on Windows, but Linux 
can be considered.


I'm not afraid to learn a (=one) new framework (that would actually be 
fun) but trying out a lot of them is not feasible. My current goal is a 
demonstration version of the project as a proof of concept. I may want 
to hand it over to a commercial solution at that stage.


I'm an experienced python programmer but not an experienced web 
developer. A few years ago I read some books about Zope and Plone, but 
never did serious development with those. I currently maintain an 
intranet site in MoinMoin. I assume Zope could still be a potential 
choice, but it may have lost the vibrancy of a few years ago. Also, I 
would not know which version to choose (Zope 4, BlueBream, or something 
like Grok). The problem seems too complicated for micro frameworks like 
bottle of Flask. Django could be the next alternative.


Finally, for a new project, I would not like to be confined to Python 2.7.

What are your ideas?

Thanks in advance,

--
Patrick
--
https://mail.python.org/mailman/listinfo/python-list

Re: choice of web-framework

2017-10-22 Thread Chris Angelico 
On Sun, Oct 22, 2017 at 9:24 PM, Patrick Vrijlandt  wrote:
> Hello list,
>
> I would like your recommendation on the choice of a web framework.
>
> The project is completely new, there are no histories to take into account
> (current solutions are paper-based). The website involves questionnaires
> that will be developed, filled out and stored. Users are not programmers or
> developers. They should be authenticated. Version control is required.
> Internationalization is not an issue. I expect that the project will add
> additional requirements and complexity later on that I can not foresee yet.
> I'm targeting a single deployment (maybe a second on a development machine).
> I usually work on Windows, but Linux can be considered.
>
> I'm not afraid to learn a (=one) new framework (that would actually be fun)
> but trying out a lot of them is not feasible. My current goal is a
> demonstration version of the project as a proof of concept. I may want to
> hand it over to a commercial solution at that stage.
>
> I'm an experienced python programmer but not an experienced web developer. A
> few years ago I read some books about Zope and Plone, but never did serious
> development with those. I currently maintain an intranet site in MoinMoin. I
> assume Zope could still be a potential choice, but it may have lost the
> vibrancy of a few years ago. Also, I would not know which version to choose
> (Zope 4, BlueBream, or something like Grok). The problem seems too
> complicated for micro frameworks like bottle of Flask. Django could be the
> next alternative.
>
> Finally, for a new project, I would not like to be confined to Python 2.7.
>
> What are your ideas?

First off, a huge thank-you for laying out your requirements in such detail!

My personal choice, under the circumstances, would be Flask. Your
project isn't too complicated for it, and it's not too hard to learn
(I learned it in a weekend, though YMMV of course). Authentication is
a fully-supported plugin (flask-login). I'm not sure what you mean by
"version control" - of course your actual Python code can be managed
in git/hg, but if it's a project requirement that your *data* be
git-managed too, then you'll need to plan something out for that. Not
too hard though.

Python 3 is fully supported by Flask, so that's not a problem. Windows
and Linux are also both viable platforms. I strongly recommend
creating a file "requirements.txt" in your project root and recording
every package you use there, so you can just run "pip install -r
requirements.txt" to grab everything. Using a virtual environment
("python3 -m venv env" in recent-enough Pythons) can help to keep your
dependencies clean. If you do that from the start of the project, and
avoid using anything that's Windows-specific, you shouldn't have much
trouble changing OSes.

For the database, I generally use PostgreSQL, unless the job's so
simple it can be done with flat files. Using Flask with SQLAlchemy is
a popular option, but you can also dive into psycopg2 directly (the
"2" doesn't mean "Python 2.7 only", it's fine). The tech stack
Python+Flask+SQLAlchemy+PostgreSQL+Linux is an extremely solid one, or
you can switch out any part fairly easily.

Disclaimer: I use Flask and SQLAlchemy when I'm teaching my students
how to build web apps in Python, but I don't have much experience with
any other frameworks or ORMs. Other options may very well be viable
and/or superior; all I can say is that the above *is* viable.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: choice of web-framework

2017-10-22 Thread Chris Warrick 
On 22 October 2017 at 12:24, Patrick Vrijlandt  wrote:
> Hello list,
>
> I would like your recommendation on the choice of a web framework.
>
> The project is completely new, there are no histories to take into account
> (current solutions are paper-based). The website involves questionnaires
> that will be developed, filled out and stored. Users are not programmers or
> developers. They should be authenticated. Version control is required.
> Internationalization is not an issue. I expect that the project will add
> additional requirements and complexity later on that I can not foresee yet.
> I'm targeting a single deployment (maybe a second on a development machine).
> I usually work on Windows, but Linux can be considered.

If you intend to put this on a server, and you probably do since
you’re talking about web frameworks, a Linux machine is your best bet
for that. Windows isn’t a good platform for making web servers out of.
(Your development machine can probably run Windows.)

> I'm not afraid to learn a (=one) new framework (that would actually be fun)
> but trying out a lot of them is not feasible. My current goal is a
> demonstration version of the project as a proof of concept. I may want to
> hand it over to a commercial solution at that stage.
>
> I'm an experienced python programmer but not an experienced web developer. A
> few years ago I read some books about Zope and Plone, but never did serious
> development with those. I currently maintain an intranet site in MoinMoin. I
> assume Zope could still be a potential choice, but it may have lost the
> vibrancy of a few years ago. Also, I would not know which version to choose
> (Zope 4, BlueBream, or something like Grok). The problem seems too
> complicated for micro frameworks like bottle of Flask. Django could be the
> next alternative.

Zope is effectively dead these days. IMO your best bet would be Django:

* built-in database support
* built-in user authentication support
* built-in administrator panel
* i18n support available for when you need it
* it’s a modern, friendly web framework

If you went with Flask, you’d end up with a pile of plugins (for auth,
for databases, for other things) and reimplement half of Django,
badly.

-- 
Chris Warrick 
PGP: 5EAAEA16
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: choice of web-framework

2017-10-22 Thread Lele Gaifax 
Chris Warrick  writes:

> Zope is effectively dead these days.

Except it's alive and kicking: https://blog.gocept.com/

:-)

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
l...@metapensiero.it  | -- Fortunato Depero, 1929.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: choice of web-framework

2017-10-22 Thread Chris Warrick 
On 22 October 2017 at 13:25, Lele Gaifax  wrote:
> Chris Warrick  writes:
>
>> Zope is effectively dead these days.
>
> Except it's alive and kicking: https://blog.gocept.com/
>
> :-)
>
> ciao, lele.

A few people still care, sure. But how alive is a project with 16
(sixteen) people on IRC (freenode #zope), 85 (eighty-five) stars on
GitHub, and 205 issues on GitHub (since 2013)?

-- 
Chris Warrick 
PGP: 5EAAEA16
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: choice of web-framework

2017-10-22 Thread Chris Angelico 
On Sun, Oct 22, 2017 at 10:34 PM, Chris Warrick  wrote:
> On 22 October 2017 at 13:25, Lele Gaifax  wrote:
>> Chris Warrick  writes:
>>
>>> Zope is effectively dead these days.
>>
>> Except it's alive and kicking: https://blog.gocept.com/
>>
>> :-)
>>
>> ciao, lele.
>
> A few people still care, sure. But how alive is a project with 16
> (sixteen) people on IRC (freenode #zope), 85 (eighty-five) stars on
> GitHub, and 205 issues on GitHub (since 2013)?
>

I'm not too bothered by number of stars, nor necessarily by the issue
count (maybe a lot of their work is discussed by email, not the
tracker). Most important, to me, is the number of commits. And that
doesn't look too bad; there aren't a huge number of them, but they're
fairly consistently being made. So I'd say the project isn't dead,
though you could very well argue that it's merely playing catch-up. (I
didn't look at the content of the commits in detail or anything.)

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

EuroPython 2017: Videos for Thursday available online

2017-10-22 Thread M.-A. Lemburg 
We are pleased to announce the third batch of cut videos for
EuroPython 2017.

To see the new videos, please head over to our EuroPython YouTube
channel and select the "EuroPython 2017" playlist. The new videos
start at entry 96 in the playlist.


  * EuroPython 2017 Videos *

http://europython.tv/


Next week we will release the last batch of videos currently marked as
"private".

Enjoy,
--
EuroPython 2017 Team
http://ep2017.europython.eu/
http://www.europython-society.org/

PS: Please forward or retweet to help us reach all interested parties:
https://twitter.com/europython/status/922070616836071425
Thanks.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: choice of web-framework

2017-10-22 Thread Chris Warrick 
On 22 October 2017 at 13:48, Chris Angelico  wrote:
> On Sun, Oct 22, 2017 at 10:34 PM, Chris Warrick  wrote:
>> On 22 October 2017 at 13:25, Lele Gaifax  wrote:
>>> Chris Warrick  writes:
>>>
 Zope is effectively dead these days.
>>>
>>> Except it's alive and kicking: https://blog.gocept.com/
>>>
>>> :-)
>>>
>>> ciao, lele.
>>
>> A few people still care, sure. But how alive is a project with 16
>> (sixteen) people on IRC (freenode #zope), 85 (eighty-five) stars on
>> GitHub, and 205 issues on GitHub (since 2013)?
>>
>
> I'm not too bothered by number of stars, nor necessarily by the issue
> count (maybe a lot of their work is discussed by email, not the
> tracker). Most important, to me, is the number of commits. And that
> doesn't look too bad; there aren't a huge number of them, but they're
> fairly consistently being made. So I'd say the project isn't dead,
> though you could very well argue that it's merely playing catch-up. (I
> didn't look at the content of the commits in detail or anything.)
>
> ChrisA
> --
> https://mail.python.org/mailman/listinfo/python-list

Mailing lists are quiet as well:
https://mail.zope.org/pipermail/zope/
https://mail.zope.org/pipermail/zope-dev/

For a web framework, the daily commit count is much less important
than the size of the active community. An active community means more
people that can offer support, fix bugs, write docs, provide
ready-made modules to achieve common tasks. Zope’s community is
nonexistent. Django has 1483 contributors and 29k stars on GitHub.
Zope has 83 and 85 respectively.

https://blog.gocept.com/2016/10/04/zope-resurrection-part-2-defibrillation/

> Zope is not dead. On the sprint there were nearly 20 people who use Zope for 
> their daily work.

Oh my, twenty people! That’s a lot! A lot!

Yeah, no. Zope is dead. With a few folks going through the “denial”
phase. Or the “I’ve got legacy code from last decade I can’t be
bothered to rewrite” phase.

-- 
Chris Warrick 
PGP: 5EAAEA16
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: choice of web-framework

2017-10-22 Thread Tim Chase 
On 2017-10-22 12:24, Patrick Vrijlandt wrote:
> I would like your recommendation on the choice of a web framework.

Might depend on what skills you already bring to the table.  If you
already know an ORM like SQLAlchemy or a template language like
Jinja, you might want to take the "bring the pieces I know/like
together" approach.  For this, Bottle & Flask are the top contenders
last I saw (I did some CherryPy contract work but the docs were
pretty horrible at the time).

If you are genuinely coming to this greenfield, Django's
docs/tutorials make it really easy to get up to speed with all of the
parts involved as it has its own ORM and templating language.  They
can be swapped out if you later need to, but for the average project,
they're sufficient and well documented.

I happen to be in the Django camp, but based on my experiments with
Bottle/Flask, can also recommend them without much hesitation.

> The project is completely new, there are no histories to take into 
> account (current solutions are paper-based). The website involves 
> questionnaires that will be developed, filled out and stored. Users
> are not programmers or developers. They should be authenticated.
> Version control is required.

I'm not sure what "version control is required" means in this
context.  Is this version-control of the users' answers? Or
version-control of the source code.  If it's the source code, the web
framework won't help you there, but git, mercurial, or subversion are
all good/reasonable choices.  If you want to version your user's
answers or other aspects of your application, you'll need to design
it into your app.  There might be plugins/modules to facilitate this
on either side of the Django / Flask/Bottle/SQLAlchemy divide.

> I'm targeting a single deployment (maybe a second on a
> development machine). I usually work on Windows, but Linux can be
> considered.

While both *can* be deployed on Windows, a Unix-like OS (whether
Linux, a BSD, or even a Mac) will likely give you a better deployment
experience and better docs.

> I'm not afraid to learn a (=one) new framework (that would actually
> be fun) but trying out a lot of them is not feasible. My current
> goal is a demonstration version of the project as a proof of
> concept.

I personally find that Django excels at these fast proof-of-concept
projects, as you have less concern about integrating disparate pieces
together at that stage.

> I'm an experienced python programmer but not an experienced web 
> developer.

Both sides offer a "Pythonic" feel to them (some feel less Pythonic)
so it's easy to come up to speed on either.

> A few years ago I read some books about Zope and Plone,

Hah, Zope was my first introduction to Python and I ran screaming.  I
eventually came back around, but it was a jarring first experience.

> The problem seems too complicated for micro frameworks like bottle
> of Flask. Django could be the next alternative.

Django is the top contender, so if you only have time to investigate
one, I'd urge you in that direction.  But Flask or Bottle can also
certainly handle a project like the one you describe.

> Finally, for a new project, I would not like to be confined to
> Python 2.7.

Flask/Bottle and Django are both Python3 ready.  Django, since v2.0
is now 3.0 only.

-tkc






-- 
https://mail.python.org/mailman/listinfo/python-list

Re: choice of web-framework

2017-10-22 Thread Patrick Vrijlandt 

Op 22-10-2017 om 14:05 schreef Tim Chase:

> I'm not sure what "version control is required" means in this
> context.  Is this version-control of the users' answers? Or
> version-control of the source code.  If it's the source code, the web
> framework won't help you there, but git, mercurial, or subversion are
> all good/reasonable choices.  If you want to version your user's
> answers or other aspects of your application, you'll need to design
> it into your app.  There might be plugins/modules to facilitate this
> on either side of the Django / Flask/Bottle/SQLAlchemy divide.

The version control I was referring to, is indeed users' data. I plan to 
use Mercurial for the source code. The questionnaires being developed 
will go through many revisions. The questionnaires being filled in, are 
enough work to have a provision for mistakes. The idea is much like the 
"revert" option that MoinMoin and other wikis provide.


--Patrick
--
https://mail.python.org/mailman/listinfo/python-list

Re: choice of web-framework

2017-10-22 Thread Chris Angelico 
On Mon, Oct 23, 2017 at 12:26 AM, Patrick Vrijlandt  wrote:
> Op 22-10-2017 om 14:05 schreef Tim Chase:
>
>> I'm not sure what "version control is required" means in this
>> context.  Is this version-control of the users' answers? Or
>> version-control of the source code.  If it's the source code, the web
>> framework won't help you there, but git, mercurial, or subversion are
>> all good/reasonable choices.  If you want to version your user's
>> answers or other aspects of your application, you'll need to design
>> it into your app.  There might be plugins/modules to facilitate this
>> on either side of the Django / Flask/Bottle/SQLAlchemy divide.
>
> The version control I was referring to, is indeed users' data. I plan to use
> Mercurial for the source code. The questionnaires being developed will go
> through many revisions. The questionnaires being filled in, are enough work
> to have a provision for mistakes. The idea is much like the "revert" option
> that MoinMoin and other wikis provide.

Then what I'd do is make sure the questionnaires are saved in a text
file (maybe use a Markdown-like notation), and then just call on
Mercurial from inside your app.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Modern website

2017-10-22 Thread Andrew Z 
I realize the following has little todo with python per se. But i hope to
get a guidance on how these types of tasks are done nowadays.

The task:
Ive been asked to create  an integration process. That is a few webpages
with questioneer with the submission to a "mother" company using its API .
Ideally this process will be used by a few 3rd party "child" companies -
they would just point a link from their site to mine. Prospective users
will fill out the pages and "mother company" will create accounts
associated with the referal "child" company.

The problem:
  how do i create a modern , slick  website. I also see , that each of
"child" would want to extend the look and feel of their site onto this
process. So, my process should have at least a few templates i can offer ,
that would match their site...
Im not too excited to use weebly or squarespace - id rather host everything
myself.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Modern website

2017-10-22 Thread Chris Angelico 
On Mon, Oct 23, 2017 at 2:21 AM, Andrew Z  wrote:
> I realize the following has little todo with python per se. But i hope to
> get a guidance on how these types of tasks are done nowadays.
>
> The task:
> Ive been asked to create  an integration process. That is a few webpages
> with questioneer with the submission to a "mother" company using its API .
> Ideally this process will be used by a few 3rd party "child" companies -
> they would just point a link from their site to mine. Prospective users
> will fill out the pages and "mother company" will create accounts
> associated with the referal "child" company.
>
> The problem:
>   how do i create a modern , slick  website. I also see , that each of
> "child" would want to extend the look and feel of their site onto this
> process. So, my process should have at least a few templates i can offer ,
> that would match their site...
> Im not too excited to use weebly or squarespace - id rather host everything
> myself.

Design a single web site, then have the child sites customize the CSS
*only*. That way, you guarantee that all functionality is the same,
but you can change the look and feel as you need. Modern CSS can do
rather a lot in terms of redefining the layout of a page.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: choice of web-framework

2017-10-22 Thread justin walters 
On Sun, Oct 22, 2017 at 3:24 AM, Patrick Vrijlandt 
wrote:

> Hello list,
>
> I would like your recommendation on the choice of a web framework.
>
> The project is completely new, there are no histories to take into account
> (current solutions are paper-based). The website involves questionnaires
> that will be developed, filled out and stored. Users are not programmers or
> developers. They should be authenticated. Version control is required.
> Internationalization is not an issue. I expect that the project will add
> additional requirements and complexity later on that I can not foresee yet.
> I'm targeting a single deployment (maybe a second on a development
> machine). I usually work on Windows, but Linux can be considered.
>
> I'm not afraid to learn a (=one) new framework (that would actually be
> fun) but trying out a lot of them is not feasible. My current goal is a
> demonstration version of the project as a proof of concept. I may want to
> hand it over to a commercial solution at that stage.
>
> I'm an experienced python programmer but not an experienced web developer.
> A few years ago I read some books about Zope and Plone, but never did
> serious development with those. I currently maintain an intranet site in
> MoinMoin. I assume Zope could still be a potential choice, but it may have
> lost the vibrancy of a few years ago. Also, I would not know which version
> to choose (Zope 4, BlueBream, or something like Grok). The problem seems
> too complicated for micro frameworks like bottle of Flask. Django could be
> the next alternative.
>
> Finally, for a new project, I would not like to be confined to Python 2.7.
>
> What are your ideas?
>
> Thanks in advance,
>
> --
> Patrick
> --
> https://mail.python.org/mailman/listinfo/python-list
>

I think your best choice here would be Django. I've been doing web
development with Python
for about 5 years now and I have only found thing that Django can't handle:
replacing the authentication
framework's dependence on SQL. i.e., if you wanted to use a noSQL db like
MongoDb for your user
model, it's not really possible without a ton of work. Other than that
though, Django can pretty much do everything.

Since you need to get this prototype done quickly, I think Django is your
absolute best choice. The projects motto is
"The web framework for perfectionists with deadlines." You get a ton of
stuff out of the box:

- User authentication
- Sessions
- Admin interface
- Fantastic ORM
- Templating (like jinbja2 but with some Django flair)
- The best documentation I have ever seen
- A huge ecosystem of third party libraries and plugins
- A strong and heavilly opinionated MVC architecture
- One of ths strongest and best suppported OSS projects in existence
- Built in and versioned schema migrations

The reasons I would not reccomend Flask/Bottle for your project
specifically:

- Flask is better for more "customized" solutions or simpler projects
- Flask is a great framework, but offers very little out of the box as far
as modern web application features
- It takes longer to get rolling with Flask(a lot more initial
configuration)
- When using Flask, many devs end up building their own version of Django
anyways
- Flask's tutorial is a lot less in-depth than Django's
- Although learning Flask in its entirety is much simpler than learning
Django in its entirety, it takes more time
  to get up and running with Flask in my experience.

Web2py/zope/other small and old frameworks:

- I would stay away from these unless you have a lot of experience with
them.
- These projects do not have a modern ecosystem of libraries and may not
have full Python3 support
- You will find less community memebers that are able to help you as there
are less people using these frameworks

Hug/Sanic/Falcon/ApiStar:

- These are designed around the idea of REST apis
- They offer less than Flask does out of the box(except for building REST
apis)
- They are fairly new and have not had the time to build up a supportive
ecosystem yet
- They rely on new language features like async
- Their docs are not up to par with more mature projects

Pyramid:

Though I haven't ever worked with Pyramid, I have met several people who
are very happy with it. I also met with
one of the project's core contributors and he spoke about how the
architecture is "plugin based". There is the the core library
and all of the other layers of the application such as the data layer or
authentication, for example, are officially supported
plugins. Might be worth looking into.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: choice of web-framework

2017-10-22 Thread Tim Chase 
On 2017-10-22 15:26, Patrick Vrijlandt wrote:
> The version control I was referring to, is indeed users' data. I
> plan to use Mercurial for the source code. The questionnaires being
> developed will go through many revisions. The questionnaires being
> filled in, are enough work to have a provision for mistakes. The
> idea is much like the "revert" option that MoinMoin and other wikis
> provide.

Depends on how much version-control'y'ness you want.  Having a
"current version with previous version" and if "resurrect version
$CURRENT-$N as the most recent version" is sufficient, then it's
pretty straight-forward.  If you also want to implement diffing,
merging diffs between various versions, diffing more than a single
text-field blob (a model spread across multiple normalized tables,
versioning changes there), etc, you're looking at a whole different
game.

Additionally, one needs to consider how responses get tied to a
questionnaire.  If I make a questionnaire that reads

 "Do you like ice cream?"
 [ ] Yes
 [ ] No

and you answer "Yes", but then I edit that question so that it reads

 "Do you like to kick puppies?"

you have a problem if it keeps your "Yes" answer and thinks it's
linked to the "same" question.

All that to say that version-control is often domain-specific and
non-trivial.

-tkc



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: right list for SIGABRT python binary question ?

2017-10-22 Thread Karsten Hilbert 
On Sat, Oct 21, 2017 at 07:10:31PM +0200, M.-A. Lemburg wrote:

> > Running a debug build of py27 gave me a first lead: this
> > Debian system (Testing, upgraded all the way from various
> > releases ago) carries an incompatible mxDateTime which I'll
> > take care of.
> > 
> > *** You don't have the (right) mxDateTime binaries installed !
> > Traceback (most recent call last):
> >   File "./bootstrap_gm_db_system.py", line 87, in 
> > from Gnumed.pycommon import gmCfg2, gmPsql, gmPG2, gmTools, gmI18N
> >   File 
> > "/home/ncq/Projekte/gm-git/gnumed/gnumed/Gnumed/pycommon/gmPG2.py", line 
> > 34, in 
> > from Gnumed.pycommon import gmDateTime
> >   File 
> > "/home/ncq/Projekte/gm-git/gnumed/gnumed/Gnumed/pycommon/gmDateTime.py", 
> > line 52, in 
> > import mx.DateTime as mxDT
> >   File "/usr/lib/python2.7/dist-packages/mx/DateTime/__init__.py", line 
> > 8, in 
> > from DateTime import *
> >   File "/usr/lib/python2.7/dist-packages/mx/DateTime/DateTime.py", line 
> > 9, in 
> > from mxDateTime import *
> >   File 
> > "/usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/__init__.py", line 
> > 13, in 
> > raise ImportError, why
> > ImportError: 
> > /usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so: 
> > undefined symbol: Py_InitModule4
> 
> This error suggests that you have 32- and 64-bit versions of
> Python and mxDateTime mixed in your installation.
> 
> Py_InitModule4 is only available in the 32-bit build of
> Python. With the 64-bit build, it's called Py_InitModule4_64.
> 
> Since you're getting the same error from faulthandler,
> this is where I'd start to investigate.
> 
> "nm" will list all exported and required symbols. As first step,
> you should probably check the python binary for its symbols and
> see whether it exports Py_InitModule* symbols.

Thanks for your input !

The python2.7-dbg build is 32 bits:

root@hermes:~# nm /usr/bin/python2.7-dbg | grep Py_InitM
00155b9f T Py_InitModule4TraceRefs


python2.7-dbg:
  Installiert:   2.7.14-2
  Installationskandidat: 2.7.14-2
  Versionstabelle:
 *** 2.7.14-2 500
500 http://httpredir.debian.org/debian unstable/main i386 
Packages
100 /var/lib/dpkg/status
 2.7.13-2 990
500 http://httpredir.debian.org/debian stretch/main i386 
Packages
990 http://httpredir.debian.org/debian buster/main i386 Packages

The python2.7 build (no -dbg) does not have symbols.

mxDateTime really should be 32 bits, too:

python-egenix-mxdatetime:
  Installiert:   3.2.9-1
  Installationskandidat: 3.2.9-1
  Versionstabelle:
 *** 3.2.9-1 990
500 http://httpredir.debian.org/debian stretch/main i386 
Packages
990 http://httpredir.debian.org/debian buster/main i386 Packages
500 http://httpredir.debian.org/debian unstable/main i386 
Packages
100 /var/lib/dpkg/status

Let me check the .so file:

root@hermes:~# nm 
/usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime_d.so  | grep 
Py_InitM
 U Py_InitModule4TraceRefs

It seems it is - hm ...

Thanks,
Karsten
-- 
GPG key ID E4071346 @ eu.pool.sks-keyservers.net
E167 67FD A291 2BEA 73BD  4537 78B9 A9F9 E407 1346
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: right list for SIGABRT python binary question ?

2017-10-22 Thread Karsten Hilbert 
On Sun, Oct 22, 2017 at 10:15:51PM +0200, Karsten Hilbert wrote:

> The python2.7-dbg build is 32 bits:

...

> The python2.7 build (no -dbg) does not have symbols.

More to the point:

/usr/bin/python2.7: ELF 32-bit LSB shared object, Intel 80386, version 
1 (SYSV), dynamically linked, interpreter /lib/ld-linux.so.2, for GNU/Linux 
2.6.32, BuildID[sha1]=91226399fb434d138f166a5c8eca04385ea2e41f, stripped

/usr/lib/python2.7/dist-packages/mx/DateTime/mxDateTime/mxDateTime.so: 
ELF 32-bit LSB shared object, Intel 80386, version 1 (SYSV), dynamically 
linked, BuildID[sha1]=f09c7cc78b41421d3cee1f418b4d45772a3db701, stripped

Regards,
Karsten
-- 
GPG key ID E4071346 @ eu.pool.sks-keyservers.net
E167 67FD A291 2BEA 73BD  4537 78B9 A9F9 E407 1346
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Sandsifter software finds hidden instructions inside processors.

2017-10-22 Thread skybuck2000 
Hi,

I hope you have the following newsgroup in case you are highly interested in 
knowing every last detail and every thought I have on this subject matter:

https://groups.google.com/forum/#!forum/alt.comp.hardware.pc-homebuilt

There I have written some detailed postings.

In the other newsgroups I will constrain myself to the most important 
matter/summary of my activities, findings and productions for your usage.

The most important information I want to share with you is the following:

1. I was successfull in running SandSifter software with Linux Mint 18.2 
booteable DVD, downloaded ISO from the internet and undocumented instructions 
have been found for AMD X2 3800+ Dual Core processor.

2. All files are available on my webdrive:

www.skybuck.org/SandSifter/

Explore the "unzipped folder" to see what it's all about.

3. I have written two tutorials how you can also run this software on your 
computer in case you have a DVD drive and DVD disc to burn this software onto.
One manual tutorial and one automatic tutorial. The automatic tutorial is the 
easiest one which I will post here, the automatic tutorial includes a run.sh 
script which I will also post here, this is to help you run this software on 
your machine, at the end of this posting I will discuss any possible risks to 
doing so in case you are worried.

Automatic tutorial:

Step 1. Download Linux Mint ISO (Successfully tested on Linux Mint 18.2 Sonya)

https://www.linuxmint.com/

Step 2. Burn Linux Mint ISO to DVD (Windows 7: Right click on file and choose 
burn to disc).

Step 3. Boot Linux Mint ISO from DVD (Restart computer, if needed go into bios 
and change boot order, or press F8 to bring up boot menu or something like that)

Step 4. Start FireFox Web Browser

Step 5. Download SandSifter software and extract to a folder.

https://github.com/xoreaxeaxeax/sandsifter

(Click "clone or download", then click "download zip", then click "open with 
archive manager", then click "extract" (top left icon), click "other 
locations", choose a harddisk or other storage
medium which is persistent, click on the storage medium, click create new 
folder (top right icon), name for folder could be "test", click "extract", 
click "show the files")

Enter the folder "sandsifter-master" by left clicking on it.

Step 6. Download Skybuck's Flying run.sh script file 

Download and save the "run.sh" script file to/inside the "sandsifter-master" 
folder.

http://www.skybuck.org/SandSifter/unzipped/run.sh

Step 7. Open terminal window and resize it to make it bigger

Right click in the empty space and choose "open in terminal"

A window and a prompt/blinking cursor should now come up looking similar to:

mint@mint /media/mint/Windows 7 System (New)/test/sandsifter-master $ 

Make the window bigger so that the summarize script at the end doesn't crash !

Drag and Drop the window at the bottom right corner to make it bigger (Hold the 
left mouse button to drag and make it bigger then let mouse button go)

Step 8. Run Skybuck's Flying Bash Script to install software and run SandSifter

type the following command:

bash ./run.sh

Step 9. Guide the software installation and upgrade process

Sometimes it will ask if you want to continue ? Press the Y key.

Once it's done installing SandSifter will automatically run and finally a 
summary will be created.

Step 10. Wait for the analysis to complete

Once you see instructions scrolling/flying over the screen go take a sleep and 
wait many hours until it is completely done.

Once it is done it will show something like: "May the Force be with you ! 
Always !" then you know the script is done !

Step 11. Do not open the log files !

The log files (in data folder) may be to big for the Linux Mint 18.2 text and 
office editors to handle ! This will probably crash/hang the system !

Step 12. Go into the data folder and send the files to the e-mail address:

xoreaxeax...@gmail.com


The run.sh script:

echo "Step 1. Install standard C library software"
sudo apt-get install libc6-dev

echo "Step 2. Install python pip"
sudo apt install python-pip

echo "Step 3. Update python pip"
sudo pip install --upgrade pip

echo "Step 4. Install setuptools"
sudo pip install setuptools

echo "Step 5. Install capstone binaries"
sudo apt-get install libcapstone3

echo "Step 6. Install capstone dev source"
sudo apt-get install libcapstone-dev

echo "Step 7. Install capstone python bindings (this will take a while)"
sudo pip install capstone

echo "Step 8. Make sandsifter"
make

echo "Step 9. Run sandsifter"
sudo ./sifter.py --unk --dis --len --sync --tick -- -P1 -t

echo "Step 10. Summarize"
./summarize.py data/log

echo ""
echo "Bash script"
echo "Version 0.01 created on 22 october 2017 by Skybuck Flying"
echo "To Install, Make, Run, Summarize SandSifter Software and Software 
Dependencies"
echo "Successfully tested on Linux Mint 18.2 Sonya on AMD Dual Core X2 3800+ 
processor"
echo "May the Force be with you ! Always ! =D"
echo "Have fun anal

Re: right list for SIGABRT python binary question ?

2017-10-22 Thread Karsten Hilbert 
On Sat, Oct 21, 2017 at 09:31:54AM +0200, dieter wrote:

> > Debug memory block at address p=0x717b7c: API ''
> > 0 bytes originally requested
> > The 3 pad bytes at p-3 are not all FORBIDDENBYTE (0xfb):
> > at p-3: 0x03 *** OUCH
> > at p-2: 0x4e *** OUCH
> > at p-1: 0x00 *** OUCH
> > Because memory is corrupted at the start, the count of bytes 
> > requested
> >may be bogus, and checking the trailing pad bytes may segfault.
> > The 4 pad bytes at tail=0x717b7c are not all FORBIDDENBYTE (0xfb):
> > at tail+0: 0x00 *** OUCH
> > at tail+1: 0x00 *** OUCH
> > at tail+2: 0x00 *** OUCH
> > at tail+3: 0x00 *** OUCH
> > The block was made by call #0 to debug malloc/realloc.
> > Fatal Python error: bad ID: Allocated using API '', verified using API 
> > 'o'
> > ...
> > Can anyone give more guidance on what the above python debug
> > output might vaguely point to ?
> 
> It points to a memory corruption.
> 
> I would approach the problem by means of debugging: put a write
> breakpoint at the corrupted address "0x717b7c" and check what part
> of the system accesses it (this assumes you are using a CPU
> supporting write breakpoints).
> It may be very tedious as the address might be accessed very often
> legally before it gets corrupted.
> 
> Another approach may be to use a tool designed for memory debugging,
> e.g. "valgrind".

Hi Dieter,

thanks for your guidance. I fear this approach is out of my class.

Karsten
-- 
GPG key ID E4071346 @ eu.pool.sks-keyservers.net
E167 67FD A291 2BEA 73BD  4537 78B9 A9F9 E407 1346
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Compression of random binary data

2017-10-22 Thread danceswithnumbers 
In Short, I cannot find a single mathematical proof that says you cannot 
compress random numbers. Pigeon hole and other conjectures are just that. In 
fact, the biggest fallacy when people start talking about compression is to say 
that all compression alg rely on redundancies, or repetitive sequences. The 
million random digits is compressed down to 415,241 kb. I have been only able 
to compress that down to 352,954 kb. A very small amount. It has taken six 
years to come up with that small amount.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Compression of random binary data

2017-10-22 Thread danceswithnumbers 
On Monday, July 11, 2016 at 11:52:27 AM UTC-6, jonas.t...@gmail.com wrote:
> What kind of statistic law or mathematical conjecture  or is it even a 
> physical law is violated by compression of random binary data? 
> 
> I only know that Shanon theorised it could not be done, but were there any 
> proof? 
> 
> What is to say that you can not do it if the symbolic representation is 
> richer than the symbolic represenatation of the dataset. 
> 
> Isn't it a fact that the set of squareroots actually depict numbers in a 
> shorter way than their actual representation. 
> 
> Now the inpretator or program must know the rules. And i have very good rules 
> to make it happen.

In Short, I cannot find a single mathematical proof that says you cannot 
compress random numbers. Pigeon hole and other conjectures are just that. In 
fact, the biggest fallacy when people start talking about compression is to say 
that all compression alg rely on redundancies, or repetitive sequences. The 
million random digits is compressed down to 415,241 kb. I have been only able 
to compress that down to 352,954 kb. A very small amount. It has taken six 
years to come up with that small amount.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Compression of random binary data

2017-10-22 Thread Steve D'Aprano 
On Mon, 23 Oct 2017 02:29 pm, Stefan Ram wrote:

> r...@zedat.fu-berlin.de (Stefan Ram) writes:
>>When we have a source of 2 random binary digits,
>>there are 2^2 possible outcomes:
>>( 0, 0 ),
>>( 0, 1 ),
>>( 1, 0 ), and
>>( 1, 1 ).
>>. One cannot aggree upon a code to represent each
>>of those four possibilities with just one bit.
> 
>   If the probabilities of each outcome is 0.25.

When people talk about "random data", they are normally referring to equal
probabilities. Just as when they talk about compression in this context, they
mean lossless, reversible compression.

If the probability of certain codes (either single codes, or sequences of
codes) are non-equal, then you can take advantage of that by encoding the
common cases into a short representation, and the uncommon and rare cases
into a longer representation. As you say:


>   Otherwise, if ( 0, 0 ) is much more frequent,
>   we can encode ( 0, 0 ) by "0" and
> 
> ( 0, 1 ) by "101",
> ( 1, 0 ) by "110", and
> ( 1, 1 ) by "111".
> 
>   And we could then use /less/ than two bits on the
>   average. 

That's incorrect. On average you use 2.5 bits.

(1*1 bit + 3*3 bits divide by four possible outcomes, makes 2.5 bits.)

Where you can actually get compression is if you limit yourself to only
encoding specially selected input data where the code pairs are non-random
and you have many more (0,0) than other combinations. But that is highly
non-random.

For example, if we are compressing a black and white bitmap, say a fax, the
most common data is a page which is mostly white with only a bit of black
text. That gives us a bitmap with lots and lots of (0,0) pairs and relatively
few of the other combinations, which means for the common case of "a page of
black text on a white background" the compression works well.

(I'm treating 0 as white and 1 as black.)

But for purely random input, there will be roughly equal numbers of each pair
of bit and on average the encoding will expand the data by 25% (two bits of
input maps to 2.5 bits output, on average).



>   But two bits are (maximally )random if 
>   the probability of every combination /is/ 0.25.

In the example you give, we have four possible pairs of bytes:


Input:  (0,0) (0,1) (1,0) (1,1)  # 2 bits on average
Output: "0"   "101" "110" "111"  # 2.5 bits on average


So if the incoming data is random (uniformly-distributed), this will expand
the size of the data by 25%.

Only if the data is non-random and biased heavily towards (0,0) will you see
compression savings.

Of course in the real world most data we want to compress is non-random, and
we can get really impressive compressions. But the cost of that is that you
can rarely compress data twice (if you do, you either get very little savings
the second time, or it expands) and you cannot compress random (uniformly
distributed) data.

>   (If other combinations than ( 0, 0 ) were extremely
>   rare, one could even improve the compression more.)

What you say is true, but it has nothing to do with Danceswithnumbers'
ignorant and ludicrous claim that he can compress random data and that there
is no mathematical proof that you cannot.



-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Compression of random binary data

2017-10-22 Thread Gregory Ewing 

danceswithnumb...@gmail.com wrote:

On Monday, July 11, 2016 at 11:52:27 AM UTC-6, jonas.t...@gmail.com wrote:



What is to say that you can not do it if the symbolic representation is
richer than the symbolic represenatation of the dataset.


The more symbols you have in your alphabet, the more bits
are needed to encode each symbol.


Isn't it a fact that the set of squareroots actually depict numbers in a
shorter way than their actual representation.


No.


In Short, I cannot find a single mathematical proof that says you cannot
compress random numbers. Pigeon hole and other conjectures are just that.


No, they are not conjectures, they are proofs. If you don't
recognise them as such, then you haven't fully understood them.


In
fact, the biggest fallacy when people start talking about compression is to
say that all compression alg rely on redundancies, or repetitive sequences.


I think it's more correct to say they rely on the fact that
the input data is encoded *ineffiently* in some way -- it uses
more bits than are necessary to represent the information.


The million random digits is compressed down to 415,241 kb. I have been only
able to compress that down to 352,954 kb. A very small amount. It has taken
six years to come up with that small amount.


You may well have an algorithm that encodes *some* million
digit sequences in less than 415,241 bytes. But I can guarantee
that there will be other million-digit sequences that it
encodes to *more* than 415,241 bytes. The average over all
possible million-digit sequences, assuming they are all
equally likely, *cannot* be less than 415,241 bytes. Anything
else would be a logical impossibility.

Note that "equally likely" is important here. That's what
"random data" means in this context -- that all possible
input sequences have the same probability.

This provides another way of thinking about how compression
works: it relies on all input sequences *not* having the
same probability. If some are more likely than others, then
you can encode the more frequent ones with less bits and
the less frequent ones with more bits, and gain overall.

--
Greg

--
https://mail.python.org/mailman/listinfo/python-list

Re: Compression of random binary data

2017-10-22 Thread Steve D'Aprano 
On Mon, 23 Oct 2017 08:22 am, danceswithnumb...@gmail.com wrote:

> In Short, I cannot find a single mathematical proof that says you cannot
> compress random numbers. 

Three seconds of googling finds this:

http://www.faqs.org/faqs/compression-faq/part1/section-8.html

which demonstrates a simple, trivial proof.

It took me a little longer to find this:

https://en.wikipedia.org/wiki/Kolmogorov_complexity#Compression

but only because I couldn't remember how to spell Kolmogorov, and that lead me
to this:

https://en.wikipedia.org/wiki/Incompressible_string

for a concrete example.

Of course *occasionally* one will, by pure luck, compress a stream of random
bits. I run my random number generator and by a fluke it generates a stream
of 50 zeroes in a row:

00

which even a simple run-length encoding scheme can compress:

(50*0)


But flukes are not what people mean when they talk about compressing random
data. They mean some algorithm which can:


- transparently (no hiding data in the decompressor, or the file name, 
  or in a secret file hidden on the disk, or anywhere else)

- reversibly (there must be a decompressor which when given ONLY the
  compressed file, and NOTHING else, reconstruct the original)

- losslessly (no throwing away data: we must reconstruct the EXACT
  original, not merely something "close enough")

- and *consistently* (flukes don't count: every input must be compressed)


compress random data of size N bits to at most N-1 bits.

We can prove that this is impossible by simply *counting* how many random
strings there are.

Let us pick a concrete example, N=8. Then there are 2**8 = 256 possible
strings. We need to compress down to at most 7 bits. If we allow the output
to vary in length, there are:

2**7 = 128 seven bit strings;
2**6 = 64 six bit strings;
2**5 = 32 five bit strings;
2**4 = 16 four bit strings;
2**3 = 8 three bit strings;
2**2 = 4 two bit strings;
2**1 = 2 one bit strings;
2**0 = 1 zero bit strings

which add up to 255 compressed strings in total. But there are 256 possible
inputs, and only 255 possible outputs: so at least one input cannot be
compressed, or the compression must be lossy (two inputs compress to the same
output).




By the way: here is a very clever trick for hiding information in the file
system:

http://www.patrickcraig.co.uk/other/compression.php


but as people point out, the information in the file, plus the information in
the file system, ends up being *larger* than the original. Well done to
Patrick Craig for finding a clever loophole to "win" the challenge, but he
did so without actually compressing the original data.




-- 
Steve
“Cheer up,” they said, “things could be worse.” So I cheered up, and sure
enough, things got worse.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Modern website

2017-10-22 Thread Abdur-Rahmaan Janhangeer 
do it as django does it
.in django you have templates like take the example of a single webpage

other pages only modify what they need to. like the footer will remain the
same on all pages, so only the top part needs to be specified

similarly for your website, you can go along that line ^^,

Abdur-Rahmaan Janhangeer,
Mauritius
abdurrahmaanjanhangeer.wordpress.com

On 22 Oct 2017 19:21, "Andrew Z"  wrote:

> I realize the following has little todo with python per se. But i hope to
> get a guidance on how these types of tasks are done nowadays.
>
> The task:
> Ive been asked to create  an integration process. That is a few webpages
> with questioneer with the submission to a "mother" company using its API .
> Ideally this process will be used by a few 3rd party "child" companies -
> they would just point a link from their site to mine. Prospective users
> will fill out the pages and "mother company" will create accounts
> associated with the referal "child" company.
>
> The problem:
>   how do i create a modern , slick  website. I also see , that each of
> "child" would want to extend the look and feel of their site onto this
> process. So, my process should have at least a few templates i can offer ,
> that would match their site...
> Im not too excited to use weebly or squarespace - id rather host everything
> myself.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: choice of web-framework

2017-10-22 Thread Abdur-Rahmaan Janhangeer 
have you considered Django? i've found found it to be nice !

Abdur-Rahmaan Janhangeer,
Mauritius
abdurrahmaanjanhangeer.wordpress.com

On 22 Oct 2017 14:25, "Patrick Vrijlandt"  wrote:

> Hello list,
>
> I would like your recommendation on the choice of a web framework.
>
> The project is completely new, there are no histories to take into account
> (current solutions are paper-based). The website involves questionnaires
> that will be developed, filled out and stored. Users are not programmers or
> developers. They should be authenticated. Version control is required.
> Internationalization is not an issue. I expect that the project will add
> additional requirements and complexity later on that I can not foresee yet.
> I'm targeting a single deployment (maybe a second on a development
> machine). I usually work on Windows, but Linux can be considered.
>
> I'm not afraid to learn a (=one) new framework (that would actually be
> fun) but trying out a lot of them is not feasible. My current goal is a
> demonstration version of the project as a proof of concept. I may want to
> hand it over to a commercial solution at that stage.
>
> I'm an experienced python programmer but not an experienced web developer.
> A few years ago I read some books about Zope and Plone, but never did
> serious development with those. I currently maintain an intranet site in
> MoinMoin. I assume Zope could still be a potential choice, but it may have
> lost the vibrancy of a few years ago. Also, I would not know which version
> to choose (Zope 4, BlueBream, or something like Grok). The problem seems
> too complicated for micro frameworks like bottle of Flask. Django could be
> the next alternative.
>
> Finally, for a new project, I would not like to be confined to Python 2.7.
>
> What are your ideas?
>
> Thanks in advance,
>
> --
> Patrick
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: grapheme cluster library (Posting On Python-List Prohibited)

2017-10-22 Thread Rustom Mody 
On Monday, October 23, 2017 at 8:06:03 AM UTC+5:30, Lawrence D’Oliveiro wrote:
> On Saturday, October 21, 2017 at 5:11:13 PM UTC+13, Rustom Mody wrote:
> > Is there a recommended library for manipulating grapheme clusters?
> 
> Is this  any good?

Thanks looks promising.
Dunno how much it lives up to the claims 
[For now the one liner from regex's findall has sufficed:
findall(r'\X', «text»)  

[Thanks MRAB for the library]
 
> Bear in mind that the logical representation of the text is as code points, 
> graphemes would have more to do with rendering.

Heh! Speak of Euro/Anglo-centrism!

In a sane world graphemes would be called letters
And unicode codepoints would be called something else — letterlets??
To be fair to the Unicode consortium, they strive hard to call them codepoints
But in an anglo-centric world, the conflation of codepoint to letter is 
inevitable I guess.
To hear how a non Roman-centric view of the world would sound:
A 'w' is a poorly double-struck 'u'
A 't' is a crossed 'l'
Reasonable?

The lead of https://en.wikipedia.org/wiki/%C3%9C has

| Ü, or ü, is a character…classified as a separate letter in several extended 
Latin alphabets 
| (including Azeri, Estonian, Hungarian and Turkish), but as the letter U with 
an 
| umlaut/diaeresis in others such as Catalan, French, Galician, German, Occitan 
and Spanish.
-- 
https://mail.python.org/mailman/listinfo/python-list