I haven't had time to verify the other findings, but there are
definitely file handle leakages. I can never delete an application if
I just visited it once.
On Aug 2, 12:21 pm, mdipierro <mdipie...@cs.depaul.edu> wrote:
> I am about to catch a plane so I will be short. We can talk more about
> this in a couple of days.
>
> > > First a word of warning. I send you this mail in private because I
> > > want
> > > to share my thoughts on the web2py code and because I hope I can help
> > > with that somehow. You're free to publish this mail if you want.
>
> This is constructive criticism so I appreciate it.
>
>
>
> > > As you might remember I came in contract with web2py a long ago when
> > > you
> > > were evaluating pygments as a syntax highlighter for the web2py
> > > documentation or something and you decided to implement your own
> > > because
> > > you were unable to "ship" pygments as part of web2py. As far as I
> > > remember py2app or py2exe were not able to pick up the on-demand
> > > imports
> > > in the lexer module.
>
> > > I have been watching the development ever since because (And please
> > > don't think bad of me because of that) it appeared to me that web2py
> > > was
> > > a joke. I remember reading some very early revisions of your PDF
> > > documentation where I remember a code that looks something like that:
>
> > > db.define_table('something', db.Field('meh', 'date',
> > > default=date.today()))
>
> > > That line freaked me out. This line alone showed me that web2py is
> > > doing something very, very weird. From this line I could see that it
> > > means one of the following two things:
>
> > > - either the code has a bug for long running applications (in this
> > > case
> > > when the application code runs longer than a day in which case the
> > > default would have to change dynamically)
>
> > > - or the global scope has different semantics and is reevaluated each
> > > request.
>
> > > Turns out the latter is what happens in web2py.
>
> Correct.
>
> > There still is a race
> > > condition but only a small one.
>
> No there is not. Why do you think there is a race condition?
>
> > The bigger problem with this
> > > however is
> > > that this means the (not so) static definitions are reevaluated each
> > > request.
>
> Correct. A small price to pay for a lot of flexibility.
> We are working on lazy evaluation of tables.
> There is also a speedup if you press the "compile" button in admin.
>
> > > Especially for the database table definitions (which cause many
> > > function
> > > calls) this means an enormous per-request penality if you have many of
> > > them. I even remember reading a case like this in the web2py
> > > newsgroup
> > > about someone who had twenty table definitions or something and the
> > > majority of the request time was spent setting up the tables.
>
> There was a slow down problem that has been fixed by Alexey. I agree
> there is still room for improvement.
>
>
>
> > > However that also means another thing. Standard Python
> > > optimizations no
> > > longer apply. In Python usually modules are cached in an interpreter
> > > bound dictionary for reusing. This of course does not apply for
> > > web2py.
> > > However the second problem here is that only the local scope gets
> > > certain optimizations in cpython such as fast-local lookup. Instead
> > > of
> > > storing the variables in a dictionary they are instead stored in an an
> > > object where items are looked up by index (fast locals) rather than
> > > by a
> > > string.
>
> > > Now of course one can argue that this is the way it's intended.
> > > That's
> > > nice but for that, the documentation is lacking.
>
> Yes it is true that documentation is lacking.
>
> > There is no
> > > explanation of what implications this has on the code. And in
> > > comparison with "enterprise ready" that gives me a strange feeling
> > > about
> > > the project.
>
> > > I'm not even exactly sure what kills web2py for me in the end, but I'm
> > > pretty sure that part of the reason is the way the project is
> > > advertised.
>
> We prefer to talk about "advocating" not "advertising" since we do not
> sell it.
> Mistakes were made on my side.
>
> > > web2py does not have worse code than many popular libraries out there.
>
> Thanks. I take is as a compliment.
>
>
>
> > > BeautifulSoup comes to mind. That library does some really ugly
> > > things
> > > there including monkeypatching of the standard library. It does
> > > different things depending on the version of Python it's running on
> > > and
> > > in the code are some really ugly idioms like a unicode subclass that
> > > has
> > > a reference to the parsed tree that results in memory leaks very
> > > easily
> > > because it's nearly impossible to get rid of the reference (actually
> > > just by exploiting some weird Python internals that are mostly
> > > undocumented). But the author of BeautifulSoup does not advertise
> > > it as
> > > a backwards-compatible, enterprise ready HTML parser.
>
> > > The web2py code on the other hand has some really problematic idioms
> > > as
> > > well. The execution of models, views and whatnot in a made-up
> > > namespace
> > > comes to mind.
>
> Yes. It was a design decision to be closer to Rails than Django in
> this respect. I think this is what is making web2py popular, more than
> anything else.
>
> > Due to the decision of being backwards compatible it
> > > will be nearly impossible to "fix" those things.
>
> We think of them as features, not bugs.
>
> > However there are so
> > > many implications nobody yet knows about which is terribel for large
> > > projects.
>
> > > For example the database table redfinition problem cited above. Of
> > > course that particular problem can be worked around with making the
> > > definition code internally lazy or something but there can be more
> > > nobody yet knows about.
>
> Alexey has already done this. It achieves a 10-20% speedup. I am
> rewriting the DAL completely to make it easier to port on other
> backends and faster. The new one may support lazy evaluations of
> tables. Time permitting,
>
> > > Especially side-effects of code execution can do some terrible damage.
> > > Decorators that would register code would instantly leak for example.
> > > And some libraries provide auto-registration that you would not
> > > instantly know about.
>
> Can you provide an example of what you mean? I am pretty sure there is
> no leak.
>
> > > Also libraries like pickle do not support weird situations like this.
> > > So I guess in the end I'm not (only) complaining about the web2py code
> > > or decisions there, but mainly how web2py is advertised as enterprise
> > > ready when nobody knows how these idioms scale or behave.
>
> I disagree with "nobody knows how these idioms scale or behave"?
> We have been doing this for 2 years. We know very well how it works
> and scales.
> It actually works very well because as soon as a request is completed
> the environment is deleted, this is clean, avoids conflicts when
> multiple apps are running, and prevents memory leaks.
>
> > > Another part is that web2py (and this is something web2py has in
> > > common
> > > with so many other web frameworks) is the craze to reimplement
> > > everything. I know you don't want to pull depedencies in but from
> > > what
> > > I can see you're providing one-click installers for the framework
> > > which
> > > could ship external dependencies. Both py2exe and py2app provide
> > > support for external dependencies and I can't see what "not
> > > depending on
> > > webob/werkzeug" gives you there.
>
> web2py started as a teaching tool. It was important (and still is)
> that I can go over every line of code with my students know exactly
> why it is there.
>
> > > The reason why I'm addressing this is that there are so many
> > > limitations, problems and weird aspects in WSGI and HTTP that are hard
> > > to tackle and many people get wrong.
>
> I agree.
>
> > I'm not saying you're getting it
> > > wrong, I haven't even studied the code in question, but I'm pretty
> > > sure
> > > it would make sense to collaborate on that.
>
> I like the idea. Some conditions have to be met:
> 1) we have a new contributor license agreement on web2py-developers
> 2) patches must not break backward compatibility
> 3) patches must either make the code smaller/faster, or add a needed
> functionality.
>
>
>
> > > About the code quality itself, not the (IMO) broken concepts I picked
> > > some of the snippets I found quickly searching with the help of ack:
>
> > > - Missing `close()` calls with files (ignored matches from
> > > compileapp):
>
> > > gluon/fileutils.py:160: f.write(open(tarname, 'rb').read())
> > > gluon/fileutils.py:246: data = open(filename, 'rb').read()
> > > gluon/highlight.py:313: data = open(sys.argv[1]).read()
> > > gluon/languages.py:37: lang_text = open(filename, 'r').read()...
> > > gluon/languages.py:153: data = open(file, 'r').read()
> > > gluon/main.py:107:web2py_version = open(os.path.join(web2py_path, ...
> > > gluon/rewrite.py:26: exec open('routes.py', 'r').read() ...
> > > gluon/template.py:104: text = open(os.path.join(path, ...
> > > gluon/template.py:118: parent = open(t, 'rb').read()
> > > gluon/template.py:134: child = open(t, 'rb').read()
> > > gluon/tools.py:1934: return urllib.urlopen(url).read()
> > > gluon/widget.py:55:ProgramVersion = open('VERSION', ...
>
> > > - Same in streamer.py for checking for file existence
> > > (open(static_file))
>
> I do not see the problem. Files are automatically closed when the file
> object goes out of scope.
> Can you provide an example when it does not?
>
> > > - Template engine seems to be missing proper escapes. Having ''' in
> > > the
> > > template should be enough to break it.
>
> I do not believe that is the case but I may be wrong. Can you provide
> an example of what you say?
>
> > > - Whitespace between "else" and colon in the template engine is
> > > unlikely
> > > to work with the current parsing rules as far as I can see.
>
> I agree it will not work.
>
> > > -
>
> ...
>
> read more »
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"web2py-users" group.
To post to this group, send email to web2py@googlegroups.com
To unsubscribe from this group, send email to
web2py+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/web2py?hl=en
-~----------~----~----~----~------~----~------~--~---
