[web2py] Can't configure web2py with apache via xampp
Nothing I am doing works. None of the web pages have answers. I am not confident in the web2py book, which contains many errors. The problem appears to be that web2py insists that the admin pages are accessed via ssl for the login but doesn't require (of course) ssl for the ordinary applications. The problem is that apache on windows appears to not allow the use of two different ports with one ip address (127.0.0.1 or localhost) in this case. Note that I have verified that mod_wsgi is installed properly and works. In a directory, c:\webapp I have a tiny .py that successfully returns "hello world." I would like to keep this around so that I continue to have a simple way to verify the mod_wsgi setup. I read somewhere (I think the mod_wsgi site) that more than one WSGIScriptAlias directives were allowed. No, I don't have skype or anything else listening on port 80. I am not even accessing this machine over the web. This is purely for local development on a single machine. I don't want to use the builtin Rocket wsgi server because I want to get apache configured properly. Later, I will shift to a SuSELinux development server. After that, I will prop to Linode. I suspect the latter two environments might work better than xampp. it is shocking how poorly documented web2py is. Deployment especially seems an afterthought. Here is my httpd.conf file. I have moved most of the web2py specific settings to httpd-alt-web2py.conf referenced via Include (below). # # This is the main Apache HTTP server configuration file. It contains the # configuration directives that give the server its instructions. # See http://httpd.apache.org/docs/2.2> for detailed information. # In particular, see # http://httpd.apache.org/docs/2.2/mod/directives.html> # for a discussion of each configuration directive. # # Do NOT simply read the instructions in here without understanding # what they do. They're here only as hints or reminders. If you are unsure # consult the online docs. You have been warned. # # Configuration and logfile names: If the filenames you specify for many # of the server's control files begin with "/" (or "drive:/" for Win32), the # server will use that explicit path. If the filenames do *not* begin # with "/", the value of ServerRoot is prepended -- so "logs/foo.log" # with ServerRoot set to "C:/xampp/apache" will be interpreted by the # server as "C:/xampp/apache/logs/foo.log". # # NOTE: Where filenames are specified, you must use forward slashes # instead of backslashes (e.g., "c:/apache" instead of "c:\apache"). # If a drive letter is omitted, the drive on which httpd.exe is located # will be used by default. It is recommended that you always supply # an explicit drive letter in absolute paths to avoid confusion. # # ServerRoot: The top of the directory tree under which the server's # configuration, error, and log files are kept. # # Do not add a slash at the end of the directory path. If you point # ServerRoot at a non-local disk, be sure to point the LockFile directive # at a local disk. If you wish to share the same ServerRoot for multiple # httpd daemons, you will need to change at least LockFile and PidFile. # ServerRoot "C:/xampp/apache" NameVirtualhost *:80 NameVirtualhost *:443 # # Listen: Allows you to bind Apache to specific IP addresses and/or # ports, instead of the default. See also the # directive. # # Change this to Listen on specific IP addresses as shown below to # prevent Apache from glomming onto all bound IP addresses. # #Listen 0.0.0.0:80 #Listen [::]:80 Listen 127.0.0.1:80 Listen 127.0.0.1:443 # for web2py secure admin pages--this didn't work by itself?? needed more than one argument? # # # # Dynamic Shared Object (DSO) Support # # To be able to use the functionality of a module which was built as a DSO you # have to place corresponding `LoadModule' lines at this location so the # directives contained in it are actually available _before_ they are used. # Statically compiled modules (those listed by `httpd -l') do not need # to be loaded here. # # Example: # LoadModule foo_module modules/mod_foo.so # LoadModule actions_module modules/mod_actions.so LoadModule alias_module modules/mod_alias.so LoadModule asis_module modules/mod_asis.so LoadModule auth_basic_module modules/mod_auth_basic.so LoadModule auth_digest_module modules/mod_auth_digest.so #LoadModule authn_alias_module modules/mod_authn_alias.so #LoadModule authn_anon_module modules/mod_authn_anon.so #LoadModule authn_dbd_module modules/mod_authn_dbd.so #LoadModule authn_dbm_module modules/mod_authn_dbm.so LoadModule authn_default_module modules/mod_authn_default.so LoadModule authn_file_module modules/mod_authn_file.so #LoadModule authnz_ldap_module modules/mod_authnz_ldap.so #LoadModule authz_dbm_module modules/mod_authz_dbm.so LoadModule authz_default_module modules/mod_authz_default.so LoadModule authz_groupfile_module modules/mod_authz_groupfile.so LoadModule authz_host_module mod
[web2py] Re: Can't configure web2py with apache via xampp
I solved this. Dumb error. Still the doc's aren't great. Error was that ifmodule for ssl section has an include that brings in a virtualhost for port 443. I was doing it twice. Fixed that and apache runs. Web2py runs behind apache. Now my problem is that I can't get to the admin pages via apache though I do have ssl configured, created a new cert, and imported the cert to both Chrome and Firefox. Got any ideas for me if you read this far?
[web2py] why won't web2py create the databases
I guess I like SQL better than frameworks. SQL is a query language for
DDL and DML that is well documented and completely explicitly.
Frameworks are opaque and invent a new syntax subject to ambiguities
of parens, square braces, curly braces, dotted names, sometimes
tablenames being implicit, sometimes fieldnames being implicit. Why?
So, I have repeatedly tried to make a link table like the famous dogs
and owners tables in the Web2py book to no avail.
Each time there is a problem I must do the following:
- drop the database from mysql
- stop apache
- stop mysql
- delete the web2py application via the filesystem
- restore the web2py application via the filesystem making sure that
the cache and session files have been deleted
- restart apache
- restart mysql
- create the empty database in mysql
- run the web2py application
But, web2py's DAL fails to create the tables. Because the errors
appear to be in the framework itself, the diagnostics are completely
opaque through several layers of calls. The ultimate error is that one
of the tables is not being created. But, that is circular--that is
because the DAL failed to create the table for a brand new web2py
application in an empty database.
Judging from the references below it is because of a faulty
migration. it's not like the most trivial migrations EVER work. it
appears that some detritus of the previous application or tables
exists so that web2py attempts its failed magic by doing a migration.
So, perhaps the real question is how does one purge every possible
vestige of both the tables and application so that web2py does not try
any of its so-called magic (which is supposed to be very unpythonic,
in any case).
I've wasted nearly 60 hours on web2py and am about to stop wasting any
more time. There seem to be many reasons that php, jquery, and RonR
have dusted python for web development.
Here is the traceback:
TRACEBACK
Traceback (most recent call last):
File "C:\web2py\gluon\restricted.py", line 204, in restricted
exec ccode in environment
File "C:/web2py/applications/pyjokes/models/db.py", line 55, in
Field('category_id', 'integer'))
File "C:\web2py\gluon\dal.py", line 5097, in define_table
polymodel=polymodel)
File "C:\web2py\gluon\dal.py", line 728, in create_table
fake_migrate=fake_migrate)
File "C:\web2py\gluon\dal.py", line 816, in migrate_table
self.execute(sub_query)
File "C:\web2py\gluon\dal.py", line 1359, in execute
return self.log_execute(*a, **b)
File "C:\web2py\gluon\dal.py", line 1353, in log_execute
ret = self.cursor.execute(*a, **b)
File "C:\web2py\gluon\contrib\pymysql\cursors.py", line 108, in
execute
self.errorhandler(self, exc, value)
File "C:\web2py\gluon\contrib\pymysql\connections.py", line 184, in
defaulterrorhandler
raise errorclass, errorvalue
ProgrammingError: (1146, u"Table 'jodb.joke_category' doesn't exist")
[web2py] Re: why won't web2py create the databases
I mean, why won't web2py create the tables through its famous DAL.
I have created the database directly in mysql.
On Jan 11, 8:44 pm, Likit wrote:
> I guess I like SQL better than frameworks. SQL is a query language for
> DDL and DML that is well documented and completely explicitly.
> Frameworks are opaque and invent a new syntax subject to ambiguities
> of parens, square braces, curly braces, dotted names, sometimes
> tablenames being implicit, sometimes fieldnames being implicit. Why?
>
> So, I have repeatedly tried to make a link table like the famous dogs
> and owners tables in the Web2py book to no avail.
>
> Each time there is a problem I must do the following:
> - drop the database from mysql
> - stop apache
> - stop mysql
> - delete the web2py application via the filesystem
> - restore the web2py application via the filesystem making sure that
> the cache and session files have been deleted
> - restart apache
> - restart mysql
> - create the empty database in mysql
> - run the web2py application
>
> But, web2py's DAL fails to create the tables. Because the errors
> appear to be in the framework itself, the diagnostics are completely
> opaque through several layers of calls. The ultimate error is that one
> of the tables is not being created. But, that is circular--that is
> because the DAL failed to create the table for a brand new web2py
> application in an empty database.
>
> Judging from the references below it is because of a faulty
> migration. it's not like the most trivial migrations EVER work. it
> appears that some detritus of the previous application or tables
> exists so that web2py attempts its failed magic by doing a migration.
>
> So, perhaps the real question is how does one purge every possible
> vestige of both the tables and application so that web2py does not try
> any of its so-called magic (which is supposed to be very unpythonic,
> in any case).
>
> I've wasted nearly 60 hours on web2py and am about to stop wasting any
> more time. There seem to be many reasons that php, jquery, and RonR
> have dusted python for web development.
>
> Here is the traceback:
>
> TRACEBACK
>
> Traceback (most recent call last):
> File "C:\web2py\gluon\restricted.py", line 204, in restricted
> exec ccode in environment
> File "C:/web2py/applications/pyjokes/models/db.py", line 55, in
>
> Field('category_id', 'integer'))
> File "C:\web2py\gluon\dal.py", line 5097, in define_table
> polymodel=polymodel)
> File "C:\web2py\gluon\dal.py", line 728, in create_table
> fake_migrate=fake_migrate)
> File "C:\web2py\gluon\dal.py", line 816, in migrate_table
> self.execute(sub_query)
> File "C:\web2py\gluon\dal.py", line 1359, in execute
> return self.log_execute(*a, **b)
> File "C:\web2py\gluon\dal.py", line 1353, in log_execute
> ret = self.cursor.execute(*a, **b)
> File "C:\web2py\gluon\contrib\pymysql\cursors.py", line 108, in
> execute
> self.errorhandler(self, exc, value)
> File "C:\web2py\gluon\contrib\pymysql\connections.py", line 184, in
> defaulterrorhandler
> raise errorclass, errorvalue
> ProgrammingError: (1146, u"Table 'jodb.joke_category' doesn't exist")
[web2py] web2py fails when mysql tables already exist
web2py has failed when the authuser table already exists. Well, ideally the auth tables are meant to persist. I thought the DAL created tables if they DON'T exist. I guess I am getting really tired of the unbearable fragility of web2py. Just pretty bush league, I guess. Here is the traceback: Traceback (most recent call last): File "C:\web2py\gluon\restricted.py", line 204, in restricted exec ccode in environment File "C:/web2py/applications/pyjokes/models/db.py", line 9, in auth.define_tables() File "C:\web2py\gluon\tools.py", line 1285, in define_tables format='%(first_name)s %(last_name)s (%(id)s)')) File "C:\web2py\gluon\dal.py", line 5097, in define_table polymodel=polymodel) File "C:\web2py\gluon\dal.py", line 705, in create_table self.create_sequence_and_triggers(query,table) File "C:\web2py\gluon\dal.py", line 1348, in create_sequence_and_triggers self.execute(query) File "C:\web2py\gluon\dal.py", line 1359, in execute return self.log_execute(*a, **b) File "C:\web2py\gluon\dal.py", line 1353, in log_execute ret = self.cursor.execute(*a, **b) File "C:\web2py\gluon\contrib\pymysql\cursors.py", line 108, in execute self.errorhandler(self, exc, value) File "C:\web2py\gluon\contrib\pymysql\connections.py", line 184, in defaulterrorhandler raise errorclass, errorvalue InternalError: (1050, u"Table 'auth_user' already exists")
[web2py] don't understand forms processing
I really don't get the use of {{form}} in views. It's really too
opaque. The form=crud and form=SQLFORM really only do trivial cases
involving a single table.
What I want to do is just create an html input form in the view with a
submit action.
When the http request comes back with variables for each input field
in the form I want to grab the returned values and put them in the
databases (plural) myself. This is the only way to get the required
control and make the code and view obvious.
Do I use request.vars to get the returned values? There is something
like form.vars and form.accepts in the manual. But, there is simply
not enough information. I have to process form.accepts before return
dict(form=form).
The flow is really confusing. When is the form/view sent to the
client? After submit is clicked by the user, when does the controller
receive the request (with Get/Put values)? The problem as I see it is
that a function in a controller only "sends" its bits to the view when
the function hits "return dict(form=form)" . But, then the function
is over--where do I put the logic for processing the user's reply
(e.g.--request.vars)? It appears BEFORE the return--but the view
hasn't gone to the client yet. Or is it effectively a loop--as long
as the request (the user's reply) comes back to the same url the
controller runs again, but this time with the submitted form. I am
thinking this must be what happens. It is very hard to experiment to
figure this out because of all my runtime errors and the very
challenging diagnostics (layers upon layers of calls through the
framework itself).
This whole flow thing would be more obvious if I could use yield
instead of return. It seems like what is really needed is something
like this:
def controller_func():
prepare info to send to the user in the view
send the view--e.g., yield dict(form=form)
receive "request" from the user after submit (or other action)
do things with what has been received from the user
if form.accepts(?, ?):
x = request.vars.x
db.foo.insert(name=x) etc.
redirect or next URL
return (end of function--nothing sent to view)
The flow above is far more obvious and follows the stateless
transactional model of http.
Sorry if I am hopelessly stupid, but the manual offers a woeful lack
of complete examples. There isn't even complete documentation of each
class and all of its methods. I did look at the doc strings on the
FORM function (is it a function? a class? ???). Accepts seems to be
the method I want, but it is not clear when it runs. It is not clear
when the validators run. It is not clear if I access returned values
(request) with request.vars or form.vars.
Again, sorry but there is just not enough for me to comprehend it.
Decent examples go a long way.Either no one has time or everyone's
expertise is so far beyond mine. I have looked through all items
tagged form on web2py slices but these are also very fragmentary. I
apologize profusely for my stupidity. I just need more complete code
examples to run and study.
[web2py] examples app doesn't run on apache
I guess I could learn a lot from the examples. It's what they are there for. But, when run with mod_wsgi and apache, you get service unavailable--which really doesn't make a lot of sense on localhost, does it? It appears to work on rocket. But, when I navigate to the actual examples, I get the web2py web site instead of the local copy (which I would access via rocket server on my machine as localhost). This seems sort of broken. I guess no one else needs such basic help or has noticed.
[web2py] Re: web2py fails when mysql tables already exist
So, all of the above problems appear to be related to some sort of corruption that resulted in the default.py controller. I noticed that in the traceback it would be shown with 2 additional lines that I did not enter. I did not see these in any external editor. I had previously backed up the application in the file system so I could delete it and restore it if needed. I tried to delete the app and it wouldn't delete (likely because I hadn't stopped apache and python/web2py were still running--an explicit "stop" would be useful). But, web2py would hang. I rebooted my system. When I returned, the app was still there, but without any controllers. Interesting. I copied the controllers back and lo, and behold, it all worked. It has worked since but I shall be wary of the internal editor though it is convenient for small changes and debugging. The issue with documentation is that it just a bit scattered. Some of the very most helpful bits appear to have some sort of configuration issue. The examples and the api help available via the admin interface are useful and cover areas not in the web2py book. When I attempt to access these while running on mod_wsgi and apache I get error 503 (service not available or no capacity). When I run on rocket server they are accessible. Any thoughts? Is anything internally coded to port 8000? It would be nice not to have to go in and out of apache to use this doc. Generally, I have had a lot of config challenges not covered in the config recipes. Apache web based documentation was essential to clearing these up. Some of the forms methods are a bit confusing and the examples (when I run on rocket) showed a way out. While the framework does a lot of work that I don't have to do, masking sql and introducing some new syntax/semantics on html creates some confusion for me.
[web2py] Re: examples app doesn't run on apache
Here is the returned page when clicking on the help button in the extreme upper right corner of the admin page: Service unavailable! The server is temporarily unable to service your request due to maintenance downtime or capacity problems. Please try again later. If you think this is a server error, please contact the webmaster. Error 503 127.0.0.1 01/12/12 10:05:52 Apache/2.2.21 (Win32) mod_ssl/2.2.21 OpenSSL/1.0.0e mod_wsgi/3.3 Python/2.7.2 PHP/5.3.8 mod_perl/2.0.4 Perl/v5.10.1 Doesn't seem like apache url access issue: that would appear as not having permission. I guess I should point out that the url is requested with https as it is part of the admin url. Thanks
[web2py] Re: don't understand forms processing
I had another problem accessing the examples with my apache config (separate thread). Under rocket server I could see those examples. Yes, I think the products or dog registration examples illustrate what I need. On Jan 12, 4:33 am, stefaan wrote: > Have you looked here?http://web2py.com/examples/default/examples#form_examples > > FORM (and all the other UPPERCASE helpers) is just a function that > creates an html fragment. > > The key to understanding how these forms work is to realize that forms > are "submitted to themselves". > > That's why the code in the examples first checks: > > "if form.accepts" -> this can only be true if the form was already > submitted, so you can take some actions based on the values in the > form > "else if form.errors" -> this can only be true if the form was > submitted, but validation failed, so you can take actions based on the > failures > "else" -> this can only be true if the form was not submitted yet, so > you can display the form for the first time > > Please checkout example 28 from the above link - it should be quite > enlightening.
[web2py] Re: why won't web2py create the databases
Thanks for the replies.
I deleted the tables themselves--not in app/databases but from the
mysql dir where they are kept. I then use phpmyadmin to create an
empty d-b, as required when using mysql. (Yes, I read the db chapter
on the web2py book.)
I don't believe you are seeing a period in the db name; that is the
path to a table.
I believe this problem had some bizarre association with corruption in
the controller.
Here is the model file:
# pyjokes
# -*- coding: utf-8 -*-
jodb = DAL('mysql://root:mypwdfoo@localhost/jodb') #not the actual
password!
from gluon.tools import *
auth = Auth(jodb)
auth.define_tables()
crud = Crud(jodb)
## configure email
mail=auth.settings.mailer
mail.settings.server = 'logging' or 'smtp.gmail.com:587'
mail.settings.sender = 'y...@gmail.com'
mail.settings.login = 'username:password'
## configure auth policy
auth.settings.registration_requires_verification = False
auth.settings.registration_requires_approval = False
auth.settings.reset_password_requires_verification = False
## if you need to use OpenID, Facebook, MySpace, Twitter, Linkedin,
etc.
## register with janrain.com, write your domain:api_key in private/
janrain.key
from gluon.contrib.login_methods.rpx_account import use_janrain
use_janrain(auth,filename='private/janrain.key')
#
## Define your tables below (or better in another model file) for
example
##
## >>> db.define_table('mytable',Field('myfield','string'))
##
## Fields can be
'string','text','password','integer','double','boolean'
## 'date','time','datetime','blob','upload', 'reference
TABLENAME'
## There is an implicit 'id integer autoincrement' field
## Consult manual for more options, validators, etc.
##
## More API examples for controllers:
##
## >>> db.mytable.insert(myfield='value')
## >>> rows=db(db.mytable.myfield=='value').select(db.mytable.ALL)
## >>> for row in rows: print row.id, row.myfield
#
jodb.define_table('joke',
Field('joketext', 'text',length=2048),
Field('created_on', 'datetime', default=request.now),
Field('created_by', jodb.auth_user, default=auth.user_id))
jodb.define_table('category',
Field('name', 'text'))
jodb.define_table('joke_category',
Field('joke_id', 'integer'),
Field('category_id', 'integer'))
jodb.category.name.requires = IS_NOT_EMPTY()
jodb.joke.joketext.requires = IS_NOT_EMPTY()
# jodb.joke_category.requires = IS_IN_DB(jodb,jodb.joke.id) #this
didn't work for some reason
[web2py] Re: examples app doesn't run on apache
I think the issue is with caching. Examples is the only app that I've run with caching on by default (all the funcs in its default.py are decorated). Since inmemory appears to be the default it is possible that apache is running out of RAM. Sort of surprising as it works in rocket and this is a 4G machine. Although when I run xampp I also have mysql running, which is not the case with rocket. So, maybe I just run out of RAM. I am still digging. I'll try to turn off caching--as a test--globally for web2py. On Jan 12, 10:09 am, Likit wrote: > Here is the returned page when clicking on the help button in the > extreme upper right corner of the admin page: > > Service unavailable! > > The server is temporarily unable to service your request due to > maintenance downtime or capacity problems. Please try again later. > > If you think this is a server error, please contact the webmaster. > > Error 503 > > 127.0.0.1 > 01/12/12 10:05:52 > Apache/2.2.21 (Win32) mod_ssl/2.2.21 OpenSSL/1.0.0e mod_wsgi/3.3 > Python/2.7.2 PHP/5.3.8 mod_perl/2.0.4 Perl/v5.10.1 > > Doesn't seem like apache url access issue: that would appear as not > having permission. I guess I should point out that the url is > requested with https as it is part of the admin url. > > Thanks
[web2py] Re: examples app doesn't run on apache
I think the issue is with caching. Examples is the only app that I've run with caching on by default (all the funcs in its default.py are decorated). Since inmemory appears to be the default it is possible that apache is running out of RAM. Sort of surprising as it works in rocket and this is a 4G machine. Although when I run xampp I also have mysql running, which is not the case with rocket. So, maybe I just run out of RAM. I am still digging. I'll try to turn off caching--as a test--globally for web2py. On Jan 12, 10:09 am, Likit wrote: > Here is the returned page when clicking on the help button in the > extreme upper right corner of the admin page: > > Service unavailable! > > The server is temporarily unable to service your request due to > maintenance downtime or capacity problems. Please try again later. > > If you think this is a server error, please contact the webmaster. > > Error 503 > > 127.0.0.1 > 01/12/12 10:05:52 > Apache/2.2.21 (Win32) mod_ssl/2.2.21 OpenSSL/1.0.0e mod_wsgi/3.3 > Python/2.7.2 PHP/5.3.8 mod_perl/2.0.4 Perl/v5.10.1 > > Doesn't seem like apache url access issue: that would appear as not > having permission. I guess I should point out that the url is > requested with https as it is part of the admin url. > > Thanks
[web2py] Re: Is it me or what?
Nice. Randomly noticed that the all bands/against me link is brokent... On Jan 12, 11:48 pm, Annet wrote: > Hi, > > > Thanks for pointing that out, this is probably a stupid question, but > > where do I change the metadata author? > > In previous versions of web2py this was the way to set meta data: > > In menu.py or in a controller: > > ## read more athttp://dev.w3.org/html5/markup/meta.name.html > response.meta.author = 'Your Name ' > response.meta.description = 'a cool new app' > response.meta.keywords = 'web2py, python, framework' > response.meta.generator = 'Web2py Web Framework' > response.meta.copyright = 'Copyright 2011' > > and then in the section of a layout something like: > > > > > > > ... I am not sure it still is ... > > Regards, > > Annet
[web2py] Re: examples app doesn't run on apache
Still no solution to this. I would really like not having to switch between Rocket and Apache each time I want to look at the help or examples. Error 503 is Apache saying that the server can't respond to the client. In the Apache error log, I have the following when I attempt to access help or run the examples app: [Sat Jan 14 14:02:26 2012] [error] [client 127.0.0.1] client denied by server configuration: C:/web2py/wsgihandler.py, referer: http://127.0.0.1/welcome So, Apache/mod_wsgi is properly dispatching to python and wsgihandler.py is obviously the first stop. But, then web2py fails to respond back to Apache. Note that all the other apps including the two I've started all work properly. My theory on memory seems to be wrong: with the whole stack running(xampp, apache, python, web2py), Windows reports 2G. That's certainly enough for some small caches and any transient memory use. So, I wonder if there is something wrong with routes, even though I didn't touch it. What is different about the help and examples app? I can't figure it out other than some links go back to www.web2py.com and many of the functions are cached via function decorator in the controller. Any help would be appreciated as I am stuck and this seems to be an uncommon problem.
[web2py] Re: examples app doesn't run on apache
One other bit of information: everything works as expected with Rocket server (using port 8000).
[web2py] migration not working
I am trying to use the many-2-many DAL syntax suggested by the manual
and the examples. In this approach, a field name is associated with a
TABLE name.
I have not been able to get this to work with MySQL. So, I have used
the more conventional approach of using an integer field to hold the
key value from the relationship table. Instead of dogs and persons I
have jokes and categories, but it's the same thing. The goal is to be
able to express relations for jokes with no category, jokes with one
or more categories, categories with one or more jokes, and categories
with no jokes. The ideal query returns all of these, if they exist.
Here is the model for my "conventional" approach:
jodb.define_table('joke',
Field('joketext', 'text',length=2048),
Field('created_on', 'datetime', default=request.now),
Field('created_by', jodb.auth_user, default=auth.user_id))
jodb.define_table('category',
Field('name', 'text'))
jodb.define_table('joke_category',
Field('joke_id', 'integer'),
Field('category_id', 'integer'))
jodb.category.name.requires = IS_NOT_EMPTY()
jodb.joke.joketext.requires = IS_NOT_EMPTY()
All was good except I couldn't get the query to work using the persons/
dogs many-to-many approach. I was not getting jokes with no
categories and categories with no jokes. So, I decided to follow the
manual more closely to see if I could try the queries from the
example.
Thus, I tried to change the field definitions in the relations table
to associate the id field with a TABLE as in:
jodb.define_table('joke_category',
Field('joke_id', jodb.joke),
Field('category_id', 'integer'))
Yes, I would need to do the same for category, but I wanted to do one
at a time because I had read somewhere that MySQL migrations did not
work with multiple pending changes (don't know if that is really
true...). Anyway, seemed easier to do one at a time. So, I made the
immediately preceding change in the model. In attempting to do the
migration, MySQL got an error 150. When this happens, either web2py or
MySQL will hang. So, I stopped everything.
Upon resuming everything, I tried to go back to the original way I had
it (above). This appears to cause a second migration with the
following result:
127.0.0.1.2012-01-14.19-48-00.189cb495-7851-429d-a0cb-bd23156431f1
(1060, u"Duplicate
column name 'joke_id__tmp'")
VERSION
web2py™ (1, 99, 4, datetime.datetime(2011, 12, 14, 14, 46, 14),
'stable')
Python
TRACEBACK
Traceback (most recent call last):
File "c:\web2py\gluon\restricted.py", line 204, in restricted
exec ccode in environment
File "c:/web2py/applications/pyjokes/models/db.py", line 55, in
Field('category_id', 'integer'))
File "c:\web2py\gluon\dal.py", line 5097, in define_table
polymodel=polymodel)
File "c:\web2py\gluon\dal.py", line 728, in create_table
fake_migrate=fake_migrate)
File "c:\web2py\gluon\dal.py", line 816, in migrate_table
self.execute(sub_query)
File "c:\web2py\gluon\dal.py", line 1359, in execute
return self.log_execute(*a, **b)
File "c:\web2py\gluon\dal.py", line 1353, in log_execute
ret = self.cursor.execute(*a, **b)
File "c:\web2py\gluon\contrib\pymysql\cursors.py", line 108, in
execute
self.errorhandler(self, exc, value)
File "c:\web2py\gluon\contrib\pymysql\connections.py", line 184, in
defaulterrorhandler
raise errorclass, errorvalue
InternalError: (1060, u"Duplicate column name 'joke_id__tmp'")
Note the duplicate column name. It seems that the first migration
created a tmp field to move the column values over to the newly
created column. This hung, but the tmp column must have been
created. The second migration--to get back (which I erroneously
thought wouldn't happen at all, assuming the first migration had
failed completely--clearly needed to create the second tmp column,
which was a dupe.
I can get myself back to the "conventional" approach working.
So, my real question is: how can I formulate the query that returns
all jokes and categories with their relations including no
corresponding joke or category? Should I do this using the
"conventional' way or should I use the suggested web2py model
syntax.
I know--a long question for what is probably a shorter answer.
Many thanks.
[web2py] what triggers a db migration
I had a db for which I changed the model. This correctly caused a migration. Either web2py or MySQL hung. This has sort of diminished my confidence in migrations. I realize it is a great thing, but only if/when it works. Believe me I am experimenting with a really trivial example with 3 tables (plus auth) and about 3-4 fields each with about as many rows of test data. So, if it hangs for this... Anyway, I manually purged the tmp column and make sure the data was ok. Set the table in question to migrate = False. All worked ok. Terrific. With things working I decided I would put the model back the way it was originally and removed migrate = False from the offending table. Then, when I ran the app--web2py attempted a migration. There must be something like the data-time stamp of the model compared to some inner value that triggers the migration. Is there someway I can get the inner state in sync so the migration doesn't happen? I sent another post about what went wrong with the migration itself. Now I am just trying to get everything cleaned up manually. Many thanks.
[web2py] Re: what triggers a db migration
The manual is pretty explicit about this although it simplifies a bit. The manual says it looks at the existing tables (actually looks at the physical file or looks at the most recent definition in sql.log?). I think that the time stamps also matter. Not clear what timestamp is being compared to what... I followed the manual and did the "fake_migrate=True" thing. As the db tables already really did match, that as a sure thing and the app works again without migrate = False. I am curious why the initial migration did not work. It would seem that the sequence that needed to happen is: i. create new tmp column ii. assign values of column being changed to tmp column iii. create new column iv. assign values of tmp column to new column--if appropriate v. drop tmp column I think my migration failed on step iii, create new column, because I did not specify the Field arguments properly. Let me know if you have any suggestions. I am still curious about this. Here is my sql.log file BEFORE doing the fake_migrate. Immediately following that is the sql.log after the successful fake_migrate. The Alter table sql statements are still present. that sort of concerns me. Can I simply purge the log and let web2py update it with the application's next execution? Perhaps it is the _name.table files that really determine what happens in migrations. timestamp: 2012-01-11T21:33:41.50 CREATE TABLE auth_user( id INT AUTO_INCREMENT NOT NULL, first_name VARCHAR(128), last_name VARCHAR(128), email VARCHAR(255), password VARCHAR(255), registration_key VARCHAR(255), reset_password_key VARCHAR(255), registration_id VARCHAR(255), PRIMARY KEY(id) ) ENGINE=InnoDB CHARACTER SET utf8; timestamp: 2012-01-11T21:46:05.797000 CREATE TABLE auth_user( id INT AUTO_INCREMENT NOT NULL, first_name VARCHAR(128), last_name VARCHAR(128), email VARCHAR(255), password VARCHAR(255), registration_key VARCHAR(255), reset_password_key VARCHAR(255), registration_id VARCHAR(255), PRIMARY KEY(id) ) ENGINE=InnoDB CHARACTER SET utf8; timestamp: 2012-01-11T22:04:00.147000 CREATE TABLE auth_user( id INT AUTO_INCREMENT NOT NULL, first_name VARCHAR(128), last_name VARCHAR(128), email VARCHAR(255), password VARCHAR(255), registration_key VARCHAR(255), reset_password_key VARCHAR(255), registration_id VARCHAR(255), PRIMARY KEY(id) ) ENGINE=InnoDB CHARACTER SET utf8; success! timestamp: 2012-01-11T22:04:00.162000 CREATE TABLE auth_group( id INT AUTO_INCREMENT NOT NULL, role VARCHAR(255), description LONGTEXT, PRIMARY KEY(id) ) ENGINE=InnoDB CHARACTER SET utf8; success! timestamp: 2012-01-11T22:04:00.172000 CREATE TABLE auth_membership( id INT AUTO_INCREMENT NOT NULL, user_id INT, INDEX user_id__idx (user_id), FOREIGN KEY (user_id) REFERENCES auth_user(id) ON DELETE CASCADE, group_id INT, INDEX group_id__idx (group_id), FOREIGN KEY (group_id) REFERENCES auth_group(id) ON DELETE CASCADE, PRIMARY KEY(id) ) ENGINE=InnoDB CHARACTER SET utf8; success! timestamp: 2012-01-11T22:04:00.186000 CREATE TABLE auth_permission( id INT AUTO_INCREMENT NOT NULL, group_id INT, INDEX group_id__idx (group_id), FOREIGN KEY (group_id) REFERENCES auth_group(id) ON DELETE CASCADE, name VARCHAR(255), table_name VARCHAR(255), record_id INT, PRIMARY KEY(id) ) ENGINE=InnoDB CHARACTER SET utf8; success! timestamp: 2012-01-11T22:04:00.198000 CREATE TABLE auth_event( id INT AUTO_INCREMENT NOT NULL, time_stamp DATETIME, client_ip VARCHAR(255), user_id INT, INDEX user_id__idx (user_id), FOREIGN KEY (user_id) REFERENCES auth_user(id) ON DELETE CASCADE, origin VARCHAR(255), description LONGTEXT, PRIMARY KEY(id) ) ENGINE=InnoDB CHARACTER SET utf8; success! timestamp: 2012-01-11T22:04:00.21 CREATE TABLE auth_cas( id INT AUTO_INCREMENT NOT NULL, user_id INT, INDEX user_id__idx (user_id), FOREIGN KEY (user_id) REFERENCES auth_user(id) ON DELETE CASCADE, created_on DATETIME, service VARCHAR(255), ticket VARCHAR(255), renew CHAR(1), PRIMARY KEY(id) ) ENGINE=InnoDB CHARACTER SET utf8; success! timestamp: 2012-01-11T22:04:00.222000 CREATE TABLE joke( id INT AUTO_INCREMENT NOT NULL, joketext LONGTEXT, created_on DATETIME, created_by INT, INDEX created_by__idx (created_by), FOREIGN KEY (created_by) REFERENCES auth_user(id) ON DELETE CASCADE, PRIMARY KEY(id) ) ENGINE=InnoDB CHARACTER SET utf8; success! timestamp: 2012-01-11T22:04:00.233000 CREATE TABLE category( id INT AUTO_INCREMENT NOT NULL, name LONGTEXT, PRIMARY KEY(id) ) ENGINE=InnoDB CHARACTER SET utf8; success! timestamp: 2012-01-11T22:04:00.243000 CREATE TABLE joke_category( id INT AUTO_INCREMENT NOT NULL, joke_id INT, category_id INT, PRIMARY KEY(id) ) ENGINE=InnoDB CHARACTER SET utf8; success! timestamp: 2012-01-14T19:48:00.511000 ALTER TABLE joke_category ADD joke_id__tmp
[web2py] Re: migration not working
...which I did...
Ummm... that sort of suggests I might as well ALWAYS set
migrate=False in my DAL connector for MySQL?
I don't mind--it's not too hard to make the changes and it helps focus
on db design.
Sounds like Postgres is a better longterm choice.
On Jan 14, 10:39 pm, Massimo Di Pierro
wrote:
> Mysql does not support multiple alter table in one transaction that is
> why it is possible for mysql to get into this state. notice this
> cannot happen with postgresql which perform the entire migration in
> one transaction.
>
> You need to manually ALTER TABLE and DELETE joke_id__tmp
>
> On Jan 14, 10:02 pm, Likit wrote:
>
>
>
>
>
>
>
> > I am trying to use the many-2-many DAL syntax suggested by the manual
> > and the examples. In this approach, a field name is associated with a
> > TABLE name.
>
> > I have not been able to get this to work with MySQL. So, I have used
> > the more conventional approach of using an integer field to hold the
> > key value from the relationship table. Instead of dogs and persons I
> > have jokes and categories, but it's the same thing. The goal is to be
> > able to express relations for jokes with no category, jokes with one
> > or more categories, categories with one or more jokes, and categories
> > with no jokes. The ideal query returns all of these, if they exist.
>
> > Here is the model for my "conventional" approach:
>
> > jodb.define_table('joke',
> > Field('joketext', 'text',length=2048),
> > Field('created_on', 'datetime', default=request.now),
> > Field('created_by', jodb.auth_user, default=auth.user_id))
>
> > jodb.define_table('category',
> > Field('name', 'text'))
>
> > jodb.define_table('joke_category',
> > Field('joke_id', 'integer'),
> > Field('category_id', 'integer'))
>
> > jodb.category.name.requires = IS_NOT_EMPTY()
> > jodb.joke.joketext.requires = IS_NOT_EMPTY()
>
> > All was good except I couldn't get the query to work using the persons/
> > dogs many-to-many approach. I was not getting jokes with no
> > categories and categories with no jokes. So, I decided to follow the
> > manual more closely to see if I could try the queries from the
> > example.
>
> > Thus, I tried to change the field definitions in the relations table
> > to associate the id field with a TABLE as in:
>
> > jodb.define_table('joke_category',
> > Field('joke_id', jodb.joke),
> > Field('category_id', 'integer'))
>
> > Yes, I would need to do the same for category, but I wanted to do one
> > at a time because I had read somewhere that MySQL migrations did not
> > work with multiple pending changes (don't know if that is really
> > true...). Anyway, seemed easier to do one at a time. So, I made the
> > immediately preceding change in the model. In attempting to do the
> > migration, MySQL got an error 150. When this happens, either web2py or
> > MySQL will hang. So, I stopped everything.
>
> > Upon resuming everything, I tried to go back to the original way I had
> > it (above). This appears to cause a second migration with the
> > following result:
>
> > 127.0.0.1.2012-01-14.19-48-00.189cb495-7851-429d-a0cb-bd23156431f1
>
> > (1060, u"Duplicate
> > column name 'joke_id__tmp'")
> > VERSION
>
> > web2py™ (1, 99, 4, datetime.datetime(2011, 12, 14, 14, 46, 14),
> > 'stable')
> > Python
> > TRACEBACK
>
> > Traceback (most recent call last):
> > File "c:\web2py\gluon\restricted.py", line 204, in restricted
> > exec ccode in environment
> > File "c:/web2py/applications/pyjokes/models/db.py", line 55, in
> >
> > Field('category_id', 'integer'))
> > File "c:\web2py\gluon\dal.py", line 5097, in define_table
> > polymodel=polymodel)
> > File "c:\web2py\gluon\dal.py", line 728, in create_table
> > fake_migrate=fake_migrate)
> > File "c:\web2py\gluon\dal.py", line 816, in migrate_table
> > self.execute(sub_query)
> > File "c:\web2py\gluon\dal.py", line 1359, in execute
> > return self.log_execute(*a, **b)
> > File "c:\web2py\gluon\dal.py", line 1353, in log_execute
> > ret = self.cursor.execute(*a, **b)
> > File "c:\web2py\gluon\contrib\pymysql\cursors.py", line 108, in
&
[web2py] Re: what triggers a db migration
Thanks, all. This is closed. I am progressing to more deployment complexity: locally develop on xampp with apache and mysql; occasionally run rocket/sqlite if I can't figure things out move things to SuSE server in my office and access behind firewall on own network move things to Linode (probably Centos) after they really, really work! Slowly but surely I am getting to understand this. Deployment still seems the biggest bug-a-boo. On Jan 15, 12:51 pm, Massimo Di Pierro wrote: > +1 > > On Jan 15, 1:21 am, pbreit wrote: > > > > > > > > > Migrations are pretty powerful but problems can arise. As I've suggested > > elsewhere, best to develop against SQLite and then deploy as desired. In > > development, I change models frequently. If I run into a problem that I > > can't fix with migrations, I might delete the whole database folder and > > start over. That's simple with SQLite. > > > The sql.log is just a log. I think web2py compares the .table files to the > > database to figure out what needs to be migrated. > > > I think fake_migrate rebuilds the .table files according to the model files > > without touching the DB. > > > There's probably room for improvement in the documentation, especially > > around fixing broken migrations, what the files are in the database folder > > and what migrate/fake_migrate actually do.
[web2py] Re: examples app doesn't run on apache
I am starting simple: - develop locally on xampp/mysql; try things out on rocket/sqlite if I can't figure something out - move to SuSELinux server in my office and access across lan - move to linode probably on centos for the real world after things really, really work I commented out caching decoration in the controller for examples and that did not fix things. I will look into Massimo's suggestion although it's only somewhat applicable as I'll be comparing an apache config on linux to one on windows--but there might be some hints there. On Jan 15, 8:57 am, Anthony wrote: > > I frequently hear people running into trouble running Apache and/or Python > > on Windows. I'd strongly suggest developing against the Windows download > > and then deploying on *nix. > > I don't know about Apache, but there should be no problem running the > source version of web2py on Windows along with a Windows installation of > Python (though you should also have pywin32 > installed:http://sourceforge.net/projects/pywin32/). Probably best to stick > with > Rocket for development on Windows. > > Anthony
[web2py] permissions problems on ubuntu 11.10
I thought this was going to be easy. Boy, was I wrong. Lots of good documentation for ubuntu 11.10 from both ubuntu and linode. I got apache, ssl, mysql, php, mod_wsgi, and phpmyadmin working perfectly first time, no stumbles. But, then I tried to make the mods to httpd.conf (really, its includes) to enable wsgi for web2py. No go. I think I am running into all kinds of user / group permissions problems where the OS blocks permission. Here is the apache2 error log of my last attempt: Thu Jan 19 22:42:19 2012] [error] ERROR:web2py:Traceback (most recent call last): [Thu Jan 19 22:42:19 2012] [error] File "/usr/lib/python2.7/web2py/ gluon/main.py", line 441, in wsgibase [Thu Jan 19 22:42:19 2012] [error] create_missing_app_folders(request) [Thu Jan 19 22:42:19 2012] [error] File "/usr/lib/python2.7/web2py/ gluon/admin.py", line 451, in create_missing_app_folders [Thu Jan 19 22:42:19 2012] [error] os.mkdir(path) [Thu Jan 19 22:42:19 2012] [error] OSError: [Errno 13] Permission denied: '/usr/lib/python2.7/web2py/applications/welcome/databases' [Thu Jan 19 22:42:19 2012] [error] I am really leary of running big scripts because things change that I don't want to change. The only real exceptions I need are a way to setup an alias to access phpmyadmin via ssl to administer the mysql databases. it would be nice to also have an alias for a trivial static html file so that I can quickly verify that the overall domain is accessible. I tried to follow a script that Massimo recommended, just pasting in things for the default file (defining virtual hosts). I also ran any step manually from the script that I had not previously performed. The sudo's to do what I think are creating a web2py group may not have worked. Here is the link to that script: http://code.google.com/p/web2py/source/browse/scripts/setup-web2py-ubuntu.sh When running the sudo - u python -c ... lines at the bottom I have never successfully gotten a parameters.80 or parameters.443 file (realize I don't have the exact name correct--but I am not doing it manually--trying to let the bash command do it) to be created. Likely part of the problem. The only changes I have made are to split the .conf files for virtualhost *:80 and virtualhost *:443 into two files instead of combining them and adding the aliases mentioned above. I think there is also a question of whether I need a DocumentRoot directive. I have tried with and without (which is probably right) and it doesn't matter. I think the os permissions are the problem. "web2py" needs rwx privileges on pretty much everything within the web2py directory I guess. But, I have not seen any of the recipes explicitly address this--so I had not realized that I had to manually change them. I need to learn a lot more about user and group permissions on Linux. There are just layers and layers to the onion. I thought I was mostly programming python, learning a framework, configuring apache, cleaning up html and css, and understanding how the framework does its queries-- all of which I have made much progress on. But, there is seemingly no end to the multi-layered complex dependencies. I bet this will turn out to be something simpler than I realize. At least I hope so. Thanks.
[web2py] Re: permissions problems on ubuntu 11.10
Solved. Needed to add the www-data group to the location where I installed web2py and to phpmyadmin. Everything works as I want it to including a little static file served by apache to make sure the server is up, access via ssl to phpmyadmin, access to the test app for wsgi to make sure wsgi is running, and access to all the web2py applications and admin ui. Now I need to do some cleanup for fixes that weren't fixes. I guess this user/group permission thing doesn't come up for that many people if you follow the exact recipes. Linux distros tend to ship with certain users/groups set up as owners of cerrtain directories. If you put things there then all is good. If you put things in other places, then you have to adjust. It just seems like a bad idea to put web2py in the www directory even if it set to Deny from All. It just doesn't need to be there. Pretty much nothing should be in www except "sites" built primarily of static html files. Generally linode has been a very good experience and ubuntu is legions better than xampp, of course. On Jan 19, 3:02 pm, Likit wrote: > I thought this was going to be easy. Boy, was I wrong. Lots of good > documentation for ubuntu 11.10 from both ubuntu and linode. I got > apache, ssl, mysql, php, mod_wsgi, and phpmyadmin working perfectly > first time, no stumbles. > > But, then I tried to make the mods to httpd.conf (really, its > includes) to enable wsgi for web2py. No go. I think I am running > into all kinds of user / group permissions problems where the OS > blocks permission. > > Here is the apache2 error log of my last attempt: > > Thu Jan 19 22:42:19 2012] [error] ERROR:web2py:Traceback (most recent > call last): > [Thu Jan 19 22:42:19 2012] [error] File "/usr/lib/python2.7/web2py/ > gluon/main.py", line 441, in wsgibase > [Thu Jan 19 22:42:19 2012] [error] > create_missing_app_folders(request) > [Thu Jan 19 22:42:19 2012] [error] File "/usr/lib/python2.7/web2py/ > gluon/admin.py", line 451, in create_missing_app_folders > [Thu Jan 19 22:42:19 2012] [error] os.mkdir(path) > [Thu Jan 19 22:42:19 2012] [error] OSError: [Errno 13] Permission > denied: '/usr/lib/python2.7/web2py/applications/welcome/databases' > [Thu Jan 19 22:42:19 2012] [error] > > I am really leary of running big scripts because things change that I > don't want to change. The only real exceptions I need are a way to > setup an alias to access phpmyadmin via ssl to administer the mysql > databases. it would be nice to also have an alias for a trivial > static html file so that I can quickly verify that the overall domain > is accessible. > > I tried to follow a script that Massimo recommended, just pasting in > things for the default file (defining virtual hosts). I also ran any > step manually from the script that I had not previously performed. > The sudo's to do what I think are creating a web2py group may not have > worked. Here is the link to that script: > > http://code.google.com/p/web2py/source/browse/scripts/setup-web2py-ub... > > When running the sudo - u python -c ... lines at the bottom I have > never successfully gotten a parameters.80 or parameters.443 file > (realize I don't have the exact name correct--but I am not doing it > manually--trying to let the bash command do it) to be created. Likely > part of the problem. > > The only changes I have made are to split the .conf files for > virtualhost *:80 and virtualhost *:443 into two files instead of > combining them and adding the aliases mentioned above. I think there > is also a question of whether I need a DocumentRoot directive. I have > tried with and without (which is probably right) and it doesn't > matter. I think the os permissions are the problem. "web2py" needs > rwx privileges on pretty much everything within the web2py directory I > guess. But, I have not seen any of the recipes explicitly address > this--so I had not realized that I had to manually change them. I > need to learn a lot more about user and group permissions on Linux. > There are just layers and layers to the onion. I thought I was mostly > programming python, learning a framework, configuring apache, cleaning > up html and css, and understanding how the framework does its queries-- > all of which I have made much progress on. But, there is seemingly no > end to the multi-layered complex dependencies. > > I bet this will turn out to be something simpler than I realize. At > least I hope so. Thanks.
[web2py] Ticket Issued: Unrecoverable
I have seen several posts about this problem. I just had the problem and solved it. However, none of the several posts I read had the answer that applied in my case. I did not read ALL of the posts mentioning this problem so some of them may have had an applicable answer. If you are a linux noob like me you might run into this problem. The problem is caused by the fact that web2py/wsgi do not have the privileges to write to the directory that holds you application. If you have set up wsgi to run as a daemon as www-data then www-data must have access to the applications directories. You should NEVER let web2py or wsgi (the daemon) be root. If you did, then anyone who managed to get into your web2py admin site with your root password could write arbitrary python code and execute which could do anything to your server because root has access to everything. If the daemon is www-data and you allow www-data GROUP
[web2py] Ticket Issued: Unrecoverable
I have seen several posts about this problem. I just had the problem and solved it. However, none of the several posts I read had the answer that applied in my case. I did not read ALL of the posts mentioning this problem so some of them may have had an applicable answer. If you are a linux noob like me you might run into this problem. The problem is caused by the fact that web2py/wsgi do not have the privileges to write to the directory that holds your application. If you have set up wsgi to run as a daemon as www-data then www-data must have access to the applications directories. You should NEVER let web2py or wsgi (the daemon) be root. If you did, then anyone who managed to get into your web2py admin site with your root password could write arbitrary python code and execute it which could do anything to your server because root has access to everything. If the daemon is www-data and you allow www-data GROUP to have rwx privileges only in your www directory and your web2py directory--and no other directories on your server (you may have other apps that need www-data group privileges such as phpmyadmin) then the worst that can happen is that www and web2py might be compromised (which is bad enough but quite contained). Now, once the wsgi daemon runs as www-data then so does web2py. Web2py needs rwx privileges on applications. If you don't give the www-data group rwx privileges then web2py returns an unrecoverable ticket--it can't do anything with that application.
