Re: [web2py] Re: try: for-loop

2012-05-31 Thread Doug Philips
BTW: Take a look at the Style Guide for Python Code (PEP-8) for lots
of really helpful tips.
The empty collections (of which list is one) can be found under
Programming Recommendations of PEP-8:
 http://www.python.org/dev/peps/pep-0008/#programming-recommendations

--Doug




On Wed, May 30, 2012 at 2:48 PM, Niphlod  wrote:
> I am astonished... I missed this in almost 3 years of python development :D
>
> I genuinely repent and knee before you, Anthony 
>
> Niphlod
>
> Il giorno mercoledì 30 maggio 2012 18:11:32 UTC+2, Anthony ha scritto:
>>
>> On Wednesday, May 30, 2012 11:59:13 AM UTC-4, Niphlod wrote:
>>>
>>> uhm, depends if he uses lists and articles is = [], there will be a
>>> problem.
>>
>>
>> What problem? An empty list evaluates to False.
>>
>> Anthony


[web2py] Question about layout.html

2012-06-06 Thread Doug Philips
I'm getting ready to deploy my first web2py application.
First off: THANK YOU!
I couldn't have pulled this off (I'm a systems programmer, not a web
programmer).

While I have only one application to deploy, I am hoping there are others.
As such, I want to use welcome as the 'landing page' which will show
what the other available applications are.

So, I thought, why not just tweak up the welcome app, it has a nice,
friendly, 'welcoming' name even. :-)

This app doesn't need authentication, so I've gone into the models,
etc. and removed all the references to Auth and auth and such.

When I did that, I started to get this weird 'False' showing up in the
upper right of the page.

I eventually tracked it down to applications/welcome/views/layout.html:
line 79 (or thereabouts):
 {{='auth' in globals() and
auth.navbar(separators=(' ',' | ',''))}}

So should I just remove that line?
It looks like it is written to be 'self-adusting'.
I "fixed" my mutant welcome app by changing it to:
{{=('auth' in globals() and
auth.navbar(separators=(' ',' | ',''))) or ''}}

Hopefully this question makes sense?
I guess I'd either like to see that change adopted or a comment added
to layout.html indicating that it is OK to remove that line (really, I
have no idea if that would break anything which is why I didn't remove
it already, I'm under the gun to get the app out).

Thanks,
 --Doug


Re: [web2py] Re: Question about layout.html

2012-06-06 Thread Doug Philips
Ok, I've entered an issue for it:
http://code.google.com/p/web2py/issues/detail?id=841
Thanks!
   ---Doug


On Wed, Jun 6, 2012 at 9:49 AM, Anthony  wrote:
> Good catch -- we should make that change. In your case, you can just remove
> that line.


[web2py] Setting initial password when registering?

2012-06-06 Thread Doug Philips
Hello all, again,

(I'm guessing the answer to this is in 'the book' , but that's down
right now and when I went looking last night I couldn't seem to figure
out where.)

For the app I'm deploying which does require users to authenticate
(login), I want to use the verification facility.
I have the following settings in db.py. (I just double checked and I
don't have any other code which seems to override or reset these
values)

auth.settings.registration_requires_verification = True
auth.settings.registration_requires_approval = False
auth.settings.reset_password_requires_verification = True


Everything works great, the email configuration is fine, that all works.
But there is nowhere on the registration form to set an initial password.
Beta testers are complaining that after following the link in the
email to verify their registration, they than have to go through the
hassle of the "Lost Password" process.

I'm a bit chagrined as I was so busy getting the rest of the app ready
that I didn't even notice this.
Do I have my app misconfigured? I'd really like to be able to have
folks pick a password when they register, which is what the beta
testers want, and (to my inexperienced web2py eyes) that seems a
reasonable request.

Thanks,
--Doug


[web2py] Re: Setting initial password when registering?

2012-06-06 Thread Doug Philips
Ok, so, for the record, I'm an idiot. :-)
My app was generated with the admin wizard.
Apparently when I was just starting to get my web2py feet wet, I went
in and tinkered very early on and marked the password field as
writable=False ...
removing that allows for a password and repeat/verify password field to show up.
Sorry for the hassle.
   --Doug

P.S. I've created an issue in the tracker requesting that perhaps the
wizard use the extra_fields mechanism instead of creating the
auth_user table from whole cloth. Would just make it easier to prevent
this kind of self-foot-shooting, but more importantly it means not
having to keep the code in synch between the gluon code and the admin
application. Not a big deal though.


On Wed, Jun 6, 2012 at 11:28 AM, Doug Philips  wrote:
> Hello all, again,
>
> (I'm guessing the answer to this is in 'the book' , but that's down
> right now and when I went looking last night I couldn't seem to figure
> out where.)
>
> For the app I'm deploying which does require users to authenticate
> (login), I want to use the verification facility.
> I have the following settings in db.py. (I just double checked and I
> don't have any other code which seems to override or reset these
> values)
>
> auth.settings.registration_requires_verification = True
> auth.settings.registration_requires_approval = False
> auth.settings.reset_password_requires_verification = True
>
>
> Everything works great, the email configuration is fine, that all works.
> But there is nowhere on the registration form to set an initial password.
> Beta testers are complaining that after following the link in the
> email to verify their registration, they than have to go through the
> hassle of the "Lost Password" process.
>
> I'm a bit chagrined as I was so busy getting the rest of the app ready
> that I didn't even notice this.
> Do I have my app misconfigured? I'd really like to be able to have
> folks pick a password when they register, which is what the beta
> testers want, and (to my inexperienced web2py eyes) that seems a
> reasonable request.
>
> Thanks,
>    --Doug


[web2py] SQLFORM.grid - questions/suggestions

2012-06-07 Thread Doug Philips
I know that SQLFORM.grid is marked experimental and thus I shouldn't
expect it to be as refined as some of the other parts of web2py, but
I'm hoping that also means it is still a bit open to adjustment?

SQLFORM.grid has saved me a ton of work, but there are few things I
wish it did slightly better.

1. The 'Add' button's button text seems to be hard coded. I'd like to
make that a parameter with a default value of 'Add' (in some forms,
'Add' is ok, and in others I'd like to set it to something else, such
as 'New Proposal').

2. The submit button's button text on the edit forms for a row (the
ones you get when use the 'edit' button from the tabular view) is hard
coded to 'Submit'. I'd like to suggest changing it to a parameter and
making the default value of that parameter be 'Save Changes'.

3. Would it be OK to add a "separator" parameter, as in the base
SQLFORM? (I've gone into the code to just change the hard coded
value.)

I'll make bug tracker entries (one or three) unless the current
behaviour can't be changed.

Thanks!
 --Doug


Re: [web2py] SQLFORM.grid - questions/suggestions

2012-06-07 Thread Doug Philips
On 6/7/12, Richard Vézina  wrote:
> I am almost sure I saw a email were Massimo explain how to set a different
> text for "submit" button, can't find it back...
>
> Have you try with SQLFORM attribute ? Or SQLFORM attribute with "_" before
> : _submit_button = 'Submit'

Well, I looked at the code in gluon/sqlhtml.py
the SQLFORM class @staticmethod grid() does not take submit_button or
_submit_button and doesn't (so far as I can tell) accept *args or
**kwargs parameters. Further, the code itself for handling the edit
case creates a SQLFORM and passes in the submit_button parameter with
a hard coded value. But I'm new to this code base and there may be
other paths or other places where things are changed after they are
made...

Thanks!
 --Doug


Re: [web2py] SQLFORM.grid - questions/suggestions

2012-06-07 Thread Doug Philips
On Thu, Jun 7, 2012 at 2:12 PM, Anthony  wrote:
> I'm not sure if this is in stable, but in trunk you can specify dictionaries
> of args to pass to SQLFORM, including separate dictionaries for create,
> edit, and view
> forms: http://code.google.com/p/web2py/source/browse/gluon/sqlhtml.py#1500.

I'm running:
Version 1.99.7 (2012-03-04 22:12:08) stable

and that code isn't there. Glad it will be in the stable tree at some
point though, thanks for the pointer/info!

-=Doug


[web2py] Symbolic links in the applications directory...

2012-06-07 Thread Doug Philips
I can only find one mention of symbolic links in the book, under:
http://web2py.com/books/default/chapter/29/4#Application-init

That seems to imply that symbolic links at the applications directory
level are ok.
Specifically, if I have the following applications:
ProjectX2010
ProjectX2011
ProjectX2012
and then I symbolic link ProjectX to point to one of them, say
ProjectX2012, is the underlying database safe if I have clients using
the ProjectX2012 and ProjectX applications at the same time?

The one mention of symbolic links seems to say so, but that's pretty
'backhanded' way of saying symlinks won't cause any corruption with
simultaneous access, etc.
I'm interested because I have several instances of ProjectX (and
ProjectY) planned, and if symbolic links are safe, for my use case
here, symbolic links seem a lot easier than doing URL
rewriting/routing. If I've missed where this is talked about in the
book, a link or even a better, a better term to search for would be
great!

Thanks,
 --Doug


Re: [web2py] Re: Symbolic links in the applications directory...

2012-06-08 Thread Doug Philips
On Fri, Jun 8, 2012 at 8:12 PM, pbreit  wrote:
> You can certainly symlink to an application. Having two applications access
> the same database is a different story. In theory it should work fine (but I
> would never do it).

Thanks. I guess I don't understand how web2py can tell the difference.
With URL rewriting I understand that what might look like different
apps to the client become the same app, but with symlinks wouldn't it
look to web2py as two different apps that are touching the same
backend (SQLite at least)?

-=Doug


Re: [web2py] Re: Symbolic links in the applications directory...

2012-06-09 Thread Doug Philips
On Sat, Jun 9, 2012 at 12:01 AM, pbreit  wrote:
> Hard to say how URL rewriting would affect all this. Maybe I don't
> understand what you are doing or trying to do.

Isn't a matter of effecting this, it is a matter of presenting one
application under several names, which, I thought, URL rewriting could
do.


> The SQLite DB for each app is usually in the app's "databases" directory.
> Have you done something to make each app use the same SQLite DB?

No, I have not. There is really one one SQLite DB; it is accessible
via two different paths, because of the symboic link.

> If you do something like this:
>
> $ ln -s applications/projectX2012 applications/projectX
>
> Then http://myserver.com/projectX and http://myserver.com/projectX2012 will
> be running the same app. Web2py thinks the symlink is just another app.

Right, that is what I was asking about, Web2py thinks it is two apps,
but those two apps are sharing the same database because of the
symbolic link.

-=Doug


Re: [web2py] Re: Symbolic links in the applications directory...

2012-06-10 Thread Doug Philips
On Sat, Jun 9, 2012 at 11:50 PM, pbreit  wrote:
> Hmmm...I still don't totally understand what you are trying to do and what 
> your setup looks like.

I have a series of applications:
ProjectX2012, ProjectX2011, ProjectX2010, etc
I want to provide a short-cut URL:
ProjectX
that resolves to the current project.

I thought I was going to have to use URL rewriting to do that, until I
came across
  http://web2py.com/books/default/chapter/29/4
  Here are four ways to set the default application:

  Call your default application "init".
  Set default_application to your application's name in routes.py
  Make a symbolic link from "applications/init" to your
application's folder.
  Use URL rewrite as discussed in the next section.

So, what I am trying to find out is how I should choose between the
last two options mentioned above, specifically, for simple application
name aliasing, should I use symbolic links or should I use URL
rewrite?

Since, as far as I have been able to find, that is the only place that
symbolic links are mentioned, I have no idea what the implication and
repercussions of using symbolic links are.

>From what I can tell of reading the URL rewrite description, to change
routing I have to either:
restart the server   OR
get to the admin interface and use the reload routes button (which
for all I know, restarts the server behind my back).

Both of those options are a lot more involved and disruptive than just
changing a symbolic link.
BUT would changing a symbolic link on a running server cause problems?
Where do I find that out?

Thanks,
 --Doug


Re: [web2py] Re: Symbolic links in the applications directory...

2012-06-10 Thread Doug Philips
On Sun, Jun 10, 2012 at 10:38 AM, Anthony  wrote:
> Reloading routes does not restart the server -- it just re-reads the routes
> file.

OK... just checked the code for the stable release... but, unless the
docs say so, I can't really rely on that behaviour not changing...


>> Both of those options are a lot more involved and disruptive than just
>> changing a symbolic link.
> I don't think clicking a button is any more involved or disruptive than
> changing a symbolic link, but you can make it even easier by simply visiting
> the /admin/default/reload_routes URL (if you're not logged in to admin at
> the time, you'll be prompted to login before routes.py is reloaded).

Actually, that is assuming a debug/development server.
The docs say not to leave that enabled for a production server (I
followed that advice).
So, I have to get to the headless production server (ssh in).
Start a localport only duplicate web2py server.
Setup an SSH tunnel so I can get to the locaport only server.
then I can finally point my browser at it and click to get the admin
and click to get the routes reloaded.
 OR
I can just ssh to the server.
then move a symlink.

Moving a symlink is a lot easier. If it is trustworthy.



>> BUT would changing a symbolic link on a running server cause problems?
>> Where do I find that out?
>
>
>  Probably won't be a problem. Best way to find out is to try it.

Well... that's not a good enough answer to trust for production. Sure
I can test it on a dev server, but all that really means is that it
didn't cause a problem the few times I tried it.

"It works for me the few times I tried it" isn't an answer I'm going
to trust for a running production server. I've been doing software for
far too long to fall for that! :-)

Thanks,
-Doug


Re: [web2py] Re: Symbolic links in the applications directory...

2012-06-10 Thread Doug Philips
On Sun, Jun 10, 2012 at 11:20 AM, Anthony  wrote:
> Aside from the built-in Rocket server (which is primarily intended for
> development), web2py doesn't have any control over the server so can't
> generally restart your server -- so you should be able to rely on the fact
> that reloading routes will not be achieved by restarting the server.

Hmmm, OK. That wasn't clear to me from that part of The Book.


> No, admin (including reloading routes) can be made available on a production
> server -- you just have to access it via HTTPS. This should be fine for
> production.

That's one way to go, but I was following the example from
http://web2py.com/books/default/chapter/29/13

An easy way to setup a secure production environment on a server is to
first stop web2py and then remove all the parameters_*.py files from
the web2py installation folder. Then start web2py without a password.
This will completely disable admin and appadmin.

1.
nohup python web2py --nogui -p 8001 -i 127.0.0.1 -a '' &
Next, start a second web2py instance accessible only from localhost:

1.
nohup python web2py --nogui -p 8002 -i 127.0.0.1 -a '' &
and create an SSH tunnel from the local machine (the one from which
you wish to access the administrative interface) to the server (the
one where web2py is running, example.com), using:

1.
ssh -L 8002:127.0.0.1:8002 usern...@example.com
Now you can access the administrative interface locally via the web
browser at localhost:8002.


> What's the potential problem? Changing the symlink either points to the new
> app or it doesn't -- if it does, you should be good to go.

I have no idea from reading the docs if it should be safe, or not.
For all I know if I happen to change the symbolic link at the wrong
time, web2py will traverse the path 'in the middle of something
important' and end up in a different application than it was before
(projectX links to projectX2011 and I get "luck" and switch the link
to projectX2012 at just the wrong moment).

You're asking me to already tell you what could go wrong.
I'm asking for some help to know whether or not using symlinks is
safe. I'm not a core developer, I've just been using web2py for a few
weeks and so far as I know, The Book is what I am supposed to rely on
to figure out how web2py works and how to use it. What the code does
today doesn't help. I'm running the stable release for a reason,
because I don't want to have to track all the commits to figure out if
my server environment in production is making some assumption(s) based
'on reading the code' that just happen to be accidents of
implementation and not design goals to be preserved as the project
evolves.

At this point, it would be great if there could be a little content
added to the book, just after the only existing reference to symbolic
links:
Symbolic links can be used to alias application names, but web2py does
not guarantee that it is safe to move symbolic links while the server
is running. (In contrast, using the admin interface 'reload routes'
button is guaranteed to be safe (i.e. not affect any operations 'in
flight' when it is activated)).

Thanks,
  --Doug


Re: [web2py] Re: Symbolic links in the applications directory...

2012-06-10 Thread Doug Philips
On Sun, Jun 10, 2012 at 2:26 PM, Anthony  wrote:
>> But I think if you have users in mid-session and you switch apps on them,
>> it could present problems no matter how you try to solve it.
>
> That's true. Although using routes is probably the safer method (should be
> no problems within a single request), a single page typically involves
> multiple requests (i.e., the HTML, plus requests for the static files, plus
> possible Ajax requests), so changing the routing could result in the
> multiple requests from a given page hitting two different apps.

Thanks for the clarifications.
What I am hearing is that even the 'reload routes' button is not safe
to use on a busy server.

I've created an issue
http://code.google.com/p/web2py/issues/detail?id=847 with some
potential/suggested minor content changes.
I doubt they match the voice of the rest of the document, but
hopefully some concrete suggested wording is better than none.

Thanks,
 --Doug


Re: [web2py] Re: Web2Py printed book: is it up to date?

2012-06-12 Thread Doug Philips
On Tue, Jun 12, 2012 at 4:49 AM, Alec Taylor  wrote:
> o.O - Just found this: "web2py Application Development Cookbook" -
> http://www.packtpub.com/web2py-application-development-recipes-to-master-python-web-framework-cookbook/book
>
> Looks quite useful, I think I will get that instead :)
>
> [I won't buy the book for a few days so feel free to share your advice]

Check out the TOC and see if the topics interest you.
I bought it because I was going from "Web2py? what's that?" to
"deployed server" in two weeks, but I wish I had read the TOC first as
a lot of the recipes in the cookbook didn't address what I wanted to
know. I don't mind having bought it because that's a way to give
back/show appreciation, but I do wish I'd clicked on the "Sample
Chapters" tab and looked at the TOC to set my expectations first. :-)

-=Doug


Re: [web2py] web2py book now free in PDF

2012-06-12 Thread Doug Philips
On Tue, Jun 12, 2012 at 9:17 AM, Massimo Di Pierro
 wrote:
> The official web2py book is now free for everybody:
>
> https://dl.dropbox.com/u/18065445/web2py/web2py_manual_4th.1.pdf

Thank you!

-=Doug


Re: [web2py] Re: Book down?

2012-06-12 Thread Doug Philips
isup.me says its not just a few of us. :-(

On Tue, Jun 12, 2012 at 3:57 PM, Anthony  wrote:
> Same here.
>
>
> On Tuesday, June 12, 2012 3:49:40 PM UTC-4, pbreit wrote:
>>
>> The Book is down for me since 12:45 PT.


[web2py] What parts of web2py won't work without scripting?

2012-06-15 Thread Doug Philips
Hey all,
  I'm looking at deploying a web2py app in an environment where any
client side scripting is forbidden. Is web2py usable in that case?
There seems to be a lot of scripting going on in the default
layout.html. As a systems programmer with no web experience, I have no
idea what will stop working if I clients can't use Javascript, etc.
The Book isn't very clear on that either. SQLFORM, SQLFORM.grid,
SQLFORM.smartgrid, and Crud seem very much intertwined with scripting,
but I don't really know. I don't even know where to look.
Thanks,
--Doug


Re: [web2py] Re: What parts of web2py won't work without scripting?

2012-06-16 Thread Doug Philips
On Fri, Jun 15, 2012 at 6:10 PM, Anthony  wrote:
> Note, web2py will have the same limitations as any framework in this case.

Right. I just didn't know how much of the Web2py forms "magic"
depended-on/required client-side scripting.


> Most things should work fine, except of course anything Ajax related
> (including Ajax components). Most of what you'll lose can be discerned by
> inspecting web2py.js. Javascript is used for the following:

Well, as a back-end programmer, what I "discern" is a lot of gobblety
gook. I'm using Web2py because I have the backend experience to do the
business logic, etc. and want web "front end/gui" interfaces, so I
want to "subcontract" all the web stuff that I don't know (or at least
not yet) to the expert (Web2py). :-)


..
> So, overall, you don't lose anything major, and there are easy alternatives
> in most cases.

Thanks for that list!
As long as things generally work (though it sounds as if the menu
might not, I'll have to try that and find out) I'll be fine.


> By the way, why is client-side scripting forbidden?

I'm looking to deploy a little helper web-app in a corporate
environment that's pretty locked down.

-=Doug


[web2py] database locking web2py vs. "external" access...

2012-06-16 Thread Doug Philips
I'm looking to potentially use background task(s) to offload sending
of emails and updating of other offline resources, and I'm trying to
sort out what the implications are.

In the PDF of the book (just released for free, thank you!), page 389,
section 8.5 "Sending messages using a background task" is the
following text:
"Also notice that it is important to commit any change as soon as
possible in order not to lock the database to other concurrent
processes.
As noted in Chapter 4, this type of background process should not be
executed via cron (except perhaps for cron @reboot) because you need
to be sure that no more than one instance is running at the same
time."

So I'm puzzled, if the database is locked, then shouldn't multiple
accessors should be fine?

The email task example, spanning pages 388-389, does a db.commit()
every in body of a loop that is processing the rows rather than at the
end of the loop.
What/where are the locks obtained in this sample script?
Since the script never exits, it can't be just running the script?
Is the lock obtained on the update call and then released on the
commit? I'm not having much luck figuring out how to avoid locking
issues when I can't seem to figure out when Web2py takes and releases
locks. I've just re-read chapter 6 and am still puzzled. Pointers to
where else to look are appreciated.

Thanks
   -Doug


Re: [web2py] Re: database locking web2py vs. "external" access...

2012-06-16 Thread Doug Philips
On Sat, Jun 16, 2012 at 2:38 PM, Niphlod  wrote:
> if you're familiar with database transactions, web2py istantiate a new
> transaction every request and commit (if no exceptions are thrown
> automatically) at the end of the request.

Yup, that part makes sense. And I assume it is true regardless of
which DB engine(?)
Which is why sending even one email in your controller is likely to
cause a delay for other users of the site.


> in a module, if you want to roll your own "daemon", a transaction is
> initiated at the start of the script. You have to manually commit to "save"
> those changes. Issuing db.commit() every loop assures you that the db is
> consistent with what your daemon has processed already.

Ok, so a transaction is opened when the script starts (actually, when
DAL is called/created)?
But here is where I get confused. First time through the loop, a
record is updated and db.commit() is called.
So far so good, the DB is committed and the transaction ends.
Second time around the loop, a record is updated (???) and db.commit()
is called. What transaction is that in?
When did that transaction start and why isn't the web app talking to
the same DB just as blocked by my daemon as it would be if I had done
the loop in my controller?


> The "blocking" part is referenced to databases like SQLite, that remain
> "blocked" if someone writes to the database and the commit is called later,
> i.e.
> - you have a queue of 500 messages
> - you start to send emails, updating or removing row by row to mark that
> email as sent
> - you commit at the end of the 500 block.

Yup, I get that you want to commit every loop and release a lock so
that other processes can get access to the DB...


> if the first daemon is still processing the queue, you'll have two processes
> running for the same task, and if you don't prepare things for multiple
> "workers", things can get ugly (e.g. two identical emails sent - one from
> the first process, the second from the second)

Right. What I'm trying to find out is how to write a separate daemon
that won't interfere with Web2py's access to the DB while it is
running and I'm not yet getting a good mental model of how to
structure the main loop (as per the example in The Book, on pages 388
and 389) as I cannot see there where transactions are started, just
where they are commited.

Thanks for trying to explain I hope I can come to an understanding but
I'm not quite there yet!

--=Doug


Re: [web2py] Re: database locking web2py vs. "external" access...

2012-06-17 Thread Doug Philips
On Sat, Jun 16, 2012 at 5:56 PM, Niphlod  wrote:
> The bit of magic that I forgot to tell explicitely is that db.commit()
> commits every opened transaction and starts a new one :D

Ok, just to double check, that is for all DB back-ends or just for SQLite?


> Note, if your site does mostly reading, than SQLite is fine with that.
> Additional note, if you're not using SQLite there are no "locks" at all.

Ah, OK. So it would be easier to write code that uses any DB except
for SQLite... Hmmm

> Take it like that. Transactions are a way to provide consistent data to
> every reader/writer connected to a database.

Right, I've got that part down, it is the "what is the magic that
Web2py is doing behind my back" part that I was (and still a little
bit) unclear on.


> So, if there is only one daemon running, it's safe to commit every 500 or
> every 1 with all the dbs: if you commit every row everytime you app will try
> to see the queue will see a "freshier" version of it, if you commit every
> 500 the app will see a "older" version of the queue.
> The caveat of locking is valid for sqlite only: if you commit every 500 the
> app won't be allowed to read or write to the db while processing 1 to 500.
> If you commit every 1, the app will "lock" less.

Ok, so I think we've established that is a special limitation for SQLite.
For all the other DBs, I can have my Web2py app inserting new records
willy nilly and the background process updating them (mail sent, mail
failed, whatever) without the Web2py app having performance/response
time issues due to database contention.

The trick comes in when SQLite is being used.
Just for completeness, if I do a commit on every loop, what is the
window where Web2py is causing the SQLite lock to be taken?
Is it just between the update_record and the commit?

I ask because I'm not yet clear on how it is that the commit on every
loop will allow the Web2py app to be more responsive than the commit
after all 500 cases. It must be that in the commit on every loop there
is a substantial chance for the Web2py app to get to the database. But
if a commit causes a new transaction to start, how does Web2py get in?
In fact, the example in the book shows an outer loop that looks for
new work, and when done, sleeps for 60 seconds. If the commit call
starts a new transaction, the on the very last of the 500 records,
we'll fall out of the inner loop and then sleep for 60 seconds, which
should lock Web2py out for a full minute? how does the server process
drop the transaction in that case so that the Web2py process has a
chance to get in to the DB?

(I'm beginning to think that The Book should just say: This example is
for any DB except SQLite, don't use SQLite for this kind of processing
???)

Thanks,
 --Doug

-- 





Re: [web2py] Re: database locking web2py vs. "external" access...

2012-06-19 Thread Doug Philips
On Mon, Jun 18, 2012 at 2:07 AM, Anthony  wrote:
> I don't think web2py is doing any magic -- I think it just relies on the
> standard database and database driver behavior.

"There in lies the rub" - the problem is that The Book isn't really
very easy to use to find out just what the DAL does for each DB and
what "shows through" as the behaviour of the underlying DB.

That is, The Book isn't really a reference manual, which is fine, but
it also leaves me wondering if there is something the DAL does but
that I just haven't found/remembered the right part of the The Book to
find it.



> A commit closes the open transaction, but it doesn't cause the db to lock
> again immediately. A new transaction doesn't really start until another
> insert/update/delete statement is issued.
>
> For more on SQLite locking, see http://www.sqlite.org/lockingv3.html.

Thanks!
I guess the issue here is that I don't really want to tie my code (or
the example in The Book) to any one DB, but there are DB "quirks"
which manage to show through, and as a new Web2py user, its hard to
build up a model of what the DAL does and what leaks through. :-)

For example, the content at the link you provided ends with "7.0
Transaction Control At The SQL Level" and talks about autocommit mode
vs. "out of autocommit mode". I searched The Book for "autocommit" and
found nothing. I also searched for SQLite and didn't find anything
mentioning that (that I was able to recognize).

Given the "commit at the end of a request" model that Web2py does, I
might guess that autocommit is being disabled?

It matters because with autocommit off, SELECTs will acquire SHARED lock.
If I am reading things correctly, that might cause this server script
to block any writes/updates from the Web2py app.

Specifically, if the server does not find any records to update (i.e.
no work to do), the query/SELECT it does to find work items would
provoke a SHARED lock, but with no records to update the server won't
do a commit (or anything else that i can see) that would drop that
lock?

Thanks,
-Doug


Re: [web2py] Re: database locking web2py vs. "external" access...

2012-06-19 Thread Doug Philips
On Mon, Jun 18, 2012 at 4:02 AM, Niphlod  wrote:
>> Ok, just to double check, that is for all DB back-ends or just for SQLite?
>>
> This is true for every backend.

Thanks!


> Web2py is not blocked, and again, it's only with SQLite that there is the
> problem. From the first update on the queue to the commit, no-one can access
> a SQLite database, hence an app trying to read from that will "hang".
> Anyway, in the "sleeping" loop, no modifications to the db are made so also
> with SQLite there will be no problems.

Ok. I was just about ready to believe that, but as per the link
Anthony sent (see my separate reply just sent), depending on whether
or not SQLite is in autocommit mode, doing a SELECT (query) could
cause it to take a SHARED lock which, while held, would block other
processes from acquiring the locks needed to update. That is, the
server script, just by looking for work and then sleeping, would lock
out any updates. But that depends on the mode of access (autocommit or
not)...


>> (I'm beginning to think that The Book should just say: This example is
>> for any DB except SQLite, don't use SQLite for this kind of processing
>> ???)

> Yep, I'll be fine with it ;-D

I was being half kidding, but only half. :-)


> Maybe I should make a slice on how to fix this behaviour also for
> SQLite. it's just a bit complicated because it requires to rebuild the
> sqlite module with an updated library.

Oh, that's unfortunate. I have to say, I having the SQLite as part of
Python (i.e. not rebuilding) is a huge win for getting new user's into
the early success that makes them (or at least me) happy to keep using
and expanding their Web2py usage. :-)

Thanks,
--Doug


Re: [web2py] Re: database locking web2py vs. "external" access...

2012-06-20 Thread Doug Philips
On Tue, Jun 19, 2012 at 1:33 PM, Anthony  wrote:
> I'm not sure it's feasible or desirable for the DAL to try to normalize
> every last behavior across all database backends. What exactly would you
> have the DAL do differently in this case?

Actually what I'm looking for is in the documentation, not the code,
so I don't know yet if there is anything I want the DAL to do
differently.


> web2py is just using the Python standard library sqlite3 module. According
> to the docs, it looks like it does not default to autocommit mode.

"It looks like" is my problem here! :-)

I'm looking for The Book to tell me what to expect.
If it doesn't tell me, then I don't know if the current behaviour
(once I discover it) will be carried forward to future versions of
Web2py as part of the backwards compatibility promise, or, if the
current behaviour is just a quirk of the current implementation that
might change and which could bite me if I  rely on it.

Can I count on the DAL setting or not setting autocommit mode? How do
I have any way of knowing if changing that "behind the DAL's back"
would break anything or not. Does the DAL even care?

Since the DAL is trying to abstract away details about particular DB
engines so that Web2py users can write code that "doesn't care", it
also needs to say (for each DB Engine) what assumptions it is
making... that is, what matters to it and what doesn't. It's perfectly
fine to say, for each DB Engine, what configurations/settings/etc.
that Web2py 'cares about' and which it doesn't.


> At some point, it might be worth doing some testing to see if you get the
> behavior you want. If you do db.commit() as in the example, I suspect it
> will be fine (assuming traffic levels aren't too high).

Indeed, I'll have to do that anyways.

Thanks,
--Doug


Re: [web2py] Re: database locking web2py vs. "external" access...

2012-06-20 Thread Doug Philips
On Tue, Jun 19, 2012 at 3:31 PM, Niphlod  wrote:
> Yep, I don't know exactly the defaults, but I happen to see some major
> differences against various systems with standard python distribution.

Ugh, at least, for the most part, it still all 'just works'. :-/


> Moreover, it's just for this kind of functionality for webapps that could be
> beneficial to update the module, so I can understand a little bit the
> "standardized" api and "different" actual sqlite linked version. For having
> the possibility to avoid this locking one must activate WAL
> (http://www.sqlite.org/draft/wal.html) on the db. Unfortunately WAL needs
> sqlite version > 3.7.0

Yeah, I'm not looking to go quite that far... yet. :-)

> Anyway, as Anthony suggested, writing something and apply "the theory" to
> that code can't replace a sane "test live functionality": every app has its
> kinds of loads, and percentages of writes/reads are absolutely not a
> standard.

Agreed. I just wanted more documentation about the
assumptions/requirements that DAL makes on the underlying database
engine interface.

Thanks
 --Doug


Re: [web2py] Re: database locking web2py vs. "external" access...

2012-06-20 Thread Doug Philips
On Wed, Jun 20, 2012 at 5:53 PM, Anthony  wrote:
> Can you explain how whether SQLite autocommits or not matters in this case?
> The DAL documentation recommends doing a db.commit() in scripts and
> background tasks like this precisely because it doesn't want to assume any
> autocommitting is happening. Within the context of HTTP requests, web2py
> automatically does a commit at the end of the request. It's not clear the
> documentation needs to say anything else here -- if you follow what the
> documentation currently says, you should be fine.

>From http://www.sqlite.org/lockingv3.html, "7.0 Transaction Control At
The SQL Level":
"The SQL command "BEGIN TRANSACTION" (the TRANSACTION keyword is
optional) is used to take SQLite out of autocommit mode. Note that the
BEGIN command does not acquire any locks on the database. After a
BEGIN command, a SHARED lock will be acquired when the first SELECT
statement is executed."

So, if the DAL takes the DB out of autocommit mode, the "select"
(which is done in the main/outer loop of the server script) will cause
a SHARED lock to be taken, right?
Then, let's say, there are no new records to process, so the server
script goes to sleep and does another select/query on its next trip
around the loop.
So far, just fine, the server script doesn't block, right?
However, if the Web2py application wants to update the DB, it can't,
because the server script now has a potentially long-lived SHARED
lock.
Quoting from the same document, "3.0 Locking":
"SHARED  The database may be read but not written. Any number of
processes can hold SHARED locks at the same time, hence there can be
many simultaneous readers. But no other thread or process is allowed
to write to the database file while one or more SHARED locks are
active."

Sooo Have I misunderstood the SQLite locking description? or
somewhere in the the DAL documentation where it says it'll do a commit
after a select or is it relying on (in the case of SQLIte) the
autocommit to handle that for it?

Thanks!
   --Doug


Re: [web2py] Re: database locking web2py vs. "external" access...

2012-06-22 Thread Doug Philips
On Wed, Jun 20, 2012 at 7:04 PM, Niphlod  wrote:
> You're forgetting another "layer".  DAL leverages on DBAPI implementation of
> sql drivers.

Well, that's not exactly good news for transparency and ease of documentation.

> A little more complicated: get on an infinite loop inserting 10 rows,
> without committing. while the loop is running on 1, try a loop of 100
> updates to those records, then commit. Occasionally you'll be presented with
> "Database is locked" error.

OK.

But what I would prefer to do is actually reason about what should be
happening rather than just slapping code together and trying to hope
that my tests will be close enough to real world scenarios to be
valid.

Sure, there is documentation about the quirks of SQLite, but nothing
(unless I've missed it after repeated reading) about the
assumptions/defaults, etc of how the DAL uses the intermediate glue
layers (pysqlite, etc.). At this point, I think I'll just avoid using
the DAL unless it is part of the web2py application, it's easier to
write the external server to talk directly to the relevant database.

Thanks for your help!
-=Doug

-- 





Re: [web2py] Re: database locking web2py vs. "external" access...

2012-06-22 Thread Doug Philips
On Wed, Jun 20, 2012 at 9:06 PM, Anthony  wrote:
>> Sooo Have I misunderstood the SQLite locking description?
>
>
> I believe so. The shared lock does not last forever -- only as long as it
> takes to execute the select. Once the data have been read, the shared lock
> (for that particular select) is released.

Yes, if autocommit is on. The part you didn't quote was that
autocommit can be disabled in which case a SHARED lock is held.
Since the documentation about what the DAL uses/needs/assumes about
the underlying Python layer (pysqlite in this case) isn't there, its
just easier to avoid using it outside of a Web2py application.

Thanks for all the help,
   -Doug

-- 





Re: [web2py] Re: database locking web2py vs. "external" access...

2012-06-27 Thread Doug Philips
On Sat, Jun 23, 2012 at 1:54 PM, Anthony  wrote:
>> > You're forgetting another "layer".  DAL leverages on DBAPI
>> > implementation of sql drivers.
>>
>> Well, that's not exactly good news for transparency and ease of
>> documentation.
>
> DB API is just a specification for Python database drivers to follow to
> ensure some degree of similarity and
> portability: http://www.python.org/dev/peps/pep-0249/. On net, this is
> probably good news for transparency and ease of documentation because you
> have some consistency across drivers. What alternative would you propose for
> greater transparency and ease of documentation?

Well, earlier it was claimed that the DAL for SQLite left autocommit
on, but the DBAPI says:
"Note that if the database supports an auto-commit feature, this must
be initially off."
So now I have an extra place to go look and try to figure out, does
the DAL actually implement DBAPI with autocommit or does the DAL
implement DBAPI but with an exception re: autocommit? Having more
places to look is not 'ease of documentation' from a user's
perspective. :-/


> Again, what specific assumptions about the "intermediate glue layers" would
> affect the code you would write in this case? If the issue is
> auto-committing, I believe the recommended code would work in either case.
> I'm not saying I would mind seeing a little more documentation regarding the
> differences across some of the databases and their drivers -- I'm just
> wondering how much it matters in this particular case.

It matters if the query at the top of the loop holds a lock on the DB,
which means that when there are no rows found to process, the database
remains locked while the server is sleeping...

-=Doug

-- 





Re: [web2py] Re: database locking web2py vs. "external" access...

2012-06-27 Thread Doug Philips
On Sat, Jun 23, 2012 at 1:24 PM, Anthony  wrote:
> No, when autocommit is disabled, that doesn't immediately result in a shared
> lock being acquired and held indefinitely. A shared lock is acquired when a
> select is made, but only held as long as it takes to actually do the reading
> for that select. Just do a commit after your inserts/updates/deletes, and
> you should be fine.

I don't think you are understanding, I'm not concerned about the case
when the query/select finds work to do, I'm concerned about the case
when the server looks, and finds nothing. I agree, when autocommit is
on, the lock will be dropped after the query returns. However, it
autocommit is off, and again referring to
http://www.sqlite.org/lockingv3.html section 7:
  "After a BEGIN command, a SHARED lock will be acquired when the
first SELECT statement is executed. ... The SQL command "COMMIT" does
not actually commit the changes to disk. It just turns autocommit back
on."

Again, with autocommit off:
The server process does a query/select to see if there is any work to
do. This provokes SQLite to acquire a SHARED lock. Then the server
process decides "oops, no work to do, let's go to sleep for a minute".
Notice that it does nothing to release that lock. So the Web2py
process is now locked out of doing any updates because the SHARED lock
will block the acquisition of locks needed to actually write/update
the database. I didn't find anything in the SQLite documentation that
would indicate otherwise, but if you do, please point me to the
specific document/section. (I just searched through The Book again and
found no mention of autocommit. Searching for just 'commit' finds only
a handful of places that make no mention of it either.)



>> Since the documentation about what the DAL uses/needs/assumes about
>> the underlying Python layer (pysqlite in this case) isn't there, its
>> just easier to avoid using it outside of a Web2py application.
>
>
> Seems rather extreme. What information do you need that would impact your
> usage? Note, the good thing about open source is that you can just look at
> the source code.

Technically True. And that has allowed me to do my own work arounds
(and submit patches), which has been extremely helpful.
So for fixing bugs, it's great!
However...
One of the things I really like about Web2py is the promise of
backwards compatibility.
Is that promise meant to encompass all the current decisions embodied
in the implementation?
I wouldn't assume so.
The Book does not say: "For anything not covered here, read the source
and base your expectation of backwards compatibility on that."
The Book does say: "web2py has always been built from the user
perspective and is constantly optimized internally to become faster
and leaner, whilst always maintaining backward compatibility." and
"The source code of the Database Abstraction Layer was completely
rewritten in 2010. While it stays backward compatible, the rewrite
made it more modular and easier to extend."
So I should fully expect the implementation to change; anything I find
by "reading the source" is clearly something that is, and has been,
subject to change. Sorry, but the only sane thing to do is to rely
only on what is documented and covered under the blanket promise of
backwards compatibility.

-=D

-- 





Re: [web2py] Re: database locking web2py vs. "external" access...

2012-06-27 Thread Doug Philips
On Wed, Jun 27, 2012 at 10:13 AM, Anthony  wrote:
>> Again, with autocommit off:
>> The server process does a query/select to see if there is any work to
>> do. This provokes SQLite to acquire a SHARED lock. Then the server
>> process decides "oops, no work to do, let's go to sleep for a minute".
>> Notice that it does nothing to release that lock.
>
>
> Yes, it does release the lock, as soon as the select is complete. You do not
> have to do db.commit() after a select.

Please show me where that is documented.
Section 7 of http://www.sqlite.org/lockingv3.html says that ONLY
happens when autocommit is on, and you have stated that it is off
(contrary to my mistaken memory). Further, The Book says that only
inside of a Web2py app does commit/rollback back happen automatically.
Thanks,
--Doug

-- 





Re: [web2py] Re: database locking web2py vs. "external" access...

2012-06-27 Thread Doug Philips
On Wed, Jun 27, 2012 at 11:53 AM, Anthony  wrote:
>> > Yes, it does release the lock, as soon as the select is complete. You do
>> > not
>> > have to do db.commit() after a select.
>> Please show me where that is documented.
>> Section 7 of http://www.sqlite.org/lockingv3.html says that ONLY
>> happens when autocommit is on, and you have stated that it is off
>> (contrary to my mistaken memory).
>
>
> End of Section 4:
>
> Once all reading has completed, the SHARED lock is dropped.

Thanks, I had missed that part.


>> Further, The Book says that only
>> inside of a Web2py app does commit/rollback back happen automatically.
>
>
> Commit is not needed for a select. Committing is for committing changes to
> the database -- selects only read from the database.

Ok, so if (and I'm looking for a more general that just SQLite answer
here), I want to get exclusive access to the DB, I should first do a
'dummy' change.

More explicitly, if the server process were first to 'update' a record
(solely for the purpose of locking the DB), then read through pending
entries, it could mark them as 'working on by server process x',
commit, and be sure that no other process (such as another
worker/server or even a user coming in through web2py proper) would be
able to also pick up that work? Even if that just works for SQLite,
I'm cool, but I'd like a solution that worked for all DBs if that is
possible without a lot of machinations.

In slightly more detail, there could be a server's table, and as each
server 'wakes up' looking for work to do, it could udpate its own row
in that table (hence provoking a lock), look for work, flag the work
as being handled by itself, then commit. It could then take its own
sweet time to do the work, knowing that no other server process would
have also claimed that work?

-=Doug


Re: [web2py] Re: database locking web2py vs. "external" access...

2012-06-28 Thread Doug Philips
On Thu, Jun 28, 2012 at 12:08 AM, Anthony  wrote:
> SQLite is a file-based database, so has to have more course locking than a
> typical client/server database. Other databases have "select for update"
> functionality, which allows you to select a set of records and lock them.
> web2py actually supports this, via .select(..., for_update=True).

Thanks, "for_update" is not documented in The Book, but is exactly
what I am looking for.

Checking the gluon code for Version 1.99.7 (2012-03-04 22:12:08) stable
shows that SQLite tries to support 'for_update', unfortunately it
doesn't work (causes an exception). So far as I could tell by
searching, that wasn't already reported so I created a new issue:
http://code.google.com/p/web2py/issues/detail?id=864 for it.

> Anyway, you might be better off looking into using the scheduler, or a
> third-party task queue like Celery (a web2py Celery plugin was started at
> some point, though not sure it is complete).

I don't understand, how will that let me do a read-modify-update
transaction? It seems as if for_update is precisely what I need to do
that.

Thanks,
-Doug


Re: [web2py] Re: database locking web2py vs. "external" access...

2012-06-28 Thread Doug Philips
On Thu, Jun 28, 2012 at 1:02 AM, Jonathan Lundell  wrote:
> The SQLite adapter simulates select for update by wrapping the select+update
> in a transaction. Not too efficient, since it has to lock the entire
> database, but it'll do it.

Thanks, I wish it worked in the released version, and I'm hopeful that
it can be made to work in the next version!
-=D


Re: [web2py] Re: database locking web2py vs. "external" access...

2012-06-29 Thread Doug Philips
On Thu, Jun 28, 2012 at 8:04 AM, Anthony  wrote:
>> > Anyway, you might be better off looking into using the scheduler, or a
>> > third-party task queue like Celery (a web2py Celery plugin was started
>> > at
>> > some point, though not sure it is complete).
>>
>> I don't understand, how will that let me do a read-modify-update
>> transaction? It seems as if for_update is precisely what I need to do
>> that.
>
> I thought your original goal was to set up a task queue -- those suggestions
> are specialized for that purpose.

Yes, it is. The email-queue example in The Book has a caveat that only
one server can be used. That's because with more than one server
process, the same work item might be processed several times. If I had
a read-modify-update transaction, then a server could mark a
particular work item as taken/in-progress and it wouldn't matter how
many servers I had. It would work with just one, or if I needed to
spin up a few more, I could that without introducing errors,
race-conditions, etc. Right?

-=Doug


[web2py] Getting auth.settings.register_next to work...

2012-06-29 Thread Doug Philips
I've been customizing how registration works for my app as a few of my
app's users are getting confused by the registration verification
email.
Ok, so I will just have my app send them to a nice hand-holding
explanatory page after they've submitted their registration
information.

According to The Book, "all" I have to do is set
auth.settings.register_next to the URL I want, and I'm good to go.
But I can't get that to work.

(all line numbers are for
https://github.com/web2py/web2py/blob/master/gluon/tools.py as of the
time of this message)

First off, it seems that the Register link I get from auth.navbar()
always has a _next parameter appended to it,
and that _next seems to override anything I set in my model for
auth.settings.registration_next.
Fine, I go into the navbar function of the Auth class in tools.py and
delete the "+next" part of the registration link. (line 1242)
That doesn't work, as for some reason I don't understand, the ?_next
parameter in the URL is still present when the navbar is constructed
from a regular page, but the ?_next parameter is (properly) missing
from Register link when I first click on Login. Very odd to me.

After trying a few more fixes, I decide that I do not understand how
next processing is done for Auth.
My fix is a huge hammer.
I go into the register function and change lines 1983 and 1984.
Actually I delete them and replace them with:
next = self.settings.register_next

I have no idea what the lines I deleted are trying to do, but they
were keeping my expressed intent from working.
Now my users are directed to the page I want after submitting their
registrations, but I have no idea what else I have subtly screwed up.
I don't even know if I should submit a bug report or if there is some
other configuration I could do that would avoid having to make this
code change.
Help!

-Doug


[web2py] Re: Getting auth.settings.register_next to work...

2012-06-29 Thread Doug Philips
On Sat, Jun 30, 2012 at 2:37 AM, I wrote:
> and that _next seems to override anything I set in my model for
> auth.settings.registration_next.

Typo, I should have written:
and that _next parameter seems to override anything I set in my model
for auth.settings.register_next

-=D


Re: [web2py] Re: Getting auth.settings.register_next to work...

2012-06-30 Thread Doug Philips
Looking at tools/gluon.py again now that I have something working and
I am not in a rush, I notice something curious, lines 1226 and 1227:

li_next = '?_next='+urllib.quote(self.settings.login_next)
lo_next = '?_next='+urllib.quote(self.settings.logout_next)

So it looks as if maybe a 'more correct'? :-) solution might be to do
something similar with register_next being used to create the "_next"
parameter?

Nonetheless,  how Auth implements where to go next just seems
excessively tangled. There are next parameters to the methods, next
parameters in the URLs, and finally, auth.settings._next
fields which are of the last resort, and it is hard to see under what
circumstances the auth.settings._next fields will ever get
used (except for login_next and logout_next, as per the code quoted
above). It would be nice to understand what the use case(s) are for
all these next 'control points'...

At this point I still don't think I understand what's going on well
enough to submit a coherent bug report (I like to submit proposed
fixes too, but I'm not even close to thinking I can do that here).

Thanks for the help,
 -Doug



On Sat, Jun 30, 2012 at 10:13 AM, Anthony  wrote:
> Note, the first two options below will disable the _next functionality for
> all Auth actions. The last solution disables it only for the register
> action.
>
> Anthony
>
>
> On Saturday, June 30, 2012 10:12:50 AM UTC-4, Anthony wrote:
>>
>> Looks like auth.navbar() automatically adds the current URL as the _next
>> parameter so whenever someone clicks on an auth.navbar() link, after
>> finishing that action, they are returned to their original page. Also,
>> auth.settings.register_next (and similar) are only used as backup defaults
>> when there is no _next parameter in the URL (i.e., _next takes precedence).
>> Perhaps we should make it easier to override these two behaviors. For now,
>> though, I think you have a few options.
>>
>> First, right after you define auth, do:
>>
>> auth = Auth(...)
>> auth.next = None
>>
>> That should remove the _next value, which will allow the
>> auth.settings.register_next setting to take effect. Alternatively, right
>> before you define auth, you can remove _next from request.vars:
>>
>> request.vars._next = None
>> auth = Auth(...)
>>
>> You can also pass a "next" argument directly to the auth.register()
>> function (and other Auth functions):
>>
>> def user():
>> if request.args(0) == 'register':
>> form = auth.register(next=auth.settings.register_next)
>> else:
>> form = auth()
>> return dict(form=form)
>>
>> Anthony
>>
>> On Saturday, June 30, 2012 2:37:36 AM UTC-4, Doug Philips wrote:
>>>
>>> I've been customizing how registration works for my app as a few of my
>>> app's users are getting confused by the registration verification
>>> email.
>>> Ok, so I will just have my app send them to a nice hand-holding
>>> explanatory page after they've submitted their registration
>>> information.
>>>
>>> According to The Book, "all" I have to do is set
>>> auth.settings.register_next to the URL I want, and I'm good to go.
>>> But I can't get that to work.
>>>
>>> (all line numbers are for
>>> https://github.com/web2py/web2py/blob/master/gluon/tools.py as of the
>>> time of this message)
>>>
>>> First off, it seems that the Register link I get from auth.navbar()
>>> always has a _next parameter appended to it,
>>> and that _next seems to override anything I set in my model for
>>> auth.settings.registration_next.
>>> Fine, I go into the navbar function of the Auth class in tools.py and
>>> delete the "+next" part of the registration link. (line 1242)
>>> That doesn't work, as for some reason I don't understand, the ?_next
>>> parameter in the URL is still present when the navbar is constructed
>>> from a regular page, but the ?_next parameter is (properly) missing
>>> from Register link when I first click on Login. Very odd to me.
>>>
>>> After trying a few more fixes, I decide that I do not understand how
>>> next processing is done for Auth.
>>> My fix is a huge hammer.
>>> I go into the register function and change lines 1983 and 1984.
>>> Actually I delete them and replace them with:
>>> next = self.settings.register_next
>>>
>>> I have no idea what the lines I deleted are trying to do, but they
>>> were keeping my expressed intent from working.
>>> Now my users are directed to the page I want after submitting their
>>> registrations, but I have no idea what else I have subtly screwed up.
>>> I don't even know if I should submit a bug report or if there is some
>>> other configuration I could do that would avoid having to make this
>>> code change.
>>> Help!
>>>
>>> -Doug


Re: [web2py] Re: Getting auth.settings.register_next to work...

2012-07-01 Thread Doug Philips
On Sat, Jun 30, 2012 at 11:11 AM, Anthony  wrote:
> I don't think it's a bug, per se, but the behavior (and documentation) could
> be improved.

??? The documentation is quite direct and clear that you can set
auth.settings.register_next to control where the user goes next. It
doesn't work. It's not even clear that it can work given the current
code base. I've just created a bug report for it
http://code.google.com/p/web2py/issues/detail?id=869.

login_next and register_next are mentioned exactly once in the
documentation and there is no indication why login_next should work,
but register_next not.

Thanks,
 -Doug


[web2py] using deck.js with web2py?

2012-07-01 Thread Doug Philips
>From my 'search engine' results, it looks as if the last this was
discussed was back in Sept of last year.
Including a very intriguing comment from Massimo on Sept 9, 2011:
> This is really nice. I can see this integrated with markmin in a
> simple web app that we can all use... I need such a thing so I am
> going to build one soon.

Did anything ever happen with that?
I'd like to use deck.js to display some of the content my app is
gathering, and am wondering what issues might crop up.
Thanks!
 --Doug


Re: [web2py] Re: using deck.js with web2py?

2012-07-02 Thread Doug Philips
On Sun, Jul 1, 2012 at 11:14 PM, Massimo Di Pierro
 wrote:
> In the end I preferred reveal.js to deck.js
>
> This integrates reveal.js with markmin.
> https://github.com/mdipierro/markmin-reveal-slides
...
> Hope this is useful. It would be nice to make a web app for it where the
> source is stored in DB and can be edited online.

What I think you are saying is that you've integrated with markin but
that it is stand-alone, not part of a web2py app?

After sleeping on it, what I think I can do is have some views use
another set of js libraries (either reveal or deck or whatever)
instead of extending layout.html. The downside is that I'll "lose" the
website theming that layout.html gives me, but in this case I think
that is OK.

Thanks,
 --Doug


Re: [web2py] Re: Getting auth.settings.register_next to work...

2012-07-02 Thread Doug Philips
On Mon, Jul 2, 2012 at 1:03 AM, Anthony  wrote:
> Here's what the book says:
> These must point to the URL you want to redirect your users to after the
> various possible auth actions (in case there is no referrer):
>
> Notice the highlighted text. The _next URLs are default redirect URLs when
> there is no referring page, and auth.settings.register_next does in fact
> work when there is no referring page (i.e., when there is no _next var in
> the URL).

I searched the book (free PDF) for "referrer".
The term "referrer" only occurs once, where you quoted above. No
explanation given, no mention of a "_next" var in the URL.
Then I searched for "referer" thinking, OK, maybe the explanation I'm
looking for just has a typo, but found only an example of
request.env.http_referer... Maybe if I search for "_next" I'll find
some explanation that doesn't use 'referrer' but a similar term... the
only other use of "_next" is on page 346 (free PDF) where _next fields
are described as "the URL to redirect to after a successful
 record." THAT'S WHAT I WANT, YAY! Woo! Hoo! No? Oh well.
It would be really nice if things with similar names had similar
meanings...


> The issue is not with auth.settings.register_next, but with the
> auth.navbar() helper. The navbar helper always adds the current page URL as
> the _next var in the register link, so if you use the navbar to get to the
> registration page, you will be redirected back to the page from which you
> clicked the link.

Unless of course you set auth.settings.login_next, in which case, no,
it doesn't; the automatically generated auth links use
auth.settings.login_next and not the page from which I clicked the
link.


> But I still don't see a bug with auth.settings.register_next.

I don't really care if the cause of the bug is the navbar function or
something else. Since auth.settings.register_next is a passive
variable, of course that can't BE the bug.

I'm trying to report what, as a mere user and not implementor, is an
anomalous behavior.
Setting auth.settings.login_next "does the right thing" (the
automatically generated navigation bar), setting
auth.settings.register_next does not have that same behavior. Maybe
that one is "has a bug." Of course since that one does what I
want/expect (see above), then from my point of view as a user of
web2py and not an implementor, it is auth.setting.register_next which
has the bug.


> The intended behavior is that a referrer takes
> precedence, and auth.settings.register_next is the default when there is no
> referrer -- and that's how it works. We now just need an easy way to prevent
> the navbar from automatically adding a referrer.

And to make it treat all the auth.settings' _next fields uniformly (or
document why it shouldn't).
Oh, and to document what "referrer" means in this context. I'll go add
a bug report for that when I've sent this.

Thanks,
-Doug


Re: [web2py] Re: Getting auth.settings.register_next to work...

2012-07-02 Thread Doug Philips
On Mon, Jul 2, 2012 at 11:03 AM, Anthony  wrote:
> Seems to me this has precisely the same meaning (though in a different
> context). In the context of Auth, register_next is the URL after you
> register, and in the context of Crud, create_next is the URL after
> submission of a create form. What's the problem?

By "precisely the same meaning", do you mean that there is a _next URL
parameter that can override the Crud create_next (and others) field
settings and it is a bug that The Book doesn't say so?

...

> Moreover, your suggested fix (to have
> auth.settings.register_next take precedence within the register() function)
> would actually break the intended behavior. It's important to identify the
> real problem before attempting a fix.

Since you wrote: "it does appear that the navbar does not do the same
thing with the login link, and I'm not quite sure why" I don't have
much confidence in your claim to knowing what the intended behavior
is.  As I wrote in my first message:
"...I have no idea what else I have subtly screwed up. I don't even
know if I should submit a bug report or if there is some other
configuration I could do that would avoid having to make this code
change."

Having spent hours reading through the book and then the code to see
what mistake I had made configuring auth.settings and/or my app in
general, I finally had to give up and just use my big hammer solution
so that I could move on to doing more substantial work on my app. No
one has explained why login_next is different nor how it fits into
some notion of what the current design/intent is. I deeply regret that
I did anything other than file a bug report, in total:
"auth.setting.login_next and auth.settings.register_next are
documented to behave the same, but do not." I esp. regret all the
investigative work and time I spent on my first message of this
thread.

-=Doug