@Tim +1 On Jul 8, 4:39 pm, Tim <[email protected]> wrote: > Hi Greg, > > On Thursday, July 7, 2011 8:14:57 PM UTC+1, Greg D wrote: > > > I'm happy to give clarification, but I need a little more information on > > the sort of clarification you want. When you say what GAE is aimed at, do > > you mean language support? Uptime? Scale? Business model? Customer type? > > Developer type? > > I put some more details of my particular scenario in the longer parent post > that may fill in some details. > > What I'm looking for is the idea of the type of developer / application > architecture that you see GAE aimed at. > > ** TL;DR alert ** > I was going to send the below to Greg off-list, but figured I'd do it on > list to see if other agree or not. Apologies for anyone not interested :) > > To be quite explicit, early on, it looked like GAE was aimed very much at > what I'd call a purely reactive web service - plenty of capacity for serving > static content and some mutable state with a good solid user authentication > model built in, but the focus very much on the fact that processing only > occurs when a request comes in (no background tasks, no execution beyond the > response, cap on maximum execution model, no push to clients mechanism), but > with a richer API in that it can also accept requests via email and > scheduled tasks and similar as long as these handlers are short and sweet > too. > > This was quite a clean model and aligned with what I planned to use it for - > a cloud based application for users to maintain and manipulate a body of > data in a browser but with most of the complexity in the client, and GAE > providing centralisation of storage and some processing and perhaps some > interaction between users, but no real autonomous processing on the server, > and the request handling code is quick to start and finish as it maintains > no in-memory state in the process. And this felt like an outgrowth of an > attempt to monetise of spare capacity in google's existing infrastructure - > a good deal for both sides. > > As new features were introduced (Channel API, increased runtime limits) then > this seemed to reinforce this impression - the server can run a bit longer, > and can push data out to clients to reduce dumb polling queries, but it > seems still that a system will only startup when needed, should do all it > needs, and close down as soon as possible. And then we let Google identify > bursts when it might actually keep your processes in memory (the 'if > __name__ == "__main__":' trick), but that's part of playing nicely, not a > change in the type of application you write. > > So although no one needs to write yet another wiki/blog engine, it seemed > that style of usage was what was being envisioned, and as long as you play > nicely and don't store too much data then it's the platform for you, with > little things like memcache thrown in so you make your app run a bit quicker > when it's hit with a burst of traffic, and you can schedule tasks to do > things like age your data, clean up stale items, send email admin reports of > who's been doing what etc. > > So it's somewhat ideal for services that are available 24/7 but may be idle > for long periods, get a few queries at other times, and then might get some > bursty periods when they get hit by a load of traffic: > "You could get a dedicated server or two, but in a typical day it'll > spend 10 hours doing absolutely nothing, 7 hours ticking over quietly with a > few requests, 6 hours pretty consistently busy, and one hour desperately > overloaded and unresponsive. Instead, let us (Google) host it for you, and > in return for writing it using our datastore etc then we'll look after > making the above all work by spreading your app over 0, 1, 2 or 10 machines > for the different times and you don't have to worry about it, lots of other > people are doing the same but we think they'll have different bursts, and we > can spread the load over our server farms - it'll be cheaper than buying too > much capacity that sits idle and more robust than skimping on capacity and > then getting slashdotted etc. And if, over time, you need it to be 10, 20 or > 200 machines depending on the time of day, you won't have to rewrite it, > we've got the capacity if you pay for it, and the infrastructure is well up > to it." > > And let's be clear, I have no major gripe with the datastore (I've hated SQL > for a long long time) but I wouldn't choose to be shackled by such a > restrictive data model unless I thought it was being done to make it easy > for you guys to spread the data persistence around so you can can scale the > data and the agents around without big startup times when a new host is > suddenly invoked. > > So far, so good: it's an engine for applications, it starts when the > applications need it, and turns off when the applications stop making > demands, and in between it automatically changes from being an electric > hybrid with a frugal 4 cylinder engine, to a high-revving turbo charged F1 > engine, and to a shedload of industrial diesel workhorses depending on the > load, and bills you (to push an analogy over the edge) based on the total > power delivered to the drivetrain. Google do the engine magic using same > mechanisms they've developed for managing the spare capacity in the massive > farms that they use, and we all go home happy. > > Then over time we quite quickly got task queues and backends and processes > can run for much longer, and there's talk about using MapReduce and other > background heavy lifting, and maybe SQL, and a new pricing model that seems > to invert much of the above emphasis - it seems that App Engine is being > re-positioned for people who want persistent server processes which is > running more or less steadily, with clients connecting to push fresh data or > request reports of what's been happening. > > This is not so much an engine for external applications, but the server > process becomes the application itself, and GAE is an engine for this large > application. Depending on what I want my application to do, I can configure > more or less resources of different types, and I still have the scalability > of what's actually needed, but the focus is now very much on the server > processes being self-sustaining, and with 15 minute startup costs etc it > feels more like a way to build always on (or mostly on) data processing > grids. > > The argument is now "Why write all your own infrastructure for that 10 > machine grid and then have to re-do it when you scale up to 100 machines, > this is our proven model that we'll license to you, just say how many > machines you want to use, and we'll host it all for you and throw in a load > of free toys that we predict you'll find useful". > > And the billing has moved from "power delivered" to "fuel consumed" - a > subtle but important change. > > Now I think both models are valid, and I can understand the appeal in the > latter target, but it does seem that the former application architecture has > been not just de-emphasised, but is actually being discouraged. GAE now > becomes a better way to buy the dedicated servers that you think you'll need > for whatever business logic you're implementing, but putting a reporting web > front end on this is now nice and simple and distinct from the dedicated > servers. > > The first model gets used by small web startups, web-apps, "appstore" apps > etc, some of which go bust, others grow massive, a load just pootle along. > The second model gets used by enterprises and SaaS providers who don't want > to build and admin and manage their own server farms, and appreciate being > able to use things like MapReduce for data crunching rather than writing > their own bad port of it. > > And yes, that's all overly simplistic (some web startups are based on > massive data processing, enterprises have a lot of simplistic data servers > for rich front ends) but it illustrates what I see as the change in > positioning for GAE over the last 18 months. > > Now it's easy to come out and say something like "We want GAE to be the > finest managed server platform for anybody, with all these rich features and > easy admin and great scaling" and it's always a bit hard to say "We don't > want these sorts of people", but the fact is that Domino's and that trendy > Italian brasserie that just opened up are NOT aiming at the same market > other than "people who want to eat food". > > If you want to aim at both markets at once (and why not), then I'd hope to > get a more definite statement about how you're aiming to deliver value and > features for each market individually, and one way to do that explicitly > might be to provide 2 or more complete pricing models - not just "small, > medium and large", but a properly segmented pricing model that can be made > to fit the demands and use of each market. > > Thanks for reading :) > > -- > Tim // @schmerg
-- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
