Hi Greg, On Thursday, July 7, 2011 8:14:57 PM UTC+1, Greg D wrote: > > I'm happy to give clarification, but I need a little more information on > the sort of clarification you want. When you say what GAE is aimed at, do > you mean language support? Uptime? Scale? Business model? Customer type? > Developer type? > > I put some more details of my particular scenario in the longer parent post that may fill in some details.
What I'm looking for is the idea of the type of developer / application architecture that you see GAE aimed at. ** TL;DR alert ** I was going to send the below to Greg off-list, but figured I'd do it on list to see if other agree or not. Apologies for anyone not interested :) To be quite explicit, early on, it looked like GAE was aimed very much at what I'd call a purely reactive web service - plenty of capacity for serving static content and some mutable state with a good solid user authentication model built in, but the focus very much on the fact that processing only occurs when a request comes in (no background tasks, no execution beyond the response, cap on maximum execution model, no push to clients mechanism), but with a richer API in that it can also accept requests via email and scheduled tasks and similar as long as these handlers are short and sweet too. This was quite a clean model and aligned with what I planned to use it for - a cloud based application for users to maintain and manipulate a body of data in a browser but with most of the complexity in the client, and GAE providing centralisation of storage and some processing and perhaps some interaction between users, but no real autonomous processing on the server, and the request handling code is quick to start and finish as it maintains no in-memory state in the process. And this felt like an outgrowth of an attempt to monetise of spare capacity in google's existing infrastructure - a good deal for both sides. As new features were introduced (Channel API, increased runtime limits) then this seemed to reinforce this impression - the server can run a bit longer, and can push data out to clients to reduce dumb polling queries, but it seems still that a system will only startup when needed, should do all it needs, and close down as soon as possible. And then we let Google identify bursts when it might actually keep your processes in memory (the 'if __name__ == "__main__":' trick), but that's part of playing nicely, not a change in the type of application you write. So although no one needs to write yet another wiki/blog engine, it seemed that style of usage was what was being envisioned, and as long as you play nicely and don't store too much data then it's the platform for you, with little things like memcache thrown in so you make your app run a bit quicker when it's hit with a burst of traffic, and you can schedule tasks to do things like age your data, clean up stale items, send email admin reports of who's been doing what etc. So it's somewhat ideal for services that are available 24/7 but may be idle for long periods, get a few queries at other times, and then might get some bursty periods when they get hit by a load of traffic: "You could get a dedicated server or two, but in a typical day it'll spend 10 hours doing absolutely nothing, 7 hours ticking over quietly with a few requests, 6 hours pretty consistently busy, and one hour desperately overloaded and unresponsive. Instead, let us (Google) host it for you, and in return for writing it using our datastore etc then we'll look after making the above all work by spreading your app over 0, 1, 2 or 10 machines for the different times and you don't have to worry about it, lots of other people are doing the same but we think they'll have different bursts, and we can spread the load over our server farms - it'll be cheaper than buying too much capacity that sits idle and more robust than skimping on capacity and then getting slashdotted etc. And if, over time, you need it to be 10, 20 or 200 machines depending on the time of day, you won't have to rewrite it, we've got the capacity if you pay for it, and the infrastructure is well up to it." And let's be clear, I have no major gripe with the datastore (I've hated SQL for a long long time) but I wouldn't choose to be shackled by such a restrictive data model unless I thought it was being done to make it easy for you guys to spread the data persistence around so you can can scale the data and the agents around without big startup times when a new host is suddenly invoked. So far, so good: it's an engine for applications, it starts when the applications need it, and turns off when the applications stop making demands, and in between it automatically changes from being an electric hybrid with a frugal 4 cylinder engine, to a high-revving turbo charged F1 engine, and to a shedload of industrial diesel workhorses depending on the load, and bills you (to push an analogy over the edge) based on the total power delivered to the drivetrain. Google do the engine magic using same mechanisms they've developed for managing the spare capacity in the massive farms that they use, and we all go home happy. Then over time we quite quickly got task queues and backends and processes can run for much longer, and there's talk about using MapReduce and other background heavy lifting, and maybe SQL, and a new pricing model that seems to invert much of the above emphasis - it seems that App Engine is being re-positioned for people who want persistent server processes which is running more or less steadily, with clients connecting to push fresh data or request reports of what's been happening. This is not so much an engine for external applications, but the server process becomes the application itself, and GAE is an engine for this large application. Depending on what I want my application to do, I can configure more or less resources of different types, and I still have the scalability of what's actually needed, but the focus is now very much on the server processes being self-sustaining, and with 15 minute startup costs etc it feels more like a way to build always on (or mostly on) data processing grids. The argument is now "Why write all your own infrastructure for that 10 machine grid and then have to re-do it when you scale up to 100 machines, this is our proven model that we'll license to you, just say how many machines you want to use, and we'll host it all for you and throw in a load of free toys that we predict you'll find useful". And the billing has moved from "power delivered" to "fuel consumed" - a subtle but important change. Now I think both models are valid, and I can understand the appeal in the latter target, but it does seem that the former application architecture has been not just de-emphasised, but is actually being discouraged. GAE now becomes a better way to buy the dedicated servers that you think you'll need for whatever business logic you're implementing, but putting a reporting web front end on this is now nice and simple and distinct from the dedicated servers. The first model gets used by small web startups, web-apps, "appstore" apps etc, some of which go bust, others grow massive, a load just pootle along. The second model gets used by enterprises and SaaS providers who don't want to build and admin and manage their own server farms, and appreciate being able to use things like MapReduce for data crunching rather than writing their own bad port of it. And yes, that's all overly simplistic (some web startups are based on massive data processing, enterprises have a lot of simplistic data servers for rich front ends) but it illustrates what I see as the change in positioning for GAE over the last 18 months. Now it's easy to come out and say something like "We want GAE to be the finest managed server platform for anybody, with all these rich features and easy admin and great scaling" and it's always a bit hard to say "We don't want these sorts of people", but the fact is that Domino's and that trendy Italian brasserie that just opened up are NOT aiming at the same market other than "people who want to eat food". If you want to aim at both markets at once (and why not), then I'd hope to get a more definite statement about how you're aiming to deliver value and features for each market individually, and one way to do that explicitly might be to provide 2 or more complete pricing models - not just "small, medium and large", but a properly segmented pricing model that can be made to fit the demands and use of each market. Thanks for reading :) -- Tim // @schmerg -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/a8IFJxhxZdwJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
