Re: [google-appengine] full text search and/or external search services

2011-08-02 Thread Robert Kluin
Have you tried running whoosh (not whoosh-appengine) on a backend? Backends are expensive, but might help you out a bit. Robert On Thu, Jul 28, 2011 at 11:19, andreas schmid wrote: > hi guys, > > i really need to have a full text search on my app and i wanted to know what > kind of extern

Re: [google-appengine] Unserializing large Blobs from the Blobstore

2011-08-02 Thread Robert Kluin
On Mon, Jul 11, 2011 at 05:55, Luca Matteis wrote: > Hello, > > I store large Blobs in the datastore. These are XML files that can be > as large as 20mb. Storing a single 20mb XML file is fine, however the > issue comes when I need to unserialize it. > > There are all sorts of limitations to this:

Re: [google-appengine] Re: Federated Login redirecting to Google OpenID and not /_ah/login_required on Java GAE

2011-08-02 Thread Robert Kluin
Hi Vishal, Did you go through Wesley's example doc: http://code.google.com/appengine/articles/openid.html You might try asking on the app engine java group. Robert On Tue, Aug 2, 2011 at 00:49, Vishal wrote: > Anyone? I still can't get Federated Login to work correctly. > > Vishal

Re: [google-appengine] Problem fetching db.ListProperty data

2011-08-02 Thread Robert Kluin
On Mon, Aug 1, 2011 at 02:43, Simon wrote: > Hi, > > I'm new to appengine and I'm trying to write an app in python, however I > don't think that my problem is python specific. > If somebody else already asked the same question, I'm sorry for repeating > it. I searched for quite a while and couldn'

Re: [google-appengine] interpreting Logs

2011-08-02 Thread Robert Kluin
You could give analytics a try: http://www.google.com/analytics/ I suppose you could also log some additional info in each request (if needed) and periodically grab the logs if you wanted to. http://code.google.com/appengine/docs/python/tools/uploadinganapp.html#Downloading_Logs You could pr

Re: [google-appengine] most efficient bulk delete code?

2011-08-02 Thread Robert Kluin
Just a note, I've heard the built in datastore admin delete code is running much faster recently. You might give it a shot. I have not tried it myself to verify it is running better though (previously it was quite poor). Robert On Tue, Aug 2, 2011 at 14:54, Stephen Johnson wrote: > I use

Re: [google-appengine] Increase apps size restriction above 10 megabytes?

2011-08-02 Thread Robert Kluin
Yes. http://code.google.com/p/googleappengine/issues/detail?id=5472 On Tue, Aug 2, 2011 at 10:52, Johan Euphrosine wrote: > Did someone fill a doc bug for this ? > > Thanks in advance. > > On Sun, Jul 31, 2011 at 7:30 PM, Robert Kluin wrote: >> I don't think that is correct, at least not a

Re: [google-appengine] Website Spam posing by appid: aking-741 using IP 64.233.172.1

2011-08-02 Thread Robert Kluin
On Tue, Aug 2, 2011 at 14:54, Murray W wrote: > I did find the abuse link. > It's nice but not practical for webmasters and developers to report network > abuse manually. What do you want to do, monitor a site and if some app is hitting it too frequently automatically report it to Google for abus

Re: [google-appengine] Re: Crazy number of Deadine Exceeded exceptions recently

2011-08-02 Thread Robert Kluin
Interesting. I've been seen exactly the same strange behavior across several apps as well. Suddenly instances will get killed and restarted in large batches. This happens even with low request latency, small memory usage (similar to yours < 50mb), low error rates, and steady traffic. I pretty c

Re: [google-appengine] Data Center (CDN) Locations

2011-08-02 Thread Robert Kluin
I wonder how many people will think what you said here is true. On Tue, Aug 2, 2011 at 15:11, Brandon Wirtz wrote: > After doing some research it turns out that the reason GAE is so fast is > that it is really a bot net running on every machine with Chrome installed. > The speed comes from the

[google-appengine] Re: Much more efficient implementation of db.ListProperty(int) - created a lib

2011-08-02 Thread Greg
See also http://bit.ly/6emftF I've been using ArrayProperty for a couple of years. Cheers Greg. On Aug 3, 4:18 am, Andrin von Rechenberg wrote: > So you want to store a "very long list of numbers" in your AppEngine Model > which doesn't need to be indexed. > > Well, you could just do this: > >

[google-appengine] Re: High CPU usage and how to improve performance

2011-08-02 Thread observer247
Issue number is 5415 It has been more than a week and I am yet to get a useful response on the ticket. On Jul 21, 4:20 am, Johan Euphrosine wrote: > > > Feel free to open a Production Issue for problem like this affecting > directly your application > operation:http://code.google.com/p/googleap

Re: [google-appengine] Re: JDO schema change nightmare, app going live

2011-08-02 Thread Pascal Voitot Dev
You can also give a try to Siena http://www.sienaproject.com It provides the same level of features as Objectify just above Datastore. But it's not specialized to GAE and aims at providing a bridge between NoSQL DB (and also SQL DB when it's possible). Currently we support GAE (sync/async) + MySQL/

Re: [google-appengine] Re: Stackoverflow

2011-08-02 Thread Stephen Johnson
It is done. On Tue, Aug 2, 2011 at 2:29 PM, Ikai Lan (Google) wrote: > So I saw this in my Google Alerts earlier, and I thought, what the heck, > people ask development questions on Yahoo Answers? > > http://answers.yahoo.com/question/index?qid=20110726102419AAFUzgy > > Can someone with a Yahoo

[google-appengine] Re: JDO schema change nightmare, app going live

2011-08-02 Thread nischalshetty
I would recommend you to scrap JDO completely and use objectify. We were in a similar situation as you, changing over to objectify was the best thing ever! And it's easier than you could ever imagine. -- You received this message because you are subscribed to the Google Groups "Google App Engi

Re: [google-appengine] Re: Stackoverflow

2011-08-02 Thread Ikai Lan (Google)
Welcome to Knowing Me In Real Life. =) -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Tue, Aug 2, 2011 at 2:43 PM, Robert Kluin wrote: > Had never saw funnyyahooquestions.com before. There goes another hour. > > As a side note, if this was a

Re: [google-appengine] Re: Stackoverflow

2011-08-02 Thread Robert Kluin
Had never saw funnyyahooquestions.com before. There goes another hour. As a side note, if this was a SO question we'd have to downvote your answer for being off topic. :P On Tue, Aug 2, 2011 at 17:29, Ikai Lan (Google) wrote: > So I saw this in my Google Alerts earlier, and I thought, wha

Re: [google-appengine] Re: Stackoverflow

2011-08-02 Thread Ikai Lan (Google)
So I saw this in my Google Alerts earlier, and I thought, what the heck, people ask development questions on Yahoo Answers? http://answers.yahoo.com/question/index?qid=20110726102419AAFUzgy Can someone with a Yahoo account point him here? I realized that I haven't logged into Yahoo in years and c

Re: [google-appengine] Re: Data Center (CDN) Locations

2011-08-02 Thread Ikai Lan (Google)
Our external roadmap is here: http://code.google.com/appengine/docs/roadmap.html We don't currently have regional data centers on our roadmap, but it's something we're always thinking about. -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Tue,

[google-appengine] interpreting Logs

2011-08-02 Thread Bruno Sandivilli
Hi, is there anynway to analyse the user actions using the log of requests? I'm thinking in log every action a user does , in my application, like "the user have clicked in homepage". In an apache server for example, i can interprete the log and see the sessionid and the get request. -- You recei

Re: [google-appengine] Re: Data Center (CDN) Locations

2011-08-02 Thread Ian Wilson
Ikai, Thank you for the information. I will keep in mind the optimization tips. Do you guys have local data centers as part of your road map? (I know Amazon recently set up a dedicated center for AWS in Tokyo). Ian -- You received this message because you are subscribed to the Google Groups "

Re: [google-appengine] "Enforced Rate" in task queues ruining performance

2011-08-02 Thread Dan Lines
Johan, I have submitted a production issue for this problem. Here is the link http://code.google.com/p/googleappengine/issues/detail?id=5471 If you have any questions or need to contact me feel free to email me directly. Thank you, - Dan -- You received this message because you are subscr

[google-appengine] Re: Crazy number of Deadine Exceeded exceptions recently

2011-08-02 Thread Eduardo Garcia
I'm afraid this could be related to this issue: https://groups.google.com/d/topic/google-appengine-java/u4R40A7a2X0/discussion We are using M/S too. HTH, Eduardo G. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion

RE: [google-appengine] Data Center (CDN) Locations

2011-08-02 Thread Brandon Wirtz
After doing some research it turns out that the reason GAE is so fast is that it is really a bot net running on every machine with Chrome installed. The speed comes from the fact that in many cases when you hit your own app it is very fast because it is running locally, but since very few servers a

Re: [google-appengine] Much more efficient implementation of db.ListProperty(int) - created a lib

2011-08-02 Thread Ikai Lan (Google)
Thanks Andrin! If anyone is wondering why this is, it's because the List property is serialized and deserialized using protocol buffers. Protocol buffers are very useful for things like validation and cross language compatibility, but you can sometimes gain some performance benefits by serializing

Re: [google-appengine] Connect from GAE to external PostgreSQL DB?

2011-08-02 Thread Ikai Lan (Google)
Yeah, you'll need a REST proxy. I wonder if someone has built a version of DBSlayer for Postgres? http://code.nytimes.com/projects/dbslayer DBSlayer's biggest fault: no authentication. -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Tue, Aug 2

Re: [google-appengine] Website Spam posing by appid: aking-741 using IP 64.233.172.1

2011-08-02 Thread Murray W
I did find the abuse link. It's nice but not practical for webmasters and developers to report network abuse manually. Q1: Is there a way to lookup the applications author / developer using the appid: information from the useragent? For every problem I believe a solution is just around the

Re: [google-appengine] Re: Data Center (CDN) Locations

2011-08-02 Thread Ikai Lan (Google)
The primary data centers serving App Engine are in North America, however, assets and items that are edge cached are served from Google edge caches which are located wherever there are Google data centers. When evaluating page speed, it's typically assets that cause page speeds to appear slow: ht

Re: [google-appengine] most efficient bulk delete code?

2011-08-02 Thread Stephen Johnson
I use the low-level datastore to do the delete by keys. On Tue, Aug 2, 2011 at 10:52 AM, Carter Maslan wrote: > thanks; what delete do you use? > It looked like deletePersistentAll() accepted only entities (rather than > key values) > > > On Tue, Aug 2, 2011 at 9:46 AM, Stephen Johnson wrote: >

Re: [google-appengine] most efficient bulk delete code?

2011-08-02 Thread Carter Maslan
thanks; what delete do you use? It looked like deletePersistentAll() accepted only entities (rather than key values) On Tue, Aug 2, 2011 at 9:46 AM, Stephen Johnson wrote: > Change your query to something like this: > > Query query = pm.newQuery("select ek from " + UploadedContent.class > .getNa

Re: [google-appengine] most efficient bulk delete code?

2011-08-02 Thread Stephen Johnson
Change your query to something like this: Query query = pm.newQuery("select ek from " + UploadedContent.class .getName()); Where "ek" is the entity key name you use in you JDO class. For example, mine is defined like the following: @PrimaryKey @Persistent(valueStrategy = IdGeneratorStra

Re: [google-appengine] Persisting Lists

2011-08-02 Thread MiuMeet Support
By the way, have a look at: http://devblog.miumeet.com/2011/08/much-more-efficient-implementation-of.html It's much more efficient than db.ListProperty(int) Cheers, -Andrin On Thu, Jul 28, 2011 at 6:17 PM, Pascal Voitot Dev < pascal.voitot@gmail.com> wrote: > good idea also :) > > > On Wed,

[google-appengine] Much more efficient implementation of db.ListProperty(int) - created a lib

2011-08-02 Thread Andrin von Rechenberg
So you want to store a "very long list of numbers" in your AppEngine Model which doesn't need to be indexed. Well, you could just do this: *class Foo(db.Model): numbers = db.ListProperty(int, indexed=False) * However, you will quickly notice that the performance and the memory consumption of thi

Re: [google-appengine] Re: Java: Precompilation fails.

2011-08-02 Thread David Hellekalek
Hi Johan, Thanks for the feedback. For me, the problem solved itself. What was troubling for me was that, besides the info that precompilation failed, no info was given. So I was left clueless whether it was caused by something on my side (my app's codes and libs) or an issue with the server that

[google-appengine] Re: Back-ends instances are dying and not restoring

2011-08-02 Thread Galoch
Seeing the same behavior. Found the url to the issue: http://code.google.com/p/googleappengine/issues/detail?id=5465 On Aug 2, 8:06 am, Kirill Zinoviev wrote: > Done - issue posted. Thanks for advice. > If will be resolved then will post results here. -- You received this message because yo

Re: [google-appengine] JDO schema change nightmare, app going live

2011-08-02 Thread Jeff Schnitzer
Sorry, just noticed this. Yeah, schema changes really don't seem to have been a design consideration for the JDO/datanucleus integration. If you want help with JDO, best to describe the exact changes you are making. If you want to port to Objectify, it shouldn't be too hard since Objectify direc

Re: [google-appengine] Huge latency spike on one of my apps

2011-08-02 Thread Johan Euphrosine
Can you share with us your application id ? Feel free to fill a production issue if this is directly affecting your operation: http://code.google.com/p/googleappengine/issues/entry?template=Production%20issue On Tue, Aug 2, 2011 at 5:43 PM, Grant wrote: > Hi > > I have had a huge spike in one of

[google-appengine] Huge latency spike on one of my apps

2011-08-02 Thread Grant
Hi I have had a huge spike in one of my appengine instances starting 4 or so hours ago (and still ongoing). Here is a picture of the graph milliseconds/request http://twitpic.com/5zvvd8 I have only noticed this on the one app (it is a python app), the others all seem to be behaving (python and

Re: [google-appengine] Re: Back-ends instances are dying and not restoring

2011-08-02 Thread Kirill Zinoviev
Done - issue posted. Thanks for advice. If will be resolved then will post results here. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/5v0QiIzVLj

Re: [google-appengine] JDO schema change nightmare, app going live

2011-08-02 Thread Johan Euphrosine
Hi Manish, Can you link the corresponding Stack Overflow question so we can get a bit more context. Thanks in advance. On Sun, Jul 31, 2011 at 6:47 AM, miwcoder wrote: > Hi, >   We are in the process of going live with our app but struggling > with schema changes. We intentionally adopted JDO a

Re: [google-appengine] Problem fetching db.ListProperty data

2011-08-02 Thread Johan Euphrosine
Hi Simon, A wild guess would be that your are not looking at the same entities in both methods, try to check the entity key using topic.key() You might also want to check that article about Modeling your data on App Engine: http://code.google.com/appengine/articles/modeling.html Hope that helps.

Re: [google-appengine] Connect from GAE to external PostgreSQL DB?

2011-08-02 Thread Pascal Voitot Dev
Not exactly the answer you expect certainly but wouldn't it better to implement a Web/REST service somewhere in the same network as the Postgres DB (if you can) which would just extract from Postgres and serve the data you need from your GAE app? Like that, you don't expose directly Postgres DB to

Re: [google-appengine] Re: Back-ends instances are dying and not restoring

2011-08-02 Thread Johan Euphrosine
Agreed, The form is there btw: http://code.google.com/p/googleappengine/issues/entry?template=Production%20issue On Mon, Aug 1, 2011 at 3:00 PM, Tim Hoffman wrote: > If you haven't already done so I suggest you also lodge a production issue. > T > > -- > You received this message because you are

Re: [google-appengine] Increase apps size restriction above 10 megabytes?

2011-08-02 Thread Johan Euphrosine
Did someone fill a doc bug for this ? Thanks in advance. On Sun, Jul 31, 2011 at 7:30 PM, Robert Kluin wrote: > I don't think that is correct, at least not according to some of the docs: > > - maximum size of an application file: 10 megabytes > - maximum size of a static file: 10 megabytes > - m

Re: [google-appengine] Can't download all data with bulkloader from non-default namespace

2011-08-02 Thread Timofey Koolin
Yes, I can. My app ID: my-optimiser (my-optimiser.appspot.com). It isn't very need personal for me now - I have only one Entity type and I can download it with --kind option, but I think it is bug in platform or SDK. Now I try use https://my-optimiser.appspot.com instead of https://3.my-optimiser

Re: [google-appengine] Can't download all data with bulkloader from non-default namespace

2011-08-02 Thread Johan Euphrosine
Can you share your application id, so we can investigate the issue ? Feel free to fill a production issue if this is directly affecting your operation: http://code.google.com/p/googleappengine/issues/entry?template=Production%20issue On Sat, Jul 30, 2011 at 10:19 AM, rekby wrote: > I use Python

Re: [google-appengine] Re: Java: Precompilation fails.

2011-08-02 Thread Johan Euphrosine
Can you two share your application id, so we can investigate the issue ? Feel free to fill a production issue if this is directly affecting your operation: http://code.google.com/p/googleappengine/issues/entry?template=Production%20issue On Sun, Jul 31, 2011 at 9:45 PM, Pavel Kaplin wrote: > Exp

Re: [google-appengine] Re: How do I do this query in JDO on server?

2011-08-02 Thread Johan Euphrosine
You should use Query filters, see the following page for more information: http://code.google.com/appengine/docs/java/datastore/queries.html#Restrictions_on_Queries Hope that helps. On Sun, Jul 31, 2011 at 11:35 AM, SergiBC wrote: > any help? > > On 26 jul, 21:11, SergiBC wrote: >> Hello, >> >>

Re: [google-appengine] "Enforced Rate" in task queues ruining performance

2011-08-02 Thread Johan Euphrosine
Can you both share your appid, so we can investigate deeper ? Also feel free to fill a production issue about it if this is directly affecting your operation: http://code.google.com/p/googleappengine/issues/entry?template=Production%20issue On Tue, Aug 2, 2011 at 3:08 PM, Dan Lines wrote: > I am

[google-appengine] Connect from GAE to external PostgreSQL DB?

2011-08-02 Thread wouterdeg
Hi All, For an app I need to obtain data from an external PostgreSQL database, do some calculations on it and then show the results to the clients in an MVP set-up. However, from some posts I gather that GAE cannot connect with a PostgreSQL DB over the internet out of its servlet context. Others

Re: [google-appengine] "Enforced Rate" in task queues ruining performance

2011-08-02 Thread Dan Lines
I am having the same problem. Today has been the worst performance so far with the enforced rate down at 0.16/s. The application is basically non functional at this rate Could we please get some information on this. - Dan -- You received this message because you are subscribed to the Go

Re: [google-appengine] Re: Stackoverflow

2011-08-02 Thread Gary Frederick
I was forced to vote for Groups over Stackoverflow. My real vote is both. Stackoverflow is ok for a focused question that does not get booted Groups are better if you have a question that does not fit into the criteria for Stackoverflow or you are more having a conversation. I do a lot with Goog

[google-appengine] Re: Data Center (CDN) Locations

2011-08-02 Thread Simon Knott
Hi, As far as I'm aware, the most accurate information the Google staff have divulged is that all data centres are located in North America somewhere. Cheers, Simon -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion

[google-appengine] Data Center (CDN) Locations

2011-08-02 Thread Ian Wilson
I have been searching around but have not seen information on where App Engine data centers are located and how loads are spread? So I have two questions: - Where are the App Engine data centers? (i.e. West Coast, East Coast, Europe...) - What criteria are used to serve an app from one data cent

[google-appengine] Re: Crazy number of Deadine Exceeded exceptions recently

2011-08-02 Thread Alexis
Hi, I've got a similar issue: lots of DeadlineExceeded errors since a few weeks. I'm on the master-slave datastore too, but what I'm reporting happened again one hour ago. These errors happen in bursts, and I recently realized that it was in fact shutting down ALL instances of the application. (I

Re: [google-appengine] Re: Unserializing large Blobs from the Blobstore

2011-08-02 Thread Luca Matteis
Hi Matt, There are some techniques for using as little memory as possible when serializing those large XML files, but you'll most likely go over the limit. Backends solve this issue. I think they need a little improvement since they require a little of sys-admin work to setup... but they're meant

Re: [google-appengine] Username/Password in Eclipse Plugin

2011-08-02 Thread Matthias Luft
Hi, On 08/02/2011 10:19 AM, Matthias Luft wrote: > I'm using the Eclipse plugin for GAE java development. Unfortunately, my > deployment directly from the plugin fails and I'm strongly suspecting > that I entered a wrong username/password at first try. Is there a > possibility to change this usern

Re: [google-appengine] Google App Engine for commercial purposes.

2011-08-02 Thread Gopal Patel
you can but it will not work. try typhoonae ro appscale. On Mon, Aug 1, 2011 at 11:19 PM, jhatzics wrote: > Hello, > > Does anyone know if I can deploy an application in my Windows 2003 > Server using GAE for commercial purposes? > > Thank you in advance, > > -- > You received this message becau

Re: Re: [google-appengine] Re: App Engine Chat Time!

2011-08-02 Thread Ian Wilson
Hi Brett, I have just finished a Facebook App built on Google App Engine and am currently engaged with another client to build their Facebook App. I would be happy to take a look at your requirements and discuss the project in more detail. You can contact me at [my first name] at [emotionai.com

[google-appengine] Username/Password in Eclipse Plugin

2011-08-02 Thread Matthias Luft
Hi, I'm using the Eclipse plugin for GAE java development. Unfortunately, my deployment directly from the plugin fails and I'm strongly suspecting that I entered a wrong username/password at first try. Is there a possibility to change this username/password? When I click deploy, there is no userna

[google-appengine] Re: Unserializing large Blobs from the Blobstore

2011-08-02 Thread Matt Myers
Hi Luca, did you ever figure this out? -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/L2H3-B0iq-kJ. To post to this group, send email to google-ap

[google-appengine] Google App Engine for commercial purposes.

2011-08-02 Thread jhatzics
Hello, Does anyone know if I can deploy an application in my Windows 2003 Server using GAE for commercial purposes? Thank you in advance, -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengin

[google-appengine] Re: Stackoverflow

2011-08-02 Thread Rodrigo Moraes
+1 Objective questions have reduced usefulness here, as there's more noise and they get lost with time. SO's peer review forces people to stay on topic, thus reducing noise and making questions more "reusable". The group is better for less objective discussions, of course. -- rodrigo -- You

Re: [google-appengine] Re: Stackoverflow

2011-08-02 Thread Johan Euphrosine
I created a quick poll for tracking the community feeling about this. https://spreadsheets.google.com/spreadsheet/viewform?formkey=dEFERTJWTTJxZEZ1dU91VXFvUFpzVnc6MQ Feel free to keep the discussion going in this thread. On Tue, Aug 2, 2011 at 8:56 AM, MLTrim wrote: > StackOverflow is perfect to

[google-appengine] most efficient bulk delete code?

2011-08-02 Thread Carter
Is this the most efficient code for bulk delete? Is there a "SELECT __key__" technique that's faster? This currently deletes 100 at a time, but it's slow. public long deleteBefore(Date d, long maxRecords) throws DaoException { PersistenceManager