Hi Stefano, If I understand your post correctly, you're using automatic scaling <https://cloud.google.com/appengine/docs/java/modules/#automatic_scaling_1> and have only configured the min_idle_instances parameter to be 1. You're wondering why you've seen a certain request distribution pattern or dynamic instance lifetime on your app. I think that the answer to these questions lies in understanding the App Engine Autoscaler.
Putting your scaling settings into default means that you give up control over max_idle_instances (that is, there is no max), and so the Autoscaler can and will spawn idle instances to preempt traffic or respond to sudden load spikes. This is based on past request loads on your app and projections based on spikes. You will probably want to determine your current and projected request load and set a sensible maximum to this value, since at that point you could be assured that you won't be charged for more idle instances, if they are spawned, than your configuration specified <https://cloud.google.com/appengine/docs/java/modules/#max_idle_instances>. As I can see from looking at the data you supplied, an instance which was only active for 34 minutes handled 1108 requests. While it's certainly possible that this load was handled well by the instance, if the same load was observed on the other two machines at some point in the past, which the overall graph of requests seems to imply, especially given how spiked the load can tend to be within even the past 6 hours (the timeframe of the graph), it makes sense that the Autoscaler would keep some instances in reserve and route requests to them once the load went higher, but not spin them down in case a new spike were to arrive. The Autoscaler scales aggressively in order to ensure that you won't be left overloaded, and it's up to your scaling configuration to make sure that this matches your expected load. I suggest you look into the max and min_pending_latency configuration options as well as min and max_idle_instances. Now, as to your question about resident instances: a resident instance is an instance which is always ready for requests, and won't spin down due to inactivity. You specified one min idle instance, so you have one resident instance. Idle instances ("resident" instances) are meant to allow your app to handle traffic while all dynamic instances (if there are any) are overloaded, and new dynamic instances need to be created, so generally they will only receive traffic during the front edge of a load spike, acting as a buffer as opposed to the main force of request-handling instances. While resident instances won't spin down due to inactivity, they can shut down due to errors or memory limits being reached, in which case the instance will receive a request to /_ah/stop which allows it to attempt to save any delicate state it's holding. Instances can also (on a rare occasion) restart due to datacenter events and maintenance. However more common reasons for seeing a dynamic instance with a smaller age than other dynamic instances is either because a change has just been made which deploys a new resident instance (such as a configuration change to specify 1 resident instance), or because a resident instance which is serving traffic and becomes quite busy with that task might transition to being a regular serving dynamic instance as a small efficiency gain, and another resident instance will be created to take its place as a buffer for even higher load spikes. So, overall, my advice is to keep in mind that the Autoscaler will scale your instances to meet spiking loads (which it seems you have), and that your scaling configuration can help keep this within certain boundaries. I hope this has helped explain what you're seeing, and feel free to ask any further questions you might have. Regards, Nick On Friday, August 7, 2015 at 7:11:49 AM UTC-4, Stefano Ciccarelli wrote: > > Hello to all! > > Our application has always suffered from high latencies due to instances > respawning continuously. > We always used custom settings in "automatic-scaling" to try to save money. > > Following this post ( > http://googlecloudplatform.blogspot.it/2015/08/How-to-Troubleshoot-Latency-in-Your-App-Engine-Application.html) > > I've decided to reset the "automatic-scheduling" settings of our app to > default values. > > So I've configured only min-idle-instances to 1 (otherwise the /_ah/warmup > handler is never called). The instance class is F2. > > After two days this is our state. > > So: > - why I have 4 instances 2 days old not used? (The logs says that one > instance served ~20 request today, but the other instances stopped to serve > yesterday) > - why the scheduler stop and spawn instances every 40/50 minutes if 4 > instances are there to do nothing? > - why the resident instance is restarted every 40/50 minutes if it is > supposed to be... resident? > - is there something wrong with our code? > > Thanks > > > > > [image: nimbus-image-1438944970211.png] > -- > *Stefano Ciccarelli* > GAE Application Division > / Director > [email protected] > > *M.M.B. s.r.l.* > via Granarolo, 177/7 - 48018 Faenza (RA) - Italy > tel. +39.0546.637711 - fax +39.0546.46077 > www.mmbsoftware.it - [email protected] > > Le informazioni contenute in questa comunicazione sono riservate e > destinate esclusivamente alla/e persona/e o all'ente sopra indicati. E' > vietato ai soggetti diversi dai destinatari qualsiasi uso, copia, > diffusione di quanto in esso contenuto sia ai sensi dell'art. 616 c.p., sia > ai sensi del DL n. 196/03. Se questa comunicazione Vi e' pervenuta per > errore, Vi preghiamo di rispondere a questa e-mail e successivamente > cancellarla dal Vostro sistema. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/b4e88571-6790-4fb7-b129-afb82d27821b%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
