Amazon diagnosis

2011-10-10 Thread Kristen Eisenberg
Sure they can, but as a thought exercise fully 2n redundancy is difficult on a small scale for anything web facing. I've seen a very simple implementation for a website requiring 5 9's that consumed over $50k in equipment, and this wasn't even geographically diverse. I have to believe that scalin

RE: Amazon diagnosis

2011-05-06 Thread Kenneth M. Chipps Ph.D.
I am preparing a graduate level course for network managers. As part of this course I would like to use a series of case studies looking at problems such as described in the report from Amazon. If anyone has something similar or knows where I could find such things, I would appreciate a copy or

Re: Amazon diagnosis

2011-05-06 Thread Jay Ashworth
- Original Message - > From: "Ryan Malayter" > On May 5, 3:51 pm, Jay Ashworth wrote: > > - Original Message - > > > From: "Ryan Malayter" > > > I like to bag on my developers for not knowing anything about the > > > infrastructure, but sometimes you just can't do it right becau

Re: Amazon diagnosis

2011-05-06 Thread Ryan Malayter
On May 5, 3:51 pm, Jay Ashworth wrote: > - Original Message - > > From: "Ryan Malayter" > > I like to bag on my developers for not knowing anything about the > > infrastructure, but sometimes you just can't do it right because of > > physics. Or you can't do it right without writing you

Re: Amazon diagnosis

2011-05-05 Thread Jay Ashworth
- Original Message - > From: "Ryan Malayter" > I like to bag on my developers for not knowing anything about the > infrastructure, but sometimes you just can't do it right because of > physics. Or you can't do it right without writing your own OS, > networking stacks, file systems, etc.,

Re: Amazon diagnosis

2011-05-05 Thread George Herbert
On Thu, May 5, 2011 at 10:45 AM, Ryan Malayter wrote: > > > On May 1, 2:29 pm, Jeff Wheeler wrote: > >> What it really boils down to is this: if application developers are >> doing their jobs, a given service can be easy and inexpensive to >> distribute to unrelated systems/networks without a hug

Re: Amazon diagnosis

2011-05-05 Thread Ryan Malayter
On May 1, 2:29 pm, Jeff Wheeler wrote: > What it really boils down to is this: if application developers are > doing their jobs, a given service can be easy and inexpensive to > distribute to unrelated systems/networks without a huge infrastructure > expense.  If the developers are not, you end

Re: Amazon diagnosis

2011-05-03 Thread Phil Pierotti
Unlike the US of A, here in Australia the industry has gone *very* heavily down the path of requiring/expecting certification. They have bought into the faith that unless your resume includes CC?? you're worthless. There are "colleges" (er, I mean training businesses) who will *guarantee* you will

Re: Amazon diagnosis

2011-05-03 Thread Jason Baugher
On 5/2/2011 4:11 PM, George Herbert wrote: On Mon, May 2, 2011 at 2:04 PM, Jeroen van Aart wrote: valdis.kletni...@vt.edu wrote: On Mon, 02 May 2011 12:27:34 PDT, Jeroen van Aart said: It surprised me because I, perhaps naively, assumed IT workers in general have a rather broad knowledge So

Re: Amazon diagnosis

2011-05-02 Thread James Smith
wledge you have about anything and everything that your systems interact with then the better you will be at your specialty. Sent from my "contract free" BlackBerry® smartphone on the WIND network. -Original Message- From: Jeroen van Aart Date: Mon, 2 May 2011 19:27:34 To:

Re: Amazon diagnosis

2011-05-02 Thread George Herbert
On Mon, May 2, 2011 at 2:04 PM, Jeroen van Aart wrote: > valdis.kletni...@vt.edu wrote: >> >> On Mon, 02 May 2011 12:27:34 PDT, Jeroen van Aart said: >> >>> It surprised me because I, perhaps naively, assumed IT workers in general >>> have a rather broad knowledge > >> Sorry to break it to you. >

Re: Amazon diagnosis

2011-05-02 Thread Jeroen van Aart
valdis.kletni...@vt.edu wrote: On Mon, 02 May 2011 12:27:34 PDT, Jeroen van Aart said: It surprised me because I, perhaps naively, assumed IT workers in general have a rather broad knowledge Sorry to break it to you. That's ok, the past tense in my story testifies to the fact I was alread

Re: Amazon diagnosis

2011-05-02 Thread Paul Graydon
On 05/02/2011 09:27 AM, Jeroen van Aart wrote: Jeff Wheeler wrote: IT managers would do well to understand that a few smart programmers, who understand how all their tools (web servers, databases, filesystems, load-balancers, etc.) actually work, can often do more to I fully agree. But much t

Re: Amazon diagnosis

2011-05-02 Thread Valdis . Kletnieks
On Mon, 02 May 2011 12:27:34 PDT, Jeroen van Aart said: > It surprised me because I, perhaps naively, assumed IT workers in > general have a rather broad knowledge No, the average IT worker is always a mere 3 keystrokes away from getting their latest creation listed on www.thedailywtf.com. They'

Re: Amazon diagnosis

2011-05-02 Thread Jeroen van Aart
Jeff Wheeler wrote: IT managers would do well to understand that a few smart programmers, who understand how all their tools (web servers, databases, filesystems, load-balancers, etc.) actually work, can often do more to I fully agree. But much to my dismay and surprise I have learned that dev

Re: Amazon diagnosis

2011-05-01 Thread Stefan
On Fri, Apr 29, 2011 at 2:35 PM, Joly MacFie wrote: > *http://aws.amazon.com/message/65648/* > > ___ > -- > --- > Joly MacFie  218 565 9365 Skype:punkcast > WWWhatsup NYC - http://wwwhatsup.com >  htt

RE: Amazon diagnosis

2011-05-01 Thread Robert Bonomi
> Subject: RE: Amazon diagnosis > Date: Sun, 1 May 2011 12:50:37 -0700 > From: George Bonser > > They apparently had a redundant primary network and, on top of that, a > secondary network. The secondary network, however, did not have the > capacity of the primary net

Re: Amazon diagnosis

2011-05-01 Thread Valdis . Kletnieks
On Sun, 01 May 2011 11:07:56 PDT, Mike said: > I am still waiting for proof that single points of failure can > realistically be completely eliminated from any moderately complicated > network environment / application. So far, I think murphy is still > winning on this one. For starters, you a

Re: Amazon diagnosis

2011-05-01 Thread Robert Bonomi
> Date: Sun, 01 May 2011 11:07:56 -0700 > From: Mike > To: nanog@nanog.org > Subject: Re: Amazon diagnosis > > On 04/29/2011 12:35 PM, Joly MacFie wrote: > > http://aws.amazon.com/message/65648/ > > > > ___ > > > So, in a nut shell, Amazon had a sing

Re: Amazon diagnosis

2011-05-01 Thread Brett Frankenberger
On Sun, May 01, 2011 at 12:50:37PM -0700, George Bonser wrote: > > From my reading of what happened, it looks like they didn't have a > single point of failure but ended up routing around their own > redundancy. > > They apparently had a redundant primary network and, on top of that, a > secondar

Re: Amazon diagnosis

2011-05-01 Thread Paul Graydon
On 5/1/2011 9:29 AM, Jeff Wheeler wrote: On Sun, May 1, 2011 at 2:18 PM, Andrew Kirch wrote: Sure they can, but as a thought exercise fully 2n redundancy is difficult on a small scale for anything web facing. I've seen a very simple implementation for a website requiring 5 9's that consumed ov

RE: Amazon diagnosis

2011-05-01 Thread George Bonser
> I am still waiting for proof that single points of failure can > realistically be completely eliminated from any moderately complicated > network environment / application. So far, I think murphy is still > winning on this one. > > Good job by the AWS team however, I am sure your new procedures

Re: Amazon diagnosis

2011-05-01 Thread Jeff Wheeler
On Sun, May 1, 2011 at 2:18 PM, Andrew Kirch wrote: > Sure they can, but as a thought exercise fully 2n redundancy is > difficult on a small scale for anything web facing.  I've seen a very > simple implementation for a website requiring 5 9's that consumed over > $50k in equipment, and this wasn'

Re: Amazon diagnosis

2011-05-01 Thread Andrew Kirch
On 5/1/2011 2:07 PM, Mike wrote: > I am still waiting for proof that single points of failure can > realistically be completely eliminated from any moderately complicated > network environment / application. So far, I think murphy is still > winning on this one. Sure they can, but as a thought exe

Re: Amazon diagnosis

2011-05-01 Thread Jay Ashworth
- Original Message - > From: "Mike" > On 04/29/2011 12:35 PM, Joly MacFie wrote: > > *http://aws.amazon.com/message/65648/* > > So, in a nut shell, Amazon had a single point of failure which touched > off this entire incident. > > I am still waiting

Re: Amazon diagnosis

2011-05-01 Thread Mike
On 04/29/2011 12:35 PM, Joly MacFie wrote: *http://aws.amazon.com/message/65648/* ___ So, in a nut shell, Amazon had a single point of failure which touched off this entire incident. I am still waiting for proof that single points of failure can reali

Amazon diagnosis

2011-04-29 Thread Joly MacFie
*http://aws.amazon.com/message/65648/* ___ -- --- Joly MacFie 218 565 9365 Skype:punkcast WWWhatsup NYC - http://wwwhatsup.com http://pinstand.com - http://punkcast.com VP (Admin) - ISOC-NY - http