if your industry lacks meaningful measurements, is devoid of independent, accurate, assessments, your industry has no good signals.
--- "No college,huh?" "How many PhD's do you have?" - someone selling security using credentials rather than capabilities as a signal. --- https://www.schneier.com/blog/archives/2007/04/a_security_mark.html A Security Market for Lemons More than a year ago, I wrote about the increasing risks of data loss because more and more data fits in smaller and smaller packages. Today I use a 4-GB USB memory stick for backup while I am traveling. I like the convenience, but if I lose the tiny thing I risk all my data. Encryption is the obvious solution for this problem -- I use PGPdisk -- but Secustick sounds even better: It automatically erases itself after a set number of bad password attempts. The company makes a bunch of other impressive claims: The product was commissioned, and eventually approved, by the French intelligence service; it is used by many militaries and banks; its technology is revolutionary. Unfortunately, the only impressive aspect of Secustick is its hubris, which was revealed when Tweakers.net completely broke its security. There's no data self-destruct feature. The password protection can easily be bypassed. The data isn't even encrypted. As a secure storage device, Secustick is pretty useless. On the surface, this is just another snake-oil security story. But there's a deeper question: Why are there so many bad security products out there? It's not just that designing good security is hard -- although it is -- and it's not just that anyone can design a security product that he himself cannot break. Why do mediocre security products beat the good ones in the marketplace? In 1970, American economist George Akerlof wrote a paper called "The Market for 'Lemons'" (abstract and article for pay here), which established asymmetrical information theory. He eventually won a Nobel Prize for his work, which looks at markets where the seller knows a lot more about the product than the buyer. Akerlof illustrated his ideas with a used car market. A used car market includes both good cars and lousy ones (lemons). The seller knows which is which, but the buyer can't tell the difference -- at least until he's made his purchase. I'll spare you the math, but what ends up happening is that the buyer bases his purchase price on the value of a used car of average quality. This means that the best cars don't get sold; their prices are too high. Which means that the owners of these best cars don't put their cars on the market. And then this starts spiraling. The removal of the good cars from the market reduces the average price buyers are willing to pay, and then the very good cars no longer sell, and disappear from the market. And then the good cars, and so on until only the lemons are left. In a market where the seller has more information about the product than the buyer, bad products can drive the good ones out of the market. The computer security market has a lot of the same characteristics of Akerlof's lemons market. Take the market for encrypted USB memory sticks. Several companies make encrypted USB drives -- Kingston Technology sent me one in the mail a few days ago -- but even I couldn't tell you if Kingston's offering is better than Secustick. Or if it's better than any other encrypted USB drives. They use the same encryption algorithms. They make the same security claims. And if I can't tell the difference, most consumers won't be able to either. Of course, it's more expensive to make an actually secure USB drive. Good security design takes time, and necessarily means limiting functionality. Good security testing takes even more time, especially if the product is any good. This means the less-secure product will be cheaper, sooner to market and have more features. In this market, the more-secure USB drive is going to lose out. I see this kind of thing happening over and over in computer security. In the late 1980s and early 1990s, there were more than a hundred competing firewall products. The few that "won" weren't the most secure firewalls; they were the ones that were easy to set up, easy to use and didn't annoy users too much. Because buyers couldn't base their buying decision on the relative security merits, they based them on these other criteria. The intrusion detection system, or IDS, market evolved the same way, and before that the antivirus market. The few products that succeeded weren't the most secure, because buyers couldn't tell the difference. How do you solve this? You need what economists call a "signal," a way for buyers to tell the difference. Warranties are a common signal. Alternatively, an independent auto mechanic can tell good cars from lemons, and a buyer can hire his expertise. The Secustick story demonstrates this. If there is a consumer advocate group that has the expertise to evaluate different products, then the lemons can be exposed. Secustick, for one, seems to have been withdrawn from sale. But security testing is both expensive and slow, and it just isn't possible for an independent lab to test everything. Unfortunately, the exposure of Secustick is an exception. It was a simple product, and easily exposed once someone bothered to look. A complex software product -- a firewall, an IDS -- is very hard to test well. And, of course, by the time you have tested it, the vendor has a new version on the market. In reality, we have to rely on a variety of mediocre signals to differentiate the good security products from the bad. Standardization is one signal. The widely used AES encryption standard has reduced, although not eliminated, the number of lousy encryption algorithms on the market. Reputation is a more common signal; we choose security products based on the reputation of the company selling them, the reputation of some security wizard associated with them, magazine reviews, recommendations from colleagues or general buzz in the media. All these signals have their problems. Even product reviews, which should be as comprehensive as the Tweakers' Secustick review, rarely are. Many firewall comparison reviews focus on things the reviewers can easily measure, like packets per second, rather than how secure the products are. In IDS comparisons, you can find the same bogus "number of signatures" comparison. Buyers lap that stuff up; in the absence of deep understanding, they happily accept shallow data. With so many mediocre security products on the market, and the difficulty of coming up with a strong quality signal, vendors don't have strong incentives to invest in developing good products. And the vendors that do tend to die a quiet and lonely death. _______________________________________________ Sent through the Full Disclosure mailing list http://nmap.org/mailman/listinfo/fulldisclosure Web Archives & RSS: http://seclists.org/fulldisclosure/
