Notatypewriter's Blog

Umm… what?

Lecture Notes: UMD Cybersecurity Seminar “The Argument for Data-driven Security” by Prof Stefan Savage, UCSD

leave a comment »

I attended a lecture (video) given by Professor Stefan Savage at the Google & University of Maryland Cybersecurity Seminar Series on the need for data in computer security.

He says the computer security field today is driven by patching or mitigating the vulnerability of the week. Presenters at the premiere computer security conferences like Black Hat and DEFCON talk about their latest exploit. Savage says this approach is ineffective at actually keeping people secure. Savage proposes that viewing security problems through the lens of business and economics can be used to gain insight into the effectiveness of measures taken to protect against or attack the computer criminals and those who enable them.


  • Savage says in the business context, the effectiveness of security is not a yes-no answer (Did it work?) but rather it should be phrased in a cost-benefit analysis. By how much did a security measure increase costs for the bad guys? Does a security measure trash the investments the bad guys made into their infrastructure? In the case of takedowns, can the bad guys quickly switch to another provider of services you just took down? Economics and business can help us understand mitigations.
  • I would tend to agree. His analysis of the spam ecosystem made sense, not that it has been put into practice… yet. I do agree with the need for more data.
  • Savage’s data collection infrastructure is massive and awesome. He claims to be able to view 1% of global Internet traffic. Thousands of instances of Firefox clicking on millions of spams. Dozens of Internet connections distributed across the world used to browse spammer’s sites. Automatic clustering and classification of illegal pharmacy sites to tie these sites to the affiliate marketer that made the site. Without this infrastructure, Savage’s research and ultimate recommendation would not have been possible.
  • “Before we had cloud, we had botnets.”
  • Apparently, bank scams can be run out of Iran. It wasn’t clear whether this was officially sanctioned, officially ignored, or just the result of lax enforcement.
  • Kill the illegal payment processors now.

Savage says traditional defense forces defenders to operate in an arena where they suffer from four structural asymmetries:

  • Initiative: Defenders have to play catch up. For example, antivirus vendors only push out new signatures after malware is in the wild and owning boxes.
  • Innovation: Defenses such as IDS or even security reviews of software or other systems represent a sunk cost. On the other hand, attackers require only low capital investment to get around these systems. Antivirus is hilariously ineffective.
  • Incentives: There’s no effective deterrent against attackers. Furthermore we have no reward for effective security, in large part because we have no good way of measuring the effectiveness of a defender or a mitigation.
  • Measurement: Attackers can measure effectiveness by dollars stolen, bots acquired, etc.

Notice how these four asymmetries sound so similar to defenders in meatspace. Think castles.

Savage next asks how the security field intends to solve the problem. With SCIENCE of course, but which science…

Math? A large part of security is about how humans interact with other humans using computers and the Internet as the intermediary. However, so far there’s no axioms that can define human interaction, so this field can’t help us here. (Program proofs, though currently a pipe dream, will help add security but users are still too fucking stupid.)

Savage says we should turn to the social sciences. Yes, the one field we computer science people disdain as a soft science. (Ironically, Savage studied Applied History at a university that also contains one of the top Computer Science schools in the nation, Carnegie Mellon.) But it is social science and economics and business in particular that have given us much insight into human motivations and desires. After all, malware authors, spammers, and their victims are humans as well (we hope anyway).

So let’s apply a business analysis to a current security problem: the spam problem.

Savage says in 2004, malware became commercialized and a market developed to sell bots to bad guys. Spammers saw that their usual spam machines were increasingly getting blocked so they wanted bots to relay spam.

Then we saw a specialization of roles. Malware people wrote malware to acquire bots. Affiliate marketers rented bots from the malware people to send spam for illegal pharmacies or suppliers of herbal supplements and other snake oils. What he called “click support” is responsible for the web page the user lands on when he clicks on the spam and the payment processing for handling the order. And the pharmacies handled the production and distribution of the purchased drug.

These four roles hold value in different arenas. Pharmacies have goods which hole direct value in the real world. Affiliates, spammers, payment processors, email and website designers, and malware authors hold value only in the underground economy. These guys represent capital investments and infrastructure in this market. Therefore, the market operations according to this inequality:

advertising cost < conversion rate * marginal revenue

This inequality is essentially the same inequality that legitimate stores like Walmart have to settle.

If we look deeper at the costs in this market, we find that affiliates have pushed off liability, innovation, and supply costs to the malware authors and the pharmacies.

Having understood this market, let’s try to measure how well CAPTCHAs work.

Since a mail server can’t really block all of Yahoo, Hotmail, Gmail, etc, without severely impacting their users, spammers love these accounts valuing them at up to 200x more then bots, according to Savage’s research. Webmail sites frequently use CAPTCHAs to prevent bots for signing up for accounts, so people will try to create a bunch of accounts and consequently have to solve a bunch of CAPTCHAs.  Savage found that the underground economy values CAPTCHAs at the rate of $1 per 1000 CAPTCHAs. This represents an increased cost to the spammers.

It also yields an interesting observation where the high skill labor was priced out of the market by more efficient lower skilled workers. The high-skill workers were programmers writing software to solve CAPTCHAs. These never got above 20% effectiveness and the website simply could switch CAPTCHA software and the CAPTCHA solvers became useless. In this case, the CAPTCHA solver became the capital investment. Since switching CAPTCHA software didn’t really impact normal human users, it was essentially cost-free so the webmail site was able to get ahead of the bad guys.

It turns out that there are people in Bangledesh and China who are paid around $3/day to solve CAPTCHAs all day long. A whole ecosystem has sprung up around this market, where people who sign up for email accounts using bots submit CAPTCHAs to a web service that displays it to the solver and returns the result. There are three distinct business lines in this transaction and each can be occupied by whoever thinks they can be efficient (eg, making money) at it.

So does CAPTCHA work?

Savage says this is the wrong question. In the business context, security is not a binary question to be answered yes or no. Instead it is a question that asks by how much did CAPTCHA increase the bad guys’ costs? In this case, CAPTCHAs killed off the bad guys who were writing the solver programs. In other words, it killed off inefficient businesses.

How do we kill off this market?

Savage proposes we take the traditional approach — find bottlenecks in the system where all the bad guys depend on a few providers and eliminate the ones that are worth eliminating — eg a traditional cost-benefit analysis.

  • Registrars: 34% of affiliates used one Russian registrar, but there was a long tail of other registrars being used. It’s easy to switch to another registrar and there are a lot of alternatives, so attacking registrars probably won’t work. (Liu, et al 2011 PDF warning)
  • DNS/Web hosting: Long, long tail, longer then registrars. Therefore there are a lot of alternatives and low switching costs

He says look at the banks that handle the credit card transactions for the affiliates. 3 banks control 95% of the market. There are few alternatives so there is a huge switching cost. Banks represent a large proportion of the costs in this market because they hold onto the money for two weeks to account for chargebacks and affiliates typically have to register for accounts in-person. These represent capital costs to the bad guys.

If US banks denied certain transactions to merchants with pharmaceutical merchant codes with accounts at these three banks, then we would see costs for the bad guys shoot way up. This represents an asymmetry that the good guys have.


Written by notatypewriter

2011 September 2 at 12:30 am

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: