Chaordic Mind

This is a re-post of an article I wrote for IT Compliance Advisor, a part of TechTarget.com, in August 2009.  I find the material to be just as applicable now as it was then.  You can find a list of reference material here.

One of the problems that many companies face is staying ahead of the information security curve. Go too fast and you run the risk of wasting capital, but run too slow and you run the risk of being compromised. So how a company can escape the hamster wheel of pain? Be proactive in managing risk and implementing a maturity framework for the organization.

In an attempt to balance the two domains of cost and security, a continual tradeoff, many companies have implemented regulatory compliance standards. These are good tools for measuring ones security to a known industry baseline. The classic example of this is the Payment Card Industry Data Security Standard (PCI DSS). Using standards like PCI DSS, companies can measure their adherence to eliminating sensitive data and protecting the remaining in-scope systems.

There are two problems with aligning an entire information security model along any singular guideline. It should be noted that, in the absence of any information security program, PCI DSS is a very good baseline standard.

The first challenge is the 0-to-100 problem. Some companies start with no information security program and try to adhere to something like PCI DSS. Much like measuring the acceleration of a car by how fast it can go from 0 to 100 miles per hour, these companies struggle with getting from 0 to 100 percent compliance in under 12 months. For these companies this means implementing security for the sake of a deadline, which means not always having the time to test what works and what does not.

The second challenge is the security limiter problem. Once companies reach 100 percent adherence to a given standard, many times they stop developing their information security program. These companies then enter a vicious cycle of identification and remediation. Each year, their auditors alert them to a new set of issues and, each year, the companies fix those and then relax until the following year.

So how do we escape this endless cycle of identification and remediation? How do we provide a way for companies to go from 0 mph to 50 mph in year one, 50 to 100 in year two, and still be inspired to go from 100 to 150 in year three? How do we become proactive instead of being reactive? One option for addressing these problems is the capability maturity model (CMM) that involves risk management.

A CMM is nothing new or innovative. It’s a useful approach for managing the maturity in a system. The Computer Security Handbook 4th Edition reveals that CMMs originated from software development. This book states that a CMM “can be used as a way to assess the soundness of a security product builder’s engineering practices during the many stages of product development.” If a CMM can be used for measuring the soundness of engineering practices, then why not leverage it to measure the soundness of information security practices?

A maturity model encourages continual growth rather than strict adherence to Procrustean boxes of information security. It’s the mathematical equivalent of the integral or the continual variable transmission of an automobile. It provides a smooth curve instead of designated endpoints of information security. For companies suffering from the 0-to-100 problem, a maturity model enables growth from 0-to-50 initially, with the projection of moving from 50-to-100 at a later date. Companies that suffer from the security limiter problem have the ability to continuously and proactively plan information security development to parallel growing business needs, instead of an independent set of criteria.

The Information Security Management Maturity Model (ISM3, or ISM-cubed) provides us with the intersection of information security and a maturity model for growing an information security program. ISM3 describes the process this way:

“Rather than focusing on controls, it focuses on the common processes of information security, which are shared to some extent by all organizations.

Under ISM3, the common processes of information security are formally described, given performance targets and metrics, and used to build a quality assured process framework. Performance targets are unique to each implementation and depend upon business requirements and resources available. Altogether, the performance targets for security become the Information Security Policy. The emphasis on the practical and the measurable is what makes ISM3 unusual, and the approach ensures that ISM systems adapt without re-engineering in the face of changes to technology and risk.”

In fact, the ISM3 is based in part on extending the Systems Security Engineering Capability Maturity Model (SSE-CMM), which is ISO standard 21827. The SSE-CMM “describes the essential characteristics of an organization’s security engineering process that must exist to ensure good security engineering.”

In addition, consider the Building Security in Maturity Model (BSIMM), which is “designed to help you understand and plan a software security initiative.” As well there is the, Open Software Assurance Maturity Model (OpenSAMM) project that can “help organizations formulate and implement a strategy for software security that is tailored to the specific risks facing the organization.” These frameworks exist as tools for helping develop the maturity of organizations and software through the use of measured metrics.

And metrics is where all the magic really happens. Only by measuring the maturity of an organization and matching it to the development and progress of known attacks can we demonstrate that we are maintaining the balance between costs and security. There is a saying that if you and your friend are being chased by a bear, you don’t need to outrun the bear — you need only outrun your friend. In the world of ever-increasing compromises, many companies struggle to stay ahead of the curve. A maturity model, with proper metrics, can help your organization do just that. The best part? Companies that implement a maturity model and show measured growth are many times more likely to adhere to industry standards such as the PCI DSS.

This post clarifies an earlier one “Considering an Opt-Out Program on PCI Validation“ and helps explain how PCI compliance validation is changing based on risk measures present in the merchant’s environment.  Regulation and deregulation cycles happen in response to market forces.  In this case selective deregulation is happening in the form of reduced validation based on risk and fraud reduction measures present in merchant organizations.

Present State

When many companies think of PCI compliance they immediately think of a third-party QSA auditor.  For mature organizations this is the old way of thinking as both Visa and MasterCard permit merchants of any level to self-assess.

• Visa (Inc. and Europe) permits a report on compliance from an internal auditor provided it is signed off by an officer of the corporation.
• MasterCard permits self-assessments but internal auditors must “attend PCI SSC ISA Training and pass the associated accreditation program”

Although organizations must validate annually, they are relieved of this in the following  situations (as noted by Simon Sharp):

• Visa Inc.: merchant does 75%+ EMV transactions = no requirement for ongoing external assessment (major abbreviation)
• Visa Europe: merchants meet 1-4 milestones of Prioritized Approach are in a safe harbor even if breached (major abbreviation)
• Visa Asia: merchants who implement end-to-end encryption or process EMV chip transactions in countries where iCVV penetration is >75% have the following options:
  • Validated compliance with milestones 1-4 of the PCI SSC’s Prioritized Approach are recognized as fulfilling Visa PCI DSS validation requirements.
  • Attested to not storing prohibited data and process EMV chip transactions in markets where iCVV penetration is higher than 75 percent – you may define merchant level by the annual volume of non-chip transactions.

Reducing Risk = Reduced Validation

Visa Inc’s Technology Innovation Program (TIP) notes organizations that reduce fraud risk using technologies such as EMV (Chip/PIN) no longer need to validate compliance annually.  Visa Europe has their own version of TIP that goes a step further to say that for merchants who validate against the Prioritized Approach 1-4, Visa Europe will:

• waive penalties for non-compliance or non-progression
• grant ‘safe harbour’ from penalties and allocation of incremental counterfeit fraud losses in the event of a data compromise

Sure there are caveats and I’m not certain what “allocation of incremental counterfeit fraud losses” entirely means, but the idea that a merchant will achieve safe-harbor from anything is a pretty big carrot with which to lead merchants.

Certainly the pendulum has moved from encouraging compliance to encouraging risk and fraud reduction.  To this end the Visa has changed from incentivizing compliance, via the Visa CAP program in 2007, to incentivizing risk and fraud reduction, via the Visa TIP programs in 2011.

PCI Deregulation

Perhaps it’s premature to say that PCI compliance as an industry is in a deregulation phase.  Clearly PCI compliance for regions that have not seen wide adoption such as Asia/Australia still need movement towards full compliance and validatoin.  Conversely, if a merchant has >95% of transactions using EMV (Chip/PIN) with iCVV and CDA authentication – the need for PCI compliance may be limited.

Although deregulation may never fully occur, the need for annual third-party validation is no longer necessary for companies that have either: reduced the risk to payment card data or have highly-mature internal controls and validation capabilities.

Abstract

As regulation-deregulation cycles rise and fall, it is important to understand how the evolving landscape of compliance impacts your future. This post proposes maintaining compliance but making validation an opt-out optional component – a radical change from the status quo.  Evidence already suggests the industry is moving in this direction and changes to compliance are necessary for the continuance of risk management.

Please understand that when I say opt-out, I am referring to mandated external, third-party validation requirements. I think internal validation is more important than ever.

Special thanks to idea people: @lennyzeltser, @mckeay, @alexhutton, @kindervag, @joshcorman

Background

I recently read Lenny Zeltser’s blog titled “Could Regulatory Compliance Encourage Weaker Security?” This is a valid question and one that needs addressing. The question can be rephrased as, “Who does compliance work best for?” To answer that question we need to understand why compliance exists.

In a blog post I wrote on How Compliance Regulations Gets Made we focus on the natural regulation-deregulation cycles and how they exist in response to an increase or decrease in data breach/loss. The ultimate goal of compliance is to set a baseline of standards within an industry. The question Lenny raises is one I’m often asked by opponents of such standards, “what about the big/little guy (who do not fall within the Bell Curve norm for best practices)?”

It’s true that regulatory compliance is targeted not only at setting a minimum standard for technical security (firewalls and IDS) but also a minimum standard for security maturity (policies and procedures) within an organization. So let’s think about this graphically. There are four quadrants within which to place organizations: those with either high/low-level of security and high/low-level of maturity.

Security vs Maturity

For the purpose of this conversation let’s assume that maturity encompasses the Check and Act aspects of the PDCA Cycle and security refers to the Plan and Do components. The reason I break it down this way is to directly reflect the results of the Verizon PCI Compliance Report (PCIR). This report found that:

“Organizations are better at planning and doing than checking. If the check phase is broken, they cannot act to maintain the state of security over time.”

The Verizon PCIR found that organizations are great at Planning and Doing but not great at Checking and, as a direct result, Acting on those changes. To me this disconnect is the difference between organizations with a high-level vs low-level of maturity within their security practice.

With this in mind, let me suggest that regulatory compliance standards should most impact those organizations with a lack of either security or maturity, but not both. So let’s break this down and the types of organizations they embody.

1. High-Security / Low-Maturity: These companies care about security but have never documented policies and procedures. They have log management systems but have slowly stopped reviewing them. Regulatory compliance can have a positive impact here.
2. Low-Security / High-Maturity: These or organizations run well but with little funding for sorely needed security projects. There has never been a “hammer” to drive spending. Regulatory compliance can have a positive impact here.
3. Low-Security / Low-Maturity: These are organizations that do not care about security or compliance. Perhaps they are too small (mom-and-pop companies) or those that will validate compliance but never maintain it through the year. There is no changing these companies and little that compliance can do for them. Validating compliance for them is a waste of time and money and since there is no driver to maintain a state of security.  (Instead new technologies such as tokenization, end-to-end encryption, and validated payment applications will have the highest impact here.)
4. High-Security / High-Maturity: These are companies at the top-tier of their breed. They don’t manage security, they manage risk! They adopt and implement custom risk management solutions based on careful analysis of data classification and impact analysis reports. These companies see regulatory compliance as a roadblock and implementing industry “best practices” as a deviation from their perfect path.

I propose that regulatory compliance will most help groups 1 and 2, but not groups 3 and 4.  (Unless you consider regulatory compliance the driving force for said technologies above, though I would argue data breaches and word of mouth have a higher impact here than compliance.)

Although I believe in the need for increased education, flexibility of controls, and more data for risk modeling – I’m going to save us a bit of time and skip to the chase.

• Companies in group 3, who do not care about compliance or security, will not change their tune by forcing them to validate compliance.  Instead the end result will most likely be in them checking a box and ending up in the 80% of companies (see: Verizon PCIR) that do not maintain their state of compliance.
• Companies in group 4, who care passionately about risk and security, need a reprise from continually validating against a standard that is built for the average individual. Although, the stated way to address this for PCI compliance is through documenting a set of Compensating Controls, what other options do we have out there? What other ways are there for such companies dealing with compliance validation?

Remember, the stated goal of regulatory compliance, taken from regulation-deregulation cycles, is to reduce the number of data breaches and data loss. In both groups 3 and 4, continual validating against a standard may, in my opinion, have little to no impact on the number of data breaches/loss. The reason is that group 3, though validating will not maintain that validation, and group 4, treat validation as an exercise in documentation.

Other Options

On February 6, 2011, Visa launched its Technology Innovation Plan (TIP) “to recognize and acknowledge merchants in Visa Inc. regions outside of the United States that have taken action to prevent counterfeit fraud by investing in EMV technology.” (Since Visa Europe is a franchise, the “outside the US” may only apply to Asia-Pacific and Latin-America & Caribbean, but it’s a bold change we should view as the tip of an iceberg.)

In essence, they are saying that organizations that have achieved the following, need not continue to validate their compliance against the PCI DSS standard:

• Implemented a sufficient level of controls so as to reduce fraud* (see: EMV)
• Validated their state of compliance once
• Have not suffered a data breach

* Yes, fraud is discernibly different from data breaches but one leads to the other and as a result are interconnected.

Wow, what an innovative approach. I’ve talked about the TIP program with industry insiders and they are mostly in agreement that we don’t know if this will result in positive or negative changes. I feel it will be a great success and here is why.

Opting Out of Validation (Not Compliance)

Presently companies that validate their state of compliance need to submit two things: a validation document (either a self-assessment questionnaire or a report on compliance) and an attestation of compliance (AOC) document. The AOC is nothing more than a memo that reiterates that organizations commitment to following the payment-brand rules for protecting payment card data.

I think organizations that choose to opt-out of compliance validation should still need to sign the Attestation on Compliance (AOC) to reaffirm their social contract and commitment to protecting payment card data. If they fail to achieve this within their, alleged, super-robust security and risk program then they deserve to undergo the same forensic review and financial implications that come with any other organization. If they instead achieve in protecting payment card data and are able to repel the wily-hacker then they should continue their reprieve from annual compliance validation (perhaps they can externally-validate every 2 or 3 years).

The reason I suggest this is because, and here’s the kicker, you cannot tell the difference between a PCI compliant organization and one that has let security and compliance lapse until they experience a data breach. Until that point, both organizations appear, from the outside, to be operating in the same manner.  (Sure, you can tell a difference internally, but so far very few organizations that achieve compliance once organically maintain it year-over-year.)

But Wait – It Already Exists

The PCI Council has already rolled out the Internal Security Assessor (ISA) program and MasterCard has begun listing this qualification as part their validation program requirements.

“Effective 30 June 2011, Level 1 merchants that choose to conduct an annual onsite assessment using an internal auditor must ensure that primary internal auditor staff engaged in validating PCI DSS compliance attend PCI SSC ISA Training and pass the associated accreditation program annually in order to continue to use internal auditors.”

(Ok, so Visa has not adopted the same stance and companies that store, process, or transmit payment card data for both brands must adhere to the minimum standard for both, but still it’s a change.  Also, the payment card brand validation guidelines are guidance for the acquiring banks who have the ability to manage their validation programs on a case-by-case basis.)

This means that many organizations (there are exceptions) who wish to opt-out of formal validation can do so leveraging their internal assessor team.

Conclusion

What we have is a directional movement towards, what I will call, selective deregulation. Step 1 is the PCI SSC ISA program.  Step 2 is the Visa TIP program.  What is the next step? The only way to know is to wait and see.

I’m not proposing that we do away with validation entirely, but instead that we move into a hybrid approach towards validation that is based on risk, maturity, pas performance, and future commitment.  The market has spoken and the Council and payment brands are already responding.

My suggestions for you?

If you fall into category 1, 2, or 4 above – prepare the following:

• Develop a “heat map” of your internal risks, data types, and exposures
• Adopt a risk management strategy, however rudimentary or advanced
• Attend the PCI SSC ISA training ASAP

If you fall into category 3 above – investigate the following:

• Are you using a PCI validated payment software and hardware?
• Are you using PCI DSS validated service providers?
• Are you area of technologies that reduce the need for PCI compliance such as: tokenization and end-to-end encryption?

The Placebo Effect

Irving Kirsch, psychologist at the University of Connecticut, and Guy Sapirstein did several experiments on the effectiveness of the placebo effect.

[Irving Kirsch] and Guy Sapirstein analyzed 19 clinical trials of antidepressants and concluded that the expectation of improvement, not adjustments in brain chemistry, accounted for 75 percent of the drugs’ effectiveness (Kirsch 1998).  ”The critical factor,” says Kirsch, “is our beliefs about what’s going to happen to us. You don’t have to rely on drugs to see profound transformation.” In an earlier study, Sapirstein analyzed 39 studies, done between 1974 and 1995, of depressed patients treated with drugs, psychotherapy, or a combination of both. He found that 50 percent of the drug effect is due to the placebo response.

The problem is that “all placebo effects eventually wear off, thus making the placebo effect impractical for long term or chronic medical matters.”

Fear, Uncertainty, Doubt

In the same way, the information security industry, and arguably the nation-state at large, regularly uses fear, uncertainty, and doubt – or F.U.D. – as a method of enticing people to take certain actions/reactions.  We take this easy way out because it’s a lot easier to tell a scary story than to explain the complexities of reality.

A large data breach happens and it cases a state of fear that, for a short term, triggers a fight-or-flight response.  Some people will use this to reign in new regulation, laws, or increased spending.  We saw this in response to 9-11 and we see it every day in businesses.

The problem with this method, aside from the ethical issues with its use, is that, like the placebo effect, it eventually wears off and thus is ineffective for long term use.  At which point, you either need to reinforce the fear, which typically leads to acceptance (sometimes in the form of cynicism), or you need to replace the placebo of fear with facts.

P.T.S.D. and Data Breaches

Cognitive behavioral therapy is a well known and accepted method for dealing with post traumatic stress disorder (PTSD).  It works by slowly and gradually exposing the individual to a feared state in a safe and reassuring manner.  The old memories are not erased but the new memories are additive in providing a more positive association with the memory experience.  Reinforced FUD takes that same method but drives us in a regression path.  Instead of moving beyond the fear it reinforces it further driving it inward and eventually preventing the subject from functioning (rationally) all together.

Only reinforced facts about a situation can help enable individuals with the self confidence they need to survive potentially negative situations (such as a data breach) and move beyond them instead of reacting negatively to them.  Once armed with knowledge you can make rational decisions based on evidence rather than emotion and knee-jerk responses.

Equipped with knowledge and well reasoned data enables us to plan and prepare rather than always existing in a reactive state.

Measuring Risk

One way to arm ourselves with confidence is to measure the risk in a system so our response to securing it can be made in a planned manner.  When people discuss measuring risk there are a number of items that come to mind.  It is important to remember that we are not trying to measure technical risk, though that is one part of the equation.  We want to measure financial risk.  By measuring the financial risk that a system, department, or enterprise  exposes us to we can calculate and plan a method of securing the data.  This plan should take into account the financial liability or loss we are trying to avoid or mitigate.

This method differs from others in that it does not attempt to calculate the cost of a data loss per record, as that could vary based on the exposure in a system.  It does not attempt to calculate the technical risk of a system or department because that could have no direct correlation on the financial losses.  It does not attempt to calculate the value of the money spent, as without a threshold for success (or associated data breach) there is no way to optimize this measurement. The focus is entirely on the overall risk associate with data loss based on legal, regulatory, and operational costs.

Presently, we each need to create this calculation and thus reinvent the wheel for every environment, but why?

The risk of exposure should be accessible in data breach reports.  The cost of financial fines and/or penalties is publicly listed by the FTC and payment card brands.  The cost of state data-breach-notification costs is generally accepted within a range.  We know data breach statistics by industry and type of business.

Why can’t someone model this data in such a way that each organization can enter in their environmental attributes, adjust the risk levels as per their individual thresholds, and have it calculate a financial risk or exposure of each system, department, or enterprise?

It’s the future and it’s happening faster than we think.

I am a big proponent of risk management and risk-based security.  I also work (mainly) in a very specific, yet large, segment of information security that pertains to the payment card industry (PCI).  Since I’ve been involved in this space for a long time I sometimes suffer from the curse of knowledge.  This helps when analyzing information and determining which is valuable and which is not.

Two weeks ago at Mini-Metricon, Pete Lindstrom said, “we have solved the problem of information security over 200 times, the problem is we don’t know which one is right.”  He went on to explain that different people are experts in their own domain.  The curse of knowledge hits me in that of all the information available in the payment card industry, I know which is useful to me and which should be discarded or is more applicable to another individual.  I do this without thinking and as a result my mental concept of risk management is shifted from that of others in the general public.  My network includes a strong background in the PCI industry of over 6 years and the opportunity to work closely with many smart people including Alex Hutton, Adam Shostack, Branden Williams, Walt Conway, Paul Guthrie, Andrew Jamieson, Anton Chuvakin, Lucas Zaichkowski, Martin McKeay, and many many other industry experts.  Having access to this holistic source of information provides me a wealth of information that others may simply not have.  (It also helps that my job involves QA and I end up reading hundreds of reports or case studies every year.)  Also, it’s not a point in time, but I call upon these individuals all the time to help shape and crystallize my understanding of the ever changing landscape of risk.

Two years ago when I met up with Adam Shostack at RSA and as we talked about the industry he explained to me that what we need as an industry is more data in order to form proper conclusions.  The main idea being that the more data you have on a specific topic the more easily you, and everyone else, can make a rationale decision about how to best protect it.  The problem with the lack of data is the ability to trust the limited data and conclusions you want so very much to rely on.

This is why when I finally met up with Donn Parker I asked him to explain his concept of diligence-base security vs risk-based security [PDF].  In a nutshell, Donn explained that risk-based approaches are nothing more than data alchemy as there is simply not enough public data available to make any sort of statistically significant conclusion when you assume that the entire population of data breaches or security failures (realistically unknown) is vastly larger.  Indeed it is very difficult to measure and make statistical decisions about the unknown-unknowns.

The example I like to reference is that of scanning for rogue devices (i.e. wireless access points) on a computer network.  Detecting rogue devices (unknowns) is very different than examining known devices, and logic breaks down when trying to apply traditional sampling methods to this unknown landscape.  Traditionally, sampling of a population is done when the population is uniform, or in some way known.  In general, the more uniform the population the smaller the sample size may be to determine a statistical conclusion.  The problem with rogue devices is that the population is unknown.  If you try to sample from an ever changing population the results you get at any point in time may be statically non-reflective of the total population.

Mr. Parker advocates that since we do not have a population of data breaches significant to the total number, and since the total number and type are ever changing, there is no scientific way to apply risk-based controls.  Instead he advocates a diligence-based approach towards security.  Since we cannot measure and thus appropriately apply risk-based metrics we should take the agreed upon “best practice” controls we have and be diligent about their application and maintenance.

Arguably, one could take the same cynical approach towards the traditional baseline “best practice” baselines such as BS7799, ISO 27001, ISO 27005 (for that matter the entire ISO 27000 series), or even HIPAA (HHS guidelines), or GLBA (FFIEC guidelines).  How do we know that these are sound practices upon which we should build an information security program?  With technologies changing and evolving over time there are many different ways to envision security.

So if we cannot base our foundation on best practices, and we cannot apply risk-based controls, what then is left?  This is where I propose holistic information security.  The diligence method is based on factors such as budget, management directives, staff talent and availability, and organizational policies.  Although this sounds right from a business perspective, following these methods provides a ‘good enough for the current business’ which may or may not be the best direction for the business to protect itself.  Arguably, one cannot know what the best direction is for the business due to lack of data.  See also, chicken-and-the-egg.

I’ve watched over the years as analysts, experts, and individuals claim to have the correct answer, when in fact all they have is their one piece of the pie of truth.  Instead, I advocate taking a holistic approach towards security and assimilating as much data as you can before making a decision.  Talk with as many stake holders as possible so you can elevate your level of knowledge about your industry from amateur to expert.  Only by reviewing others’ piece of the pie can you approach seeing the bigger picture.  In fact, Donn Parker advocates this in his ISSA Journal 2008 paper by proposing that practitioners of the art of information security seek out other sources of information from other organizations of comparable size, type, structure, and threat exposure.

If we are actually dealing with an unknown-unknown that we cannot measure or (honestly) see the entirety of, then we are left with only one option.  The only option left is to assimilate as much of the whole as we can.  The goal should be to “seek first to understand and then to be understood“.  This approach enables us to make more informed decisions about what is valuable information and what is fodder.

Update: I also highly recommend you watch Alex Hutton’s Security B-Sides talk on, Risk Management – Time to blow it up and start over? [slides]

I don’t think many people have ever asked themselves what regulatory compliance has in common with immunization, but they should.  The fact of the matter is that these two have more in common than you think and understanding one will help you better understand the other and how to make better educated decisions.  In addition, there are trade-offs — both heath and economic — to the choices one makes in participating in vaccination and immunization programs.  The following addresses a few of these items and opens the doors for further conversation.

Why Comply? Why Vaccinate?

Immunization and vaccination are the process by which an individual or population is treated in order to fortify itself against attack from foreign bodies.  Vaccination against disease can help prevent contracting that pathogen in the future, and preventing multiple individuals in a population from becoming infected helps prevent the widespread outbreak and transmission of diseases such as smallpox, polio, measles, mumps, and anthrax.  By elevating the level of a population that is resistant to such attacks vaccines help protect the entire population from harm.

The problem is that although most all agree that vaccination is positive for the population not everyone agrees that it is positive for the individual.

Since vaccination began in the late 18th century, opponents have claimed that vaccines do not work, that they are or may be dangerous, that individuals should rely on personal hygiene instead, or that mandatory vaccinations violate individual rights or religious principles.

Have we not heard similar arguments against regulatory compliance?  Individuals stating that:

• My environment is already secure
• I know how to manage risk better than the regulatory bodies
• My environment is special and unique and does not fit into your Procrustean boxes

I’ve listened to people sing the virtues of regulatory compliance as often as I’ve heard other individual tell me “that sounds good but it’s not for me.”  I feel as if I’m mediating between the Center for Disease Control (CDC) and a troubled parent about why their child should be vaccinated before entering grade school.

Perspective

Part I

One of the problems with understanding the complexity of the problem is that of perspective.  The CDC and the parent have very different perspectives on vaccines and immunization.  In the same way, the regulatory bodies and those who must comply with them have very different views on how to best apply data security practices.

For example, it is widely known by the payment card industry (PCI) that the majority of small and medium merchants use one of a few brands of payment application.  Many retail merchants use a Micros, VeriFone, or Radiant Aloha (restaurants) point of sale (POS) application.  This high level of homogeneity in a population lends itself to attract attackers (pathogens) who wish to take advantage of any vulnerabilities they can identify in these systems.

The PCI Council, who act as the CDC, along with the card brands mandate that software companies validate their applications against a given security standard (in this case the PA-DSS).  They then introduce these more secure applications into the population and the governing bodies mandate their use over less secure payment applications.

So why not just stop there?  If things were that easy, the CDC would only ever have to worry about one pathogen using one attack vector.  If we secure the retail payment applications, attackers will just move to other industries such as petrol (gas) stations, ski resorts, and florist shops.  To which the industry responds with Dresser Wayne or Gilbarco, SKIDATA, and Teleflora Dove validated payment applications respectively.  The validated payment application program targets to inoculate every industry against the dangers of retaining data most valuable to attackers.

Part II

But what about the individual restaurant owner who says they don’t need a validated payment application?  They claim all the reasons mentioned above from the specialized nature of their business or network to the secure risk management platform they have already implemented.  Why should they comply?

I do not have a good answer to the ‘why’ but I do have one for the ‘how’.  In fact, about 95% of the ‘PCI Wars’ debate going on today try to answer the question of “why” when this is as futile as debating intelligent design vs evolution (because both are based on separate and unequal premises.)  Debating why one should comply is futile as the rules state that everyone who “stores, process or transmits” such data must comply (as per the card brand operating regulations.)

The more interesting question is that of how one should comply.  These examples reference the PCI standards but could apply to just about any regulatory compliance mandate.  The way in which one complies can be taken at a high level.  For the PCI standards it implies preventing the paper and electronic theft of payment card data.  In fact, any way that your company decides to do this implies compliance with the standard.

If parents didn’t mind sending their children to school in hermetically sealed bubbles, then there would be less of a public policy need for them to be vaccinate against disease.  In this way, the parent and child could make their own decision about data security without harming or posing a risk to the rest of the population of school children and their parents.  If your company can, via whatever means at your disposal, hermetically seal itself against attacks then the matter of compliance is simply an exercise for the user in creative documentation, reporting, and compensating controls.  The problem is, many companies over estimate their security controls and thus cause a break in the structure of data security.

Economics of Immunization and Compliance

When approaching the economics of immunization one cannot ignore the population at hand.  For example, a poorer population will benefit more strongly from an immunization program than one that maintains a high level of sanitation, health care, and treatment programs.  To the same degree a more vulnerable population (e.g. retail, restaurants, higher education, e-commerce, etc.) will benefit more from regulatory compliance than one that is more highly secure (e.g. government systems).

In fact, one of the primary catalysts for regulatory compliance is the build up of problems (e.g. data breaches) within an industry followed by the punctuated equilibrium that brings about a response founded in legislative and regulatory action.

The cost of making a population more secure is relatively simple: require them to use more secure applications and systems.  The cost to the individual can vary along with the benefits.  The same applies to vaccines.

One could go their entire professional life without contracting the flu but this is rather rare in my experience.  Instead many people will get the flu vaccine each year on the off chance they will come in contact with the virus because being bed ridden for 1-2 weeks can be both painful and detrimental to the company.

So what!

The cause of action to vaccinate a population is to immunize them from each other.  The process involves a uniform across the board preemptive treatment that is meant to mitigate risks, not prevent them entirely.  In the same way, regulatory bodies craft legislation as a one-size-fits all in order to protect the population from each other.  The individual implementation should see this as guidance and not a rule without exceptions.

The details of how one protects themselves against attack and infection may be unique to each individual, but they still must comply with the overarching industry agreement to protect themselves and thus the population against attacks.  The implementation will vary, of course it will.  One size does not fit all.  But the industry needs a standard, a baseline, against which it can measure risk.  As new infections and outbreaks occur, the industry will change the baseline to match the new attacks.

Those who can visualize the various perspectives will have a greater visibility into how they can better fortify their individual organizations to both validate against industry mandates and manage risk based on their specific organizational behavior.

Recently Nick Selby posted on FudSec his article on Showing the Oblomovs the Door.  For those who care, an Oblomov or Oblomovism is considered a lazy or apathetic person or belief.  The blog post claims that information security professionals are “well-trained, well-intentioned” but “reduced [to] a series of relentless box-ticking” due to being “saddled with compliance management.”

The blog post further claims:

The CEO who lets the Security organization become the compliance department has abdicated to the government and Payment Card Industry his responsibility to understand and manage organizational risk. That is a fiduciary breach of CEO responsibility to shareholders. In addition to firing your ass, this should also be a floggable offense.

I agree one should use compliance as a guideline but manage it with respect to the business process.  I disagree with the fiduciary statement on grounds that one cannot claim a breach based on sparse case study and singularity statements.  The writer says this to bring grandeur to their claim.

The important part of this statement is that we are focused on the individual company here and their personal responsibility.  Remember, if you ever want to get something done don’t pass the buck.

The author, frustrated with the current implementation of compliance, states, “I stomped away from trying to influence security as an analyst because compliance … has managed to suck every ounce of oxygen from the room that is the security industry.”

Let’s just remember that history has shown that in the absence of legislation there exists a downward spiral of corporate responsibility towards protection of customer/consumer information and the well being of others. To support this I point to the moments of punctuated equilibrium that lead to things such as the Food and Drug Administration (FDA), the Securities and Exchange Commission (SEC), marginally improved ecological laws in China, and the current global financial crisis — to name a few.

Let’s also take a moment to remember that regulatory compliance has been raising the bar of information security since 1999, starting with GLBA, then with HIPAA and SOX, and finally with PCI DSS.  Is it because PCI DSS impacts most all business verticals on a global basis that it receives the most abuse from those who feel burned out?

Might I remind you that without such efforts the number of data breaches would be higher, much higher, than we see now because people find it easier to blame someone or something else rather than take personal responsibility for their own work.  The Information Security Management Handbook, by Tipton and Krause, has a section on diffusion of responsibility.

People behave differently based on the perception of being part of a group as opposed to being an individual.  It has been commonly observed that people tend to work less in a group than as individuals when only group output is measured.  People, in addition, tend to feel less responsibility in a group than as a single individual.  The bigger the group, the lower the felt sense of responsibility. Social scientists call this diffusion of responsibility and the phenomenon is commonly observed across all cultures.

I believe that instead of blaming others, we as information security professionals need to become an agent of change starting with ourselves and our current environment and expanding outwards.

The blog then claims:

At this writing it’s unclear whether Black Hat and DefCon demonstrations will include the PCI-compliant account skimmers we’re heard of, but the fact that they’re out there stands testament to the Pyrrhic victory that is the PCI Data Security Standard.

Please remember, the PCI DSS is meant to protect against the electronic and paper theft of payment card data.  It is not meant in any way to prevent credit card skimming. If you wish to raise the issue of skimming, please use the correct approach which is to clarify the need for a more secure payment card.  That of course gives way to the larger question of what is proper capital allocation and the conundrum of offline transactions and backwards compatibility.

I agree, sadly, with the blog post when it says, “PCI is not the minimum standard, it’s the maximum effort that many organizations make.” The question I have is, based on historical precedent (see above): are we better off with or without a carrot-and-stick approach? What impact has HIPAA had on the security of health care records vs PCI on the payment card industry?  In which area do we see more movement?

Certainly, movement does not always imply movement in the correct direction, but I would claim that basic items such as PCI DSS Requirement 3.2 which tells merchants and service providers to not store sensitive authentication date post-authorization has done wonders to the security of our payment card data.  How better to secure the data than to remove it in the first place?  We are seeing trends in this direction more and more in this industry and others.

But isn’t it better to have a minimum standard than none?  What if the minimum was for companies to do nothing?

Jeremiah Grossman stated, “nothing did more to build webappsec awareness than pci-dss. Now we need something to improve webappsec security.” I could not agree more, but let’s please remember that without awareness of a problem you cannot bring clarity or correction. People love to lambaste and transfer responsibility to others, all the while stomping away from personal responsibility.

If your company or those around you fail to see the forest through the trees of ‘industry best practices’ when I wonder if they are fit to run the information security department.  Those who complain that ‘compliance’ is the problem are transferring responsibility to industry standards instead of working to secure their own infrastructure.

Do such standards need correction and evolution to mirror the evolving threat of attackers and the continued evolution of information security practices and technology?  Certainly!  I support Mr. Selby in his goal to drive higher standards and move towards risk management, but let’s do so by taking individual responsibility for our own management of risk.

Mr. Selby claims, “all this compliance stuff is preventing us from addressing risk and performing, you know, security.” Why?  Did someone tell you that you cannot secure your data? Did someone tell you that by using proper the proper risk management practices you claim work so well that you cannot pass the “minimum standard”?  I support you in questioning and ferreting out anyone who makes such statements.  For the rest of the unwashed masses, we need standards.

Mr. Selby ends his rant with a statement everyone should agree with, “Compliance – the state of being – is achieved as a by-product of well-managed risk, not through a relentless ticking of boxes”, which is then followed by high-level statements of positive thinking.  The problem is that we need some tactical examples and guidelines to match the ever increasingly vague strategic statements.  GLBA says to safeguard customer information, but how?  And left to their own devices most companies will chose the cheapest possible way to implement optics of compliance.

I argue, that the PCI DSS has given concrete statements to how one secure their infrastructure, while giving the flexibility one needs to adjust for business and risk management (e.g., compensating controls, wireless and end-t0-end encryption guidelines.)

The problem lies not with our industry “best practices” but with the diffusion of responsibility that happens throughout every company.  Let’s reference back to that Information Security Management Handbook article:

The effects of de-individualization and individualization are real and play a role in how users perceive their role in an information security awareness program.  In the credit card processing call center example, de-individualization can encourage theft, carelessness, and loss of productivity.

I’d like to stop the blame game and see everyone start at home, transforming their company and being neighborly enough to share the information and results with others.  Revolution has often come from emerging evolution of ideas and conversations. I commend Mr. Selby for the conversation, but wish it involved a greater focus on personal responsibility.

Take responsibility for your own security, risk management, and data protection. Start today.

When a situation is not risky, there is little need to manage or measure the risk involved.  This applies equally to lending money to friends, reading utility meters, and until a few years ago, handling credit card transactions.  With the growing risk to financial transactions there is a need to improve the ways acquiring banks, processors, gateways, and even merchants manage and measure their risk.

In fact, prior to the PCI DSS the metrics involved in measuring the risk in an acquiring bank’s merchant portfolio were rather basic.  You look at the number of transactions per month and categorize the merchants into business categories.  One would say that online gambling would be riskier than grocery stores.  The logic seemed flawless, at least for the environment at the time.

Unfortunately the environment changed and hackers turned from fame and glory seekers to those wanting large financial payoffs from their prowess.  They began attacking merchants and even banks by finding the low-hanging-fruit never imagined by the industry.  The hackers began targeting:

• POS systems directly connected to the Internet with weak remote access methods
• Weak or insecure wireless located at physical stores
• Insecurities in partner and vendor connections to companies
• Insecure web application software

The hackers identified, by brute force, holes in the security of an industry that were never imagined by those creating the metric for managing their risk.  When the compromises reached a tipping point, the industry began to shift the focus to security.  The banks and card brands formed the PCI Security Standards Council (PCI SSC or Council) in 2006 and invited merchants, POS vendors, and other industry experts to participate (as Participating Organizations.)

In order to realize the importance of this change you have to first understand that people do things based on incentives, and for most companies security is not something they are very willing spend money on.  This can be seen in the ever increasing number of data breaches that occur every day.  Anyone who has had teenagers can tell you that in order to “encourage” a person to do the right thing you need to properly incent them.  Regulatory compliance has been the thing that has incented companies to place an importance on information security for the last 10 years, and PCI DSS compliance has been the leading force for the last three years.

This might pass over as just another regulatory issue like GLBA or SOX if not for the fact that it’s not specific to a business vertical.  The payments industry is sometimes called a “horizontal” because it cuts across so many areas such as banking an finance, travel and entertainment, health care, power and energy, etc.  In fact PCI DSS is the first globally enforced, industry regulated, cross-industry compliance program.  It’s goal is simple: prevent the electronic and paper theft of payment card data.

But why?  Don’t you like it when we ask why?  The reason was not to do this because it’s the right thing.  We all like to say we are acting “green” by composting when really it’s just a way of reducing the cost of our garbage bill.  We want to prevent the loss of this data because someone is paying for the fraud: other merchants, acquiring banks, and many more.

So, we begin to understand that acquiring banks use the binary aspect of PCI DSS compliance as a measuring stick to determine the risk within their merchant portfolio.  Sure there are still the number of transactions, type of business, and size of the organization, but now there is the check box of security.  What does this really mean in practical terms?

Well, of all the PCI DSS requirements, the most important by all accounts is 3.2 which mandates that sensitive authentication data should not be retained post-authorization.  The other requirements for security act to protect this data from being intercepted in the first place.  The end result is that hackers should never have any access to this sensitive authentication data.

OK, so the standard exists to protect the super secret data so banks can measure how risky their merchants are to them.  This acts much like the credit rating agencies such as Moody’s and Standard & Poors.  The question is, “Are the current  metrics sufficient for measuring risk in a merchant portfolio?”

I would argue that, much like the credit rating that says “AAA”, the PCI DSS is only one part of the holistic approach merchant banks should take to measuring risk within their portfolio.  Think back to the collateralized debt obligation (CDO) market that just exploded.  People were packaging mortgages together into one security that people could then trade against.  The problem is understanding the impact that all those thousands of mortgages and the people behind them will have on the value of that one security.

In a similar way, banks need to look at PCI DSS as just one factor in analyzing the risk to their portfolio.  I’d argue that to keep the model working one should look to the Verizon Data Breach report and analyze attack vectors to determine what areas should be measured.

If we look at the data breach landscape we see the following numbers:

• 74% resulted from external sources
• 20% were caused by insiders
• 32% implicated business partners
• 39% involved multiple parties

The metrics companies and banks should use for measuring risk should include:

• Third Parties and the data they share (all three types)
• Deployment of a wireless network (proximity to acceptance channels such as POS)
• Number and size of business processes (POS network, databases, applications)
• Connected business units (call center, data warehouse, or physically insecure locations)

Individual merchants need to prioritize attack vectors.  If we know that more hacking events occur due to weak passwords or default passwords we should focus on eliminating things like “<blank>” or “password” or “<vendor name>” rather than focusing on achieving 7 character, alpha-numeric ones (which for the record are no better than 5 character ones in theory.)

I argue that we need more focus on attack vector trending threat models for regulatory compliance before we focus on the broad spectrum of security best practices.

Many people debate the efficacy, effectiveness, and overall utility of the Payment Card Industry Data Security Standard (PCI DSS).  Some people involved in this debate suffer from a bounded rationality, wherein their rationality is bounded by the few articles they read online, their perspective as a merchant, or their view as an information security professional.

I’d like to outline the good, the bad, and the ugly about the PCI DSS.  I do this not to condemn or defend any one party but instead to raise the level of conversation and debate beyond some of the fallacy ridden discussions we have been having up until now.

The Good

One thing that people do not realize is that the Payment Card Industry is unlike any other.  The reason we all talk about it is because it is more of a “horizontal” than it is a “vertical” industry.  Instead of talking about the banking & finance, agriculture, or even retail industry we are talking about just about every company that utilizes credit cards in some way.  It is the case that regulatory compliance has driven the dollars behind information security since 2001, and the greatest motivator in the past few years has been the PCI DSS.  If we like it or not, this standard has been the carrot (and stick) necessary to making companies care about securing your payment card data.

Until the PCI DSS was created (and the CISP/SDP before it) there was little to no standardized way for acquiring banks to measure the risk present in their merchant population. Sure, they could have reviewed varying security reports but none of these went to the heart of the risk matter by eliminating the retention of sensitive authentication data.  In fact, it was not until the PCI mandates for not storing such data that we saw real change in the payments industry.

Thought the original PCI DSS compliance deadlines were September 30, 2004, it was not until the Visa Compliance Acceleration Program (CAP) in 2007 that substantial movement occurred.  The CAP program provided the motivation necessary to make merchants validate they were not retaining sensitive authentication data.

Information security writer and reporter, George V Hulme, gets it and says, “I still contend, PCI DSS has done much to raise merchant security from the dismal state it was in — to the better state it is in today.”  Things are certainly getting better.

The PCI DSS has given acquiring banks, merchants and service providers a method of measuring their exposure to electronic and paper data compromise, the most important of which is keeping hackers from accessing the payment card data.  This is something that no other security standard specifically calls out.  (There exist many other information security programs but none define industry terms such as “sensitive authentication data” or “cardholder data”.)

So where would we be without the specific PCI mandate to eliminate sensitive authentication data and protect the remaining cardholder data?  Well, I argue we would have even more data breaches than we see today.  In the most recent (3.31.09) Visa Inc statistics, we see that within the U.S. all of the top two merchant levels have either validated compliance or are in the process of remediation.  This is a big change from even 3-4 years ago where many merchants had only begun to protect their payment card data.

Additionally, there has been significant work done to help the medium and smaller merchant levels.  Visa spearheaded the PA-DSS program, formerly the PABP, in an effort to provide secure payment applications to merchants who may not otherwise care about security or compliance.  If we follow the 80/20 rule, it is easy to imagine that 80% of the small (Level 4) merchants use the top 20% of payment applications/terminals.  If the industry can verify these payment terminals are secure they can help reduce the risk of data loss for 80% of the small merchant market.

The Bad

The PCI SSC has put in place a structured 2-year cycle for updating and improving the PCI standards.  These changes hope to move the standard in the direction of protecting data compromises based on evolving attack and threat patterns.  In the move from v1.1 to v1.2 we saw the addition of Requirement 6.6 in direct response to the rise in the number of web application attacks.  These are very positive moves, but I would not say everything is roses.

The PCI DSS references the need for a “risk based” approach but it is buried deep in Requirement 12.1.2, which states, “[security policies must] include an annual process that identifies threats and vulnerabilities, and results in a formal risk assessment.”  I like that it is here, but I think this sentence should be the overarching measure by which all requirements are addressed.

There have been many positives moves in this direction.  In April 2008, Bank of America released a document titled Beyond Minimum Compliance: PCI Risk Management.  Though not a risk management framework, this document outlined some Top 10 PCI Best Practices that included: becoming compliant, using secure payment applications, reducing the scope of data, validate compliance, and maintain a state of security throughout the year.  In 2009, the PCI Security Standards Council released a Prioritized Approach to Pursue PCI DSS Compliance.  This takes the PCI SAP and prioritizes each requirement based on, what I assume is, that which may prevent the exposure of payment card data to the greatest number of attacks.

The executive summary of the PCI Security Audit Procedures (SAP) should clearly state the need for a risk management methodology and approach.  Moreover, it should designate a call for action that companies create and maintain a capability and maturity model (CMM) to track their progress in reducing risk.

I’ve found that one of the major roadblocks to compliance adoption is the confusion about to what degree each requirement should be implemented.  These fears could be lessened if the risk management dialogue happened at the beginning of the standard.  Similar approaches have been seen in COBIT and even the ISO27001 (with the ISO27005 framework) to name a few.

The Ugly

What I find ugly about the industry is the continual flame wars that erupt around the standard itself.  I find that many, if not all of these, have to do with the improper implementation of the standard, or the lack of a deeper insight to the industry and the history of regulatory compliance.

Anyone who has worked in a cross section of regulatory compliance arenas can tell you that there is a major difference between government regulated industries and self-regulated industries.  With the exception of NAR’s REALTOR® Secure Program, I don’t know any other industry self-regulation that exists other than the PCI DSS.  (Branden Williams, Alex Hutton and David Mortman remind me of the HiTRUST program for the health care industry.)  This is a welcome anomaly in the world of government mandates.  Those who think PCI DSS is hard to comply with may not have dealt with GLBA or SOX.  With the GLB Act, if the Fed gives you a low rating (higher number) on your bank, they don’t fine you, they just shut you down.  The payments industry is trying to enable commerce in a more secure way than it was done before.

A favorite topic of Dan Geer is that of punctuated equilibrium.  The historical context of punctuated equilibrium has shown that without incremental action, a buildup of force can result in great reactionary action.  The simplest example of this is that of a volcano or earthquake.  In public policy and the payments industry this buildup has been noted several times.  We can see this historically where the government has intervened after the industry itself could not properly self-regulate.  The Great Depression gave rise to the Securities and Exchange Commission (SEC), the once poor food standards gave rise to the Food and Drug Administration (FDA), and we are now asking if the lack of personal data privacy will give way to a government managed compliance standard.  I argue that self-regulation with continual improvement will make faster strides towards data protection than will omnibus legislation.

People complain that the 2-year life cycle of the PCI DSS is too slow, but they must not be that familiar with the 4-year life cycle for other government mandates.  Also, changing a standard any faster would cause merchants’ heads to spin!  Though the payments industry is trying to self-regulate, this never seems to be enough for some people.

The “ugly” side of the industry is that which everyone becomes a pundit, spinning the story to meet their specific pain.  People say that PCI DSS is all about “risk transference” which is a flat out fallacy of belief.  The truth is that Issuing banks have always picked up the tab for fraud, and have long been able to recover that money from merchants via the various card brand issuer reimbursement programs (i.e. Visa ADCR).  The only thing the PCI DSS ever did was give acquiring banks (and all organizations handling payment card data) a measure against which to protect the payment card data.

This angry snarl of punditry is really the sad conclusion to our state of affairs.  I fully support those who wish to improve the industry but many times people feel that change should be revolutionary and not evolutionary.  They are quick to call your baby ugly and then tell you how to parent your child.  Everyone has their own input, and I just wish people would be more constructive with their comments, feedback, and advice.  We are listening!  I am listening!

As a final note, I’d like to remind the implementers of the PCI standards (merchants and service providers) to take responsibility for their own actions.  The PCI DSS should not be used as a stand alone tool.  Companies need to wrap the DSS in a comprehensive risk management program that is measured through time by a proper capability and maturity model.  Only by staying vigilant and continuously reevaluating our current security posture can we properly protect against the ever changing attack vectors that confront us.

Additional Reading

I also highly recommend PCI Shrugged: Debunking Criticisms of PCI DSS.  Please suggest others in the comments.

Update: It should be noted that I am a believer in risk management, especially quantitative risk management, but simply want to highlight some of the effects that bounded rationality has on our ability to manage risk.  I want to push us towards a more optimized view of rationality and risk management.

When we think of how to protect our most sensitive data we have one of two approaches.  Security is a tactical approach and risk management is a strategic approach.  Security implies the implementation of sound risk management practices.  While technical people like to talk about ‘vulns’ it is the risk management people who wax philosophically about long term strategy, data centric vs system centric approaches, and drink from the fountain of Utopian kool-aid.

I too have paid my dues and talked about risk management in its perfect form.  This approach involves metrics, models, threat vectors, CIA triad, and a multitude of other factors.  Risk management was married long ago to the maiden of Capability and Maturity Models for long term vitality.  Combined, these two go hand in hand to protect data from the foes.  Or so the story goes.

Now, I’m not about to become a risk management heretic, but as Mahan Khalsa says in his infamous sales books, “let’s get real!”  One thing that risk management does not (typically) take into account is that people, humans, are irrational beings.  When it comes to assessing risk, managing hazards, making decisions, managing a crisis, navigating office politics, and altering perceptions we have a roadblock called emotion.  Within emotion are all the factors that influence our decision making capabilities, such as: fear, uncertainty, doubt, misdirection, and oh so many more.

History has shown that human fear the small possibility of a quick immediate death much more than the larger possibility of a long term slow death.  The World Health Organization (WHO) reported that from 2003 – 2009 the total number of global deaths from Avian Fluwas 257.  That’s not enough to even be a statistical anomaly but we saw it on cover of just about every magazine and newspaper around the world for a few months. The WHO does not even rank influenza, of any sort, in the top 10 causes of deathby the WHO.  In fact, chronic heart disease killed 7.2 million people in 2004, and road traffic accidents killed 1.27 million people.  We worry more about contracting a rare form of the flu and dying than we do of driving to the grocery store on a Friday night.

Proper MetriCon people might say that numbers don’t lie or have emotions, but the question is, “how good are those numbers?”  I recall one year an analyst group put out a press release saying that it costs companies $200 per lost credit card.  The following year many vendor companies ran with that number and sold their product as costing only $100 per record to protect.  This could result in a 50% savings.  The problem came the following year when the analyst firm revised their numbers to say that it only cost companies $80 per lost credit card.  (Numbers have been rounded and changed to protect their creators.)

I have been sold on the need for more metrics in risk management and security, but the problem is we need to temper our reaction to data the same way we wait for Service Pack 2 before purchasing software.

We need to temper our risk management approach to one that accepts the hesitation of people to make precise and accurate decisions, especially if they are not satisfying an immediate need.  I’ve spoken with many PCI Qualified Security Assessor (QSA) companies and many agree that companies focus on satisfying compliance first and push off risk management for a later date, that sometimes never arrives.  The economics do not even need to matter as long as the immediate need is being satisfied.

People would rather spend more money now to satisfy compliance even if they could spend less over the long term to pave the road for a sound security strategy.  Why?  Well, there are many reasons but some of them include:

• High turnover
• Annual management based objectives (MBOs)
• Immediate need for “compliance”
• Lack of enterprise visibility
• Siloed departments/divisions
• Lack of information/education

It is the lack of awareness, information, and education that causes many companies to ignore the long term death and focus on the short term threat.  This can be like putting a band-aid on a bloody stump and calling it a mere “flesh wound“.

We need to accept that people are going to make irrational decisions and devise new and creative ways to re-educate them about the decisions they are making.  I think that better and better metrics are certainly a way to get there, but we are a long way from the panacea of payment security and risk metrics.