Chaordic Mind

Abstract

As regulation-deregulation cycles rise and fall, it is important to understand how the evolving landscape of compliance impacts your future. This post proposes maintaining compliance but making validation an opt-out optional component – a radical change from the status quo.  Evidence already suggests the industry is moving in this direction and changes to compliance are necessary for the continuance of risk management.

Please understand that when I say opt-out, I am referring to mandated external, third-party validation requirements. I think internal validation is more important than ever.

Special thanks to idea people: @lennyzeltser, @mckeay, @alexhutton, @kindervag, @joshcorman

Background

I recently read Lenny Zeltser’s blog titled “Could Regulatory Compliance Encourage Weaker Security?” This is a valid question and one that needs addressing. The question can be rephrased as, “Who does compliance work best for?” To answer that question we need to understand why compliance exists.

In a blog post I wrote on How Compliance Regulations Gets Made we focus on the natural regulation-deregulation cycles and how they exist in response to an increase or decrease in data breach/loss. The ultimate goal of compliance is to set a baseline of standards within an industry. The question Lenny raises is one I’m often asked by opponents of such standards, “what about the big/little guy (who do not fall within the Bell Curve norm for best practices)?”

It’s true that regulatory compliance is targeted not only at setting a minimum standard for technical security (firewalls and IDS) but also a minimum standard for security maturity (policies and procedures) within an organization. So let’s think about this graphically. There are four quadrants within which to place organizations: those with either high/low-level of security and high/low-level of maturity.

Security vs Maturity

For the purpose of this conversation let’s assume that maturity encompasses the Check and Act aspects of the PDCA Cycle and security refers to the Plan and Do components. The reason I break it down this way is to directly reflect the results of the Verizon PCI Compliance Report (PCIR). This report found that:

“Organizations are better at planning and doing than checking. If the check phase is broken, they cannot act to maintain the state of security over time.”

The Verizon PCIR found that organizations are great at Planning and Doing but not great at Checking and, as a direct result, Acting on those changes. To me this disconnect is the difference between organizations with a high-level vs low-level of maturity within their security practice.

With this in mind, let me suggest that regulatory compliance standards should most impact those organizations with a lack of either security or maturity, but not both. So let’s break this down and the types of organizations they embody.

1. High-Security / Low-Maturity: These companies care about security but have never documented policies and procedures. They have log management systems but have slowly stopped reviewing them. Regulatory compliance can have a positive impact here.
2. Low-Security / High-Maturity: These or organizations run well but with little funding for sorely needed security projects. There has never been a “hammer” to drive spending. Regulatory compliance can have a positive impact here.
3. Low-Security / Low-Maturity: These are organizations that do not care about security or compliance. Perhaps they are too small (mom-and-pop companies) or those that will validate compliance but never maintain it through the year. There is no changing these companies and little that compliance can do for them. Validating compliance for them is a waste of time and money and since there is no driver to maintain a state of security.  (Instead new technologies such as tokenization, end-to-end encryption, and validated payment applications will have the highest impact here.)
4. High-Security / High-Maturity: These are companies at the top-tier of their breed. They don’t manage security, they manage risk! They adopt and implement custom risk management solutions based on careful analysis of data classification and impact analysis reports. These companies see regulatory compliance as a roadblock and implementing industry “best practices” as a deviation from their perfect path.

I propose that regulatory compliance will most help groups 1 and 2, but not groups 3 and 4.  (Unless you consider regulatory compliance the driving force for said technologies above, though I would argue data breaches and word of mouth have a higher impact here than compliance.)

Although I believe in the need for increased education, flexibility of controls, and more data for risk modeling – I’m going to save us a bit of time and skip to the chase.

• Companies in group 3, who do not care about compliance or security, will not change their tune by forcing them to validate compliance.  Instead the end result will most likely be in them checking a box and ending up in the 80% of companies (see: Verizon PCIR) that do not maintain their state of compliance.
• Companies in group 4, who care passionately about risk and security, need a reprise from continually validating against a standard that is built for the average individual. Although, the stated way to address this for PCI compliance is through documenting a set of Compensating Controls, what other options do we have out there? What other ways are there for such companies dealing with compliance validation?

Remember, the stated goal of regulatory compliance, taken from regulation-deregulation cycles, is to reduce the number of data breaches and data loss. In both groups 3 and 4, continual validating against a standard may, in my opinion, have little to no impact on the number of data breaches/loss. The reason is that group 3, though validating will not maintain that validation, and group 4, treat validation as an exercise in documentation.

Other Options

On February 6, 2011, Visa launched its Technology Innovation Plan (TIP) “to recognize and acknowledge merchants in Visa Inc. regions outside of the United States that have taken action to prevent counterfeit fraud by investing in EMV technology.” (Since Visa Europe is a franchise, the “outside the US” may only apply to Asia-Pacific and Latin-America & Caribbean, but it’s a bold change we should view as the tip of an iceberg.)

In essence, they are saying that organizations that have achieved the following, need not continue to validate their compliance against the PCI DSS standard:

• Implemented a sufficient level of controls so as to reduce fraud* (see: EMV)
• Validated their state of compliance once
• Have not suffered a data breach

* Yes, fraud is discernibly different from data breaches but one leads to the other and as a result are interconnected.

Wow, what an innovative approach. I’ve talked about the TIP program with industry insiders and they are mostly in agreement that we don’t know if this will result in positive or negative changes. I feel it will be a great success and here is why.

Opting Out of Validation (Not Compliance)

Presently companies that validate their state of compliance need to submit two things: a validation document (either a self-assessment questionnaire or a report on compliance) and an attestation of compliance (AOC) document. The AOC is nothing more than a memo that reiterates that organizations commitment to following the payment-brand rules for protecting payment card data.

I think organizations that choose to opt-out of compliance validation should still need to sign the Attestation on Compliance (AOC) to reaffirm their social contract and commitment to protecting payment card data. If they fail to achieve this within their, alleged, super-robust security and risk program then they deserve to undergo the same forensic review and financial implications that come with any other organization. If they instead achieve in protecting payment card data and are able to repel the wily-hacker then they should continue their reprieve from annual compliance validation (perhaps they can externally-validate every 2 or 3 years).

The reason I suggest this is because, and here’s the kicker, you cannot tell the difference between a PCI compliant organization and one that has let security and compliance lapse until they experience a data breach. Until that point, both organizations appear, from the outside, to be operating in the same manner.  (Sure, you can tell a difference internally, but so far very few organizations that achieve compliance once organically maintain it year-over-year.)

But Wait – It Already Exists

The PCI Council has already rolled out the Internal Security Assessor (ISA) program and MasterCard has begun listing this qualification as part their validation program requirements.

“Effective 30 June 2011, Level 1 merchants that choose to conduct an annual onsite assessment using an internal auditor must ensure that primary internal auditor staff engaged in validating PCI DSS compliance attend PCI SSC ISA Training and pass the associated accreditation program annually in order to continue to use internal auditors.”

(Ok, so Visa has not adopted the same stance and companies that store, process, or transmit payment card data for both brands must adhere to the minimum standard for both, but still it’s a change.  Also, the payment card brand validation guidelines are guidance for the acquiring banks who have the ability to manage their validation programs on a case-by-case basis.)

This means that many organizations (there are exceptions) who wish to opt-out of formal validation can do so leveraging their internal assessor team.

Conclusion

What we have is a directional movement towards, what I will call, selective deregulation. Step 1 is the PCI SSC ISA program.  Step 2 is the Visa TIP program.  What is the next step? The only way to know is to wait and see.

I’m not proposing that we do away with validation entirely, but instead that we move into a hybrid approach towards validation that is based on risk, maturity, pas performance, and future commitment.  The market has spoken and the Council and payment brands are already responding.

My suggestions for you?

If you fall into category 1, 2, or 4 above – prepare the following:

• Develop a “heat map” of your internal risks, data types, and exposures
• Adopt a risk management strategy, however rudimentary or advanced
• Attend the PCI SSC ISA training ASAP

If you fall into category 3 above – investigate the following:

• Are you using a PCI validated payment software and hardware?
• Are you using PCI DSS validated service providers?
• Are you area of technologies that reduce the need for PCI compliance such as: tokenization and end-to-end encryption?

When measuring compliance results the data available is sparse and the analysis ranges from those who love it to those who hate it. Regardless of your personal, political or analytical beliefs the question remains about the efficacy of the practice itself.

There are several approaches and conversations surrounding the aspect of regulatory compliance, but the question remains, “What leads to excellence in organizations and the security they implement?”

The conclusion of this post is simple: Excellence within an organization is not achieved by measuring the compliance of an organization but by measuring the compliance of individuals and employees.

So how do we come to such a conclusion? To start we identify the main points for and against regulatory compliance:

• Pro: It raises the minimum level of security
• Cons: It creates a glass ceiling that either (1) prevents proactive organizations from implementing better security or (2) encourages reactive organizations to never excel above a certain point

Regardless of your opinion, both arguments focus around the precise level at which compliance measures an organization. Isn’t that interesting? Both the pro (for) and con (against) opinions seem to claim the same thing. This would be a good thing if it encouraged security without limiting organizations.

Already, PCI compliance (credit card security) has been compared with the No Child Left Behind or Elementary and Secondary Education Act (ESEA) within the United States. If this program is broken, let’s identify why, dispel false truths and come to a conclusion of how to learn from its lessons.

The fact remains, “nationwide: only 6 percent of U.S. students perform at the advanced-proficiency level in math, a share that lags behind kids in some 30 other countries, from the United Kingdom to Taiwan.”

Davi Ottenheimer wrote a summary of an article in The Atlantica titled, “Your Child Left Behind”. It goes on to dispel many myths about the No Child Left Behind Act. Eric Hanushek has spent “40 years calmly butchering conventional wisdom on education” including the following:

• More money does not tend to lead to better results
• Smaller class sizes do not tend to improve learning
• No U.S. state does very well compared with other rich nations
• Even relatively privileged students do not compete favorably with average students in other well-off countries. (In Illinois, the percentage of kids with a college-educated parent who are highly skilled at math is lower than the percentage of such kids among all students in Iceland, France, Estonia, and Sweden.)

“Per student, we now spend more than all but three other countries—Luxembourg, Switzerland, and Norway—on elementary and secondary education. And the list of countries that spend the most, notably, has little in common with the outcomes that Hanushek and his colleagues put into rank order. (The same holds true on the state level, where New York, one of the highest-spending states—it topped the list at $17,000 per pupil in 2008—still comes in behind 15 other states and 30 countries on Hanushek’s list.

If money, class size, and affluence of individuals do not impact classroom performance then why do we think that money, attack landscape, and intelligence of security professionals will impact the security of organizations?

Davi writes, “The exception to this lazy approach is the state of Massachusetts, which has followed a path that found success in other countries. It has directly intervened and introduced compliance.” But what kind of compliance? Was it simple testing that measured student performance against standardized tests? Nope.

The Atlantic gives us a glimpse into the answer. “What did Massachusetts do? Well, nothing that many countries (and industries) didn’t do a long time ago. For example, Massachusetts made it harder to become a teacher, requiring newcomers to pass a basic literacy test before entering the classroom. (In the first year, more than a third of the new teachers failed the test.) The state also required students to pass a test before graduating from high school–a notion so heretical that it led to protests in which students burned state superintendent David Driscoll in effigy. To help tutor the kids who failed, the state moved money around to the places where it was needed most. “We had a system of standards and held people to it–adults and students,” Driscoll says.”

So what works?

“Massachusetts, in other words, began demanding meaningful outcomes from everyone in the school building. Obvious though it may seem, it’s an idea that remains sacrilegious in many U.S. schools, despite the clumsy advances of No Child Left Behind. Instead, we still fixate on inputs—such as how much money we are pouring into the system or how small our class sizes are—and wind up with little to show for it. Since the early 1970s, we’ve doubled the amount of money we spend per pupil nationwide, but our high-schoolers’ reading and math scores have barely budged.”

Problem Set

My personal opinion is to focus on a capability and maturity model (CMM) of security and making regulatory compliance a natural side effect rather than an end goal. Sounds academic; so where’s the beef?

The practical implementation of this is shown in the recent Verizon PCI Compliance Report wherein it showed that organizations fail at tasks that: require human intervention or reoccurring activity. Many organizations that focus on compliance as an end-goal, fail to validate or maintain security throughout the year. No shocker there, but how do we overcome the human side of security? Security professionals have been talking about addressing the end-user for a long time, but these are not end-user problems they are security-professional problems.

Can we just throw more money at the problem? Reduce the scope of compliance? Train our security staff? Well, studies of the U.S. education system show these methods to be ineffective since they do not encourage well funded security professionals to do things like review audit logs on a daily basis. Even the most automated of systems are often times ignored for a variety of reasons.

Conclusion

Excellence within an organization is not achieved by measuring the compliance of an organization but by measuring the compliance of individuals and employees.

Building maturity models for information security implies an ever increasing maturity and level of security. This helps break the proverbial “glass ceiling” of compliance by having the security of an organization grow in proportion to the ever evolving attack landscape. This is so much easier said than done.

Our ability to achieve this goal hinges on our ability to encourage individual participation. Encouraging individual security-professionals to take action towards this goal.

So how useful is regulatory compliance? I advocate that compliance is good, but only in measuring the security of an organization at a point in time.  We need something much more than this to achieve real security.  We need something that will encourage validation and maintenance of security.

Final <rant>

Better security will not come from automation (DLP, audit log aggregation, etc.)  Better security will not come from more intelligent tools.  Better security will come from a higher standard within organizations to focus on maintaining security.  This leads to a discussion of cross-training security-professionals on conversational business-speak and helping them build measurable, results-driven risk models… but that is for another day.

Last year’s list was such a hit, I decided to do it again. This year I took into consideration nominations from comments on Facebook, Twitter and the blog. All those suggestions, as well as my own, took a trip  through the gravitron accelerator and here are the results!

There are so many freaking awesome people. I try not to repeat people from year to year, and even then there are so many options. Long time friends and people I’m yet to meet. There should probably be a top 100 list!

My opinions are obviously biased by my network, projects, and perspective.  I encourage you to make your own list as well.  As always, feel free to disagree or add your own using the comments.

10. Michelle Schafer, MC Petermann, Katherine Nellums (our PR/Marketing trio)

Who would care about information security if it wasn’t widely distributed? These three women are the trio that connect with people, engage the conversation and raise the level of intrigue around the spurious conversations we create.  I know when they are in town a posse of infosec people are close-by conversing about the next big thing.

09. Genevieve “Banasidhe” Southwick

When you think of security, don’t forget the physical side.  Banasidhe is part time bouncer, part time physical security / facilities manager, and former Vinyl Vanna.  She ran security for most every major Security B-Sides event, and is the bouncer you don’t want to cross paths with – unless you’re an invited VIP.

08. Leigh Honeywell

Leigh is co-founder of HackLab.TO, a hacker community space with many events on hardware hacking.  She’s a mainstay at most security conferences and many times on stage discussing malware, hardware hacking, or women in infosec.

07. Chris Nickerson

Chris is co-organizer of Security B-Sides, former cast of Tiger Team TV, and all around good guy.  He throws massive parties in Las Vegas (BSidesLV), flies around the world to present at conferences, and maintains the international-man-of-mystery persona.  His opinionated, straight-talk is often grounded in experience both on the digital and physical side of the house.

06. Alexander Sotirov

Sotirov is a founder and organizer of the Pwnie awards and was on the program committee of the 2008 Workshop On Offensive Technologies (WOOT ’08) as well as Hackito Ergo Sum 2011.  In his spare time he bypasses memory protection and creates rogue certificate authority certificates.

05. Bill Brenner

Bill has been covering the information security industry and documenting its past, present, and future for over [classified-redacted] years.  He knows and writes about those who need to be known, and covers stories ranging from the technical to the social, but always staying on or ahead of the next big thing.  His openness about himself is one of the many reasons people open up to him to share their stories.

04. Gordon “Fyodor” Lyon

Gordon is an infosec luminary having written NMAP, a tool that many claim is the reason they got into the industry in the first place.  He’s incredibly open and willing to talk to anyone. He is one of the few computer geeks who is sharply dressed at all times.  If you don’t know him you are missing out.

03. Allison Miller

Think you know fraud? Think again! Ally is a key player on the front lines of preventing fraud.  She presents on risk/fraud across several continents.  Her work includes detecting automated crime malware, running her own band, and mentoring others in their career and passions.

02. Jennifer Granick

Although she recently moved to another firm, her work for the EFF has protected the digital rights of hackers and citizens alike.  I can’t say enough about the work that her and others at the EFF have done to prevent the abuse of our legal rights.

01. Andrew Hay

As they say in Hitchhiker’s Guide to the Galaxy, “Vell, Andrew’s just zis guy, you know?”  The Zaphod Beeblebrox reincarnated, Andrew made “D” the new “A” when it comes to “A-Listers”.  He brings humor and levity to every event.  He co-organized BSidesOntario2010.  He probably even got 4chan to rig this contest so he topped the list.  He’s so good, we still can’t count the hanging chads.

I am a big proponent of risk management and risk-based security.  I also work (mainly) in a very specific, yet large, segment of information security that pertains to the payment card industry (PCI).  Since I’ve been involved in this space for a long time I sometimes suffer from the curse of knowledge.  This helps when analyzing information and determining which is valuable and which is not.

Two weeks ago at Mini-Metricon, Pete Lindstrom said, “we have solved the problem of information security over 200 times, the problem is we don’t know which one is right.”  He went on to explain that different people are experts in their own domain.  The curse of knowledge hits me in that of all the information available in the payment card industry, I know which is useful to me and which should be discarded or is more applicable to another individual.  I do this without thinking and as a result my mental concept of risk management is shifted from that of others in the general public.  My network includes a strong background in the PCI industry of over 6 years and the opportunity to work closely with many smart people including Alex Hutton, Adam Shostack, Branden Williams, Walt Conway, Paul Guthrie, Andrew Jamieson, Anton Chuvakin, Lucas Zaichkowski, Martin McKeay, and many many other industry experts.  Having access to this holistic source of information provides me a wealth of information that others may simply not have.  (It also helps that my job involves QA and I end up reading hundreds of reports or case studies every year.)  Also, it’s not a point in time, but I call upon these individuals all the time to help shape and crystallize my understanding of the ever changing landscape of risk.

Two years ago when I met up with Adam Shostack at RSA and as we talked about the industry he explained to me that what we need as an industry is more data in order to form proper conclusions.  The main idea being that the more data you have on a specific topic the more easily you, and everyone else, can make a rationale decision about how to best protect it.  The problem with the lack of data is the ability to trust the limited data and conclusions you want so very much to rely on.

This is why when I finally met up with Donn Parker I asked him to explain his concept of diligence-base security vs risk-based security [PDF].  In a nutshell, Donn explained that risk-based approaches are nothing more than data alchemy as there is simply not enough public data available to make any sort of statistically significant conclusion when you assume that the entire population of data breaches or security failures (realistically unknown) is vastly larger.  Indeed it is very difficult to measure and make statistical decisions about the unknown-unknowns.

The example I like to reference is that of scanning for rogue devices (i.e. wireless access points) on a computer network.  Detecting rogue devices (unknowns) is very different than examining known devices, and logic breaks down when trying to apply traditional sampling methods to this unknown landscape.  Traditionally, sampling of a population is done when the population is uniform, or in some way known.  In general, the more uniform the population the smaller the sample size may be to determine a statistical conclusion.  The problem with rogue devices is that the population is unknown.  If you try to sample from an ever changing population the results you get at any point in time may be statically non-reflective of the total population.

Mr. Parker advocates that since we do not have a population of data breaches significant to the total number, and since the total number and type are ever changing, there is no scientific way to apply risk-based controls.  Instead he advocates a diligence-based approach towards security.  Since we cannot measure and thus appropriately apply risk-based metrics we should take the agreed upon “best practice” controls we have and be diligent about their application and maintenance.

Arguably, one could take the same cynical approach towards the traditional baseline “best practice” baselines such as BS7799, ISO 27001, ISO 27005 (for that matter the entire ISO 27000 series), or even HIPAA (HHS guidelines), or GLBA (FFIEC guidelines).  How do we know that these are sound practices upon which we should build an information security program?  With technologies changing and evolving over time there are many different ways to envision security.

So if we cannot base our foundation on best practices, and we cannot apply risk-based controls, what then is left?  This is where I propose holistic information security.  The diligence method is based on factors such as budget, management directives, staff talent and availability, and organizational policies.  Although this sounds right from a business perspective, following these methods provides a ‘good enough for the current business’ which may or may not be the best direction for the business to protect itself.  Arguably, one cannot know what the best direction is for the business due to lack of data.  See also, chicken-and-the-egg.

I’ve watched over the years as analysts, experts, and individuals claim to have the correct answer, when in fact all they have is their one piece of the pie of truth.  Instead, I advocate taking a holistic approach towards security and assimilating as much data as you can before making a decision.  Talk with as many stake holders as possible so you can elevate your level of knowledge about your industry from amateur to expert.  Only by reviewing others’ piece of the pie can you approach seeing the bigger picture.  In fact, Donn Parker advocates this in his ISSA Journal 2008 paper by proposing that practitioners of the art of information security seek out other sources of information from other organizations of comparable size, type, structure, and threat exposure.

If we are actually dealing with an unknown-unknown that we cannot measure or (honestly) see the entirety of, then we are left with only one option.  The only option left is to assimilate as much of the whole as we can.  The goal should be to “seek first to understand and then to be understood“.  This approach enables us to make more informed decisions about what is valuable information and what is fodder.

Update: I also highly recommend you watch Alex Hutton’s Security B-Sides talk on, Risk Management – Time to blow it up and start over? [slides]