When measuring compliance results the data available is sparse and the analysis ranges from those who love it to those who hate it. Regardless of your personal, political or analytical beliefs the question remains about the efficacy of the practice itself.
There are several approaches and conversations surrounding the aspect of regulatory compliance, but the question remains, “What leads to excellence in organizations and the security they implement?”
The conclusion of this post is simple: Excellence within an organization is not achieved by measuring the compliance of an organization but by measuring the compliance of individuals and employees.
So how do we come to such a conclusion? To start we identify the main points for and against regulatory compliance:
- Pro: It raises the minimum level of security
- Cons: It creates a glass ceiling that either (1) prevents proactive organizations from implementing better security or (2) encourages reactive organizations to never excel above a certain point
Regardless of your opinion, both arguments focus around the precise level at which compliance measures an organization. Isn’t that interesting? Both the pro (for) and con (against) opinions seem to claim the same thing. This would be a good thing if it encouraged security without limiting organizations.
Already, PCI compliance (credit card security) has been compared with the No Child Left Behind or Elementary and Secondary Education Act (ESEA) within the United States. If this program is broken, let’s identify why, dispel false truths and come to a conclusion of how to learn from its lessons.
The fact remains, “nationwide: only 6 percent of U.S. students perform at the advanced-proficiency level in math, a share that lags behind kids in some 30 other countries, from the United Kingdom to Taiwan.”
Davi Ottenheimer wrote a summary of an article in The Atlantica titled, “Your Child Left Behind”. It goes on to dispel many myths about the No Child Left Behind Act. Eric Hanushek has spent “40 years calmly butchering conventional wisdom on education” including the following:
- More money does not tend to lead to better results
- Smaller class sizes do not tend to improve learning
- No U.S. state does very well compared with other rich nations
- Even relatively privileged students do not compete favorably with average students in other well-off countries. (In Illinois, the percentage of kids with a college-educated parent who are highly skilled at math is lower than the percentage of such kids among all students in Iceland, France, Estonia, and Sweden.)
“Per student, we now spend more than all but three other countries—Luxembourg, Switzerland, and Norway—on elementary and secondary education. And the list of countries that spend the most, notably, has little in common with the outcomes that Hanushek and his colleagues put into rank order. (The same holds true on the state level, where New York, one of the highest-spending states—it topped the list at $17,000 per pupil in 2008—still comes in behind 15 other states and 30 countries on Hanushek’s list.
If money, class size, and affluence of individuals do not impact classroom performance then why do we think that money, attack landscape, and intelligence of security professionals will impact the security of organizations?
Davi writes, “The exception to this lazy approach is the state of Massachusetts, which has followed a path that found success in other countries. It has directly intervened and introduced compliance.” But what kind of compliance? Was it simple testing that measured student performance against standardized tests? Nope.
The Atlantic gives us a glimpse into the answer. “What did Massachusetts do? Well, nothing that many countries (and industries) didn’t do a long time ago. For example, Massachusetts made it harder to become a teacher, requiring newcomers to pass a basic literacy test before entering the classroom. (In the first year, more than a third of the new teachers failed the test.) The state also required students to pass a test before graduating from high school–a notion so heretical that it led to protests in which students burned state superintendent David Driscoll in effigy. To help tutor the kids who failed, the state moved money around to the places where it was needed most. “We had a system of standards and held people to it–adults and students,” Driscoll says.”
So what works?
“Massachusetts, in other words, began demanding meaningful outcomes from everyone in the school building. Obvious though it may seem, it’s an idea that remains sacrilegious in many U.S. schools, despite the clumsy advances of No Child Left Behind. Instead, we still fixate on inputs—such as how much money we are pouring into the system or how small our class sizes are—and wind up with little to show for it. Since the early 1970s, we’ve doubled the amount of money we spend per pupil nationwide, but our high-schoolers’ reading and math scores have barely budged.”
Problem Set
My personal opinion is to focus on a capability and maturity model (CMM) of security and making regulatory compliance a natural side effect rather than an end goal. Sounds academic; so where’s the beef?
The practical implementation of this is shown in the recent Verizon PCI Compliance Report wherein it showed that organizations fail at tasks that: require human intervention or reoccurring activity. Many organizations that focus on compliance as an end-goal, fail to validate or maintain security throughout the year. No shocker there, but how do we overcome the human side of security? Security professionals have been talking about addressing the end-user for a long time, but these are not end-user problems they are security-professional problems.
Can we just throw more money at the problem? Reduce the scope of compliance? Train our security staff? Well, studies of the U.S. education system show these methods to be ineffective since they do not encourage well funded security professionals to do things like review audit logs on a daily basis. Even the most automated of systems are often times ignored for a variety of reasons.
Conclusion
Excellence within an organization is not achieved by measuring the compliance of an organization but by measuring the compliance of individuals and employees.
Building maturity models for information security implies an ever increasing maturity and level of security. This helps break the proverbial “glass ceiling” of compliance by having the security of an organization grow in proportion to the ever evolving attack landscape. This is so much easier said than done.
Our ability to achieve this goal hinges on our ability to encourage individual participation. Encouraging individual security-professionals to take action towards this goal.
So how useful is regulatory compliance? I advocate that compliance is good, but only in measuring the security of an organization at a point in time. We need something much more than this to achieve real security. We need something that will encourage validation and maintenance of security.
Final <rant>
Better security will not come from automation (DLP, audit log aggregation, etc.) Better security will not come from more intelligent tools. Better security will come from a higher standard within organizations to focus on maintaining security. This leads to a discussion of cross-training security-professionals on conversational business-speak and helping them build measurable, results-driven risk models… but that is for another day.
Mike,
Your comparing security/compliance with education/NCLB is inspired. But I would like to suggest one additional item to add to the mix.
As a former teacher, I saw the issues you raise. To me, the most important factor in my students’ education was not money or computers in the classroom. It was the active support of the parents. That is, the parents gave the kid a place to study, then encouraged and supported them.
If we follow that logic, success in security will need the active support and encouragement of the organization’s senior management (i.e., the “parents”). While that is not enough in itself, it will allow security professionals to do their jobs, reward them (another topic: how do you reward someone for NOT having a data breach?), and encourage training and information sharing.
How do we get this support for security? Compliance is a good argument for budget, but it is not a positive factor. Besides, compliance is too often sold (forced?) on the basis of FUD. That is, what we need is the positive business case for security, then to make that case to the “parents?” Maybe then we can achieve your goal of a “higher standard within organizations to focus on maintaining security.”
Pingback: Tweets that mention Your Security Left Behind : How compliance and security can play well together | Chaordic Mind -- Topsy.com
Walt, yes, I’ve always said it’s the parents who we should blame for poor security. We would not need PCI compliance tests if corporations had better parenting.
Not having a security breach IS the reward.
Thanks for the link!
Walt,
I agree with you regarding the need for better parenting on behalf of corporations. Over the last 10 years I’ve been around the world and in/out of tens of thousands of companies. People argue about compliance being about the right requirements but none of that matters.
Plain and simple, left to their own devices most companies will do a little as possible when it comes to any cost center – and most companies see security as a cost center.
We are an industry of intelligent-idiots. We can spin elegant code to re-write dynamic sections of memory and circumvent assumed security controls, but we cannot justify our own existence to a board of directors.
I blame us for not educating the higher-ups about the risk their organizations face, but should instead blame the higher-ups for not hiring more competent people who can both measure and articulate risk the same way as wall street traders and insurance actuaries.
Davi,
I’d like to thank you for pointing me to the article in question. Your point is spot on that we would not need PCI compliance if corporations had better parenting. They are absent parents and absent managers.
It’s easier for a company to deal with the annuals of compliance than to fix the problem, but that is not an MBO that can be measured in a quarterly bonus.
Dare I say, that real security will not occur until we can show organizations that investing in security is a positive return on investment, and I do not mean some etherial measurement of un-quantified and intangible things such as the illusional “repetitional risk.”
I am talking about actually making money on security, and not just the vendors and consultants. We need to start showing our customers and bosses the direct and immediate return on investment that security has on the business.
I deal daily with retailers who tell me they would love to do security and open a new store but only have the money to do one of them. They tell me that if I can show them how to do both, but until that happens they will keep opening stores. This brings in short term gains which are measured in bonuses, and if a breach happens you can either blame it on the predecessor or spin it into a I-found-religion story about security. At least the latter warms the cockles of readers hearts and paints the poor risk as a common mistake anyone could have made.
But I digress. Focus on the money. It’s the only way we will see a change.
Mike
Pingback: Payments, Software, Technology on January 3, 2011 : emergent commerce and technology
Pingback: Your Security Left Behind : How compliance and security can play well together | Consulting & Business Intelligence Services Private Limited