Book Review: The VisibleOps Handbook

The VisibleOps Handbook: Implementing ITIL in 4 Practical and Auditable Steps

by Gene Kim, Kevin Behr, and George Spafford
Publisher: Information Technology Process Institute
ISBN: 0975568612
Number of Pages: 100
Date Published: June 15, 2005


VisibleOps is one of my favorite computer geek books of all time.  This book is a no-nonsense, straight forward guide to running a highly successful IT department.  But, VisibleOps is not just some flavor of the week self-help management book.  The lessons and goals presented in VisibleOps are the culmination of years of observation and research by the authors, who happened to notice that successful organizations had IT departments that operated in very similar ways.  This book is a distillation of those observations into a methodology that is easy for anyone in IT to grok.  Loosely based on the ITIL framework, VisibleOps cuts straight to the chase with four basic steps. 
 
The Four Steps of Visbile Ops

Phase 1. Stabilize the PatientPhase 2. Catch & Release and Find Fragile ArtifactsPhase 3. Establish Repeatable Build LibraryPhase 4. Enable Continuous Improvement

Stabilize the Patient

In the first phase of VisibleOps, the goal is triage.  Can you reduce the number and impact of outages?  Some of the key ways to accomplish this goal is to implement and strengthen Change Management processes, only allow scheduled changes, and have a defined maintenance window.

Another huge benefit to the Change Management process that often gets overlooked is its ability to act as a communication tool and a way to publish a schedule of changes.  With these processes in place, you will have better visibility for outage responders:

1. What changed? 
2. How to back out that change

Fragile Artifacts

The second phase is all about using a risk based approach to identifying and cataloging critical systems.  Some of the key indicators include:

• Systems with the highest Mean Time To Recovery (MTTR)
• Systems with low change success rates
• Systems with the highest downtime costs

But being able to understand and identify the cost of downtime requires understanding the business processes that each system supports.  That is why this phase is based on the Configuration Management process and includes implementing a Configuration Management Database (CMDB).  Once these processes are in place, you should see a reduction in variance, increased conformity in your systems, and it will be easier to detect anomalies within the environment.

Repeatable Build Library

In order to overcome the limitations imposed by the Fragile Artifacts, you must create a way to commoditize these systems.  Phase three is all about implementing proper Build and Release Management processes to further reduce variance and increase your understanding of what your systems are actually doing.  The thing that makes systems fragile in the first place is your lack of understanding about how that system operates.  

Once you are able to obtain that level of understanding, it is much easier to swap out interchangeable components than it is to ad-hoc a resolution out of random troubleshooting steps that you can't really explain WHY those steps "fixed" the issue.

Continuous Improvement

You would think that phase four would be self explanatory.  It is anything but that.  In terms of implementation, I have found that this can be the absolute most difficult because it requires a major shift in the culture of most organizations.  The VisibleOps Handbook provides some key indicators and metrics that can help track your progress on this journey.  It does not, however, provide much advice on how to steer your Titanic to avoid icebergs along the way.

Reflection

The thing I love the most about the Visible Ops approach to ITIL and managing IT in general, is how corporeal it is.  The word "visible" in the title obviously wasn't an accident; it is visible because the steps for implementation, the explanation of the methodology, really everything about it is so clearly evident that [almost] anybody should be able to thumb through this booklet and pick up some ideas that they can put to use right away and see results almost as fast.

#ChecklistIsDead

I keep seeing tweets and blog posts and hearing talks at various cons that keep repeating statements such as: 

"[insert unpopular framework/checklist here] has done nothing to improve cyber security, and in fact it has probably made security worse"

And I don't believe it!

I recently wrote about how the InfoSec echo chamber keeps dogging on "outdated best practices", and today I started wondering if these echo repeaters all work for Gartner?  So I'm proposing that all framework/checklist bashing should use the hashtag #ChecklistIsDead from now on.

My point is that one of the biggest reasons InfoSec is failing is not because we are using a bad checklist.  We are failing because we aren't actually following through with implementing *any* checklist consistently, whether it is the PCI DSS, FFIEC, FISMA, NIST, or the SANS Critical Security Controls.  I don't really care which checklist you are being graded on (most of them can be cross-referenced with each other anyway, just different wording for the same basic goals), but if you can't make a list of your key business process, a list of your critical information assets, and updated diagrams for your network and data flow... then what makes you think that you are going to do any better with the newest #RiskManagement flavor of the week?

For example, I hear a lot of people complaining about the PCI DSS in one breath and then calling for the need to replace checklists with a risk based approach to security.  That's all fine and good, but if companies can't comply with the intent of PCI DSS v2.0 Requirement 12.1.2 to perform a risk assessment [only] once per year, then how well are they going on their own without such a requirement?

Establish, publish, maintain, and disseminate a security policy that "12.1.2 Includes an annual process that identifies threats, and vulnerabilities, and results in a formal risk assessment. (Examples of risk assessment methodologies include but are not limited to OCTAVE, ISO 27005 and NIST SP 800-30.)"

I have read some articles lately (here and here) that talk about how security policies and frameworks are too silo'd and need to span across functional boundaries.  I'm sorry, but show me what framework or checklist specifically calls for its implementation to be contained within silos?  These failed implementations are the direct result of bad decisions made at the highest levels of most companies who don't understand the threats and vulnerabilities facing their organizations.  Yet these same decision makers are supposed to magically understand the risk derived from these same threats and vulnerabilities in order to invent a better #RiskManagement program that fixes their security failures?

All the while, taking time to actually implement the items on the existing checklists keeps slipping through the cracks or falling down the priority list (and just getting a QSA to submit your RoC to the cardbrands doesn't mean your company has actually implemented all of the requirements on the checklist).

There were several interesting items listed in a recent paper by James Lewis of the Center for Strategic & International Studies, Raising the Bar for Cybersecurity.

"In the last few years, in 2009 and 2010, Australia’s Defense Signals Directorate (DSD) and the U.S. National Security Agency (NSA) independently surveyed the techniques hackers used to successfully penetrate networks. NSA (in partnership with private experts) and DSD each came up with a list of measures that stop almost all attacks.

"DSD found that four risk reduction measures block most attacks. Agencies and companies implementing these measures saw risk fall by 85 percent and, in some cases, to zero."

<sarcasm>Too bad checklists are dead.</sarcasm>

Best Practices

Great comment in this week's SANS NewsBites (Vol. 15 Num. 029) from Alan Paller, director of research at the SANS Institute.

[Editor's Note (Paller): As organizations discover there is economic liability for lax cybersecurity, and lawyers smell blood in the water, the recognition will dawn on policymakers that their reliance on high level "guidance" was a really bad idea and made government cybersecurity a terrible model for protecting the critical infrastructure and businesses.  This week the Australian Attorney General established a legal requirement that all agencies implement a small number of critical security controls. No company can pretend they don't know the basic controls they must implement. The U.S. government will do that, too, but, as Winston Churchill said so long ago, "Americans will always do the right thing - after exhausting all the alternatives." You can get a head start on doing the right thing if you can get to London on May 1-2 (http://www.sans.org/event/critical-security-controls-international-summit) or listen in on the briefing on April 18.  (http://www.sans.org/info/128297]


I found this comment somewhat ironic, given the recent twitter conversation with @joshcorman:

@cindyv for 1); stop dogmatically clinging to outdated "Best Practices", Dogma, and static compliance checklists.[cont]
— Joshua Corman (@joshcorman) April 2, 2013

Maybe "Best Practices" really aren't the absolute "Best" that we can do in every individual situation.  And can they really be called "Practices", if they aren't actually practiced? (i.e. repeated performance or systematic exercise for the purpose of acquiring skill or proficiency). Having cursory familiarity with an established checklist of known good security measures such as the SANS Critical Security Controls, does not qualify as practicing or best.  ;)

@sudosec @cindyv hasn't SANS migrated away from "the 20" ? Also, all controls need to subordinate to context of Risk Program
— Joshua Corman (@joshcorman) April 11, 2013

@cindyv @sudosec I need to look at their newest list; but my main point was the context/program is king
— Joshua Corman (@joshcorman) April 11, 2013

Also, check out Cindy's article about being Consumers of Security Intelligence here.

Book Review: Made to Stick

Made to Stick: Why Some Ideas Survive and Others Die

by Chip Heath and Dan Heath 
Narrated by: Charles Kahlenberg
Publisher: Random House Audio
Total Length: 8 Hours, 37 Minutes
Date Published: September 17, 2007


Any book that starts off with a variant of the kidney thieves heist automatically gets a +1 rating in my evaluation process.  Another +1 for having duct tape on the front cover and you know this is going to be a great book.  

Made to Stick was incredibly fun to listen to.  The authors decided to explore a topic mentioned by Malcolm Gladwell in The Tipping Point to see whether you could measure or influence how "sticky" an idea is.  The book is full of great examples of how to craft and alter the message you wish to convey so that it has a better chance of being remembered by others.  The book is laid out in chapters explaining each of the six criteria for the S.U.C.C.E.Ss of a sticky message:

• Simple 
• Unexpected 
• Concrete 
• Credible 
• Emotional 
• Stories
• s - and another 's' on the end for good measure.

As I started putting together some notes on each chapter for this blog post, I had a difficult time condensing the material covered by this book.  Each chapter is full of great information and superb examples for supporting their point of view.  Here are a few of the over arching concepts that I got the most out of.  Each of these are a massive body of knowledge unto themselves, but perhaps it will provoke you to do some additional research on these subjects, and read this book!

The Curse of Knowledge

As described in the Harvard Business Review article The Curse of Knowledge, most people make the mistake of assuming that other people are going to understand the message they are trying to convey.  The example of Tappers and Listeners makes it clear how easy it is for the meaning of a message to get lost in transmission.

Gap Theory

What makes a subject or situation "interesting"?  How do you get someone to pay attention to your message?  The answer, in short, curiosity.  In George Loewenstein's 1994 article, The Psychology of Curiosity: A Review and Reinterpretation (PDF), Loewenstein "interprets curiosity as a form of cognitively induced deprivation that arises from the perception of a gap in knowledge or understanding."

Priming

In the chapter on Emotion, Mental Priming is discussed as a way of not only getting people to pay attention to your message, but for them to care about your message in a way that causes them to take action.  These concepts about mental priming stood out to me as I read through Blink: The Power of Thinking Without Thinking by Malcolm Gladwell and Predictably Irrational and The Upside of Irrationality by Dan Ariely as well.  Based on the studies and examples cited in these books, it is obvious that Mental Priming is an extremely powerful tool.

Mental Simulation

The results of the Mental Simulation experiments and studies mentioned in Chapter 6 are astounding.  The studies show that "mental practice alone (sitting quietly, without moving, and picturing yourself performing a task successfully from start to finish) improves performance significantly. [...] Overall, mental practice alone produced about two thirds the benefits of actual physical practice."  With these types of results it is hard to ignore the value of being mentally prepared and engaged for a task.  "The more that training simulates the actions we must take in the world, the more effective it will be."
 

Deep Dive (Chapter Reviews Coming Sometime)

Chapter 1 - Simple
Chapter 2 - Unexpected
Chapter 3 - Concrete
Chapter 4 - Credible
Chapter 5 - Emotional
Chapter 6 - Stories

Security Interconnected

In a recent blog post, Is Your Security Harming Someone Else’s Business?, Tripwire CTO, Dwayne Melancon, talks about mapping out the relationships your business has with other outside entities to "connect security to the businesses we depend on and those who depend on us."

This is a novel concept considering that many organizations have such a hard time mapping out their own internal processes, let alone ones that stretch outside their environment.  One of the main points that Dr. Eric Cole discussed this year during his SANS 2013 keynote in Orlando was that when he has been called into organizations to do an investigation or analysis, the first thing he asks for is a network diagram and a list of locations of critical data.  He then conducts a discovery of critical data on the client's network and maps out the true location of critical data to find that it rarely matches the client's list.

One of the questions that Melancon asks in his blog is, do you know what impact changes in your organization might have on contractual commitments with outside parties?  Many times the people engaged in writing, reviewing and signing contracts within an organization do not have the level of technical understanding to know what good security practices are, let alone whether they are properly included in the contract.  In the financial sector, there are regulatory requirements for this sort of thing (see Interagency Guidelines Establishing Information Security Standards)

Under the Security Guidelines, each financial institution must:

• Develop and maintain an effective information security program tailored to the complexity of its operations, and 
• Require, by contract, service providers that have access to its customer information to take appropriate steps to protect the security and confidentiality of this information.

You Get What You Pay For (and Prepare For) 

If the people writing and signing these contracts do not understand InfoSec, then this whole process seems a bit like going off a cliff and not knowing what's up ahead.   

I was reviewing some BCP/DR documents for a small financial institution not long ago and found a contract with a technology service provider (TSP) that was storing off-site backups for them.  The TSP provided the proof of breach insurance that was requested and it showed that the coverage limit was $1 MM per incident.  That sure doesn't sound like much given the amount of money lost by small and medium sized banks recently due to phishing and account take over attacks which led to ACH/Wire Fraud.  But then after taking a closer look at the contract, I found this statement in the section titled "Limitation of Liability":

"If VENDOR becomes liable to the CUSTOMER under this Agreement for any other reason, whether arising by negligence, willful misconduct or otherwise, (a) the damages recoverable against VENDOR for all events, acts, delays, or omissions will not exceed in aggregate the compensation payable to VENDOR […] for the lesser of the months that have elapsed since the Operational Date […]"


Say what? [Unnecessary Risk]

Somebody obviously didn't take the time to read this garbage before whipping out their John Hancock.  Let's just say the amount paid to this vendor each year is much less than the value of the data the customer was backing up with this vendor.  

Managing the security of interconnected systems is not just an IT issue.  It is a business issue which means taking the time to read and understand the contracts that your business is agreeing to abide by.  Does your company have a process for reviewing contracts?  Is your IT/InfoSec team involved in that process?  Are contractual requirements communicated to your IT/InfoSec teams?  If you are missing these steps, then it will be impossible to do any sort of impact analysis across interconnected outside entities... just say'n.

Book Review: Joker One

Joker One: A Marine Platoon's Story of Courage, Leadership, and Brotherhood

by Donovan Campbell 
Narrated by: David Drummond
Publisher: Tantor Media
Date Published: Apr 9, 2009
Total Length: 11 Hours, 41 Minutes

Publisher: Random House
ISBN: 1400067731
Date Published: March 10, 2009
Number of Pages: 336


I first heard about Joker One on NPR during an episode of Fresh Air with Terry Gross interviewing Donovan Campbell.  I was struck by the depth of character and conviction to responsibility that I heard in this interview.  Several weeks later, I remembered listening to the interview and decided to download the audio book.  Wow.  I was not disappointed.  Impressed by not only the quality of the story itself but also the quality of the story telling.  First Lt. Donovan Campbell of 1st Platoon, Company G ('Golf'), 2nd Battalion, 4th Marine Regiment, led a group of 40 marines in 2004 during the beginning and the height of the insurgency.  They were stationed in Ramadi and had to deal with much of the fallout from the first and second battles of Fallujah.  Ramadi soon came to be labelled the most dangerous place in Iraq.

The book tells of Campbell's officer training at Quantico, his first days on the job as Lieutenant of his infantry platoon (call sign 'Joker One'), building relationships and respect with his men, patrolling the streets in Ramadi during some of the most intense resistance of the Iraq war, and returning to 'normal' life back home after serving in combat. 'Golf' Company suffered a 50% casualty rate, which exceeded that of any other Marine or Army combat unit since Vietnam.  

Throughout this book it is clear that Campbell truly embraces the meaning of "servant leader".  This excerpt taken from page 5 has truly profound insight: 

"If you are a Marine lieutenant in a firefight, a situation that's probably as good a proxy as any for hell, then it's your job to figure out at least 50 to 70 percent of what is going on around you so that you can make intelligent decisions, which translate into good orders, which lead to focused, effective and decisive action.  This whole process needs to be rapid to be relevant, but if you're too hasty, then you can lead your men to their deaths, all the while believing you are leading them to safety.  It's not an easy tension to manage on an ongoing basis.

"However, it can be done, and to do it well you must have absolutely no concern for your own safety.  You can't think of home, you can't miss you wife, and you can't wonder how it would feel to take a round through the neck.  You can only pretend that you're already dead and thus free yourself up to focus on three things: 1) finding and killing the enemy, 2) communicating the situation and resulting actions to adjacent units and higher headquarters, and 3) triaging and treating your wounded.  If you love your men, you naturally think about number three first, but if you do you're wrong.  The grim logic of combat dictates that numbers one and two take precedence."

Reflections

The stories told in Joker One are so vivid that I can recall many of the details even though it has been a while since I first listened to this book (I also bought a hard copy soon afterward to refer back to).  A few of the things that really stand out to me include: 

Split Second Decisions - As seen in the excerpt above, being willing to make the tough calls in the midst of chaos and without having all of the facts laid out in from of you is an amazing skill.  It is difficult to say whether decisions made in such circumstances are the "right" decisions.  But the lesson I get from this is that being able to act now and continually adapt is often more critical than the time lost through hesitating over the "right" decision.

Mastery of Details - When Joker One arrived in Kuwait and were preparing to launch into Iraq, they spent time going over and over things that most people probably wouldn't think as being all that important.  For example practicing dismounting from vehicles, as described on page 71: 

"In our world, basic tasks have to be repeatedly rehearsed in conditions mimicking predicted combat scenarios as faithfully as possible.  For example, you can never be sure which small detail might mean the difference between exiting a vehicle caught in an enemy ambush kill zone in two seconds or in ten.  That kind of time differential can be fatal.  Where is the door handle on the seven-ton truck?  Do you have to pull it up or down to get out?  How far is the drop out of the truck bed, and where exactly do you need to put your feet before you hurl yourself out the door?  Once all the little questions have been answered, those answers must be practiced again and again until they become muscle memory.  The Marines didn't like the mind-numbingly repetitive nature of such drills, and they didn't exactly love the squad leaders and me for putting them through the endless rehearsals, but every time we did something tedious and painful, we tried to lay out the reason behind the drills to everyone.  I became amazed at how much my men would tolerate if someone just took the time to explain the why of it all to them"

Trusting Your Team - One of the situations Joker One ran into while patrolling was when a group of kids started throwing rocks at his men.  The men radioed up to Campbell at the front of the patrol that the rocks were really starting to hurt.  While he was thinking about how to handle the situation, one of his men radioed back and said the situation was under control.  The men had gained the aid of an old Iraqi man who scolded the children and they ran away.  As a leader, you won't have the answer to every dilemma that pops up.  So it is important to train your team on the principals you want them to follow and allow them to think for themselves.

Succession Planning - Keeping an eye open for future leaders and take time to build on their talent.  There's a great example on page 22 talks about Campbell's first encounter with with Lance Corporal Carson on a training hike, carrying two packs and pushing another Marine up hill and shouting at him not to fall out.  Carson was later promoted to a team lead.

Dedication to the Mission - Campbell gave a recent interview with Steve Paulson, 10 Years in Iraq, where he recalls one of the events described in the book when the insurgents attacked his platoon.  The rocket-propelled grenade (RPG) that was fired at his men miss them, but instead detonated in a group of school children.  Campbell had the option of getting his men out of the situation, which meant abandoning the injured children or digging in, setting up a perimeter and helping the kids, which most likely meant facing another attack.  I'll let you listen to the interview (or read the book) to hear what happened.


Not only was I impressed by the stories told by Joker One, but what really stands out to me is the contrast I see to my own life.  In 2004, I had been out of college for a little while and making decent money for the first time in my life, and really just living the life of a slacker.  Reading this book gave me a lot more respect for the challenges that young soldiers in the armed forces have to go through and the character it can build.  I like to think of myself as someone who is up for a challenge, strong, courageous; but this book puts me to shame.  I don't think I could have made it Iraq.  However, I truly value the lessons shared within the pages of Joker One.

This book should be required reading for anyone leading a team to understand the amount of care that must be invested in your team.
 
The Leader's Code

I am anxiously waiting for Donovan's newest book, The Leader's Code: Mission, Character, Service, and Getting the Job Done to arrive.  I pre-ordered it last week and I'm looking forward to digging through it for more valuable leadership lessons.  Stay tuned for another book review!

Social Media H@x0rz

Someone once told me that they had just signed up for a Facebook account.  My reaction was to ask "Why?"  This person responded that weren't sure why they signed up, but surely 200 million people (at that time) couldn't be wrong.  That was several years ago and millions of people were rushing to sign up.  As of this writing there are somewhere north of 1 billion Facebook accounts.

This is a great video to help drive home the point that if you post information online, it is not private.

Hahaha... I guess all the H@x0rz really do wear ski masks!

Don't Surrender to Your Smartphone, Part 2

See the original Smartphone rant here.

Gotta love Meme Generator!

http://memegenerator.net/