This workshop at the upcoming Apollo RCA Symposium will demonstrate a new approach to identify systemic problems in your organization through integration of past root cause analysis (RCA) cause and effect charts.  In short, this approach will enable you to be proactive with your RCA efforts.  Sound interesting?  Attend the Apollo RCA Symposium to benefit from this and other highly-valuable workshops and presentations!  Learn more here.

Do you know how you will react when a serious incident occurs?  When the phone call comes for you, will you be confident in your training, experience, and preparation?

As investigators, we usually don’t think about how were going to react to an incident until it happens. When an incident does happen the company looks to us as the experts for what to do. That feeling of overall confidence and reassurance doesn’t happen when the reply coming back from the expert is “I’m not really sure.”

This exchange of information at the 2010 Apollo RCA Symposium is not as much to provide you with all the answers about how to react, but more of what question should you be asking yourself and your organization to determine if we, as experts, are ready. If were not ready, what needs to be developed so you, as the expert, when called upon are ready with the answers and are ready to react?

Sound interesting?  This presentation will be given at the Apollo RCA Symposium by Mike Resimius, Corrective Action Specialist at Ameren Generation in St Louis, MO.  For more info and to register, visit the Apollo RCA Symposium webpage.

Do you struggle with how to apply effective solutions to "human error"?

Human errors can be found in any adverse event.  Often, human error is listed as the primary cause of significant incidents in the news.  And in our daily work lives, incidents of less significance are also often blamed on someone doing the wrong thing at the right time - or failing to do the right thing when called upon.

Any of this sound familiar?  We'll give you practical tips and best practices you can take back to work during this Apollo RCA Symposium workshop.  We'll discuss human error within the context of a case study that analyzes an adverse event.  Attendees will gain a more complete understanding of how to manage human error as a cause, and how to reduce the systemic risk and impact of future errors.  Learn more and register for the 2010 Apollo RCA Symposium here.

What does recent cognitive science research on how our brains work tell us about how to accomplish this?

In many types of work -- such as aircraft and power plant maintenance, construction, firefighting, and healthcare -- learning from incidents and near misses can be a matter of life or death.  Cognitive theories and knowledge in the domains of memory retention, and behavioral and cultural change, have advanced significantly in the past 10 years from new tools such as Magnetic Resonance Imaging and many studies of the brain.  Some business practices have not evolved to benefit from this increased understanding of how our minds work.   Reporting on critical incidents, including crafting and delivering the message, while a crucial function for many industries, has not changed significantly - but needs to.

Sound interesting?  Beth Lay, Manager of Risk Management & Loss Control at Siemens, will be giving this presentation. Learn more and register at the Symposium webpage.

A recent article published in Quality Digest pointed out that in order to foster a problem-solving culture, managers must serve as mentors and cultural leaders -- building the systems and atmosphere that support and encourage team members at all levels to problem solve effectively.

That is absolutely true, however, once these problem-solving roles are understood, truly developing an effective problem-solving culture will require the following three elements:

• Adopting a formal, standardized problem-solving methodology for the organization
• Integrating the formal system into the organization in away that capitalizes on existing roles, responsibilities, andsystems
• Embracing a change management process to assure people have the training and coaching toaccept and perform new problem-solving functions

For specific tips on developing these elements, see the full article published in the June 18, 2010 issue of Quality Digest at http://www.qualitydigest.com/inside/quality-insider-article/got-effective-problem-solving-culture.html

(Excerpted from our article in the May 2010 issue of Industrial Engineer magazine)

Many IT departments struggle with the negative business impacts of recurring problems, and many also struggle with how to proceed with formally investigating each major problem.  The risks are so significant that IT-related problems are estimated to impact the US economy to the tune of around $60 billion per year for software errors and around $140 billion per year for info/data security breaches.

So how are solutions typically being developed for these problems?  There seems to be a reliance on the statistical analyses of industry trends to drive IT-related solutions. Information from actual IT problems categorized by type, or assumed cause and solutions, are recommended based on data trends exhibiting the highest percentages or greatest threats.

This approach leads to one of the most common reasons problem solving is often ineffective -- solutions based on categories do not necessarily address the actual causes of a given problem as effectively as solutions that are specifically designed to control the actual causes of the problem.  Categories don't cause problems - causes do.  Categories are titles created so that similar things can be grouped together, so by nature are generic on many levels. Generic, categorical-based solutions fail at a much higher rate than do solutions that control specific causes of a defined problem.

The problem management component of the Information Technology Infrastructure Library (ITIL)[1] framework sets the stage for each organization to adopt effective problem-solving strategies that will improve the quality of internal and external IT services.  Recently, IT organizations have begun to realize that some of the same problem-solving best practices long used by other disciplines such as quality, safety, maintenance and reliability are adaptable and scalable to IT.  Successful IT problem-solving organizations are increasingly implementing formal root cause analysis (RCA) within their ITIL problem management structure.

To glean specific best practices for implementing root cause analysis in an IT environment, read the rest of our article published in the May 2010 issue of Industrial Engineer magazine. Read the article here.

[1] ITIL - Information Technology Infrastructure Library. http://www.itil-officialsite.com/home/home.asp

When a major safety event takes place at one of your company's facilities, what do you do?  How do you manage the immediate issues - avoid more danger, handle employee fear, manage circling news trucks or helicopters, and the like?

We often hear stories of engineers, supervisors and safety folks attacking the scene like a herd of elephants in a china shop.  Others recount that one safety professional is sent out to do everything, which can feel like herding cats - including cats that don't accept the safety professional's authority.

Site managers and their staffs often are hesitant to call in someone from corporate because they like to maintain control of their own site and would rather handle the situation privately -- without judgment or imposition of corporate culture and opinion.

In short, it can be total chaos without a thorough plan and adequate preparation.

The heat of the moment is not the best time to decide what needs to be done and who's going to assume each role.  In some cases, organizations have thought about that ahead of time, so they create and document protocols and procedures.  But is that enough to enable people to be totally prepared?  Many organizations don't know until the major incident happens.

How do you set up a system so that everyone really knows what response will look like, and what their roles/responsibilities will be?  Plant personnel are responding with a goal to recover from the incident, so someone else needs to be prepared to respond with a mission to find the causes and solutions.

By designating, training and preparing employees ahead of time, and continuously practicing and refreshing training, organizations are most well-equipped to effectively respond to safety events...and help prevent them from recurring in the future.

For best practices related to setting up such a system, see the full article here.

• Being used to their autonomy, people are suspicious and resistant when folks from headquarters come waltzing in.
• They don't really want their problems to be in the spotlight, nor the causes to be revealed -- especially not to upper management.
• Site leaders feel like they're losing control of their domain, and that their authority is being undermined.
• They worry that they won't be able to discipline their people within their usual discretion.

So, it is possible to improve on that scenario?  From my 26 years performing corrective action in a large industrial organization, yes.  But it takes a well-planned and deliberate approach.  Following are practices that worked for me and my team.

• Recognize that every satellite location has a culture of its own - unique from corporate and from other satellite locations. Be humble and open-minded enough to understand and adapt to each site.
• Recognize that each satellite location is much like a family. Even though they may bicker, when push comes to shove they will be more loyal to each other than to anyone else.
• To effectively communicate with the people directly involved in problem incidents, you must have a solid relationship with them beforehand. Seek ways to build tangible trust.
• To change inaccurate perceptions, you first need to know what their perceptions are. Genuine motives, face-to-face contact, direct questions, and careful listening are critical.
• Communicate your motives clearly. If you don't tell them, they will draw their own conclusions, filling in the blanks with a negative attitude like, "what are you trying to do to me, make my job harder?"
• Demonstrate that the intention is to eliminate problems they relate to, that directly affect them -- and to make sure that the problem doesn't happen again to someone else.
• Give them credit. Acknowledge what they do to create results that improve the workplace. Be specific and refer to tangible results. Demonstrate that you really understand and notice.
• Don't ever revert to blame.
• Emphasize that solutions - which often will represent changes in their processes/procedures -- will benefit them, the workplace, the bottom line, etc.
• People who work in the area where the problem occurred may think they know the causes better than anyone. Indeed, they work most directly with the causes and effects. How can their experience be respected, tapped and utilized to identify the true causes?
• Quietly realize, though, that the people directly involved may not really have the big-picture perspective on the processes and systems, or on the materials/parts.
• When you use the Apollo RCA method correctly, you will make a sincere and concerted effort to glean their input through a structured process of productive input-gathering - avoiding any personal opinion, speculation, negativity, finger-pointing, excuse-making, etc.

If management buy-in could use some improvement, too, see my previous entry titled, "Feel expendable to upper management? Improve the corporate quality culture."

John Stiller | 13 April 2010

Catastrophic failures with enterprise-wide impacts are obvious and are easily identified as problems requiring formal investigation.  Fundamentally, though, many IT organizations struggle to define the difference between some types of incidents and problems, and grapple with how to classify and respond to these unwanted service interruptions appropriately.

The key is to focus on escalating incidents that are preventing your organization from achieving specific goals or targets.  One you have outlined your first draft of threshold criteria (as explained in our previous IT blog):

• Continuously improve your approach to RCA over time. Setting and working with threshold criteria in problem management is a fluid process.
• Take a look at the organization's history. Forecast what your group might be faced with in the forthcoming year. If a threshold for a given goal is set at level Z, how many major incidents would you expect to be triggered in a given year ifZ defined a major incident? Ask yourself whether your team can handle that number of investigations per year. If not, then you may need to raise the threshold criteria to be in balance with your team's capabilities to conduct RCAs.
• Review threshold criteria periodically. Adjust up or down and add new ones as needed. Threshold criteria should be in-sync with changing organizational activities/situations.
• Consider human bandwidth. Scale the threshold criteria to match your team's bandwidth to complete formal investigations on time. If the threshold is set to look good on paper and satisfy company "politics," the numbers of actual investigations triggered by that threshold may overwhelm your capacity to investigate each problem.
• Don't sacrifice quality. The quality of the investigation is often sacrificed to meet the requirement to close out an investigation by a certain deadline. This situation typically leads to ineffective solutions and the recurrence of problems. Evaluate whether it's worth reconsidering deadlines for the sake of RCA quality and problem elimination.
• Adjust your threshold criteria as you experience fewer problems. In the first few years of building your problem management process, you may have a high number of RCAs generated in any one year. But if you start to prevent some of those problems from happening again, using a robust RCA tool you can start eliminating systemic issues that reduce future risk. Then the number of RCAs your group will face is likely to decrease as your program matures. We'll cover systemic issues in more depth in a future blog.
• Demonstrate the effect that formal problem management is having on the organization's bottom line. For instance, 50% reduction in events of type A over the last year. We'll cover the bottom line more in a future IT blog.

Share your experiences and best practices.  We're building an online community to support professional development and continuous improvement.

Posted 31 March 2010 by Brian Hughes, Apollo Vice President

Managing the problems associated with the global supply chain of major products these days requires a flexible, adaptable, and consistent approach.

One need look no further than the latest commercial aircraft designed by Boeing to see just how far a company is willing to go to realize the numerous and enormous benefits of a decentralized, extended global supply chain.  Expertise and specialization is focused directly on individual components.  Profit and loss responsibility is concentrated into smaller, more manageable (and thereby accountable) business units.  Risk is diversified across multiple "baskets" of suppliers.  The benefits of local markets (such as cheaper labor and the proximity to raw materials) can be exploited.  The list goes on.

But Boeing's strategy also shows some of the inherent risks and unfortunate consequences associated with managing such a supply chain.  Unfortunately, it is one thing to map out all the potential benefits of a diversified global supply chain, yet quite another for the company and its managers to actually make it out alive given the risks involved.

While the new 787 by Boeing may be one of the most ambitious attempts at wringing benefits from an extended global supply chain, we can find other examples of all sizes, shapes, and flavors.  Globalization is just another way to say "global supply chain."  Like it or not, it is here to stay.

Companies that learn to manage the risks of a global supply chain can expect to reap, at the very least, the reward of survival.  But those that learn to proactively manage the problems encountered in such a diverse system can expect to rule their sectors.  The key component to proactive problem solving is a robust Solution Management System (SMS) built on a solid, adaptable root cause analysis program.

To proactively minimize the risk of future failures by learning from failures in the supply chain, read the rest of our article published in the February issue of Industrial Engineer magazine.

When something goes wrong in your organization, is human error often the identified cause?  Is "more (task-related) training" for the people involved often the designated solution?  As a result, have employees in your organization reluctantly endured many kinds of training - often referred to as "flavor of the month"?  And do the problems you are trying to prevent continue to occur?  Given limited resources, are you getting the most out of those resources?

Organizations have an opportunity to look more closely at the results they are trying achieve, and evaluate what actions will help achieve those results.

Consider all the time and energy it takes to provide training - from the people who initiate and drive it, to the people who plan it, to the people who deliver it, to the people who partake in it, not to mention the work that is not getting done while those people are away from their jobs, or the temps that must be hired to fill in - it is a huge commitment and investment.  Especially in the current economic climate, when everyone needs to be very careful with the way they invest their resources - time, people and money - it is all the more important to carefully focus training and make sure there's payback on the investment.

Training in an effective root cause analysis methodology and careful program implementation can resolve these issues and help accomplish training goals.  For best practices, go to our article published as the cover story in the February 2010 issue of Plant Engineering.

Link to cover and contents:

http://www.plantengineering.com/archive/2010/20100201.php

Link directly to article text:

http://www.plantengineering.com/article/447578-Is_more_training_the_solution_to_human_error_.php

One thing is for certain, you must compete for executives' attention.  A compelling case that stands out from the many other initiatives is necessary.

Executives often don't pay attention to root cause analysis programs.  Why?  It's not because they don't care.  Rather,  they don't understand the lost value that an effective RCA program could easily scoop up.  Leaders these days are simply swamped with significant problems and little time is available for them to dig in and understand what is going on, or at least that is the perception.

The reality is that if they can invest one or two hours of time and help you solve just one system-wide problem, they can not only endorse your efforts, they can help save significant money and demonstrate a proactive change that will improve the work processes or system.  You need to set the stage before this can happen.  So, where do you begin?

First, let's understand the systemic causes that enable system-wide problems.  Systemic problems are pervasive throughout all work processes in the entire organization, including reliability, manufacturing, EH&S and supply chain.  Systemic problems have considerable negative impact on overall performance and competitiveness - they generate many of the causes (problems) you have been working hard to eliminate.  Left unchecked, systemic problems will always generate new problems.

• The first step to identifying systemic causes is to build a library of thorough and sound RCA reports.
• After you have some RCAs to draw from, create a new file with a higher level starting point than most RCAs. For example: "20% more quality defects than desired," "exceeding recordable injury goal," "10% over budget," "Excessive customer complaints." Import in all existing RCAs related to, or that contribute to the new RCA.
• Organize this new, larger chart and look for common causes. Pareto the top three common causes...they will be there.
• When you communicate to executives that you've uncovered common causes in multiple reports, you can estimate what the resulting, aggregate problems are costing...AND the potential risks this presents for the future. NOW you'll have executives' attention.
• This is an opportunity to solicit executive input. Engage them to discuss the common causes. Take each of the three common causes and ask "why". This will start to drive toward systemic causes that are causing problems all over the organization. I think you'll find that not only will the leaders enjoy utilizing their previous functional backgrounds (which often feel underutilized once they get to management), you'll benefit from their big-picture view of how the company's many functions and systems interact. They'll probably find it fun, and feel it's a valuable use of their time because of the potential ROI. This is a way to get them to show their support for your RCA efforts and program....they will talk/boast to other leaders about the systemic problems being eliminated in their area because you, indirectly, caused them to take ownership of the RCA results.
• Together you'll pinpoint what solution(s) can be implemented, AND how much will be saved after the common cause is eliminated.

Question: How have you successfully eliminated systemic causes?

Question: What kind of ROI have you helped your organization to achieve?

Question: How have you involved executives?  Suppliers?  Customers?  And how has that impacted leadership support of your program?

Share your experiences and best practices.  We're building an online community to support professional development and continuous improvement.

Posted 11 December 2009 by Chris Eckert, Apollo President

Many emerging ITIL programs either don't have a formal root cause tool in place or take an ad hoc approach to identifying problems.  Either way, most don't know how to separate incidents from problems in order to right-size the investigation response, so that time and money is not spent needlessly.

It is most important to ensure that your root cause analyses are focused on the events that have -- or could have -- the greatest impact on the most important functions of your organization.  That way, your RCA effort is aligned with the organization's business goals.  To differentiate between incidents and problems -- or incidents and major incidents -- draw a line in the sand where the cost of the unwanted event outweighs the cost of a formal investigation.  Set threshold criteria that specifically outline the kind of response that is appropriate for different scenarios, make it easy for anyone including help desk staff to make the decision, and remember to clearly outline the rationale to justify your criteria.

One way to create threshold criteria that define what Problem Managers should formally investigate would be to set up a table.

• In column 1, list business goals, targets, key performance indicators and objectives. These can be set up at all levels of the organization including corporate, business unit and project.
• In column 2, list how IT supports these goals or targets, such as services provided.
• In column 3, list how an IT-related event could compromise that goal or target. Specify the magnitude and direction of an event, such as "unplanned server outage more than one hour in duration during business hours." This third column essentially becomes the threshold criteria used to separate incidents from problems.

Engage people in the organization to help you define what level of service interruption or deviation could compromise the various business goals.  Be aware that each person's definition of what constitutes a problem or major incident will be different.

Another way to approach problem threshold criteria, especially in larger organizations, would be to:

• Look at your service license agreement (SLA) and identify all the IT performance outputs - all the "IT will do X" statements.
• Outline "when X does not happen, then Y will happen." Determine what kind of response is appropriate when Y happens.

In order to do this successfully, you may need to more specifically define the general terms that are contained in many service agreements.   For example:

• "Significant number of people affected." How many people?
• "Percentage of total tasks that can no longer be performed by individuals." What percentage? What tasks?
• "Significant impact on the delivery of customer product." What is a significant impact? One day late?
• "Significant risk to safety, law, rule, or policy compliance." How do you define significant risk? One un-licensed copy of a software program on a single desktop?

How does your organization decide when to perform a root cause analysis?

Question:  What threshold criteria do you have in place?

Question:  What best practices did you follow to you create them?

Question:  Who leads the effort?  What role to systems and business analysts play?

Share your experiences and best practices.  We're building an online community to support professional development and continuous improvement.

Posted by Mark Hall | Canada Account Manager, Instructor, & Investigator | 12/2/2009

[1] Within the incident management process, an incident is defined as an event that interrupts or reduces the Quality of Service. An incident may affect a single employee (i.e., user) but it does not affect the overall business in a significant way. Within the problem management process, a problem is defined as a series of incidents that adversely affects the business because it affects a group of employees.

Question:  When you present the results of a root cause analysis, do decision makers question the conclusions, thinking they know better?

Question:  Are you denied budget for solution implementation?

These symptoms may indicate lack of management support.

Before you can change corporate culture, you need to know:

• What it is? The predominant attitudes and behaviors in an organization.
• What you want to eliminate. What's detrimental?
• What you want to add. What's a positive influence?

Eliminate the bad, reinforce the good.  Sounds simple enough.  But we all know it can be political, complicated and tricky.

Put yourself in the executive's shoes.  Play to their perspective and needs.

• They're focused on what's in it for their department or function. Achieve results that make them look effective and successful. Eliminate problems that are costing them money and wasting their resources.
• The bottom line is king. Demonstrate cost-avoidance, revenue generation and return on investment.
• They think at a high, business-goals level. Understand their business goals. Demonstrate how your work is directly helping meet those goals. Call in other internal experts to help you gather and position the data.
• They'll embrace the Quality Management System (QMS) and RCA only if the results merit. Speak their language. Present results, not process!!

Question:  How is the level of management support in your organization affecting your job?

Question:  What attitudes and behaviors do you need to eliminate or reinforce?

Question:  How have you improved the corporate quality culture?

Share your experiences and best practices.  We're building an online community to support professional development and continuous improvement.

I look forward to hearing from you,

John Stiller | November 17 2009

It was a dark and stormy night, and all the vampires were just getting to work.  As they gathered around the blood cooler drinking their cups of red 'go-juice', the conversation was dominated by the subject of a chilling event that happened a few nights prior.  Apparently, Vlad (not his real name) sprained his ankle while jumping down off a flat bed truck trailer.  He had climbed up onto the trailer to replace a tarp that had been ripped sometime during previous usage.  He threw the ripped tarp onto the ground below.  Then, instead of climbing carefully down the permanent access ladder at the back of the truck, he sat down on the edge and gently pushed himself off.  He landed on the discarded tarp, which slipped under his foot.  He then twisted his ankle.  Ouch!  The witch doctor concluded he had a badly sprained ankle, which forced the safety manager to reset the lost-time accident counter to zero.

During the interview, Vlad offered the following:

"Well, that turned out to be a stupid mistake.  I did this work as a human for five years as a contractor in Moldova before my abrupt introduction to the realm of the undead.  You think vampires are thick in Transylvania?  Well, after Gorbachev took the lid off the Eastern Block, they fanned out all over the place.  Just like Bald Eagles - you used to go your whole life without seeing them... now they're competing for road kill with the crows.  Hmmm, speaking of road kill, is it lunch time yet?  Anyway, now I've been doing the same kind of work - albeit on the night shift - for the past fifteen years.  But my undead bones must be getting weak.  I've never been hurt before.  I thought that I was being careful by sitting down first before jumping off the trailer.  If only I hadn't landed on that tarp, I wouldn't have slipped and we wouldn't be here."

Management wasn't buying it.  They knew that Vlad had been trained recently in best safety practices because they pulled the training roster.  Right there, between Keith Richards and Joan Rivers (two longtime members of the undead) was Vlad's scrawled signature.  He should have known that that his actions of jumping off the trailer weren't acceptable.

The rules were very clear.  If an employee violates a safety procedure, they are to be bolted into their coffin for two days with a sunlamp and a bag of beef jerky.  That's what the conversation at the blood cooler was all about.  Sure, Vlad may have violated a rule.  But did he really deserve to be punished in such a way?  The punishment wasn't perceived as just by the other vampires onsite because Vlad had a reputation for being safe.  He had never been hurt before.  And, everyone took shortcuts from time to time.  The supervisors seemed to be more interested in meeting their demonic production targets than making sure that every little safety rule was followed.  If Vlad hadn't been hurt, no one would have cared.

Now I know what you're thinking.  If you've got enough vampires to work the night shift, and you're worried about a sprained ankle, your priorities could use some fine-tuning.  Agreed, but since I'm already invested in the metaphor now we've got to stick it out...

Recently, Cory Boisoneau and I attended the National Safety Congress conference in Orlando, Florida.  I had the opportunity (thanks to Hilda Koskiewicz of NSC) to host a professional development session titled "Does Disciplinary Action Increase the Risk of Human Error?".  This was a panel discussion where I gave a presentation and then we opened up a dialogue with the panel members and the audience about the topic.  I was very fortunate to have Beth Lay, Manager of Risk Management and Loss Control and Mark Thomasson, Six Sigma Master Black Belt, both of Siemens Energy, as panel members.  Also, thanks to the attendees who contributed great stuff to the conversation.

My take on this, as you can see from my previous blog on this subject, as well as my recent article in the October 2009 issue of Occupational Health and Safety, is that systemic risk can be found in conditional causes of events and non-systemic risk can be found in the action causes of events.  Most investigations focus on action causes, and therefore only on the non-systemic risk elements in an event.  However, as we stress in our root cause analysis courses, the best solutions are often found in controlling conditional causes.  This is because controlling conditions reduces the systemic risk of recurrence to which all employees are subject... not just the single actions of any one employee.

Disciplinary action is intended to control action causes of an event.  It attempts to control worker's choices by providing a disincentive to acting outside the established rules.  However, my experience has shown me that disciplinary action is often ineffective in reducing risk.  This experience is apparently mirrored by many others.  In James Reason's latest book
"The Human Contribution: Unsafe Acts, Accidents, and Heroic Recoveries" page 74 discusses the concept of blame (the act of attributing to which occurs immediately before handing down disciplinary action) in the context of what he calls "the fundamental attribution error", which occurs when we assume that an error is caused by some character flaw, rather than the fact that they made choices based upon past experiences along with information available at the time of the incident.  In other words, the conditions at the time of the event were eminently important to the decisions made by the blamed (and subsequently disciplined) individual.

Disciplinary action often not only fails to reduce risk of future recurrence, it also can actually increase the risk of future errors.  When discipline is applied programmatically (as in Vlad's case above - the supervisors were 'forced' into discipline by their own rules) regardless of conditions at the time of the incident, the action will be perceived as unjust by both the person experiencing it and that person's peers.  This is a nightmare (remember, it's Halloween) scenario for an investigator of future incidents because when the time comes to understand the causes, no one's talking.  No one saw anything.  No one knows what happened.  That's spooky stuff.

This was the point I attempted to make in my presentation.  But I learned that I needed to account better for times when punitive disciplinary action actually does help to reduce risk... primarily when dealing with cardinal rule violations, such as horseplay, drinking, drugs, etc.  Beth Lay of Siemens shared a great decision tree based on the work of James Reason that they created in order to help recognize the difference between an error that resides primarily with the employee or with management systems.  I am convinced that this is the right way to go.  An organization needs guidance to know how to act, and in the future we at Apollo will work to develop our own guidelines to help decide when disciplinary action will be effective without actually adding to systemic risk.

A great article on this subject by Kevin Sharpe of Science and Spirit Magazine can be found here for those interested in reading more.

Have a happy Halloween!  Be safe!  Or be boring like me...I think I'll spend this year on the couch watching baseball.

Brian Hughes

These are typical thoughts when developing a team to investigate a problem.  Who should be on the team?  In today's environment, most employees perform many tasks and are responsible for various activities, or in other words wear multiple "hats" - so how will they react to being assigned yet another task?

RCA investigations are often eye opening, and when performed correctly very effective at preventing problem recurrence.  Most RCA team members take great pride in participating in an analysis, and value the company's decision to include them on such a pressing issue.  Putting together an effective team is crucial to the success or failure of an RCA analysis.

There are usually 2 basic roles in an investigation; a facilitator and a participant.  Here's a look at what these are and the differences between them:

Facilitator - This is the person leading the group.  Ideally this individual is knowledgeable of the incident, but would not be considered the "expert" on the issue.  However, the facilitator should be proficient in the RCA method being applied to the analysis.  Having an education on the corporate process involved in the incident is extremely helpful, but often not a requirement. Managing a team is the responsibility of the facilitator, and the rest of the team looks to the facilitator for direction and guidance on what to do in the investigation.

Participant - A participant is an individual selected to be actively involved in the investigation.  An effective team is generally made up 4-6 participants with different roles.  Industry or corporate experts are usually involved, and offer information on the process and equipment involved in incidents.  Other participants include individuals with direct knowledge of what happened and outside personnel with no direct knowledge of the incident or process.

You might ask "why have someone in the room with no expertise or knowledge of the problem?".  It is important to have someone that has no vested interest in the causes or solutions, other than self-preservation (i.e. employment).  You need someone to ask what experts might say are "dumb" or "ignorant" questions, because those questions are valuable when conducting an analysis.  Thoroughly identifying the causes of an incident include those simple causes that experts might not even consider as they might have already dismissed those as contributing causes.  As the old saying goes "there are no dumb questions" is true when applying those questions to an RCA analysis.

Another attribute of an effective RCA team is the diversity of the group.  We've briefly discussed the roles of the investigation team and what value they bring to the investigation, but having a diverse group both in education and background also helps contribute to an effective team.  You never want a team made up of similar educations and backgrounds because these similarities could contribute to the old "group think" mentality.

I suppose what I'm trying to say is: mix it up.  Expertise is great, but creativity in a group can be just as valuable.

What are your thoughts?  Have you had any successful RCA teams? What were some of the attributes of the team you found intriguing and worth mentioning?

Marcus McCoy - Apollo Account Manager/Investigator

Fundamentally, the Apollo RCA method is a collection of critical thinking principles designed to help you create a strong argument that explains why a problem occurred and shows the logical connection between solutions and the actual causes of a problem.  In an Apollo RCA the logic of your analysis is visually depicted via the Realitychart. The Realitychart also becomes the platform from which effective solutions are identified and the place where the logical connections between causes and effective solutions are evident.  Without a solid Realitychart, a problem solvers' ability to identify solution options and to select the most effective solutions is diminished.

The principles of logical thinking teach us that an argument is valid when the conclusion logically flows from the premises AND there are no other circumstances that would invalidate the argument. A sound argument is a valid argument when the premises are true. Try thinking of your next RCA as an exercise in constructing a logical argument and bringing clarity to an otherwise complex scenario as your primary job.  With that in mind, here are some tips to help you build an inductively strong argument into your next Apollo RCA:

1.     Think of your entire analysis as an argument with the primary effect being your main conclusion and the causes as the premises which support that conclusion.

2.     Be sure that the primary effect logically flows from the causes you have identified. Use Baby Steps (i.e., inserting causes between causes) to improve the logical flow of your analysis.

3.     Be sure your analysis will resonate with your colleagues. If you have to, run it by them - they will point out areas that do not make sense.

4.     Be sure that all the causes are true.  Use robust forms of evidence to validate each cause. Use more than one piece of evidence to increase the certainty for any causes you feel require a greater level of validation.

5.     Avoid vague or ambiguous wording when describing causes such as: inadequate, poor, lack of, or failed. Be specific what those terms mean but don't get too wordy (Extra Hint: use the new "Notes" or "Reference" feature in RealityCharting® version 4.0 to clarify your terms).

6.     Add any additional causes to your analysis that strengthen your argument. Try to anticipate any counter arguments or "Yes, but what about..." scenarios and address them by adding additional causes.

7.     Try focusing on the logic of what your Realitychart is saying, not on labeling the actions and conditions. Label your actions and conditions after you are confident the analysis is sound.

8.     Spend enough time on the analysis so that it describes your problem adequately enough for you to see a number of different solution opportunities.

9.     Ensure your final solutions are specific and detailed enough that they can be easily implemented by a third person by avoiding vague or ambiguous wording.

10.  Follow the four steps of the Apollo method but remember that the "ends" - creating a logical analysis, is more important than the "means" - i.e., how you get there.

What specific tips or hints do you have that have helped you to improve the strength of your analysis or success of your solutions?

Regards,

Mark Hall

Apollo RCA Instructor

Brian Hughes | Vice President

Posted by: Cory Boisoneau | May 28 2009

• How do you, or can you, justify preparing for the future at the current time?
• If you have already begun experiencing shortages of skilled employees, what are some "do's" and "don'ts" you can offer others who may soon face the same problem?
• Do you see your organization being affected by a shortage of skilled employees in the future?

o    If so, are your training and mentoring systems robust enough to handle the transition?  
o    What steps are you putting in place to address the shortage?

I would be interested in hearing your thoughts and experiences.

By Chris Eckert, 5/1/2009

You get the picture right away.  It even works outside of the workplace.  I guess I always knew this intuitively, but it’s never been directly on the surface the way it is now.  A culture is really defined by the behaviors of the members of that culture.  

So why did this make an impact on me?  I’m a sucker for specifics.  I don’t have a lot of value for categorical phrases, such as ‘inadequate maintenance’ or ‘wrong procedure’ because phrases like this don’t provide me with the specifics I need to put an effective solution in place.  I always have to ask a follow up question, such as: “Why was the maintenance inadequate?”  The same is true with a word like ‘Culture’.  What can I do with a word like ‘Culture’?  What can my clients do with a when I tell them they would be better off by developing a ‘Problem-Solving Culture’?

The value for me is the fact that the word ‘Behavior’ quickly gets me to specifics.  What behaviors do I want to change?  What conditional causes allow and encourage these behaviors?  What corrective actions can we recommend to change the conditional environment in a way that encourages the behaviors we want to see?  

Maybe I’m making too much of this – but I’ve incorporated this new way of thinking about culture into my investigations and classes and the feedback has been very positive.  Give it some thought and let me know what you think…

Brian Hughes, 4/17/09

According to the Privacy Rights Clearinghouse there has been 255 million security record breaches involving sensitive personal information in the U.S. since 2005.  26 million personal medical records were breached from the U.S. Department of Veterans Affairs in 2006. In 2005, CardSystems suffered the breach of 40 million credit card holder's personal account information and in January 2007 TJ Stores (TJX) suffered the breach of over 46 million credit and debit card holders account information. The TJX incident alone impacted 100 million accounts to the tune of $94 million dollars.  It has been estimated that the exposure of 46,000 records of sensitive personal information costs an organization on average - $76 million.

So what is being done to protect the public? There is a dizzying array of reports on the topic - each offering various solutions to safeguard sensitive data. There seems to be a reliance on the statistical analyses of industry trends to drive solutions much like a Pareto Analysis. Information from actual data security breaches have been categorized by business sector, type of data breached, proportion attributed to malicious acts, theft, hacking, careless/untrained employee so on and so on. Solutions are then recommended based on the trend data exhibiting the highest percentages or greatest threats.

Is implementing solutions based on industry trends a proactive way to solve the problem of data security breaches or for that fact any problem?  Well....maybe. It depends on how you use the knowledge. If you simply use recommended solutions without understanding what the actual causes of your own internal problems are you are certainly taking a chance that the borrowed solutions may not be addressing the causes of your own problems. In other words if the solutions are ineffective you remain vulnerable to the consequences of another breach.  When you blindly accept solutions from external sources, the risk to you comes from assuming that 1) the borrowed solution ideas are controlling known causes of other problems, and 2) your problems are caused by the same things experience by others.  Solutions generated from data categories used in trending are less likely to be effective than solutions that that are controlling specifically defined individual causes.

However, industry trend data can be useful if you use it in conjunction with your own RCAs to direct the search for causes within your own system boundaries.  Industry trend data can also help an RCA team decide if it has been diligent enough in the depth of its analysis. When conducting a multiple event analysis (i.e., an analysis of multiple RCAs used to identify common causes among multiple mutually exclusive problems) industry trends can help bring focus to the areas in which common causes might be found.

What are your thoughts - is implementing solutions based on industry trends a proactive way to solve existing problems?

Sincerely,

Mark Hall, Account Manager
Apollo Associated Services

Note: The organizations referred to in this article are not clients of Apollo Associated Services, LLC.

Brian Hughes, Vice President - Apollo Associated Services - Posted 18 February 2009

*Note: Peanut Corporation of America is not a client of Apollo Associated Services, LLC or Artemis Investigations LLC.

As the cartoon created by Sid Harris depicts, risk assessment (RA) is about quantifying and describing a hazard which is perceived to have the potential ability to cause harm to those exposed to it should the hazard be released.  Risk management is the actions taken to eliminate or mitigate harmful consequences associated with that risk.

In the paper we describe how root cause analysis (RCA) can benefit the overall risk management process by helping define and quantify risk, understand the causes of risk, and identify effective risk management actions.

One of our premises for this paper is that risk assessment focuses on anticipating events and RCA focuses on reacting to them. Because of this attitude RCA and risk assessment are separate programs led by people from different disciplines with separate training and different tool kits.  

It is our belief that RCA is fundamentally designed to minimize or eliminate risk by reactively and proactively solving problems and removing causes that contribute to risk.  

What are your thoughts?

1)    Are Risk Assessment and RCA separate entities in your organization?
2)    Do you think RCA can be leveraged to help improve the overall Risk Management process in  your organization?
3)    What barriers would have to be removed to more effectively integrate RCA and Risk Management functions?

Or -

Do you think RCA, used proactively on identified hazards, could simply replace the effort duplicated by the risk assessment / risk management process and one could simply rely on hazard analysis and RCA to management risk?

Sincerely,

Mark Hall

Author: Brian Hughes, Apollo Vice President