Why process capture is important for unstructured processes

This article on BPM.com caught my eye yesterday, “Will standard processes soon be extinct?“, particularly the comment from Software AG Chief Evangelist Theo Priestly. Citing Jim Sinur‘s ‘Dark Processes’, Theo recommends exposing these hidden processes and encouraging them. Based on my experience I’d have to agree.

In Theo’s words:

“Jim wrote last year about Dark Processes, those which lurk around the enterprise conducted by many but defy definition.”

One of the problems with undefined and undocumented processes is often they lack the benefit of process improvement experience. And let’s not forget that process improvement techniques have been incredibly effective over the past century in manufacturing and more recently with Business Process Management Systems automating, and ultimately speeding up highly repetitive and high volume transactional work.

What this article, and many others, are discussing is the limitation of process improvement in the modern World and what happens next. In my experience the one area that seems to continually get left out is the human element. These so called unstructured and dark processes are being orchestrated by humans. These are experts in their field, they have experience and skills that allow them to adapt the way they work to the changing environment. They know when to employ a standard process and when something needs to be created ad hoc.

The thing that most of these workers are not, are process experts and therefore the processes they operate are not as efficient as they could be. I find in these areas people are continually bumping into each other, duplicating work or, worse still, getting things wrong.

Simple process capture workshops can be very useful here, the objective is not to create some perfect standardised documentation. The purpose is to get those working together in the same or related areas and to discuss how things are done today. Not tomorrow but right now. In an ever changing environment it’s easy to forget exactly what your colleague is doing, what they expect of you and what you expect of them. Running regular but short and simple workshops keeps people aligned without hamstringing them with over simplified and rigid processes. It allows them to discuss real and potential issues and how to solve them in the immediate future.

I’ve had this experience recently with agile development teams. There’s a high level 4 or 5 step process but what happens in each of these steps is different in nearly every iteration. After every third or fourth iteration the retrospective is run as a process workshop. The process is torn up and re-captured every time with a focus on what went well, that we’d like to do more of, and what didn’t work, that we need to get better at. This allows the team to openly share ideas and improvements and try them out in a highly agile environment.

It doesn’t make people process experts but it acknowledges these so called ‘unstructured’ processes as being essential. It also gives a checkpoint and an opportunity for continuous improvement.


The Selfish Community

Social Collaboration

Like many I have found the power of social collaboration on projects such as Linux, Wikipedia and many others awe inspiring. As described by Clay Shirky in “Here Comes Everybody” modern technology has taken away the friction from connecting with people, based on a common interest, and working toward a common goal. Creating communities and groups has never been easier, publishing information and thoughts is as simple as pushing a button.

Social in the Enterprise

But in recent years the focus, for many of us, has been on how we can apply these models  in the enterprise. How can we harness the seemingly awesome power of the social community to improve the way our businesses work? McKinsey report that:

by fully implementing social technologies, companies have an opportunity to raise the productivity of interaction workers—high-skill knowledge workers, including managers and professionals—by 20 to 25 percent.

So far social networking has been most effective, and to some destructive, in the marketing space. Social savvy marketers have used these networks to raise awareness and drive traffic. At the same time many businesses have stood by powerless to influence these social networks one way or the other.

But what about the promise of self organizing groups within the enterprise? Getting work done more efficiently, providing value and adding new intellectual property where it hadn’t previously existed? This should be easy right? We already have goals and objectives, teams and organization, social media should just make that easier.

There still appears to be a great deal of skepticism surrounding the effectiveness of social media tools in the workplace, specifically Enterprise Social Networks (ESN) built especially for the workplace. Vendors provide services and direction to help embed these tools and build a case for return on investment. Some commentators argue that the grass roots movements, that drive some of the internet’s most successful mass collaborations, are not possible in the enterprise and that you need to provide senior management direction and organize social collaboration around a specific project.

Using the Enterprise Social Network

Here I wanted to share my own thoughts and experience using ESNs. Many of my colleagues have been generally skeptical about the use of these tools, arguments, or rather excuses, range from; I don’t have time, there’s nothing in it for me, I miss things, there’s too much information, information gets lost, the tool doesn’t support a feature I want, etc.

For many of these I think the problem lies in the fact that people have possibly been oversold the value versus effort. They expect to login on day one and get value from it straight away. Individuals must invest valuable time in the network, valuable time that is being pressured from every direction. ESNs are not like other software tools where the value is easy to define, take a word processor for instance, the choice is to write something by hand/typewriter or use the new tool.

By contrast the argument “better collaboration” or “better networking/communication” is a difficult thing to place value on, especially for workers that do not use social tools regularly in their personal life or don’t believe they necessarily have a problem with “collaboration” or “communication” today. And yet I personally have found these tools incredibly valuable, in one company I used a micro-blog to gather feedback in an open forum. Feedback was asynchronous,  fitting in with other’s schedules, there was little duplication and it was focused.

That company was acquired by a much larger organization that also used an ESN. And this, for me, was when it really got interesting. We suddenly became a very small corner of a larger city. Using the tool I steadily built up a network of connections through brief conversations and comments by taking part in open conversations. I gradually got to know people and learnt about projects that I simply wouldn’t have known about any other way.

This benefitted me in the long term as when I needed help or advice in a particular area I knew where to find it, instantly. That conversation may have taken place over the phone or on instant messenger but it wouldn’t have taken place at all had I not made the connections in the first place.

And for my team there were benefits, aside from improving my own productivity, as we integrated, sales opportunities arose across the larger organization. People that normally would have had to work to find contacts in our team knew me already. In the first six months I brokered many conversations between our own sales team and those of the larger business.

Focus On Your Own Objectives

There are many other examples I could give but the value to my team grew out of my selfish desire for self-improvement by improving my own network. This was my reason for using the ESN and the reason I continued to use it. Self organized communities spring up out of chaos as the selfish needs of many are closely enough aligned that they can work together. It is not necessarily that they are working toward a common goal, they all have their own objective. The product of this effort is the sum of these disorganized goals and objectives.

By contrast, in business, workers are working toward common goals and objectives and they’ve been doing this since people organized in order to turn a profit. Trying to sell the benefit of a social network as improving work through collaboration to an experienced business person is pretty weak. The way I have sold the ESN to colleagues is purely on a selfish basis, use it for yourself and get your own benefit from it, if you don’t engage it’s only really you that will lose out. The more people individually benefit from it the more value the organization will realize in it.

Can Process Improve Communication?

It is with some frequency that I am presented with inaccurate process documentation. The degree of accuracy varies but what’s interesting is that the degree of accuracy varies depending on who you ask. One version of a process, involving multiple actors, can range from a perfect representation of reality (impossible!) to unrecognizable (probably closer to the truth) within a single team!

Keith Swenson, referencing Cognitive Nueroscientist Michael Gazzaniga, describes how process descriptions can be full of inaccuracies because of the mind’s need for coherence. We give more weight to a coherent story than an accurate one. As a result, when interviewing process participants, they are as likely to fill in the gaps of their knowledge of the process with something that makes sense than what actually happens.

In his book Thinking Fast and Slow, Daniel Kahneman describes the human mind as two systems he calls System 1 and System 2. System 1 is your intuitive mind, it is the part of the mind that reacts instantly to sudden events, it is in control when we are completing simple tasks, it recognizes faces, sounds and smells. Often our intuitive minds fly solo, allowing us to complete tasks on autopilot without much thought or concentration.

System 2 is your rational mind, the part that calculates mathematical problems, the part that works through complex puzzles and formulates thoughtful responses to problems and is used in situations where high levels of concentration are required.

Our intuitive mind makes assumptions based on past experience, norms and memories often without consulting our rational mind. According to Kahneman our rational mind is lazy and allows the intuitive side to create these assumptions without question. When an individual describes a process, of which they have incomplete knowledge, the intuitive mind fills in the gaps as it would expect to do those steps in the process it does not know. This feels natural to us, when our rational mind is at rest, the coherence of the described process feels right and the rational mind is not called in to check the validity.

This is why I prefer running collaborative workshops involving multiple process actors rather than individual interviews. Watching them attempt to fill in the gaps for each other can be quite colorful. Process workshops force disagreements and a search for consensus and bring our rational minds into play. It forces us to both mentally and verbally question what we see and hear. While this allows, to some extent, the discovery of the truth it has another powerful effect, it creates a level of mutual understanding between players that improves communication. The act of talking through and visualizing a process can often, in my experience, actually improve the effectiveness of that process without any attempt to change it. Process improvement suggestions are common in these sessions but they are a by-product of improved communication.

The Real Inefficiency?

In my last post I outlined how concepts of product usability might be applied to the work place as Business Usability. I want to expand the idea of the conceptual model and how the modern knowledge worker is often at a disadvantage compared to their historic colleagues.

In everything we do there is a conceptual model and a mental model. The conceptual model describes how something actually works and the mental model is how we ‘think’ something works. The success in anything working properly for us is when those two models converge. When our mental model matches the conceptual model things are obvious and easy to do.

In many workplaces the conceptual model of how things work is embedded in the environment. Think of a train station, there is a ticket desk, a platform and a train. You go in, you buy your ticket at the desk, you go to the platform and then board the train. There are sometimes signposts to help you in larger stations. Here we know what the steps are because the environment, and our cultural knowledge, dictate this. Some stations/airports work better than others.

Think how relevant this is in manufacturing environments, production lines, where every machine and person has a place, the trigger is the incoming widget the output is the outgoing widget+. It’s easy to train people, and for them to sustain their understanding of what to do because their environment provides so many visual indicators. They don’t have to remember everything only enough to mentally piece together what they actually need to do. As humans we use this type of construct to save us having to store information in our long-term memory. For the vast majority of us long-term memory is incredibly inefficient, you know what it’s like when you try to remember the name of ‘that’ song or the author of ‘that’ book. What we are very good at is piecing together, or constructing, memory based on recognizable identifiers.

Now consider the typical office worker, or knowledge worker, where the environment provides no clues as to what happens next. A computer, desk and a telephone. Without these traditional indicators we have become inefficient as we have to work harder to piece together the steps required to meet our goals. We must rely heavily on long-term memory and spend a good deal of time looking for information, discussing with colleagues and guessing at the steps we need in order to get there. This is, perhaps, one reason why process improvement methodologies have not been as successful in the information domain as they have in traditional industries.

As we spend so much of our time staring at the computer screen it seems sensible that these identifiers should be embedded there. There are many software apps available that do a good job of providing the right amount of information at the right time to help you understand what you are supposed to do when you are using them. But outstanding user interfaces are still in the minority and they are generally limited to the activities for which they are used. They do not provide clues that lead you to them in the first place, nor what you do after you have finished using them.

What is required is a common platform that provides just enough information at the right time to the help user achieve their goals. Detailed procedures or complex flow diagrams are not ideal; they require examination and concentration that takes time away from real work. These new ‘conceptual models’ need to be instantly accessible and, where appropriate, interact with the underlying tools and applications that are required in order to complete the process.

Perhaps the biggest inefficiency in modern business is the time workers spend trying to piece together what they are supposed to do.

Make Your Business Usable – Business Usability

Your systems are usable right? They have great user interfaces that are a pleasure to use, everyone loves using them. But what about your business’ usability? For the end users, your employees, how usable is your business? What is the user experience like? If the user experience is bad workers are wasting their time trying to figure how to get work done. If the user experience is good workers are spending their time innovating and delivering the organisation’s goals and objectives.

So how does one make business ‘usable’? In his book, “The Design of Everyday Things”, Donald Norman sets out some simple principles for designers. 1) Provide a conceptual model, 2) make features visible, 3) map actions to features and 4) provide feedback. A conceptual model should provide a framework for what happens when and why. Sound familiar? The conceptual model for business is locked up in the process documentation, manuals and procedures that are common place within most organizations. The problem is that for the conceptual model to be effective it cannot be too detailed, overly complicated or simply unrecognisable to the user that needs it. And this is exactly the problem with much of the process documentation that is captured today. It is not created with the end user in mind unless the end user is considered to be a system or analyst. Care is not taken to provide just the right amount of information to the user completing the tasks.

Humans are intelligent creatures; they don’t need to be told exactly how to perform an action every time they do it. They need a reminder, a framework in which they can place themselves and act to produce a desired outcome. Overly detailed and complicated documentation will simply serve to confuse and frustrate the user. Thought must be given to the level of information required for that person to perform their tasks efficiently. It’s probably less detail than you think. Some areas may need more detail than others. Some processes may be highly regulated while others are completed infrequently or by unskilled labour. There’s a chance that these will need more detail but even then don’t overwhelm the user.

Make the features that make up your business, visible. These are the process documentation, the various systems and tools required by users to complete their work. Think about where the documentation is held today, some in the BPMS, some in the training system, some is on your hard drive and in various intranet and shared stores. Creating a portal and listing everything in it does not make it visible.

The conceptual model needs to be delivered to the user in context. It needs to be easy to locate. Some form of tagging is useful based on a taxonomy that is easily recognisable to the users. When the content is delivered to the user it should not just reference the system to be used it should link the user directly to it. This is about mapping what needs to be done to the tools required to do it.

Finally provide feedback to the users. Instant feedback should be provided when systems are launched, when workflows are kicked of or when data is committed. Holistic feedback should be given through real time metrics displayed in the context of the process.

I believe that improving your organization’s usability is perhaps one of the most important things you can do with regards to efficiency. It also supports the other two aspects of ‘social’ I described in a previous post. Combining all these approaches can make BPM an essential component of your social strategy.

Does social technology change the As-Is versus To-Be debate?

In a previous post I discussed how Business Process Management can, and should, be a social experience even without modern social technology. One reader sent me the following message:

 I think [you are] refering to ‘to-be’ processes. Someone still may have to describe today’s reality and sometimes just a good BA and a few knowledgable SME’s gets this job done far quicker, accurately and without too much ‘socialising’.

I don’t wish to add too much to the As-Is versus To-Be debate here except to say that every situation is different and will require an appropriate approach.

I do believe, however, that modern social technology is changing the way we approach this debate. With the new crop of BPM tools available the line between As-Is and To-Be becomes somewhat blurred. When capturing As-Is process, using a simple goal driven approach, the deficiencies in the process are often painfully clear to those present. A good Social BPM product should allow the facilitator to instantly capture those issues, and improvement suggestions, against the point in the process where they were raised. This means, to some extent, that you are simultaneously capturing As-Is and To-Be at the same time. Even if you are proposing a radically new process you can capture potential issues and essential steps during the As-Is capture.

A team of knowledgeable Subject Matter Experts (SME) could undoubtedly capture today’s reality quicker without the interference of end users. But is this the best way? Firstly you are excluding the very people that have firsthand experience of the execution of the process. There are often issues experienced during the everyday execution but there is no forum in which to express these. And if these do exist they are disconnected from the context of the process.

Worker engagement is key to the adoption of any new or improved process. Clearly you cannot invite every worker to a workshop, although I have seen attempts to do this! But you can invite some key influencers and of course the platform should allow everyone to comment on the As-Is as well as being able to see the other comments captured during the workshop.

I would argue that the long term benefits of worker engagement in understanding and improving the process is worth more than a quick capture that ends up on an intranet and never looked at.

Making BPM Usable

This is where I believe social can have one of the biggest impacts on BPM. I’ve heard the following phrase, or variations thereof, many times recently. “Social network platforms have become so ubiquitous in our personal life that people now expect them in their work life.” While there are undoubtedly great benefits to adding this capability to the work place I believe there is more to it than that.

Donald Norman’s seminal work ‘The Design of Everyday Things’ was written over two decades ago and explains in very simple terms how life is made overly complicated by poor design. He advances some very simple principles that help designers create products that are easy to use. Yet many software vendors, especially of the enterprise variety, continue to create products that require advanced knowledge in order to complete simple tasks. There is a multi-billion dollar software training industry that exists to close the gap between poor interface design and helping workers complete tasks.

Products like Facebook, Twitter and LinkedIn are easy to use. Some are better than others; each is a variance on a conceptual model which is becoming culturally recognisable to a larger and larger proportion of society. So when software vendors start to bring this type of technology into their own products they cannot, or rather should not, make them more difficult to use than what people have become used to. A majority of BPM vendors, that have released social enabled platforms, have taken care to preserve those ideas of usability and in many cases even begun to improve the underlying product.

This is a positive and constructive improvement to the industry at large. It will improve adoption by the end user community and help acceptance of BPM as a whole.To be ‘social’ BPM must become more inclusive, to be inclusive it must be accessible and that means better usability than most workers have traditionally had to endure.