Friday, August 15, 2008

The Problem with Thinking Additively

How does your company decide how to improve your products? If yours is like most, you have product managers who survey selected customers, soliciting feedback on improvements. Between their suggestions and your development staff, you come up with a list of new features for that next development cycle.

Maybe this works for your company, but it is a flawed process. By focusing on your existing product and thinking additively, you effectively warp your process into being tool-centric. The Next Great Release is only a few new features away. Developers think this way, product managers think this way, and existing customers think this way (potential customers may not).

So what exactly is wrong with this picture? Everyone is focused on the tool. It is a distraction from what is important, namely solving the user's problem. If you could really see how your tool is used by end users, you might notice that there is a lot of interaction between the user, the tool, and other things.

I'll call this workflow. It is the sequence of steps that the user is following in going from nothing to solution. It might involve using your tool multiple times in different ways. It might involve generating intermediate workproducts. It might involve interacting with others at certain points. Ultimately, it might be all this activity around your tool that determines its success, rat more than its feature set.
Workflow may not be prominent in all problem domains, but the more complex the domain, the more likely that workflow is significant. Consider my company's domain for example. We provide tools to allow technical people to work with network and server data, to build a models of their system, and to predict the behavior of the system under various loads.

Recipes

One day one of our consultants passed by and proudly flopped a meaty document on my desk and informed me that he had finally codified a methodology to solve one of our users' most common modeling problems. It was a masterpiece at 90 pages, was well organized, was valuable to our users, and made an impressive sound hitting my desk.
In a later meeting, while developers and product managers argued about getting SAN support into our tools, I was still thinking about that sound. If the problem domain was that complex, surely we should be selling guidance to our users, right? And we couldn't ship a consultant in every box, could we?

In a sense, the methodology was a recipe, and arguably was as valuable as the tool. If we could productize it, users would see real value in a productivity boost. The analogy to a cooking recipe is apt. It has ingredients (preconditions), a step by step description of what to do (a process), and a view of the expected result. Users might choose different recipes, depending on what they are trying to do.
One of the most valuable things a recipe can do is to establish context. By knowing context, any support you give the user can be made more relevant. Maybe your user has filtered a network traffic report and it is empty. Is that an error condition? Maybe. Is there sage advice to offer them - only if you know what they are trying to do. If you build tools without context, in a way, you can only offer abstract options and abstract advice. It becomes much harder to embed domain knowledge in your tool.

Steps

Steps are the building blocks of recipes. Our product is composed of more than one application, and some steps control these other applications, launching them with the appropriate inputs and outputs when appropriate. Some kinds of steps generate workproducts. Some steps encompass critique of data or state, sometimes encompassing expert advice. Some steps allow declaration of data (like a list of server names). A lot of steps have fixed or parameterized functionality, but one type of step is very highly customizable in terms of both its behavior and its designable user interface.

Flow and Dependency

There is a concept of information flow involved in recipes. Since recipes have multiple steps and steps might have inputs and outputs, there is a flow among the steps. The inputs and outputs of each step are typed and the flow matches up by type. This brings up one of the strengths of this approach, namely the managing of dependencies. If a user makes a change farther up in the recipe, all downstream steps that share that flow are now out-of-date, meaning that the steps need to be executed again to bring their outputs up-to-date. As long as this out-of-date state is visible to users, it is a simple matter of making sure that required changes propagate correctly.

Tracability

In our problem domain, users are making business decisions on modeling results, involving potentially millions of dollars. Confidence in the results is paramount, requiring an understanding of how those results were arrived at. If the results are unexpected, they need to work backwards in the flow to figure out where their expectations diverge from the results. Users might work backwards examining not only intermediate workproducts, but also look at assumptions along the way.
Consider this requirement in the context of a loop. Whether in a conventional programming language or a flow chart, a loop has a conditional, a loop body, and a loop terminator. There are undesirable characteristics of this representation. First, it assumes that the logic associated with the loop body is the same for each pass through the loop. When you consider that each pass might represent the analysis of a different set of data, and that the user might want to tweak each pass, the flow chart style logic is not flexible enough.

A way to represent this is to unroll the loop. That is, represent the loop body explicitly for each iteration of the loop. It takes more space to do this certainly, but it meets the traceability requirement and allows the user to modify each pass independently. There is also a dynamic reconfiguration aspect here. Maybe the loop is a pass for each test case in the collected data. So if the user adds a test case, a new series of steps might dynamically come into existence representing the new pass through the loop.

A final benefit in explicitly representing all steps required in a recipe, is that the user has a roadmap that tells them how far they need to go. They can tell how far they have come by a "done flag" that each step displays.

Business Politics

An easily overlooked issue with introducing workflow into your products involves business politics. You might have to champion a fairly sizable change to how you are going to spend your development dollars. If introducing workflow competes for the same resources that would be used to develop new features, you might meet resistance.

In my company, I started the sales pitch at the customer advisory council level. Being composed entirely of our biggest customers, the council readily bought into the concept. This should be expected, as users are the ones who ultimately pay the price for lack of workflow support through lost productivity.

Next up was an internal presentation that included developers, consultants, and product managers. The development staff was positive towards the idea. The product managers were uncertain about this; the workflow solution came out of some creative thinking in development, which is backwards from what they expected. The really interesting responses came from the consulting staff, who are effectively very advanced users. Most were favorable to the idea, but consultants have a history of being mavericks. As subject matter experts, each might have their own way of solving problems.

One consultant stood up in this meeting and gave the most negative response, saying something like "Tell me why I would ever use something like this. Why would we build this application?" I'm happy to say that the same consultant caught me in the hall about a year after the product was released and told me, "I don't know how I got along without this!"

I'd like to close this with an admission. To this day, I never have read every page of that original methodology. Just don't tell that consultant.

No comments: