Seeing a software project through the end and making sure it fills the needs that it was designed to fill requires more than just programming.

I had suspected that this was fairly obvious. In fact, it follows almost directly from the principles in the Agile Manifesto, especially valuing “customer collaboration” and “responding to change”. It should be obvious without needing a manifesto at all, though, shouldn’t it? The point of any of software project isn’t to have more software: it’s to have a piece of software that accomplishes a goal. (When I say “software project” here, I’m talking about a project that includes software created to fill a business need rather than a program written to learn, to teach, or for fun. The goals there are different.)

On the other hand, I keep getting surprises, especially since we started Reverso Labs in earnest. Of course, as a tech consulting company, it’s common that we take over a project that another company has already started. We’ve seen some pretty egregious mishaps when inheriting code:

• Major features that don’t work.
• Major features that don’t do what they were intended to do.
• Consultants that cash the check and insist on working in secret. (“We’ll show you when we’re done.”)
• Last but not least, rolling out a stock WordPress site, slapping on some premade themes and some plugins, and calling it a day. (Not that WordPress is bad software, but its treatment as a one-size-fits-all solution is a bad practice.)

The entire point of the project is to address concrete needs. All of the code, all of the design work, everything produced as part of the project needs to address a concrete need. Without constant communication with the client or the customers, the developers fall out of sync, and start producing work that is farther and farther from the concrete needs the project is supposed to address. Clearly, this costs the client money, although (depending on the contract structure) it sometimes makes money for an unethical developer.

The Value of a Good Feedback Loop: A War Story

The value of a properly maintained feedback loop is:

• Time saved
• Money saved
• Effort saved

I’d like to illustrate this with a story directly from the trenches, one of my favorites when explaining the value of a feedback loop.

In 2006, I started work at a consulting company called Bitscribe, a consulting shop out of Burbank. It was one of the most productive environments I’ve ever worked in, thanks in no small part to an insistence from the top level that the feedback loop stay closed and tight. (You may have heard of the very successful startup they created in 2007.) We had a big project called “Harpoon”, an inventory management system. The workflow at the warehouse that had contracted us for this software was fairly tight. Initially, to describe the contents of a truck (e.g., “14 pallets, each with 12 cartons containing 12 pieces of SKU X and 8 cartons containing two pieces of SKU Y”), we had built an attractive UI, with drop-downs and text fields. We were doing weekly demos, so it was only a few days after we built this UI that we discovered the client couldn’t use it. It was far slower for their data entry employees than the old system: they would type, then click, then wait for the JavaScript calls to populate the frontend’s drop-downs, and then click some more.

The data entry team was full of very quick touch-typists. After watching them attempt to use the new system and discussing their problems with it, we replaced the entire UI with something simpler that played to their strength. The new system had no drop-downs: it was just a large text box that accepted a special language created to describe the contents of trucks. So rather than typing “14” in the “pallet” field, then clicking on the quantity field for the “carton” section, typing “12”, clicking the “+” to add another carton, and so on, they would type “14 pallets of 12 cartons of 12 of SKU X and 8 cartons of 2 of SKU Y” or, since the language allowed abbreviations for the common names and omissions where it was unambiguous, “14 p 12 c 2 1 X 8 c 2 s Y”, which looks cryptic but which their team loved. It saved the client money, and it saved us further time and effort spent on developing the fancy UI that they had no use for.

Since the SKU numbers were not “X” or “Y” but in reality larger monsters like “WDG-23-MD-SVLR-3B”, we also added autocomplete. This was back before you could just drop in an autocomplete library (in fact, it was back when jQuery was relatively obscure, and the default tool in the box for most Rails developers was Prototype.js), so it took one of our developers a few days to build. When we did the next demo, this time with the autocomplete enabled, their data entry team didn’t notice it. They typed so quickly that the autocomplete box never had time to show up before they were done describing the truck. So we halted development on the autocomplete system, again saving the client money and saving ourselves time and effort.

The feedback loop allowed us to avoid wasting time and effort, and allowed the client to avoid wasting money on features that they didn’t like or weren’t using.

The Dangers of a Bad Feedback Loop

Losing the benefits of a feedback loop can be disastrous, whether the feedback loop is broken entirely or if information takes a long time to move through it.

Breaking the Feedback Loop Entirely

Without feedback, a project loses direction more quickly than the developers realize. Even in the highly improbable case that nothing in the original plans need to change, a skew develops and it creates a rift between the developers’ concept of the needs the project is supposed to address and reality.

That’s assuming, of course, that the direction stays constant, which is very rare for a project of any scale. There are plenty of external technical changes that a project faces. For example, Facebook changes its APIs frequently and Google Wave folded entirely. And there are plenty of non-technical circumstances that require a project to change direction (especially legal circumstances).

But even without changes forced by third parties, direction almost never stays constant. Sometimes the client changes their mind. Sometimes, once something is built, it becomes obvious that it ought to work differently. The Waterfall model of development rarely results in a good product; after all, as Eisenhower said, “I have always found that plans are useless but planning is indispensable.”

Adding Latency to the Feedback Loop

Adding latency to the feedback loop is more insidious. Even when you have a closed loop, it’s easy to add latency. However, it’s no less harmful than a broken feedback loop.

There’s a natural inclination to be fearful of scrapping something that they worked hard on and some developers are reluctant to throw work away, even when the work turns out to be useless or an impediment to progress. A good feedback loop prevents too much work being done on a dead end (and, perhaps more importantly in the long-term, helps to correct the impulse to hang onto bad code).

More subtly, slowing the feedback loop can nearly kill collaboration between the client and the developer. Ideas that can casually float by when there’s frequent feedback from both sides tend to get pushed aside when there’s more overhead or less opportunity for communication about pressing matters. Small ideas can contain the seed that alters a project fundamentally, the beginning of a concept that makes the difference between something bland and something incredible. Even if no game-changing concepts result, discussion has a way of energizing a team on both sides; it’s great for the client’s morale and the developers’ morale, and it keeps the brains buzzing. So if you create impediments to casually sharing ideas, you’re killing your project’s potential.

A high-latency feedback loop can dry out a project completely. A brittle project fails to respond well to change, and a failure to respond to change very often translates directly into failure for the project as a whole.

How to Keep the Feedback Loop Closed and Tight

When the developer builds something, the client has to know, and get a chance to review it. The sooner you know where the problems are, the sooner they can be fixed, and the sooner something is fixed, the less likely the next set of features are to rely on it.

Keeping the feedback loop closed (so that there is feedback) and tight (so that the feedback doesn’t have high latency attached) is not difficult. It requires a bit of discipline, though like any maintenance task, as long as you keep up with it, it feels more like an asset than a task.

Keep progress visible

At Reverso Labs, one of the first things we do when breaking ground is to set up development and staging environments. We often give them access to our git server, although this depends on the client and the details of the contract.

The client can see what’s being built, live, by looking at the development server. To get a better feel for how everything works when it is put together, our client can look at the staging server, which gets updated when a branch of development stabilizes. This lets the client explore what is going on when they have time, and to put some thought into how things fit together.

Sometimes there’s nothing visible to show, of course. For example, as important as refactoring is, if it changes an application’s behavior (except for the occasional free bug fix that comes from simplifying the code), it’s been done incorrectly. Note that this doesn’t apply, however, to back-end processes that front-end code will end up relying on: there might not be anything graphical that you can show directly, but the back-end has encoded the business logic, and explaining the rules that have been encoded can help make sure that they are representative of the intended behavior.

Communicate

The communication style can differ from client to client and developer to developer. Some developers have in-house tools built around their workflow, for example. Whatever the means of communication, though, there needs to be an open communication channel.

Often, the main modes of communication are email and face-to-face meetings. Make sure you’re engaged and participating (listening carefully counts) when it’s a meeting. With email, IM, bug trackers, or any of an array of project tracking tools or wikis, you have to make sure that nothing sits unaddressed for long. When there’s no response on one end, communication dries up on the other end. Sometimes a given tool turns out not to fit into the workflow that the developers and the client build out together; this is fine, but make sure that there is a useful and usable method for communication and keeping track of a project’s status.

Collaborate

The relationship between the developer and the client is two-way. Although ultimately the client is the driver, developers are hired for their insight and experience, not just to be a pair of hands at the keyboard.

Doing frequent demos is one of the best means of collaboration. It’s as valuable for the developers as for the client. The developers show what they are building, the client gives feedback, and there’s a lot of room for productive discussion. Once a feature is built and demonstrated, it becomes more concrete, and its place in the application is clearer to both parties than it was when it was a wireframe or a use case. Watching someone use a new feature for the first time has an immense value in determining if something is intuitive, if it’s fun to use, or if there’s something critical that you didn’t consider.

Of course, like every method of collaboration, you have to tailor the workflow for the client. Sometimes there’s too much physical distance to make an in-person demo possible, for example. Make sure you work out a system of collaboration.

Why Does the Feedback Loop Break?

Feedback loops break or gain latency because it’s a process to carry out rather than another item in a checklist. So, like any good habit, no matter how beneficial, complacency can set in, discipline can give way to slack, and it can be harder to reinstate a habit that you’re accustomed to breaking than to start a habit to begin with.

There are three big problems to watch out for, so as part of the process, you’ll want to keep an eye out for these:

• Lack of openness. Sometimes progress slows, sometimes there are issues with funding, and in larger companies, sometimes a point of contact leaves their company without establishing a new point of contact. It can be hard to discuss problems with people that you’re doing business with, but a failure to be open can break the feedback loop, among other problems. Besides being a good practice, openness in any business relationship builds and maintains confidence for both parties, and without it, there cannot be a functional feedback loop.
• Complacency. An unfortunate (but avoidable) side-effect of comfort. For example, making enough money that survival becomes a less pressing concern is great for a business, as it lets you focus on longer term strategies, but make sure not to forget how you made the money to begin with.
• Not understanding the value. Someone that has never worked with a tight feedback loop may not understand the value. There’s a gap between knowing that something is a good idea and actually experiencing the benefits. Just hearing something that sounds like a good idea does not instill the same confidence that experience can, and seeing the benefits first-hand leads to a real understanding of the value. Once you have experienced the benefits of a good feedback loop, though, there’s really no going back. Anything else feels like stumbling through business blindly.

Conclusion

Whether you’re a client or a developer, a feedback loop is critical for keeping the project focused on the goal. Not only that, but it saves time, effort, and money, and wise use of limited resources like those allows your business to stay effective.

Collecting and analyzing relevant data is at the very core of productive work, and the feedback loop is an invaluable source of data. In order to plan, the client needs to know what is going on with the software and feedback from the developer is critical. In order to deliver value, the developer needs to know when they’re wasting time on an unproductive path. And for both parties in the business relationship, sharing ideas creates a more interesting and vibrant development environment, which has a visible effect on the end product.

The feedback loop takes effort to maintain, but it’s very simple to do, and there’s an immediate effect when a force multiplier of this kind is applied to the process. Any business, including software development has enough headaches without running blind, so around our office, it’s unthinkable to work without one of our most valuable tools: a tight feedback loop.