Does Agile mean no Processes and no Governance?

There are two generally held perceptions about agile. One, that agile is a kind of anarchist model with no processes, and second, that agile means no governance. Let’s look at the first perception that agile means no processes. We can understand how agile looks at need of processes by applying a well known principle called Occam's razor which states that ” one should not increase, beyond what is necessary, the number of entities required to explain anything”. The principle helps us to remove those concepts, variables or constructs that are not needed to explain a phenomenon. By doing this, it becomes easier to create a model which resembles reality much better and redundancy and inconsistency is removed. In essence, Occam's razor stipulates that when multiple theories are available to explain a problem, the simplest one is preferred. Nature likes simplicity. Ocaam's razor when applied to project management methods would imply that a Project should have only those processes which are required and not more. This is exactly what agile paradigm means with its use of low ceremony, just enough processes. Agile teams chose processes which are most relevant for them to meet their goals in their context. It means you do not pick set of pre-defined, rigid processes in a bunch and deploy them on a project. A process is good as long as it adds value to project; or it should be discarded or amended. So, why is to so hard to agree with this!

It is possibly due to the fact that traditional management methods have relied heavily on process standardization and execution efficiency. But it has not helped achieve predictability of results which is clear from number of IT project failures. Today's business scenario is vastly different with constant change and dynamic markets. Project teams like Organizations need to adapt quickly. Traditional methods, which are based on "execution as efficiency”, find this hard to achieve because they inherently oppose changes in system. In a Harvard business review paper published in 2008, Professor Amy Edmondson identifies a different approach to execution, called "execution as learning" which looks uncannily similar to agile thinking of software development. She calls execution as learning “a radically different organizational mind-set, one that focuses not so much on making sure a process is carried out as on helping it evolve". Agile practitioners would easily relate to her thoughts and “execution as learning” looks very similar to agile manifesto.

Second perception about agile is that there is no or very little Governance in agile methods. But the fact is that agile makes governance more effective as it puts onus on participation of all stakeholders in project. Traditional command and control governance models usually end up creating bureaucratic hurdles and illusion of control. Agile governance model on the other hand is about enabling the team and facilitating an environment where team can meet its objective without avoidable hindrances. Whole team principle is about shared vision and goals and that there are no finger pointing and putting the blame when and if things go wrong. Should project sponsor remain a distant illusive figure or rather become part of the team! Perception of agile being low on governance perhaps comes from principle of "self-organizing teams". Scott Ambler explains this principle when he says that "self organizing doesn’t mean that you are out of control and doing your own thing. It means that team members are themselves deciding how to meet the goals. But the goals they have to meet, the resources used and timeframe, are governed by organization". Agile governance he says is about keeping the baby and throwing the bathwater. And that might well summarize processes and governance perspective in agile.

(First appeared on Capgemini technology blog )

Fixed Price projects - Inconvenient truths by Scot Ambler

In recent times, for some reason, call it bad karma perhaps, I have been part of projects which have had contractual and cost issues. Projects followed BRUF (Big Requirements Up Front) method to accumulate requirements, estimates and pricing were done and contract signed with Customer. The fixed price project was started. And the problems which ensued were typical of so many such projects in industry. They are well documented variousuly, so I am not going to add to that. But I have often wondered why customers insist on agreeing to a fixed price in beginning of a big project when most of times scope can hardly be known. What it does is that, it makes both the parties, the customer and the IT Supplier draw the lines like in a battle and start on a war footing with Sr. Management on both the sides acting like Generals engaged in high strategy. Supplier implements strict change control procedures and become fixated by change management and repelling anything not in original scope. As initial estimates done during contract signing phase can't be relied upon for accuracy, big contingencies are built into estimates but without any surety of its finality. Customer in meanwhile practicaly disengages during project execution waiting for the final delivery to happen as most of his risks are offloaded to supplier. The whole dynamics of a fixed price project creates an enviornment where both the customer and vendor are playing games to protect themselves and not working together to solve the problem for which project is being executed.

Scot Ambler in this articles has articulated what he calls Inconvenient truths about fixed price software development. In fact he goes even further and calls it an unethical practice. Most of points he has highlighted are valid and right on target. Customers have to realise that if they really care about ROI, they need to engage and work with suppliers and not look to offload thier risks in form of fixed price projects. IT vendors or providers on other hand , have to relook into thier own practices which they have been selling to customer for years . As Scot Ambler says "For several decades traditional software engineering theory has told us to do formal, up-front cost estimates. In turn, we have taught our customers that this is the way things are done " . So it is time we teach them that we were wrong.

Going Agile - Just another collection of good practices.

Agile practices for creating software within enterprises have been gaining ground in recent times. Even though shift is not tectonic but subtle, it is important nonetheless. One of main reasons for this has been that many big enterprises have realized that traditional methods have failed them at times. I have seen quite a few big enterprises spending millions of dollars on projects just to realize that projects very often overrun cost and don’t give them operational systems as they would have envisioned.

Do it Small and Detect early: Software projects fail for various reasons and going agile is not the panacea and it doesn't guarantee success either. But what Agile does is that, it reduces chances of a costly failure as progress of project is known to all stakeholders and an early decision about termination of project or change of course can be taken. A constant progress monitoring of project, continuous build and test, small iterations,sprints etc lead to early detection of problems if any. In this regard Agile is akin to RUP like iterative approach and breaking big into many smalls and doing continous health checks on them.

Focuss On People: But most importantly, What Agile also does is that it brings focus on people and project teams and put them in control. Process boundaries are lowered so there are no processes to cover for failures later. Fetish for processes is diluted so that team doesn’t drown beneath cumbersome and often stifling processes. People are the key ingredient for projects to either succeed or failure. Agile project teams are empowered to think and adapt as needed for success of project. Since teams are in control of project, they can not turn around and blame someone else or some process later for failure. Agile makes team members more visible and accountable. In principle, Agile just states a truth which many would want to ignore for sake of industrialization of software that People are most important element of code assembly line and heroes need not be discouraged.

One caution which enterprises adopting Agile has to follow is that they need to make sure that all pre-conditions for agile to work are met. Agile need some cultural changes and mindset adjustments. Agile teams are built around trust and openness to change. Most Agile teams are cross functional with Business and IT skills working in very close cooperation. Enterprises outsourcing their Software needs to vendors and using agile should keep in mind few things :

1.) Put key personnel in charge: Key Business and IT people should be put in charge of project (well, shouldn't that be true anyway ;)).

2.) Accountable Product Owners: Product owners should be people in your organization who can answer questions. Buck should stop with them. Very often cause of project failure is due to volatile requirements or difficulty in nailing them down properly and on time. Especially if organization is big and diverse, getting access to business people who can provide answers is difficult. This should be changed with empowered product owners being part of project team and not just as outsiders.

3.) Build Prescriptive Processes: Agile doesn’t mean no process. It doesn’t mean guerilla programming either. Agile means adapting quickly to changed scenario or context. Instead of a frozen, rigid, descriptive Process, a prescriptive process or Meta-process can be created so that agile teams are operating within a known process framework but with desired leeway to improvise and maneuver.

5.) Model for Offshore Agile: As is the norm these days, if agile team is distributed across locations, put some thought on how multi location latency can be managed. Decide if you would need proxy product owners in remote teams or how cross pollination would be done to maintain one team spirit of Agile. Remember that Agile can be adapted to your specific needs and constraints. Purists would say that distributed teams are not pure agile teams, but real spirit of Agile is adaptability and not maintaining purity.

6.) Agile doesn't mean cheap: Agile doesn’t mean less cost of doing software. In fact it can mean more build cost. Successful agile projects would create value in terms of reducing hidden costs which are usually accrued in traditional methods of tight control and change aversion.

7.) Fixed price versus TnM Get over fixed price mindset. Fixed cost projects don’t reduce your risks anyway. They only delay the costs by converting them into Hidden costs (Change requests, maintenance) .

8.) Not for all conditions: Agile might not be suited for all scenarios. In many cases, traditional methods of tighter control, elaborate requirements in beginning and managing change are required. There is never fit all solution.

Ultimately, Agile is just a collection of good development practices which puts value and emphasize on certain aspects of software engineering. Keep overzealous agile evangelists and those who think it is passing fad at equidistance.