Just a recent observation. Agile vs. Formal processes in the context of factories - any confusion there?
It's pretty much a given these days in software engineering we accept that we have been misled over the last couple of decades into believing that software development should follow a manufacturing process (see from your own experience the evidence in past failed development projects). Manufacturing implies 'assembly' of predictable software solutions from well designed plans. (I am sure there are more formal definitions you can insert here). For new software development projects this approach obviously fails in practice because its not so easy to plan what you don't know about what will change in the near term future, and new developments require ample amounts of discovery, learning, design, research and prototyping - all of which you are suppose to squeeze-in for free, without impact, within fixed time and resource budgets of formal processes.
Formal processes work only in practice for a small number of organisations where the outcomes of the project are highly predictable at the start, the inputs tightly controlled and the resources highly tuned and available. You can find these in very few established software businesses where the products they build don't afford large unknown variance in scope/technology etc., and can actually be predicted pretty well.
Because software development businesses need to minimize risk, meet customer expectations and a changing requirements etc., you simply can't get away with saying to your customers "it will be ready when its ready" - and herein lies the rub, because in order to have any level of predictability, you need solid planning.
Formal processes and practices were based upon the fundamental assumption of manufacturing, that you could predict how long and how much cost, and what resources any development project would take. Which is part of the reason why its taken years (and much failure) to realise the basic assumption was incorrect for the majority of new development projects today. I don't know about you but I felt robbed! What were they smoking?
We, as software engineers, all knew this smelled to be broken somewhere, but no-one really had a practical alternative to push back to meet business goals until we decided to face the music of failure and agile was allowed to emerge - sometimes out of desperation. The beauty of agile of course is that it simply pays respect to the way most of us were actually trying to develop new software within the constraints of formal processes in the first place, and that puts the natural, empirical progress foremost.
We now, as an industry, accept superior agile processes and practices deal far better to the reality of creative new software development projects (although many organisations are sadly still to make this realization). But is agile the right process to use in all cases?
Isn't the predictable assembly of products, like in manufacturing, exactly the mantra of Software Factories? On the outside, there appears to be a disparity here - or is there?
It appears that we are saying "use factories to create your solutions". Which sounds like factories are implying to take us a step backwards again to formal practices. Is it any wonder the message is being confused?
If you or someone else came to a similar conclusion, you/they are not listening close enough, and must be confusing the case when to use software factories, and when not to use software factories to build software.
You never build a software factory for building a one-off solution from scratch, where you don't already know the solution domain from past experience.
The clarity here is this:
We definitely should be moving software development (as much that is already known) to a manufactured process - which is what factories are all about helping to do, and what the industry needs to move forward. Yes, so, this also means that you should use formal processes, with the factories as part of the solution, to build software in the future! But when we say this we mean that the formal process should be built around the use of factories to assemble the solution.
We are not saying, use formal processes to build your new, one-off solutions. No, for those you should be using agile processes, simply because these solutions are likely to be unique to you and unknown to you at the start (i.e. requires: discovery, learning, design, research prototyping etc).
So, no factories in the solution -> no predictability to the development of the solution -> no formal process -> use agile process.
But, if factories in the solutions -> high predictability in the development of the solution -> use formal process.
This will hold true only for the parts of the whole solution which the factories themselves create. For all other parts of the solution, i.e. the glue, that require discovery, learning, design, research etc, you should employ agile processes to manage that. (Which for most software factory created solutions should be the last 20-40% of the development considering the expectation that factories today can't create the entire solution, and that the solution will always require some level of manual customization).
Now, all this supports the fact that: you should be using agile processes to create your Software Factories themselves. Since you also won't know what they look like at the start and they also require much discovery, design, research and prototyping etc. You can treat a software factory building project as a one-off solution development project in this respect - since it will almost always be a one-off development project.
So, I find it kind of ironic, that whilst the industry, as a whole, is still moving us to agile development, software factories appear to be countering that by moving us back to formal development! Of course, you know better than to confuse the two different types of software development now - right?
In actual fact, if you understand this clearly now you will know that agile development (process) is used to help design and implement new, unknown solutions to meet changing requirements. Whereas, Software Factories (tool) are used to help manufacture existing solutions, each with similar requirements.
