The below is a response to this Washington Post article:
Governments have a fiduciary responsibility to look after taxpayer money. When dealing with the private sector, this is normally interpreted to mean an obligation to issue a tender, and to take the lowest (viable) bid that meets the specification. This works fairly well when the scope of the work is known beforehand, when costs can be predicted, and when a schedule of work can be drawn up.
However, as the complexity of a system increases, making any sort of prediction about that system becomes exponentially more difficult. This means that the specification for a complex system must be exponentially longer than a specification for a simple system, with an exponentially greater risk of errors. Making time & cost estimates becomes exponentially more difficult, and the error in those estimates becomes exponentially greater. The number of unknowns that must be researched grows exponentially with complexity also.
The term "Agile" is a bit of a buzzword, and has attracted more than it's fair share of snake-oil salesmen, but what it comes down to is, essentially, throwing the towel in and admitting defeat. When you *cannot* make predictions about your system, what do you do? You need to find another way of managing costs and reducing project risk.
Unfortunately, because of the fiduciary responsibilities named above, these options are not open to Government contract mechanisms. There is a fundamental conflict that cannot be resolved. As a result, government cannot (should not?) attempt to implement anything other than the very simplest of systems in the traditional, mandated manner.
How then, can complex information systems be developed for public benefit? The philanthropy of private individuals & organisations is one solution; whether through crowd-funding, or open source initiatives. Political leadership and coordination of such activities is something that could easily fall into government remit, without the significant legal hurdles that contracting work out imposes.