Exploring the Agile Idea with Zigurd Mednieks

Kevin Montalbo

Welcome to episode 51 of the Coding Over Cocktails podcast. My name is Kevin Montalbo. And joining me from Sydney, Australia is Toro Cloud CEO and founder, David Brown. Good day, David!

David Brown 

Good day, Kevin.

Kevin Montalbo 

All right. Our guest for today is a consultant to leading OEMs, enterprises, and entrepreneurial ventures creating Android-based systems and software. He is the author of the upcoming book “The Agile Idea, Great management for great code”, currently available in the Manning Early Access Program (MEAP). The book explores the “why” behind Agile techniques, stripping back encrusted doctrine and jargon to reach Agile’s core principles.

Joining us for a round of Cocktails is Zigurd Mednieks. Hi Zigurd, I hope I didn’t botch your name there. Welcome to the show!

Zigurd Mednieks 

That's perfectly fine. Thank you for the introduction, and it's a real pleasure to be talking to you guys!

David Brown

You’ve got a great book there you're working on, the “Agile Idea.” I’ve read its development early release program with Manning. In it, you wrote “Agile is the response to the poor applicability of previous management approaches,” which I thought was a great way to start the conversation. What are those poor project management practices that you're referring to and has Agile addressed those poor principles?

Zigurd Mednieks 

Right. I mean, Agile came out of a crisis in project management. Project management emerged as a field of systems analysis in the 1950s and 1960s and developed a lot of very good methods for doing things, like building the Hoover Dam and the Trident submarines and the space program and things like that. And it worked very well for projects and particularly, [for] projects that are a little bit more mundane, like building ships and building skyscrapers and things like that, because those are projects [are] where you're doing things that you've done many times previously. You're just doing them in a different configuration and with that systems analysis approach, you could do things that are called, for example, a critical path analysis. 

And that's where you take all of the tasks in your project, and you write careful descriptions of those tasks and based on experience, you know what kinds of resources you're going to need to complete those tasks. And you have a pool of resources and you have a pool of tasks and you have some dependencies among those tasks, like you have to put in the wiring and the plumbing before you put up the wall board in a building. And you connect together the dependencies of these tasks and you make sure that you're not overscheduling the resources that you need to complete them. 

And, you know, in a tool like Microsoft Project, it can do automatic resource leveling, which is quite a sophisticated thing in project management. You can come up with a really exquisite project schedule analysis. And when things like Mac Project and Microsoft Project appeared, people started applying these tools to software development and it was way better than nothing. 

But they found fairly quickly that they had to replan their projects very frequently compared to other kinds of projects. There was something about software development that was fundamentally different. That was screwing things up so that every two or three weeks, you had to go through this little critical path analysis again.

David Brown 

Yeah. And you said like, the original project management principles were great for buildings and large infrastructure projects and that kind of industrial age type project management. Software was kind of different. So, they’re not repeatable processes. We’re building something new. Is that why not the same principles are applicable?

Zigurd Mednieks

That's right. So, in software development... Pardon me, while I pour my cocktail. 

David Brown

That's okay. It is Coding Over Cocktails.

Zigurd Mednieks 

That’s right. So, I’ll have a very nice martini.

David Brown

Oh, a martini. Very nice. Even better!

Zigurd Mednieks

By giving up the precision of previous project management styles, what you get is something that's much more effective for software.

Yes, yes. So, that turned out to be the problem. And in that, if you're doing the same thing in software, why didn't you just reuse the software that you already did? So, in software, you're solving new problems every day. You're doing new things every day. Even the descriptions of what you should be doing might be off because you haven't done it before. So, you run into many more surprises in a software development project. 

And there's a really interesting thing about Agile that is underappreciated in the study of Agile and that is when you do iteration towards the goal of project completion, what you're doing is you are allocating your people to tasks in a tactical way. You're not doing this critical path analysis where you need 17 plumbers because you're building a 40-storey building and you need five wallboard guys and all of that. You can't make estimates like that when you're developing software and so what you do [is], instead of this kind of exquisite beginning-to-end project planning, you assign people on a tactical basis to the tasks that you need to accomplish in that iteration. And by giving up the precision of previous project management styles, what you get is something that's much more effective for software.

And  people who graduate from these two week Agile boot camps, they never learned that and it's a very powerful idea. So, if we can impart that idea to people who are managing software development projects, I think they're going to have a lot more success at doing it.

David Brown 

You mentioned those boot camps. So, there is a lot of training, like instant Agile training type resources available out there. What are they missing and how are you addressing those?

Zigurd Mednieks

Well, it's not so much that they're missing things. It's just that they're organised toward the goal of getting you a certification as efficiently as possible. And it does teach you how to execute the Scrum model of project management very quickly. And training is a good thing. Training gets a lot of things done in this world. But once you've gotten that training and you walk into a room full of coders who think that you're just a newbie who's walked out of a boot camp, they might be a little hostile to you and you need to be able to tell them why you're doing the things that you're doing. And that's another reason why I wrote this book.

David Brown

Yeah, so you mentioned that when you have a newbie who has just perhaps done an Agile course, [they] may come into a project, manage a software solution and start throwing around Agile jargon to the developers, which may not necessarily be effective. It's perhaps more important that they understand the principles of why Agile was developed in the first place.

Zigurd Mednieks

...those enterprise software development environments can benefit from stripping Agile down to the principles and then customising it for their particular needs.

That's right. And if you look at Scrum, there's a considerable amount of backlash at Scrum and a lot of it has to do with the jargon because the jargon tends to alienate people who are outsiders to that side, that kind of project management. You know, you have a “Stand up,” you have a “Parking lot.” Those are all meetings. Meetings should all have agendas, they should all have action items at the ends of them that people actually perform. And you know, none of those terms are in the Manifesto for Agile Software Development or in the 12 Agile Principles that go with it. 

So, you know, you can strip Agile down to the manifesto, start from there and build up to things that you need to have for a particular situation. There was recently an article about how project management is done at Facebook. Project management and product management. And really, the product managers focus on being product managers. And product management, it's an elite pursuit in a lot of businesses. In any sort of consumer-oriented business, you're really the boss of the product. You are the CEO of the product and it's your objective to make the best product in that category. 

And project management, those in the “fang,” so to speak, [like] Facebook and so on, large, high-tech companies that's left largely to the engineers, they will do Agile but they'll customise it to their own needs and the way that they work. So, you're not getting the kinds of formal ISMs in those environments that you get in a lot of enterprise software development environments. And I think those enterprise software development environments can benefit from stripping Agile down to the principles and then customising it for their particular needs.

David Brown

Yeah, I'd like to talk about this a bit more, because in your book, you do talk a lot about the adoption of Agile principles within the enterprise and how there's this concept now of “SAFe,” the Scaled Agile Framework, which is a set of organisational workflow patterns implementing Agile in an enterprise. And yet, you don't seem to be a big fan of SAFe. Can you run me through what it is you don't like about that framework and what were you suggesting instead?

Zigurd Mednieks

Sure. Well, if Scrum is getting pushback among software developers and people who are in a project that is run according to the Scrum recipe, SAFe is an enormous elaboration of Scrum. The official SAFe website has this absolutely horrifying five-layer diagram right on the landing page that looks like, you know, the result of an unholy union of many PowerPoint decks in many different enterprises. 

And it's counterproductive because Agile jargon, where the jargon that is accumulated around Agile, tends to isolate project teams. And if you try and push that jargon into all aspects of an enterprise, you're going to get a lot of pushback. You're going to get, you know, a lot of sniggering among people who know how to run that enterprise and know what they're doing and don't really enjoy having a bunch of software guys trying to tell them how to run their enterprise differently. 

So, you know, this is another reason why I wrote this book. It’s that by being able to strip Agile down to the principles and and the fundamentals, and then understanding the way people in an enterprise run their business, that's going to be a much more fruitful way of spreading Agile software development practice through an enterprise.

David Brown

Well, I mean, there are companies which are large enterprises as well as software developers, companies like Microsoft, Google and Facebook presumably have adopted Agile principles in their software development, the same organisations adopting SAFe for their organisational practices.

Zigurd Mednieks

No, notably they're not. So again, you know, they're not even adopting the formal ISMs of Scrum or Kanban. They might be doing Scrum or Kanban in effect. Right? So, they might be doing iterations that kick off for the meeting and end with the retrospective. They might be continuously taking tasks off the to-do list and putting them into their workflow, but they're not doing Scrum or Kanban in a formal way, and much less would they be doing SAFe in a formal way.

David Brown

And this is what you're arguing in your book, [which] is [to] take some of the principles of Agile, but adopt them to what suits your organisation. Is that right?

Zigurd Mednieks

That's right, yes. And this also comes into, you know, that Agile is not perfect. It was written by a bunch of men who ran software projects in the 1980s or 1990s. And I think one of the key projects that influenced XP, extreme programming, was sort of one of the factors that went into the formulation of Agile. That was, I believe, an employee payroll program at Chrysler Motors. 

So, if you imagine, you know, an American car-maker running mainframe software to run their payroll program and managing a project like that, right? That's a pretty retro environment compared to the way software is developed these days. And the Agile principles have endured remarkably well, but they contain clunkers. Like, you know, the software people ought to talk to the business people face-to-face every day and that's faulty in a number of dimensions. 

First of all, every person in a business is a business person and they should be internalising the goals of that business, whether they write software or whether they build people or sell things or whatever it is they do. So, you know, Agile isn't perfect. You need to be able to understand and talk to the people in your business whose job it is to turn capital into profit. That's what senior management's job is. And you need to understand that if you are in the business of creating software in that enterprise.

David Brown

You can see where the confusion comes though. Because we talk about the Agile organisation being fast to respond to market, adopting Agile principles. Every company is now a software company in order to produce the widgets they produce. Software is eating the world. All of these things kind of suggest that now, every organisation is a software company and that you should run it as such. But you're arguing that the business people are quite separate from the software development and their processes should be different as well.

Zigurd Mednieks

Not really. I don't think so. I think that people who create software need to understand the priorities of the people who manage the company and deploy the resources of the company to various goals, including software development, and the people who are in senior management need to have a better understanding of software. And in part, that means making that process more accessible to them. Now, we talked about this a little bit before I came on this program. And one thing I didn't realise was that you guys are in the business of doing API gateways and things like that, is that correct?

David Brown

That's right.

Zigurd Mednieks

Yeah. So, one of the things about SAFe is that SAFe makes an attempt to make software understandable by using UML diagrams and UML diagrams have been around since the 90s or earlier, since object-oriented programming and modeling seemed to be the way to make structures and software understandable to other people. You have this taxonomy of objects and you can say, “Well, you know, vehicles are things that move about and cars have four wheels and some trucks have six,” and so on. 

And that was interesting. But it turns out, that's not really the best way to be designing software and it was a little bit of a naive way of trying to make software understandable to other people. And one of the chapters in my book is trying to find an alternative to that. And if you look at API gateways, API gateways let you do very interesting things like model your APIs before you implement them fully and document them and even create sandboxes for people to play around with them, so that your frontend people can start coding the frontend before people are done coding the back end,. 

It's very useful. But you know, a lot of these things are very approachable as well. If you've got your API designed and documented and you've got a sandbox going that can return sample results, it doesn't really take a coder to go in there and understand what's going on. You can see what the data behind your application is, you can see what parts of the API are going to manipulate that data. You know, the web is made of a piece, it's made of get and put requests and the parameters of those requests. And if you're going to understand what those do, you can really understand the ground truth behind a software system. 

So, there are ways of making software accessible to people who aren't software developers. And, you know, part of the book, once you get past the basics of what's compatible software development, what's compatible with Agile methods of project management, I explore a lot of things that have to do with management techniques in general, like seven-step problem solving, like decision trees, like, you know, what can you do to make software accessible to people outside of the development team? 

So, you can do a digital transformation and you can try to do something like SAFe, which is very PowerPoint friendly, and have a lot of meetings and, you know, people will go away kind of happy about it for a little while until they realise that “I don't really understand how this works.” Or you can get to the fundamentals of it and make the software people closer to management, decision-making and vice-versa, get management more involved in what the capabilities of their software is.

David Brown

I wanted to talk to you more about this, about the boundaries between management and the developers and you brought up the API application. So let's talk about that. So, when you're developing an API, you often get the stakeholders of the API involved, in which normally, business people are involved in that design process. And it is a business-friendly sort of process because the design is about what consumers want, what data they need to see and what sort of produces their reports or analysis or whatever it may be. 

In some cases, it might be another system accessing that API. Let's assume for the time being, there's a business person wanting to consume data so they might get involved in that design process and as you say, the implementation process, once the design is finalised, can be done by the backend developers. So, that's kind of a clear distinction of roles between the business user getting involved in the design process and the developer then going off during the implementation. As I understand it in your book, you argue that that interaction between the business user and the developer should be distinct phases of the project and it shouldn't be ongoing on a daily basis throughout the project, is that right?

Zigurd Mednieks

Not exactly. It's more that the Agile principles’ admonition that you should interact face to face every day is unsustainable and unrealistic. You know, you have to be able to both integrate the way people who are developing the software think with the way the people who are running the business think. But you can't be, you know, talking to each other every day without a framework for what you're going to accomplish by that.

David Brown

And so, the boundaries between what level should management be involved in this Agile process and you giving him some toolsets to get involved, what are you recommending?

Zigurd Mednieks

That's right and that's why software development people need to explore frameworks for interacting with management. A lot of times, management will have very sophisticated financial analysis tools. For example, in the pharmaceuticals industry, there's a thing that's fairly widely used called “Real Options” analysis. 

So, if you're developing a drug, you're not going to know how effective that drug is when you start developing it. And you're going to learn things along the way. And at each critical milestone in a project which is, you know, very applicable to software development, you're going to find out things about how well your project is doing and what the likely outcomes are going to be. And Real Options analysis is a very formalised way of doing it. It's based on financial options and how values are calculated for financial options. 

And if you're developing a drug that costs maybe tens to hundreds of millions of dollars to develop, it's worth getting your MBAs on the task of formulating a Real Options analysis as your development project hits certain milestones. And you might not know for certain what the outcome will be but you can adjust what you think the odds of certain outcomes are. And on the basis of that, you can calculate the value of the project and you can decide to put more money in it or less money in it or stretch it out over a longer time. 

And you should be able to do similar things to software development projects. You might not want to use Real Options Analysis but there is a less formalised approach using decision trees that you can do to achieve roughly the same thing for a software development project. So, as you hit things like the minimum viable product in the software development project and you start to get feedback, you can make decisions about future investment in that project.

David Brown

Interesting. And you also go through some exercises in the book to suggest an organisation using an Agile methodology on the take. And they can ask senior management certain questions such as what would you create or change in a product to displace a competitor from a share of the market. That kind of the thinking that goes on a digital transformation process is getting people to think outside the boundary and think of new and innovative ways to deal with the customers and other stakeholders. Agile and digital transformation, are they complementary? Are they one and the same thing?

Zigurd Mednieks

I think that the concept of “software is eating the world” is really true.

Well, I think that the concept of “software is eating the world” is really true. It's almost an understatement and that everyone is on the road to digital transformation, whether they like it or not. You know, if you make bricks, you probably need software to tell you things about how your bricks are used and where they're needed and how they've performed and how your supply chain works and what ingredients you need for your bricks and how long it takes to get them, that kind of thing. And so, even if you make something as inert as a brick, you're going to be making software and that's going to be part of your competitive advantage.

David Brown:

Hence, why it's become essential for the management to have these tools which have developed which enable them to manage this software development process without actually necessarily speaking the Agile lingo jargon.

Zigurd Mednieks

That's right, yes.

David Brown

In the last chapter of your book, which is currently still being finished off, you have titled it and now you're at the beginning. So I'm getting that, suggesting that the Agile process is an ongoing history of proprietary processes of continuous improvement. Is that what you're leading towards that?

Zigurd Mednieks

Yes. And there's really no limit to the learning that you can undertake to improve your software development processes.You know, the book starts out by laying out the fundamentals of Agile, why it was needed, why it's better than what came before so that you can justify the use of Agile for software project management. And then it goes into certain things that are very likely to be applicable to software development, things like decision tree analysis as you proceed through a project. 

But that's not the end of the road, because every enterprise is different. Every enterprise has different aspects that really can't be generalist. And you have to develop the critical thinking skills to identify those things that are going to be good for software development or not, good for software development in your particular enterprise. So, for example, there are things like statistical quality analysis that are wonderful for manufacturing. That's what revolutionised manufacturing. That's why you can buy a Korean car that is more precisely built than Mercedes, right? Because we had this sort of mythology that Germans made cuckoo clocks and therefore they're great at building cars. 

But really, you know, W. Edwards Deming realised that no, what you really want is to be able to determine whether or not you've got an acute problem in your factory or a systemic problem in your factory by sampling the goods that are coming out of your factory, in determining what the problem is and where it comes from. And that, tightening and tightening your manufacturing tolerances is going to make your manufacturing process more resilient, because you can have a little bit of variation in it. But it's not going to matter because your tolerances are so tight, that whatever variation you have isn't going to make a difference in the final product. That doesn't apply to software at all. You can't do that with software. 

So, you know, there are things like Agile metrics that you should be very skeptical of. You know, the story points are unitless measures. So velocity, which is story points per unit time, is just the inverse of time. It makes no sense, right? So, when you try to do things like that, you're really trying to import things into Agile that don't belong there. They're not applicable to software. “Lean Six Sigma” is another example of a management fad. That sort of metastasized that lean manufacturing was a brilliant idea. People discovered that accounting systems are very bad at finding idle capital that is accumulated in work in progress and parts. And stuff like that sort of accumulates in your factory. And lean goes after those things and finds them and sweats them out of the process. And you can discover millions or tens of millions of dollars in capital that had been lying around. That doesn't apply to software. 

So, you know, your Lean Six Sigma black belts are just a plague on software development if they show up and try to apply that stuff. So, you know, there are things that are good for software and things that are bad for software and only continuous learning and critical thinking skills are going to help you identify which is which.

David Brown

It's really interesting. We've spoken to a number of authors over the last 50 programs on Coding Over Cocktails and a number of those have gone back to the very origins of their discipline. In your case, project management and Agile and how it's evolved critically, what evolved, where it came from, the reasons for it. And it means it's often resulted in more critical thinking in terms of the adaption of those principles in today's world. 

It seems like you've come up with something which is in use by most organisations around the world today. But in this book you may just rethink our adoption of those principles in terms of an organisational project management frame. When do you think the book will be published in its final form? 

Zigurd Mednieks

Well, I'm also an advocate of no estimates, so...

David Brown

You can whenever you feel like it. 

Zigurd Mednieks

That's right. My publisher would like it to come out before the end of the year, but we'll see about that.

David Brown

Well, for those that have a Manning subscription, it is available in the early access program. Zigurd, it's been a pleasure talking to you [about] some really interesting concepts. There are critically some really interesting tools that you developed in the book which management can apply within their organisation. Thank you so much for joining us today.

Zigurd Mednieks

Well, thank you for having me. It was a great pleasure.