The Power of Continuous Software Delivery in Defense with Bryon Kroger

[00:00:00] Bryon: Every engagement I believe has to start with a path to production. So if you're out there listening and you want to fundamentally transform software, there's a huge draw to start with alignment activities, things like scaled agile framework or whatever, to like, how do we align on value and come up with the right requirements and the right plan and design our org and all of these things.

[00:00:20] Bryon: Big things. I would say, just figure out how to continuously deliver lines of code first. I don't even care if the code is important, because the second that you create the ability to ship code on demand, has value in and of itself,

[00:00:34] 

[00:00:53] Bonnie: All right. It's another beautiful day in the Washington DC area for me. I'm Bonnie Evangelista. I'm with the chief digital and artificial intelligence office. I'm joined by Bryon Kroger. Bryon, thanks for joining.

[00:01:08] Bryon: Thanks for having me.

[00:01:10] Bonnie: Where are you calling from?

[00:01:11] Bryon: I'm in sunny Tampa, Florida right now.

[00:01:13] Bonnie: Oh, so it is a beautiful day for you too, right?

[00:01:16] Bryon: Always, All right. So Bryon, I've, You know, loosely followed some of your work and what you've been doing, especially in the software community. I think there's a lot of good conversation that you're going to bring to the table today with regard to the software development and actual delivery of new software.

[00:01:35] Bonnie: Into the field essentially. So can you give us a quick rundown on your background and how you got to where you are today, what you're doing today?

[00:01:44] Bryon: Yeah, absolutely. So I commissioned in the Air Force in 2009, I served for 10 years before I got out. The 1st. Seven years were in intelligence. So I was an intelligence officer. I did primarily targeting operations. And I like to tell people for all seven of those years, I used really terrible software.

[00:02:04] Bryon: When I joined the military, I found out I was going to be an intelligence officer. I was like imagining a movie scene and I showed up in my email barely. And so, Very familiar story for most who have served or been in and around government, but it's really frustrating for me. And I will say, you know, doing targeting operations, you're working on some of the most critical missions, right?

[00:02:26] Bryon: Sometimes I would support special operations forces in like, Counter insurgency operations. And when you're doing those missions we have some of the most advanced hardware on the planet, but then back in the operation center, we're working with some of the worst software and the software actually becomes the limiting factor for a lot of our operations.

[00:02:48] Bryon: And I saw software literally cause. Missions to fail, you know, bad guys to get away or our inability to save people who are in harm's way, troops in contact other types of missions. And then, you know, sometimes innocent civilians end up in the crosshairs too. And after watching that happen for so long, there was a particular operation that, you know, I won't go into the details on, but I firmly believed that software was the reason that a bunch of civilians had died.

[00:03:20] Bryon: And I applied for an acquisition Intel exchange. Which was the first year it had been opened up since I joined the Air Force. The Intel was critically manned, so we were never able to really go to acquisitions, but I went and became an acquisitions officer. I pulled a bunch of strings to get myself assigned to the targeting program office at Hanscom, where they made my software.

[00:03:39] Bryon: And my idea was like, I'm going to go fix this software myself. And I showed up that day and I called DIU. I had a friend that had just got assigned to DIUx and I called him and I was like, Hey, I heard you have this thing called an OTA. I think I'm going to need one. And that eventually linked me up with Enrique Odi as well as the Air Operations Center material leader.

[00:03:59] Bryon: And that was the start of Kessel Run. So the three of us co founded Kessel Run um, and I served as a COO for two years. And then I got out and started Rise Aid so I could help other people replicate the success we had at Kessel Run.

[00:04:11] Bonnie: Well, before I do a little bit of digging on Kessel Run, when you went to Hanscom and you're like, I'm going to go back and I'm going to fix what I saw. What did you find? Like, what were some of the rocks big or small that were, that you thought maybe? Created the impact that you saw in the field.

[00:04:29] Bryon: Yeah, there were a number of things. I think that the number 1 is just a fundamental disconnect between the acquisitions community and the operations community. They don't talk. They don't talk well. You know, the requirements process, I've been on the other end of it where I would fill out a requirements spreadsheet and I never really knew how that got to the program office.

[00:04:48] Bryon: But, you know, that went all the way up through the Magicom staff, up to headquarters, up to the JROC, right? Like a joint requirements council as part of the CJ chairman of the joint chiefs before it makes its way back down. And by then most of my requirements, if they were even there you know, we're in a mutated format that were completely disconnected from what I originally wanted.

[00:05:07] Bryon: And my need had changed by now. Right, because what I found out was we were just starting to deliver on requirements that have been submitted almost six years ago. And also we had critical fixes like known operational issues that were causing operational risk that were sitting on the shelf waiting to go through the ATO process.

[00:05:28] Bryon: So between the requirements process and the ATO process I just knew we had to change something. And you know, the first one we solved via the OTA. And the ATO process we might get to, but you know, really one of my personal passion projects was continuous ATO and making it so that we didn't have finished software sitting on the shelf, waiting to go through a risk reduction process while causing operational risk.

[00:05:53] Bonnie: Is that kind of part of the legacy or foundations of Kessel Run was the CATO process.

[00:05:58] Bryon: Definitely. I mean, there's a lot of things that we did that were required to actually get software into the hands of users, but I think that's a pretty fundamental one.

[00:06:07] Bonnie: So what spurred the idea or the movement toward Kessel Run? Specifically. Cause I think in broad strokes you're describing, okay, you saw this need and you had to do something, but it takes a lot to start something new in the department. So that's kind of what I'm driving toward. Like, what, how did you get the momentum, the resources the right people, right, to do what happened.

[00:06:28] Bonnie: And then if you can also just give a little background, like what was the impact of Kesselrung? Like, why was it influential in the department?

[00:06:35] Bryon: Yeah, so maybe I'll start there, right? I mean, by the end of the first year, we had delivered an entire set of software to the end users. We'd actually delivered that in about 120 days. And then got to a point where we were delivering software updates on a weekly basis. Just a regular cadence. Like every Thursday we're shipping software not delivering it to the government, like delivering it to the end user out in the field, how you do their base in Qatar.

[00:07:01] Bonnie: and this is while you're an acquisition officer, right? Yeah.

[00:07:04] Bryon: technically I was an acquisition officer. I never did any of my training, which became a hot button issue Anscombe but yeah, so I would say a few things that, Were unique. And I have to give credit where credit is due. D. I. U. Had already they were D. I. U. X. At the time they had already done an experiment.

[00:07:21] Bryon: The first project was called the tanker planner project. And a few things that Enrique did that were very unique is he used airmen. So he called everybody he knew in the Air Force. And it's like, Hey, if you've got anybody that knows how to code, knows product management, Yeah. Please send them TDY to me. And he paired those people with industry at a place called Pivotal Labs, which I've modeled my business off of Pivotal Labs. I think there's something fundamentally unique that we should talk about at some point, because I think they gave us an insight into what a new model for systems integration could look like in a modern software driven world, like the role of an SI needs to change.

[00:07:59] Bryon: And I think they were pretty close to nailing it. And then so you've got these airmen paired with these industry professionals, like literally paired, like two people working on the same computer terminal two mice, two keyboard, but one, one CPU and they're coding. Not only that they're going out to the field, right?

[00:08:18] Bryon: So had a huge TDI budget and we would send these people forward for sometimes two, three weeks at a time. We also had an LNO that was permanently embedded out at IUD airbase in Qatar at the AOC. And. They would ship software, go out, watch users use it. Sometimes they would make changes on the floor. We had like a dev station set up for them in the back of the chaotic there.

[00:08:40] Bryon: And then they would come back home and for the next two months had a completely different perspective on what they needed to build next. So how did what they built work? What did they need to change about it? What do we need to build next? And they always, I mean, we would send people out on constant rotations.

[00:08:57] Bryon: We always had somebody out embedded with the user. And so even if your team didn't have somebody out there, there was an LNO always out there, and then a couple other dev teams that were probably out there that could gather information for you tell you what you needed to know. And I think that is really important.

[00:09:14] Bryon: Is that constant touch point with the users? And then another thing that was unique was the ability to continuously deploy the software. So

[00:09:22] Bonnie: Yeah.

[00:09:22] Bryon: I would have loved it if the Air Force and the DoD had the infrastructure for us to do that, but we had to build it ourselves. And we were able to successfully do that.

[00:09:29] Bryon: And then that continuous ATO process allowed us to continuously ship those updates.

[00:09:35] Bonnie: Yeah, I'm really interested in the delivery model you're describing. So what did that look like from a I built the software or I have new code or I have something to push. And did you create a pipeline? Like what to, in order to kind of do the compliance checks and then in order to get it out to the field, what did that look like for you?

[00:09:54] Bryon: Yeah. So you have to architect for continuous delivery. So there's a few things that And these are getting lost in the space right now. I see us kind of going backwards almost in the. What I'll call the cloud native platform space and path to production. So your development environment needs to be in parity with your eventual production environment to the greatest extent, practical.

[00:10:18] Bryon: And so that requires a lot of forethought, especially when you're pushing to secret or in our case, it was a secret rel five I environment, which is a pretty constrained environment. So, we worked backwards from our end user production environment and built a development environment and a test staging environment that allowed us to perform a lot of risk reduction during the development process itself.

[00:10:40] Bryon: Then there's things we built into the process, the pipeline. It's super valuable, but it's just one small piece of what needs to be done to reduce both functional risk and security risk. And so, some of the things that we did, right, I mentioned we had paired programming, which is a really great way to provide security.

[00:10:59] Bryon: Two person integrity on every line of code. We did test driven development. So that's where you write the test before you write the line of code. And then you write the code to pass the test. Not only does that lead to well tested code, which you obviously want, it also means you don't end up with code sprawl.

[00:11:16] Bryon: Code is a liability, right? I only want as much code as I need to solve the user's problem. Nothing more, nothing less. And so it really constrains the code. So you don't end up with software bloat, which is not only a functional or usability problem from a user's perspective, it also provides a larger attack surface, just all these things that and it is what it is.

[00:11:36] Bryon: When you have. Efforts like Kessel Run the shorthand is always going to be problematic, but there's all these little details that if you don't pay attention to them I think a lot of the community has ended up cargo culting. I don't know if you've ever heard of this concept of cargo culting, but after one of the wars we were involved in, we were on a small island and there were a lot of natives on the island and we would bring in cargo planes.

[00:11:59] Bryon: We used it as a resupply line. And When we left came back years later and saw that these people had tried to recreate what we were doing because they viewed it almost religiously where they thought these like cargo gods were bringing food to their island. And so they recreated our flight lines.

[00:12:18] Bryon: They recreated our uniforms. They would bring in the cargo planes and they were trying to get the You know, the supplies that we were bringing in. And the idea there is that if you just see things at a surface level without understanding what's going on underneath you'll never get the result that you're looking for.

[00:12:34] Bryon: And so I saw that happening a lot with Kessel Run. So I think it's interesting to dive into some of these details. But yes. Creating a DevOps pipeline that does a lot of the testing as well as the security testing and scanning is a super important part, but there's also an underlying process that has to be there to meet all of the requirements of FISMA and the RMF.

[00:12:57] Bryon: And so we did all of that as well. It's just like no substitute for hard work there.

[00:13:02] Bonnie: Yeah. When you say you did all of that, like government people were. Building the infrastructure and whatnot, writing the code to basically create this pipeline.

[00:13:10] Bryon: Yeah. We had a high percentage of government at that time. We were relatively small. We started out with one team of, you know, eight people very quickly scaled up to five teams. And I think we had about 55 people at that time. From that point on, we had to bring on a lot more contractors, but I do think the fact that we started with heavy government was important and the ATO process innovation in particular, it came from government.

[00:13:36] Bryon: And I think it had to, like, this was a government problem that needed to be solved. It's very difficult for a contractor to come in and say, like, I want to change the way you do RMF. So, Yes, that innovation was entirely led by, by government. As I said, that was one of my personal passion projects as the CEO of Kessel Run.

[00:13:52] Bryon: My job was to work on the next most valuable thing nobody else was working on. So, myself Lieutenant Wayne Starr and another lieutenant, Andrew Altizer. Really put together, Wayne focused on the technology, Altizer on the process and then me bringing it together. And we actually ended up briefing Lauren Nassenberger on the concept of operations and she, and we drafted the memo for her and she signed that first continuous ATO memo.

[00:14:18] Bonnie: Wow. How long did you stay at Kessel Run?

[00:14:21] Bryon: I was there for two years.

[00:14:23] Bonnie: And is that when you transitioned out to start Prodacity? Or did you have another assignment after that?

[00:14:29] Bryon: So, Talent management is a big issue in the department after I was at Hanscom for three years, right? And it took about one year before we started Kessel Run. So I had three years there doing software innovation, and then the air force wanted to send me back to intelligence. With nothing to do with software.

[00:14:46] Bryon: So that's why I decided to get out and I started RISE 8. We're a software consultancy. I really built my model around that Pivotal Labs model, but very geared towards federal and how can you take a different approach to systems integration in a software defined world and then Prodacity is a, is the conference that we put on every year, which is really just focused on educating government.

[00:15:09] Bryon: And giving them a place to, to learn as well as network in this space, trying to get rid of the like sales and theater. There's plenty of sales conferences out there. This one's just focused on learning.

[00:15:19] Bonnie: Yeah. So with, you know, now you're kind of, On the outside looking in, trying to influence and shape, you mentioned pivotal labs was a big inspiration for you. So how are you doing consulting differently with, in this space in particular, because I don't want consulting to be confused with more general strategic consulting, maybe that most people in government are familiar with.

[00:15:44] Bryon: Yeah. So we actually don't even refer to ourselves as consultants any of our messaging. And so I think what's different is we are focused on a change through doing. So I'm not like McKinsey or BCG that's going to come give you a bunch of PowerPoints. We're going to do what I described we did with Pivotal Labs when we started Kessel Run, which is we'll come in and pair with you one to one and deliver software.

[00:16:11] Bryon: And as we deliver software maybe we'll deliver the first app. And then we'll say, all right, let's start four more teams, but we need to organize these into a portfolio. So now we're going to teach you software portfolio management, but we're not going to teach you in an academic way. We're going to stand up five teams and teach you how to manage, just like we taught you one to one how to ship code.

[00:16:31] Bryon: Now we're going to pair with you one to one to teach you how to run a software portfolio. And then once you have four or five portfolios, we're going to pair one to one with you to teach you how to run a software organization. You know what they're calling software factories these days. I don't love the name.

[00:16:46] Bryon: I'm partly responsible for it. But and the idea there is that you change culture through doing. There's a great story of new me auto manufacturing, which is a joint venture between Toyota and GM, but Toyota wanted to enter the American market and they, Partner with GM, GM gave them the worst performing plant in the United States that had just been shut down.

[00:17:04] Bryon: The Fremont assembly people were drinking and gambling on the job. Like that's how bad the culture was. They were sabotaging vehicles, like putting coke doors and ceiling. Coke bottles inside of doors and sealing them up. So they would rattle. So I think we would all universally say this is one of the worst cultures, right?

[00:17:19] Bryon: And Toyota or GM said, Hey, we'll help you take on the union so you can get rid of some of these employees. And Toyota said that wouldn't be necessary. They took the employees out to the Toyota production system in Japan taught them. Through doing like, Hey, come and work with us at the Toyota production system.

[00:17:35] Bryon: And they brought them back to the Fremont assembly. And within three months, it was the highest performing plant in the United States. Same people, same exact people. And they didn't give them a bunch of computer based training to tell them what their culture should be. They didn't send them to agile certification schools or lean or, you know, six Sigma black belt consulting.

[00:17:54] Bryon: They just showed them how to work in a new way. And when people see the result of working in a new way, that's what changes their values and attitudes. That's what changes their beliefs and their culture. And so, we want to apply that model at scale. And we think that the new model for systems integration then, Is less about configuration control, working groups and architectural review boards, and really like a command and control style.

[00:18:20] Bryon: Lockheed comes in and hires 10 subcontractors, and that's how we're going to make good software. It's about implementing the culture first, just like I described, not some fluffy nonsense, but like, we're going to teach you our culture through execution. And then we're going to have the processes, the modern DevOps processes and tooling available that creates.

[00:18:40] Bryon: The integration through C. I. C. D. Through modern testing practices, and these are the things that are going to result in integrated systems. I don't have to tell you. Hey, you have to integrate with this person in this way through a five year process of coming up with the interface control working group.

[00:18:57] Bryon: Instead, you're going to build modern A. P. I. S. And I'm going to fire you if you don't achieve the outcome of Creating this user outcome and that user outcome requires you to integrate with another system, and you can do that because you can continuously deliver your software. So if you get it wrong, you just ship an update, and if the other team ships an update that breaks the connection, we'll find that before it gets to production.

[00:19:21] Bryon: And again, it doesn't take us a year to fix that. It takes us a few minutes to fix that. And that's the way system integration has to work in the modern world.

[00:19:29] Bonnie: So I would call that experiential learning, or that's how that's the term of phrase we tend to use in my group. Cause we talk about that from the acquisition perspective. How do we create new plays in our playbook? Well, you have to, sometimes you have to show people and in order for them to adopt it.

[00:19:45] Bonnie: So that's very familiar when you're talking about you know, shipping new software that are addressing user needs. How does that translate back to requirements? Cause this is a, an area where I think is unfamiliar and you know, we're not trained to, to think in the way you're describing and from an acquisition functional lane.

[00:20:07] Bonnie: So how do we need to shift or how should we be thinking about addressing user needs and tying those needs back to requirements? Especially if you have this continuous nature. That you're describing. Yeah. Just how are you thinking about that?

[00:20:22] Bryon: A few different ways. There's some things to unpack and I, unfortunately the agile and innovation community I think has maybe swung the pendulum too far to the right on like, Hey, we're going to put developers next to users. And That can be interesting in some contexts, but in general, if you talk to anybody in the design community, they will tell you that's an anti practice like Henry Ford is often attributed with this quote.

[00:20:44] Bryon: He never actually said it, but he said, if I would have asked people what they wanted, they would have said faster horses.

[00:20:48] Bonnie: Right.

[00:20:49] Bryon: Fundamentals of good design is understanding people's problems and then professional designers design solutions to those problems and often they design solutions in ways that users would have never imagined and perhaps could have never imagined because they're focused on what they do, which is not design.

[00:21:07] Bryon: It's conducting their mission. And so, I think like sometimes in short term scenarios where there's criticality involved, you could send somebody out and do something quick and dirty putting a developer next to a warfighter, like I won't deny that. But in general, if we want to design really beautifully designed systems that of war winning software, we're going to need design.

[00:21:28] Bryon: And so how do you bridge the 2 gaps? We certainly don't want this JROC requirements process that takes 5 years. But we also want to make sure that when we're talking to somebody, we have the entire mission context available to us. And so. I think the software acquisition pathway had some good language.

[00:21:48] Bryon: I take issue with a few things, but in general, I think it was a positive move in the right direction with capability needs statements were basically saying, look, we can't do the specification. Like we shouldn't be handing a contract or a specification. We should be handing them a list of objectives or outcomes.

[00:22:06] Bryon: And so we talk about outcomes in the Joshua Seiden context it's a Silicon Valley, like product guru. He says that. Outcomes bridge the gap between outputs and impacts. Senior leaders live in the world of impacts, like here's the results. I need this to drive for the mission. And then usually the acquisitions people and everybody involved in the delivery of software is concerned about outputs.

[00:22:31] Bryon: Outcomes are the changes in human behavior. So you deliver an output. What's the change in human behavior as a result of that output that leads to the mission impact? You have to bridge that gap. And Doing that then is a matter of somebody does still have to deliver the outcomes, not just at a user level, but also at a mission level.

[00:22:51] Bryon: If I'm building software for a small team within the Air Operations Center, I still need to understand the broader context of the AOC's mission. Because it's very easy as you, and you can read about this and manufacturing has a lot of really good analogs, but there's this book, the goal by Eli Goldratt, and they talk about the theory of constraints and how sometimes when you're trying to solve for a constraint, you can locally optimize.

[00:23:16] Bryon: And actually globally de optimized that is to say, like, if you speed up manufacturing but there's nowhere for that component that you're speeding up to go, that has to be stored. And now your storage costs could actually exceed the gains that you made from the efficiency in manufacturing. In other words you made efficiency here, but for the factory writ large, you actually de optimized it and cost more money than you made.

[00:23:43] Bryon: And so we see that in, in missions all the time. So what I would do there is I think we need to bring in a practice of service design. This is fundamentally missing. We use phrases like mission threads or similar types of words. And industry has no idea what we're talking about. Fundamentally, it's just a service.

[00:24:01] Bryon: Like the air operations center provides a service. To the combatant commander as one example, and I'll keep using air force examples because it's what's most comfortable to me, but the AOC is providing a service. And so it's just classic service design. You, we call it mission thread mapping a lot of times in the JROC process, but it's just service design.

[00:24:23] Bryon: And so if you have that service design, that service blueprint and you have a set of objectives that the commander has given you, Hey, I need to be able to detect 2000 you know, Air defense objectives coming into my airspace. I need to be able to generate 5, 000 sorties a day, you know, whatever it is could be crazy.

[00:24:40] Bryon: But like, those are your objectives. You understand the mission thread. And now I send software people out to solve those local problems in the context of the larger mission that needs to be achieved. And so. That can be done relatively quickly. If it takes anything longer than a couple months to, for any part of that process we're waiting too long because the mission is changing too fast.

[00:25:01] Bryon: And we're seeing that with counter U. S. right now, people are dying because of how fast the UAS threat is evolving. And we need to be able to move much faster from a requirements perspective.

[00:25:11] Bonnie: How do we connect the buyer community with the end user community from your vantage point? Just your opinions on that more closely, I

[00:25:20] Bryon: It's a hard one. Yeah, it's a hard one. Because interestingly enough, if. In the requirements process, a lot of times the requirements stakeholders, all of them, I won't target any specific group, but pretty much anybody that has a stake in the requirements process, they will not let. The contractor or even the airman coder go talk to the end user. They like, we'll literally not let it happen and they won't even let them go get feedback on their software. All has to be done by proxy. And it's because of what I just said, they are afraid that when you do that, and we've seen this happen, so this is a valid fear that they have that we need to address in a meaningful way that.

[00:26:01] Bryon: You send coders out to sit next to warfighters and they build solutions that deoptimize the larger mission thread because they'll say, Oh, the 18 year old airman that swaps out every six months on deployment doesn't understand the larger AOC mission thread. And to some degree, they're right. They're also wrong in that they don't understand the 18 year old airman's point of view when he has to go conduct the mission that they've given him.

[00:26:27] Bryon: And so we need both. And so I think. What I just said before is certainly like we have to build the trust through that process so that people don't feel like we will lose sight of the larger mission thread, the service design, but then I think it's just a matter of, you know, I can tell you your priorities just by looking at your budget and your calendar.

[00:26:47] Bryon: You need to fence off a portion of your money to go visit. We got into a lot of trouble at this. Be careful, but like, I'll say early day, I don't speak for Kessel Run anymore. Early days of Kessel Run. When I was there, we got in trouble for our TDY budget all the time. People were like, you're spending like, you know, half a billion dollars on travel.

[00:27:05] Bryon: Like, what is this? It's a boondoggle, you know, it was constantly getting criticized for it. But that was more important than the two developers that could have bought me. Way more important, infinitely more important. And then if you looked at our calendar, like I said, there was always somebody out there and a lot of times there were four or five people out there at any given time.

[00:27:24] Bryon: And so, literally go to your user. It's like going back to manufacturing Gemba, right? Like walk around, go see. And when you hear these people's stories too it's heartbreaking. It's hard not to become a zealot at that point. Like people are literally die or get put in harm's way unnecessarily because of bad software.

[00:27:42] Bonnie: Yeah, I'm smiling because I mean, you're, I didn't even have to say anything like you're just saying a lot of things that my team has known and felt. But it's hard to proliferate. It's been hard to proliferate. So I'm curious going back to the culture piece. Maybe we can round out with some of your, you described like you're going to teach these cultural tenants by doing what are some of those cultural tenants where you're hoping to shape.

[00:28:05] Bonnie: And influence some of these things you're talking about where you do, people should prioritize and end user engagement, or they should prioritize design up front and things like that. What are some of the other principles or tenants that are important to do continuous software delivery at scale?

[00:28:24] Bryon: We start out with values and principles and then those lead to practices and so, you know, the higher up we are in the chain, the more opinionated we are. So like values and principles really shouldn't change that often, but our practices are often evolving. Based on what we're learning from the field.

[00:28:42] Bryon: So with values we have a culture manifesto at rise eight. And we use that in our engagements with our customers too. We try to get our customers to adopt these values. The first is outcomes are bust. We don't care about anything unless it creates an outcome. And that sounds fairly obvious, but you'll see all the time activities that get prioritized that have nothing to do with creating outcomes.

[00:29:06] Bryon: In fact, you can see it. Very obviously in the platform space right now. So people are building platforms for the sake of platforms and building features into those platforms for the sake of like, this is the latest technology. We need sidecars, we need this, we need that. And then you're like, okay, well, how many capabilities are on that platform that are serving war fighters today?

[00:29:28] Bryon: And I've come across a hundred million dollars spend. In the platform space with zero zero apps in operations and so to me that they're like, Oh, well, we've achieved an outcome. No, you've created an output. There's no human behavior change, meaning it's never going to lead to the impacts that you're looking for. And so, we're constantly orienting all of our activities around outcomes. And then we have some things around keeping it real, learning the fastest things to be less certain. So, you know, one of the fundamental things that we try to get across is, you know, plans are useless, planning is essential.

[00:30:11] Bryon: But we have to take a good, hard look at the plans that we're creating. A lot of times we just think if we followed the plan better, like the reason that the project failed is because acquisitions or the contractor didn't follow the plan well enough. But the plan was the problem. It was the plan that was the problem, not the execution.

[00:30:28] Bryon: And so, reminding ourselves that every, you know, whether we call it a requirement or an outcome or anything they're just assumptions that need to be tested. And the farther out in time we go with an assumption, the more likely it is to be wrong. And so creating processes for learning instead of optimizing for being right.

[00:30:50] Bryon: So we optimize for being wrong. The community really optimizes for being right. I would say this is the biggest friction point that we have on engagements. And so going back to learning through dealing we will create. Every engagement I believe has to start with a path to production. So if you're out there listening and you want to fundamentally transform software, there's a huge draw to start with alignment activities, things like scaled agile framework or whatever, to like, how do we align on value and come up with the right requirements and the right plan and design our org and all of these things.

[00:31:22] Bryon: Big things. I would say, just figure out how to continuously deliver lines of code first. I don't even care if the code is important, because the second that you create the ability to ship code on demand, has value in and of itself, in that now, if Bryon and Bonnie get into an argument in a conference room about what we should build next. All we have to do is ship it and find out. We don't have to find out a year from now or five years from now, which is typical, which is why these fights in these boardrooms get so violent in the requirements process. Right. And it has to go up to four stars for adjudication before it comes back down to the force is now I can say like, Hey, you know what, Bonnie, I disagree with you.

[00:32:04] Bryon: Here's the things that I think we should look out for. Let's go ahead and ship that feature. And like, if I'm right, we'll see this. If you're right, we'll see this and we ship it. Or maybe we do both. And we do an AB test and see which one works best for the user. We can do that now because we've unlocked this feedback mechanism.

[00:32:21] Bryon: So that's one example of something that we do. And by doing it, we can change behaviors because when people see that the walls just melt down. They're like, Oh, you know what? If we're wrong, it's not a big deal.

[00:32:35] Bonnie: like the idea of testing assumptions. We, you just don't know until you know, and you see, like you said, with the impact or the change behavior is I like that a lot. And I think we, we need a lot more of that. So with that I can't thank you enough for sharing some of your insights with me today.

[00:32:53] Bonnie: I do have one more question. This is more of a fun question that I ask everybody. This is not really anything related to anything we've been talking about. But are you ready for the question?

[00:33:05] Bryon: Yeah.

[00:33:05] Bonnie: Yeah. Okay. So the reason I paused for effect is because it's it's a little bit out there. So can you tell the audience if you would dip your grilled cheese and hot chocolate?

[00:33:16] Bryon: No, I wouldn't.

[00:33:17] Bonnie: Yeah, I can tell by your face. Like I wish everybody could see your face. Yeah. So apparently this is a thing out in the Midwest and I was so blown away. When somebody told me this, I thought this was a secret. Nobody told me. And so I ask everybody now, cause I'm testing assumptions here and trying to see, is this real or not real?

[00:33:36] Bonnie: So thank you for playing with me. So other than that um, any last words for the audience that on anything that's on your heart or on this topic of delivering outcomes, not outputs or anything that we've talked about,

[00:33:49] Bryon: Yeah, I'll just say I recently had the pleasure of joining the CENTCOM innovation community on a tour. They brought five CEOs and tech leaders out to the AOR. So we traveled to Army, Air Force, and Navy bases in the Middle East. And it was great because I've been running my company for the last five years and been really focused on that.

[00:34:12] Bryon: And it's easy to get disconnected from my old Casa Rondes when I was going out into the theater all the time and seeing it. We've made some great strides in the innovation community and we should definitely take note of that. There's also really long ways to go. And in the one area you've hit on it a few times in this interview is that disconnect between operators and acquisitions.

[00:34:31] Bryon: And I see senior leaders, one, two, three, four stars on the operations side of the house, who do some of the most amazing things, them and their people. I mean, these are people like knocking down doors, doing the craziest missions. But then when it comes to this particular topic, it's almost like there's a learned sense of helplessness. Oh my gosh, well, that's the program of record. Like we can't change that. There's nothing we can do. It's Congress, it's the FAR, it's all these things. And what I would say is you don't have to feel so helpless, right? Like, and you don't have to do what I did, which is leave and go to acquisitions to fix it yourself.

[00:35:08] Bryon: I definitely learned a lot trial by fire doing that, but you can hack the bureaucracy. And in fact, usually when you look at the bureaucracy, It's not even written in policy, it's just in the way that we're behaving around that policy. And so I would just tell everybody out there, especially on the operations side, and even if you're an innovator on the acquisition side, don't accept powerlessness.

[00:35:30] Bryon: Like, that's what the bureaucracy wants you to do. But if you just stop for a minute and look at the bureaucracy and figure out how you can hack it, even a program of record, right? You and I had talked before there's, it doesn't mean it's tied to the technology and the vendor that happened to be on contract right now.

[00:35:48] Bryon: Even if you have a full blown program of record that's two years in, you could cancel that. That's what happened with the Kessel Run and AOC, right? They canceled the Northrop Grumman contract, took that entire budget. And assigned it to a production OTA, like those things can be done. It just takes political willpower and the way that you can get the political capital is by going and seeing the people who are living with the consequences of your decisions.

[00:36:12] Bryon: And so those would be the two things I would tell folks is don't accept the powerlessness and then take people out there to see what's really happening. And once you do that, you'll have the political capital to cancel any program. When people see people dying, it's a real quick way to get. A contract turned off and a new one stood up.

[00:36:28] Bryon: So, just do it.

[00:36:30] Bonnie: finish. Thank you so much. And we'll be, I'm sure I'll be talking to you soon.

[00:36:34] Bryon: Awesome. Thanks for having me. 

[00:36:36]