Cloud Unplugged

There's a Crisis in DevOps and Nobody's Talking About It!

Listen to a lively discussion about the state of DevOps today. 
Our hosts Tim Stahl, Graeme Colman, and Salman Iqbal discuss the biggest problems in DevOps and how to solve them.  

  • How DevOps teams interact with other roles is becoming fuzzy
  • DevOps tools and how people are using them
  • Fail fast: if you pick the wrong tools and processes, pivot and move on
  • DevOps has to be part of a cultural change - find an MPV to champion change
  • Scaling DevOps requires standardizing tools and processes

To keep the conversation going,  join our Slack community or follow along on Twitter 

Transcript

Tim Stahl:

And our topic for today is There's a Crisis in DevOps, and No One is Talking About It. So what we're going to do here is start with a quote that we found out there. And really, it's, you know, one of the problems we see is the lack of standardization. And DevOps has created these patterns of working. And they're using a litany of tools, and they don't always scale. Now, this is further exacerbated by employee turnover. And of course, you know, that quote is by anonymous, but but really, it's something that I actually believe. So that being said, what I'm going to do here is we're going to kill these slides. And we're not going to do death by PowerPoint here. We're going to stop screenshare turn on where our attendees can also join in the conversation here, give me just one second, okay, and allow to talk a lot of talk. And we'll have you join in the conversation as well. With that being said, I'm gonna go ahead and kick this off by throwing a question out there. Again, since this is a live q&a, feel free to join in, if anytime you have a point you'd like to make, start talking, we're all in the group here. We're all here to learn from each other and have a little have some good have a good time. So when that panel, including myself, maybe we'll start with an easy one here. You know, what, what do we think the current state of DevOps looks like? Anybody can feel free to take the

Graeme Colman:

one gram, you're gonna, yeah, well, no, I was. If you can hear the dogs barking in the background, let me know. The dogs are getting very excited outside. But yeah, there's, I was kind of looking at this just before this morning, and there's state of didn't realize this, there's now nine state of DevOps report, which you should be looking at, and figuring out what the state of DevOps is. But if you look across them, you know, you've got the usual unit got from puppet circle, ci, there's a bunch of them. They're all kind of saying the same thing. And they're all saying, talking about cost commodity standardization skills. On one side, and on the other side, it's kind of like, well, you know, what, DevOps is not a thing. For people who are doing DevOps, it's kind of just a way of working. So it's, and I think, if DevOps is a thing for you, if DevOps is not a thing for you, because it's just a way of working, then, cool, I think you're in a great place. But you're on the other side of it, where we've got problems with, you know, the too many tools across the organization, lack of skills to do this properly. Problems with scaling, because of lack of skills and too many tools, then, actually, you're not in a great place, because it's, you know, we're trying to follow the industry, but not doing it successfully. And it's, it's hindering a lot of people in just doing what we're trying to do, which is build apps, deliver apps and deliver business value quicker. I think that's where I've talked to customers, and they're kind of a, we'd like to do DevOps, but we just, we just can't really, we don't know where to start, or we've got small experimental projects, which we could doing it. But that's not having the huge business impact that we thought it would. So it's, it's mixed from my from my experience. Yeah. And

Unknown:

I think, Greg, what you're saying does chime with exactly what the puppets 2022 state of the DevOps report is also saying, Yeah, as Graham says, if you haven't had a chance, definitely check it out. And there's a few of them. But I'm going to talk about the puppet one, because that's the one that I read, that had a glance through. And stuff, basically, the biggest, I mean, the biggest one of the few summaries, there is the role of organization wide acceptance of DevOps practices, which is, you know, what you're referring to, you have a little project one team is working on, they've put in stuff that automates the repetitive tasks, and, you know, shift everything to the left, make sure you're doing security on the left, making sure you're doing infrastructure, everything is working, but hardly you get that accepted, organization wide, and that people struggle with and that causes a blocker, on, you know, them advancing on on the DevOps journey. That's what the report is saying, which is exactly what jives with what you're saying. And, and how the teams then interact with the boundaries of roles is becoming a bit more fuzzy. Now, who's supposed to do what, for example, we work with some clients as as, as Appvia, we work with some organizations, they used to be a specific team, they say, Oh, we are just going to do patching of our servers. And that's all they did. But now we're like, well, the patching of the servers also has to go into the code. And they're like, well, we don't know how to use these code tools. How do we do this? And yeah, the the acceptance of DevOps practices, people are struggling like organizations are struggling to, to implement them. But there are some perhaps we can talk about what things people can do at the end of it to see how You can improve this problem. But there are some issues that are coming up. And this is what the report is saying. And this is what we're seeing as well.

Graeme Colman:

As well as that. Second, the definition of DevOps varies wildly. Depending on the asking, you're going to see. So DevOps that includes the whole of the business, right? So there's all of the agile business change objectives, agenda is it within an organization is included in, in, in, in DevOps, and you get to some other organizations, and it's automation, right? It's infrastructure automation, or so that doesn't help. Because the two of you, the vendors, and the tool, the tooling vendors will perpetuate some of this, where it actually really adds business value, it's kind of well, this is maybe where I'm seeing some customers saying, well, DevOps is not really delivering the value, perhaps because it's just cleaner, because all we're doing is automating infrastructure, which is cool. Well deliver value, but if you want to business value from from, from a lot of the

Tim Stahl:

interesting that you mentioned that too, because there's when you talk about infrastructure, specifically, and automating it, you know, there's kind of only a couple of big players out there that everybody's using, and you're kind of on one side of the fence on the other. And I'm not gonna mention their names here. We all know who they are. But, you know, one organization in, you know, as a DevOps engineer, you could join, you could run, you can use that tool. And when you find another opportunity, you jump, and now your other companies using a different tool. And all of that experience, you'd have gotten got, it doesn't quite go out the window. But it does, you know, lend yourself to having to retool, you have to learn this new, this new thing to get your job done. So that's, it's it's a very interesting place to be. And maybe that's kind of our jumping off point there is, you know, when we start talking about tools and how people are using them, you know, what have we seen out there? What are some ideas around the spheres of influence related to those and how those different tool sets drive that business impact? You know, what were you guys seeing out there for that?

Unknown:

In terms of tooling for DevOps? Is that what you asked? Tim? Yeah, yep. Yeah, it all depends which area we're looking at. I mean, as as Greg was saying earlier on, by the way, welcome to the webinar, if you've just joined, please feel free to unmute yourself and introduce yourself. And you know, me if you want to make a point or ask questions, feel free, feel free to do that. We're just talking about DevOps, just hanging up, basically, we're just hanging out. So yeah, I think, I don't know if you've seen if you go to landscape.cn cf.io, I'm just going to talk about cloud native tooling, l.cn cf.io. Maybe we can share this here on the on the page, just so people can see it. I got it, you got it.

Graeme Colman:

So I didn't know that you had L dot CNC f.io.

Unknown:

You just, it's just redirects that landscape of CNC F. Now you can see in here, all these tools fall into some category of DevOps, right? So you've got, you've got these tools for it like this, if you scroll down to a CI CD tools, there's tools for doing cloud infrastructure, automation, or container registry is whatever it is. Now, if you're working, if we're talking about cloud native landscape, we are seeing a bunch of tools being used across a number of organizations, for example, application, definition and image build at the very top. If you scroll up to the top, if you're working in Kubernetes, some people are using Helm, others not using it somebody else is using customize, somebody might be using Buildpacks. So there is as always no industry wide. Standard, right. And this is one of the problems when people start, they're like, What should we do we want to do cloud native, what do we do? And then you know, they ask around in the community, maybe go to meetups, maybe talk to service providers and get their expertise in figuring out which tools should they use? So in order to answer your question, like the big one, there are continuous integration and continuous delivery. The one that on top right there, there's so many tools in there. And some of them you're familiar with, perhaps Jenkins or the chair background you come with. But basically we're saying we're seeing a number of tools. There's a few practices. For example, if you say oh, I want to do git ops, in Kubernetes, which is every change in the infrastructure has to be made via Git or the application. Every change has to go via Git and automatically syncs it. Then there's a couple of tools that stand out like the Argo and flux there on the left there on the top. But apart from that, basically it's a free for all. I don't know Graham, if you want to add anything to this.

Graeme Colman:

So this kind of goes back to some of the title of the talk and it's kind of cool Some tools are going to be good and great for the things that you're doing. And the level of experience and what people have learned tooling wise for the for the job, they're doing. Good, because there's so much choice, it's kind of a well, I've got customers who are all in Argo, who are building, you know, Kubernetes, all the production Kubernetes estates, and heavily into our other organizations, you know, using TerraForm, and Ansible, to do all of the all of the things they need or, or just TerraForm, or just Ansible. And then if you look, underneath that there's, there's a whole load of other tools, which are really, really useful, but aren't standardized in terms of, hey, here's a great tool set that's going to help you deliver and build Kubernetes infrastructure and, and deliver applications into Kubernetes. So this, this kind of goes into some of the problem. And this is, for me, a bit of a problem of the industry at the moment, you're just looking at the CNC F landscape. If you're starting in this, Hey, what? It's just a look at all the tiles on there, it's coming Well, where do I start? What are the right things to do? What's going to make keep me in terms of the tooling choices, I'm using future proof me for the skill sets that I'm going to be having to hire into for the longevity of the actual toolset, you know what, what's cool today, it probably isn't gonna be well might not be cool in 18 months time, and now I'm stuck with technical debt that I don't have now have to resolve within my, the budget and everything that I'm doing within the organization. So talking about tools is difficult for me, because it's like, well, can you suggest some tools now that that are going to be future proofed? In 18 months time? Now? I don't think again.

Tim Stahl:

And that's even even more as we call them a

Unknown:

lottery numbers.

Graeme Colman:

Yeah. I mean, there's a good set of standard tools that you'll see in the industry that you can look towards, you know, what are what are the skills that people are gaining, and what what is the skills attainment, or how easy it is to attain those skills into your organization, which is a great thing, because you know, it's, it's all cost at the end of the day, and we've got to be able to do the right thing at the right cost with the right people. So the things you can hazard a guess on, but you're guaranteed in large organizations. In a large organization, you can guarantee there's going to be little hives of teams that have gone off and use random bits of quite cool bits of software that you have now on lots of little snowflakes within the organization with different tool sets.

Unknown:

Yet, if you think skills is the bit that plays the part the most, you need to make sure you have the right skills in the organization to be able to take on some of these tools, understand how they work and implement them across your team first, and then perhaps the whole organization, what What tips do you give, when somebody comes to you say, we need to pick a tool? What should we pick? Depends on

Graeme Colman:

what we're trying to do. So I've worked in organizations where actually we're trying to drive an agile agenda, trying to drive agile software delivery, involving the business and holding, you know, making pretty much transformational change into how software is delivered. And that's taken a lot of experimentation, a lot of, you know, putting together a tool set. So you can just showcase it in front of the rest of the organization and say, Look, this is how cool this is how well we can deliver software, software quality in a great time are reduced at time. And repeatedly doing that, in an experimentation framework or in the bubble of experiment. It's pretty easy, right? Choose what you want. Because you just delivering once right? scale that out into how do we do this in an organization that's got 800 developers with with their size, and other partners that help them building and delivering their software, that becomes a real problem now. So where DevOps and agile kind of started off by saying, Hey, pick the right tools, that's gonna get your job done. And getting the job done is more important than documentation or whatever the Agile Manifesto says, great, but when you scale it, that's where that's where that's where you start to look at. What's the cost of the skills I need to to build and maintain this? What, what are the cost of the tooling? And how long will this be living within my organization? What's the technical debt I'm taking on and being aware of all that? And suddenly, we're back to again, Enterprise Architecture skills of figuring out what technology is going to drive our business forward rather than how agile can we be delivering this bit of business functionality using whatever tools we need? I don't think that answered the question.

Tim Stahl:

Well, it's almost it also goes to the rinse and repeat factor, right? You know, so you've, we've gotten all this tools, we've gotten these these pipelines set up, we have all of our stuff in place where we're, we're saying, you know, we're doing the best we can with this project. We're delivering it the the most efficient way possible, securely. And now project two comes on, does what you set up work going forward, you know, you put all this time and effort into it. But how do you make sure as you're doing this, but it's set up in a way that's scalable and maintainable in the long term, even if the long term is two years or one year, it's not just that one project. That's, that's one of the problems, you know, I've run into and seen before is, yeah, we've got this great landscape set up, we've picked, you know, whatever we needed for whatever's problem we're trying to solve. And now I'm on to problem two, and it all kind of fell on its face. So I don't know if you've seen anything like that, as well. But

Graeme Colman:

I certainly have worked for a large oil and gas company. And that oil and gas company had, he was involved in commodity trading. So trading on the oil and gas markets, and the DevOps tooling and processes and what they put in place. And that was awesome, really great. It turned the software delivery for that part of the business completely around and gave them competitive advantage. And, you know, it was great for them. That then tried to be rolled out to the and this wasn't just a project, this was like a whole department, when they tried to roll it out to the rest of the organization, which didn't have the regulatory compliant compliance requirements that didn't have some of the real stringent need and require for the amount of level of testing of environments and everything else. They tried to roll it out. And it just failed completely, developers just turned away from it, because it was too heavyweight for what they needed. What was great in one area, and the amount of skills needed to run this thing or to use this thing was just no one had it. So it was unusable outside of this one use case or because the department so that cost that company a lot of money to fix that. So it was and the value that was driving within one pick one area have to all be unwound and for them for that organization to standardize to make sure that they weren't creating pockets of snowflakes that were going to cost them in the long run. The cost and other costs a lot of money to standardize, so be aware of these things.

Tim Stahl:

That's true. Now, go ahead, please.

Unknown:

Yeah, I was gonna say something similar as well, I'm, I've seen, I used to work for a financial institution in UK, quite a massive financial institution. And they decided to make their own tooling for CI CD, right, they're like, Oh, we're not gonna go with Jenkins, we're not gonna go with any of the other stuff. Because what they had a few reasons, some more valid, which others weren't. And kind of similar to what green is saying. And as you know, this pattern, you see repeats that to work with a small number of teams. But then when it when it got rolled out, there were too many issues. Number one couldn't really scale properly. If you submit a number of jobs on there, didn't really work that well. And the biggest problem was that this thing only a few people could support. If you pick something like Jenkins, there's a massive community around Jenkins, you could search for the error, and you were damn sure that the error is going to pop up and Stack Overflow, and you know what to do. So that is kind of useful. Picking a tool, which number one meets your requirement number two, when things go wrong, knowing what to do with it. That again, just like Graham was saying in that in that organization, and this organization, they got canned, after so much time was spent on this thing. And then eventually this is buying off the shelf, right? Save, save, you know, here's the right bets, when you when you pick these tools is also. So perhaps a good and also important thing is with that project was if you think it's not going to work, there's no shame in saying it didn't work, let's shut it down quicker, rather than leading it to a slow and painful death. I think that's probably the thing that I've learned the most. With this. If you pick a tool, and it doesn't work out, not a problem, you should pivot and quickly go and pick a tool that might work for you in this scenario, because you will pick that tool when you did pick that tool based on the information that you had. And that could have changed. Maybe you didn't have all the information. Maybe the decision was made was influenced by something else, but that's okay. Because now you have time to rectify it and pivot really quickly to something that that could work. So that's what I wanted to add to exactly what Graeme said how

Graeme Colman:

do I think the kind of the fail fast mindset works within our DevOps community is Yeah,

Unknown:

so the biggest the biggest problem with specially I'm just going to talk, you know, I've worked with different kinds of organizations, from all level startups to established organizations for a little while. This this fail fast approach is not really welcomed everywhere. People are penalized for saying what you wait, the project that you did didn't really work. So we, you know, like you didn't deliver the objectives that we set up? You do? Do you know, like you, they set your objectives every three or four months? And you have to supposed to, you know, complete them those objectives that's penalized? You know, somebody's saying, is this not working? Let's shut it down. That is not, you know, celebrate. I'm not saying it should be celebrated as a win. It's just celebrate as learning and moving on to something new. Yeah, so I think behavior is changing, people are picking up that, oh, maybe we need to move fast. Because things are moving quick. Anyway, like, you know, everything changes so quickly. Yeah, I'll say, as a whole, it's not really welcome that much less, fail fast and move on. Unless you're the you're the co founder, we work and you can raise another few 100 Millions. I think there's nothing a thing. I don't know what you think, right?

Graeme Colman:

The whole idea of kind of the Agile delivery fail fast and rapidly changing to experiment. And that all kind of relies on having the right tool set underneath you to be able to do that. So I think the so to be able to rapidly change and fail fast and deliver an application. Yeah, sure, we can try different requirements, different tools, different whatever we need to different architecture, it all requires having, you know, the, the pipelines and tooling for delivering applications already there for you to be able to rapidly rapidly deploy rapidly test rapidly change. So think there's levels of fail fast. And in this in your pipelines and how you get infrastructure built and how you get applications into testing through the environments. That can fail faster. Because if that, if that's failing, we have to change that then actually, we're failing way too early on in our in trying to build and deliver business value. So it's some sometimes you find that actually, this is fairly set in stone. And we've chosen these, these tools and these these processes, and we can't really change them, because that's impacting everything that we do. And we'll ultimately delay the delivery that we're trying to be agile with. And it's a, we sometimes end up with the wrong decisions and legacy or snowflake in those pipelines.

Tim Stahl:

I can, I can see things like security and compliance being an issue with that, right, you picked a tool, whatever it happened to be, because it met some kind of guideline, you had to set up for some kind of compliance, regulatory compliance issue you had to adhere to. And then you find out while he does that one thing, but it also actually breaks three others down the line, because of, you know, however, works or whatever it happens to be. And now, like you said, you've got this one pipeline that's set up specifically for this one task that's orphaned over here. So it's just it's, it's, you know, especially if you're trying to deliver quickly, being able to fail fast, I think, is something people should, you know, if you're, if you're going to adhere to it, you're gonna say, I'm agile, I'm going to go down this route, I'm gonna sorta, you know, deliver applications this way. I don't know, my my probably unpopular opinion in some circles is you should really be failing fast and failing off. Line iterate.

Unknown:

I think you have to get the balance right, as well. Because sometimes just saying I didn't work, let's just shut this down. Every every week, you're doing something new, right? You're not really focusing on I think you have to make a decision based on and this is all us, all us engineers fall in the same category, I'd say you have to make a decision based on how does it make me help achieve the business requirements? What is it that I'm trying to deliver, to achieve what the business wants to achieve? If I say let's, let's just introduce Kubernetes, because it's going to be really good for the business. Well, the website doesn't really need scaling. It doesn't need 3d needs. Auto restarts it doesn't doesn't need any of that stuff. Why are you introducing a tool that's not going to help out so that you can waste time by introducing Kubernetes and be like, Oh, I didn't work let's just scrap that. Let's carry on. And I think just asking probing questions within the organization is perhaps useful, you know, like within the Then the teams like, challenging some decisions, definitely helps.

Graeme Colman:

So that probably brings kind of another area of culture and within the business of what what is the, what is the organization trying to do? And how does that kind of goes back to the earlier statement about what DevOps means different things in different organizations. So, for me, it has to DevOps has to be part of a cultural change within the organization that adopts and adapt ways of working that, that delivers business change quicker, not just IT and automating it, yes, it's delivering the changes from a business requirement. And it's shifting all of that right up front into the business as well. It's kind of like, well, how do we change the way of thinking from our business owners and product owners to, to really adopt agile ways of working of doing things like fail fast experimentation, and come up with? What are the things that we can do to accelerate business change by by doing things really quickly in small increments in whatever it is? And then what is the how does DevOps fit into that as a as a culture? Where I've seen it be just insanely successful? And I've worked on programs where really had to change the mindset of the whole of the business. And it's been business driven. The change in in technology being business driven, so educating the business into how does agile delivery work? And how does DevOps fit into that? Why is it important? And why is it important to get your business product owners actually, knee deep in, in the process of, of getting get ups and changing? Everything that ice is doing, it's once once that's all linked together, then there's a whole cultural change and a culture that is almost the tooling. And technology doesn't really matter, as long as you get through in terms of the level of change the level of quality associated with that change, that it's repeatable, and can deliver in a standardized way, every time I can safely make a change back it out, because I know the process is there. Sometimes the tooling isn't as important.

Tim Stahl:

Well, that's even something you know, you see in those reports, the state of DevOps, you talk about the culture? And, you know, are they are they actually all in on it, you know, are they do you have what I've run into in the past where you have, you know, pMOS, or project owners that are more comfortable going the waterfall method where it's the entire project is scoped, you know, end to end. And it's just build it until it's done. And any change has to go through 14 layers of bureaucracy and change control to make, you know, change the background color from red to blue. And, you know, but they say, Well, we were, we're DevOps. So because we're using, you know, we're using a CI CD pipeline to automate the bill. But you're not, you're still not, you're missing the point of trying to deliver in an agile way. And that's, that's, that's something I ran into numerous times where it's just, you have a legacy way of thinking in a legacy model that they know how to operate in, and they know how to make work, right? This new thing is scary. You know, what I'm giving up control? Wait, you mean, the whole product isn't designed, and we're already, you know, in the middle of developing it. So it's, it's, it's interesting to get that shift in that mindset change within organizations, that then you can start building out the actual processes.

Unknown:

So how do you do that, that education is showing what we're going to implement this thing? And there's gonna give you this this much times improvement and delivery? Or what was the steps for that for it?

Tim Stahl:

Yes. It's actually both. So it's, you know, part of it was education on the processes itself. So really, it was starting with the basics, what is agile? What does this mean? You know, what are the differences in the two different project management philosophies and delivery philosophies. But the second was, you know, taking a low hanging fruit application that needed to be built very quickly. I mean, we're talking from conception to full production within two months, that was going to in my case, was going to serve about 37,000 students in the fall. So I'm, I'm pulling from a higher ed background here. And the traditional method would not have solved that problem. So I was able to go to the go to the executives and say, Listen, you know, if we want to get this done, and we have to get this done, it's mission critical. Here's a way we can do it. Let's take this on, run with it, get the project done, get it solved. And you'll see how this will actually speed up the delivery process significantly, but also give you the ability to iterate on it as we're going through it. You know, in the traditional method, we've gotten done you said, You know what, that button up there? I don't like it. I want it down here. But we'd have to go through all this process, but I've had four committees meetings and you know that the standard thing to get that approve were in Agile, it's like, okay, well, we'll just add it to the next door, okay, pull that piece in, you know, develop it and send it. So it ended up being a massive success. I mean, of course, there's, there were a couple issues with production time, but again, because of how we built it, it was very easy to go in, correct and go. So based on that, certain other projects at that university has started down the trajectory of DevOps. And it's been a couple of years since I've worked there, but they were trending where that was where the development shop was gonna go to.

Graeme Colman:

So So it used to be for me used to be doctor when a few years ago, really difficult to get any type of buy in for anything that would be have any impact on the business, there was being driven and led by it. Now, I'm a member, when trying to do first DevOps type agile deliveries, had to have a real leap of faith from the business. Because it wasn't just it, it was, actually we're going to pull you in, because this is how agile works, we're gonna put you in as part of the business into it. And they just weren't ready for her, there was nothing, it was something that was done to them on a quarterly basis, rather than getting involved

Tim Stahl:

with magic, it's just the types that just Yeah,

Graeme Colman:

it's easier. Now I'd say I think, get finding champions for agile business delivery and agile business change is easier. But it's still something that has to be done, you have to, unless it's already happening within the organization, you have to have a champion or a set of champions, who will be able to find a MVP or a project or something that is going to be able to help them showcase just how great things can be, and how, what has to happen for that to change. And DevOps is part of all of that. Education is something it just isn't enough, sometimes it's actually I want a champion who believes in actually they can make the change in the business. But it's helping them understand the best way to do that, from my perspective, and what my job my role, my role is, it's how do how do you help them find it and MVP, small part of a slice of functionality that they can push all of their changes through, to show some value at the end of it, that the rest of the business can kind of take a look at and say, I want I want more of that. And that's that's been where it's been successful. Just doing the the it part of it. Although it doesn't drive all the value, is still a really valuable thing to do. So, you know, DevOps as a way of delivering it safer. With better quality, faster, hey, there's lots there's lots of value in that doesn't always have to be with the business. But that that in itself is valuable, but doesn't get the whole hold of the value from from DevOps.

Tim Stahl:

You know, it's interesting, it's interesting, you mentioned that because I, you know, two of the things that I had working for me in that transformation I'm talking about was, one was, I ran the department, the IT department. So I was able to make that unilateral decision that we're going to do it. But the second one is, we did have an MVP on the business side, who was who was familiar enough with software development life cycles to understand what we were trying to do. And he also wanted the control over what the product looked like, and the ability to tell us, hey, I don't really like this, I want it to look this way, as we're going through it, he really glommed onto that idea and said, Hey, this is this providing me value, because now I'm part of it. So you don't get that a lot from the business side of the house. But, you know, we're just, we're just fortunate we have someone who was and was willing to take that leap. And, you know, I basically, it was one of those, well, if we want to get it done in the time that we need it done, this is what we're going to do, you can be part of the process or not, luckily, he was very willing to be part of the process, and it ended up being a very successful outcome. So, you know, kind of, you know, pulling this all back, you know, at the start, we were talking about how there's just, you know, litany tools out there and there's you know, we've talked through culture, we've talked through teams and scalability and all that, you know, if we could come up with you know, I know there's no magic sword there's no unicorn we can come up with it says this is exactly how to fix it. But what are your thoughts on you know, the, the having the litany of tools out there, the lack of standardization and and the training and culture efforts, we're talking about, you know, if you had an opinion on what's something organizations could do to help themselves get through this, and what what do you have any ideas on? Maybe a takeaway there?

Unknown:

Yeah, maybe I'll get my My two cents, two cents, as we call it here, and then let Graham see what he says. I think the best thing you can do is to talk to people, you can talk to people who have done it, maybe you do talk to other people, or conferences and meetups, maybe talk to companies who are, you know, we'll help other companies do this, you know, maybe like some services company, companies just talk and find out, what are the practices they've deployed, have worked. There's so many talks out there, let's say if you're in Kubernetes space, there's so many talks out there and cube con online on YouTube, we're gonna find out, one of the things they implemented to improve their whole process for delivering because at the end of the day, it's all about delivering business value. This is what DevOps is delivering business value, how do they improve, it doesn't matter if they use like 60 clusters or 500 clusters, or didn't even use any clusters. So there's some examples out there. So but basically, I think just talking to people who have done this, maybe talking to organizations who have done this with other customers, you know, like we've done a lot of customers we work with customers are like, are we gonna, we're working with some right now. We have a lot of our applications that are deployed on premises, it's quite slow. You can't really make any changes. If you ask them to patch server, it takes like a few months, and the servers already quite vulnerable. What do we do? And then we go in, you have a look at this, figure out what's going on. By the way, I'm not trying to plug up yet, but I'm just saying, just talking to people is one way of figuring out what to do. That's probably my one takeaway, but grand will probably give a better answer for this.

Graeme Colman:

Not better. Another answer? And an additional answer, I think is the takeaways understand the scale at which you want to which which you are at and what you want to where you want to bake. If you're if you're starting out, then actually start small, small scale, it doesn't matter about standardization, when you're starting out, figure out what process is going to work for you and work for your organization for the things that you want to deliver. And that is start small, do MVP, get people on board, get people bought in, get as much of the tooling and automation in place for you to be able to just, once you want an MVP to iterate on that quickly, to continuously add value to every delivery that you're doing and continuously be able to show value for every every delivery cycle, or every sprint you're doing or whatever it is. That's getting into it, if you're already there. Think about the scale that you're at and where you want to be. Because lots of organizations have got to a stage where they're doing Agile delivery, they've got DevOps, they've become static in terms of the value that's been driven from their programs, then your DevOps and agile delivery, mainly because skills, resources, cost tooling skills transfer, you know, we were talking about people who are trying to scale trying to scale their DevOps capability. That's when you need to take a step back and kind of figure out, okay, we're now at a scale where we can't supply DevOps engineers or engineers to sort of to all of the project work that we need to happen and all the products that we're creating. So it's a step back and say, Well, we're only going to throw money on it and just keep on accumulating some technical debt. Or we need to figure out okay, what does our DevOps practices look like? What are the two standard tool sets? What are the what are the deployment tools, the end runtime tools, the cloud tools, the standardizing on this and providing standard ways of repeatedly growing your DevOps capabilities within different projects and products? And so it's, for me, it's about, think about scale of where you are and where you need to be and be cognizant of what the problems are just from the things we've been talking about.

Tim Stahl:

Gentlemen, I appreciate the time today. This is our first of probably, many of these. I'm not holding anyone to anything, just saying first of anything stocks. But you know, well appreciate your time today. And thank you so much. And from Appvia we'll sign off and you'll have a great day.