Speed AND Reliability: How to move fast and fix things [Webinar]

“Working closely with business and developer to find the happy ground is what you have to do, otherwise you’re going to continue to find problems. But try and map it back to your product success, and outcomes, and appetite in the market.” - Sam Hunt, VP APAC at GitHub.

What to expect in this webinar

In this Raygun webinar, our host Andre and guest host Eliza get curious about what it takes for teams to ship things faster, but still keep the focus on the end user.

They take a look at some internal processes that have worked for both companies, including keeping the end-user in mind, and how Agile and DevOps processes reinforce the culture of shipping excellent software.

Sam and Nick offer insights into how to balance chaos and control with a combination of communication and aligning engineering effort with business outcomes.

This was a rare glimpse into the inner workings GitHub - now at 40 million users over 100 million projects - and how they build software. You won’t want to miss the full recording above!

Show notes

• 4:14 How much time do teams spend resolving errors?
• 8:34 “Move fast with stable infrastucture” - not as catchy but certainly more accurate.
• 9:58 Developers are at the center of innovation for traditionally non-tech companies.
• 12:57 Does shipping get you excited, or fill you with dread? If it’s the latter, you need to revisit something.
• 16:05 How Raygun ships code
• 20:53 How GitHub maintain culture with collaboration
• 24:11 Product instabilities cause lack of trust from your customers
• 27:38 Dashboards are the only way to get visibility into issues affecting users and to maintain confidence as part of cultures
• 30:42 The metrics that teams should be measuring around software quality.
• 38:39 Does the responsibility of visibility and quality lie in the developers or the team leaders?
• 42:17 Audience question: How to manage tool overwhelm
• 45:00 Audience question: How to put ROI metrics on ideas and new features to get buy-in
• 46:25 GitHub uses early action groups for feature releases so they can understand their market
• 51.15 Closing thoughts

Resources mentioned

• Full side deck
• Raygun
• GitHub

For dashboards

• Geckoboard
• Grafana
• Raygun Dashboards

For Continuous Delivery

• Deployment Tracking
• Octopus Deploy
• Jira Software

Full transcript

Host: Andre All righty, so yeah, I think that’s it. We’ve got plenty of people on the line, let’s crack into it. So this exclusive webinar, brought to you by Raygun and GitHub. We’re going to talk about speed and reliability, so how to move fast and fix things. Really looking forward to hearing from Sam and Nick about both their kind of unique perspectives on this topic. Let’s run through the agenda very quick, I’m going to introduce these two and hand it over to those guys shortly.

They’re going to run really quickly over what is Raygun and what is GitHub for those that aren’t aware, and then we’re going to jump into the discussion part of this topic. Once again, we’ve got the Q&A section and Zoom, so if you have any questions, please throw them in there. We’re going to get plenty of time for the Q&A. So we’ll move through the slides and then we’ll move to the Q&A. We also have some prizes, don’t we, Eliza?

Eliza Sure do.

Host: Andre So if you have questions, we’re going to pick the best question or two and we’ll send those best questions, the person who asked them, some swag from Raygun and GitHub. But first, I do have a quick poll, and this is kind of just to get a little bit of information about you guys so we can kind of tailor the conversation around where you are at in the journey, so let me launch that poll right now. Everyone can see that? So the first question is, “How many developers are there at your organization?” We’ve got a multiple choice option there. “How often does your team ship code into production?” That’s quite an interesting one, and then, “How much time does your team spend on bugs and performance issues?”

Host: Andre Just, you know, you’re not going to know that exactly, but roughly how much time does your time spend on bugs and performance issue. So I’m seeing the results fly in right now. So guys, we’ve got, what, 30% have got between 11 and 50 on their development teams, 45% between one and 10. Most people are shipping code weekly, which is pretty good. We’ve got 30% multiple times per day. And then people are spending 10 to 25% mostly on fixing and spending time on bugs and performance issues. Maybe I can kind of share results with you guys just very quickly, should be able to see that now. Yeah?

Sam Hunt We can see that.

Host: Andre Excellent, so that’s pretty interesting. We’ve got a bit of a mix here: Smaller teams between one and 10, [inaudible 00:05:04] actually up to 50 make up 70% of the audience today. We’ve got almost 20% with 100 or more, so that’s pretty cool. Multiple times per day, a third of our audience are shipping multiple times per day. That’s definitely where things are heading, and then 50% weekly, so that’s pretty good going as well. Wasn’t long ago where it far less regular than that. And then yeah, a vast majority are spending between 10 and 25% on bugs, so we appreciate that. All righty, so let’s move on from that and meet the presenters. We’ve got Sam Hunt, the VP of APAC at GitHub, and Nick Harley, the Director of Product. Also dabbles in a bit of marketing from Raygun. Guys, I’ll hand it over to you guys. Take it away. Sam, you want to kick it off?

Sam Hunt Cool, yeah. Hi, thanks for the intro. Sam Hunt, I run the APAC region for GitHub, working with customers, community on delivering the world’s largest code development platform, collaboration platform.

Nick Harley And I’m Nick, I’m the Director of Product, so my job’s more or less connecting with customers and finding out what we need to build, how we need to build it, and then wrangling developers and design teams to try and get those through the pipe as soon as possible.

Host: Andre And Nick, what is Raygun?

Nick Harley Yeah, so Raygun offers a suite of products that monitor your web and mobile applications, and then when errors, crashes, and performance issues affect users, we give developers all the diagnostic details they need to fix them with greater speed and accuracy.

Host: Andre Excellent, over to you Nick, I mean Sam.

Sam Hunt Yeah, so, I mean, for those that don’t know GitHub, as I mentioned it, it’s the world’s largest developer collaboration and code management platform. It’s where most of the world’s open source code lives. There’s 41 million developers collaborating on a daily basis on GitHub. But it’s also an extension into commercial collaboration around code as well, so open source is extremely relevant to everybody’s development ecosystem, but not everything we do is open source. We provide an extension which is the same tool for your commercial collaboration and development as well.

Host: Andre Awesome. Speed & Reliability: How to move fast and fix things is why we’re all here today. We’ll start off with Mentality & Culture, Nick.

Nick Harley Yeah, so this is quite an important point to get across: The actual inspiration for this webinar title kind of came from Mark Zuckerberg and Facebook, and Facebook’s a massive organization as we know, and we shouldn’t all go out and copy exactly what Facebook’s doing. But this kind of “Move Fast and Break Things” was a motto and a mantra that Facebook had throughout their offices several years ago, and it got picked up in the press, and lots of people kind of quote it, because it’s quite catchy.

Nick Harley But a lot of people don’t realize that this actually got superseded not long afterwards, and a quote here from Mark Zuckerberg at the Facebook developers conference, so that the idea is that developers moving quickly is so important that we are even willing to tolerate a few bugs in order to do that. What we realized over time is that wasn’t helping us to move faster because we had to slow down to fix these bugs, and it wasn’t improving our speed. So they actually pulled back from this “Move Fast and Break Things” to actually go with this less catchy “Move Fast with Stable Infrastructure” type model.

Nick Harley And if Facebook, with all their resources, and money, and such a big company can’t actually pull off moving fast and fixing things, it kind of points to more of this problem. You know, I think Facebook in the early days could get away with it, because the company’s on a rapid growth trajectory, and it’s just way better for developers to actually fix things as they go along. And every company now is becoming a software company, so what do all these companies have in common? Well, they didn’t start out as software companies. They’ve had to grow into software, and better monitoring, and application performance is kind of the core aspect of what they do nowadays.

Nick Harley Because high street retailers have mobile apps, websites to manage, stock ordering systems. Even places like Home Depot, you can now order online. They’re now an online-first company, and if we don’t have good software experiences, people are just going to shop next door. And so right in the middle of there, Dominoes pizza, I mean, they’re a pizza company, retail. But if we actually look at their share price over time, they just … They use technology, if you’ve ever seen Dominoes pizza and what they come out with, with pizza trackers, there’s a pizza checker thing being advertised at the moment.

Nick Harley And if all of this technology didn’t actually work very well for users, then this share price would look a lot different. So developers are actually in this unique position at the moment where the faster they can build, it’s actually a market opportunity and a business outcome, that developers are actually empowering these companies to do well.

Host: Andre It’s interesting that the developers are in the center of this, aren’t they? The center of innovation for traditionally non-technical companies.

Sam Hunt Yeah, absolutely, and Nick, just to extend on one of your examples, look at the automotive industry. We all consume cars at some point in our lives, or most of us do, and from where the car industry competed 10 to 15 years ago, it was around colors, and wheel sizes, and speed, and gearboxes. And the reality is most of us go out and buy a car based on the technology that’s going into it. So there’s an industry that’s completely … It’s still very connected to its roots of being a manufacturing industry, but it’s completely shifted to how it competes to being a technology industry as well.

Nick Harley Cool, and if we move on to mentality within a team over monitoring, it doesn’t really matter whether you’ve got a New Relic, a Raygun, a [inaudible 00:11:13], whatever tool you’re using to monitor your software, you can buy a tool, but it won’t actually do anything for your team if you don’t use it correctly or have the team culture around monitoring of software quality. So when you are moving fast, you can actually spot issues and fix them as you go along. Just buying a tool is really irrelevant unless you use it properly.

Nick Harley And a lot of the mistakes that we see with development teams is they talk a lot about, “Oh, how many errors does my application have? How many problems does our application, our software, our infrastructure …” It’s very kind of internally focused, and we really need to break out of this kind of like, “It’s about our application, and our software, and our code,” because it’s really not. The outcomes are that it’s affecting your customers, and your end user experience, and your conversion rates, and the revenue of the business, and these are all outcomes.

Nick Harley The things in the middle are all tactics, you can choose what to monitor, you can choose which parts of your application that you need to pay attention to. But the outcomes are about your customers, and who really actually cares the most about these outcomes? It’s usually the CEO, the senior management, the executive team, the head of product. All the people in your organization that are more senior and you want to impress care about these outcomes. So when we’re actually looking at being able to communicate these things within our teams, it’s best to focus on these outcomes. It’s like the biggest career hack you can have, just to take what you’re doing at the code level and then talk about them more at an outcome level and how it affects the business.

Nick Harley One of the key measures that I look for in a team when I ask engineers is do they actually look forward to shipping new features, or does it kind of fill them with this sense of dread? Like, “Ah, if we push this live it’s going to actually go wrong, and there’s no real safety net.” So if you can actually get your team to think about this, do we have confidence in shipping what we’ve built? Have we got the monitoring in place to make sure if it does go wrong then we can pull it back? That’s kind of the litmus test of have we actually got this culture and this team mentality to the right place?

Nick Harley And deploying with confidence really comes from how will we actually deploy code safely as a team, and how can we increase the frequency to several times a day? And continuous integration and continuous deployment are essential for kind of quick development cycles. So this is kind of the first step we need to get to. On the poll there, people are at kind of mostly weekly deployments, but is there an appetite in the business to make that daily or even improve it just from where you are?

Nick Harley And if you can get there, things like code reviews, pair programming, bug bash Fridays, all these types of things can actually help you get your software quality improved, and the speed that you are shipping software can improve as well. And if we put deployment and release tracking around issues, we can actually ship with confidence, because we know that we’re going to catch them once they go out into production. So gone are the days where we used to have all our kind of changes rolled up in one big release, and we released it on the shipping day, and everyone would get really nervous around it. If we can actually increase the frequency that we ship, we’ll actually see that have huge benefits across the business, because things are going out in smaller pieces.

Sam Hunt Nick, can I just … I really like this, and I think it’s key, and I’m going to talk a little bit more about this. I’m obviously not so much from the technical side but more from the culture side, but this really, really draws to the point that tool’s a part of your team, and getting the feedback from them in context of what you’re trying to deliver as an outcome based on your previous slide is a really, really important part of trusting that whole ecosystem that’s delivering. I mean, the reality of the reason why we can deliver so much code now is because we’re able to trust and automate with the great tools that we use, the parts that just become the bog down of your day to day. And I think it’s a really important point.

Nick Harley Yeah, that’s exactly right. It has to start with your team and your culture, and the mentality you have around this stuff. So how can you actually improve from where you are today to ship faster and with better quality?

Sam Hunt And you’re kind of comparing your team to the best in the world these days, with people’s expectations and [inaudible 00:15:55] Netflix, or Slack.

Nick Harley Yeah.

Sam Hunt And so you’ve kind of got to be looking at the best of the world and kind of moving towards that directional big part of that journey, which is changing so quickly.

Nick Harley Yeah. I mean, I read something a while ago around Amazon shipping multiple times a day, hundreds of times, hundreds of times. But I thought I’d include this example of how we do it at Raygun, just to give people a perspective of our … We could always do better obviously, but we feel as though we’ve got quite a good process going on in this area. And so we use Jira for our issue tracking and prioritizing work. And then once that goes into the dev team, they use their own kind of tooling. We don’t have too many kind of restrictions on what they use, but things like Rider and Visual Studio are pretty prevalent within our team.

Nick Harley And then we use GitHub to manage our source code. And when people commit their code, we run automated continuous integration builds, which will fail if there’s a problem in the application and people can fix. And then we use Octopus Deploy to actually deploy our code to beta environments, test environments, and promote it to production when it’s ready. But while it’s on the beta environment and the integration tests are passed, we’ll get other people on the team to code review everything that will go out the door. And these changes they need to make will go through that process again, but if it’s okay we’ll merge that into the master, run that through the build again, and then we obviously put it into production, and we have Raygun watching for alerts.

Nick Harley We tag the deployments. With Raygun, there’s a deployment tracking functionality, so you can actually see which issues you’ve introduced, and we have that hooked up to Slack as well. So we know if we were to ship something to production, if we start getting errors in our Slack channel, then we know we can roll it back pretty quickly. But hopefully that doesn’t happen because of the QA, and the integration tests, and the safety nets that have been put in place. And then as we do have issues in production, because nobody’s perfect, we can … Well, we can deploy pretty easily, and we can roll back pretty easily.

Nick Harley So with Octopus Deploy, we usually run it on one of the office environments and then promote it into a beta, which is as close to production as possible, before promoting it to production. So we’re actually in this situation where we can roll back deployments within seconds, and we can ship to production within around five minutes. The biggest thing that takes the most amount of time is actually the build server in that process, to actually build the code. And if people are looking to get a bit more information on this, Ollie on our team is on our dev team, and could probably speak to it more than me. At this Bitly link at the bottom, there’s a blog post there that goes into detail about how we actually ship software like this.

Nick Harley And so the last point on this culture section is you’ve signed up to this webinar because you have an interest in actually moving faster and fixing things as you go along. But you really need to kind of stand out in your organization and actually make this happen. So I’ve spoken to multiple people who have actually had success with driving this culture within their team, and they always feel like they were a little bit going out on a limb initially. They felt like they didn’t have the buy-in. But when they spoke about these business outcomes and they tied it all back to outcome-based approaches, they actually got the buy-in of the executive team, and were able to kind of accelerate their own career as well as improve the organization in their engineering team.

Nick Harley Because they stuck their neck out and say, “Hey, well, I’m not comfortable with what we’re doing right now. We can do better.” So I think you know, it is going to accelerate your career and get you more respect if you just try and do this continuous improvement in your team, and the end result is better software for your customers. So there’s no real downside to wanting to improve software velocity and quality at the same time.

Sam Hunt Yeah, so I think sort of extending on that from the culture perspective, which is really where my expertise more comes into it than the technology side of things, there’s really three key areas that we want to focus on in this. And the first one, we’ll cover them off, they all have slightly different tangents, but the first one is really getting over the fear factor around your work. And that’s where there’s relevance around how tools can help you do that, but also how people can help you do that as well. But the work without fear is shifting as well, and it means different things to different people.

Sam Hunt You can see on your screen right now, you see a lion. That lion, kind of for me, that’s the 20th century, that’s the CEO of an organization. They work without fear, they own the risk, and they make decisions. But the reality of the 21st century is we’re not lions. We’re honey badgers, right? Honey badgers work without fear. We have to be in there, we have to be aggressive, and there’s two things that really help us do that: One is to be comfortable with having lots of eyes on things and collaborating on our mistakes and our wins as well, and the other is relying on tools. And this is a great shift in the way that businesses see the value of technology as well.

Sam Hunt So that lion’s still there, but that lion now respects what the honey badger means to their organization, because let’s face it, we talked about every company being a technology company a little earlier, it couldn’t be truer. And this, the importance of what you’re doing at a development level resonates all the way up. So if we take that, how do we make that relevant? It’s around collaboration. So we talk about collaboration within our own developer community and our own teams, but it’s also collaboration with the business. It’s really, really important.

Sam Hunt And as I said before, collaboration with tools. Tools are part of your business as well. So the relevance to the information that they’re giving back to you has to be taken in the same way that you would take feedback from a colleague that might be doing a code review, or you’ve asked a question for, and it’s all about how those pieces collaborate together to bring outcomes that are in line with what we’re all trying to achieve. Now, here’s the hard bit: We’re all trying to achieve little different things. So if we go and look at what a developer wants, well, the reality is they kind of want chaos.

Sam Hunt They want to try a lot of things, they want to play, they want to come out with innovative outcomes, but business don’t always want that, right? They want some control there, they want to mitigate risk and make sure that there’s some sort of boundaries. So this is where we talk about finding a balance between your control and your chaos, and as you see the railroad tracks on the screen, it’s kind of like the business owns one side and technology owns the other. And you need to collaborate and come to agreement on where your rails sit and how wide they are or how narrow they are.

Sam Hunt Now, the downside is you can get really, really tight on the control, but that narrows your ability to innovate and ship at speed. And then there’s a downside to being really, really open, because you lose a lot of the control. The interesting fact is, it’s usually your consumer or your industry that’s going to dictate where those rails should be. So the natural balance here is where you have the rails in a position where you can compete and stay up to date with what the market’s asking you to do. So I’d ask yourself to take a step back and look. If you feel like you’re falling behind your competitors or your industry demand, your rails are probably too close.

Sam Hunt If you feel like you have trust issues with your consumer because of product stability or inconsistencies, your rails are probably too wide. And it’s really, really important, pulling it back into the collaboration piece, because we haven’t always been good at having business talk to technology and coming out with aligned outcomes, and that’s what’s going to help us succeed. It’s not just a developer decision, it’s not just a business decision. It’s a company decision for outcomes, and remembering that those outcomes, you have measureables in the market where you can figure out what’s important, what am I missing at? It’s not hard to do.

Sam Hunt Tools like Raygun, they really give you an opportunity to have another insight, and it gives you, in addition to that, it gives you great insight at speed as well, and that’s the important thing. So I think if we can focus on those three points, working without fear, and again, that can mean a lot of things: It could be fear of criticism is one that I actually hear a lot, developers being worried that somebody’s going to put eyes on their code and criticize it. Be conscious of how you criticize, be constructive in how you criticize. Because really, you want that.

Sam Hunt We all improve better by getting constructive criticism back, and it is a cultural shift away to be comfortable with that. That ties into the collaboration, so really, really important that collaboration is a key part of our strategy moving forward. And again, as I said, it’s not just collaboration within ourselves. It’s collaboration within our industry, it’s collaboration within open source communities. We’re probably all leveraging open source technology that helps us deliver, because we don’t want to reinvent the wheel. And then the last one is identifying the boundaries and those rails, and how we’re going to work closely together with aligned outcomes that are going to allow us to develop at the speed that we have to to either compete or stay relevant.

Host: Andre That’s some healthy kind of conflict there, isn’t it, between business objectives and being safe to what basically your customer demands as fast innovation, not affecting the performance or the experience.

Sam Hunt Yeah, absolutely, and I think people forget that your product’s being consumed by something, someone. And it’s what they think of it as much as anything else. And if we look at disruptors, so the Airbnbs, the Ubers, the Netflix of the world, they changed industries. And if you didn’t stay relevant, you disappeared.

Eliza: Guest host They’ve changed the industries, but they’ve also changed our expectations.

Sam Hunt Yeah.

Eliza: Guest host I know a lot of governments are finding it difficult because the way that they operate doesn’t really allow, but we expect apps to be able to change deeds on houses and things like that now. So consumer expectations have certainly shifted.

Sam Hunt Yeah, I actually, I was on a panel in Sydney yesterday. I was talking to Eliza about this earlier, and just before I went on the panel my daughter emailed me from her science class asking me to send her a design for a mousetrap [inaudible 00:27:25], right? Her expectation is that I’m available right now. Assume that your consumer is in the exact same position.

Host: Andre I mean, it’s changing so quickly, you know? That’s constantly moving.

Nick Harley Yeah, so we talked a lot about kind of technology and teams, but how do you actually get the buy-in to some of this stuff? And for me, I am a personal fan of dashboards, because having dashboards up in the office is just really powerful. People know what’s going on, things like that. But what I’ve encountered across multiple companies is the dashboards just show the success metrics: “How many sales did we make? How many goal completions did we have?” Things like that. But it takes a very long time for those metrics to start sagging before anybody notices, “Oh, there’s actually something going on under the hood here.”

Nick Harley So I’m a big believer in having really raw metrics of like, “Tell me when this stuff is broken on the dashboard in the office.” And what does your team actually need to pay attention to? So have this discussion with your team, “What do we need to monitor?” We actually have this concept of canaries here at Raygun where we have these kind of signals, like, “Nobody can pay, nobody can do this and that.” And thankfully, these canaries don’t go off very often, but we’ve got them there to tell us straight away when something’s broken. So how can you put up dashboards in your office that kind of tell you when things are going wrong so that somebody doesn’t have to go a month later and go, “Why are sales down so much this month?” and you have a checkout issue or something like that.

Sam Hunt Yeah, well again, it comes back to tools being part of the team.

Nick Harley Yep.

Sam Hunt It’s really, really important, and they’re providing you valuable context to your business.

Nick Harley Absolutely.

Eliza: Guest host And I will say, sitting here in the Raygun office, they certainly walk the walk. There’s lots of fun things to look around on the walls.

Nick Harley And that visibility’s actually appreciated by other people in the business. So when other people are walking around, they’re kind of, they’re seeing that the engineering team are using these dashboards, and they’re kind of assured that people are on top of this stuff. Like, they’re watching it. And so marketing have had their own dashboards, had their own metrics. But in your engineering team, can you put these dashboards up to give your business visibility that the engineering team are trusted? They’re on top of it, they know things are going to go wrong, and that’ll actually give you far more kind of trust and validation from the people within your organization.

Nick Harley And what should you be really measuring? Because cycle times, and points completed, and sprints completed, and all these types of things, they’re all productivity metrics. And so they’re quite prevalent in engineering teams, but they talk about the amount of work done rather than the quality of the work done. And we actually have turned off our net promoter score system recently, because it just wasn’t giving us good, valuable insights. But that’s another thing that people use to kind of try and measure whether they’re being effective in the market.

Nick Harley But better kind of metrics to even put into this mix, so, “How many errors does our application have, how many bugs have we introduced, how many users have been affected by bugs, have we got slow performance, how slow is too slow? So if our checkout page takes three seconds to load, is that acceptable to us? What is an acceptable load speed and can we put measures in to actually alert us when these problems arise, and can we report on them over time? How can we improve?” Also, another thing we watch here at Raygun is how many technical support tickets for issues that users experience coming in each day, can we actually get that down?

Nick Harley Because any day where we get zero support tickets is going to be a great day, and it’s going to save the business a hell of a lot of money. And also, there’s two ways to look at this: If you’re kind of singling out people that are introducing issues or fixing issues, you shouldn’t use it as a stick to kind of embarrass people or beat people up on the fact that they’re introducing issues. But it can be a measure to kind of say, “Does this person need some help? Are we teaching them properly? Can we pair them up with somebody else?” But also rewarding people who are fixing the most issues as well, and that positive reinforcement about, “We care about software quality,” and singling out people for stuff they’ve done that had a big impact on the business.

Sam Hunt Yeah, and Nick, I don’t know whether this is relevant, I’m overstepping my boundaries as a technologist or not, but that person that’s introducing issues might actually be identifying something that was downstream that could be fixed.

Nick Harley Yep, absolutely. I mean, the amount of times you’ll find an error but it’s connected in some way to another error. But again, if you’re fixing things, that has a business impact. If it’s a customer-facing bug that you fix, and no longer do 50 people a week encounter something, that’s going to get the attention of the people above you, and the product person, and you’re going to get recognized for that type of work. And you might run into people who view it … I’m in the product team, so I’m always running into [inaudible 00:33:00], going, “Yeah, yeah, but we need to do this piece of technical debt, right,” and it’s always a fight between product roadmap, infrastructure roadmap, fixing technical debt.

Sam Hunt So reframing technical debt can really get you more buy-in. You don’t really want to be kind of saying to the problem, “Yeah, yeah, well, but we need to add a new node to our Elasticsearch cluster, and made some critical maintenance on our SQL server data …” To a non-technical person, they don’t understand that. They’re just not going to get buy-in, so your kind of technical debt’s going to get right down the roadmap. So four kind of things to try and tie technical debt back is does it actually help with performance, does it help with reliability, does it help with scaling, and does it help with security?

Nick Harley And if you go to the product manager or whoever runs your roadmap and you just say, “Look, if we don’t do this, we’re going to run into scaling issues. We can’t onboard new users, we’re going to run into these performance issues,” and tie it back to these four things, you’re going to speak in a language that they can’t say no to. So your technical debt priorities are going to get way more buy in, rather than talking to them like we said earlier, talking about your application, and your code, and all that, speak about it in customer outcomes and benefits to the business, and you’ll instantly, overnight get way more buy-in on technical debt issues.

Sam Hunt Yeah, and I think remembering that technology only owns one side of the rails, so if it makes a lot of sense to both sides, it’s going to get you a result much faster.

Nick Harley Yeah, we actually, we prioritize our roadmap, product, and infrastructure together, so we can have a conversation together around like, “Does this make sense to do this before this?” And it could be a technical [inaudible 00:34:44] and the engineering one to kind of complete, and they might say, “Well, we need to do that piece of work on the database before we can actually do this other piece of work.” So try and roll them up and have them as separate roadmaps. And another thing you might run into is people say, “Oh, I just don’t have time. I’m so busy trying to ship these product features and there’s a deadline to meet.”

Nick Harley But actually making time for software culture doesn’t have to be that time intensive. So one thing we’ve found with our more enterprise customers that they seem to resonate with is in Raygun, the errors get picked up. And if we just rank these by how many users are affected, and then take the top two items and include them in each sprint, it’s a super easy way to just be chipping away at software quality over time. And if you just use the highest impact errors, or issues, or performance problems you can actually fix, and then put those one or two in every sprint, you’ll actually make a big dent in your software quality over six months to a year.

Nick Harley So just to recap, talked about mentality and culture, to get your team on board with actually improving systems and processes to actually be able to move fast and fix things at the same time, and not break as much. We also talked about how do you personally invoke the change in your organization? How are you going to actually make improvements and get the buy-in? So talk about those outcomes to your leadership team, and your managers, and your product person, and speak to it in the business metrics, and the impact for customers, and stuff like that. And then Sam talked about modern teams, obviously the honey badger reference. Do you want to recap?

Sam Hunt Not the rugby player, but no, I just think it’s that work without fear, and it ties into better collaboration as well. The other point I didn’t cover off but I’d like to cover is when you collaborate, and share, and track, and keep this information, it’s all useful, it all helps you speed up your business. Because you can not only look at … It’s not your product that’s morphing, and transforming, and getting better, or getting different, or whatever, or more relevant, it’s also the decisions that are driving that, that you can now learn from and use as experience when you hit another fork down the road. And if you don’t do that, you’re missing an opportunity to help. Again, tools, people, information, these are the things that are going to drive good decisions about our product and our go to market.

Nick Harley You build that muscle memory, don’t you?

Sam Hunt Yeah.

Nick Harley But for teams who kind of constantly change and kind of incorporate new ways of doing things, they’re already ahead of the pack, because they’re doing it. They’re building their muscle memory, they’re learning, and it turns into a distinct competitive advantage.

Sam Hunt Yeah, and it could be as simple as, “Oh, we made this decision. We’re weighing up something new in a new market or demographic that we’re maybe targeting, and we’re going down a certain path.” And some of them might say, “Well actually, we looked at that. It’s very similar to this scenario two years ago, and in fact what we found was this was a much better way forward for us, and we’re able to do it really fast.” And then, to the second point or to extend on that, resusability of what you wrote back then into this new project. So things like that, the historical information and historical learnings are a massive improvement with better collaboration, and they’re a big asset in your organization.

Eliza: Guest host What I thought was really interesting about what Nick was saying, in my world I talk to the community a lot, developers, and you know, they’re obviously the honey badgers of the world, which I’m going to be saying a lot now. But then also, business leaders who are managing IT departments and the tech of businesses, and where does the responsibility lie? Does the business need to brief the developer beyond the scope so they can foresee the business strategy and where it’s going to go, or do business leaders need to learn to speak tech? But I think, Nick, you gave some really great examples of how technical people can use language and talk in business cases to actually fight that point better and vice-versa, so I think that’s really helpful.

Nick Harley Yeah, I mean, it goes both ways. I mean, developers, they need the context on why they’re building what they’re building, but also, coming back, they need to talk in the language of the people, in the product teams, because they deal with metrics and business outcomes as well. So that’s the whole reason behind putting those in, because it is a common issue, to bridge that kind of back. And that’s where kind of closing the feedback loop ties it up, because if you’re struggling to get buy-in, if you’re struggling to get the things you care about, you’re probably just not speaking the right language to the people and the business in order to get the buy-in.

Nick Harley So once you’ve got the tools set up, and the culture, and you know how you can kind of innovate quickly within your team, getting the buy-in from the rest of the business, not only will this accelerate your career, it’ll also make you kind of the hero of the organization. Because the Jack or the Jill of the development team is the one who’s pushed us into the new realm of what we can do, and now we can ship faster, and the software quality’s improved and, “Look at the error chart? You know, it’s gone way down, our application’s much faster.” But this doesn’t happen overnight, you know, to get the buy-in of the whole business to make this happen.

Host: Andre Very interesting. So we’re right on track in terms of the time, so I know we had a couple of questions that were emailed through, but how is it looking for questions over there, Eliza?

Eliza: Guest host Well, I promised Keith I’d ask his question: Keith wants to know, “What do you do about tool overload?” So there’s a new tool every week, so you’ll do really specific things, do you see consolidation coming into some areas, or how do you assess-

Sam Hunt Yeah, so that’s the chaos and control bit that I talked about, and it’s not just around having control on aligned outcomes, it’s also tools. And it’s actually quite a few of our customers that this has really resonated well. They want to give flexibility for developers to make decisions about tools they want, but they want control. So it’s like, you have to find a happy medium there, and you’re right, tools change in organizations. 70% of tools change every 12 months, but having an agreement on how closely or openly you allow that to happen is the only real way to control it.

Nick Harley Even the [inaudible 00:41:34] space has changed massively in five or 10 years, hasn’t it?

Sam Hunt Yeah.

Nick Harley The tech stack is so varied now, you know, in every department.

Sam Hunt We see it a lot. We’re effectively at the heart of a lot of development, because we’re most of the source. I mean, it’s not just GitHub, but it’s Git in general. So there’s lots of tools. When somebody brings a new tool to market, it integrates with GitHub, and so everyone plays with GitHub and their new tool as well. It’s not an easy one, but again, having a reasonable set of those boundaries to give you some flexibility for developers that want to try new stuff, because the reality is some of those tools might be really successful for your organization.

Sam Hunt So you don’t want to discount them just because it’s a lot of them, you just want to control how many you bring in, and also rationalization: If it’s like, “We have five tools that deliver this same functionality,” then you really need to look at what the difference is with the new one as well. It’s very easy to go, “Shiny new thing.”

Nick Harley We usually go in cycles of certainly the companies I’ve worked for, is every 12 or so months you do a big cull, and what ones aren’t as useful. Because you do have to have an appetite in your tools, because the ones that do kind of break through or do kind of prove to be valuable, you actually almost can’t live without them at a certain point.

Sam Hunt It’s true. Just on a potential way to manage that outside of your having drivers that are going to put pressure on production, GitHub, if people are using enterprise or enterprise cloud, you can run up instances to test new tools, to test the viability of them bringing value to your environment as well, without actually having to introduce them in.

Eliza: Guest host Another tactical question, “What are some open source dashboard tools you could recommend?”

Nick Harley I don’t really know too much about the open source ones. I mean, we use our own internal Raygun dashboards, the product here that … I hear Grafana come up a lot with our customers, and our marketing team use a tool called Geckoboard as well, which is not open source, but that’s also probably not quite the tool for developers, but Grafana is just one that seems to come up a lot with customers. But we have our own dash boarding functionality within Raygun, so I don’t really know any open source ones.

Sam Hunt Yeah, I’m not that familiar with them. We’re happy to take that offline and come back with any recommendations. Our support team are very much across a lot of these tools. What I will say is what comes up quite often is that people don’t realize that the API into GitHub is extremely open. So if you have dashboards that you’re using within your organization and what you’re looking for is access to data that you want to analyze and have presented, it may well be available already, and you can build it into something that you’re currently using. It’s definitely an area of focus at GitHub from a product perspective to deliver a better dashboard and insight experience for organizations and developers.

Eliza: Guest host Another question here, one from Ben: “How can we put …” I think, Nick, this would be a good one for you, “How can we put some ROI metrics on the ideas and new features to get that buy-in, where there’s no time to experience the features?”

Nick Harley Yeah, it’s a good question. So the way we’ve done it internally, and that’s the only kind of experience I can speak to, is we have our product roadmap, but then we’ve actually added fields alongside the work to say, “Rank the customer impact, the development time involved,” and all these kind of criteria that sit alongside, so we can actually rank them based on the business impact, and the customer impact, and the development time that it’s going to take, and we can kind of evaluate them side by side.

Because you’re never going to actually know until you put the product live or the feature live, but if you can kind of talk to the impact that that particular feature or new product’s going to have on the organization, you’ll be able to rank them against each other. So I would say adding some extra supporting information or rankings for everything you want to build so that people know, “If we actually build this, we’re going to have quite a big impact on our revenue or usability, and-”

Sam Hunt I think if it’s within the boundaries of your appetite for risk, have a good early access program, have your trusted advisors inputting into product and feature as well. GitHub does that. We’ve recently gone through early access programs for new features like actions, and it’s a really, really meaningful feedback process for us to make sure that we can make the product as relevant as possible as what people are looking to achieve.

Eliza: Guest host Another question here about making the deck available, I assume absolutely?

Nick Harley Yeah, for sure. We’ll email out the deck and the full recording as well.

Sam Hunt Yeah, just after the flash pictures of us, [inaudible 00:46:52]. I’m kidding, sorry.

Eliza: Guest host Should we go to email for a couple more?

Host: Andre Yeah, I’ve got one here, so, “What are some tangible steps I can take to speed up my time to ship without increasing the risk?”

Nick Harley Good question. Well, I guess kind of playing within the realms of what you’ve already got set up, right? So the instant one for me, well, how are you going to make the build server times actually less? You can actually do things to speed up your build times, you can upgrade servers and things like that. But you know, going out and buying new tools and stuff is just really not going to help you at that stage. You need to take a look at what you’ve already got, how you can optimize it, how can you optimize the process of shipping code, how can you reduce how many bugs get introduced by doing QA with your teams? But it’s going to be a real kind of case-by-case basis.

Sam Hunt Sorry, yeah, I was going to say, from a GitHub perspective, one of the things that I hear frequently from our technical people talking to customers is, “Get comfortable,” and this is a culture change, but, “Get comfortable with creating early [inaudible 00:48:04] requests.” So if you’ve got something you’re working on, get eyes on it fast, because assume that if you can get through coming up with an outcome with two eyes, if you have 100 eyes on it, you’re going to get there a lot faster.

Host: Andre Is everyone moving to dev ops in terms of having all the resources within that same team, to be able to manage that kind of quality and put it out there quicker?

Sam Hunt Yeah, well, I think dev ops is a marriage between the tools side and the developers side to make it faster. But I think what people assume is that dev ops will fix problems, and dev ops doesn’t fix problems. Cultural change fixes problems. Dev ops supports a better mechanism to run at speed, but it’s really the culture and the people that are going to adopt that. So it’s really, really important, but what we have seen, and we actually saw this a lot in the APAC region when GitHub started to engage more with the community down here, we saw that having a dev ops strategy did not transform the speed at which you could ship. Having a cultural change program that allowed people to get the best out of the tool set that they were putting on that was supporting that dev ops was much more effective and successful.

Host: Andre Cool, so we’ve got one more question: “How do you find the balance between control and innovation?” And you kind of spoke to the guardrails.

Sam Hunt It’s really about understanding your market. Like I said, at the end of the day, if you’re falling behind, if your competitors are bringing out product ahead of you, and you’re becoming less relevant to your audience, your rails are probably too close together. If you’ve got trust issues with your product because of failures, or stability, or inconsistencies, then your rails are too open.

Sam Hunt So what you’ve really got to do, and actually this kind of ties back into having a really good early access program as well, is if you’re running an early access program, you’re getting feedback before you hit production. Don’t underestimate the importance and the value of the feedback from your audience as part of your developer ecosystem. There’s no hard and fast rule, though. There is no, “You should do it this way.” Every organization’s going to be different. Working closely with business and developer to find the happy ground is what you have to do, otherwise you’re going to continue to find problems. But try and map it back to your product success, and outcomes, and appetite in the market.

Host: Andre It’s certainly a key theme. Any other final points guys, before we wrap this thing up?

Nick Harley From me, I just hope that there was something in there for everybody that they can take at least one point away, make some kind of change in their organization to improve. There’s no kind of quick fix, it’s just going to take that kind of culture shift, communication shift, and then putting the right tools and processes in place to actually get there eventually. You’re not going to get there overnight and there’s no silver bullet.

Sam Hunt I’d just like to say thank you to the Raygun team for inviting GitHub to be part of it. I think it’s great to be able to sit down with local innovators and actually have a discussion about our industry and what it means to an audience. So appreciate it, and appreciate everyone’s time.

Host: Andre No worries, and one thing I … When I talk to customers, and I always ask kind of how their workflow has changed, and I used to say “five years”, and I’d say “night and day”, and I’ve changed that to “three years”, and they still say “night and day”. They almost can’t believe how they used to do it three years ago. So I mean, people shouldn’t feel overwhelmed about the speed of change.

Host: Andre It’s almost an opportunity to kind of … No one has it sussed, it’s kind of you’re constantly learning, there’s constantly new tools, there’s constantly new ways to innovate, and introduce exciting new features, and leverage tools in your team, and whatnot. So it’s just having that appetite and comparing yourself to the best in the world, actually, it’s to remain competitive. And I loved your point around developers being at the center of this innovation, and it ends up, for all these companies that are traditionally not typical companies, it’s actually the developers and the product team that are driving that innovation and driving that competitive advantage. So you’re in the power seat, so it’s a really exciting place to be. All righty guys, we’re going to wrap it up. Cheers, Eliza.

Eliza: Guest host Thank you so much.

Host: Andre We’ll choose the best questions and send out some swag, so we’ll be in touch, but thanks so much for everyone for joining us today, and we’ll send you the recordings and the slides later on.

Sam Hunt See you in the next one.

Host: Andre All right, see you later. That should be done.