Hiring strategies for high-performance software teams [Webinar]

By Freyja Spaven | Posted Sep 3, 2019 | 54 min. (11458 words)

“This is a sellers market. If in many cases the best developers and skilled professionals for the most part across the board are not going to tolerate a culture that does not feed them in that way, that empowers them to do good work, and they’ll move on.” — Dave Swersky, Lead Engineering Manager and Author

What to expect in this webinar

Today, our host Andre talks with Dave Swersky, Lead Engineering Manager and author, and Jeff Langston, Freelance Software Engineer, and Consultant working with freelance developer platform Gun.io.

With almost 30 years’ combined experience, Dave and Jeff have some unique insights to share on how to build a great team - from achieving a cultural fit to onboarding new hires successfully.

These processes are used by Gun.io to hire hundreds of senior engineers for companies every year. Listen as they discuss what’s worked (and what hasn’t) when hiring for high-performance cultures, and don’t forget to check out the guest blog post by Gun.io on why building a great team starts with a great hire.

Show notes

[06:16] Leadership is a science in this competitive landscape
[08:19] Are you missing out on the talent pool by not hiring remotely?
[13:41] Developers hiring developers isn’t always the best way to hire
[15:06] Just like you want your customers to have a good experience, you also want your candidates to have a good experience
[16:35] How to test for problem-solving skills in senior engineers properly
[18:43] Authentic culture extends from values - do you have them in place?
[29:57] The role of good documentation in onboarding
[31:56] The clear channels of communication needed for fixing bugs and shipping code
[38:19] Why you can’t afford to “move fast and break things” anymore, so hire with that in mind
[47:27] How user experience metrics can help understand if you have a high-performing team (or not)
[50:40] Get developers on support early so they understand the impact of their code
[56:00] Make sure everyone has access to the best tools so they can do their job effectively
[59:54] Monitor what matters to help keep the team focused
[1:02] Assess your circumstances and apply some of the learnings from today, and you’ll be well on your way

Resources on hiring high-performing teams

Full transcript

Andre

All righty, so how to build high performance software teams. So today we’re going to talk about tips for maintaining quality and velocity when scaling your dev team. If you’re anything like the companies we work for, there’s a big demand for developers. If you hire the wrong person, it can really slow things down. Just kind of building that team and the culture and the processes to be high performance, I think is a really topical discussion at the moment.

Andre

Things have changed so quickly when we talk to some of our customers. I always ask everyone, “How’s your workflow changed in the last few years?” Everyone pretty much says night and day and there’s just so many different tools and ways people can build. We’ve got the cloud. If you’re not comparing your workflow to the folks like Netflix, you can really lag behind on that. You can lose some of your competitive advantage.

Andre

So let’s jump straight into it. Here we go. So agenda today, which is going to give you a really quick rundown of what Raygun and Gun is. Then we’re going to jump into some intros with Dave and Jeff. They can kind of tell you a little bit about themselves and why they’re joining us today and then we’ll dive into the discussion. As mentioned, we’ve got a lot to get through. So the challenge will be to kind of keep this moving.

Andre

Also, we really want some interaction with you guys. This is a discussion as you’ll see how we structure this. We’ve got different questions and Dave and Jeff come from different perspectives so it’s going to be really interesting to hear kind of their take on some of these key topics. But we’d also love to hear where you are in this journey as well.

Andre

So if you have any questions or even comments, contributions to the discussions we’d love you to put that in the discussion box in Zoom. There’s also a Q&A section. If you have any questions, please put the questions in there. We’re going to have plenty of time at the end to cover all those questions. So please use those features. We’d love to hear your thoughts as well.

So just really quickly, if you haven’t heard about Raygun, it’s a monitoring tool. It helps you kind of monitor the health and performance of your app or website. We’ve got three products, real user monitoring, which is the front end so you can see the user’s experience, how quick things are loading and what the flow looks like, where they’re getting some snags.

Andre

Then we’ve got error reporting that covers front and backend and that will let you know about any error that’s happened in your app. Also, we’ve got integrations with the likes of GitHub so you can dive straight into the line of code. The idea is to give you the answers so you don’t have to look through logs.

Taylor

Hi guys. I’m Taylor again from Gun.io. Just a super quick overview of what we do. We hire senior level freelance engineers with 7 to 10 years of experience. All of our engineers are pre-vetted through technical interviews, live code tests and reference checks, which allows us to match candidates with our clients in 48 hours or less. Given that our engineers are passionate about the project and they meet the clients tech stack needs. Our engineers are available part-time, full-time and freelance to hire. So you can learn more if that’s a pain point for you. You can learn more at Gun.io.

Andre

Excellent. All right, so let’s jump into some introductions. Who wants to start? Dave, you want to give us a little bit of a rundown about where you’re at, how you got to where you are today?

Dave Swersky

Sure. My name is David Swersky. I’m a staff writer for Raygun, but my day job, I’m an engineering manager for a software and services company focused on healthcare based here in Kansas City. I’ve worked for a lot of different companies, a lot of different types of companies. I’ve worked for software companies, real estate companies, banks, federal government. A lot of my career starting in about 2014 has been focused on dev ops, kind of the managerial side of it and also the technical side.

Dave Swersky

So that’s been a real theme for my career, leading up to now and then going forward. I actually wrote a book on dev ops tools, Docker, Git, and Jenkins. If you go to the website, devopskatas.com, there’s a link there for a discount if you’re interested in those tools. A lot of really useful exercises to learn those key dev ops tools. Just a lot of different types of background experience. .NET developer by heart, but also, dev ops and engineering management.

Andre

Excellent. Thanks mate. Jeff.

Jeff Langston

My name is Jeff Langston. I am based out of Utah and currently a freelance software engineer and consultant primarily working with Gun.io on a variety of different contracts. So my background, I’ve been in the engineering sector for a little over the last 10 years, working in a variety of different problem domains around ERP systems and e-commerce, GIS. I am a full stack developer.

I got started in Java and spent some time in sort of an enterprise setting and then have shifted through a bunch of different language and had a lot of different exposure or different problems with Gun and doing that for now. So, happy to be here.

Andre

Cheers Jeff. What I’m really excited to hear both your opinions because you’ve got from one side working for an enterprise growing a team, and the other side coming into teams in different various companies, various projects. So it’s really interesting to hear what works from your perspective. Seeing all the different ways people do things, I think will add a lot to this conversation. All right guys, well let’s dive into it.

Something we kind of started with was leadership. It’s kind of, you’ve really got to take this seriously and you’ve really got to treat it as a little bit of a science, building high performance teams. How do you attract and select high performance talent for starters? I know how competitive it is out there. So Jeff, I might start with you here because you’ve had a little bit of a different experience across various different roles. What are the key things to attracting high performance talent?

Jeff Langston

Attracting high performance talent is sort of a two fold from the contracting world because as an organization, and we want to appeal to engineers who maybe want to sort of go out on their own and attract talents that we bring onto our bench. So that comes through really a referral system and to reaching out to folks and a recommendation system. All of the engineers that are on the Gun.io bench have been through a vetting process like Taylor mentioned through live coding exercises and technical interviews and sort of cultural fits.

Jeff Langston

Then also, for everybody that matches up with projects, they go through and an opportunity to interview with clients. But broader than that, really I think the perspective between first comes down to the decision of the type of role that you’re looking for and if you should be looking for somebody full-time or if maybe bringing somebody on through a contracting organization might be a better fit as far as looking at the what type of permanence is the role.

Jeff Langston

Is it somebody that you need to come in for a short term project and to get something done or maybe an outside expertise on a particular new technology that you’re looking to introduce into a project and really what is the opportunity cost? Because one of the things from a contracting perspective is that we try and lower the opportunity cost as far as by quickly matching somebody who fits the experience needs and or the domain experience.

Jeff Langston

If you’re in either healthcare or finance or different areas where we have engineers from a wide background that can bring years of industry experience to a different problem domain and can apply that to the problem at hand. Really come in and sort of be technical pinch hitters. Really one of the things that we’ve seen a lot of emphasis on is the growing remote work force. Really the talent pool is that a high number of the engagements that I’ve been through have been in a remote setting and even the talent pool there is that, what’s your [inaudible 00:08:59] and the opportunity to seek out engineers who might not have the physical proximity to where you’re based out of but still be able to source them from.

Really all of the world has been an industry shift over the last several years and the tooling exists now to make that a really highly productive work setting.

Andre

It is quite a recent shift, isn’t it? To actually have the tools to be able to work really efficiently with people anywhere in the world. You just look at all of us guys working on the cloud kind of chatting with each other in real time. Is there something that companies need to kind of come to grips with or take a different approach when they’re hiring people out of the office? I know we spend a lot of time hiring and we’re hiring people in a city where there’s a big demand for developers. How do you shift that kind of mentality into looking kind of for remote workers? What are some steps people can take?

Jeff Langston

Well, that’s a great question and part of it comes down to I think and something that we’ll talk a little bit about is what the culture of the company is like. The process of moving a large like onsite company towards remote can be difficult, but depending on the size of the company, it might be easier as far as allowing users who were onsite to work part-time remotely.

Jeff Langston

Whether that be a couple of days a week or whatever arrangement works best for the company in the setting. But the other thing is [inaudible 00:10:22] some of that is as you’re going through recruiting and bringing on engineers who can be successful and that really comes down to some of the different attributes that they might possess aside from just their technical skill, their ability to communicate and decompose problems and really them being accountable and able to focus and be self-directed and work autonomously.

Those are all characteristics that as you’re building a remote team can help to help them be very effective. Then the other part of it is through some of the tools that you use, other different forms of chat applications, video conferencing and other ways that you incorporate communication in the remote culture can really help build the effectiveness and efficacy of that sector of your company. Whether that be a few individual or some companies that have been highly as full with an entirely remote workforce too. But you really just need to adapt the process to what your needs are.

Andre

That’s a bit of a shift and a lot of change. I mean, we use Slack and we’re sitting next to each other because it’s that helpful. What about you, Dave? When you’re hiring key people on your team, what are some really important things to make sure you hire the right person?

Dave Swersky

Just following onto the comments about remote work, it is really wonderful to have the flexibility to hire remotely for all of reasons. I would say that the ability to work remotely and having experience working remotely is a skill that can be developed. So if you’re looking for remote workers, anyone who has already done it, has been through that process of understanding you need a set space at the home. You can’t do it in the kitchen, especially if you have a family. Being familiar with the tools, Slack and the other conferencing tools.

Dave Swersky

So finding people that already have that experience if you’re going to start hiring remotely is useful. It’s not absolutely necessary, but it’s a kind of a nice to have plus bonus for someone if you’re going to hire remotely having had that experience. As far as from the interviewing selection process, it’s really important to have a process. What that process exactly is, is very dependent on what kind of business you’re running, what you already have. If you’re bootstrapping from scrappy startup all the way to a multi billion dollar company, having a process that is a set process, developing that process and making it your own is really important.

Dave Swersky

The reason is that without that consistency, you don’t know whether or not you’re getting the same experience. You’re providing the same experience to your candidates and you don’t know if you’re assessing them in a consistent way. So you may be passing up on good candidates that deserve a chance. You could be hiring the wrong people and that meeting for cultural fit maybe not as strong technically as they need to be in the particular domain you’re hiring for. So having a consistent process is really key and it’s something you have to do.

Dave Swersky

It’s not something that just sort of evolves. If you leave the interviewing process to your developers when you’re hiring developers, more often than not they’re going to be pretty bad at it because it’s not something that you just know how to do. On average we all go through a set number of interviews in our lifetimes throughout our career that’s not enough practice to be good at the other side of it. So you have to have trained interviewers in your process so that you can provide that experience.

Dave Swersky

What that consistent process provides is a consistent interviewing process that pulls in the right people, excludes ones that are not a good fit. It also provides a positive experience for the candidates. So again, whether you’re a scrappy startup or a multi billion dollar company or somewhere in between, which most companies are, you want people coming into the experience and leaving it feeling like they were treated fairly. Like the interview was relevant to the job posting that they came in for.

Dave Swersky

Many, many cases where, “Hey, I came in here to be an SD2 and they were talking about product management the whole time.” That is really going to turn people off. Back in the day, when I first started in the industry, if you had a bad interview experience, you went home and you grouse to your cat or your significant other your terrible experience. That’s as far as it went. In the days now of Glassdoor and Blind, a bad experience is going to follow your company for a long time and that reputation is really hard to live down if you earn a poor interviewing reputation.

Dave Swersky

So for the same reason you want your customers to have a good experience, you also want your candidates to have a good experience. Beyond that, the way you treat your prospective candidates is going to bleed into the way you treat the people who you hire. That’s goes again to culture. So focusing on that positive experience throughout the interviewing process, you want people to come away feeling like they were treated fairly, even if they’re not a good fit. That’s really what you want to look for is, it’s not a question of bad or good candidates. It’s, is this person a good fit culturally, technically domain experience, the whole thing.

Dave Swersky

Then finally on the nuts and bolts side when it comes to skills based hiring, and this is true for any skill, but especially I think for software development, there’s just simply no replacement in that process for live coding and problem solving. So the live coding part of it in many cases in the Googles and Microsofts and Oracles and Netflixes of the world, the live coding experiences, technology and language agnostic, you can pick the one you’re best at, use that or even like pseudo code.

Dave Swersky

The purpose of the process is to understand the way that candidate solves problems. That’s really in today’s world what you want to focus on. The tools that are used C#, .NET, Java, JavaScript, whatever, I can go on and on are important. But especially at the mid to senior level, it shouldn’t matter whether you’re using a hammer or a saw, you should understand how to build the house. That part of the process is again, it goes back to the consistent process. How they way to consistently across interviewers assess those skills, it’s going to be an art no matter what you do.

Dave Swersky

But if you invest in that process and there are consultancies and all sorts of different ways to develop that process, once you have that process, then the live coding and problem solving gives you on the spot, in the moment experience with that person’s process. It’s still limited depending on your process a couple of hours, three hours, four hours, but it gives you a much better view into their capabilities than some kind of camed system online. It gives them a score or what have you. So it’s really important. Never skimp on that. Build it into the process so that you can see that person’s thought process live there in the interview.

Andre

Very good. We both touched on culture as well. So it’s transitioned into us, the role of culture and I think a lot of these things are finding that right fit but also kind of courting the good prospects you want them to feel comfortable about joining a great company where can do their best work. How does culture play into that, Dave?

Dave Swersky

The culture of a company flows from the leadership. If you’ve worked for a number of companies, then you know that somehow kind of the same way that pets tend to look like their owners, companies look like their CEOs. Especially in private companies or relatively small companies where there’s not much space between the rank and file and the leadership, the culture and the values of the company as a whole flow very directly from the leadership. So we’ve just been talking about how bringing in the right people, good fit is really important.

Dave Swersky

If you don’t have a generative culture to plant them in, then all of that good work that you’re doing to hire the right people is going to be for nothing. Because again, you’re going to bring in people, highly skilled people who are highly paid for what they do. You want to create conditions for success and that starts with leadership. So I would say of all of the things we’re going to talk about here, my feeling is that leadership is the single most important thing because you can’t develop a good hiring process without good leadership.

Dave Swersky

You don’t have an environment for success without good leadership. Leadership is where it all begins. So with poor leadership, if you’re familiar with the Western typology, if you’ve ever followed the dev ops stuff, Gene Kim talks a lot about that. It’s a bureaucratic, pathological and generative cultures. Generative cultures are ones that trust people to do the right thing. You give them a goal and then they go out and go from the goal and you don’t micromanage them. So without those things with the pathological and bureaucratic cultures, it’s kind of like buying this beautiful new plant and planting it in a trash can.

Dave Swersky

It’s not going to thrive. It’s not going to grow. So the leadership leading from the front, not asking anyone to do the things that they wouldn’t do for themselves, and then being clear about what those values are. So culture extends from the values. You can’t just expect that kind of thing to just happen. You can model those good things, but if you’re explicit about these are our values, these are the things we do, these are the things we don’t do. These are the ways we interact with each other at the office. These are the things that are unacceptable. If you’re clear about those things up front, throughout the hiring process and continuing to repeat those things as just part of your interaction with the rest of the company from a leadership perspective, then the culture kind of takes care of itself.

Dave Swersky

Again, in a seller’s market, every one of us works for a company unless we’re an entrepreneur and we’re selling our services to a company. This is a sellers market. If in many cases the best developers and skilled professionals for the most part across the board are not going to tolerate a culture that does not feed them in that way, that empowers them to do good work, and they’ll move on. So that you end up with turnover and again, it goes back to the reputational aspects. Leadership at all levels, not just the center, the corner office, but leadership at all levels is really super important.

Andre

Excellent. What about you Jeff from your perspective coming into different cultures, different companies, how do you see that playing a role?

Jeff Langston

Really I mean, sort of echoing off of some of the points that Dave shared is that, values are very centric to how a culture sort of emanates through the different departments and levels of leadership depending on the size of the company. But one of the things as far as even in trying to match a contract engineer against a fit for that is really if a prospective hire can sort of match up with the ideals that company embodies and holds. What is sort of like the ideal culture in the workplace is somewhere that’s collaborative, that feeds several different surveys.

Jeff Langston

Aside from just outside of compensation, one of the things that a lot of engineers and especially senior engineers can look for is the ability to continue learning through like the opportunity to grow. The other thing that really I have seen play into keeping cultures thriving and growing is that, especially coming down from the leadership, is really a clear understanding of expectations can factor in that. So that different relationships that form within an organization when there’s clear expectations about how to handle different interactions and those things are docked in.

Jeff Langston

That really helps us to build an authentic culture that, even when you’re marketing your organization to attract talent when you have great advertising, for the fact when somebody does come in and takes on a new role with the organization that they can see that all that advertising is reality, can really show in a body. The fact that it is an authentic culture and help and especially as you might be potentially you’re utilizing [inaudible 00:23:07] it’s a bit of a revolving door where you have people coming in and out for different needs. Where you can let that culturally sort of rub off on the contractors as they go somewhere else so that you can sort of spread those good attributes throughout the different places that they might go to as well.

Andre

Excellent. Well let’s keep rolling on. So in terms of onboarding, Jeff, you’ve probably seen a few different ways when you join a new company, a new team. Now what are some things that work for you to make sure you hit the ground running?

Jeff Langston

That is definitely something that in the contract setting that you get to do a lot is that you get to interact with the contract being somewhat shorter duration in comparison to many full-time roles is that, that golden question is sort of how do I start is what are the sources of truth. Really getting into starting with that kickoff meeting with the stakeholder and depending on the circumstance of the business, it might be a rather small company where the stakeholder might be a CTO or a co-founder of a small startup or maybe a bigger organization where you’re maybe augmenting an engineering role within a larger in-house engineering team.

Jeff Langston

Or maybe they’re coming to you with a gray field product to start from the ground up. In some of those instances there might be a very established onboarding role where it’s just a matter of getting your new hire or a new contractor access to the tools that they need, access to the repo and access to the ticketing system, whether that be Jira or something else. Where does the documentation work? But the other thing is they’re really getting a feel for maybe the hierarchy in an organization.

Jeff Langston

Especially within sort of the technical aspects if you’re coming into like an existing code base and sort of more established product or company is really, a sense of ownership for who can you ask questions to about different aspects of the organization or the product too, and really getting a feel and as early as possible a feel for the priorities on what are sort of the expectations that the stakeholder has within the engagement and how can I best serve them and deliver value as quickly as possible, to really hit the ground running and as far as to get up and started with either running the application locally within my own environment to start contributing to features as quickly as possible. All can make a difference to really deliver value early on and throughout the engagement.

Andre

Nice. Even for them to give you some solid ownership of certain areas, it’s very empowering when someone says, “We hired you to do this job, so go for it.” How do we enable you rather than getting in your way? I think that some good stuff. What about you Dave?

Dave Swersky

When it comes to onboarding, getting people started quickly, once you have found the right person, you feel like you’ve got a good fit. It’s important for them to be able to onboard, like you said, independently. You will always have the senior people in the organization that are doing most of the work. A lot of what you want to do is protect those people’s time. They need to be available for people to answer questions and so forth. But they also need to be available to do work. At the tech lead level, you want them splitting their time between supporting other developers and mentoring and that kind of thing, but you don’t want that time to grow beyond a certain point.

Dave Swersky

One of the things that you can do to help with that is documentation. That’s not the only thing that documentation is good for, but it’s one very important thing. Onboarding a new developer, so there should be a process for it and one of those processes should include reading the documentation. The documentation needs to be up to date enough to be relevant. The challenge there is that developers are hired to write code, not to write documentation. So a couple of things you can do about that.

Dave Swersky

You can say, “We expect you to write a certain amount of documentation. We expect you to take the time to do that. We’re going provide the time to do that.” You can set incentives, you can make it part of a review process, which you can say, “Hey, documentation is part of your job.” You can also find people who are especially good at it. There are a few developers out there that actually like writing documentation. I happen to be one, I have a background in enterprise architecture and so documentation, diagrams, blog style posts about the architecture, about decisions that have been made, has been a part of my job for a long time. I actually kind of enjoy it.

Dave Swersky

Play to your strengths, find ways to make documentation part of your process so that most importantly among all of the different reasons, so that you don’t have to keep redrawing the same picture every single time you want to talk about your system. Non trivially, complex systems are difficult to reason about. So you should have a relevant current picture and set of high level documented artifacts that describe it. So that anyone can come in from anywhere and relatively quickly get up to speed and also helps you as you’re discussing evolution of design of those systems.

Dave Swersky

The regular knowledge transfer. So, knowledge transfer, KT is generally thought of as something that happens kind of opportunistically. I’ve got a new person coming in. I’m going to do a KT. I’m transitioning to a new job. I’m going to do a KT so that the work I’m working on right now doesn’t get lost. I tend to think of KT as an ongoing thing. Again, complex systems always have many, many moving parts and there can be people on your team who have been with the company for years who still may not fully understand elements of that system.

Dave Swersky

So having a regular, we have a weekly KT, and what ends up happening is that as people come into the team, the cohort that came in before them, we tend to, in our group, this particular group, we bring in two or three people, maybe sometimes four or five at a time. Then they’re with the team for awhile and then they move on. So there’s a constant rolling membership and I think that’s probably typical in a lot of different teams. If you’re doing that regular KT or it’s, the intent of this hour per week or whatever, is to share knowledge about the system. Every time a new group of people come through, you can say, “Okay, the last people who came through who are now kind of familiar with the system, it’s their job to do the KT.”

Dave Swersky

That constant sharing of information, of fairly fundamental knowledge about the system, whatever systems you’re developing, keeps it fresh for everybody. So the old veterans don’t take that knowledge for granted. They’re constantly having to package it up and deliver it to a group of relatively newer people. As the newer people get more experience on the team, they then turn around and they’re the ones to deliver that information to the next group of people that come in. Again, there’s so much information in a complex system, you can’t cover it in a month worth of Thursdays. It’s just, there’s so much to know. So rather than-

Andre

David, do you record these sessions so they’re on demand in the future? Just out of curiosity,

Dave Swersky

Yes, we do. We take notes and when we can, we record them on video in chat. The notes are usually more useful than the videos, but we definitely do. Because every time we do it, we refresh. The information stays fresh. It’s current as of that week, if we need to do it on a weekly basis because these systems move fast. So treating knowledge transfer as something that is not something you just do when you feel like something’s happening, but you do all the time is really important, especially on a complex system.

Dave Swersky

Then again, nuts and bolts, solid source control practices are absolutely critical. It’s kind of become table stakes in the industry. Everybody knows you got to have source control and you’ve got to use it well. I think at this point, we’re at the point where when we say source control, most people think Git. Knowing how to use Git, knowing how to adapt it to your particular use cases, the applications you’re working on, the source that you’re working on is really super important. Because productivity in a software development environment and especially in a dev ops environment is generally measured by production deployments. Without source control, you don’t have a CICD, without CICD, you have no velocity. So that’s why source control is so key.

Andre

This might get us onto the next slide, actually talking about workflows in particular because now with leading teams to remain competitive, the shipping a lot more often, the having more ownership. So you’ve got to have some pretty strong workflows in place, right?

Dave Swersky

Yes, absolutely. The process it’s kind of the business of IT. The business and the processes and workflows that take a concept all the way at the very beginning of the process, all the way through whatever agile process that you’re probably hopefully already using and moving that work through a constantly moving process that involves whatever your agile process is, managing your backlog. Again, in the same way that Git is kind of the defacto for source control, I think dev ops is becoming the de facto for a framework for those kinds of processes and workflows.

Dave Swersky

So when I say co-location for example, what I mean is, you want ops people and developers to be as close physically to each other as you can get. My team right now of developers sits literally on the same floor and right next to the ops folks, infrastructure people that we work with. So when we need capacity in a cluster, when we need VMs, where we need those things, it’s not get in a car, drive to an office, it’s not get on the phone. It’s not Slack. It’s turnaround to the guy behind you or girl behind you and say, “Hey, I need a VM.” And they’re right there.

Dave Swersky

You’d be shocked at how important development can be because even just the friction of having to pick up a phone or send an email or definitely file a ticket. We use tickets. But again, the people are available for that conversation. Depending on a ticket for a conversation back and forth, that asynchronous communication is what makes something that should be simple take a week or more.

Andre

Jeff might have something to say about that one.

Jeff Langston

In a remote setting that availability, trying to ensure that like your teammates knowing what their skills are, where you might be coming from overlapping time zones and then really just sort of knowing like what clear channels of communications are. I mean, sometimes that falls down to like where we like leveraging tools to help with monitoring and setting up special like [inaudible 00:34:22] and even alerting policies so that when we need to sound the fire alarm for a problem in production that we do so.

Jeff Langston

Then that really sort of goes back to how we get started and really adapting into a team or whether we might be supplying the entire development team is really being able to take either features through the agile process and the planning backlogging. Then executing on those is really having a known workflow for how features get planned and then built. Then how they integrate into the architecture of the software. Then also sort of what mechanisms they go through to ensure that you can reduce the number of possible regressions and really raise the level of quality.

Jeff Langston

That sort of fits into that feature delivery flow. Really that can fit into maybe a highly automated portions of that process through some of the things that Dave mentioned like, through like what you might call [inaudible 00:35:22] where a developer might create a branch off of [inaudible 00:35:27] and then make the changes that would encompass a particular product and then they could submit a poll request and then you can integrate a peer review. In a setting you have a large enough or you have other peers that can review your work and can pull requests in the engineering editing that’s a great.

Jeff Langston Then in some smaller engagements might just be maybe me with a particular client, one of those things that, that can be is frequent demos. So that the client can see iterative feedback and progress with that. That can be the same in larger settings and teams as well as leveraging staging environments and being able to have that. If your organization has dedicated quality assurance individuals or that can fall back to the habits of how that is [inaudible 00:36:13] development or whatever level of automated testing acuity incorporated into the product.

Jeff Langston Really help in sort of enforce and build on that iterative workflow to continually ship features of products and bug fixes. Really that can be an end to end thing and that’s sort of built into one other point that it is that, once you take an idea from conception and build it and ship it, how do you support it?

Andre From your perspective coming into different workflows, I guess scale is one thing if you’re dealing with a team of two versus a team of 50. I get that that would be quite different. But is there an ideal workflow that people are moving towards in terms of best practices or does it really depend on the product and I guess the framework they have chosen in their particular case and making sure everyone’s on board?

Jeff Langston I mean, the particular tech stack that might be in use can play into maybe the workflow of how things are. Also, I have primarily worked on false back web applications or even supporting mobile applications, where depending on the product of an atrium, maybe what the reason the tilling might affect that little bit. Especially with like at web applications and APIs, the shift towards virtualization and containerization and moving infrastructure as code and some of the other topics that Dave has a lot of background within dev ops is that, that’s ultimately where we’ve seen a lot of stuff going in.

Jeff Langston Especially with sort of the advent of, how easy it can be to set up cloud hosting for products and to really put stuff into the public domain, to where you can make your applications available quickly and can scale as much as traffic as you can onboard through that too.

Andre I think what’s interesting from some of the customers we’ve talked to is, we’ve had people like Mark Zuckerberg say, “Build it fast and break stuff,” but you can’t really afford to break it these days. You’ve got to build fast and have reliability, otherwise people will go somewhere else. Kind of it’s that competitive. But we’ve got customers that dare I say, ship once a year, which is just shocking. Then other customers that ship every 11 minutes. I think, you know what’s quite funny, I think the guys that ship least or I might be exaggerating here, and I don’t want to name names.

Andre But I’m the guys that ship least frequently, they see that as I’m de-risking that shipping process. However, if you talk to the folks that are shipping every 11 minutes, they actually feel really confident about shipping regularly because they’ve got such a good process in place. If any errors occur or if they’ve got something that they need to roll back, have developed that muscle where they can identify the issues before the customer does, roll it back seamlessly, fix it, reship. So it’s just kind of it’s pretty interesting.

Dave Swersky The dev ops way of moving comes from the investment that is made in the infrastructure that supports and partly in the investment in infrastructure that’s made. So you’re in a situation where it is faster to fail forward than it is to fail back. But if you read the Phoenix Project Accelerator, all the good literature on dev ops, the velocity and quality can be maintained with the right workflows and the right infrastructure. The idea is that you can have your cake and eat it too. Because one of the reasons why high velocity is so important and is actually better is that if you’re shipping small things fast, then every time if one of those things fails, you know exactly what failed.

Dave Swersky It’s not 50 things that failed. It’s one thing that failed and you can very quickly go in and fix it and fail forward as opposed to if you’re in a world where every time you deploy, you’re deploying a million different things, you might not know what broke. You could have so many dependencies in there. That goes also to some degree to application architecture, microservices, decoupling and decomposing monoliths so that the components of the application are independently deployable.

Dave Swersky Once you have those things in place, then it is better to move fast because no one thing that you’re deploying is going to be so complex that it can’t be fixed quickly. If it breaks, you know that that broke, exactly what broke as opposed to having to go spelunking through logs to figure out. If you’re changing 50 things at one time, it’s really hard to know what broke in when anything breaks. Is it a regression? Is it a regression in something you deployed? Is it a regression in something that was already there?

Dave Swersky When you’re making those many changes, then at one time there’s no way to know. So you can say that de-risking by going slower is actually a fallacy because your customers are not going to want things any less fast. Bundling all that stuff together in one big deploy is way more risky than just a constant stream of very small deploys.

Andre Just quickly, if someone had a kind of antiquated workflow like that, would that put you off the job?

Dave Swersky Yes.

Jeff Langston So getting some insights into what the existing workflow might be can to some degree, I mean legacy systems exist everywhere and that’s one of the things that I frequently [inaudible 00:41:49] jumped into projects and help you to modernize. Really try and reduce friction within the process and help automate certain pieces of the workflow or an early to, but really it depends on the stakeholder are really what that fits into with what the end user is too.

Dave Swersky Real quick, just a couple quick comments on the co-location. The technology is available now to make co-location virtual. As long as it serves, as long as you have that, like what we’re doing now, conferences, video, building a culture around those tools that says that if you want to talk to somebody, get on a video chat. Don’t just make it a phone call. Don’t just use Slack. Use synchronous communication to replace what you lose in a virtual environment, I think can work just fine. But you have to be deliberate about that.

Dave Swersky Oh, on the topic of, will it puts me off a particular job, we’re all going to work on legacy systems. As long as it’s a well defined scope, that’s fine. I think that over the long term if you’re using those older workflows that don’t emphasize more of the dev ops styles workflows, you’ll have trouble hiring because people don’t want to work on those legacy systems. Especially [inaudible 00:43:14] old workflows and your business is going to suffer. For both of those reasons if you’re dealing with those constant legacy issues, then you should probably be thinking about what you can do to modernize.

Andre I think in the last slide we touched on some interesting points around, it’s not just for the developers and having an efficient workflow where they can feel like they’re shipping quickly and they’ve got a lot of security. It’s actually about the demands of the customer. They want to see innovation quickly. If the experience it’s like, it used to be a competitive advantage just to release something that other people don’t have. But now a lot of people have that so that it turns into the experience or the performance being the feature. How do you kind of bring that customer, and first of all, is it important to bring the customer into the center of everything you do? Then how do you actually connect your developers with the impact they have on the customer?

Dave Swersky Yeah, it is absolutely critical. The fundamental rule of business is if we don’t, someone else will. Someone else will deliver faster, better, more. Everyone’s always looking for that competitive advantage and having a modernized workflow with modernized systems is in itself a competitive advantage. A customer focus is really critically important in any business. That’s not just the typical traditional point of sale. I’m selling a widget. Somebody walks away with a widget, they’re happy with the widget. There’s services, there’s internal customers, there’s external customers.

Dave Swersky So everyone in any given organization can have a customer focus even if they’re not in sales basically. So if you’re in development, your customers are the stakeholders that hired you to make a product. In the last slide there was a bullet about product versus project. Basically if you go from a product perspective, if your developer is working on product, products live for years at a time. As a developer if you’re working on a product, the product is the thing that you’re selling to your stakeholders. Your stakeholders are your customers.

Dave Swersky Those stakeholders could be probably in development teams. In many cases, they are internal stakeholders, executives stakeholders, and so forth. Their job is to represent the end customer to those internal teams. So, the headline is customer satisfaction is absolutely important and there’s never a case, I think, you can always use a customer focus and a customer perspective to help people understand why they’re doing the things that they’re doing. So I’m working on this particular feature because it’s related to this epic, which is related to this capability which our customers have asked for explicitly. Or we believe that our customers are going to love.

Dave Swersky So having that customer focus at every level and not letting anyone think, “Well, I’m just a dev. I’m just an ops person. I just start up VMs.” You always need to have that connection to how what you do feeds into the larger vision of the company or even just of your team. That I think it’s motivating. I think it can be inspiring. It makes you feel like you’re coming in not every day just to turn cranks and make widgets but to a larger purpose.

Andre

Well, then that that kind of context allows people to problem solve a little bit and start kind of think, they get more context to think outside the kind of the ones and zeros and come up with some new innovation. What about you Jeff, what works for you? I mean, if sometimes if you work at a company for a long time, you might understand the customer better. But maybe you are also a little bit limited in what you think the customer is and maybe you come in with some fresh perspective. How do you get up to speed quickly when you join a company to kind of-

Jeff Langston

That’s one consistent question that I tend to always ask within sort of the initial onboarding phase or even within the kick start, or like the project kick start meeting is really, who are like the end customers and who are stakeholders. Me being clear about who within the engagement that sort of I’m into and who am I [inaudible 00:47:36] , is that like, who is the business providing value to? Really trying to get and sometimes that answer might come from them in a more established organization.

Jeff Langston

They might actually have even drafted personas where they have usability studies that go into the UX behind the product. Sometimes it might be not quite so organized about where they have like [inaudible 00:48:00] and maybe it’s a product that hasn’t hit the market yet. So it’s just based on research that they’ve done. But within that is often either the founders for smaller startups or getting through the larger organization is that having users and all the different parts of an organization really dog fooding the product and becoming very familiar and even sort of from the support aspect.

Jeff Langston

Several companies, I mean some like Acura and others apply sort of like an all hands approach where you don’t necessarily just have dedicated support individuals, but where you have users from different departments that are contributing and interacting with customers directly. Having a short feedback loop from first off can be, having customer be very, very clear on how they access support and being able to see that sort of maybe even moving into some of the topics on this is that that short feedback loop between code and the customer is that, one of the things that I’ve seen on multiple projects that has made a significant difference is getting tooling in place to monitor application behavior and to also be able to turn around.

Jeff Langston

From being able to identify a defect to pushing your fix for it and getting it out to the customer. So having a short feedback cycle with support to triage issues and turn around fixes for that is extremely useful. There’s a lot of tools that exist today that can help provide tremendous insight into application behavior and transparency for what’s going on within your system. Even a tools that might monitor application behavior from the user so that you can actually see what the user did as they were doing it with like live backend of the session where they ran into a particular error can help cut down on the developers AI.

Jeff Langston

“I can’t reproduce this because it works on my machine,” to, “Here’s literal proof of what exactly was happening in all the different variables that might’ve been within the environment or the different pieces of data that might’ve been involved to because a particular error.” But really, leveraging tooling to provide answers and really cut down on the amount of time that it takes to get to the bottom of issues can really shorten that feedback loop between a code that’s being written and the customer who’s going through workflows and are trying to accomplish their own goals with whatever software they might be using.

Andre

Yeah, I think that’s really interesting. Obviously that’s a place that we play in and is in the toll. One thing we also do is get the developers to handle support. They do a little bit of a roster system and that just kind of puts them in the shoes of the customer and kind of gets them to communicate with them and go back and forth about whatever issue that looking at what their experiences. Then obviously there’s a lot of information out there so you can kind of see what device they’re using, what browser, etc, etc. Is there any metrics that you look at when you’re trying to kind of understand the health of your platform? Is there any key metrics that you’re looking at or even the user experience? [inaudible 00:51:18]

Jeff Langston

Yeah. So within some of the different [inaudible 00:51:22], part of it might be with things like an API, like I trying to keep response times down and to try or that you’re looking at. Part of it is maybe taking a pulse on sort of what the sort of normal is for application behavior and then really tuning your monitoring around trying to find abnormal behavior. Whether that might be caused by security is a major concern on application that are exposed to the internet and with the widespread audit acts and different things to spam networks and really keeping as security in check. To make sure that, you know, people aren’t trying to leverage or looking for vulnerabilities within particular applications and keeping it on that.

Jeff Langston

Then the other thing is, is setting metrics for a user being able to accomplish their goals as far as. That can vary widely based on sort of the genre of application that you might be working on. But really within something like a CRM system or something where making sure that reporting is working and the different key workflows are functioning as appropriate. But really that can go into making sure that you have a good visibility into the entire stack and how things are working for end users.

Andre

That’s interesting cause you can kind of look at how people are adopting different features or whatever that metric is. Also you can look at averages which can be average performance, average load time, which can actually be quite dangerous. Because if your top 5% are having a terrible time, then you really need to dial in to the outsiders as well. What about you Dave? How do you close that loop between feedback and between code and customer? I’m conscious that kind of out of time. We’ve got a couple of minutes, so let’s-

Dave Swersky

So real quick, just well on the points that I’m making here. Unhappy customers tend to be noisy, but not all unhappy customers are noisy. If you’re not hearing from them, that doesn’t necessarily mean they’re happy.

Andre

Which is also like NPS is dangerous, right? Because that’s at the end, isn’t it?

Dave Swersky

Yes. So you want to hear about your customers experience from them directly on a regular basis. That goes to the engagement surveys, feedback. You want to ask them, “Is this working out for you? What can be better?” Don’t assume that if they’re quiet, that doesn’t necessarily mean they’re happy. It might just mean they’re busy. You’ll likely hear from the most unhappy clients, but you may have very many clients who are not thrilled with your services, who are just not taking the time to pick up the phone to tell you.

Dave Swersky

So the middle one there, low error rates do not equal success. So just because you get into hundreds across the board doesn’t necessarily mean that your successful in the aggregate. You can’t have success with high error rates. But again, low error rates is not necessarily the same thing as success. So it’s an element of success. But don’t assume that just because your application is not on fire or even seems to be performing appropriately, that’s an element of success, but it’s not the whole story.

Andre

What are some other key metrics that you might look at beyond just error rates? What are some other key indicators for you if you had to choose a couple of key metrics?

Dave Swersky

Like on the next slide we talk about site reliability engineering, which is really important. I would say that it’s related to dev ops and you could call it pro to dev ops in the sense that it’s been around for a long time. Google has been doing kind of initiated the concept. A lot of site liability engineering is about how to interpret the signals you’re getting from your telemetry and how to reason about what that data means? How to set service level agreements and then service level objectives and service level indicators?

Dave Swersky

So basically back in the day when I started in this industry, you put code out there whether it worked or it didn’t. If it didn’t, you spent a lot of time manually sifting through logs and shaking chicken legs at it. We’re hoping something comes out of the data that you’re getting. Now, log aggregation is basically a solved problem. The amount of data we’re getting from these systems is immense. We’re getting lots of information. Knowing how to interpret that data is really important and that’s one of the things that SRE talks about. It also talks about how to scale.

Dave Swersky

So SRE was born from Google when they realized that in order to scale the way their infrastructure and their support for that infrastructure to what they expected to scale with in terms of just physical hardware, they would need tens of thousands of engineers. That was never going to work. So they had to figure out how do we support, how do we get our ratio of support staff to infrastructure down to like one engineer per hundred thousand VMs or whatever? That’s a lot about understanding how to use the tools you have, how to get the data that you need, identifying what data you need and then how to interpret it.

Dave Swersky

That all kinds of feeds into site reliability engineering. I would say that anybody working on really any scale, that where you expect to expand and grow, you should look into SRE for sure.

Andre

Around that tooling and being scalable actually every company has access to the best tools. I’m going to plug Raygun as one of those tools to give you actionable insights. Because there’s a ton of data and you need to actually turn that into what are the actions I need to take without spending too much time. But the good news is everyone can access the best tools out there because you’re usually charged on number of errors or like it’s very scalable. So if you’re low level, you’re going to be paying a real low level. I think that’s really interesting that everyone can access the same tools as the best companies out there for sure. What about you Jeff, how do you monitor what matters mate?

Jeff Langston

Even the smallest of tools can incorporate tools like that too to where you don’t have to necessarily, like you said, have an enormous budget to have the most useful tools in play. The other is that with that in that large source of it is really being able to have an accurate pulse sort of what expected versus abnormal behavior is into the application. Really being able to key off of, setting alerts to pick up things that are going to make inferences. That also you can identify and fix abnormal behavior within the application before very many users might be impacted.

Jeff Langston

The other thing, some of the things that can help for that is that within a lot of the, or some of the things that you can set up different sort of dashboards so that they can give you high level overview to making sure that the different pieces of your system are behaving over a linear period as expected. The other is to periodically review sort of historical performance because those can be good ways to create tickets for the backlog to go in and tune performance on particular pieces of information. It’s really just sort of utilizing those tools throughout the development life cycle as well as just in supporting production.

Jeff Langston

Using them in staging rooms too so that you can see what impacts different feature changes might make into performance of the product or ability to do that. One other interesting note with that is like leveraging different types of the least patterns like feature flags where you might be able to release a feature with a feature flag segmented to a small portion of users. Then to watch the two linger around the behavior of that particular feature to know when you can roll it out to a wider user base too.

Jeff Langston

Again, this is going to vary based on the sort of the stage of the product that you have and the size of the user base. But just because you have maybe like a small web application idea that might not even have any users yet that you can’t leverage some these different goals to sort of, as you start to pick up traction, you can really find those actual insights and act on them quickly too. So these tools can be useful from the largest enterprise down to the small startups as well.

Andre

I think the way things are going and we’ve actually been been doing an event series around kind of monitoring what matters and asking CTOs and what not. Everyone’s actually trying to tackle this problem. It’s not something that kind of solves at this point. We’re getting a ton of information and everyone’s trying to figure out how to kind of find the best actions and make the best decisions. Because it’s no longer about features. It’s around making it fast, making that experienced top notch, looking at the top and worst percentages in terms of your users.

Andre

I think there’s a lot happening in this space and I think it’s something for people to get their head around and to start to really monitor the full stack, get full visibility to empower the team to start making some really impactful decisions. I think it’s quite an interesting space and something that a lot of companies don’t do very well at all and they really need to kind of need to look a lot more closely at the formats. Well, just on the hour. Next time we’ll definitely make it an an hour, not 45 minutes. Taylor, do you have any questions in the Q&A section?

Taylor

No.

Andre

No questions. So it sounds like everything’s pretty well covered. Well on that note, guys, any kind of closing remarks on a couple of the key things today and we’ll wrap it up?

Jeff Langston

Yeah, just an interesting thoughts and sort of that overarching theme of building a high performance engineering team, that really assess your circumstances and consider different things that we’ve may be shared today. Really sort of adapt them to fit your needs too is that if having somebody from the outside who is a tenure, there’s companies like Gun that can help, that can have people come in and pitch it and help for whatever integration that you need. As well as there’s a wealth of information. We’ve covered a lot of different high level topic [inaudible 01:01:30], but please reach out with any questions [inaudible 01:01:32] afterwards too.

Dave Swersky

On the topic of interviewing and pulling in the right people, I think it’s important to remember that like if you’re on tech Twitter, a few weeks ago there was kind of a thread that went through. A lot of people were reacting to a comment made about the concept of a 10x developer. I think there’s a such a thing as a 10x team and I think any team can be a 10x team in the right environment. I think that you should be skeptical when people talk about 10x rockstar, Ninja developers. Individual developers can be highly competent, but the sort of archetype of the 10x developer who is 10 times smarter, 10 times better, 10 times more powerful than everyone else on the team, tends to a bit of a fallacy.

Dave Swersky

I think that it is better to have competent developers who can work together toward a goal than it is to have one person who is going to work independently without communicating with anyone else, who doesn’t like to talk to people, who hates meetings. Nobody loves meetings, but everybody’s got to be able to participate in the process. We talking about workflows and process. Everyone needs to be able to participate equally. So don’t favor that 10x concept over someone who is going to be highly communicative and work well culturally and just interpersonally on a team. So on balance, take the person who’s going to be a team player as opposed to someone who’s just all the way up to 11 on just the technical hard skills.

Andre

Very interesting. Excellent guys. This has been a really, really good chat and I appreciate both of your perspectives. Thanks Taylor for joining us for this, for collaborating on this webinar and I think these days it is a real collaborative effort among teams and are working quickly and working side by side. So kind of giving them the full picture, having full visibility of the app, giving them all the information and making sure they’re a bit more of a rounded developer who cares about the customer and can work in the teams and kind of problem solves on the fly I think is going to really help with the performance and the velocity of the problems you’re solving, the app or the website, etc.

Andre

So I think it’s a pretty exciting space. With every company having a digital aspect these days, it’s just going get more and more ferocious out there, which is pretty interesting. All right Taylor, thank you very much. I will stop it there. Much appreciated everyone. We’ll see you later and we’ll share this out with everyone after the show.