CTO Connection Chicago: How to Make Product and Engineering Work Together

David's Notes:

Product Management and Engineering are most commonly seen as two separate groups with separate and inward-facing measures of effectiveness. When the effectiveness of these teams are not evaluated together, the two groups are typically driven out of alignment with each other and worse, with the rest of the company. Although not commonly practiced, there are better measures of effectiveness that will not only align Product Management with Engineering and align both groups with the rest of the company but will also aid in alignment among executives and recruiting. In this talk, we discuss common evaluation metrics of Product Management and of Engineering, why they fail, and propose a better structure that aligns their goals and unites them, as opposed to setting them at odds with each other.

Here are the key moments from today's topic on aligning goals and actions:

0:00 - Introductions

00:46 - Topic introduction

01:11 - Introduction to Interna

2:45 - The three states of alignment

8:32 - How do you know if you are producing value

9:45 - Which metrics do you report?

11:05 - Thesis of Delivering Value

11:56 - Secondary benefit to metrics (Alignment)

14:16 - Retro your epics' value creation the way you retro your sprints

16:05 - Let's talk about transparency

18:05 - Iterations

19:38 - Final words

Transcript:

David Subar: Thank you. This is my button to go ahead.

Speaker 1: Yes.

David: Awesome. This is an audience participation presentation, so repeat after me. This is an audience participation presentation.

Audience: This is an audience participation presentation.

David: Thank you for participating [laughs]. Today I'm going to be talking about culture clash, product management and engineering. These are two groups that fundamentally get unaligned and I'm going to talk to you about how to get them aligned and why it's important. Let me tell you how I came to this and how I came about-- I started my career doing research and development on AI and machine learning in the military-owned think tank. Sounds exciting? Terribly boring, because what you do when you're doing research is you're writing a paper and perhaps if you're Newton or Einstein, you've changed the world.

Turns out I'm not. I went from there to being a developer in an AI tools firm where I could actually really build software that affected the world. Became a director, eventually became a CTO, Chief Product Officer. The thing that mattered to me was, what does this mean? What are we doing? Why are we here? Why is my work important? I did that, about eight years ago, I started in Interna where we consult with product engineering groups on the stuff that I learned. There's a bunch of us now that do it, but how to make product managing engineer more effective.

We've done it with a whole bunch of companies. I'm only telling you this because you have the context of what I'm about to tell you in a minute. You'll learn some patterns with small companies, startups and large companies like the Walt Disney Company become very similar about how to make some of these things more effective. I'm going to talk to you today about two culture clashes and five things you can do to overcome them. Now audience participation, how many culture clashes?

Audience: Two.

David: Awesome. How many tools?

Audience: Five.

David: Okay. You're going to yell at me if I don't give you all five, right? Awesome. Great. First I'm going to talk about alignment. There are three states of alignment, aligning within your team, bonding within the team, and a greater kind of alignment we're going to talk about.

As an engineering leader, you might think that your job is to make your team more efficient. How do we run faster? How do we get more out? This creates a fundamental problem because if you're an engineering leader and you're thinking about how do I get my team faster? If I'm a product leader, I think about how to get my team faster. I'm looking at my boundaries, what my inputs are, what my team's inputs are, what our processes are, and then what our outputs are.

Product, same thing. What are our inputs? We have some kind of roadmap, we write some user stories, we put those out to engineering, they're just going to execute them. I've created two teams, two groups of people who can easily say, "I did the user stories that they gave us, my team did the user stories they gave us. If it's not right, it's on product," or I gave engineering the things they were supposed to do. If they didn't get it up fast enough, it's on engineering. What you've now created is the first culture clash.

Remember I told you [unintelligible 00:04:24] give you two. The first culture clash, attention between product management and engineering, where there's ability to point fingers and get those two teams out of align. If that's not going to work, what do you do? Next, the result thing is to focus on the customer. We're here to deliver value for someone. This was my transition when I went from R&D into building product.

We're here to create value for someone. Who is it we create value for? Who's your customer? A little more audience participation. Everyone who's in engineering, stand up for a minute. Great. That's almost the whole room. How many of you guys have this conversation? There's business and there's technology. If you don't have that conversation in your company, sit down. Okay, for the rest of you, stop it. Stop it.

This is the next culture clash. You guys can sit down now. This is the next culture clash because now you are taking orders from the business side. This is not your customer. Business side is not your customer. Sales are not your customer. Marketing is not your customer. Those are your colleagues. You are here to serve the same folks. Your customer are the people that use your product, your service, your customer are the people that actually pay you money.

Don't mistake sales for being your customer. I'm here to create customer value. We've identified that sales is not our customer, that our customer are the folks that actually pay us money. How does that get aligned then? How do we think about the alignment in getting it? It turns out that creating customer value, the focus that you should have in product management engineering, we're building a product for somebody. We'll get your alignment with your CEO.

Now, Brian just talked about some metrics about CEO alignment. I'm going to talk about a slightly different set and why they're different, why they're important. Turns out, markets only buy products from companies that whose products produce value for them. Your CEO's job is to create a company of value. They create a company of value by being able to sell product. People want to buy product that creates value for them.

You should think about revenue and profitability as a side effect of creating a product of value. Drucker talked about company's job is not to create revenue, is to create customers. You create customers by having something people want to buy. By focusing on the customer, product management engineering are going to get aligned. [unintelligible 00:08:12] by that and the CEO is going to be able to have more revenue and profitability as a side effect of that.

How do you know whether or not you're producing value? Look at the stuff on the roadmap. If the majority of that stuff is rooted on change for the customer, that's a good sign. You should be able to measure the value your team has created for the customer. You should have a specific metric for that. Everybody on your team should know why they're doing it. What value are we creating? Why are we here? What value do we bring to the world? By the way, I forgot to mention something in the beginning.

I have a 20-minute present-- I have 20 minutes to give a presentation, I have about an hour and 40 minutes of presentation. We're not going to get this all done. At the end, I'm going to put a QR code up. If you guys want to have questions about this, if you want to guys want to see the rest of this presentation, hit me the QR codes. You'll get that. You can send the email to me. I'll be around here. Glad to answer your questions. I'll give the rest of the presentation to you guys. Also, do audience participation. We can exercise if you all want. It's all good.

Okay, so I talked about the first thing, alignment. That's great, but what does it mean to be aligned if you can't measure it? The second question is, what are you measuring? What metrics do you report? Okay, let's stand up again. Everyone who reports metrics about velocity to their CEO, Cycle time, PR response, on any of these, you all stand up. Okay, rest of you guys stop this, don't do this. These are interesting metrics internally but they're poison when talking with your CEO, they're poison when talking with your colleagues in sales and marketing. These are important metrics internally to your department to know how efficient you're being but they address activity, not outcome. How fast you're running but not what you're getting to where you want to go, so what do you do?

Now, remember, we're talking about product management engineering as a combined group, combined delivering value. Everything on the roadmap that is meant to deliver value should have a thesis and the thesis looks like this. We believe by doing a feature for these kinds of users, we will deliver this kind of value for them and we will know it when we see this metric move.

Everything on your roadmap that's meant to deliver value, I'm parsing my words carefully because there should be some things on roadmap, they're going to be refactoring or rearchitecting things like that. Everything on your roadmap, that's meant to deliver value, you should know how you're going to deliver value. If you don't know don't start.

There's a secondary benefit to metrics and that's alignment. I talked about alignment before, talking about it again. You can think about what you're asking your engineering teams to do, what you're asking your product teams to do with three things. What you're asking them to do, it's always very clear. How they're going to do it is often clear. Why they're going to do it turns out to be the most important thing. If all of your engineers, if all of your park managers, you think about these vectors.

Now, I don't know how many of you took linear algebra in high school or college but vectors have two attributes. They have a heading, a direction and they have a magnitude. We all hire just perfect engineers, so their magnitude is high, they're really strong, they can pull really well but if they're not aligned, if they're heading in the same thing, it's like a tug of war. They're pulling in opposite directions. By knowing the why your engineers everything they do, there's three or four ways of doing it. The odds of them consistently picking the right way, if they don't know why they're doing something, is really small.

By them knowing the why, by having these epic statements, they're more likely to be able to make the right choices. Additionally, they're more likely to be able to contribute to the product management process to say, "I understand what you're asking me to do, I can get that done, that's going to take us three months, that seems like that sucks. We could do this other thing, we'll get 80% of it done, it'll take about two weeks and we can get these other benefits."

By having your engineers know why, they're better connected to product, they're better connected to delivering value. By the way, there's another side effect, we talk about people resigning, the great resignation. It's a subjective thing that is unique about your company if your engineers are invested in who you serve, and why you serve them, they will tend to stay around more.

Okay, so promised you five, we've done alignment measurement. Let's talk about review. Here's the big mistake people make when people have things on the product roadmap. It's lovely to have things in a product roadmap, it's lovely to have a thesis about why it's on the product roadmap. It's useless unless you measure how good your bets on the product roadmap were. Just like you retro a sprint and you learn at the end of the sprint, here's stuff that went well. Here's stuff that didn't go too well, we [unintelligible 00:14:45] want to do it differently. You should read in the same way, you should retro the things on your product roadmap. They're all bets, they're all bets based on the epic statement or some other format.

Here's what we're doing. Here's why we're doing it. Here's who we serve. Here's how we know if we're actually serving them. We should go back and look at the things that we put on the roadmap, the things we actually released, and say, "Did we or did we not, actually, produce value for our customers?" If we didn't, that said, what did we learn, what do we do differently next time?

It's also interesting to ask, "Oh, we overperformed. That's interesting, that wasn't what our bet said, what do we learn?" Retroing for releases, not the day you release but two or three weeks later, maybe four weeks, it depends on your company, is just as important as retroing at the end of a sprint. If retroing at the end of the sprint talks about efficiency, how fast from running, retroing at the end of a release says, "Did we or did we not run to the right place?"

We did alignment measurement, and review. Let's talk about transparency. It's lovely, remember before I said, that sales are your colleagues, the CEO is your colleague, you're here to serve a customer together. If we've talked about efficiency, just with this, the general agile process effectiveness, with how are we putting things in the roadmap. How are we talking about them, how we're measuring them? It's incumbent on us to be transparent to our colleagues about how we actually did.

If we screwed up we should damn well say it, if we overperformed we should say why. Turns out they know some stuff we don't know too, your sales folks may be closer to the customer. They may understand things about the customer, your market people might know stuff about the market. You want to have them understand what's going on with your engineering, with your product teams, so they can give you feedback.

Brian was talking earlier about metrics that are reported around the executive team, didn't say quite like that but that was a similar kind of thing he said. The salespeople do metrics. When sales are down, it's kind of the conversation you want to have. When marketing isn't generating enough leads, it's kind of conversation you want to have or if they're overperforming, the same way. The same with the things in your product roadmap, it requires vulnerability, it requires trust, it requires collegiality. By the way, I would argue if you don't have those in your company, kind of problem, might want to think about fixing them or find a place that you do have those.

Last thing is iteration. Sorry, I have a slight problem. After you do the stuff on the roadmap, after report, there's nothing about roadmaps, they are a religious artifacts. Just because it's on the roadmap today, we've released something, we've learned something, we've shared it with our colleagues, we should revisit the roadmap based on what we learned. No roadmap should live for a year. No battle plan survives first contact with the enemy, that's famously said. Same is true with roadmaps, after you release something and you decide whether or whether you've not been effective, you should [unintelligible 00:18:52] that, not just for your process but look for your roadmap.

Audience: Antoine de Saint-Exupéry.

David: Yes, right. Great, good job. [laughter] He wrote The Little Prince, he said this. "Love does not consist of gazing into each other's eyes but looking outwardly in the same direction." This is the same as true with product management engineering. It's not sitting in the room together and staring at each other. It's all about who we serve. Last audience participation. If you want to talk more, hit me up, I'll be around here, feel free to email. I spend time thinking about this stuff, sorry, I'm a nerd.

[applause]