Product and Engineering Culture: The Most Important Things to Know to Make Eng and Product Effective

David's notes:

Product Management and Engineering are most commonly seen as two separate groups with separate and inward-facing measures of effectiveness. When the effectiveness of these teams are not evaluated together, the two groups are typically driven out of alignment with each other and worse, with the rest of the company. Although not commonly practiced, there are better measures of effectiveness that will not only align Product Management with Engineering and align both groups with the rest of the company, but will also aid in alignment among executives and recruiting.

In this talk, we discuss common evaluation metrics of Product Management and of Engineering, why they fail, and propose a better structure that aligns their goals and unites them, as opposed to setting them at odds with each other.

---

Transcript:

Ron Lichty:

I discovered David Subar as a source of engineering leadership insight when I heard him talk with the UK's CTO Craft Bytes Group. He was talking on Modes of CTO Success and Failure. His talk just really resonated with me more than about any talk I heard in a really long time because: one, he really gets agile where there are a ton of people out there who don't; he gets the importance of context, of the why of motivation for engineers; he called out empathy for users, which is dear to my heart having managed the UX of Macintosh.

He recommended demoing frequently in order to keep the executives out of your team room. He advised the socializing of ideas and the roadmaps to get executives on board. He counseled not doing everything but asking what are the things that only I can do and not doing all the things that anybody else can do. I investigated a little bit and discovered he's on the board of the LA CTO Forum. I discovered his gigs have included Disney, and Lynda.com, and Frog Design, and a ton of startups. I discovered that he was not only excellent as a CTO but a CPO, a Chief Product Officer. Then, I discovered that he had done a podcast with one of our sponsors, the CTO Connection, a talk called Improving Communication Between CEOs and CTOs.

With that, let me turn it over to David. David, it's all yours.

David Subar:

Thanks, Ron. I'm going to have you introduce me everywhere I go. Good evening, everyone. Thanks for spending time with me. I'm talking about product and engineering culture tonight. My argument is that the culture of the product management team and the engineering team are critical for those teams to be effective. We're going to talk about some of the attributes and how to make them effective.

Now, Ron, already sung some of my praises. I won't go too far in-depth, but I'm going to tell you about how I got here and why I got to thinking about Product and Engineering together as a group and about culture.

I started my career doing research and development in AI and machine learning at a military-owned think tank in DC. It sounds pretty cool, but it wasn't. What I found was my job was to- we were advancing science somewhat and we would write a paper and if it was a good paper, we'd present at a conference and maybe 200 people listened to it and then nothing would happen. I found it really frustrating.

I'm congenitally a nerd. I was born a nerd and I was programming computers when I was 12. A lot of you people have the same story that I have. But I wanted to do something. My uncle was a physicist, but I wanted to do something with technology and have an impact on the world. I moved from doing research and development to being a programmer at an AI tools firm, then managing teams and being a director, becoming CTO-- and I was almost where I wanted to get but as Ron mentioned, I became CPO as well because I wanted to be able to shape what was happening, to build something that mattered.

My career went from being a developer and developing software to developing teams that's developing software, kind of getting very meta. I did that. I was CTO and Chief Product Officer with a bunch of different companies. In that, I learned methodologies about what worked and what didn't work about building product management teams and engineering teams.

Seven years ago I started Interna. At Interna, we just help companies with product management and technology teams and make them more effective. As Ron mentioned, companies like the Walt Disney Company, and Lynda.com, and Frog that Ron mentioned, a bunch of startups that you might not have heard of and companies you might have heard of. But the point here is that there are similar things that happen to cultures in product management and engineering at technology companies that are the same no matter the size of the company. There are some things that are different but some things are the same. Tonight, I'm going to talk about some of those things, and hopefully, some things that you can use.

My argument is that there are six attributes you need to make an effective culture for product management and engineering. I'm going to talk about them: alignment, measurement, review of what you're doing, having a great team to do it, empowering the team, and the environment for them to be successful.

The most important thing for alignment is for the CTO and the Chief Product Officer themselves to be aligned. Now, in this conversation, I'm going to talk about whoever runs Product as the CPO, whether that be the Chief Product Officer, or the VP of Product, or the SVP of Product, I'm going to talk about that person as CPO. And same with engineering, that being the Chief Technology Officer. I'm going to talk about this person who runs technology as a CTO, although it might be a VP or an SVP.

I’m arguing there's three states of this alignment between the Chief Product Officer and the Chief Technology Officer. Once again, it's critical: if those two people are not aligned, it's very unlikely their teams are going to be aligned.

The first thing is thinking about optimizing your team. Now, a lot of people come to running their team and think about their primary job as optimizing their team's efficiency. There's an input, there's stuff that we do; and then there's an output. That input might be, "Hey, we have some strategy," and from there we have a roadmap, we write a bunch of user stories; and the output, the user stories and the engineers get that, or, "Hey, we get some user stories, and then we write up a bunch of code, and we put it on servers," that's our input, our process, and our output.

A lot of people think about how to optimize their team's efficiency going through this. "Hey, I'm going to build a bunch of infrastructure so I can release code quickly. I'm going to build an architecture that aligns with our three-year vision of what we're going to build so we can build quickly on top of that, as an example." My argument is if the primary thing that you're looking at is the efficiency of your team, it's the surest way to fail, particularly around culture for product management and engineering.

The reason for this is because when you're thinking inward on your team-- your inputs, your process, and your outputs-- it creates essentially a wall. When something bad happens, and something bad surely will happen, you're not looking at the holistic picture. "Hey, something got shipped late." Well, we in product wrote the user stories, we gave them to engineering. If it's late, it's on them." or, "We did what product management told us to do. User stories weren't really clear. It was hard to know so if we're late, it's on them," or, "The product went out there and the users didn't like. Well, we did what the product managers told us. That's not our fault."

The problem is, when something goes wrong, this sets up a cancer in the organization where product management and engineering start pointing fingers at each other. This creates the opposite of a good culture. This creates an opportunity for people to look where the problem exists somewhere else.

Participant 1:

Dave, you're going to give me nightmares tonight.

David:

It's all right. I'm going to get you out of it. I'm going to get you out of it.

Participant 1:

I'm looking forward to that.

David:

The next reasonable solution is product management and engineering to bond together and say, "Our job is to ship a product." That's much, much better because now-- and by the way, I think this is a next good solution, where they say, "Our job is to ship a product and product management might have an opinion about the way things should be built because they have some insight to it." Engineering might be able to say things like, "Hey, can we build this," or product management, "I understand what you're asking me to do. That's going to take a really long time. I can get 80% of it done in 20% of the time if we just do this instead." Now you have bonded your product management and engineering; and that's better, but it's not right still.

What you've done now is shifted the goalposts between product management and engineering to outside. Oftentimes, you'll hear language like, "We have to talk to business about what they want. We have to go tell someone else," or, "We just do what sales tells us to do." This is something that I see a lot, is people will say, "This whole company is really run by sales, and we just do whatever we're told." This product management and engineering is better, but still not what you want.

The optimal thing to do is align product management and engineering with the customer. This is what Ron was mentioning before that I'm very passionate about, is having empathy for the customer and getting that alignment; and this solves a bunch of problems I'm about to explain, but the first question is, who's the customer? Is it marketing? Is it sales? There's this concept of, "Our customer is someone internally." My argument is those people are not the customer. Your customer is actually the customer of the company. It's only with that alignment that you can create an environment that is successful for culture, that's going to be successful in product management and engineering.

Here's why. You need not only alignment between product management and engineering, you need alignment within the company. What we know about companies is that your job in the company is to create enterprise value. You create enterprise value by having revenue and having profitability. That's all true, but I argue that revenue and profitability are side effects of something more important, which is creating value for your customers. If you do something for your customer that creates value, they will tend to buy from you. It's not the only thing you have to do, but it is a necessary and a critical thing you have to do, to create products that have value for customers. When you do that, you will generate revenue and profitability as a side effect.

Now you're solving not just the customer's problem, you're solving the CEO's problem, and product management and engineering get alignment with the company. Now you're not, "Hey, we respond to what sales says." Now you are solving the problem that the company has by solving the problem with the company's customers. This is one of the critical components of the culture, is understanding who you should be aligned to.

How do you know? How do you know if you're creating value for the customer? I say, look at the activities that you're running. If the majority of the activities are not rooted in direct change for your users, you're not concentrated on creating customer value. Not everything-- by the way, you do need to do things that create your own efficiency. Create CICD systems, for instance; but the majority of the stuff you need to do is about creating customer value. Then you should measure whether your team did or didn't create customer value-- we're going to talk about how to do that-- and the members of the team need to know why it is what they're doing. How is what they're doing creating customer value? Why is it important?

We've talked a little about alignment and we're going to talk about it again, but alignment to customers only has value if you can measure whether you're actually creating by just saying, hey, we're here to create customer value. We do a bunch of stuff. If you don't have a way of measuring it, it's not that interesting.

We'll take a break for a second of my talking. I'm going to ask everyone to simultaneously chat about what metrics do you use in your teams. Actually, what metrics do you report from your teams to the people you report it to, to the CEO?

If everyone can go in the chat just for a moment. Great. Thank you. There's a bunch of them here and we're going to reference a few of them: churn, velocity, bugs fixed, NPS.

Here's my argument. CTOs often communicate things like velocity or cycle time or PR response time. Chief Product Officers often talk about bugs or user stories or roadmaps, and those are interesting metrics, but those are interesting metrics that are internal to the department.

If you want alignment, if you want something that you can measure to know if you have alignment to customers, you need a different metric set. These metrics are good and you should use them. You should use them to manage your department, but you need to think about what it is you're putting on the roadmap and the value that that's going to create. My argument is that everything on the roadmap is there as a bet. You only have so many bets you can do simultaneously.

Let's just say you have 50 developers and you have a six-month roadmap. You essentially have 300 developer months that you can spend. That's essentially 300 small bets you can make. Every bet should have a reason. Every epic you put on the roadmap should have a reason. My argument is that things like talking about velocity, not interesting when you're thinking about alignment. Interesting for your department and for managing the department internally, but if you want to align with customers, you need to have a statement about how you are going to generate value for them.

Here's a structure you can do that with:

We believe by doing this feature for this user, we're going to achieve this outcome for them. We will know it when we see this metric move. And that every feature on the roadmap needs to have a statement like this. It doesn't have to have exactly this format, but you need to have a thesis about why you're making this bet.

How does that affect culture? Why do you care? Every person that works in your department is like a vector. Now, I don't know how many of you guys remember linear algebra. Vectors have two attributes, magnitude and direction. Magnitude is: this person's a really good developer, really good product manager, and can get a lot out. Direction is: which way are they going?

If you have a bunch of people that work for you that are really good but all have been going in opposite directions, the sum of your vectors is going to be zero. You're going to have a lot of effort expended and not a lot of outcome. You're not going to be creating a lot of value. If, on the other hand, people know why they're doing something and what bet they're trying to fulfill, the vectors will tend to align. When they align, the sum of the vectors will be value creation for the customers. Value creation for the customers means they're more likely to buy a product from you, with revenue being a side effect.

If that revenue is done at a unit margin, with unit margin you're going to generate profit; sum of all profit over time creates corporate value. Great, that's a win. Having these bets aligns your vectors, aligns your people to produce value, it produces empathy. We can have a whole different conversation-- glad to have it someday-- about how do you organize your squads if you use the squad model? And how you organize your squads-- which I argue-- for particular customers and not for features, but for products because it creates empathy. But if you do that and people know why they're doing things, then they'll tend to align and will tend to increase value better.

We talked about alignment. We talked about measurement. Great. We know what we're going to do. We're aligning to customers. We know what we're going to provide for them because we've stated it specifically; but that’s not enough to create the culture we want.

Participant 2:

I'll just sort of comment if you don't mind. What I've seen is top management puts out statements of why but because they address so many different corporate departments, they're almost forced to make it such a broad umbrella that this vague statement of how we're going to help the customer is not really boots on the ground for the people in the trenches. You're dependent on the intermediate layers of management to say, "Hey, given this broad statement from up high, for us, it means this, dadada.

I so rarely see this refining of the vague broad statement into, "This is what it means for engineers. This is what it means for manufacturing. This is what it means for service," and so on. I rarely see that. I guess what I'm trying to say is I support your thesis. I just see it so hard to be implemented in a tractable manner.

David:

Let me call it. That's a good question. It's not one of my slides, but let's talk about this because what you're saying is a real thing. If you're a believer in OKRs, for instance, that's a method to do this, but it's also very rarely executed well. It's the same problem. There's some broad statement, but what does it mean? There is a whole process from product management to talk about how are we going to think about creating value? Now the epic statements are part of that but the product manager department's job is to say, "Okay, we're going to place some bets in order to create this value and to draw in information and data from various departments to do it."

Going from that broad statement, the CEO or the board has this general statement, to the implementation to say, "Okay, we've got 10 choices of what to do this quarter. We could only do three. Here are the ones we're going to pick and here's why." That's part of the product manager's job. That is a whole another presentation, which I'm willing to give at some point but it's a product manager job-- There's some nuance there and how that gets done because it's not just drawing ideas in from other departments, but then it's organizing the ideas into ones that will create the most value for relative costs and then socializing that within product management and engineering and among the various departments so you can get alignment.

You can go back to the CEO and say, "We had a choice of bets to make. There were 10 we considered. We chose to do this and here's why--" That's where the epic statements then become helpful again.

Some of this is a science, a lot of this is just interpersonal interaction. It's softer, but it's incumbent and it's the responsibility of product management to be able to do that refinement of, "Here's a broad vision," to, "Here's how we're going to do value creation." If a product management organization can't or won't do that, you have a problem. Now, there are some- I've been in companies where the CEO, in particular, has a thousand good ideas, and it's disruptive to the product management process. I was in one organization where the CEO was brilliant. He was brilliant. He came to me one day, he was like, "Okay, here's four more things we need to do." I said to him, "Keith--" By the way, his name was actually Keith. I'm not changing the name to protect the innocent. He was not at all innocent at all. I said, "Keith, those four things are four of your 16 brilliant ideas this week. They were really, really smart." They were. They were. Keith was brilliant.

It helped me think about, among the things that we can do, how do these rank? I'm describing this conversation as if it happened in the two-minute session, It was longer than that, it was more nuanced than that. But it's product management's job to do that refinement that you're mentioning- plus it's part art and it's part science. That’s not meant to be the topic for this presentation, but it’s an interesting aside. Let's go to the point that you have, the epics on the roadmap. It's not sufficient.

Now we're aligned in product management and engineering to the customer. We know what we're going to do. Now people understand how to retro sprints and some people are quite good at it. The end of sprints, we go and we look and we say, "Hey, we thought we're going to get this much philosophy, we’ve taken care of this many points. We got more done than that. Hooray.

What did we learn? How are we going to do it again? We got less done. Oh, that's sad. What did we learn? What are we going to change in the future?" People know how to do that, the Kanban, there's some more ways to look at it with things like that. Those are good. Those are important. Those are necessary, but it’s just as necessary to retro your epics after they're done.

You don't retro your epics the day you do the release, you need some time for the users to use the product, but you made a bet. You need to know whether your bet paid off or not. You need to take the epics and the epic statements. You should take the epic statements and say, "How did we do?" Some of them you're going to get right and some of them you're going to get wrong. That's okay.

As a matter of fact, if you get none of them wrong, you've probably been too conservative. You should hope to get some of them wrong, but then you should ask how you do it, and you should share the conversation with the engineers because now we're talking about engineering and product culture. If you're telling the engineers and your product that this stuff's important, then they should be part of the retro process. They should hear how you did because-- By the way, you want them to be empathetic to the customers. It's hard to be empathetic if you didn't know if you helped or not. Retro’ing the epics is as important as retro’ing the sprints.

Participant 3:

David, what happens if some of those engineers, the epics they worked on were the ones that didn't benefit, where the value didn't play out?

David:

That's going to happen. That is certainly going to happen. Some of them, the things that you worked on, were not successful. That should be an expectation. The thing that they are bonding to isn't the epic they worked on-- or you hope it's not, it might be-- it's the value they're creating for the customer. There's a bunch of things in the world that needs to be refactored: code, features, or design. You should expect that some of these things aren't going to work but their experiments are going to put it out there. Once again, another side, I'm a big believer in doing MVP. All product managers are bad at their job, every one of them. The reason they're bad at their job is because you can't be good at the description of the job that product managers are given.

Tell me exactly what the users want. Our expectation should be that some of these are going to be wrong and we're going to learn from it. Our expectation, therefore, is because we know some of them are going to be wrong, we should invest as little as we can into good software. Oh no, sorry, I said it wrong. Let me say that differently. All the software we built should be good software, but we should build this smallest increment of it so we can learn, so we don't have to overinvest in something that might be wrong.

Engineer, think about what you're doing that way. Don't fall in love with your code. Expect to refactor it because the environment that you're in, the bets that we're making might be wrong. Our goal here is to create more value for the customer. Even if this thing that we just released wasn't right, the sum of them are and you're part of that journey.

Okay. We talk about alignment, aligning the teams to customers, measurement, review. That's awesome but none of this works unless you have great people. How does this stuff help me get a great team? As a manager of a team, or a director, or CTO, or Chief Product Officer, if you do three things well, 90% of your problems go away. If you recruit well, retain well, and fire well. I'm not going to talk about fire well right now. I'll talk about recruit well and retain well. Product managers and engineers have choices of jobs. They can leave your company today and get a job in three days.

By the way, if they can't do that, you probably don't want them to work for you either, but you have to assume that they can get another job in three days, and so retention is really hard. Recruiting is also really hard for exactly the same reason. You have to have a differentiator to keep great people on your staff. The typical ones that people think about are money or stock options or interesting projects. I argue that none of these are sustainable. Facebook can pay more money than any of us, and any of our companies, unless you have to work for Facebook or Google or Amazon. Money's not a differentiator. Options are hard because if you have a downpour, it's hard. There's a lot of interesting projects.

The key differentiator is creating value for customers and being able to say, "Hey, mom, look what we did. Here's how we changed the world." That is non-fungible. People talk about non-fungible tokens. Money is fungible. Your dollar bill and my dollar bill, exactly the same. The differentiator for your company, for your product, the thing that you can do to be more attractive, to retain, to generate, to attract great engineers and great product managers, to retain them, is have them working on something that's of importance and have them tied to it. That's where the empathy becomes not just important to creating a great product, but to having a great team.

Okay. By the way, I've already mentioned two or three other presentations. There's a whole other one on how to recruit and retain people, being good in recruiting, how to think about how to align them with what attributes you're going to look for, but not for today.

Now we have our teams that are aligned with customers, we have things we're going to do to measure to do them. We're going to review them. We've got a way to attract people and retain them. It’s not enough, because if you micromanage them, they will tend to leave.

By the way, there's a two-by-two matrix that looks like, you have two choices of the kind of people you could hire, smart ones or stupid ones, and then you could empower them or micromanage them. That's four quadrants. Hiring stupid people and micro-managing them takes too much time, too costly. That's a fail. Hiring stupid people and not empowering them, that outcome fail. Hiring smart people, micromanaging them, they'll tend to leave, fail. There's only one quadrant that works, hiring smart people and empowering them.

I'm going to talk about this briefly. I think we're on- you saw my Commander's Intent presentation. Did you? Yes. I'm going to talk about it briefly here. It's a longer- it's 45 minutes, which I'll reference here. Commander's intent is basically this. It talks about-- It was invented by the Prussians and then Napoleon did it. It started with the military. The problem they needed to solve was: I have a large number of people in an army, communication is unclear and maybe slow, and I want them to go and achieve some purpose. It turns out that those attributes are very much the same in technology development and where there are large, complicated things to do; it's not clear how to do them and the communication is hard.

The answer to that is-- and I'm doing this in brief-- the manager says, "Here's what our goal set is. We're trying to get this done for this customer--” very much using the epic, so drawing down to here's who we serve and here's how-- “Ask me a bunch of questions about how to do it.” You can ask questions, you finalize the goal, you finalize the epic statement, and then the manager says to the team, "I'm not going to tell you how to do this. I'm not going to tell you how to do your job. I want you to go away and figure out how you are going to solve this problem and bring me back the solution." Then the manager goes, asks a bunch of hard questions to the people that are going to execute it.

It's important here that the manager asks and not tells. If you tell someone who works for you how to do something, they will tend to do it even if it's wrong. If you ask them, you would learn. Maybe they know something you didn't know, they're closer to the problem. Maybe you didn't communicate it well. By learning, you will be able to harden against the problem. Then, just as you've decided the goal, now you jointly decided on the implementation, let them do it. That's not all you need to do in commander's intent. Now they know your intent, they know the implication, then your job as a manager is to get impediments out of their way so they can execute. Then just check in with them. That's commander's intent standing on one foot.

Participant 1:

I love the title of this empowerment, which is really saying, admitting that-- Subtly, they're saying, "I don't know everything." Well, what do you know? Encouraging them to grow. It's actually just- empowerment is great. Tell me something. You don't have to be nakedly saying, "I don't know this." I actually in the past have told people, "You're closer to the solution than I am. Tell me something." That's the way of empowerment, saying you're closer than I am and admitting that I'm not the person doing the day-to-day work. I think it's very good.

David:

Yes. There's this story about Abbottabad and Bin Laden. They were going after Bin Laden. I think it was General McRaven-- I might have picked the wrong general here-- he went to the seal team and said, "We think we've found bin Laden and here's a model of where we think he lives." There's an article in the Atlantic about this. The Seals all walked around, looked at the model, and then, McRaven said, "Okay, I'm leaving the room now. I'm going to come back in an hour or so. Tell me what you need to successfully capture or kill Bin Laden." Then he walked out the room. That is classic commander's intent. Now think of what you want about that mission, about Bin Laden, however you feel about it, but that was classic good management because the people whose job was to do it, who knew the job better, they made the decisions. They not only bought into the solution because they crafted it, they crafted a better solution than McRaven could have crafted himself.

Okay. Now we've got product management team aligned to the customer. They know what they're going to do. They have some way of measuring it. They're going to review the measure so they can tune the product management process. They've got a great team of people to do it. They're empowered to do it so they can execute. They're aligned because they know how to do it. All that's great. You need one more thing: you need an environment where they can be successful.

Environment's your responsibility. To create a great culture, the environment is your responsibility. You need to be in a space where the team can get what they need. I talk about it here, about the company growing or can grow. You need the tools, the money to give the team what they need. You have to be able to lead the team, right? You have to believe in what the mission is and who you serve. If you don't believe who you're serving, your team won't believe it either.

You yourself have to be empowered. It might mean that you have to fight for empowerment to lead the team but you yourself have to empower to create this environment for them. You have to be able to provide the tools and see the table-- now, when I talk about providing your team with the tools, I mean it sometimes in a complicated fashion, like, "Hey, I'm going to get you all the credits you need at AWS. You want to use Lambda, that's awesome," or, "Pick Azure if you want, whatever." The tools also mean, back when people worked in offices, "Hey, everyone here is working late, who's hungry? I’ll order the pizza. All the garbage cans are full because the pizza boxes filled them up? I'll empty the garbage cans.”

You need to do those kinds of things. Your job is to create an environment where your team can be successful. It's not enough for them to be empowered and aligned. They have to have the tools and you've got to provide that environment.

My suggestion is try to get as many of these right as you can. The more you get right, the better your culture's going to become. The thesis is if you create a great culture for product management and engineering, you're going to get a more effective product management and engineering team. Get as close to perfection as you can get. You might be-- by the way, there may be some environments that you're in, where you can't be effective, where you don't have a seat at the table, and maybe you should decide to quit those companies.

I'll leave you with a quote. Now you'll see that my French is awful, well, actually non-existent, but Antoine de Saint-Exupéry. He also wrote The Little Prince.

He had this quote about "Love does not consist of [two people] gazing at each other, but in looking outward together in the same direction." I think that's true, but it's not only true for people in love with each other. It's true for teams that want to function well together. This whole presentation I gave could have been summed up by this one slide. Go find the customer you serve, figure out what makes that customer happy, have everyone aligned to it, and the culture will tend to end well.

That's what I have for you. If anyone has questions for me, wants to talk offline, feel free to reach out to me by email. You can hit me up on LinkedIn. You can hit the Interna website. Glad to chat about some of the things that we think about when we coach CTOs and Chief Product Officers-- I missed this in the beginning. There's four things we do. Sorry, I'll just say as quickly in case-- One is we evaluate product management and engineering teams and say things like, "Here are some stuff you could do better. Here are some stuff you should keep doing." We coach and mentor CTOs and Chief Product Officers, and we function as interim CTOs and Chief Product Officers when a company doesn't have someone and help fix the teams ourselves.

If you want to ever have a conversation about, "Hey, I'm seeing some stuff like this. What have you guys seen?" and just spend an hour on the phone with me or with one of the folks, just call us. Glad to be helpful.

[00:51:32] [END OF AUDIO]