The Thought Leaders Podcast With Thierry de Pauw of ThinkingLabs

In the latest episode of "The Thought Leaders Podcast", host David Subar welcomes software development strategist Thierry de Pauw to delve into one of the most misunderstood topics in software development today, feature branching. For many, feature branching versus trunk/mainline development seems like a solved problem, but as with many things that are thought solved, the solutions are poorly implemented. Thierry takes us through the difference between feature branching and mainline development and tells us how feature branching is not a panacea but it is an impediment.

De Pauw brings a rich background in electromechanical engineering and continuous development consulting, providing a fresh perspective on traditional software methodologies. The conversation quickly dives into the complexities of branch-based development versus trunk-based development, igniting a detailed examination of the impact these practices have on lead times, team dynamics, and overall product quality.

Moreover, Thierry touches on an often-overlooked aspect of software development: the human element. He discusses the importance of senior developers mentoring newcomers, the transformative potential of practices like pairing and mob programming, and how these can be integrated into different organizational cultures. Particularly intriguing is his take on the evolving role of software engineers in the age of AI and code generation tools. How does one balance the benefits of automation with the invaluable human touch in coding?

This episode is for anyone involved in software development, from CTOs grappling with legacy systems to developers keen on refining their craft. Tune into "Interna Thought Leaders" to explore these topics and more, and discover whether your current practices are facilitating or hindering your team's success.

For more about us: https://www.interna.com and find more posts like this at https://www.interna.com/blog

Don't miss a single episode of "The Thought Leaders Podcast"! Subscribe to our YouTube channel to be notified as soon as new episodes are released.

Timestamps

00:00 Thierry de Pauw on continuous development.

06:12 Product managers struggle to create value effectively.

07:12 Reducing lead time to measure user satisfaction.

12:09 How improving communication and visibility can avoid rework.

16:31 Why automated and exploratory testing are both necessary.

23:10 Returning to mainline development and deployment challenges.

26:12 How to lose $460 million with bad toggle implementation.

35:07 GitHub Flow: long main, feature branches, deployment.

49:39 On-the-job mentoring for effective junior training.

Transcript

David Subar [00:00:01]:

Today I interviewed Thierry de Pauw. He's a consultant on continuous development and cares very much like I do, like we do at Interna, about how do you make product managing engineering teams effective. We talk about mainline development and what that means, why that might make, why that should make developers able to release code faster, get it out to users faster, and therefore find out whether the bets on the roadmap are right or wrong. We talk about a lot of other philosophical issues, but in heart, we're talking about making product development more effective. I hope you enjoy. Hello, everybody, and welcome to the podcast. Today I have the pleasure of talking to Thierry de Pauw, who I got to know through the Agile alliance and presentation he did. Thierry, is a thought leader in multiple things about continuous development, proper software development, process and technique. But before we get started, I'm going to ask theory to say his name properly, because I don't have those skills.

Thierry de Pauw [00:01:28]:

Yes. Hello, David. Thank you. So, yeah, my name is Thierry de Pauw. So it's a combination of French and Dutch.

David Subar [00:01:39]:

So thank you. Well, before we jump into it, let's talk about your background. If you're like me, obviously process matters, but I didn't get there. Graduating from college and my first job in software development, I didn't know anything about the concept of process. I just knew how to do some coding. So tell us a little about your background, how you got here, how this became something that you were passionate about and started pushing.

Thierry de Pauw [00:02:13]:

All right, so my background is actually not computer science. So I have a degree as a Master of Science in Electromechanical Engineering. Well, I was actually destined to study computer science until someone at the university said, yeah, it's a lot of statistics. And I hated statistics. So I went for my second choice, which was electromechanical engineering, which is, yeah, big machines. And then I went for production planning for fast race. This was my thing, and I actually started working in this industry. Like, my first job was actually building a factory for aluminum.

Thierry de Pauw [00:03:06]:

Aluminum foundry was quite a thing when you leave college or when you leave university. And after that, I went working for a small medium enterprise, like 50 people. We were two engineers, and we did everything. We did R&D, the IT, the software, we built the machines, we did all the software for the human-machine interfaces. And that's how I got into IT finally. So two years later, I transitioned to IT, and that's.

David Subar [00:03:50]:

Did you work in factories? In the process there, yeah, I worked in.

Thierry de Pauw [00:03:55]:

Factories also as a student. I worked. So my student job was working in factories.

David Subar [00:04:02]:

The process you learned from working in factories and inferring there was maybe lean manufacturing, things like that affect or help you think about process and software development or were they completely unrelated?

Thierry de Pauw [00:04:18]:

No, well actually I learned all these things and it's quite embarrassing. So I learned all these things in university. So we had a theory of constraints at university and my father was also working in a factory and so he was like this whole just in time thing doesn't work. When I look back I'm like, yeah, yes, it did work, but you were just not ready for that. But in fact, I forgot everything about that. And it's only like ten years later when I was cleaning the attic, I fall back on my courses from university and I was like, here it is, all the things I'm now hearing about. I actually learned these things and then I started reading my courses, I was going to be absorbed by it.

David Subar [00:05:10]:

Okay, so let's put a pin in that for a minute. I want to jump to another subject and we're likely to circle back to this. Let's talk about the purpose of software development.

Thierry de Pauw [00:05:25]:

Yeah, so the purpose of software development, well, I have a take on this. It's like, yeah, well, the purpose of software development is actually to solve problems, problems based on success. And it might be that we can solve the problem without writing any software, which is the best thing. But if we do have to write software well, we want to do this well in the fastest way, getting the fastest feedback. So we want to sustainably reduce lead time to create a positive business impact. Creating positive business impact is when software helps the business and the business can generate money. That's my take on it. So in the end, what we want is generating money.

David Subar [00:06:12]:

So let me, let me pause you there because I think you and I agree on this. I think it's not surprising that we agree on, this is why we're having this conversation. I have this thesis that all product managers are bad at their jobs. All product managers are bad at their job because it's not a job you can be good at, where if good is the measurement of the user specifically wants this and we specifically need to build that, it's an impossible job. And so if you believe that theory and you believe the purpose of software development is creating value, which is, I'm rephrasing what you say, creating value for a user. So I believe too, then the thing that you're saying, which is how do you reduce the lead time to get the knowledge of where the product manager made a good decision, where they made a bad decision to iterate, right? So I think we're, I, well, I think we're in agreement about that. Tell me if I'm wrong.

Thierry de Pauw [00:07:12]:

Well, well, we want to reduce the lead time because we want to know as fast as possible that the thing we've just implemented actually satisfied the user or not, or if the user is actually using what we have implemented, which is not always a given. But I've seen it many times happen that, yeah, teams spent six months to implement something and then realize that, well, nobody was actually waiting for this, which is painful. So that's why we want to reach your sleep time. We want to know this as fast as possible, and we want to be able to course correct. Yes. In that case, our product managers. Well, it's always like making a hypothesis. I think the user might need this, but we are never really sure to know this, or we need to have something in production, then we need to measure.

David Subar [00:08:08]:

Okay, great. So we have hypothesis. Product managers are building hypotheses. I agree with that. I would say product managers are putting bets on a roadmap. And until the rubber hits the road, until the user has a product you don't know, which then goes to, you're talking about reducing the lead time. So, so far we're in complete agreement. Um, we'll see if, we'll see if that continues.

David Subar [00:08:33]:

Well, we'll see. Um, so, trunk-based development, that's what we're really here to talk about. Uh, trunk-based development. The process. Actually, I was going to find it. I'll let you define it for a moment, but then I'm going to, I'm going to be the contrarian for, I'm going to be contrarian right after that. But define, define for us, if you could. Trunk-based development.

Thierry de Pauw [00:08:56]:

Yeah, so, well, actually not everyone is happy with this term trunk-based development because what we actually mean is continuous integration, but on the right, that's the idea behind it. But that happened so, well, the term continuous integration started to be badly interpreted. Many people say like, we do continuous integration, but in fact they don't. Then somewhere around 2006, people started, especially the people from thoughtworks started to say, yeah, but no, what we really mean is that we commit straight to mainline, which is trunk. When you work on two versions, and we continuously commit to trick, this is really what we mean. So it's not having a build system that is running against branches. That's the idea. And so that's why they wanted to make this distinction.

Thierry de Pauw [00:10:00]:

Like, yeah, this is string-based development as opposed to continuous integration as the whole industry was speaking. About continuous integration, meaning we have a built of running against branches.

David Subar [00:10:14]:

So the, so building against trunk. Right. So I'm gonna, I'm gonna be, be the contrarian and then tell, tell me how I got this wrong and then we can debate if we want.

Thierry de Pauw [00:10:30]:

Yeah.

David Subar [00:10:31]:

Okay. So I got a bunch of, I've got some user story that I've gotten from my product manager. I'm a software developer, of some sort, and I want to do a branch, I want to develop on that branch. These user stories are small because I have a good relationship with the product manager. I have a good product manager. They're, they're small and they're atomic. And so my branch is going to last for a day or two or three. And over those days I'm going to rebase from once what's on, what's on main.

David Subar [00:11:05]:

Why, why is that evil? Why wouldn't I do that? And then me and a bunch of my fellow coders can all have our own branches and we keep rebasing and then we merge and then life is happy. Why is that, why is that a bad idea?

Thierry de Pauw [00:11:22]:

Well, because we don't have feedback. So. Well, that's the first thing. So we don't have feedback as obviously as everyone's working on his individual isolated branch or they don't know how their work will integrate with the work of all the other things that are happening in parallel. And yes, you can rebase main onto your branch, but that is only main that you are integrating with your branch. You are not integrating all the changes that are happening on all the other parallel branches. So that's one, the second thing is, well, because they all work in isolation while we are hiding code. So I don't know what is happening on your branch and I don't know what is happening on the branch of someone else.

Thierry de Pauw [00:12:09]:

And so, well, when all these things go together, well, we might have some surprises which can introduce rework and this will increase lead time so it takes longer to, to get into production. Whereas if we are all committing into mainline, we all see the changes of each other, so we are communicating changes with each other and so we can adapt immediately. Like, oh, this is happening there, I might need to change this and this and this so it's more visible. And then there is a myriad of problems like it discourages refactoring because, well, once we refactor, we introduce new concepts and your abstractions and this is difficult to integrate with other people because they don't have to refactor. So they build on top of other assumptions. But in the meantime, you have affected code. Some concepts don't exist anymore or new concepts have been introduced. And this can introduce like semantic conflicts, not necessarily merge conflicts.

Thierry de Pauw [00:13:29]:

It might be that the code merges correctly, but there is a semantic error because, well, concepts have changed meaning and that you can only detect. Well, if there are automated tests we can have the chance to detect that. But if there are no automated tests, it's even worse to detect these problems.

David Subar [00:13:53]:

So actually you bring up, we have like five things to discuss just from that answer. But I want to go to automated tests for a while because I find companies do this very poorly, typically, and I would argue that without automated testing, it's very hard to have frequent releases and therefore do the thing that we agree on, which is you want to decrease the lead time before the software is to the customer. You want, you want to get that. So, is one trunk-based development or branch better or worse in the case where you have low coverage on automated testing, and then how do you solve that problem in either case?

Thierry de Pauw [00:14:43]:

So, well, the low coverage by automated test is problematic regardless of your use of branches or trunk-based development. It's a fallacy to think that because you are using branches, you are protecting yourself against this problem because the problem is still there. You still don't know if your changes broke something. Not so, but I hear this often. Yeah, yeah. No, we don't have automated tests. Do we need to use branches? It doesn't change anything actually. But that's a perception.

Thierry de Pauw [00:15:22]:

Yeah, some people then think, yeah, but we don't have the pull request and the code review. And the code review will also not save you from this problem. And I have yet to see someone detecting problems just by reading code. Well, you will detect the obvious things, but not the less obvious things. What else?

David Subar [00:15:46]:

I completely agree with what you're saying. So if you agree with me, which you may or not, that many companies have a problem, that their test coverage by automated test is too thin, there's just not enough. In either case, if you want to decrease the lead time to deployment so you can learn faster, you would want to have automated testing. How do we create an environment where we generate automated testing? How do we, and that becomes part of what we do culturally, whether it's trunk-based or branches or whatever, doesn't really matter.

Thierry de Pauw [00:16:31]:

Yeah. So obviously, yeah, we, it helps to have automated tests, but I want to add a side note is that automated tests won't. Automated test will just say that the thing we assumed it should do has been implemented as we assumed it should be implemented. But we don't know anything about all the things we don't know about the application. This is where then exploratory testing comes in and will detect all these things. Now this is a side note, so we need both automated testing and manual exploratory testing. Now how do we create a culture for automated testing? The only way I know about is that from the start we say as a team we are going to apply test prism development and acceptance test-driven development. So this means we add test-driven development that will create unit tests and then we have the acceptance test-driven development that will create the automated acceptance test, which are also called like functional testing, but they are still automated and that's how we will grow with test shoots and that we will have something that will be generally called a regression test shoot.

Thierry de Pauw [00:18:02]:

But it comes like, well, somehow for free because it's part of your development process. It's not like an afterthought, it's not like oh yeah, I've implemented this and now I'm going to add automated tests.

David Subar [00:18:17]:

So we're straying from trunk-based development, but I promise we'll get back to it. So you have a TDD environment or where people do TDD. That's accepted practice. Better to pair to make that happen. Doesn't matter.

Thierry de Pauw [00:18:37]:

Well, it helps. Well, it helps at older stages in the development process because once we start pairing, the need for code reviews falls away because we have reviewing while we are writing code. Also while software teaming, more known as mob programming, also helps, which makes string-based development even more easy because we are going to and work in sequences like we work on this feature and then on this feature. On this feature. We don't have parallelism anymore when we do software teaming, but yeah, so pairing absolutely helps.

David Subar [00:19:30]:

Hard to implement, at least in my experience.

Thierry de Pauw [00:19:33]:

It's hard to do, it's a cultural thing, and not all organizations are open to it. I've had teams where I've suggested it didn't work and then you look for other solutions. It's a cultural thing and if you introduce it in an existing organization it's more difficult. Whereas if it is present in the organization and you start recruiting, well, this is one of your requirements. Do you feel comfortable in working in pair programming or in mob programming?

David Subar [00:20:09]:

So let's go to the slightly harder situation. I am a new CTO in an organization that has some amount of legacy code because it's been around for three or four years and there's no automated test suite or a small one. And I want to say, hey, okay, I'm the CTO, so I have the power to tell you to do whatever I want. So from here on out, everyone's a do TDD and everyone's going to listen to me because I'm the CTO and there'll be no problem with that. Let's just assume that's true. What do I do about the legacy code base?

Thierry de Pauw [00:20:49]:

Yeah, so the theoretical most ideal way of handling it is by using a concept that is called approval tests, also known as validation tests. So you start by setting up a test harness around what is visible of the application. Might be the user interface, might be logging produced by the application, might be just sending some input and you get some output. And so, well, you just record these things as is without thinking about it. And then you record it once and then afterward use that as a test. When we send this input, we receive this output, we have a test. Now once we have that we can start refactoring in safe way because we now have this test that will eventually detect any problems on weekends. Well, and the idea of the refactoring is that we can introduce designs that allow us to introduce unit testing because unit testing requires that we decouple the code base.

Thierry de Pauw [00:22:13]:

So this is the preferred way of working. But setting up an approval test is cumbersome. It's not always easy. I've done it against logging. Well, capturing logging is not evident. I've done it in c, it's even less evident. And sometimes you just know what you need to do. Well, from my experience I know what I need to do.

Thierry de Pauw [00:22:38]:

When I look at the class, I know, like okay, yeah, we are instantiating classes inside the method. Bad idea. We need to introduce dependency injection. Let's refactor until we have a dependency injection. Now there is a risk by doing that without tests. However, if we are able to do these refactorings in an automated way using tooling, than just refactoring while we are kind of safe.

David Subar [00:23:10]:

Okay, so let's. Okay, I want to pop back to mainline development, trunk-based development, call it what you will. Okay, so we're all developing against, you know, against the trunk, against mainline and we're all developing there simultaneously. And we're all testing our code and we're good testers and it's a happy day, but, and we might be deploying, we might be doing automated deployment. So maybe we're deploying every day, several times a day, once a week. I don't care. I'm halfway through my development when the deployment cycle goes, how am I protecting the product from my half-developed code?

Thierry de Pauw [00:24:02]:

Yeah. So the easiest way to do that, and people are most often surprised when I say that you don't necessarily need toggles feature toggles to do that. You might be able to just hide unfinished functionality. So, like, for instance, we are adding a new screen to a user interface. Well, as long as the screen is not finished. Well, we just don't wire the screen into the navigation screen is there. It's not finished, but nobody can access it. It's okay.

Thierry de Pauw [00:24:44]:

Or we are adding a new backend service. While this backend service can be deployed, as long as it is not used by a user interface, it's okay. Same for API endpoints. As long as there is no documentation about the API endpoint. Well, nobody knows it exists. Can be there. It's now. The thing is, that doesn't always work.

Thierry de Pauw [00:25:11]:

Sometimes we need to change a little widget on an existing screen, or we need to change the behavior of an existing backend service. And this is where the toggles come in. That will help us in protecting us from ongoing changes that are not yet finished, but that allow us to deploy frequently even when the change is not yet, not yet completely implemented.

David Subar [00:25:43]:

Okay, so we have feature toggles. Feature toggles make me, I make me itchy.

Thierry de Pauw [00:25:49]:

Yeah, absolutely right.

David Subar [00:25:55]:

There's a risk. There's a code craft. There's like, people don't seem to clean them correctly. You just get a back-end feature toggle debt. Do you have a good solution? I don't, by the way, I don't have a good solution to this problem. Do you have a good solution to this problem?

Thierry de Pauw [00:26:12]:

So, well, so, yeah, so we have an example of a company that went bankrupt because of a bad implementation of a toggle together with manual deployments. It's what happened with Nite Capitals Services, which lost like $460 million in 45 minutes just because they reused the toggle between two releases, and then they forgot to deploy on one server, on one of eight servers, and then, wow, crazy things happened. So most of the problems with toggles, and each time I suggest, yeah, well, toggles is a solution. And then people come to me and say, yeah, yeah, we tried toggles, didn't work. The problem is, well, teams don't manage their toggles, so we need to manage toggles. And the first thing is, like work in progress, we want to limit the number of active toggles. So as soon as a toggle is not needed anymore, we remove it from the code base immediately. Now, to keep track of these toggles, well, it's important to have an inventory of both which toggles would have and to have tickets.

Thierry de Pauw [00:27:26]:

And so, well, it comes with some administration. So this is one, second, we want also to limit the number of toggles. And I've read some articles where they suggested like, yeah, we set a threshold number of toggles we want to maintain. And let's say we want a maximum of five toggles whenever a new toggle is introduced. Well, another one needs to go, so it's a choice we need to make. When we say toggle, we also say branching logic and branching logic needs to be tested. So we need to execute our automated test both against the toggle, on and off.

Thierry de Pauw [00:28:13]:

This is important. And then, yeah, well, branching logic means complexity in the code base maintainability problems. So there are many ways on how we can implement this. So the most naive, classic way is to use just an if-else check in the code base. Like is this the rule on or off? The most advanced way is to work with factoring and strategy and the strategy pattern so that we don't have conditionals anymore in the code.

David Subar [00:28:46]:

Okay, I feel like I have not given you, I've not given you the opportunity to detail why feature branches. Feature branches are evil. We started talking about that. I played the contrarian a bit, but I don't feel like I've given you the opportunity to talk about all the reasons why you'd like to do mainline development. So, yeah, what else do we need to say about that? I know there's a lot.

Thierry de Pauw [00:29:19]:

Yeah, well there is a lot. I've written a whole article on the subject. It's called the evilness of feature branching. The problems. It's quite long because there are many things that go wrong. So, yeah, so the first thing is, as I've already said, is it delays feedback. That's one. So the longer the branches open.

Thierry de Pauw [00:29:42]:

Well, the longer that we don't receive any feedback, we only receive feedback. The longer that we merge back into mainline. That's when continuous integration kicks in. The second problem is the integration of multiple features that are being developed in parallel, but each on their own branch. So integrating this becomes exponentially harder with the number of features that are being implemented in parallel and the number of changes required to implement those features. I've already mentioned it hides work for team members. So we are all in the dark of what is happening in all those parallel branches. It also working against adopting connective code ownership, which is one of the base values from extreme programming, which instills better code quality because, well, if everyone owns the code, well, it's easier to fix problems we detect and to just say like, oh, yeah, well, this is not good, I'm just going to change that because, well, I know this code, I also have the ownership, it's not the ownership of someone else.

Thierry de Pauw [00:31:00]:

Whereas if we use feature branches while people work in isolation, and then we start to think like, oh yeah, this piece of code is their ownership, and this piece of code is the ownership of someone else. I've already mentioned that it discourages the adoption of refactoring. Then the thing is, and this comes from manufacturing, it induces batch work. And from manufacturing, we know that batch work is bad. So Toyota discovered that the bigger the batch sizes was, well, the lower the quality was, and that when they worked in smaller batches, well, they had better quality and that they had faster feedback. They knew faster everything was right or wrong. But it requires skilled people and motivated people, so requires a different culture to work like that. It also introduces more risks, because if we work in branches while we are accumulating changes on the branch, and the longer the branch lives, the more changes we are accumulating.

Thierry de Pauw [00:32:10]:

And then all of a sudden we are merging this into mainline. So we are letting go of a big amount of changes onto the mainline, and eventually, this will be deployed in one go into production. And so if something goes wrong, well, finding the root goals will be more difficult because our change sets are too big. And so we will spend more time in recovering from an eventual failure. It's expensive also working on branches because we spend a lot of time keeping the branch up to date. With mainline, it's the whole rebasing. So every day you need to rebase, eventually fixing large conflicts and reworking the code, and we don't keep it up to date with all your parallel branches. And so it creates what altruists calls merged debt.

Thierry de Pauw [00:33:15]:

So we are creating a debt of potential merging that will happen in the future, but we don't know how much work it will introduce. And so it makes the whole delivery process very unstable and unpredictable. And then it adds a lot of cognitive overload, because, well, there is a lot of work that needs to be done on a day-to-day basis that engineers need to execute so well. They need to rebase the mainline onto their branch to reduce merge complexity. When starting a new feature, they need to create a branch and then create a reboot branch. If they want to communicate changes between features, they need to cherry-pick commits between branches. And then once the feature is finished. Well, we may not forget to delete the branch.

Thierry de Pauw [00:34:09]:

So these are lots and lots of operations engineers need to execute. And so engineers need to know all the commands to execute those operations, whereas with string-based development is fairly simple, we just pull the latest changes, add local changes, commit and push. It's the only thing we need to do. It's fairly simple.

David Subar [00:34:33]:

So what about GitHub flow? Right? It's neither fish nor foul computer.

Thierry de Pauw [00:34:41]:

Yeah, so GitHub flow is somehow a sensible branching strategy. Let's say it's possible to be compatible with continuous integration, and it's according to Ecxel.

David Subar [00:34:56]:

We just pause because some of the listeners may not know what GitHub flow is. So give, if you will, 30 seconds on what GitHub flow is and how.

Thierry de Pauw [00:35:07]:

It's different from, yeah, so GitHub flow is the branching strategy that GitHub started to use internally for their development process. So essentially it means they have, there is one long-running branch, the main branch, and then every feature is implemented on what is called a feature branch, supposedly short-lived. Now, there is quite a difference on how most organizations implement GitHub flow and how GitHub implements GitHub flow, and not many people are aware of that. So most organizations, what they do is they implement a feature on the feature branch, and once the feature is finished, they merge to main and then deployment happens from main, whereas GitHub, they deploy from the branch, and they only merge into main once the deployment was successful. The rationale behind this is that if something goes wrong in production, they can roll back by really going mainline and production. The result of a welcome like that is that they need to sequence merging of their branches and releasing of their features. So it's one branch at a time that is deployed into production and then merged into inline, where most organizations will merge multiple branches into main and then deploy multiple features at once into production.

David Subar [00:36:54]:

So when would you say GitHub flow is the right thing to use? When would you say mainline development is good to use? And then, just to be possibly contrarian, when would you pick feature branches, or never?

Thierry de Pauw [00:37:12]:

So it quite depends on my role in the organization. So I've always said you're God if.

David Subar [00:37:24]:

You get, you get to be God. Your role in the organization.

Thierry de Pauw [00:37:27]:

Yeah, so like if I would be in charge, there's no automated test, I would just tell the teams we're doing trunk-based development and we'll fix it. I'm aware it will introduce quite some stress, but at least it has the benefit of a way forward and imposing change because all of a sudden teams will have to do something about everything that is wrong because then they will have to look for, oh yeah, but how can we do this in a less risky way? Oh, maybe if we work in small increments, this is less risky. Well, here you go. You work in small incremental steps. This is one way to mitigate the lack of automated tasks. Because now I change this, what happens? I change that, what happens? You're more in control. It's not like you're releasing a bunch of changes all at once. And then, oh, something got wrong.

Thierry de Pauw [00:38:35]:

What might this be? We did a lot of changes. Damn it. So this is one way if I'm consulting coaching. Yeah, well it's different. And then I'll say to them, yeah, well, stay on your branches, it will feel safer for you and we'll start working on implementing to get a test.

David Subar [00:39:04]:

And then eventually go to mainline development.

Thierry de Pauw [00:39:08]:

Wow. So the thing is, what I suggest is to start collecting metrics about your build system. Like what's the lead time of your build, what's the failure rate of your build? If it fails often it's not good. And something is odd. If it doesn't fail that often anymore, you're going in a good direction. And then also collecting metrics about branch lifetime, like how long are the branches open? If this is starting to reduce, which we won't, well, this is going in the good direction. Eventually, at some point, we can say, maybe we don't need branches anymore.

David Subar [00:39:50]:

And when would you say GitHub flow is the way to go as the end state?

Thierry de Pauw [00:39:57]:

I see it as a transition to trunk-based development because, well, so what we say is like an organization is in a state of continuous integration when everyone commits at least once a day into mainline, which means while we are merging branches at least once a day into mainline. And then there is also the thing that while every commit triggers an automated build, and when the build fails, we can fix it within ten minutes. So these are the conditions for contingency integration. Now, my take on this is if we manage to keep branch lifetime reduced to a couple of hours or a maximum of one day, why would we bother to still create branches? Because it's an administrative overhead. Why would we still do that?

David Subar [00:40:50]:

Oh, right. Okay, so basically if your branches are short-lived enough, the problem solves itself and so you don't need them.

Thierry de Pauw [00:41:00]:

Exactly.

David Subar [00:41:01]:

So what is the big, I have a two-part question. You can answer the parts in any order you want. The questions are, what are the biggest resistance? What's the biggest resistance to doing mainline development, trunk-based development, and talk a bit about your consulting and how you help companies be more effective in doing software development.

Thierry de Pauw [00:41:30]:

So resistance is mainly, because they don't know. The whole industry is using some sort of branching strategy. And it has been popularized with GitHub as a development platform, and it was popularized especially for the open source industry where it really made a difference because, well, I remember that we were sending batches over email now we had this all request. Yeah. And then we applied the batch on the code base we had, then we start reviewing the code. Well, it's an interesting way of working. Of course, the pull request model made total sense for the open source world, but now we have a whole commercial IT industry that started adopting this open source model, which makes absolutely no sense for co-located corporate environments because, well, the open source world uses this because there is a trust issue. So we want people to contribute code, but we don't know the people that contribute code, so we want to verify the code they contribute.

Thierry de Pauw [00:42:56]:

Makes sense in a corporate environment. We are working in teams, we know the people that are working in the team and they all are supposed to be owners of the code. So there is no trust issue or they shouldn't be a trust issue. So everyone should be able to commit into mainline. But most often people are afraid that, well, it's gonna be a mess. And most of the time it's because they lack certain software engineering skills. And once you start showing them those engineering skills that make the difference, well then it starts, they start to get it, but you need to sit next to them, show how to work. And once they experience the difference and how much more easy the workflow becomes and how much more simpler, well, most of the time they cannot imagine going back to branches.

Thierry de Pauw [00:44:00]:

One case I like to tell it's a French engineer who was used to work with branches all his career, and then he enters quite advanced organization, quite extreme, programming oriented, that the string-based development all the time. And when he arrived he was like, yeah, no, no, this cannot be possible, this cannot work. And one year later, he's one of the biggest advocates of string-based development on LinkedIn, because he experienced this, saw the difference, saw how much simpler everything was, and also saw how much higher quality is delivered because it imposes a certain way of working. So you need to adopt certain engineering skills that will lead to better quality.

David Subar [00:44:55]:

So back to the second part of the question in your consulting, talk a little about your consulting, what you do and how help you help companies create more effective software development organizations.

Thierry de Pauw [00:45:13]:

Yeah. So my consulting is, well, it started to be focused on adoption of continuous delivery and so improving the whole delivery process. But most of the time it starts with adopting continued integration right way and solving the problematic branching strategy that is in place. So most of the time they have a crazy branching strategy in place and then you start asking why is it in place? And most of the time it's a fear thing. And then you start working on this. How can we mitigate this fear? What would help? And then, well, I'm most of the time there to just to explain principles and practices and show them different ways of working. Sometimes I'm just sitting next to them, like, show me the code, which is sometimes surprising because you never know how the code will look when you see it. So it's unprepared.

Thierry de Pauw [00:46:20]:

And then you get this code in front of you and you have like 10 seconds to make a reasoning, like, I think you are lacking this. Well, and then you start to show them like, well, most of the things that are missing is dependency injection. And they just don't know this, this concept, dependency injection. Then you show them like, look, when we do this now, we can test this, this piece of code in isolation.

David Subar [00:46:50]:

Yeah.

Thierry de Pauw [00:46:50]:

Then it was a big “aha” moment.

David Subar [00:46:55]:

So it starts with the, it starts with the tactical and goes with the conception, it goes to the conceptual and cultural. So one last question. Code pilot, code-whisperer, all those code generation tools, do they change any of this or is it all just the same from continuous development?

Thierry de Pauw [00:47:22]:

So what I think will change is that all the repetitive work, all the boring things will be generated by artificial intelligence, which also means that the need for specialists will be fewer. But if we will need even more generalists, people who know about a broad area of broad expertise, not really in-depth, but really have an overview of everything that is required to release an application. Just to judge what has been generated by artificial intelligence, because in the end, well, it's still computers that are generating code based on models that has been defined by humans.

David Subar [00:48:26]:

Actually, I said that was the last question. Clearly, by this next one, it wasn't. I claim that these gen AI tools empower the senior developer and are either do nothing for or scary to put in the hands of junior developers because they can't judge what was created and senior developers can't.

Thierry de Pauw [00:48:52]:

Yeah, exactly. So I don't believe they will suddenly start generating applications. We've been there before in the early two thousand. The same was said about the CASE tools that businesses would be able to just create an application by clicking around and then, well, here you go. You have an application. Didn't work. I still don't believe this will happen. It will facilitate our work.

Thierry de Pauw [00:49:20]:

That's true. It won't replace software engineers.

David Subar [00:49:28]:

So how will we train junior engineers to become senior engineers in the Viceroy world of code assistance tools?

Thierry de Pauw [00:49:39]:

Well, the same way as we trained them in the past, by having a senior sitting next to a junior. It's expensive, but it's, in my experience, the best way to train a junior. During six months, you have a senior sitting next to them showing ways of working, and that is okay. And then they are at a level where they can start reasoning and argument with seniors. And this is wonderful. I've seen this happen, like people in one year's time getting at my level and starting to read books and then starting to, oh, I've read this. What do you think? And I found this book. Have you read it? No, I didn't.

Thierry de Pauw [00:50:23]:

What do you think about it? It's wonderful.

David Subar [00:50:27]:

So Thierry. Thank you very much. I posed this conversation, the beginning about some things we agreed on, and then I took the contrarian view of things that we, in air quotes, didn't agree on. But as you know, as I think people listening to this will learn that we agree on many, many things. And including that last statement about the necessity, the necessity for senior programmers and the training method for junior programmers, probably not changing very much. So thank you very much and appreciate you spending time with me.

Thierry de Pauw [00:51:03]:

Thank you for inviting me.