361: Working Incrementally

thoughtbotter Stephanie Minn joins The Bike Shed as co-host! 🎉 Joël and Stephanie talk about continuing on a rewrite and redesign of a legacy Rails app and working incrementally. This episode is brought to you by Airbrake. Visit Frictionless error monitoring and performance insight for your app stack. Stephanie's listener question Stephanie's older episodes Case Expressions Specialized Vocabulary Mike Burns' smelly list Previous episode about note taking systems Transcript: JOËL: Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Joël Quenneville. And today, I'm joined by someone very special, Stephanie Minn, who will be joining the podcast as a co-host. STEPHANIE: Hi, Joël. It's me, Stephanie Minn. [laughs] JOËL: Welcome to The Bike Shed. STEPHANIE: Thanks. I am really excited to be here for the third time, I guess, but now in a more official capacity. JOËL: And together, I think both of us are now excited to share a little bit of what we've learned along the way. So for the first time as a co-host, I'm happy to ask you the question, what's new in your world? STEPHANIE: Well, I think I would have to say co-hosting this podcast. It's pretty big news for me, personally. I'm really pumped and also really nervous. I never thought I would co-host a podcast. But I have been a long-time listener of the show, and it feels very surreal to be here. I have big shoes to fill, even if I do have the same name as former co-host Steph Viccari. It's funny; I listened to a previous episode of The Bike Shed this morning, where I had submitted a listener question about project estimation. It came out earlier this year. It was really funny because Steph and Chris had a whole bit about speaking to this other Stephanie out in the world. That was me. And now it's kind of come full circle that I am in this position now. So I thought that was kind of fun. Now I could just say hello to all the Stephs out there. JOËL: Regular listeners will have recognized you because you've been a recent guest on a couple of different episodes, one where you talked about case expressions and how they fit into our style of programming. And then another one where we talked about the use of domain-specific or industry-specific vocabulary and jargon as a form of communication and how that plays out in the workplace. And for those who have not listened to those, we'll link them in the show notes if you want to catch up on the backlog. STEPHANIE: I really enjoyed being a guest on the last few episodes. It was cool to see things that we talk about internally come up in a way that we want to share with a broader audience because I think some of the things that we get into in book club or in the dev channel on Slack end up being really interesting and sometimes not things that we always see out in the content world, and we can dig really deep into that with this format. So that is really exciting to me. I think, in general, being on this podcast, I'm hoping in some ways will also be a growth opportunity for me because I am wanting to get more comfortable sharing my thoughts and ideas in a public space more frequently and informally. I'm very comfortable hoarding my thoughts until they're perfectly refined. And I'm like, okay, now I finally feel ready to share them in the form of a blog post or a talk. But I kind of want to open myself up to hearing from others, different perspectives, being more comfortable being wrong in public, and changing my mind, and evolving how I work and what I think in this way because I think it's important to normalize that. Yeah, I don't know how I can be any more vulnerable than being on a podcast on the internet. JOËL: Podcasting is really interesting because it's much more raw and unfiltered as opposed to something like a blog post or a conference talk where you've gone through a whole editing phase; you've rehearsed it. And you've spoken at conferences before, and you're speaking again. You'll be at RubyConf Mini in a few weeks. STEPHANIE: Yeah, I was thinking about my background because I actually come from a journalism background; that's what my degree is. And so I'm very comfortable in the role of editor, and I am a very obsessive and tedious editor when it comes to my own personal work in words and in code, I would say. The idea of putting stuff out there in a more unfiltered and less polished way is uncomfortable for me, but I want to get better at it, which is why I'm here because, like you were saying, in some ways, it's more realistic in just how we talk about some of these things at work. I kind of want to remove the Instagram filter of talking about software and technical topics. And sometimes, we are just learning about things along the way. And I think that's one of the things that I've really appreciated about The Bike Shed in the past. JOËL: Have you ever tried improv? STEPHANIE: Oh my God. Once I took an improv class as a team bonding activity at an old job, and that was really something. I think we all collectively were not into the idea, but that's what we were doing. So we went to I a think comedy club or something in New York with my team at my first job. And it was actually a lot more fun than I thought it was going to be. I think the instructors knew that most people would not be super comfortable with improv, and so they did provide a lot of structure in terms of the types of exercises and games we played. And we weren't doing improv comedy; we were just doing the exercises that would make us feel a little more comfortable and just having fun with it. Given some prompts, we would maybe walk across the room in a silly walk. And then the other person that we would walk to would imitate it, but it would be slightly different, and it would kind of evolve that way. So that's funny that you mentioned that. I had to really dig that memory from the archives [laughs] because I probably repressed it at some point. JOËL: I've also done improv a couple of times in a similar setting to you, like a team-building activity. I really enjoyed it. And in many ways, I feel like podcasting can feel like the improv version of the content world. STEPHANIE: Yes, and I agree. [laughter] JOËL: Yes, and very true. STEPHANIE: So, Joël, what's new in your world? JOËL: So you'd mentioned earlier how our developer channel on Slack is just a fantastic resource. There are some great conversations that happen there, and a lot of them eventually I like them so much I want to pull them into The Bike Shed and make an episode inspired by them. And I had a conversation today about the impact of what matchers you choose when you write tests and how that might impact the code that you write. So, for example, an equality matcher is almost like the primitive obsession version of testing matchers in RSpec and how which one you pick might impact the implementation of your code. An example of that might be if you have... let's say you're testing behavior on an order, and you might say you do some things, and you expect the order status to be the string or to equal the string pending. You've now exposed this internal of this order status string where instead you might using some of RSpec's automatic matchers for predicate methods, expect the order to be _pending because you now have a predicate method pending defined on your order. So by choosing to use a more rich matcher, you may have actually improved the encapsulation of your object. I had never thought about matters in that way before. It kind of blew my mind, and so I'm still kind of chewing on that, and maybe some of the implications of it. STEPHANIE: That's really interesting because, in my experience, I think I would reach for the more general matchers, the ones that seem to be more top of mind for me. And it takes an extra intentional thought to be like, oh, actually, I want this particular specific matcher to better reflect the behavior that I'm desiring. JOËL: This just came out of a conversation because fellow thoughtboter, Mike Burns, has a blog post that just lists a bunch of almost code smells or things that make him raise an eyebrow during code review that might lead to a follow-up comment and asking for clarification about why that choice was used. And on that list is the RSpec eq matcher that checks for just regular equality. And so we went a little bit deep into why that might be the case, and that brought up a lot of really interesting ideas that I had not thought about before. STEPHANIE: I'm curious if you are doing test-driven development if using a more specific selector. This is kind of the opposite of what I said earlier, but I guess I'm wondering if there's a possibility that it pigeonholes you into a particular implementation. JOËL: It might pigeonhole you into a particular interface, and I'm still exploring the idea. I think that the richer matchers move you away from implementation. So, in this case, or in the case of the example I talked about earlier, all you know is that there's a predicate method pending on an order. You don't know whether it's implemented as a Boolean internally or if it's a string that is being checked or some other thing. Maybe it's a status code. STEPHANIE: That's cool. Do you think you might explore this in your own work moving forward? JOËL: In a recent episode with Amanda Beiner, we had a whole conversation about note-taking systems, and I talked about having a sort of personal knowledge base where I keep deeper thoughts about code. And I definitely added a couple of entries to that today based off of that conversation. STEPHANIE: That's awesome. I totally know what you're talking about when you learn things or pick up little bits and pieces of information that you want to hold on to. But you might not have a particularly applicable project or codebase you're working in at the moment to apply that, but you want to hold on to it when you encounter that situation in the future. So I really like what you're saying about just adding it to your knowledge base and coming back to it. JOËL: I think the next time I'm writing a test and I feel the need to reach for eq I will immediately think of this and ask myself, is there something else that might be better? And if other matches feel awkward, why? So it's going to definitely cause me to be a lot more thoughtful about the way I write assertions. I'm curious to see if that will have an impact on the types of designs that my tests drive. STEPHANIE: Yeah, so maybe not a code smell but a code whiff. JOËL: Definitely. MID-ROLL AD: Debugging errors can be a developer's worst nightmare...but it doesn't have to be. Airbrake is an award-winning error monitoring, performance, and deployment tracking tool created by developers for developers that can actually help cut your debugging time in half. So why do developers love Airbrake? It has all of the information that web developers need to monitor their application - including error management, performance insights, and deploy tracking! Airbrake's debugging tool catches all of your project errors, intelligently groups them, and points you to the issue in the code so you can quickly fix the bug before customers are impacted. In addition to stellar error monitoring, Airbrake's lightweight APM helps developers to track the performance and availability of their application through metrics like HTTP requests, response times, error occurrences, and user satisfaction. Finally, Airbrake Deploy Tracking helps developers track trends, fix bad deploys, and improve code quality. Since 2008, Airbrake has been a staple in the Ruby community and has grown to cover all major programming languages. Airbrake seamlessly integrates with your favorite apps to include modern features like single sign-on and SDK-based installation. From testing to production, Airbrake notifiers have your back. Your time is valuable, so why waste it combing through logs, waiting for user reports, or retrofitting other tools to monitor your application? You literally have nothing to lose. Head on over to airbrake.io/try/bikeshed to create your FREE developer account today! STEPHANIE: So one thing I've been thinking a lot about in my client project currently is working on features more incrementally. Right now, we are working on a rewrite of the front end of a legacy Rails app. So they did a big, modern refresh on the look of the app. And we are rewriting a bunch of the views to those specs. And all of this is happening behind a feature flag. So we are able to ship work incrementally and not have as much of an issue where all of this work is happening on one big branch. But because a lot of this work builds on top of each other, I have been experiencing folks cutting branches off of feature branches, off of feature branches. And I've been thinking a lot about the intentional steps that we made to be able to deploy this in pieces and more safely but the friction that people still might have to work incrementally. One thing that I've noticed is that there are different levels to how this shows up in our work. You can work incrementally on an individual level where you are in your own work writing small commits that capture individual pieces, and you can feel good about that. But then you also have it at a team level where we have to collectively decide how we want to ship features. And I am trying to figure out how to encourage other people to agree on an approach and encourage the benefits of shipping features in small chunks. Have you ever noticed a discrepancy between individual work and how a team works in that way? JOËL: It sounds like what you're describing is that by encouraging the team to work in smaller chunks, it has made the Git and branching and merging process more complicated for the team as a whole. Maybe there's a bit of that tension whereby increasing the individual granularity; you're making the merging for the team more complex. STEPHANIE: Yeah, we've had a lot of rebasing issues where we do spend a lot of time in that mode because one branch went through code review and had some changes and then went through UAT and had some changes. And then we had to reconcile those changes with another feature that had been started and cut off of that feature branch. JOËL: That's really interesting because I feel like I've almost had the opposite experience. When I was a new developer, I would constantly get Git conflicts, and you have to figure out the merge, and it's a mess. And then eventually, I got decently good at resolving conflicts. Nowadays, I very rarely encounter conflicts, and the ones I get tend to be really minimal. And I think that's because I've started to really work incrementally and to keep my change sets small. And so Git conflicts are not really something I run into very much anymore. And I tend to think of them now as more of a symptom of large patches and code changes that might be bigger than they need to be. Does that line up maybe with your experience as well? STEPHANIE: Yeah, I think it does. But one area of friction that we are experiencing right now is that we may be working incrementally, but those changes don't make it all the way to our integration or production branches, and so they are still just hanging around. And we have had to fix merge conflicts, not necessarily with other people's work but with our own work where changes happened upstream, and then they kind of cascade down. So we are working in pieces. But because there is a little bit of process challenges that we're facing, we haven't quite closed the loop on that feedback cycle in a way that allows us to move as quickly as I think what you're describing. JOËL: It sounds like maybe your team is bundling multiple incremental changes and then trying to merge a bundle of them together. So while it's composed of small, incremental changes, the effect is similar to merging a branch that's got a large change set together. STEPHANIE: Yeah, that's exactly it. JOËL: In my own work, I tend to really view commits as the atomic chunks of the work that I'm doing, so each of them is going to be very tightly scoped and do a single thing. Ideally, also pass all the tests and be independently mergeable. I tend to view PRs not so much as like a unit of work but just a unit of review, a way to get feedback on one or more commits. I don't want to put too many commits in a PR because then it's painful for the reviewer. But also, another reason that you got me thinking about is that trying to shove fewer commits in a PR will also make it easier to do that final merge. STEPHANIE: Yeah, I know that I really appreciate when PRs are just one, maybe a few commits, but the overall diff is small. And in my opinion, I think we are able to move faster that way because reviews are quicker, conflicts are fewer. And it better captures the idea of working incrementally, even if it does involve more than a single, small atomic commit. And I would love to figure out how to move in that direction. Right now, in my client project, one of the barriers is the processes we've built into the agile methodology we're following, where our PRs have to get a couple of approvals and then be tested by folks from product. And so it's sometimes easier to just add a little bit of things to an open PR already. But what we were talking about with incorporating more of that intentionality really pays the cost up front rather than just pushing it until later when we do run into problems with conflicts or having to go back to debug something that went in into a big bundled PR and then having to spend time and energy at that point in the lifecycle of our work. JOËL: It's really interesting that you highlight the organizational impact and the process impacts. Definitely, when you increase the cost of merging when you make it like to merge a single PR, you're going to need to wait at least 24 hours because of all the other checks that need to go through, then people will tend to make larger PRs. And so sometimes it's not about programmer discipline or good habits even; it's about the process pressures that just really incentivize making larger PRs because of how expensive it is to open a new one. I'm curious, though, in a perfect world, when you are reviewing a PR for some code written in Ruby on Rails, what is the max amount of lines that you'd want to see in a diff before you start thinking this is too big; I wish we could split it up? STEPHANIE: I don't know if I have a number in my head. I want to say somewhere in the couple hundreds, maybe. I will love a PR where the diff is less than 100. That feels great to review and just feels right. I don't know; it's I don't open it and be like, ah, I have to now read through tens of files, some of which I have no context about. I like a tidy PR where everything that's changed is related to what the PR title is. What about you? Do you have a heuristic? JOËL: I think I'm probably similar to you, 100 lines is probably about the cut-off for where it starts to become more of a chore to review a PR. I've definitely had moments where somebody sends me a link and says, "Hey, can you review?" And I click the link and I open it up, and I'm like, oh okay, well, I should set aside a half hour here to really get into this because this is not going to be quick. STEPHANIE: For sure. And it also makes addressing that review feedback more difficult because you have to likely make more changes because there's just more to review and more to improve if it wasn't quite right the first time. The only time I do a big diff is if it's all in red. [laughter] JOËL: Yes. STEPHANIE: We have a Slack channel at thoughtbot called Dead Code Society where people post screenshots of their negative diffs, and it's so fun. JOËL: I'm all for that. When I look at code that has been broken down into nice, small commits, it just looks so clean. It looks so natural. But when I try to write code like that, it's anything but; it doesn't feel natural. Have you had a similar experience? STEPHANIE: I completely agree. I think it's really hard, and it's something that I am still practicing. Because when you are first learning to code, no one teaches you to write it incrementally; at least, that was my experience. It requires a lot of discipline to think about code in little, tiny chunks when you are just so excited to get your feature working and seeing it in a browser and playing around with it. When I first started doing it, I thought it was impossible. I thought it was wild to have a single commit be passing CI all the time because when I was writing code, there were so many work in progresses. And then I would run the test suite and be like, ah, 20 test failures. Now I have to go through and fix all of them. I guess what I learned from that was the pain of not working incrementally, and that is what motivates me to be disciplined. And it doesn't always happen. Sometimes I'm lazy and just decide that it's fine for now, and then will usually have to come back through when I am in that kind of headspace where I'm like, okay, let me really get down to business. And I'm able to see the seams of the code that I wrote to be able to extract them out into encapsulated pieces because that doesn't always come supernaturally. So yeah, I would say that it's an upward journey for sure. Do you find that to be true for your work? JOËL: It has definitely been a journey. I think as I have gained experience, as I have discovered new techniques, even picked up different perspectives and mindsets, all of these have helped hone that skill. And I feel like it's one of those skills that feels very mundane, but it's actually one of the more valuable skills I have as a developer is able to take something complex and decompose it into atomic pieces. STEPHANIE: Do you ever find yourselves in a position where there are obstacles that keep you from doing that, or would you say it's just an internal state of mind? JOËL: Maybe a little bit of both. You'd mentioned earlier that there can be process or organizational obstacles that make it much harder to try to scope down the work that you have. It can be internal in that you don't know the tools that you need for this particular scenario. But it's something where you're constantly on the lookout for ways to learn to be better. Over the course of your career, it is a skill that I think you're going to keep improving probably forever. I don't think you'll ever get to the point which is like, yep, I've mastered this. I'm as good as I will ever be, and that's the end. I'm curious, are there some tools or techniques that you like to use when trying to keep your work focused? STEPHANIE: From a commit level, I really love the git add --patch command. I think that it is really helpful because I like to litter my code with random debuggers and little changes that I don't end up wanting to bring into my commit later on. And so that is a great place for me to discard things that I know are distractions or were part of little rabbit holes I went down along the way. And so I highly recommend folks to check that out if that's not something that's already familiar to them. At a higher level, TDD has helped with staying focused because you have that built-in feedback loop. And once you are green, you can commit essentially and know that you did the least amount of work possible to get the behavior that you wanted without starting to sprawl into other territory. Joël, do you have any other tips or tools you want to share with our listeners? JOËL: A sort of mental tool that I've gotten into recently is drawing things out as a directed graph to understand what are the changes that I need to accomplish my goal and then what changes rely on other changes being there first? And a directed graph, you just draw a bunch of circles with arrows pointing towards things that they depend on. Anytime there is a cycle, that chunk of changes all has to go out together. They're all sort of interdependent on each other. And if you can find a way to restructure the graph or introduce a new step in there that will break the cycle, now individual pieces of that can be shipped independently. I've started using that visual approach sometimes to look at changes that feel like they're one sort of big blob of changes that can't be broken down and to say, oh, well, in order to ship this change, I need to introduce this new gem. I'm going to need to change behavior in this part of the system, but that will also need a change in another part of the system. And now that I can see how things are connected to each other, it gives me a clue to where I can rearrange the changes. Maybe there's a refactor that I can ship out first completely separate that makes the follow-up work much easier. Or maybe there's a way that I can introduce some of the changes without needing to do all of them at once. STEPHANIE: That's a really cool way to identify those pieces because it can be nearly impossible to figure it out from files in a repo or to keep in your head. And I think what's even cooler is that you can share those graphs too so someone else can come along and pick up that work, as well as just have the same level of understanding of what things depend on each other just through our clients. JOËL: I've really fallen in love with directed graphs and dependency graphs, in particular over the past year, and I feel like now I see them everywhere. And regular listeners of the show will have noticed that I have mentioned them multiple times. And I almost feel now like I'm that Parks and Recs meme where he's got [laughter] all his conspiracy board with all the threads connecting to each other and just like, let me tell you about graphs and how everything is all connected. They're behind everything. STEPHANIE: I'm with you. I think everything is connected, even just in our informal conversations on the pod and off the pod. We're constantly being like, oh, that's a great idea. Like, we can do a completely different episode if we go down this rabbit hole, but it also still maps back to things we talked about in another episode. And yeah, that feels very true to me in terms of software and also in terms of life, not to make that sound too deep. [laughs] But it's cool that you have found a way to manage that complexity for yourself at work and a way to share it with others. JOËL: I think one of these days, we're going to have to do a dedicated episode all about dependency graphs. STEPHANIE: Yeah, I would love to hear more. Too bad the podcast is just an auditory platform and not a visual one. [laughs] JOËL: That's the challenge, right? Because it's such a visual topic. STEPHANIE: That's cool. I'm really looking forward to hearing more about them. On that note, shall we wrap up? JOËL: Let's wrap up. The show notes for this episode can be found at bikeshed.fm. This show is produced and edited by Mandy Moore. If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review in iTunes. It really helps other folks find the show. If you have any feedback, you can reach us at @_bikeshed, or reach me at @joelquen on Twitter, or at [email protected] via email. Thank you so much for listening to The Bike Shed, and we'll see you next week. Byeeeeeeeee!!!!!!!!! ANNOUNCER: This podcast was brought to you by thoughtbot. thoughtbot is your expert design and development partner. Let's make your product and team a success.Sponsored By:Airbrake: Deploy fearlessly and fix bugs faster with Airbrake Error & Performance Monitoring. Airbrake notifiers are available for all major programming languages and frameworks, and install in minutes, with an open-source SDK-based install and near-zero technical debt. Spend less time tracking down bugs and more time developing. Visit Frictionless error monitoring and performance insight for your app stack.Support The Bike Shed