CI/CD and how data is the gatekeeper of productivity [Q&A]

Using CI/CD (continuous integration and continuous deployment) is supposed to streamline and speed up application development.

However, outdated cloud-based approaches negate many of the benefits. Buildkite's co-founder and CEO, Keith Pitt helped pioneer the hybrid CI/CD approach, which combines elements of both managed and self-hosted CI/CD. With hybrid, users control the build environment and can customize their needs. We spoke to Keith to learn more

BN: What issues are you seeing today with product development that are not talked about?

KP: For the most part, people don't quit jobs, they quit cultures. When developers are in an environment that is too data-driven and overly obsessed with the time it takes for code commits to make it to the product, developers can feel stagnant in their creativity. The industry often downplays the role developer tools play in overall developer happiness. It's 2024 -- we need to wake up and realize that using archaic and subpar tools will result in lower productivity, leaving a negative impact on the developer experience. Imagine you're a carpenter and every time you hit a nail you had to wait six hours to see if it worked. When building code, developers regularly have to accept this as their reality. Not to mention that heavy data-driven approaches tend to hinder a developer’s opportunity to focus on actual problems and come up with novel solutions. In my experience, a balance between data and creativity results in faster development, more tests run, and an energized team of developers.

BN: How does focusing on the joy of invention help the productivity and happiness of developer teams?

KP: Creating software that works is the best feeling a developer can experience and is ultimately fundamental to their success and happiness. Teams that feel inspired by real challenges craft the best solutions. I understand that every developer is unique, driven by different passions and goals. In my opinion, there are two personalities one encounters in software development: 'poets' and 'librarians'. Poets are inherently creative and think of big ideas, whereas librarians desire structure and make sure everything is in order. While they may have different working styles, poets and librarians ultimately have the same goal: make software that works. Focusing on the right mix of poets and librarians in an organization ensures that software teams don't lose sight of creativity, but are also able to develop code quickly and make deadlines.

When I first started working on Buildkite in my free time and on weekends, I didn't have a grand vision of creating a global software company with over 100 employees. I built it out of necessity so I didn't have to keep using poor CI/CD tooling. As it turns out, I wasn't alone in my frustration and here we are over a decade later with a product that continues to evolve and is used by some of the most sophisticated developer teams in the world who are continually on the forefront of innovation. Hundreds of other founders have the same story, and hundreds more will have it in the future, and it’s all driven by the initial intent to improve their personal experiences.

BN: You mentioned that breakthrough products don't come from focus groups. How does this challenge the traditional approach to product development?

KP: The best decisions occur when developers are unencumbered by popular opinion. Product design by committees often leads to bland and unremarkable products that excite no one, not even the designers. This is because products or ideas that are subjected to arduous approvals by stakeholder after stakeholder can be diluted of any real creativity for the sake of compromise. The end result may be safe and effective, but I'd be willing to bet it lacks unique qualities that stand out. The best products across all tech industries stand apart because of the opinions those teams held. Breakthroughs often spring from the minds of those who have strong opinions on how things should be done.

BN: Software development is often regulated by rigorous testing. How does 'trusting your gut' change that process?

KP: In reality, the process of building anything of value requires thousands of decisions and attempting to test and validate each one is a daunting task. This is when relying on intuition or a 'gut feeling' becomes critical. You can test all you want, but the real test is whether or not someone wants to buy your product.

BN: How might the advancement of AI impact the perspective of DevOps?

KP: LLMs, in their current form, will play a large role in helping developers kickstart code, so they don't have to constantly start from scratch. In that same vein, more workloads will be completed with AI. For example, we'll likely see an increase in the actual volume of code written and automated tests completed due to the integration of generative AI. Developers are always going to want tools that will make their lives easier, and AI may be the answer to a lot of their problems. In fact, LLMs will allow us to be more creative and live our best ideas. We may see more 'poets' and less 'librarians.' With all of this said, the AI space is changing day to day and my opinion is constantly shifting, just like the rest of the world’s. When using AI, human review cycles will help keep things honest and innovative. Instead of relying on technology built a couple of years ago, we should ultimately rely on our gut which has been trained for over a thousand years!

Image credit: ArtemisDiana/depositphotos.com