Disruptive News – Nick Mills, CircleCI
NICK MILLS [00:00:13] Hi, my name is Nick Mills, the general manager for EMEA for CircleCI. CircleCI is a software delivery platform centred around continuous integration and continuous delivery. The proposition is to try to help teams move as quickly as possible from idea to code delivery and value creation for their company. And we do that by providing a set of deep and feature rich tools for software teams to build, test and deploy code. And then once that code is live, to quickly validate, change and ship that code to end users.
[00:00:57] Just at the end of 2020, we produced a report called The State of Software Delivery to give you an idea as to why CircleCI is fairly uniquely positioned to create a report on how software teams do what they do. And we what the first reason is in the context of our scale. So we work with over forty thousand organisations truly global as a business across North America, Latin America, EMEA and APAC. And that gives us access to tens of millions of data points in the context of many millions of workflows, which are units of code which are integrated and then subsequently deployed into live production environments. And so on the basis of the data that we have, we the intention was to produce one of the first reports on software delivery using real observed usage data. Many reports typically focus on surveys, which obviously have a lot of value, but are, of course, based on the questions that are asked of particular participants to do surveys and indeed how they choose to answer those questions. So we focussed on real observed data, and the intention was to try to provide a set of consistent measures and then some benchmarks for software engineering teams to use to to baseline and then assess how they individually as teams and companies are performing. And we focussed on four metrics. The first was duration, which is the length of time it takes for a workflow to run from the point of a developer committing a change to getting some sort of signal whether that is a pass or fail. The second was mean time to recovery, which is the average time it takes between a failure and your next next success. The third was throughput, which is the average number of workflows per day that a team runs or kind of units of change, if you like. And the fourth was success rate, which is the successful number of workflows divided by the total number of workflows. And the intention was not to provide answers specifically for any given company. Of course, it very much depends on what a company is trying to do, what they’re trying to optimise towards, and which elements of an interconnected system that is their engineering team is most important to them in that their business goals. And we certainly don’t believe in a one size fits all set of metrics. And what we did want to do is provide some benchmarks and to try to help companies understand what to measure, how to measure it, and ultimately to try to instil a framework for and a culture of measurement.
[00:03:44] Specifically in the context of the report that we produced. We saw a couple of interesting data points to bring out to us in relation to the UK and UK tech in general. And this was interesting, partly because we work with, of course, a lot of Start-Up tech companies and growth to scale up tech companies, but also a lot of more traditional businesses that have gone through some form of digital transformation or may be running an innovation lab. So we come across a lot of sectors and types of companies. We have some interesting insights into how software teams in general are building technology. One thing that stood out was throughput, which the UK actually benchmarked incredibly highly against the median global performer median, taking into account the high performing companies or companies that have chosen to optimise towards a certain goal and normalised for that. And the UK on average, using that median benchmark, actually achieved an 80 percent increase in the total amount of output or throughput, which means that UK software teams are incredibly productive against the typical global company. The second interesting data point was success rates. So success rate is a very good indicator of just how innovative a team is. At the end of the day, you expect that if you’re moving quickly and you’re trying to encourage a culture of innovation within you, within your business, you you would see errors made and you wouldn’t expect every single code build or change that you make to be successful. For the U.K., we actually saw that the the median success rate was over ninety five percent versus what we typically see in very innovative, fast paced companies of more like 50 to 60 percent. So that’s quite unusual because most highly innovative and successful teams don’t spend lots of time, lots of extra time polishing and perfecting code before they make a change. They’re much more likely to take measured risks and rely on a high degree of automation and test coverage and really create a culture whereby individual developers and teams can move quickly and have confidence in the testing and operational processes they have in place.