In any engineering organization, it’s important to balance investments in product development with those in tooling and other productivity enablers. It can be easy to slip into a mindset of focusing solely on new products and features, but the reality is that developer productivity, efficiency, and happiness also impact a business’s bottom line. But if you’re going to devote resources to improving tooling and other support systems, you want to make sure that your investment will have a positive net effect.
We frequently measure customers’ satisfaction with their product experience—why should our approach to tooling and developer productivity be any different? With that in mind, we overhauled our system at LinkedIn for surveying engineers about their satisfaction with the tools and resources at their disposal. As a result, we’ve seen improvements in our understanding of the specific pain points of different teams, enabling us to more effectively adapt certain tools or processes in order to improve the user experience.
Originally, we used an annual developer survey at LinkedIn to measure engineers’ satisfaction with the marquee parts of our tooling ecosystem. However, we realized that, as the company continued to grow, we needed to expand this survey in terms of both frequency and scope. We’ve since introduced a quarterly developer survey that covers a wider range of tools and systems used by each engineering team at LinkedIn, and we continue to work on creating feedback opportunities for engineers at every phase of the software development lifecycle.
As we’ve gone through this overhaul, we’ve learned several best practices that would be valuable for any company looking to better understand developer satisfaction and ways it could be improved.
Use partial anonymity
Privacy is a core value at LinkedIn, for both our members and our employees. In the interest of transparency, we do share certain aggregated results from the survey with the engineering organization as a whole, with the results often broken out by teams. However, no employee names are attached to survey results, and any team that has five or fewer respondents is not included to ensure that employee privacy remains protected. To encourage participation in the survey, it’s important that employees know their responses will remain anonymous.
When administering the survey, however, we do need to be able to identify certain characteristics, like which engineering team the employee taking the survey works on—this is why the process isn’t fully anonymized. In addition to enabling us to understand specific pain points for each team, this approach also allows us to customize the survey so that participants are asked to evaluate only the tools and processes relevant to their particular jobs. Making the survey as short and as targeted as possible is another key aspect of increasing participation.
Use different types of questions
Our survey is a combination of standardized scoring (evaluating on a five-point scale) and free-form text responses. We believe this mixed approach provides the most actionable results because it gives each element a “component score” that can be tracked quarter-over-quarter for improvement, while also allowing engineers to elaborate on issues they might have with certain tools.
For the scoring, we use Very Satisfied, Satisfied, Neutral, Dissatisfied, and Very Dissatisfied as the standard options. We ask participants to rate each tool or system from a given list on this scale, and then compute the component score based on the responses, with five being a perfect score. For the free response questions, in addition to evaluating each answer individually, we also create a heatmap of commonly occurring phrases, allowing us to better understand overall sentiment.
Commit to making changes
The whole purpose of conducting developer surveys is to discover which tools and systems could use improvement in order to increase developer happiness, so it’s important that you commit at an organizational level to act on the survey results. This also helps encourage participation in future surveys. After all, no one wants to spend time completing a questionnaire if their responses are ignored. At LinkedIn, our leadership is committed to investing in pain points that the survey uncovers. So when we send out the subsequent quarter’s survey, we include a report on the progress we’ve made in areas identified in the previous quarter’s results.
In order to have confidence in acting on survey results, it is important to make sure that you have established a margin of error that is deemed acceptable, and that the results fall within that margin. This will vary depending on the size of your organization and the survey completion rate, but in general, the smaller the margin of error, the better.
For instance, in one of our recent surveys at LinkedIn, the response rate was high enough that we could say with 95 percent confidence that our scores fell within a +/- 5 percent margin of error. Having a high level of certainty makes it much easier to get the whole organization behind acting on the results.
Adapt and refine over time
It’s important to remember that your survey process should remain adaptable as time goes on. Avoid the trap of automatically using the same questions and processes just because that’s what you’ve done previously. Just like the tools and systems we seek to improve, the survey should be refined whenever needed to meet your company’s needs.
For example, we’re working on developing a real-time feedback system that will prompt developers to share their experiences with the CI/CD tooling pipeline at the time of use. With this system, we will not have to wait until the end of the quarter to gather feedback signals on certain key areas, and the length of the quarterly surveys will be reduced—a win-win situation.
Creating a strong developer survey program takes time for both the survey architects and respondents, but it’s a worthwhile investment to increase developer happiness and productivity.