How to Evaluate Video Performance in Developer Relations

Did you know that 82% of developers prefer watching videos to learn about new technologies and tools? That's right; this is backed by surveys and statistics from SlashData State of the Developer Nation Report, DataProtocol, Wistia State of Video Marketing 2024, and, as one last confirmation, this week's Hashnode survey on LinkedIn shows this result:

Video has become the go-to medium for technical content consumption, surpassing traditional formats like documentation and blogs. This statistic alone should be a wake-up call for anyone in developer relations (devrel) – if you're not leveraging video, you're missing out on a massive opportunity to engage with your target audience.

But simply creating videos isn't enough; you must measure their effectiveness and ensure they align with your DevRel goals. DevRel video detractors will say, "You can't trust video metrics; they are unreliable." Here's my tip on how to measure video for DevRel.

This is where the Developer Journey Map comes into play, and for this article, I will use the fantastic resource DevRel.Agency Developer Journey Map.

Measuring Video Success Across the Developer Journey

The Developer Journey Map outlines five stages: Awareness, Interest, Evaluation, Adoption, and Growth.

Each stage has specific goals and questions that need to be addressed to progress developers to the next level and different metrics to call it "success." Here's how you can leverage video to measure success at each stage:

Awareness

The goal at this stage is to capture the developer's attention and introduce your product or service. Video metrics to use:

• View count

• Play rate

• Audience retention(average view duration)

• Description Links (UTM tracking)

Keeping track of the historical performance of these data points can help you gauge the effectiveness of your awareness videos. High view counts and strong audience retention indicate that your videos are resonating with developers and piquing their interest at this stage, you want an average view duration of at least 50% of the video length, if you are below this, your video is probably not interesting and people are just moving on, you've lost the opportunity.

Another important thing to do for awareness videos is to give the viewer a call to action, normally close to the end, to click a link in the description to sign up, download, learn more, etc. You need to track the clicks on these links using UTM codes or similar to know from the total number of video views how many of them clicked on these links.

Interest

Once developers know your offering, the next step is to pique their interest and provide more detailed information. At this stage, here are the engagement metrics I am looking for:

• Video Likes

• Comments

• Shares

• Description Links (UTM tracking)

These 4 become crucial for videos at this stage. If your videos are generating active discussions, being shared within developer communities, and conversion rate (e.g., developers signing up for a trial or downloading a sample project) it's a clear sign that you're successfully nurturing their interest, if not, while you might get a lot of views and a healthy play rate, we could say you failed if there are no likes, follow up questions, comments, etc.

Evaluation

As developers dive deeper into evaluating your product or service, they'll likely seek out technical walkthroughs, code samples, and demos. Here, the metric I look for the most is video completion rate. This can tell the effectiveness of your evaluation videos. High completion rates indicate that your videos are providing the necessary information and guidance for developers to make an informed decision.

If you have a demo video or tutorial on "How to do X or Y" and their completion rate is low, it means you failed since they probably didn't finish the tutorial.

Here's a scenario: On an evaluation video, you get a lot of views (more than average) - thinking you succeeded, but low completion rate, it turns out you failed.

On the other hand, on an evaluation video, you get a very low view (less than average) - thinking you failed, but high completion rate, it turns out you succeeded.

Adoption

At this stage, developers have decided to adopt your product or service and are actively using it. Video metrics I look for:

• Tutorial Viewership

• Support Ticket Deflection

• Adoption Rate

These can help you measure the impact of your adoption videos. If developers consistently watch your tutorial videos and adoption rates are high, it's proof of the effectiveness of your video content in onboarding and supporting new users.

Here's a personal example: at my current position as DevRel Advocate, working along with the support team(this is why collaboration is key for DevRel), we realized we had a lot of developers opening tickets and asking help about how to implement oAuth2 in our product. We support oAuth2 with no problem, and the feature works, but the implementation is different depending on the service to use. So, we created a series of videos about oAuth2 integration using different third-party auth services.

If we see the video metrics for these videos, we'll see a low view count (very low) but a large viewership and the support team saw a large reduction in tickets related to oAuth2. This is an example of success where low views, low likes, and almost no comments could be a misleading metric, but the fact that the few people who watched the video saw it until the end and reduced tickets, this is a success.

Growth

The final stage is all about retaining and expanding your developer community, this is probably the hardest to measure for video, I must acknowledge. Here, metrics like community engagement, user-generated content, and net promoter score (NPS) can help you gauge the success of your growth videos. If developers are actively participating in your community, creating their own content, and recommending your product or service to others, it's a clear indication that your video strategy is driving long-term growth and advocacy, so you will need to measure data outside of YouTube (or your video platform).

Let me give you a personal example: I was on a Livestream with one of our customers going over a success case of our product. Normally, these conversations are more value-driven than technical to showcase to the community how other companies have adopted our product to achieve their goals and solve their problems.

During the livestream, the guest mentioned that one of the reasons why they didn't move to our competition, apart from being a good product, is that every time they had questions, issues, or blockers implementing our tool, they could always find a video tutorial on our YouTube channel that explained how to do it. Our videos also gave them ideas on how to expand product usage.

So literally, they grow product usage thanks to video. At that moment, everything made sense.

Conclusion

But this is a written article, isn't it? Why isn't this a video if it's a powerful medium - well, embracing a multi-format approach aligns with the principles of inclusive and accessible developer education. By catering to different learning styles and preferences, you increase the chances of your content resonating with a wider audience, fostering greater engagement and understanding, I don't just do videos, I do a lot of written articles, docs, podcasts, etc.

Remember, video is not just a medium for developers to consume content; it's a powerful tool for building relationships, fostering trust, and cultivating a loyal community. Embrace video as a strategic pillar of your devrel efforts, and watch as your developer engagement soars to new heights.