Ben White
Strategic systems

Measuring experience quality across a product area

Role: Product Design Lead, 2023

Goal: Prioritising experience solutions on evidence

Improving quality across a large set of established products starts with knowing where the problems are. We had no objective way to see how each product’s experience compared, or where investment would make the most difference.

I designed an experience scorecard that gave teams and leadership a consistent way to assess product health - covering interaction patterns, accessibility and responsiveness - so quality could be tracked and prioritised across the product area.

The impact

  • A consistent view of product health Six products assessed against the same criteria, giving leadership one comparable view of quality across the product area.
  • Accessibility and responsiveness made visible Checks built into every assessment surfaced accessibility and responsiveness issues alongside the rest of a product’s health, rather than as a separate exercise.
  • Teams able to assess their own products Self-serve audit tools let product teams check their own quality without waiting for a central review.

No objective view of experience debt

Across a varied set of products, inconsistent interaction patterns and different levels of technical maturity build up over time. Some products were clearly showing their age, but we had no objective way to measure how far each had fallen behind, or where fixing it would matter most. Without that, prioritisation tended to follow whoever made the strongest case rather than where the problems were greatest.

Making quality measurable

I wanted to move reviews away from subjective judgements about how a product looked, towards a consistent health check that could be applied to any product. I ran sessions with leadership across Product, Engineering and Design to agree what good looks like, which gave everyone a shared language for experience quality. Measuring experience debt in a consistent way made it easier for leadership to treat quality as a product risk, not just a design preference.

This is a great assessment, and I feel I’ve now a clear overview of where opportunities are.

— Design Director

A shared way to track product health

The result was a scorecard and a heat map that showed where experience debt was highest across the product area, alongside self-serve audit tools and scoping kits that let teams assess their own products.

What did I do on this project?

  • Agreed what good looks like Ran cross-functional sessions with leadership to define experience quality in terms Product, Engineering and Design could all use.
  • Designed the scorecard Designed the scorecard and the criteria behind it, including accessibility and responsiveness checks.
  • Made priorities visible Developed a heat-mapping approach that showed leadership where experience debt was highest across the product area.
  • Built self-serve tools Designed audit tools and scoping kits so product teams could assess their own products without depending on a central review.