Beyond Reach and Clicks

Katharina Knott, Chief Operating Officer, Mediteo GmbH 

 

Which Key Metrics Should Be Measured?

The right metrics depend on the specific objective of the project. A project focused on activating patients requires different measures than one aimed at driving repeat usage or providing access to specific information.

Typical metrics may include:

• Reach: How many people were able to see a project-specific touchpoint?

• Activation: How many people interacted with content that was displayed or made available to them?

• Usage intensity: How frequently is a specific piece of content or feature used?

• Repeat usage: Do users return to the offering?

• Drop-off points: At which stage of a defined user journey do users most frequently drop off?

• Target actions: How many users complete the action defined as a project objective?

• Development in time: Does usage change following the launch, the start of a campaign, or during the later phases of the project?

Different cohorts can, of course, also be compared with one another (e.g. demographic cohorts, patients currently undergoing treatment with an alternative medication, new vs. long-term app users, and many more).

Each metric should be linked to a specific question from the start. A click rate alone says little without knowing which user group was targeted, where the touchpoint was placed, and what action was expected after the click. In many cases, meaningful insights therefore emerge from a combination of the pharmaceutical company’s research objectives and vision and the digital health company’s expertise and experience.

Flexible Participation in Real Time

Reporting does not begin with the first analysis; rather, it starts as early as the project planning stage. If objectives and metrics are only defined after launch, baseline data may be missing or relevant events may not have been adequately accounted for.

1. Define the project objective and measurement logic

At the start, it should be clear which user behavior the project aims to observe.

In the case of an information service, for example, it may be relevant to understand how many users open the content and whether they proceed to further information. For a service offering, the focus may be more on whether a feature is used repeatedly or whether certain user groups remain active for longer. To determine the appropriate key performance indicators for a project, a kick-off meeting or workshop between the subject matter experts at the pharmaceutical company and the project manager on the app side is essential.

Based on this, the interactions to be measured and the reporting period are defined.

2. Capture relevant interactions

After the launch, the previously defined events are tracked. However, not every technically measurable action should automatically be included in reporting.

A meaningful report focuses on events that are relevant to the project decision. If there are ten possible click paths, it may be sufficient in some cases to look at three key steps if these accurately reflect the actual patient journey.

Tracking must always be GDPR-compliant. At mediteo, actions are recorded exclusively without any personal identifiers, meaning that it is no longer possible to draw conclusions about individual users. In addition, raw data is aggregated to generate meaningful insights across the overall population. This helps protect the anonymity of individuals, minimize regulatory hurdles, and reduce risks for clients.

3. Put metrics into context

Absolute numbers are often not sufficient for interpretation.

500 interactions can represent either very strong or very weak performance. This depends, among other things, on how many users saw the touchpoint, how long it was available, and how much friction was involved in completing the desired action.

Reports should therefore put metrics into context wherever possible, for example by looking at:

• Interactions relative to the user base reached

• Development compared with the previous reporting period

• Usage before and after a content adjustment

• Differences between individual touchpoints

• Changes in usage over several weeks or months

To this end, it is beneficial for the pharmaceutical company commissioning the project to share additional influencing factors that are known to them with the digital health company (e.g., previous launches, competitive situations, new approvals, etc). This can help ensure that the analyses address the specific questions and uncertainties that need to be explored.

How such Real-World Data can be leveraged beyond the individual project to address broader pharmaceutical questions is demonstrated in our article on Real-World Data and Real-World Evidence: https://www.mediteo.com/en/patient-engagement-evolving-how-real-world-data-opens-up-new-perspectives-on-treatment-success/ .

4. Jointly evaluate results

Simply providing numbers is of limited value to pharmaceutical project teams if it is not clear what conclusions can be drawn from a change in the data.

If usage of a touchpoint declines, several potential causes may be responsible. These could include unfavorable placement, an overly complex user journey, a seasonal effect, or a change in the user group that can be reached. Reporting should distinguish between these possibilities rather than automatically attributing a cause to a single metric.

Particularly due to inherent cohort bias, the need for technical expertise, and potential technical limitations, it is essential that the digital health company contributes to the interpretation of the data. Therefore, data should not only be shared in raw, complex tables, but also presented through clear visualizations, accompanied by relevant annotations and helpful interpretations. Where appropriate, expert recommendations can also be highly valuable to pharmaceutical companies, particularly when tailored to the target audience in a digital context. However, it is equally essential that the commissioning party contributes its own expertise and asks questions in order to extract the maximum possible value and insight from a project.

How a mediteo report can be structured

The structure should be guided by the decisions the pharmaceutical team needs to make based on the data.

Management Summary

The first level should provide a concise summary of the most important developments. This should not include every available metric, but rather focus on the changes that are critical to the progress of the project.

Examples:

• Activation rate increased compared with the previous period

• Usage of a specific touchpoint significantly below expectations

• Repeat usage remained stable despite a decline in reach

• Noticeable drop-off at a specific step in the user journey

KPI Overview

In the next step, the key performance indicators are presented alongside comparison values.

Useful metrics include:

• current value

• previous reporting period

• absolute and relative change

• agreed target value, if applicable

• brief professional interpretation

In addition, including representations of the content published in the app or digital health application, or visualizations of the user journey, has proven valuable for increasing understanding of the medium.

Detailed Analysis

When a metric stands out, the report should narrow down the potential cause as far as possible.

For example, the following questions can be examined:

• Does the decline affect all touchpoints or only one?

• Has reach changed, or only the interaction rate?

• Is usage one-time or recurring?

• Did the change occur following an adjustment to the project?

• Does the pattern persist across multiple reporting periods?

At this stage, targeted additional analyses are often worthwhile, as they can add further dimensions and depth to the insights gained from the project.

Next Steps

A report becomes particularly valuable when the data leads to concrete test actions.

This can include:

• Adjusting the placement of a touchpoint

• Revising the copy or call to action

• Removing a little-used step from the user journey

• Making a successful interaction more visible

• Testing a specific hypothesis in the next reporting period

At this final stage, pharmaceutical companies have a particularly important role to play. How results and insights are used internally, and what actions are taken as a result, help to better prepare future projects, identify and eliminate technical hurdles at an earlier stage, and ensure that long-term collaborations are optimally coordinated. A detailed feedback and debriefing meeting is therefore often already the starting point for a new project.

A Caveat and the Appropriate Solution

Even comprehensive reporting has methodological limitations.

Usage data shows that a particular behavior is taking place. It does not automatically explain why it is happening.

A decline in usage may, of course, be caused by a lack of interest. However, other explanations are equally possible, such as an unfavorable user journey, a change in the patient situation, or a technical issue. This distinction is important because it prevents a correlation from being prematurely interpreted as a causal explanation.

If the underlying cause of an observation is of interest, tracking should therefore be complemented by additional methods, such as qualitative interviews, tests, or short surveys. The latter, in particular, can be conducted quickly and easily through in-app surveys, directly within the environment and context in which the measures are being implemented.

Conclusion

Performance tracking at mediteo is not about providing pharmaceutical teams with as many metrics as possible. Its value lies in regularly evaluating project-relevant interactions in a consistent and comparable way. This enables teams to assess which touchpoints are actually being used, where users drop off, and whether adjustments have measurable effects.

For pharmaceutical companies, this creates a tangible advantage in project management: decisions do not have to wait until the final evaluation. They can be reviewed and adjusted based on ongoing usage, and the impact can then be reassessed in the next reporting cycle.