Collecting data to steer the product decisions (part 1)

Nothing is perfect. Your product is not perfect in its own special manner. You want to make it better and have some groundbreaking exceptional ideas? Wonderful! Consider this article as an input to bring these ideas to live and understand if you actually bring them live.

My best guess, that you want to identify the biggest pain points and solve them for the users you care about. You want them to be happy, productive, and praise your product instead of the product of your competitor.

Say no more! The only things you need to make it happen: your hypothesis, usage data, users feedback, and some guts to combine them together into product decisions

(I’ll skip the hypothesis part because you should already have them before starting reading this article)

1. Why are data-driven decisions important?

Data-driven decisions are the most realistic ones because they are based on the reality of usage and facts, instead of personal feelings and guessing.

Continuously gathering data allows you to iteratively deal with issues to minimize their impact. It can also help prioritize features and fixes when budgets and timeframes are scratched.

It’s a common issue when product guys start addressing imaginable issues instead of real ones. And data (or fact) -based analysis will help you to not fall into this trap.

2. What should we consider as data?

2.1. Step 1: The statistics of a daily usage

Yes, if a user clicks a button, it’s a good idea to collect this clicking event into data flow. Like

- session

- when

- what was clicked

- who clicks it

The last one is the most important, actually. Sessions are starting with something meaningful and business-related and end with… well, if everything goes well — with something meaningful and business-related.

I.e. The session starts with the user logged in and ends when the user logged out. During this session, the user clicks one million buttons of our application. All those million-buttons-clicks are united into one session.

Why is this important? Because the combination of facts (clicks) and timing will give you a real user-app interaction flow. A combination of those flows over some time will give you a user profile and how the user changes the behavior over time. A combination of those profiles will give you an insight into the most common flows (the core) and lead to the creation of user cohorts.

2.2. Step 2: the technical data

As we know from the very beginning of the article — nothing is perfect. Bugs in browsers, connection issues, old smartphone, etc. This is not product-related, but a major part of a common user experience

So, it’s a good idea to enrich the session data collected in the first step with a piece of information about the user’s environment:

- a browser the user is using

- a quality of the internet connection

- display resolution

- etc..

Combined with session data, it will give better visibility of potential issues and flow impact. Like if the frustration is only in a cohort of users who are using new iPhones with a bad internet connection, and not directly related to the product’s features.

2.3. Step 3: the user’s input

Yes, surprisingly (not), the user experience is a major part of making decisions on how to steer the product.

Gathering user feedback is one of the most effective ways to acquire data on issues with the product, and improve it. You need to know what customers’ subjective experience is, and it needs to be backed by evidence.

How to integrate the feedback into a data-driven decision process?

Surprisingly (not, again) — surveys are serving this purpose. But these surveys should be designed in a way that they will serve a clear purpose.

In the survey, you should only ask questions that fulfill your end goal and are designed in the context of the user’s session and the user’s cohort

Common tip: if you do not intend to use the information, do not ask that question.

The trick is to think about the user’s input as a result (endpoint) of the user’s session from Step 1 and create the infrastructure and approach to collect this input in the context of the session.

To make it happen, don’t force users to write an essay, but offer them to answer on 1–3 meaningful questions on this stage with predefined answers.

No need to ask if the customer is happy on the bug-reporting window — you already know that the user is unhappy.

If you want to know if the user likes the new payment window — the survey should be about the payment process and not about overall satisfaction

“Are you satisfied with the process?” with 1–5 stars will not give you many new ideas to think about. However simple, but specific questions like:

“Did you find the payment option you are used to?” yes/no

“Do you want to make the payment process easier” yes/no

Can give you some insights and will help you in steering using the user’s data

In the second part, I’ll write how to use the collected data for decisions