I’m writing about one hundred things I’ve learned as a product manager.
If you were to inherit a portfolio of ninety legacy products, some of which hadn’t been updated in years, with varying numbers of users relying on them, what would you decide to do with them? And how would you prioritise which ones to work on?
Those were the questions posing the Tactical Products team, one of several teams I helped out at Ministry of Justice (MOJ) Digital while I was there as interim head of product. The team distinguishes itself by its responsibility to provide day-to-day technical support on all these legacy products, as well as figuring out which to keep and improve and which to retire. This is a case study of how we went about prioritising their backlog of ninety products.
To complicate matters, many of these products were built to work with Internet Explorer 7 and Windows XP, a version of Windows even Microsoft has finally killed off, and the MOJ Technology team, which is separate to the Tactical Products team, was in the process of upgrading sections of the 70,000 staff to a much more recent version. Some other serious network renovation was also due to start a few months later. There was no guarantee that any of these legacy products would work after either set of infrastructure changes.
In an ideal world, the Tactical Products team would have had plenty of time and people available to investigate each of the ninety products in any order, choose a course of action, and carry it out. In reality, with both time and people in short supply, the team needed to prioritise the list of products to ensure they were working on the right ones first.
And this was the real challenge: with such a diverse set of products to manage, most of which were performing quite different tasks to each other, how should they compare and rank them?
Context tells you what matters most right now #
Any prioritisation exercise relies on using consistent criteria to compare one thing with another. However, the choice of criteria is subjective – it depends on a good understanding of your context.
Ask yourself: what is your organisation’s most important concern right now? It’s okay (and normal) for this to change over time. Or as Alistair Croll and Ben Yoskovitz put it in Lean Analytics, what is your one metric that matters? In other words, what is the one main indicator of how successfully your product is achieving your main goal?
Coupling this concept with Dave McClure’s pirate metrics, a company may initially want to focus on acquiring users – bringing them to its metaphorical front door. This would be a test of whether the message, “Hey! We can solve your problem!” is reaching and appealing to the people we think would care.
If user acquisition was what mattered most to us, then the one metric that mattered might be the rate of how many of the people to whom we sent the message came to the company’s website to find out more. We would also benchmark this rate against a control group (people who didn’t receive the message but came to our site anyway) to determine what would count as background noise. In other words, we’d want to know whether our product was achieving an acquisition rate better than what we’d see from people completely uninterested in the message. (Despite being uninterested, people still manage to click on stuff.)
Back to prioritisation #
To get back to prioritisation, once you’ve figured out what your main objective is (what matters most), you filter your work depending on whether it brings you closer to that goal or not, and you start with the things first that are most urgent and important to achieve this goal.
In this case, the task of user acquisition is the most important thing for the company right now (= most urgent). Other things may well be important, like converting those visitors into activated users, turning them into paying customers and retaining them, but none of those is as urgent. There’s no point in worrying about how to convert and retain customers you don’t have yet.
So rather than assuming that the highest priority is always making money or a high Net Promoter Score or whatever, figure out what matters most to your company right now, then ask: “to achieve that goal, what are the most urgent and important things we should do?”
Defining “important” #
To start shaping our thinking, the team and I set out some basic assumptions. We proposed that products that were more important were those that:
- were used most often;
- were used by more people; and that
- people couldn’t do their jobs without.
We could already pull usage statistics from the products, meaning in most cases we could identify who specifically were the users of a product, as well as how often they each used a product. We then put together a two-minute survey to ask four questions about each of the products:
- What impact does the product have on the work you do?
- On average, how frequently do you use the product?
- Do you know of anything new coming up that would replace the product?
- Is there anything else you’d like to tell us about the product?
The combination of the answers to the first two questions will give us a way of figuring out which products were the most important to the users. As an aside, we also thought it would be interesting to look for:
- instances where products are used very infrequently, but are seen as having a high impact on users’ ability to do their jobs; and
- mismatches between users’ perception of their usage patterns and the hard data of the usage logs.
The second two questions will allow us to tap into the wisdom of the crowd of users, partly to gain insight into the longevity of the product, partly to start uncovering any significant gripes (or even the level of satisfaction) that people have with the current product.
You might reasonably ask at this point why we have to survey users to find out whether there are plans to replace it. It’s important to remember that dissemination of information in a large organisation is imperfect at best, especially one as large as the MOJ. It’s not uncommon for similar projects to be running in parallel without any knowledge of the other. (But what a waste. *sigh*.) Hence, it was helpful for us to uncover any leads that might point to hitherto unknown replacement products.
Establishing urgency #
Once the Tactical Products team have the results back from their survey (not available yet at time of writing), they’ll be able to rank the products by their importance to users. With limited time and people available, they want to be concentrating our efforts in the right place. In their case, they want to be worrying first about the products that are most critical to users’ jobs and are used by the most people.
Doing this step first simplifies their next task, which is to figure out which ones within that smaller group of important products are the most urgent and so need their attention first.
The time factors affecting their choices come from planned infrastructure changes being made by the MOJ Technology team:
- the phased deployment of Windows 8 / Internet Explorer 11 to replace Windows XP / IE6-8 already underway; and
- the reorganisation of the intranet IP network addresses probably starting in a few months.
It’s not necessarily the case that every product will break as a result of these changes, but until the Tactical Products team have a chance to test them, they can’t know for sure, much less understand how to fix them. Testing the products to find this out will itself be relatively time-consuming (few of them will have automated regression tests, if any diagnostics at all), but because they’ll be able to filter the ninety products by their importance, they’ve at least cut the scale of the problem down.
At the time of writing, the team is yet to get the survey results back, so testing will start soon. Once they’ve completed testing the most important products, they’ll be able to establish how urgent each is depending on which set of IT infrastructure changes are going to break them. That way, they can be confident that they’re working on the most important and urgent products first.
Lessons I’ve learned #
When thinking about what to do in what order, whether it’s your product roadmap, a bug list or your household chores, it always boils down to focusing on which tasks are the most important and urgent.
The trick is recognising that what you define as importance and urgency is subjective: what is important to your organisation right now will likely be different in six months’ time.
Notice also that you can combine several factors into the one measure: in the example above, the MOJ Tactical Products team defined the most important of their ninety legacy products as those which were most critical to users’ jobs and which were used by the most people, most frequently.
Remember that we’ve been dealing with subjective (qualitative) data here. If you’re assigning values in a spreadsheet to different survey responses (e.g. ‘5’ is most critical, ‘1’ is not at all critical), then using those values to calculate priorities, don’t stress about decimal places – the input data simply isn’t that precise to begin with.
This process is simply a means to make a more informed judgement on priority – rather than guessing – in the context of what is important to the organisation right now, so that you’ll be able to answer with confidence the question: “why did you decide to do this before that?”