New patients are not showing up for their first telehealth appointment. How would you solve this issue?

Question

New patients are not showing up for their first telehealth appointment. How would you solve this issue?

Acme inc is Tele-counselling service in which people complete weekly live video-sessions with a therapist. The company gets paid by an health insurance plan for each session that the patient completes. The basic funnel is that the company get a list of prospective patients from a health insurance plan. The company operates a call center that contacts each of these prospective patients and schedules an initial counseling session for them. During this initial call, the prospective patients generally agree to signing up for the service, and they schedule their first session for a future date. However, the problem is that a large percentage of these new patients never show up for their first session. How would you solve this problem?

Asked at Shopify over a year ago

3.3k views

Problem Solving category

Asked at

3.3k views

How to answer Problem Solving questions

Interview Guide

Top Problem Solving interview questions

Invite members
Invite by email

Add another

gabrielsimon · Answer 1 · 2020-04-19T21:14:15+0000

Here's what you should do to answer this problem solving question:

So as I understand it, Acme has a telehealth product. Patients meet weekly with a counselor to do video sessions, and the company gets paid for each session that the patient completes. The basic funnel is that you get a list of potential patients from a health plan (your target list). Then you have a call center that contacts each of these prospective patients and sets them up for an initial session. They generally agree to set up a session. However, the problem is that a lot of them never show up for that first session. This is the problem we’re trying to address. I also understand that the healthcare plan covers the full cost of the session and the patients didn't actively seek out this care. Instead, the health plan is suggesting it to them. Is that right?

Interviewer: Yes, that’s all correct.

Okay, great. So, I want to follow this process. First, I want to understand the user journey of showing up to that first appointment. And then I’ll identify pain points in that user journey, and then I’ll find solutions for the highest impact pain points?

Can I take a couple of minutes to think through the user journey?

Interviewer: Sure, go ahead.

Okay, here’s what I came up with:

User remembers they have an appointment
User decides they want to go to the appointment
User downloads app and sets it up/logs in
User has first appointment (meets with counselor)

Are there any steps in the user journey that I’m missing there?

Interviewer: No, that’s accurate.

Okay, so now i’m going to think through some pain points for each of those steps. Can you give me a couple of minutes to think that through?

Interviewer: Sure.

User journey stage	Pain points
Remember the appointment	Forgets about the appointment. Didn’t look at calendar or didn’t put it in calendar in the first place.
Decide to go to the appointment	Don’t see the value in it. This is definitely possible since the service is free for the user (covered by insurance) and the user didn’t actively didn’t seek out the treatment. Not at a convenient time Something came up / conflict
Download app and set it up / login	Has technical difficulties downloading the app, setting it up, and logging in.
Have the appointment (meet with counselor)	Not applicable. This is the final step that should occur if the other three steps are accomplished.

Before, I go any further. Do any of those particular pain points resonate more than the others with what you’re seeing in the data? In the real world, if I was working on this problem, I would send out a survey to the users that don’t show up and ask them why they didn’t show up, so that I could better identify the actual pain points.

Interviewer: I do have data, but I want to see what you'll come up with without any additional information.

Ok, no problem. Then I will prioritize them based on what I believe to be the highest impact. More specifically, I want to prioritize the pain points based on whether they’re a ‘trigger’ problem, an ‘ability’ problem or a ‘motivation’ problem. These three dimensions come from BJ Fogg’s model of behavior change, B = MAT. Behavior = Motivation + Ability + Trigger. When trying to change user behavior, he always recommends targeting Trigger, then ability, and then motivation. Triggers are the easiest to fix, just give some kind of cue/reminder for the behavior. Then ability is second easiest to fix, and then motivation. It’s hardest to change someone motivation is they are deeply ingrained based on people’s personality and values. E.g if someone is not interested in getting better, then it’s very difficult to convince them to.

The framework looks like this:

Interviewer: Oh, interesting framework. I'd never heard of that and looks really interesting. Sure, go ahead and prioritize based on those dimensions.

Okay, please give me a couple of minutes to apply them to the pain points.

User journey stage	Pain points	Trigger, Ability or motivation
Remember the appointment	Forgets about it. Didn’t look at calendar or didn’t put it in calendar in the first place/	Trigger
Decide to go to the appointment	Don’t see the value it Not at a convenient time Something came up / conflict	Motivation Ability/Motivation Ability/Motivation
Download app and set it up / login	Has technical difficulties downloading, setting it up, and logging in	Ability
Have the appointment (meet with counselor)	Not applicable. This is the final step that should occur if the other three steps are accomplished.

So based on that, I would choose to focus on the following pain points

Forgetting about the appointment (this is a trigger, which is easiest to address)
Technical difficulties downloading and setting up the app (this is pure ability. Particularly for older people who I understand to be a large proportion of your user base).

The other pain points had larger motivational components, so I’m not going to prioritize those, as they will be significantly harder to move the needle on. I want to start with the lowest hanging fruit: increasing ability, and providing relevant triggers.

Interviewer: Make sense.

Okay, so now I want to come up with solutions for each of the pain points that I prioritized. Please give me a couple of minutes to come up with some solutions.

Okay, here’s what I have

Pain point	Solution
Forgetting about the appointment	Reminders prior to appointment Push notification in app Email notifications Text reminders Automated phone call reminders Alter call center script to encourage user to put it in their calendar right then when they schedule the appointment
Technical difficulties downloading and setting up the app	Have call center rep guide the user through the process of downloading and setting up right then and there when they schedule the appointment Do a usability test to see where people are running into issues Improve the usability of the set up flow and process. Send people reminders (email/text) to set up the app prior to the session Create better instructions on how to download and set up the app. Can include this with the reminders mentioned above.

Okay, so I came up with 6 solutions. Now I want to prioritize them based on impact and implementation effort.

Solution	Impact	Effort
Reminders prior to appointment Push notification in app Email notifications Text reminders Automated phone call reminders	High	High
Alter call center script to encourage user to put it in their calendar right then when they schedule the appointment	low	low
Have call center rep guide the user through the process of downloading and setting up right then and there	High	Moderate (could be expensive depending on how long it takes and how much call center operators are paid)
Improve the general usability of the set up /signup flow. Run a UX research project to understand this better.	High	Moderate
Remind people to set up app prior to session Email reminder / phone reminder	Moderate	Moderate
Improve instructions on how to set up the app	Moderate	Moderate

So based on this prioritization, I would implement the following three solutions

Set up a series of reminder notifications 1 day and 1 hour before the appointments. While it is high effort to set up all these processes, this will have a very high impact, as it will serve as the trigger for people to show up to the appointments. We can also incldue reminders and instructions to download the app and set it up, if we see that the user hasn’t signed in yet
Have the call center representative guide the user through downloading and setting up the app while they are on the phone with them. We have success getting the users on the call initially to schedule the first appointment. We can take advantage of that captive attention, to guide the user through the account set up at that time. Of course, this all depends on the ROI in terms of the additional call center cost of longer scheduling calls versus the additional revenue gained by having people show up to their sessions. We’d need to model this out to reach a decision.
Improve the general usability of the set up/sign up flow. This isn’t a solution itself, but I would definitely run a UX study to understand how users go through the download, signup and set up flow to identify any major pain points, and then prioritize solutions from there.

Finally, I want to identify how I would measure the success of these solutions.

The success metric is very clear: the show rate for scheduled first sessions. Specifically, that would be calculated as the number of new patients that show up for their first session divided by the number of patients that schedule a first session.

I would run A/B tests where I would randomly assign people into two groups. One that experienced the new solution (e.g. reminder notifications) and one that did not experience the new solution. I’d then look for a significant difference (p=0.05) between those two groups for the first-session show rate.

So to summarize, our goal was to increase the first-session show rate. I identified a series of pain points, and then prioritized them based on whether the underlying issue was a trigger, ability or motivation. This led me to prioritize the pain points of 1) forgetting about the appointment and 2) difficulties with app download and account set up. I then identified several solutions, which I prioritized based on impact and level of effort. The two that I prioritized highest were a program of reminders prior to the session and having the call center rep guide the user through download and account setup during their initial phone call. I would also prioritize doing a UX research project to examine pain points in the setup flow to identify friction points and uncover potential solutions.

Prateek Srivastava · Answer 2 · 2024-09-30T10:04:28+0000

We begin by asking clarifying questions:

Do we have any information on the nature/type of ailment & therapy (i.e are they non critical or critical?)
What kind of service was promised as part of the first call?
Has any new player come in this landscape?

Key assumptions:

TG has a decent level of technological proficiency
Nature of ailments is non-critical.
Insurance company is providing the right type/group of patients.

Internal factors leading to individuals not joining the first call:

The individuals comfortable with the app interface.
Individuals have not understood the benefit of the offering at hand.

External factors due to which individuals do not join the first call:

Is there a competitor app?
Patients are not convinced of the utility of the application at hand (skeptisism).

Based on the user feedback, we frame the appropriate hypothesis and then use data from app usage/ onboarding flow & competitor study to validate our hypothesis.

Solution will involve triggering reminders, improving App UI, spreading awareness about the convenience & efficacy of the app, and fighting off competitor influence.

Ekta Mishra · Answer 3 · 2024-07-31T14:55:48+0000

Even though the question is pretty detailed we don't get a lot of perspective of the company Acme. Let us clarify about their process

Clarification questions

Is Acme a well-known company in its space? - No, it is an upcoming business
Does the client have to download an app to have a call with the therapist? - Yes like Practo
Is the information about the therapist shared in advance? - No, it is not
How much in the future is the call scheduled? Is there a cancellation fee? - This is based on the availability of the client and there is no cancellation fee.
Since Acme is a mental well-being app how do they select their user group as in whom to call? - this could be a tricky question but if there has been an incident with some like an accident then they might be flagged for PTSD that's when their profile is shared with Acme
What are the details collected in this initial call? - Just basic information, in-depth is for the therapist to unpack
Is there a follow-up when the person becomes a no-show? If yes then what is the response? - I was busy and I will schedule on my own later when I am free
Who is supposed to call on the date of the appointment? - Them or therapist? - Therapist

Let us draw a user journey

Client PoV

Gets a call from a random number asking them to meet a therapist -> Caller collects initial data like name, age, gender, phone number, and email ID -> Schedules an appointment

On the day

Client downloads the app -> Gets a call from the therapist -> Has a conversation -> Creates a schedule with the therapist

Acme PoV

Get basic information from the health insurance company -> Call the person -> Take information -> Schedule an appointment

Let us use the funnel for the client to see where the drops happen

Do they download the app before the appointment?
Do they pick up the call when the therapist calls?

Based on these numbers we can do the following

When scheduling a call the email should have detailed information about Acme's positive customer reviews, and basic information about the company, founders, mission, etc.
Not mandate the app and make it a WhatsApp call to remove the friction
Send them detailed information about the therapist like their education, experience, and positive customer experience. Give them the option to switch if possible.
Engage with the client between the time when the detail was collected and the first call was scheduled - borrowing this idea from the recruitment industry
A survey about why they didn’t join the call with some pre-set options could also give an insight

Tarun Kumar · Answer 4 · 2023-10-20T17:54:59+0000

Case in hand, new users who are signing up for the services are not showing up for therapy sessions.

Clarification needed:

I'm working as a PM for Acme Inc., safe to assume this?
The objective is to improve revenue, as many sessions are completed the revenue will increase, is that correct?
The task at hand is for users to adopt, engage, and retain with the counseling services, is that correct?
What's the geography we target here?
How long has this been happening?
What does the entire process of counseling sessions look like?
- What goes on in the counseling sessions?
- Does the user have to prepare something beforehand?
Is it a one-time session or a recurring session?
On the logistics side:
- Does it require a high-speed internet connection?
- Is there a device preference i.e., mobile vs desktop for these sessions?
I'd also like to understand how we convince users to sign up for therapy sessions today.
What time do we keep therapy sessions? Can user modify these invites once created?

Interviewer responds:

Yes, to PM for Acme Inc.
Though the ultimate objective is to hit revenue, for users we want them to adopt and engage
The geography is across the US
This has been happening forever
Counseling session:
User joins the session > Meet therapist > Therapy session begins > At the end therapist asks for a next date and then closes the session
This therapy session requires users to complete 3-5 sessions in 3 months
It requires high-speed internet and it is a website
Therapy session typically is one hour long
While signing up, we talk to users and ask them about a preferred time for therapy, let them know about the therapy benefits and that it's free of cost for them.
This is purely user preferred time and once created user has to call the Acme Inc call center to modify the timings.

Moving ahead, now that we have some information around the booking for therapy session.

Here are following information I'd like to seek.

Is this driven solely due to the clause present in insurance company, that is users may not want this but they are being targeted because of insurance policy.
-> Response: Yes it is coming from Insurance policy

I'd like to make a case considering the data points discussed so far

Users are not actively looking for insurance policy, there are not aware of the benefits.
Users are not interested in getting a counseling session.
Users want to reschedule the appointment
Users feel they would be more comfortable with a mobile app on the same

Recommendations:

Create mobile app that allows users to reschedule and book counseling session
Drive awareness for the users via targeted email and referral programs, seek more reviews from the users
Create a small quiz and QnA section in mobile to help users understand the need and importance of counseling session
Give users some personalized content, like an XYZ person in your position had these issues and now they are resolved, get a peek into their life.

Based on criteria like impact to the users, organization and effort required here, I'd pick following points

A small quiz targeted as an email campaign to the users
Target users with benefits around counseling sessions
Allow users to have a brief conversation with the counselor first and then let them enrol in the sessions
Have a referral programs to retain more users, the users who are referred are likely to retain higher

Check with the interviewer around the same.

Metrics:

1. No. of users open quiz >>> Completed the quiz >>> Signed up for therapy thereafter

2. No. of users read benefits >>> Came to the website >>> booked therapy session

3. No. of users signed up via referral program

Summarize the scenario once.

Looking for the feedback.

Kaushik · Answer 5 · 2021-03-02T16:12:05+0000

Clarifying Questions

Which geography is this activity happening? - US
Has this been an issue since inception or did it start happening recently? - Forever
Is it necessary that users do a video call with the professional? - Yes
Is this issue happening more on any specific platform - web, android, ios - or is it across the board - Across the board

Goal: Get all the users who have signed up to join the call at the scheduled time

To do this, I will first examine the funnel for the users, then talk about various issues that could have cropped up at each stage and propose a few solutions to alleviate them

Let's examine the funnel that is in place for this set of users:

Call center executives call the patient and sign them up for the service
Call center executives setup a call with a telehealth professional for a future date
The patient installs the app and gets ready for the call
The patient and the professional both join the call to complete the process.

We can assume that there is no drop-off between step 1 and step 2 since it is all happening on just one call. However, the dropoff between steps 2 and 3 could happen due to various reasons. There is also a possibility of dropoffs between steps 3 and 4.

Some possible issues:

The patient might have forgotten about the schedule
The patient might not have the right app installed on the phone or
The instructions to join the call might not have been clear
The patient might be worried about joining the video call on a phone
The patient wasn't fully prepared for the call and needs to reschedule

For each of these issues, there are various possible solutions

Reminders

Since the initial contact with the patient has been via phone, implement SMS based notifications and email notifications to ensure that the user signs up for the call
1. Impact - High
2. Effort - High (depending on the current setup)
Call the patient a day prior to the scheduled appointment to remind them about the call
1. Impact - High
2. Effort - Low (since the call center employees are already engaged on this)

Calendar Blocking

Block the calendar of the user to ensure that all the instructions are available for the user
1. Impact - Medium (we easily tend to miss blocked calendars also)
2. Effort - Low

App Issues

Get the user to install the app while on the call and have them log in - the app can help in further coordination
1. Impact - High (essentially we are eliminating the step 3 challenges by doing this)
2. Effort - Medium (it might take time for the call center employee to provide instructions for this)

Issues with starting the call

Get the professional to initiate a call with the patient instead of waiting for them to join the call
1. Impact - High
2. Effort - Low
Provide alternate options to join via voice only
1. Impact - High
2. Effort - Low

Privacy Issues and Concerns

Assure that the call center employees are trained accordingly and are providing them all the necessary information to assure the patient of privacy concerns
1. Impact - High (although depends on how many users care about this)
2. Effort - Medium (they might already be doing this, but script reviews are required)

Reschedule

It is possible that users are not available at the time to join the call, provide easy options for them to reschedule a call
1. Impact - High
2. Effort - Medium to High (depending on the current tech setup)

General Disinterest

Re-engage with people who haven't joined the call to see if there is still interest to do the call
1. Impact - Low (can't do much if they don't want to take the call because they don't have any reason to do so)
2. Effort - High

List of Prioritized Solutions

Call reminders
Get the professional to start the call
Provide an option to do a voice call
Help users install the app in advance
Email reminders