In May of this year, the Federal Communications Commission (FCC) launched the Emergency Broadband Benefit (EBB), providing a temporary discount on monthly broadband bills for qualifying low-income households. Eligible households that sign-up receive:
- Up to a $50/month discount on broadband service and associated equipment rentals
- Up to a $75/month discount if the household is on qualifying Tribal lands
- If the broadband provider offers it, a one-time discount of up to $100 for a laptop, tablet, or desktop computer (with a co-payment of more than $10 but less than $50)
What did we do?
The California ODI team wanted to better understand the mental models of Californians applying for this new benefit, and identify what specific pain points and blockers might prevent eligible individuals and households from successfully applying for the program. The CA Public Utilities Commission and ODI partnered to create a page on covid19.ca.gov to provide information about and elicit feedback from Californians about their experience with the Emergency Broadband Benefit program. (Because ODI manages covid19.ca.gov we were able to spin up a landing page quickly.) We directed visitors to the page by utilizing Google AdWords, targeting Californians who searched for “low-cost internet.”
We then set up a temporary ethn.io intercept for page visitors who wanted to leave a question/comment with their contact information, and followed up within 24-48 hours for live assistance. Many of the people we spoke with didn’t have home broadband access and had searched for the EBB page on a mobile phone or tablet. While we would usually have recorded participant’s screens, most participants were not comfortable getting on a Zoom. Instead, we recorded our phone calls, with permission.
Over the span of six weeks, our researchers spoke to over 60 individuals and families who were interested in applying for the EBB program. These calls ranged anywhere from 2-90 minutes; some of the respondents were in a rush and had a few high-level questions about the program, while others wanted to be walked through the process step by step.
What did we learn?
Most respondents we spoke to weren’t sure what steps they needed to take, or weren’t sure what to do once they completed the EBB application. As shown below, we also observed general anxiety and hesitancy around submitting personal information to the EBB website, along with gaps in knowledge around program eligibility, plan eligibility, and the expected program end date.
We also heard and lifted up several specific and actionable user pain points that could be addressed in a relatively straightforward way.
What challenges did people share with us?
Password validation: We noticed that many of the people we talked to were using the site on a tablet or mobile device. (This also makes sense given they were searching online for options for “broadband.”) This meant that they weren’t seeing the desktop view, in which the password hints were set-up to be easily viewable. For applicants on a mobile device, the password help was below the “Confirm Password” section.
ID requirements: In the clip below, an applicant thinks she needs to upload her driver’s license, but her current broadband is so slow, she can’t get it to upload. However, she didn’t need to upload it because she’d already provided the last four digits of her social security number. The EBB website stated that you only needed to verify through one method, but this applicant hadn’t seen it and assumed that because you could upload a file, she should.
Error messages: Applicants also showed us a few error messages they received. We know that these errors are hard to plan for when you’re building a site, and speaking to real people early on is a great way to unearth them. They may not affect a lot of users, but for the users they do affect, they can be show-stoppers.
For example, one applicant received a message about being in a “National Verifier state.” Another received an error because they only entered their middle initial, instead of name, and didn’t know how to reverse the error.
Additionally, some applicants who were interested in EBB ended up accidentally applying for Lifeline, a related but distinct benefit program, because the EBB and national Lifeline application portals have a similar user experience.
What was the result?
With our colleagues at the Public Utilities Commission, we had the opportunity to present our research to Chairwoman Rosenworcel’s office at the Federal Communications Commission. Her team facilitated a second meeting with the team that built the EBB site and, just as we had been, they were appreciative that these applicants had taken the opportunity to provide specific feedback to us.
We were elated when the FCC’s EBB team told us that they would implement six changes in response to the issues we uncovered, and were putting in tickets to get them done. We were even more excited to see the following changes go live just a few weeks later:
- Reminder Pop-Up Notification: A pop-up notification appears now after successful account creation reminding consumers that they still need to log-in in order to submit an application for Lifeline or EBB.
2. Identity Verification Section: Re-formatted the Identity Verification section (SSN4, Tribal ID, alternate ID) to only collect information based on the option the user has selected.
3. Username Field: Added language to the username field to clarify that consumers may use their email address as their username.
4. Middle Name Field: Added language to the middle name field to clarify that consumers attempting to enter their middle initial should provide their full middle name instead.
5. Password Requirements: Re-formatted this section to make the requirements more prominent for mobile screens so consumers see this information prior to the password entry fields.
6. Opt-Out State Messaging: Improved the messaging for consumers in California, Oregon, and Texas (the Lifeline NLAD opt-out states) by providing information about how to apply for EBB if they attempt to access a Lifeline application instead.
While the quick changes that were made may seem somewhat minor, they were in response to watching real people get stuck at these points. And for some, it would have made the difference between successfully applying for the benefit or not making it through. At the scale of a program like EBB, those many small problems faced by lots and lots of people add up.
Why did this go so well?
While we hadn’t planned on undertaking this research, we knew that the EBB program would provide a great opportunity to Californians. We were fortunate to talk with some colleagues at the US Digital Service early on, and knew that the EBB website was incorporating user research and other best practices to make it as easy as possible for people to use. We also knew that no matter how good the 1.0 of an application is, there will always be opportunities to reduce friction by observing real users using it.
Four elements helped this relatively limited effort yield such positive effects:
- A great partnership between the California Public Utilities Commission and the FCC. The PUC administers the California Lifeline program and had been tracking the EBB program in the state and nationally.
- Our ability to quickly build off of existing work our team at the Office of Digital Innovation had been doing throughout the pandemic to create and maintain the covid19.ca.gov site, using ongoing and consistent user research. This solid foundation made launching a new page in plain language quick and easy.
- Federal partners at the FCC and the Universal Services Administrative Committee (USAC) who welcomed feedback and had already built a structure that made it possible to make changes.
- A commitment (and low internal barriers) to identifying and speaking with Californians already looking to get the help this benefit provides. The only true experts of whether a service is working for users are the users themselves.