Nava partnered with the Commonwealth of Massachusetts to support the launch of their brand new Paid Family Medical Leave (PFML) program, which guarantees paid time away from work to care for a new baby, a sick loved one, or one’s own illness.
We designed, tested, and built a new digital service — in just 13 months — to help Massachusetts workers as they apply for and manage benefits. Since we launched in January of 2021, the program has paid out $1 billion in benefits to people who were experiencing a big life event. We built digital tools for both claimants and employers using the program, helping to make managing paid leave simple across the board.
In tandem with this work, we built the API integration layer to connect multiple state systems with paidleave.mass.gov and other paid family and medical leave systems. In our shared long-term vision, Massachusetts workers, families, and employers will be able to confidently navigate life’s critical moments because paidleave.mass.gov’s personalized support makes managing paid leave simple.
We approached building this program by using pilots to continuously test our products with real users, ensuring that our tools met their needs and minimizing the risk of releasing untested tools for public use during critical times of need for beneficiaries. We’ll walk through how we used pilots to rapidly deliver paidleave.mass.gov.
Approach
We launched this project with several pilots. In a pilot, we incrementally develop pieces of the end-to-end experience and test them with real users (pilot participants) before the service launches to the public. This helps us identify and mitigate risks early on so we can affirm that what we ultimately deliver works for users, integrates with downstream systems, and collects the information needed to meet program goals. In a multi-vendor environment with a strict legislative deadline, pilots also serve as milestones for teams to align on so we can collaboratively build and validate new features.
For our first pilot, we designed and built an application that allowed claimants to create an account, log in, verify an account, reset the password, log out, and take the first steps toward creating a claim. We also tested a key technical integration with the Department of Revenue (DOR), which provides contribution and wage data that is essential to determining a claimant's eligibility.
In the subsequent pilots, we expanded the application to allow claimants to submit an application and ensured that Massachusetts administrators receive the complete and accurate data needed to adjudicate a claim. We then used a pilot to confirm that payments are made to the right person, in the right amount, at the right time.
Building and testing piece-by-piece, over multiple pilots, also allowed us to test the application with a diverse set of participants to further ensure that it’s simple and easy to use for everyone.
Outcomes
Pilots let us build, test, and see results quickly. Within a few months, we confirmed that most claimants could easily navigate account creation and that the DOR integration was successful. Our initial findings:
73 percent of participants were able to create an account
90 percent could reset their password
Of the 73 percent able to create an account, 100 percent could take the first steps to create a claim
28 items in the account creation flow were identified for improvement
100 percent of DOR data was successfully delivered, received in the correct formatting, and added to a database with zero errors
In the short term, our partners in Massachusetts were assured of our ability to solve increasingly large parts of the picture. In addition, we could quickly prioritize our next steps — based on the 28 items we identified for improvement — to ensure that 100 percent of claimants will be able to navigate account creation and that paidleave.mass.gov would be accessible to all at launch.
Process
Mitigating risks with real users
Nava often works in multi-vendor environments with strict legislative deadlines. In Massachusetts, we had 13 months to build a service to support an estimated 161,000 claims annually — from scratch. And on this particular project, we onboarded before the other vendor teams. So we wouldn’t be stalled, we specifically chose a scope for our first pilot that was independent of other teams.
We focused on the account creation user flow, from account creation through the first step of making a claim. In addition to being something we could start immediately, the account creation flow is also high-impact; it’s one of the first things PFML claimants will experience. Getting this foundational component right is essential to building a seamless user experience.
A simple log in page helps make the first steps as easy as possible.
For the pilot, we used the production environment. Our 11 participants completed the testing remotely, either on a desktop computer over Zoom, or on a mobile device, using a tool called Validately. Participants were Massachusetts state employees (we didn’t yet have identity proofing enabled and needed to be able to otherwise verify participant identities) who would eventually be eligible to use the service.
The ability to test a service with the people who would actually use it was essential to a successful pilot. It let us observe and identify user needs and fix negative user experiences that could ultimately hinder adoption and use of the new PFML service.
Each session had a participant, facilitator, and notetaker. Occasionally, someone from the Massachusetts program would also join as an observer. Seeing how participants interact with an application in real time gave program partners firsthand knowledge of how their programs are experienced by real claimants — it’s the most informative part of this process. So we encourage program partners to participate in pilot sessions whenever possible.
Identifying user needs
During this first pilot, a participant encountered an unexpected error message when attempting to verify their account. When entering a verification code, they accidentally added an extra space at the end. The participant couldn’t easily see the mistake. And the consequent error message used confusing, technical language; only the developers on our team understood it.
Instead of just updating this confusing error message, we streamlined the user experience to avoid it all together.
Getting an error message that doesn’t help users solve the error is a frustrating experience that erodes trust. So, from this finding, we eliminated the error message issue by simply allowing users to enter a verification code even with an extra space, streamlining the user experience.
In another instance, we iterated on the dashboard based on participant feedback. We revised both the content and the layout until we had a version that met user needs. Ultimately, we discovered that users need to see: their progress through the application, what types of information they’d need to complete each step, and new applications and in-progress or completed applications under different tabs. These features—a direct result of the findings in our pilot—provided a more clear user experience.
To best organize the information on this page, we tested several different versions of both the content and the layout with users.
Reducing administrative burden
In another testing session, we discovered a loophole where participants were stuck because they were unable to reset their password. This was important because we learned from other states with PFML programs that password resets are a primary cause of high call volume at call centers. We knew that creating a simple self-service reset password flow would reduce the burden on the call center, saving staff time and resources. (And in “Administrative Burden,” Pamela Herd and Donald Moynihan emphasize the importance of reducing unnecessary, often time-consuming tasks, so that government agencies can better serve citizens.)
In this case, a participant mistyped their email address when completing the flow for resetting their password. When they went back to the login screen to try again, the site had saved their mistyped email address and would only let them create a new account. So, we updated the flow to allow users to fix a mistyped email address and still successfully reset their password.
Conclusion
Pilots let us discover unforeseen and unpredictable errors like these in a low-risk environment, before they become a barrier to claimants and a burden to staff. In this first pilot, we were able to build and test the first steps of the account creation process — an essential part of the user experience — with real users. Working continuously and incrementally, we’re able to address issues as they appear so that we ultimately deliver a simple and effective experience for all users.
Written by
Senior Product Manager
