Validation of VA’s Self-Scheduling Tools Gets GAO Stamp of Approval

The Faster Care for Veterans Act of 2016 was passed to address veterans’ access to care issues and mandated the development of an online tool for patients to self-schedule appointments.  The act also required VA to “hire a non-governmental, not-for-profit entity with expertise in health information technology to perform independent verification and validation (IV&V) of the (new) patient self-scheduling system and any other patient self-scheduling system used by the department.” VA hired MITRE in April 2017 for the IV&V work.

According to the legislation, the self-scheduling tool(s) needed to fulfill a minimum of seven core requirements:

• Schedule, modify, and cancel appointments for primary care, specialty care, and mental health.
• Support appointments for the provision of health care regardless of whether such care is provided in person or through telehealth services.
• View appointment availability in real time.
• Make available, in real time, appointments that were previously filled, but later canceled, by other patients.
• Provide prompts or reminders to veterans to schedule follow-up appointments.
• Be available for use 24 hours per day, 7 days per week.
• Integrate with VistA, or such successor information technology system.

The act required GAO to evaluate whether the IV&V of the scheduling tool included the seven minimum capabilities specified and “was performed consistent with practices included in the Institute of Electrical and Electronics Engineers’ (IEEE) Standard for System and Software Verification and Validation.”

To fulfill the requirements of the Faster Care for Veterans Act, VA acquired the On-line Patient Self Scheduling

(OPSS) system.  Prior to the enactment of the legislation, VA had developed the Veteran Appointment Request (VAR) system to also support self-scheduling. VA began national deployment of VAR in January 2017 and began implementing OPSS in December 2017. As prescribed for in the act, MITRE performed IV&V evaluations on both systems and GAO subsequently reviewed these assessments.  

GAO found that MITRE’s IV&V of the scheduling systems did include evaluations of the seven core requirements specified in the Faster Care for Veterans Act.  GAO also concluded that that MITRE’s IV&V work was consistent with the IEEE standards for conducting verification and validation activities. Specifically, it met the required IEEE standards for verification and validation for systems, software and hardware.

GAO was asked to evaluate the IV&V of the systems and gave its approval for that work with no recommendations to VA.  However, this is not to say that VA’s scheduling systems actually met all of the core requirements of the legislation. MITRE’s evaluation indicated deficiencies of the scheduling tools in several areas. 

The only details available regarding MITRE’s report are found in GAO’s evaluation of their IV&V activities on the two VA systems. GAO provides a table of the seven core system requirements and MITRE’s IV&V findings for each system.  MITRE’s evaluation indicates that the VAR system met five requirements, but only partially met two, and the OPSS system only fully met one requirement.

Although GAO’s review of MITRE’s work was positive, MITRE’s evaluation indicates that VA still has work to do to comply with the self-scheduling tool requirements of the Faster Care for Veterans Act.