Virginia Inter-Rater Reliability (IRR) Program

Tools and Resources

Virginia Medicaid – Department of Medical Assistance Services (DMAS) contracts with Maximus to develop a baseline reliability study on its Universal Assessment Instrument. This study will demonstrate the degree of consistency and reliability of long-term care screenings for children and adults, verifying that the tools and training are effectively and consistently applied for the benefit of individuals across the Commonwealth, regardless of region or screener. This is a study only, and outcomes for individuals will not change. 

The Inter-rater Reliability (IRR) Study seeks to demonstrate the degree of consistency and reliability of long-term care assessments for children and adults, as well as verification that individuals have the same opportunity to obtain services regardless of where they live or when a screen is conducted and ensure that screeners are identifying the most integrated and appropriate settings, minimizing unnecessary institutional placements.

Stay connected to important program updates and helpful program resources. If you aren't currently on the Virginia IRR contact list, please reach out to the Help Desk with your name, title/role and facility name: VAIRR@maximus.com

Contact the help desk
Phone & Email

833.73 VAIRR (833.738.2477)

VAIRR@maximus.com

Business Hours

8:00 am - 5:00 pm EST

Announcements

  • Virginia Inter-rater Reliability (IRR) Study: DMAS Study Officially Launched on January 3 | Register for Upcoming Webinars | Review Supports & Reminders

    Monday, January 3 marked the official launch of the Inter-rater reliability (IRR) study through the Virginia Department of Medical Assistance (DMAS) and contractor, Maximus. As shared in previous communications, this IRR study was mandated by the General Assembly as a review of Virginia’s LTSS screening process with several key goals in mind, including:

    • Validation of the children's criteria used with the UAI instrument to determine eligibility for Medicaid Long Term Services and Supports (LTSS) 
    • Design and implementation of an inter-rater reliability test for the pre-admission screening process


    Continue reading below for insights into some upcoming learning opportunities, frequently asked questions (FAQs), available resources and Help Desk support pertaining to the IRR study. 

     

    Timeline-VA-IRR-2.png

     


    REGISTER: IRR Overview Webinars | Jan 26 - 27

    Would you like to learn more about this important new study? Be sure to join the program team at Maximus for one of these hour-long overview sessions, as they walk through the key aspects of the IRR study. There will also be a few minutes set aside to answer attendee questions. Click the link below and select the date/time that work best with your schedule.

     



    FAST FACTS: Understanding the IRR Study Process

    In the coming months we'll continue to share some frequently asked questions about the IRR study and your role in the process. Below are four fast facts focused on how this new initiative will work.

     

    1. How will the study be conducted? Individuals will be randomly selected across regions, to include both adults and children.  
       
    2. How will rescreens be selected? Rescreens will be randomly chosen from a statistically significant sample of conducted screens and analyzed based on assessment type (child vs. adult), region, and screening entity (NF, hospitals and CBTs).   
       
    3. Who will perform the secondary screening? All screeners working with the IRR study will have taken and passed the VA LTSS modules.   
       
    4. How will the rescreening process work? Maximus will conduct face-to-face screenings for the Baseline IRR Study using two techniques:
    • Screen/Rescreen—Entails repeating a subset of required questions within a short time frame after the initial screen. This will begin in January 2022 
       
    • Observational IRR will include silently observing a live screen with the initial screener. This is expected to begin mid-year 2022.

     

    TOOLKIT: Screener Resources

    Ensuring that you have access to all the necessary tools and supports is critical to help ensure the successful implementation and the findings that will be gathered as a result of the IRR study. Check out a few helpful guides below that have been developed with Virginia’s Screeners in mind. 

    Tool: Screener Leave-Behind | Download
    This handy printout (2 copies per page) provides the persons you assess with some brief information about the IRR study and lets them know that a follow up screening may potentially be scheduled, as well as offering options for them to learn more.

    Guide: Virginia IRR Study Overview | Download
    A one-page overview including the purpose, processes and planned goals for the study.

    Guide: Frequently Asked Questions (FAQs) | Download
    This resource guide covers the questions most often fielded about the IRR study and its related processes. This will continue to be updated as we move through the year.



    CONTACT: IRR Study Support

    Have questions about the IRR study and can't find the answer in the resources listed above? Contact the Help Desk team for more information:

    Email: VAIRR@maximus.com

    Phone: 833.73. VA IRR (833.738.2477)

  • New DMAS Program Implementation with State Partner, Maximus, Coming January 3

    The Virginia Department of Medical Assistance (DMAS) has contracted with clinical services company, Maximus, to launch an Inter-rater reliability (IRR) study beginning on Monday, January 3rd
     
    The General Assembly has mandated this initiative as a review of Virginia’s LTSS screening process. Once implemented, this study will demonstrate the degree of consistency and reliability of long-term care screenings for children and adults, verifying that the tools and training are effectively and consistently applied for the benefit of individuals across the Commonwealth, regardless of region or screener type.  
     

    As the state prepares to implement this program, below are answers to some key questions about the IRR study, the project’s timeline, as well as available supports and resources you might find helpful. 
     

    What is IRR?  
    IRR, or Inter-rater reliability, refers to the extent to which two or more individuals agree. The process will be used to evaluate the consistency of processes, rating decisions and outcomes. The study will validate the children’s criteria and determine the consistency of screening across populations. 
     

    How will the study be conducted?  
    Individuals will be randomly selected across children, adults, regions. 
     

    How will rescreens be selected?  
    Rescreens will be randomly chosen from a statistically significant sample of conducted screens and analyzed based on assessment type (child vs. adult), region, and screening entity (NF, hospitals and CBTs).  
     

    How will the rescreening process work? 
    Maximus will conduct face-to-face screenings for the Baseline IRR Study using two techniques: 

    1. Screen/Rescreen—Entails repeating a subset of required questions within a short time frame after the initial screen. This will begin in January 2022 
    2. Observational IRR will include silently observing a live screen with the initial screener. This is expected to begin mid-year 2022.  


    Who will perform the secondary screening? 
    All screeners working with the IRR study meet or exceed requirements and will have taken and passed the VA LTSS modules. 
     

    How many individual screenings are estimated to go through a secondary screening? 
    Out of approximately 40,000 LTSS screenings a year Maximus anticipates sampling just over 3% 
     

    Will the secondary screenings affect findings from the initial screen? 
    No. DMAS and Maximus recognize the importance of minimizing disruptions to normal processes. This is a study only, and outcomes for individuals will not change. No part of this study will negatively impact a completed screening and no changes will be made to any previously completed screenings, outcomes or services received. 
     

    What are key dates ahead for this initiative? 
    There are a few milestone dates to keep in mind for the IRR study including: 

    • IRR activities (screen/rescreen) will begin on January 3, 2022 
    • Observational IRR-related activities will take place mid-late 2022 
    • Baseline study recommendations due in the fall of 2022 


    How will the study results be used? 
    At the close of the IRR study, the results of this instrument-focused initiative will serve as a foundation for several important next steps:

    • Ensure reliability across different settings 
    • Assure confidence in the LTSS screening process  
    • Inform best practices ad develop training materials (if required) 
       

    What will Maximus need from screeners?
    Maximus may contact you to request certain information and copies of the supporting documentation you used during the original screen.
     

    Are we to add Maximus to our consent to exchange information, or are they covered under DMAS?  
    Maximus is included in Business Associate Agreement (BAA) for DMAS. 
     

    Should initial screenings be faxed to Maximus?
    No, the initial screening should not be faxed or emailed to Maximus.
     


    Program Support

    Ensuring that stakeholders across the Commonwealth have access to the information they need about the IRR study will remain a critical goal throughout this process. The Help Desk contacts and supports shown below are now available for any questions regarding IRR study processes, next steps and expectations. Communication updates and will be shared by the state and Maximus as the study moves forward in the months ahead. 

     
    Help Desk Contacts 
    VAIRR@maximus.com | 833.73 VAIRR (833.738.2477) 


    Virginia IRR Program Overview
    Download a one-page overview here to learn more.
     

    About Maximus

    Maximus is nationally recognized for its work with states across the country to manage complex programs and ensure that individuals receive the appropriate care and supports necessary to improve their quality of life. Enduring commitment to process innovation and service excellence has led the company to a role as the industry’s leading provider of high-quality, conflict free assessment services for persons with intellectual, behavioral health, and developmental disabilities, as well as older adults and individuals with physical disabilities and complex medical conditions. Learn more about Maximus services and capabilities at: https://maximus.com/clinical-services