Assessing reliability and validity of the outcome and assessment information set: Implications for health policy, practice and research

The Outcome and Assessment Information Set (OASIS), the Medicare-mandated home health assessment, is used to establish agency reimbursements, evaluate home health quality and for research. Thus, it is critical to ensure the validity and reliability of the items within the OASIS tool. A comprehensive, systematic review of the empirical literature testing either validity or reliability of the OASIS was conducted, yielding twelve studies between 1999 and 2010. Data were extracted in a three-step process using a structured narrative template that reviewed the type of validity/reliability tested, methods/design, sampling procedures, measures, significant findings and limitations of each study. Methodologies employed in the reviewed studies included secondary analysis, video simulation, retrospective and prospective cohort designs, some with random but most with convenience samples. Seven studies analyzed validity; both construct and criterion-related validity were studied as well as unidimensionality of certain domains. Validity was evaluated by comparing various OASIS measures to existing “gold-standard” scales, expert-derived correct answers, or hospital admission rates. Some studies used home care staff, while others used research personnel to complete assessments. Nine studies evaluated reliability, including intra-rater, inter-rater, and internal consistency reliability. Empirical findings indicate the validity and reliability of the OASIS varies depending upon the item studied. Differing methodologies and statistical measures make comparison between studies difficult. Further research is needed to investigate the psychometric properties of the OASIS to confirm its value in measuring patient outcomes for research, quality improvement and policy. Researchers must use caution in studying home health outcomes via the OASIS data.