Criterion-Referenced Exit Examinations: An Institution’s Internal Process for Psychometric Analysis

Authors

  • Cristian Lieneck Texas State University-San Marcos
  • Eileen Morrison Texas State University-San Marcos
  • Larry Price Texas State University-San Marcos

Keywords:

healthcare administration, alternate form reliability, comprehensive exit examination, test psychometric analysis.

Abstract

The Texas State University-San Marcos undergraduate healthcare administration program requires all bachelors of health administration (BHA) students to pass a comprehensive examination to demonstrate their knowledge of specific core competencies. This also demonstrates completion of their didactic coursework in order to enter a practical internship or residency experience. Since this examination provided important documentation of student learning, the program conducted a detailed psychometric analysis of its three most recent undergraduate comprehensive exit examinations.  In order to determine the value of this examination psychometrically, an evaluation of item validity evidence, between-exam reliability, and related assessment of descriptive statistics with regard to overall exam results and individual healthcare administration competency outcomes was necessary. Using Classical Test Theory (CTT) as a methodological framework, the psychometric analysis involved calculating item-level indices that assessed descriptive, validity, difficulty, and discrimination characteristics. This allowed the program’s faculty to better interpret student exam outcomes at the overall exam and within-exam competency levels. Additionally, this analysis provided an evaluation of the score reliability of the three alternate exam forms, as well as within-exam healthcare administration competency items, furthering the program’s comprehensive exit exam test development process. The outcomes of the analysis included an increased awareness of potential non-equivalent test forms for the total exam and within each exam (competency) level, increased level of interpretation the descriptive results for each exam, and the establishment of a more robust test development process to guide future comprehensive examination efforts.

Author Biographies

Cristian Lieneck, Texas State University-San Marcos

Dr. Lieneck has over 10 years of experience in the field of healthcare administration with a focus in medical group practice management. Initially entering the field as a Medical Service Corps officer in the United States Army, Dr. Lieneck led a medical clinic and evacuation platoon in an armored cavalry unit. As a Captain, he served as an Executive Officer and Company Commander, managing four dental practices and one hospital-based oral surgery clinic. As a civilian, Dr. Lieneck held positions of leadership in organizations such as the Texas Medicaid-Waiver Program (Texas DADS), Austin Radiological Association (ARA), as well as serving as the medical group practice administrator of an Austin-based physiatry and pain medicine group practice, consisting of multiple locations and an outpatient procedure suite in central Texas.  Dr. Lieneck is a fellow and board certified in medical group practice management by the American College of Medical Practice Executives (ACMPE), the credentialing body of the Medical Group Management Association (MGMA). He is also a fellow and board certified in healthcare management by the American College of Healthcare Executives (ACHE), and a fellow in the Academy of Healthcare Management (AHM).

Eileen Morrison, Texas State University-San Marcos

Dr. Morrison has her EdD in Counseling & Administration from Vanderbilt University and her MPH from the University of Tennessee. Prior to joining the Texas State University faculty 2004, Dr. Morrison had extensive experience in curriculum design, course development, and teaching in graduate programs in health administration. She also holds several licenses that include dental hygiene, Licensed Professional Counselor, and Certified Health Education Specialist. Her health administration experience involves positions in corporate and public health. Dr Morrison has served as a consultant to the U.S. Department of Justice, the United States Public Health Service, Head Start, and the United States Army. She also writes in the field of health administration and education and including two textbooks on ethics published by Jones and Bartlett. A new book on health care ethics issues is under contract.

Larry Price, Texas State University-San Marcos

Dr. Larry Price is Professor of Psychometrics & Statistics and Director of the Interdisciplinary Initiative for Research Design and Analysis at Texas State University. Prior coming to Texas State University, he served as a Senior Psychometrician and Statistician for the Emory University School Medicine, Departments of Psychiatry & Behavioral Sciences. Dr. Price was employed at The Psychological Corporation in San Antonio as a Senior Psychometrician during 1999-2002. He is a Fellow of the American Psychological Association, Division 5 – Evaluation, Measurement & Statistics, and a member of the American Statistical Association, the Psychometric Society, the American Educational Research Association and the American Mathematical Society. He has published extensively with over 90 peer-reviewed articles, presentations, books and book chapters in journals and books such as: Structural Equation Modeling, Journal of Educational & Behavioral Statistics, Psychological Assessment, Psychological Methods, Elementary School Journal, Journal of Experimental Education, and The Journal of Clinical and Experimental Neuropsychology and the Oxford Handbook of Quantitative Methods.

Downloads

Additional Files

Published

2013-08-11

How to Cite

Lieneck, C., Morrison, E., & Price, L. (2013). Criterion-Referenced Exit Examinations: An Institution’s Internal Process for Psychometric Analysis. Current Issues in Education, 16(2). Retrieved from https://cie.asu.edu/ojs/index.php/cieatasu/article/view/1052

Issue

Vol. 16 No. 2 (2013)

Section

Articles

License

Authors retain copyright without restrictions. Unless otherwise indicated, from 2021 all articles are published under the Creative Commons CC-BY-SA license. For more information visit https://creativecommons.org/licenses/by-sa/4.0/. Articles published prior to 2021 used a CC-BY-NC-SA license.