Manager of Psychometrics

The American Registry of Radiologic Technologists (ARRT) seeks a Manager of Psychometrics to lead the exceptional team responsible for the psychometric integrity of ARRT's certification programs. This fulltime position is located onsite at our office in St. Paul, Minnesota with potential for hybrid onsite/telework. You'll drive innovative and strategic initiatives for:

• systematic valuation of modern testing methodologies and innovative item types;
• contribution to increased understanding through psychometric research;
• and dissemination of information through publication in professional journals and presentations at professional conferences.

The Manager- Psychometrics works to maintain the psychometric integrity of ARRT’s certification programs and drives the strategic priorities for structuring the ARRT certification programs to the future. They will advocate high-quality care through ARRT credentials; increase participation and partnerships with ARRT; promote innovation and foresight; and commit to operational excellence.

Duties and Responsibilities: Psychometric Support

Provide leadership, staff consultation, technical support, and project management for initiatives such as alternative assessment formats. Create guidelines and training materials for item writers. • Develop proposals for collaborative areas of certification programs. • Investigate competence and maintenance of certification issues. • Develop procedures for conducting routing post-equating studies. Create reports to monitor adequacy or pre-equating and scaling procedures. • Consult with standard setting, job analysis, survey development and other psychometric issues. • Analyze data and write reports. • Create and respond to accreditation-related inquiries.

General Supervisory Responsibilities as listed in the full job description.

Community Relations • Promote ARRT’s good will and commitment to the professional community. • Present papers, conduct workshops related to psychometrics and/or medical imaging. • Publish research-based articles. • Attend occasional state or inter-organizational meetings. • Function as liaison with the University of Minnesota Psychometrics and Educational Measurement programs, collaborate on research projects, maintain and strengthen ties between the ARRT and the University faculty and graduate students.

Psychometric Work Tasks • Support the Practice Analysis Activities and work with the Assessment team on periodic practice analysis project, support of committee selection, survey development, survey data analysis and report writing. • Present results of survey to practice analysis committees and answer any questions or concerns. • Develop standard presentation of practice analysis/job analysis survey data and incorporate documentation into manual. • Provide training, guidance, and oversight to psychometric staff in presenting and interpreting results of practice analysis survey data for committees.

Examination Committee Support• Determine psychometric quality assurance criteria for item statistics, exam statistics, and examinee content review. • Review exams and projected exam performance characteristics (including projected cut score) prior to the examination committee meeting. • Examination Scoring, Analysis, and Reporting •Review item statistics, exam results, and examinee comments for assigned programs to assure accuracy of keys, passing scores, and exam content. • Serve as back-up for exam results processing, scoring, and Quality Experience (QX) for all programs. • Maintain documentation required for exam processing and scoring. • Analyze exam results, collaborate with staff in production and dissemination of annual examination summary reports and technical reports. • Calibrate items and calculate cut scores and scaling equations for exam forms as needed; monitor pass rates, cut scores, and other exam statistics over time. • Respond to letters and calls regarding score reporting. • Recommend enhancements to scoring procedures and reports. • Conduct standard setting studies. • Analyze data and prepare reports.

Case Study and Associated Items • Determine psychometric quality assurance criteria for developing and scoring the case study questions and assure these criteria are met. • Prepare materials to systematize the development of case studies. • Establish mechanisms for scoring case studies, quality control procedures and policies for rescoring case studies; investigate automated (computerized) scoring. • Evaluate and adopt methods for setting performance standards, scoring, and scaling responses to case study questions, and for determining the reliability of case study scores and total scaled scores. • Evaluate and adopt statistical models for monitoring the psychometric properties of cases over time; incorporate statistical information into the item bank or other database.

Develop and Maintain Psychometric Exam Security Program • Design exam security programs in concert with psychometric research findings. • Set criteria for flagging suspected security breaches. • Design procedure to systematically psychometrically assess security risks. • Assign psychometric work task responsibilities to put the exam security procedure into practice. • Escalate suspected security breaches. • Monitor outcomes from exam security programs for potential security breaches and investigate them.

Miscellaneous • As listed in the full job description.

Minimum Qualifications: • Doctorate in Psychometrics, Quantitative Methods in Psychology, Educational Measurement, or related field • 5+ years of applied and research psychometrics • Supervisory experience Preferred Qualifications: • ICE-CCP certification • Experience with NCCA-accredited programs

Skills and Abilities as listed in full job description • Comprehensive knowledge of certification processes and their business impact • Strong understanding of non-profit organization management (preferred) • Excellent analytical and problem-solving skills, with strong attention to detail. • Data collection and analysis skills, both qualitative and quantitative, to make informed decisions. • Ability to gather data, compile information, prepare reports, and manage databases. • Ability to leverage Artificial Intelligence (AI) psychometric tools and techniques. • Skilled in working both independently and collaboratively. • Strong application of ethics and confidentiality. • Proficiency across various software platforms and technologies including the Microsoft Office Suite, learning management systems, video conferencing and presenting platforms, SPSS, IRT Calibrates, R Studio, and Item Analysis software applications.

Visit www.arrt.org for more detailed information, benefits overview, and to review a full job description, including physical demands. This fulltime position is located onsite at our office in St. Paul, Minnesota with potential for hybrid onsite/telework. Relocation assistance may be available. Salary Description: $119,673 - $140,791 per year. Please contact your recruiter or email human-resources@arrt.org with any questions or requests for reasonable accommodations during the interview process.