Description

The American Registry of Radiologic Technologists (ARRT) seeks a Manager of Psychometrics to lead the exceptional team responsible for the psychometric integrity of ARRT's certification programs.  You'll drive innovative and strategic initiatives for:

•         systematic evaluation of modern testing methodologies and innovative item types;
•         contribution to increased understanding through psychometric research;
•         and dissemination of information through publication in professional journals and presentations at professional                  conferences

The Manager- Psychometrics works with the Assessments Department to maintain the psychometric integrity of ARRT’s certification programs. The Manager- Psychometrics also drives the ARRT’s strategic priorities for structuring the ARRT certification program to the needs of the future; advocating high-quality care through ARRT credentials; increasing participation and partnerships with ARRT; promoting innovation and foresight; committing to operational excellence. Responsibilities listed include those of the Psychometrics area.  No distinction is made between activities directly performed and those delegated since the Manager – Psychometrics is ultimately responsible for their performance.  This includes responsibility for the quality of the processes as well as the quality of products.

Duties and Responsibilities:
Psychometric Support
   Provide leadership, staff consultation, technical support, and project management for initiatives including, yet not limited to:
   Alternative assessment formats such as: Situational Judgment Tests (SJTs), select multiples, including guidelines and training materials for item writers.
   Proposal development for collaborative areas of certification programs such as the Society for Imaging Informatics in Medicine (SIIM).
   Assist other staff with efforts to investigate issues related to continued competence and maintenance of certification.
   Develop procedures for conducting routing post-equating studies and reports to monitor adequacy or pre-equating and scaling procedures.
   Consult with staff on standard setting, job analysis, survey development and other psychometric issues.
   Analyze data and write reports for Board of Trustees (BoT) and/or other senior staff.
   Create and respond to National Commission for Certifying Agencies (NCCA) accreditation inquiries. 

General Supervisory Responsibilities
   Monitor work responsibilities of staff. Ensure coverage for departmental responsibilities.
   Oversee staff schedules, approve leave, review, and finalize pay period timesheets.
   Develop and coach staff through one-on-one meetings and additional methods, respond to staff needs and suggestions, and provide ongoing feedback.
   Orient, onboard, and train new staff.  Assess the need for, and provide or coordinate, any ongoing training.
   Enforce staff policies as set in ARRT Employee Handbook.
   Manage staff performance, correlate job descriptions with performance management, participate in talent acquisition process, and participate in salary review sessions. 
   Work with staff to develop an effective teamwork environment.
   Maintain and elevate working relationships with other ARRT departments. Mentor the team on effective ways to communicate and work as a group.
   Participate in ARRT projects as assigned.
   Participate in professional development activities.
   Perform other duties as required and modified as business needs arise.

Community Relations
   Participate in activities that promote ARRT’s good will and commitment to the professional community.
   Present papers and conduct workshops at conferences related to psychometrics and/or medical imaging.
   Publish research-based articles.
   Attend occasional state or inter-organizational meetings.
   Function as liaison with the University of Minnesota Psychometrics and Educational Measurement programs to collaborate on applied research projects, as well as maintaining and strengthening ties between the ARRT and the University faculty and their graduate students.

Psychometric Work Tasks
   Support of Practice Analysis Activities
   Work with Assessment Development Coordinators on periodic practice analysis projects for assigned examination programs, including support of committee (sample) selection, survey development, survey data analysis and report writing.
   Present results of survey to practice analysis committees and answer any questions or concerns. Respond to internal requests for additional analyses of the survey data.
   Develop procedures for the standard presentation of practice analysis/job analysis survey data where possible and incorporate documentation on those processes into the Assessment Development Manual.
   Provide training and/or provide guidance and oversight to psychometric staff in presenting and interpreting results of practice analysis survey data for committees.
   Examination Committees
   Determine psychometric quality assurance criteria for item statistics, exam statistics, and examinee content review in conjunction within the psychometric team and ensure those criteria are met.
   Review exams and projected exam performance characteristics (including projected cut score) prior to the examination committee meeting.
   Examination Scoring, Analysis, and Reporting
   Periodically review item statistics, exam results, and examinee comments for assigned programs to assure accuracy of keys, passing scores, and exam content.
   Serve as back-up for exam results processing, scoring, and Quality Experience (QX) for all programs.
   Assist with the maintenance of the procedures manual and other documentation required for exam processing and scoring.
   Analyze exam results and collaborate with department staff in production and dissemination of annual examination summary reports and technical reports.
   Calibrate items and calculate cut scores and scaling equations for exam forms as needed; monitor pass rates, cut scores, and other exam statistics over time.
   Respond to letters and calls regarding score reporting.
   Recommend enhancements to scoring procedures and reports.
   Conduct standard setting studies.
   Analyze data and prepare reports.
   Train new psychometric staff in conducting standard setting studies and provide oversight and monitoring of any standard setting work delegated to them once they are sufficiently prepared to undertake such tasks.

Case Study and Associated Items
   Determine psychometric quality assurance criteria for developing and scoring the case study questions and assure these criteria are met.
   Prepare materials to systematize the development of case studies, including instructional materials for authors and reviewers.
   Establish mechanisms for scoring case studies, inclusive quality control procedures and policies for rescoring case studies; investigate feasibility of automated (computerized) scoring.
   Evaluate and adopt methods for setting performance standards, scoring, and scaling responses to case study questions, and for determining the reliability of case study scores and total scaled scores.
   Evaluate and adopt statistical models for monitoring the psychometric properties of cases over time; develop specifications for incorporating statistical information into the item bank or other database.
   Determine long-term needs of the item bank and work with IT staff on how we might achieve and prioritize those goals.

Develop and Maintain Psychometric Exam Security Program
   Design exam security programs in concert with psychometric research findings.
   Set criteria for flagging suspected security breaches.
   Design procedure to systematically psychometrically assess security risks.
   Assign psychometric work task responsibilities to put the exam security procedure into practice.
   Escalate suspected security breaches to the CAO or CEO.
   Monitor outcomes from exam security programs in conjunction with other Assessments staff for potential security breaches and investigate them.

Miscellaneous
   Review and edit ARRT publications as requested, including website material.
   Analyze data and write reports for BoT and/or senior staff.
   Conduct applied psychometric research in support of ARRT's mission.
   Assist ARRT staff with programming and data analysis needs.
   Respond to data analysis requests from the public, educators, or state agencies.
   Assist with specialized analysis requests.
   Keep CAO informed of departmental activities via regular meetings or immediately in case of serious or potentially serious situations.

Requirements

Minimum Qualifications:
   Doctorate in Psychometrics, Quantitative Methods in Psychology, Educational Measurement, or related field
   5+ years of applied and research psychometrics
   Supervisory experience

Preferred Qualifications:
   ICE-CCP certification
   Experience with NCCA-accredited programs   

 Skills and Abilities:
   Comprehensive knowledge of certification processes and their business impact
   Strong understanding of non-profit organization management (preferred)
   Ability to organize, prioritize, and delegate tasks effectively.
   Effective and professional communication, interpersonal, and presentation skills including public speaking, along with the ability to adapt style and approach to various levels and audiences.  
   Excellent analytical and problem-solving skills, with strong attention to detail.
   Data collection and analysis skills, both qualitative and quantitative, to make informed decisions.
   Ability to gather data, compile information, prepare reports, and manage databases.
   Ability to leverage Artificial Intelligence (AI) psychometric tools and techniques.
   Skilled in working both independently and collaboratively.
   Ability to analyze complex business processes, identify improvement opportunities, and develop innovative solutions to enhance operational efficiency.
   Strong application of ethics and confidentiality. 
   Proficiency across various software platforms and technologies including the Microsoft Office Suite, learning management systems, video conferencing and presenting platforms, SPSS, IRT Calibrates, R Studio, and Item Analysis software applications.

 Physical Demands:
Close Visual Acuity: Preparing and analyzing data; viewing a computer screen; and extensive reading
Fingering: Typing, or otherwise working, primarily with fingers
Hearing: Ability to receive detailed information through oral communication, and to make fine discriminations in sound.
Repetitive Motions: Substantial movements (motions) of the wrists, hands, and/or fingers
Sedentary Work: Sitting most of the time; occasional walking and standing 
Talking: Spoken word activities can be relayed accurately, loudly, or quickly
Travel: Ability to travel based on business needs.

The American Registry of Radiologic Technologists (ARRT) is a leading credentialing organization. ARRT offers certification and registration in a wide range of radiologic disciplines, helping people who work in medical imaging and radiation therapy to develop their careers. With nearly 360,000 registrants, we promote high standards of patient care with each new credential we award. Visit www.arrt.org for more information and benefits overview.

 Review a full job description, which includes additional information on duties and responsibilities, skills and abilities, core competencies, and physical demands. This fulltime position is located onsite at our office in St. Paul, Minnesota with potential for hybrid onsite/telework -approximately 50/50. 

 Salary Description: $119,673 - $140,791 per year

 Our Benefits: We offer competitive wages, and a variety of excellent benefits, including health insurance, retirement contributions and paid time off.  Jobs at ARRT - ARRT

 Please contact your recruiter or email human-resources@arrt.org with any questions or requests for reasonable accommodations during the interview process.