Test Development Specialist
THE AMERICAN BOARD of PEDIATRICS
Chapel Hill, NC
Full-time
Administration / Clerical
Posted on May 13, 2022
- Manage test development tasks for a variety of medical examinations, assessments, and supporting material, ensuring accuracy, consistency, integrity, and quality throughout all phases of development
- Manage item banks for ABP examinations, performing gap analyses as needed to ensure coverage for all domains in the examination blueprints
- Consult with psychometricians to set exam selection parameters and determine linkages to previous examinations as needed; facilitate key validation meetings; assist psychometricians with scoring related tasks as needed (eg, standard setting, job task analysis)
- Select items and build forms for in-training, initial certification, continuing certification examinations and longitudinal assessments; verify that all examinations conform to blueprints and specifications
- Maintain item banks; applying standardized and systematic processes
- Build exam delivery package and ensure successful exam transfer and publication by the vendor
- Initiate and lead process improvement activities
- Guide the ABP sub boards and examination committees through all phases of examination development and production processes
- Develop training materials and train subject-matter experts (SMEs)
- Develop question writing and review assignments based on blueprints and gap analyses
- Lead the process for reviewing and approving examination forms
- Ensure quality by providing test development expertise and knowledge of best practices to SMEs and staff
- Serve as program manager for multiple assigned examination programs
- Manage the multiple and overlapping requirements of examination development, and other ongoing external and internal projects, to maintain an orderly and efficient workflow, while ensuring that deadlines are met
- Develop, update, and coordinate assignments and project schedules with department colleagues and SMEs to ensure a successful outcome
- Plan, coordinate, and facilitate in-person and virtual meetings with SMEs
- Oversee creation and maintenance of standard policy and procedure documents
- Collaborate with the Director of Test Development in the design, critique, and implementation of test development processes
- Analyze, document, and resolve internal issues as they emerge
- Provide technical support for internal staff and SMEs; represent ABP with vendors to document and resolve issues
- Analyze needs, explore solutions and alternative applications or platforms
- Develop, revise, and maintain test development reports for both internal and external audiences
- Participate in testing research as needed
- Perform testing on application updates and patches as needed
- Implement the strategic initiatives of the ABP related to testing
- Build and maintain strong relationships with committee members and ABP staff to achieve required
- Represent the ABP when making presentations to stakeholders and other professional audiences (eg, ATP, ABMS, ICE)
- Assist the Director of Test Development with other duties as assigned
Knowledge, Skills, and Abilities
- Expertise in question writing and examination construction; understanding of measurement principles
- Strong program management skills
- Ability to collect and analyze data using question banking application to inform decisions about question writing and exam selection
- Strong program/project management skills
- Advanced knowledge of test development methodology
- Ability to learn medical terminology
- Strong Excel skills including the ability to use pivot tables and a variety of formulae
- Ability to synthesize complex medical information and apply edits during live or virtual meetings
- Excellent communication skills: ability to write and speak clearly and professionally to a variety of audiences
- Excellent organizational skills with demonstrated ability to manage complex processes and multiple, concurrent, and overlapping projects while meeting deadlines
- Ability to collaborate with a team in pursuit of quality and continuous improvement
- Ability to use (or learn to use) web-based item-banking applications (e.g., ExamDeveloper), test delivery platforms, project management software (e.g., Wrike) Microsoft Office 365 (e.g., Excel, PowerPoint, SharePoint), Adobe Photoshop
Special Requirements
- Must be available to attend meetings outside of regular office hours
- Must be available to provide email coverage, as needed, outside of regular office hours, including evenings and weekends, to respond to SMEs regarding item development and exam review
- Provide technical support for item banking system outside of regular office hours, as needed