About the Company

The client is a national not-for-profit certification and assessment organization serving the healthcare professions. The organization is responsible for high-stakes national certification examinations and is focused on maintaining rigorous standards for assessment quality, validity, reliability, fairness, and defensibility.

The organization's identity is confidential and will be disclosed to candidates selected for further discussion.

The Role

Reporting to the Chief Executive Officer and serving on the Senior Management Team, the Interim Vice-President, Psychometrics and Data Analytics will provide strategic and technical leadership across psychometrics, data analytics, business intelligence, data governance, and enterprise data strategy.

The role will ensure the quality and defensibility of high-stakes assessments while positioning organizational data as a strategic asset for decision-making. The VP will also lead the Psychometrics and Data Analytics department and work closely with executive leadership, technology, operations, and the Board.

Key Responsibilities

Psychometrics & Assessment Quality

• Oversee psychometric quality across all assessments, including validity, reliability, fairness, and defensibility.
• Provide oversight across test design and administration support, standard setting, equating, scaling, scoring, and results reporting.
• Ensure consistent application of psychometric methodologies in accordance with professional standards and best practices.
• Direct statistical and forensic analyses of examination outcomes, candidate performance, and testing incidents, including remote proctoring.
• Advise on psychometric and standard-setting decisions with high-stakes or policy-level implications.
• Provide statistical and forensic expertise related to testing irregularities.

Data Strategy & Governance

• Lead the development and implementation of an enterprise data governance framework.
• Establish standards for data quality, integrity, security, privacy, retention, definitions, and stewardship.
• Develop supporting capabilities including metadata management, data cataloguing, and master and reference data management.
• Chair the data governance operating model, including named data owners and stewards across functions.
• Develop and maintain the multi-year data strategy and analytics roadmap in alignment with the organization's strategic plan.

Analytics, AI & Business Intelligence

• Lead enterprise data analytics and business intelligence initiatives.
• Govern analytical and AI/machine-learning models used in or affecting assessments.
• Oversee model validation, bias and fairness testing, performance monitoring, drift detection, and documentation.
• Own the reporting, dashboard, and data-visualization environment.
• Develop self-service business intelligence capabilities for senior leadership, the Board, and stakeholders.
• Establish a governed single source of truth and semantic layer.
• Advance analytics from descriptive and diagnostic reporting toward predictive and prescriptive capabilities.
• Champion the use of data as a strategic organizational asset.

Executive & Board Leadership

• Partner with executive leaders across Assessments, Technology, and Operations.
• Present psychometric results, validity evidence, and recommendations to the Board's examinations committee.
• Translate complex psychometric and data concepts for non-technical audiences.
• Provide strategic guidance on high-impact assessment, data, and analytics matters.
• Collaborate with technology leadership, which owns the underlying data platform and engineering.

Team & Department Leadership

• Lead the Psychometrics and Data Analytics department.
• Manage recruitment, training, retention, and performance management.
• Establish annual departmental priorities, projects, and budgets.
• Inspire and develop technical teams while maintaining high professional and scientific standards.

Qualifications

• PhD required, ideally in psychometrics, educational measurement, statistics, or a closely related quantitative discipline.
• Significant progressive senior leadership and management experience, including leadership of technical teams or functions.
• Typically 8+ years of experience as a psychometrician in educational or psychological testing, including high-stakes assessment.
• Strong expertise in psychometric methodologies, including standard setting, equating, and scaling.
• Experience leading or overseeing enterprise data strategy, data governance, analytics, and business intelligence/reporting.
• Proven ability to design, conduct, interpret, and present psychometric analyses and studies in high-stakes assessment environments.
• Strong data analysis and scientific-computing capabilities.
• Familiarity with business intelligence and data-visualization tools.
• Experience using data science, machine learning, or emerging AI tools for measurement and analytics.
• Strong strategic leadership, analytical judgment, and communication skills.
• Ability to operate effectively in a high-stakes assessment environment.

Preferred Qualifications

• Experience with predictive and prescriptive analytics.
• Experience with AI/ML model governance.
• Professional certification and licensure experience.
• Proficiency in French.

Ideal Candidate

The ideal candidate is a seasoned technical and strategic leader who has successfully redefined the direction of a technical function rather than simply managed it.

They will bring a strong commitment to assessment quality, fairness, and defensibility, combined with sound judgment on complex psychometric and data issues. The successful candidate will be able to balance strategic leadership with technical rigor, maintain high standards for data integrity, security, and privacy, and communicate complex concepts clearly to non-technical audiences.

Work Arrangement

• Fully remote.
• Candidates must be located in Canada or the United States.
• Occasional travel may be required.

Application Process

This is an interim executive engagement, with the anticipated term to be confirmed with shortlisted candidates.

Interested candidates should submit a cover letter and résumé in confidence to Vanessa at Peak Performance HR at vanessa@peakperformancehr.com, referencing “Interim VP, Psychometrics and Data Analytics” in the subject line.

The client organization's identity will be disclosed only to candidates advancing in the process.

The client is committed to an inclusive, barrier-free recruitment process, with accommodations available upon request.

Note: This role is a replacement, and AI will not be used during the recruitment process.

‍