Skip to main content
Unlocking the Potential of Assessments

Unlocking the Potential of Assessments

By Questionmark

This monthly series delves into creating, delivering and reporting on valid and reliable assessments. "Unlocking the Potential of Assessments" offers advice and thought leadership to those just starting out with assessments, those who have been in the industry at length and anyone with a keen interest in the future of assessments lead by host, John Kleeman.

Join Questionmark on our quest to discover and examine the latest in best practice guidance with a wide array of guests - including assessment luminaries, industry influencers, SMEs and customers - and discuss "all things assessment."
Available on
Apple Podcasts Logo
Google Podcasts Logo
Overcast Logo
Pocket Casts Logo
RadioPublic Logo
Spotify Logo
Currently playing episode

Ep. 8 - Assessment Luminary, Jim Holm, Examity

Unlocking the Potential of AssessmentsJul 22, 2020

00:00
30:34
The changing face of fairness: context, inclusivity & AI

The changing face of fairness: context, inclusivity & AI

Join host John Kleeman in an insightful conversation with Dr. Ada Woo, VP of Innovative Learning Sciences at Ascend Learning and seasoned expert in assessment and psychometrics.

Dr. Woo shares her journey from aspiring clinical psychologist to becoming a leading figure in the assessment industry. They discuss various topics like the role of psychometrics, the importance of fair and reliable assessments, the impact of test anxiety, and the evolution of fairness in testing.

Dr. Woo also delves into the integration of AI in assessment and learning, emphasizing the need for adaptability and innovation in the field.

Tune in for a deep dive into the complexities and advancements in educational and professional assessments.

Feb 27, 202426:45
EdTech Revolution: Rethinking Learning Models with Carla Aerts

EdTech Revolution: Rethinking Learning Models with Carla Aerts

Join host John Kleeman in an insightful conversation with Carla Aerts, a trailblazer in the reinvention of education. Carla, with an unconventional journey from the performing arts to education technology, shares her accidental foray into the sector and delves into the importance of making impactful contributions to teaching and learning.

Carla explains what "perfect EdTech" looks like to her and critiques traditional exam-style assessments, advocating for a shift towards formative assessments and diverse models that prioritize skills, collaboration, and social engagement.

The discussion extends to the role of AI in education and Carla's predictions of how it will impact the future. The episode concludes with three key takeaways: the significance of social learning, the need for skills development in reflection and metacognitive thinking, and the promotion of learner agency for effective lifelong learning.

Tune in for a thought-provoking exploration of education, technology, and assessments, guided by Carla Aerts' forward-thinking perspectives.

Nov 29, 202330:06
Workday: Being Ready for What’s Next - Skills as Strategy
Oct 09, 202330:44
Ep. 39 – Assessment Luminary, John Weiner, Lifelong Learner Holdings
Aug 14, 202330:18
Ep. 38 – Assessment Luminary, Mark Lynch, Learnosity
Jul 19, 202327:11
Ep. 37 – Assessment Luminary, Matt Wingfield, e-Assessment Association
May 17, 202332:12
Ep. 36 - Assessment Luminary, Maria Hamdani, Center for Measurement Justice
Apr 26, 202330:16
Ep. 35 - Assessment Luminary, Norihisa Wada, EduLab

Ep. 35 - Assessment Luminary, Norihisa Wada, EduLab

John Kleeman is joined by assessment luminary, Norihisa Wada, executive adviser of EduLab in Japan, and director of the Japan Institute for Educational Measurement. Nori has experience with AI-driven voice recognition and has worked with Carnegie Mellon University and Kyoto University. He is also involved in EdTech investment. Delve into his insights on assessment and his background in the industry. Nori shares how he transitioned from his work on game development for consumer educational games and fitness games for Nintendo to the assessment industry, the future of assessments and gives advice on how assessment organizations can be more inclusive.

Mar 29, 202327:39
Ep. 34 - Assessment Luminary, Dr. Gary A. Gates, Pearson VUE

Ep. 34 - Assessment Luminary, Dr. Gary A. Gates, Pearson VUE

John Kleeman is joined by assessment luminary, Dr. Gary A. Gates, the Managing Director of Pearson VUE. Originally a nuclear physicist, Dr. Gates details his journey into the world of assessments. He discusses his key priorities going into his second year leading Pearson, the future of test centers during the rise of online proctoring and whether new AI technology is harmful or helpful to proctoring and assessments.

Feb 15, 202330:29
Ep. 33 - Assessment Luminary, Mike Olsen, Proctorio

Ep. 33 - Assessment Luminary, Mike Olsen, Proctorio

John Kleeman is joined by assessment luminary, Mike Olsen, the CEO and Founder of Proctorio. Mike discusses what drove him to create a company focused on proctoring, how it has expanded beyond the education sector, and how the tool can be used to address test fraud. 


Jan 25, 202335:02
Ep. 32 - Assessment Luminary, Phillip Dawson, Deakin University

Ep. 32 - Assessment Luminary, Phillip Dawson, Deakin University

John Kleeman is joined by assessment luminary, Professor Phillip Dawson, the Associate Director of the Centre for Research in Assessment and Digital Learning (CRADLE) at Australia's Deakin University. With an academic background in education, artificial intelligence and cybersecurity, he leads CRADLE's work on cheating, academic integrity and assessment security. This work spans hacking and cheating on online exams, training academics to detect contract cheating, student use of study drugs, the effectiveness of legislation in stopping cheating and the evaluation of new assessment security technologies. Phillip shares his thoughts on constructing an assessment that gives feedback, reducing test cheating and the privacy concerns associated with remote proctoring.

Dec 14, 202231:14
Ep. 31 - Assessment Luminary, Thomas Goettsche, SAP

Ep. 31 - Assessment Luminary, Thomas Goettsche, SAP

John Kleeman is joined by assessment luminary, Thomas Goettsche, SAP Certification Principal Production and Psychometrics Consultant. A trained psychologist, he has designed key performance indicators, e-learning programs, best practice standards and much more over the course of his career. Thomas identifies the intrinsic value of certifications, his contributions as a member of SAP's development team and explains the ins and outs of standard setting.

Nov 17, 202229:10
Ep. 30 - Assessment Luminary, Dylan Wiliam, University College London

Ep. 30 - Assessment Luminary, Dylan Wiliam, University College London

John Kleeman is joined by assessment luminary, Dylan Wiliam, Emeritus Professor of Educational Assessment at University College London. Dylan's academic work is focused on the use of assessment to support learning. A leader in formative assessment, he explains that testing can go beyond just measuring a student's learning as it can also be used to improve it. Dylan also shares his thoughts on test validity, online learning and the college admissions testing controversy.

Oct 12, 202230:35
Ep. 29 - Assessment Luminary, Isabel Nisbet, University of Cambridge

Ep. 29 - Assessment Luminary, Isabel Nisbet, University of Cambridge

John Kleeman is joined by assessment luminary, Isabel Nisbet, Affiliated Lecturer at the University of Cambridge. Isabel discusses the two fundamentals of fairness, the importance of public confidence in educational assessment and shares her thoughts on the steady push for test-optional college admissions in the US.

Sep 14, 202231:46
Ep. 28 - Assessment Luminary, Will Thalheimer, TiER1

Ep. 28 - Assessment Luminary, Will Thalheimer, TiER1

John Kleeman is joined by assessment luminary, Will Thalheimer, Principal at TiER1. Will details his 30-year journey to becoming a learning expert, which started as an instructional designer, trainer, and simulation architect. He explains which key findings from learning science we should put into practice, the four pillars of learning effectiveness, and the best way to evaluate how effective learning is.

Aug 04, 202233:12
Ep. 27 - Assessment Luminary, Siddhartha Gupta, Mercer | Mettl

Ep. 27 - Assessment Luminary, Siddhartha Gupta, Mercer | Mettl

John Kleeman is joined by assessment luminary Siddhartha Gupta, CEO of Mercer | Mettl. Over the course of his career, Sid held key leadership positions at some of the most prestigious global IT companies. It was during that time that he discovered the niche of assessments and eventually moved to Mercer | Mettl. Sid shares his opinion on technology and bias, assessment innovation, and Mercer | Mettl's partnership with Questionmark for the upcoming Bring Your Own Proctor feature.

Jul 13, 202232:45
Ep. 26 - Assessment Luminary, Suzanne Heiob, SAP

Ep. 26 - Assessment Luminary, Suzanne Heiob, SAP

John Kleeman is joined by assessment luminary Suzanne Heiob, the Global Certification Product Owner of SAP, who began her journey with the company as a student. Among her responsibilities, she oversees the product roadmap and keeps an eye on market trends to ensure that they are adopted into the program. Suzanne shares her thoughts on the best practices and future of certification programs, along with discussing international challenges, the computer translation capability co-innovation with Questionmark, and the ITCC award it recently won. 


Jun 08, 202227:35
Ep. 25 - Assessment Luminary, Andre Allen, FifthTheory

Ep. 25 - Assessment Luminary, Andre Allen, FifthTheory

John Kleeman is joined by assessment luminary, Andre Allen, Managing Member and Executive Vice President of FifthTheory, who has over 23 years of continuous experience in the test publishing industry. Andre has pioneered many of the technologies that are the basis for many of the assessment delivery and scoring options for today. Andre shares how he got into assessment and discusses how you can assess leadership, admissions testing and how we can do better around diversity in testing. 

Mar 29, 202229:32
Ep. 24 - Assessment Luminary, Linda Anguish, SPHR, Institute for Credentialing Excellence (ICE)

Ep. 24 - Assessment Luminary, Linda Anguish, SPHR, Institute for Credentialing Excellence (ICE)

John Kleeman is joined by assessment luminary, Linda Anguish, SPHR, Director of Accreditation Services at the Institute for Credentialing Excellence (ICE). She is responsible for the accreditation programs at ICE including the NCCA, ACAP and ICE’s newest accreditation program for ISO 17024 compliance. Linda shares how she got into assessment and credentialing and discusses the advantages of credentialing, the ICE certification program, diversity and the future of certification.

Feb 23, 202229:52
Ep. 23 - Assessment Luminary, Stephen Sireci, UMass Amherst

Ep. 23 - Assessment Luminary, Stephen Sireci, UMass Amherst

John Kleeman is joined by assessment luminary, Stephen Sireci, Professor in the Psychometrics Program and Director, Center for Educational Assessment at University of Massachusetts Amherst. His work is well known in the world of assessment. Steve shares how he got into assessment and discusses his research around validity, the five sources for validity evidence and his good practice advice.

Jan 26, 202232:04
Ep. 22 - Assessment Luminary, Jarrod Morgan, Meazure Learning

Ep. 22 - Assessment Luminary, Jarrod Morgan, Meazure Learning

John Kleeman is joined by assessment luminary, Jarrod Morgan, Founder of ProctorU (now known as Meazure Learning) and Chief Strategy Officer. Jarrod has been a leader in the educational technology field for over 16 years. Jarrod shares how he got started in assessment and discusses all things proctoring, including where he sees proctoring going, good practices in remote proctoring and the human first approach.

Dec 21, 202128:20
Ep. 21 - Assessment Luminary, Marten Roorda, Riiid Labs

Ep. 21 - Assessment Luminary, Marten Roorda, Riiid Labs

John Kleeman is joined by assessment luminary, Marten Roorda, Chief Measurement and Learning Officer at AI and education company, Riiid Labs. Marten is a pioneer in educational technology, a leading innovator in computer-based testing, e-learning and adaptive learning and has an extensive background, which includes ACT, CITO and more. Marten shares how he got started in assessment and discusses how to use AI in assessment, adaptive learning and his best practice advice.

Nov 09, 202128:42
Ep. 20 - Assessment Luminary, Alina von Davier, Duolingo

Ep. 20 - Assessment Luminary, Alina von Davier, Duolingo

John Kleeman is joined by assessment luminary, Alina von Davier, Ph.D, Chief of Assessment at Duolingo. At Duolingo, Alina and her team operate at the forefront of computational psychometrics. Alina shares how she got into assessment and discusses how to make a high stakes test equitable with multiple instances, her experience as a female leader in the assessment space and her advice to people who want to progress in assessment.

Oct 13, 202132:59
Ep. 19 - Assessment Luminary, Riz Khan, SAP

Ep. 19 - Assessment Luminary, Riz Khan, SAP

John Kleeman is joined by assessment luminary and "Learning Enthusiast," Riz Khan of SAP. Riz is Global COO and Vice President of Training and Adoption at SAP and frequently partners with CxOs, executives and leadership groups to help them understand the importance of corporate education and the power of user adoption. Riz shares his background in assessment and learning and discusses the value of investing in learning and training and how assessments fit in, his best practice advice and the future of learning and adoption.

Sep 15, 202129:12
Ep. 18 - Assessment Luminary, Gavin Cooney, Learnosity

Ep. 18 - Assessment Luminary, Gavin Cooney, Learnosity

John Kleeman is joined by assessment luminary, Gavin Cooney, CEO and co-Founder of Learnosity, the leading provider of assessment infrastructure for the learning industry. As you may have heard Learnosity recently acquired Questionmark. Gavin discusses how he got into assessment and dives into various topics, including the importance of accessibility and inclusivity in education, the trends he's noticed and how Questionmark and Leanosity fit together.

Aug 18, 202127:32
Ep. 17 - Assessment Luminaries, Sharon Shrock and William Coscarelli

Ep. 17 - Assessment Luminaries, Sharon Shrock and William Coscarelli

John Kleeman is joined by assessment luminaries, Sharon Shrock and Bill Coscarelli, co-authors of award winning book "Criterion-Referenced Test Development." Sharon and Bill discuss how they got into assessment and tell the story of their book. The pair also shares patterns in testing they've seen working with various organizations and their advice.

Jul 27, 202131:04
Ep. 16 - Assessment Luminary, Dr. Leanne Furby, National League for Nursing

Ep. 16 - Assessment Luminary, Dr. Leanne Furby, National League for Nursing

John Kleeman is joined by assessment luminary, Dr. Leanne Furby, Director of the Assessment Services Division at National League for Nursing. Leanne discusses how she got into assessments, talks about the assessment program at NLN and remote proctoring during COVID-19 and explains spaced learning and how it helps.

Jun 17, 202125:00
Ep. 15 - Assessment Luminary, Akil Bello, FairTest

Ep. 15 - Assessment Luminary, Akil Bello, FairTest

John Kleeman is joined by assessment luminary, Akil Bello, who is an expert in assessment reform. He currently serves as the Senior Director of Advocacy and Advancement at FairTest, the National Center for Fair and Open Testing. Akil discusses how he got interested in assessments and talks about his work with admissions tests, whether recruitment tests are fair and his thoughts on what the testing industry should be doing better.

May 19, 202127:06
Ep. 14 - Assessment Luminary, Helen Baron, Psychometric Test Development

Ep. 14 - Assessment Luminary, Helen Baron, Psychometric Test Development

John Kleeman is joined by assessment luminary, Helen Baron, who is an expert in psychometric measurement and assessment. Helen works with test publishers and employers on developing tests and assessment processes. Helen discusses how she got into assessment, and dives into her time at SHL, the different types of assessments used in recruiting and where personality tests come in and her work on assessment standards.

Apr 20, 202127:16
Ep. 13 - Assessment Luminary, Wendy Kirkpatrick, Kirkpatrick Partners

Ep. 13 - Assessment Luminary, Wendy Kirkpatrick, Kirkpatrick Partners

John Kleeman is joined by assessment luminary, Wendy Kirkpatrick, President and Founder of Kirkpatrick Partners. As a global driving force of the use and implementation of the Kirkpatrick Model, Wendy leads companies to measurable success through training and evaluation. Wendy discusses how she got into training and assessments, and dives into the four levels of evaluation, why level three is the most important and how to measure behavior and her tips for evaluating training while remote working.

Mar 09, 202132:26
Ep. 12 - Assessment Luminary, Chris Norton, Mentor Group

Ep. 12 - Assessment Luminary, Chris Norton, Mentor Group

John Kleeman is joined by assessment luminary, Chris Norton, Director at Mentor Group, a global sales transformation company based in UK. Chris shares his background and experience with assessments and discusses the value and best practice advice on using assessments in the sales space, the new Questionmark B2B Sales Qualification by Mentor Group and the future of digital selling.

Feb 10, 202131:34
Ep. 11 - Assessment Luminary, Bernt Nilsen, Datakortet

Ep. 11 - Assessment Luminary, Bernt Nilsen, Datakortet

John Kleeman is joined by assessment luminary, Bernt Nilsen, founder of Datakoret (ECDL Norway) and Norsk Test, the leading test developer and supplier of national certification programs in Norway. Bernt discusses how he got into the assessment world and shares what's important when running a testing service business, his best practice advice for creating high-stakes assessments and the value in anonymous testing.

Jan 12, 202127:30
Ep. 10 - Assessment Luminary, Alan Thiemann, Attorney and General Counsel to ATP

Ep. 10 - Assessment Luminary, Alan Thiemann, Attorney and General Counsel to ATP

John Kleeman is joined by renowned attorney, Alan Thiemann of Hans Santos, PLLC and General Counsel to the Assocation of Test Publishers (ATP). Alan discusses how he got into testing and his role at ATP and dives into topics such as privacy and remote proctoring, how to avoid being taken to court for a bad test and testing controversies in the US.

Nov 24, 202028:17
Ep. 9 - Assessment Luminary, Liberty Munson, Microsoft

Ep. 9 - Assessment Luminary, Liberty Munson, Microsoft

John Kleeman is joined by assessment luminary, Liberty Munson, who is the Senior Psychometrician for Microsoft. Liberty discussess how she got into assessments, why people stay in jobs, the value of certifications and the future assessments, among other topics. Liberty also shares her aha! moment - listen to hear more!

Sep 09, 202033:54
Ep. 8 - Assessment Luminary, Jim Holm, Examity

Ep. 8 - Assessment Luminary, Jim Holm, Examity

John Kleeman is joined by assessment luminary, Jim Holm, CEO of Examity, the leading solution for learning validation and test security. Jim discusses how he got into his current role and the challenges he's faced with COVID-19, and dives into remote proctoring, including the value, his best practice advice, the future of proctoring and his views on artificial intelligence and proctoring.

Jul 22, 202030:34
Ep. 7 - Assessment Luminary, Paul Crump, Cambridge Assessment

Ep. 7 - Assessment Luminary, Paul Crump, Cambridge Assessment

John Kleeman is joined by assessment luminary, Paul Crump of Cambridge Assessment, the non-teaching department of the University of Cambridge that develops and delivers examinations and tests around the world through their three exam boards.  Crump discusses his background in assessments and dives into the Thinking Skills testing initiated by Cambridge Assessment and his best practice advice for those creating questions and assessment.

May 19, 202028:50
Ep. 6 - Assessment Luminary, Sue Martin, Certification Expert

Ep. 6 - Assessment Luminary, Sue Martin, Certification Expert

Host, John Kleeman is joined by assessment luminary, Sue Martin, a pioneer in the assessment industry responsible for implementing the first test center network in Europe. Sue shares her story and advice on transforming certification programs. Discover the five most important considerations when setting up your certification program.

Apr 15, 202029:53
Ep. 5 - Assessment Luminary, William G Harris, ATP

Ep. 5 - Assessment Luminary, William G Harris, ATP

John Kleeman is joined by assessment luminary, William G Harris, Chief Executive Officer of the Association of Test Publishers (ATP), the leading international organization for publishers of assessment programs. G Harris discusses his background in assessments and dives into his advice for those starting out in assessments, the importance of validity in assessments and the different sets of standards for testing, among other topics.

Mar 17, 202026:14
Ep. 4 - Assessment Luminary, Dave Foster, Caveon

Ep. 4 - Assessment Luminary, Dave Foster, Caveon

John Kleeman is joined by assessment luminary, Dave Foster, CEO of Caveon, the first test security company. Dave is a psychologist, psychometrician and recognized luminary in the measurement industry. In this episode, Dave discusses his background in assessments and dives into the main risks and threats of testing, his recommendations on the key measures you should take with test security and how statistical analysis fits into this, among other topics.

Feb 18, 202027:21
Ep. 3 - Assessment Luminary, Steve Dept, cApStAn

Ep. 3 - Assessment Luminary, Steve Dept, cApStAn

John Kleeman is joined by assessment luminary, Steve Dept, CEO and founding partner of cApStAn, a high-profile language service provider. Steve received his education in English, Dutch, French and German but he is essentially an autodidact and a field practitioner. In this episode, Steve discusses his background in assessments and dives into good practice advice for translating tests, including the translation process, translation technology and other translation resources.

Jan 14, 202037:06
Ep. 2 - Assessment Luminary, Steve Lay, Questionmark

Ep. 2 - Assessment Luminary, Steve Lay, Questionmark

John Kleeman is joined by one of Questionmark's product managers, Steve Lay. Steve is an expert on scalable, computerized assessment and integration between systems, and has ensured that the million of assessments delivered by Questionmark are robust and scalable. He is also active in the IMS and other e-learning standards groups. In this episode, John dives into many topics with Steve including the impact of technology on the assessment process, QTI and LTI and proctoring.

Dec 04, 201934:56
Ep. 1 - Assessment Luminary, Jim Parry, Compass Consultants

Ep. 1 - Assessment Luminary, Jim Parry, Compass Consultants

Host, John Kleeman is joined by assessment luminary, Jim Parry, who has over 40 years of experience as a course designer, developer and instructor. Jim discusses his 12-year tenure at the Coast Guard as a civilian employee as the Test Development and e-testing Manager, during which he guided the move from paper to online testing for the entire Coast Guard.

Nov 01, 201935:42