Skip to main content
AB Testing

AB Testing

By AB Testing
Alan and Brent talk about Modern Testing (which isn't all that modern, and not really about testing) - including Agile, Lean, Delivery, DevOps, Data Science, Leadership, and more. Between them, Alan and Brent have nearly 60 years of software experience and are here to tell stories and talk about what they're seeing in the world of software.

Intro / Outro music on the AB Testing podcast is The One by Rivet. License: CC BY

Intro / Outro music on ABT343 is from
“Werq” by Kevin MacLeod (
License: CC BY (

Listen on Spotify
Where to listen
Apple Podcasts Logo

Apple Podcasts

Google Podcasts Logo

Google Podcasts

Pocket Casts Logo

Pocket Casts

RadioPublic Logo


Spotify Logo


Currently playing episode

Episode 146: The answer is always data

AB Testing

Episode 167: Oh my god! Get to the point!
We record an entire podcast without getting to the point - or maybe we just created a lot of points. We talk about podcasting, how we learn, and async work - and about 30 other things.
October 03, 2022
Episode 166: The Modern Modern Testing Podcast
We chat a bit about filling offices, then a lot about A/B testing, and share a few stories along the way.
September 15, 2022
Episode 165: It IS Modern, and it IS about Testing
After a very brief political discussion (apologies), we talk about what we'd do as testers today, and along the way, accidentally reinvent Modern Testing.
August 15, 2022
Episode 164: Matt LeRay from Speedscale
We are joined by Matt from Speedscale. We talk a little about his company, but talk a lot about everything else. Important links Speedscale on twitter - Matt on twitter -
August 01, 2022
Episode 163: Layoffs
Interspersed with a ridiculous number of tangents, we talk about layoffs.
July 18, 2022
Episode 162: Hanging out with Kirk Marple
We are joined by Kirk Marple, CEO of Unstruk. We talk about data mostly, but also baseball (which is also data) - and share a few microsoft stories.
July 05, 2022
Episode 161: msft stuff, and the state of modern testing
We talk about a lot of stuff - including posting salary ranges for jobs, SDETs making devs cry, and the pesky Principle 7
June 20, 2022
Episode 160: The Modern Testing Podcast: featuring Bert Jernsen
We talk about...a lot of things. From testing podcasts to A/B (with the slash) testing to the idea of Internal Developer Relations. It's a lot, so apologies in advance.
June 06, 2022
Episode 159: Darko Fabijan from Semaphore
We have Darko from Semaphore - that's the good news. The bad news is that we lost the last 5-ish minutes of the podcast due to me not planning ahead. We will work with Darko to crash an episode of his podcast in the near future and complete the conversation.
May 02, 2022
Episode 158: Finding the Sweet Spot with Goal Setting
We talk about setting goals...or commitments...or /PLANS/ for our team members.
April 11, 2022
Episode 157: Kanban - and then it gets boring...
We play the kanban game - but stop, because playing the kanban game ( on a podcast is super boring. So then we just gave up and started talking and wound up being even more boring. Enjoy!
March 27, 2022
Episode 156: Customers and Quality
We talk a lot about Modern Testing Principle number 5 - the one that says only customers can evaluate quality, and why - despite controversy, we believe it is true. We also discuss whether or not Deming would be a fan of the principles.
March 14, 2022
Episode 155: Quality Coaching with Anne-Marie Charrett
We are honored to have Anne-Marie Charrett join us this week to talk about quality coaching.  Anne-Marie currently works at CultureAmp where she puts quality coaching into practice. You can find her on twitter (@charrett), and on her website. Her ongoing book on quality coaching is available here, with fresh installments monthly.
February 22, 2022
Episode 154: The Complete Software Tester w/ Kristin Jackvony
Kristin Jackvony - author of The Complete Software Tester - ( joins us for a wonderful discussion on testing, testers, and the value of a music education. 
February 07, 2022
Episode 153: Quality Coaching with Henry Golding
This week, we are joined by Henry Golding, who doesn't call himself a quality coach, but that's what he does anyway.  For more about Henry, check out this preview of Henry's talk at GDC, and his Game Automated Testing Resource Hub 
January 24, 2022
Episode 152: AAB Testing with Al Shalloway
We are joined by Al Shalloway. We talk about Agile mostly, but we finally get to talk about SAFE. You can contact Al at al.shalloway at successengineering dot works
January 10, 2022
Episode 151: Past, Future, and Covid
It's our last episode of 2021, so we take some time to reflect on the last year, look forward into the next year.
December 13, 2021
Episode 150: Ramble On
It's our 150th episide. With absolutely zero celebration or anything else special, we ramble our way through discussions on Customer-Centric Monitoring, Alan's 3 biggest accomplishments in 2021, and our favorite video games.
November 15, 2021
Episode 149: Perks and Tips
We talk a little about work place perks (for reasons unknown), and then talk more about the new fun challenges with Alan's new org.
October 25, 2021
Episode 148: Us, Them, and the (new?) MT Principles
Eventually, we talk about how we could gear the Modern Testing Principles more towards development teams.
October 11, 2021
Episode 147: Coaching...and stuff
We attempt to talk about coaching and quality coaching...and we do - mostly.
September 13, 2021
Episode 146: The answer is always data
We chat a bit about the rise (?) of Modern Testing practices in the industry, and then look at a few TestSphere cards - and eventually end up talking a whole lot about data.
August 23, 2021
Episode 145: Diversity, Safety, and Brent's grody stories
We kick off the episode with 5 painful minutes of discussion on Brent's hernia surgery before making it into a discussion on fairness, diversity, psychological safety, and other stuff. And we talk about a lot of books.
August 09, 2021
Episode 144: Days of Future Past
We talk a bit about Alan's recent AMA on the future of test automation and then about his past with Microsoft Teams.
July 26, 2021
Episode 143: Questions from Larry
We dive (wade? splash??) into some listener questions on Theory of Constraints applied to different aspects of software development and Modern Testing.
July 12, 2021
Episode 142: Wild Ideas with Reflect
We're joined by Nick and Todd from to talk about the future of test automation (sort of) and developers creating UI automation. We loved having these folks on the show, and hope you enjoy the conversation.
June 14, 2021
Episode 141: Dichotomies and Tautologies
In this episode, we discuss deep vs. shallow testing, choosing what to automate, and the future of testing roles (sort of). The link we discuss is
May 31, 2021
Episode 140: AB DevOps
We discuss Alan's experiences at TestingFestival, talk a bit about DevOps and answer (sort of) a mailbag question on BDD
May 17, 2021
Episode 139: The Genius of Tangents
We talk more about Lencioni's working genius model, and attempt to attach it to a mailbag question about how projects get done in a modern testing world.
May 10, 2021
Episode 138: So you want to "get into automation"
We discuss one of the age-old questions of software testing. How does a "manaual tester" 'get into' automation. 
April 26, 2021
Episode 137: Personality Types
We dive into our history of personality type tests. Starting with the classic MBTI, we move to Insights, Strengths Finders, and then Working Genius (we also briefly mention our Harry Potter houses). Meyers Briggs Insights Strengths Finders Working Genius
April 05, 2021
Episode 136: Learning stuff, and why testing isn't quality
This time we talk a bit about learning (including an upcoming webinar from alan -, and talk about quality vs. testing - which is just another framing of principle 5 of the modern testing principles. 
March 22, 2021
Episode 135: Record, Playback, and Personality
We talk a bit about personality types (more on this in 136), touch on some thoughts on record & playback automation, and a few other tangential topics. Books mentioned on this podcast include: Accelerate (Forsgren et al) Radical Candor (Scott) Where Good Ideas come from (Johnson) How to Measure Anything (Hubbard) Lean Startup (Ries) The Leaders Guide to Radical Management (Denning) Drive (Pink) One more for the bookshelf that we didn't call out is Principles of Product Development Flow (Reinertsen)
March 08, 2021
Episode 134: Easier, Faster, Safer
It's a bit of a meandering discussion this time, but we dance around the role of teams that support other teams success. Here's the blog post link Brent refers to in the podcast - Test Doesn't Understand the Customer
February 22, 2021
Episode 133: Test Ownership and Test Automation
We reflect a bit on Alan's recent article (inspired by episode 132), and then talk quite a bit about the correlation of test automation ownership and quality. We hope you enjoy the show.
February 01, 2021
AB Testing - Episode 132: Mindsets and Metrics
It's our first episode of 2021 - where we talk (a little) about the US political situation, then discuss mindsets and skillsets - and then wrap things up with a discussion on estimation metrics.
January 11, 2021
AB Testing – Episode 131: The 2020 Year End Extravaganza
This is our last recording of 2020, so we reflect a little on 2020 and look forward to 2021.
December 07, 2020
AB Testing – Episode 130: The Pre-Election Episode
We talk a bit about the upcoming U.S. Election, discuss ideal team size, and then workshop some ideas on what “advanced” training for Modern Testing may look like.
November 02, 2020
AB Testing – Episode 129: Alan’s Journey to Modern Testing
This time it’s the story of Alan – from music teacher to bicycle messenger to modern testing (and just a few steps in between). It’s another view into the origins of modern testing.
October 19, 2020
AB Testing – Episode 128: Brent’s Path to Modern Testing
It’s origin story time. We talk about the early days of Brent’s testing career and how he found his way into the thing we now call Modern Testing.
September 21, 2020
AB Testing – Episode 127: Brent’s Hand & Alan’s Org
Apologies for the audio issues – Brent was having technical challenges. We cover stuff related to Alan’s growing team (and responsibilities) at Unity – plus the usual assortment of tangents.
September 08, 2020
AB Testing – Episode 126: Azure Powered
We talk about what’s going on with Alan’s expanded role at Unity and answer (sort of) a mailbag question about bottlenecks in delivering mobile apps.
August 17, 2020
ABT 343 – Andreas Worm
This time, Andreas Worm – aka @globalworming in our Slack group joins me for conversations, mostly about his experiences with scrum.
August 10, 2020
ABT 343 – Danny Faught
My guest for the ABT 343 is Danny Faught. We talk about his move out of being a dedicated tester, and well as the biography he’s putting together. Find Danny on twitter and read his blog.
August 03, 2020
AB Testing – Episode 125: P* Management and ATAOSQ
Just Brent and Alan this week. We talk about how Program Management can help with Modern Testing and Accelerating the Achievement of Shippable Quality… …but mostly Alan tries to explain to Brent that Product Management is different than Program Management, no matter what titles Microsoft use. Worth reading on the topic:
July 27, 2020
AB Testing – Episode 124: Learning with Maaret
Brent and Alan are joined by Maaret Pyhäjärvi, who talks with us about her role, the role of testing in general, mob programming, and a whole lot of other topics. I hope this was the beginning of a longer conversation, and we hope to have Maaret as a guest again soon. Maaret’s blogMaaret’s book on […]
July 13, 2020
AB Testing – Episode 123: Sue Loh’s Evil Plan to Save the World
Sue Loh (@suedeyloh on twitter). Sue has a lengthy history of finding and fixing nasty performance issues across a lot of different operating systems at Microsoft, but we spend most of our time talking about her Evil Plan To Save The World Get a copy of Sue’s excellent book, Raven, today
June 29, 2020
AB Testing – Episode 122: Julian Bucknall and TestCafe
We are joined by the CTO of Devexpress to talk about TestCafe.
June 08, 2020
AB Testing – Episode 121: Peter Senge is Old
We talk a bit about Covid (again), touch on a few random subjects, and then find time to answer a few mailbag questions. We also now have an AB Testing Podcast instagram.
May 18, 2020
AB Testing – Episode 120: David Bishop on Metagility
For this episode, we are joined by David Bishop – the author of a new book called Metagility. There’s a lot of good stuff here, and we think you’ll find it interesting. Links for more info: Book information – David’s company – Current course schedule –
May 04, 2020
ABT 343 – Melissa Eaden
Melissa Eaden is in the hot seat this week to talk about management, leadership, modern testing, and more. Melissa’s blog: Melissa on Twitter: @melthetester
April 23, 2020
AB Testing – Episode 119: Retrospectives
We, of course, talk about working through Covid-19, but we spend most of our time talking about retrospectives.
April 20, 2020
ABT 343 – Yuliya Bagriy (aviskase)
Yuliya Bagriy (known across the internet as aviskase) is my guest this time. We talk about her path into testing, a bit about her new role, and the origin of the alias, ‘aviskase’ – which includes a tangent on how she broke Postman.
April 09, 2020
AB Testing – Episode 118: Covid continues; and Velocity
We talk about Covid (because every meeting does these days). And we also (sort of?) talk about velocity.
April 06, 2020
ABT 343 – Chris Kenst
I’m joined by Chris Kenst, who talks about his path into testing, his job as an automation engineer (which he convinces me not to hate), and his work with AST. He also mentions these links. His job – AST – Blog – List of test conferences – Testing education resources – […]
March 31, 2020
ABT 343 – Melissa Pohlman
343 is back with Melissa Pohlman, who has thoughts for testers, advice for managers, and stories about a whole lot more.
March 23, 2020
AB Testing – Episode 117: The Covid-19 WFH Show
We talk quite a bit about working from home, and a bit about test coaching / consulting in the Modern Testing world. We hope everyone is staying healthy and sane out there.
March 17, 2020
AB Testing – Episode 116: What’s the Deal with ISTQB?
We have an in-depth discussion on the recently published Vision on the Future of Software Testing from ISTQB
March 02, 2020
AB Testing – Episode 115: Gates and Balmer and Satya (oh my)
This time, we reflect on the Gates, Balmer, and Satya eras at Microsoft.
February 17, 2020
ABT 343 – Conor Fitzgerald: The Sequel
Conor is back for more questions on Modern Testing. As with last time, you can follow him on Twitter (@conorfi) and read his blog here. Intro Music from “Werq” by Kevin MacLeod ( License: CC BY (
January 22, 2020
AB Testing – Episode 114: Random Reflections and Reading
Happy New Year! This week, we react to a 15 -year-old paper, and then somehow segue into talking about a few other papers. Papers mentioned: The Ongoing Revolution in Software Testing By Cem Kaner State of DevOps 2019 An Analysis of the Effect of Code Ownership on Software Quality across Windows, Eclipse, and Firefox by Nachi Nagappan
January 14, 2020
ABT 343 – Conor Fitzgerald
In this 343, Conor Fitzgerald and I talk about his experiences in testing, his experiences in Modern Testing, and a whole lot more. More info: Follow Conor on twitter: @conorfi Read his blog Also worth linking is Elisabeth Hendrickson’s Explore It!
January 06, 2020
ABT 343 – Perze Ababa
In this 343, I’m joined by one of the long-time listeners and supporters of the AB Testing Podcast, the one and only, Perze Ababa. We talk (mostly) about leading a quality culture, but as you’d expect, we take a few tangents along the way. Perze is @perze on twitter – follow him!
December 29, 2019
AB Testing – Episode 113: The 2019 Year End Extravaganza
We reflect on our December 2018 predictions, look back on the wonderful year of 2019, and talk a bit about what we expect in 2020. Happy Holidays from the two of us to the three of you.
December 17, 2019
AB Testing – Episode 112: The Penultimate Potpourri
Lots of topics this time, including CI/CD, UI automation, and ABT ulterior motives. Notes (more to be added later): Rockstar language Advent of Code
December 11, 2019
ABT 343 – Maciej Wyrodek
Maciej Wyrodek joins the 343 podcast to talk about his experiences with modern testing, The Unicorn Project, and various rants about the state of software testing. Follow Maciej On twitter – @maciejwyrodek His blog – The Broken Test Also mentioned: The Modern Testing Principles mindmap Polish article on the state of testing
December 02, 2019
ABT 343 – Areti Panou
I am joined this time by Areti Panou. Listen in to hear more about her changing role, and her experience with modern testing. Areti’s blog is, and you can folllow her on twitter – @unremarkableqa
November 25, 2019
AB Testing – Episode 111: Oredev, DevOps, and Stuff
We talk a bit about Oredev, some more about devops, controversies in the principles, and a bit more about Alan’s role at Unity.
November 18, 2019
ABT 343 – Dave Westveld
Our guest is Dave Westerveld. We talk a bit about accessibility, challenges in testing LMS software, and a lot of “modern” topics. Follow him on twitter @offbeattesting, and his blog at
November 13, 2019
ABT 343 – Andrew Morton
Our first real episode is a quick chat with Andrew Morton. Andrew is the Dev Boss at Ministry of Testing, and can be found at @testingchef on Twitter
October 31, 2019
AB Testing – Episode 110: PM, PM, and MT
We spend most of this podcast trying to understand each other while we wind through conversations on Product Management, Project Management, Program Management and how they all fit into quality.
October 29, 2019
AB Testing – Episode 109: The MT Guide to Promotions
Some discussion (and rants) on career maps (aka career ladders, levels, etc.) – and our experiences in how to make these fail spectacularly (and sometimes work).
October 15, 2019
AB Testing – Episode 108: Lack of Testers vs. Lack of Testing
it’s a mailbag show, where we talk about whether or not bugs in Windows 10 are due to a lack of a test team, and how you can apply MT principles if you’re a consultant asked to write automation. Once again, the audio isn’t awesome, but quality debugging continues.
October 01, 2019
AB Testing – Episode 107: A Plethora of Principles
Good news: We have a new episode!Bad News: I am doomed to never having another podcast without audio issues. My mic cable must have been lying across the speakerphone in the conference room, as it was picking up a massive amount of static. I was able to remove the static (you’ll thank me), but removing […]
September 16, 2019
AB Testing – Episode 106: Weapons of Math Destruction
Steve Rowe is back (for the third time!) on AB Testing to take part in a discussion on ethics in data science. Hope you enjoy.
August 12, 2019
AB Testing – Episode 105: A World of Modern Testing
Heads up – our technical producer (me) messed up levels and microphone choice and we ended up with a little bit of bleed between microphones that ends up not meeting our normally high quality standards. The next one will be better. For content, we talk a lot about what we’re seeing in the industry that […]
July 31, 2019
AB Testing – Episode 104: Leading Quality (with Ron Cummings-John)
We are joined by the author of Leading Quality, Ron Cummings-John to talk about his upcoming book. More info at
July 01, 2019
AB Testing – Episode 103: The Croissants are Selenium (with Jason Huggins)
We are joined by the inventor of Selenium, Jason Huggins. Listen in for stories about the early days of Selenium, Jason’s thoughts on our UI rants, and much much more. Be sure to check out Jason’s latest venture – Tapster
June 03, 2019
AB Testing – Episode 102: Modern Testing takes on UI Automation
Alan recently took some heat on Twitter for disliking Selenium – or more accurately, how it was being used. This episode is a discussion and expansion of why Alan and Brent both have strong feelings about the industry infatuation with UI testing.
May 20, 2019
AB Testing – Episode 101: AB…not testing?
We realize during the intro, that this is the first time we’ve recorded an AB Testing podcast when neither Alan nor Brent are in a dedicated testing / QA role. What does that mean for Modern Testing? Turns out – not that much. Listen as we discuss.
May 06, 2019
AB Testing – Episode 100: The AB Testing Fanfest
Thanks to everyone who helped get us to this milestone episode. For episode 100, we share cast-time with all three of our listeners – who may or may not have been paid for their kind words.
April 15, 2019
AB Testing – Episode 99: Post-Modern Alan
This episode brought to you by Practitest – We spend some time preparing for episode 100, and talk just a bit more about what Alan may do next at Unity.
April 01, 2019
AB Testing – Episode 98: Stuff about Alan and a Mailbag Question
We talk very abstractly about Alan’s new role, and dive into a mailbag question on the role of leadership in a quality culture. This episode is brought to you by Practitest – check them out at
March 12, 2019
AB Testing – Episode 97: Questions About Developers and Tests
An all mailbag show where we talk about skills needed by testers today, how to get developers to write tests, and hiring. Register for Brent’s AST webinar on Thursday –
February 18, 2019
AB Testing – Episode 96: Principle #8???
Happy New Year everyone – amongst catching up, we spend most of the time discussing and debating a potential eighth Modern Testing Principle.
January 15, 2019
AB Testing – Episode 95: The 2018 Reflection / Prediction Show
Happy Holidays to each and everyone of our three listeners. We review and reflect on 2018, and talk about some things we think may happen in 2019. Listen all the way to the end for a cliffhanger from Brent. Thanks again for listening – in 2019, we will hit episode 100, our 5 year podacting […]
December 25, 2018
AB Testing – Episode 94: Modern Testing meets Context-Driven Testing
It’s the penultimate show of 2018, with a lot of (hopefully interesting) preamble on this one, but we spend a chunk of time reviewing the Context-Driven Testing Principles and how they relate to the Modern Testing Principles.
December 03, 2018
AB Testing – Episode 93: The Quality Culture Transition Guide
We eventually talk about a model Alan built to help a team discover where they need to grow in order to build a quality culture or move towards modern testing (they’re kind of the same thing). Some rants and ramblings also included.
November 12, 2018
AB Testing – Episode 92: Customers, Bugs, and Triangles
We almost (but not quite) blast through the banter at the beginning, but eventually get a chance to talk a bit more about interpreting Principle #5, and dive into a James Bach talk on the Test Automation Pyramid.
October 22, 2018
AB Testing – Episode 91: Stories and Models and Things
We answer a mailbag question,  begin to dive into what attributes make up a quality culture, and talk about other stuff. And we have a new intro.
October 08, 2018
AB Testing – Episode 90: A Lot About Nothing
It’s almost an all tangent episode, but we do (eventually) get a chance to discuss how ML and AI are changing testing and quality, and we pay our respects to the great Jerry Weinberg.
September 24, 2018
AB Testing – Episode 89: Modern Testing Mailbag
All mailbag episode as we answer questions about combined engineering teams, life after Modern Testing, and early project “data” samples
September 10, 2018
AB Testing – Episode 88: Testing Isn’t Dead (Testers, OTOH…)
It’s the last of our deep-dive series on Modern Testing Principles. This time, we discuss the (sometimes controversial) principle number 7: We expand testing abilities and knowhow across the team; understanding that this may reduce (or eliminate) the need for a dedicated testing specialist. Not too many tangents this time. Next time, we’ll get caught up […]
August 20, 2018
AB Testing – Episode 87: The One About Data
On to the next principle in our deep dive series. Principle Number 6 says: We use data extensively to deeply understand customer usage and then close the gaps between product hypotheses and business impact. Note that episode 82 makes a nice listening companion.
August 06, 2018
AB Testing – Episode 86: Not the Customer’s Champion
We’re back to discuss another of the modern testing principles. We believe that the customer is the only one capable to judge and evaluate the quality of our product Discussions include a brief discussion of the Kin phone from Microsoft. Finally – if you missed it, the Modern Testing AMA recording is here.
July 24, 2018
AB Testing – Episode 85: The Community Principle
Our dive into the Modern Testing Principles continues with a dive into principle number four: We care deeply about the quality culture of our team, and we coach, lead, and nurture the team towards a more mature quality culture.  
July 09, 2018
AB Testing – Episode 84: Stories about Changes and Nets
On to Principle number 3 (full list at We are a force for continuous improvement, helping the team adapt and optimize in order to succeed, rather than providing a safety net to catch failures. As promised in the show, there will be an AMA on Modern Testing hosted by Ministry of Test on July […]
June 25, 2018
AB Testing – Episode 83: Accelerating the Team
We’re back to the MT Principle Deep-Dives (which may just be medium dives). Principle #2 says: We accelerate the team, and use models like Lean Thinking and the Theory of Constraints to help identify, prioritize and mitigate bottlenecks from the system. We talk about that – but of course, there are a few tangents we […]
June 05, 2018
AB Testing – Episode 82: Brent Talks to Alan About Data
We are already taking an interlude from principle deep-dives to reflect a bit on the episode 81 feedback, and for Brent and Alan to have short (but in depth) discussion on how to get started creating actionable metrics. We’ll be back to Modern Testing principles in episode 83.
May 22, 2018
AB Testing – Episode 81: Business Rulz!
It’s time to deep dive into the Modern Testing Principles with #1 – Our priority is improving the business. We also dive into some experiences and observations from Alan’s recent trip to STAR East.
May 07, 2018
AB Testing – Episode 80: The Mailbag returns: Modern Testing questions
It’s a mostly mailbag show, and we talk about the role of a modern testers as a role that transitions an organization to following the modern testing principles.
April 23, 2018
AB Testing – Episode 79: The Modern Tester Goes to Testbash
We talk quite a bit about testbash, Modern Testing, and give deeper answers to a few of the questions that came up in around the first public appearance of the Modern Testing Principles.
March 26, 2018
AB Testing – Episode 78: Digging in on Modern Testing Principles
We don’t reiterate the Principles directly (see the Episode 77 show notes for that), we talk a lot about how we are adapting the principles slightly based on feedback and discussions with the three – and how much better the principles are for the effort. We also discover that both Brent and Alan love to cook. […]
March 12, 2018
AB Testing – Episode 77: The Conception of the Modern Testing Principles
We discuss, vet, and rewrite (and not for the last time), the Modern Testing Principles. These are a work in progress, but we’re happy to share the current final form. Listen to the podcast to hear these developed in real time. The customer is the only one suitable to judge and evaluate the quality of […]
March 05, 2018
AB Testing – Episode 76: Communing with Testers
More on community, a few tangents, and a few strong opinions.   Mentioned in this episode: Influencer: The New Science of Leading Change Situational Leadership – Training, etc can be found here
February 19, 2018
AB Testing – Episode 75: From Spider to Starfish
We talk about how testers (and test specialists) can use the power of community to discover what they don’t know they don’t know. And other stuff.
February 13, 2018
AB Testing – Episode 74: Where in the World is the Weasel?
First show of 2018, with a variety of topics that are all mostly about Modern Testing. Items of note include the State of Testing survey, podcast plans for 2018, Alan’s presentation schedule, and the usual amount of tangents.   Links: AST Webinar (Alan presenting) State of Testing Survey
January 29, 2018
AB Testing – Episode 73: 2017 Year in review and Predictions episode
It’s the 2017 year end extravaganza with discussions about 2017, thoughts and predictions about 2018, and more…including… The AB Testing Merchandise shop – And, sign up for Audible with AB Testing   We’ll be back in 2018.
December 18, 2017
AB Testing – Episode 72
It’s the penultimate AB show of 2017. More discussions about modern testing, a bit of a preview of what’s coming up in episode 73, the role of a modern tester in code craftsmanship, and a bit on this shift-left thing.
December 05, 2017
AB Testing – Episode 71
In this eclectic episode, we discuss college savings plans, personal finance (wtf?), definitions of “Modern Testing”, and recap the Unity Services Test Summit (where Brent presented on moving to a data-centric culture). Enjoy.
November 21, 2017
AB Testing – Episode 70
Steve Rowe is back for another visit, and he shares some of his experiences from STAR Canada, discusses logical fallacies, and we all chime in on the transition from traditional to modern testing.
November 06, 2017
AB Testing – Episode 69
We had a brief break in recording, but we’re back with another fun-filled fifty minutes where we take mailbag questions on the pros and cons of ISTQB certification, the impact of easy-to-create GUI tests on the testing pyramid, and whether or not you really need a data scientist.
October 16, 2017
AB Testing – Episode 68
We spend a chunk of time talking about a report (The State of Functional Testing), and how it aligns with modern testing. We answer a #mailbag question elaborating on the role of a modern tester, and we get to hear Brent functioning on just a few hours of sleep.
September 18, 2017
AB Testing – Episode 67: The Traditional vs. Modern Test Manager Debate
For this episode, we shoved Brent into a time machine and asked 1990s “Traditional Test Manager” Brent to have a conversation with “Modern Testing Manager” Alan. The usual arguments and antics follow.
September 07, 2017
AB Testing – Episode 66
A bit of a follow up episode with more management advice. We talk about how to give feedback to our employees and about frameworks and approaches we like for giving that feedback.   This episode brought to you by STAR Canada.  If you are thinking about attending, STARCANADA is offering 20% off your conference fees […]
August 21, 2017
AB Testing – Episode 65
It’s the AB Testing Management episode. We reflect a bit on our growth as managers and share some of the tools and philosophies that (we hope) allow us to succeed. Some of the books mentioned include (also note the link on the right for a free month of Audible): The Leaders Guide to Radical Management […]
August 15, 2017
AB Testing – Episode 64
We talk about Agile Testing (both the concept and the book), and how it relates to the “Modern Testing” we’ve talked about before. Also – there’s a mailbag question on Multi Armed Bandit Testing. This episode brought to you by the folks at Techwell. Attend Agile Dev East, the premier agile event covering the latest […]
August 08, 2017
AB Testing – Episode 63
More talk about data, CIRA (collect-inform-recommend-act), and Brent gives Alan free advice on maturing a data aware organization. We mention the great Douglas Hubbard book – How To Measure Anything This AB Testing brought to you by Kobiton Test your mobile app on real cloud based devices today! Kobiton is the complete mobile device cloud […]
July 18, 2017
AB Testing – Episode 62
It’s an all mailbag show, with questions (and answers) about Model-Based testing; if testers are being (or will be) phased out – or if the role of the tester is simply evolving and expanding (and if the expanded role needs a new name); and some expansion on the Collect-Inform-Recommend-Act model. This episode sponsored by the […]
July 10, 2017
AB Testing – Episode 61: INTP – Intermittently Normal Tangential Podcast
It had to happen – it’s an all-tangent show! But, somehow we manage to cover a lot about technical testing, data science, tester mindset and more. Alan mentions a lot of books on this episode – here’s the list with amazon links. Steal like an artist Everybody Lies Wonderland What every BODY is saying And […]
June 26, 2017
AB Testing – Episode 60: #ATAOSQ – Information providers no more
Here at AB Testing, we like this episode a lot. Give it a listen and let us know what you think. More on modern testing, and we break down “accelerating the achievement of shippable quality” in the mailbag segment.  
June 12, 2017
AB Testing – Episode 59
We are back with Rants, Rages, and Regurgitated information from the world of software engineering, and software quality. Topics include just a bit of chat about checking and testing, and then a deeper dive into the topic of “Modern Testing”. This episode brought to our listeners by Kobiton. Test your mobile app on real devices […]
May 29, 2017
AB Testing – Episode 58
Topics this time include a brief recap of episode 57, discussion on viewing a test organization as a community, and a mailbag question on dealing with budgets.
May 07, 2017
AB Testing – Episode 57: Hans Buwalda explains action-based testing
This time on AB Testing, we share the microphone with automation expert, Hans Buwalda. We talk with Hans a lot about his views on test design and test automation, including action-based testing (one of the other AB testing types), as well as his upcoming talks at STAR conferences. This episode brought to our listeners by Kobiton. […]
May 01, 2017
AB Testing – Episode 56
Took a while, but we’re back and ranting away. This time, we talk about the role of Test / Quality as, Accelerating the achievement of shippable quality, and, in a newly named segment called, “Alan talks about working at Unity”, Alan aswers a mailbag question about his approach to change working at Unity.
April 03, 2017
AB Testing – Episode 55
It’s the double-nickel episode of AB Testing. Included in today’s ramblings are: Alan and Brent talk about graduate school, and learning in general, as well as our thoughts on Dr. Seuss’s greatest works Our sponsor, Techwell, is offering a $250 discount on upcoming software training classes in Boston and Chicago using the promo code 17ABW – […]
March 06, 2017
AB Testing – Episode 54
First ever AB podcast with A&B working at two different companies. As expected, Alan talks about Unity the whole time.
February 21, 2017
AB Testing – Episode 53
Alan’s last day at Microsoft, Tequila, and a mailbag. Oh yeah – and this show sponsored by Techwell. Use the promo code ABTesting to get $200 off of you STAR East registration.
January 26, 2017
AB Testing – Episode 52
Some bad recording (but only at the beginning this time), and some big news. Listen and find out the latest on AB Testing.
January 19, 2017
AB Testing – Episode 51: 2016 Year in review and Predictions episode
It’s our 2016 end-of-year extravaganza – if extravaganza means Brent and Alan randomly talking about their favorite software moments of 2016. We also recorded our 2017 predictions…but audio went haywire, so we’ll have to wait for a re-hash of that topic in early 2017. Notes! Blogs: – – Blog Posts: – – […]
December 23, 2016
AB Testing – Episode 50
It’s the Big Five-Oh for Alan and Brent. For our golden anniversary, we talk about TLAs, PMs gone wild, the Concierge MVP, and Brent takes a scrum master quiz.
December 14, 2016
AB Testing – Episode 49
More about TestBash, and Alan (finally) gets into specifics on Microsoft Teams. And tangents….
November 21, 2016
AB Testing – Episode 48: Alan hosts the Three at Test Bash Philly
Live from Test Bash in Philadelphia, this is VAMP Testing with three of the three (Vernon Richards (@testerfromleic), Michael Richards (@saltygunner), and Perze Ababa (@perze)). We discuss sessions at TestBash along with the appropriate number of tangents.
November 18, 2016
AB Testing – Episode 47
This episode includes a discussion of Alan’s new car (along with ideas on learning and saving), and a long discussion on Alan’s long journey to continuous delivery. And a mailbag.
October 24, 2016
AB Testing – Episode 46: Steve Rowe discusses Data . . . and Stuff
In episode 46, we have our very first guest, as Steve Rowe joins us for a discussion on his 3 years in a data science role (plus the usual tangents and random bits of trivia)
September 28, 2016
AB Testing – Episode 45
Pretty much more stuff on what it takes to become data-centric. And weirdly, Alan gives Brent a hard time.
September 09, 2016
AB Testing – Episode 44
This episode is dedicated entirely to the topic of how to transition from data oblivious to a data-centric organization. Very little Alan yelling at Brent, but a whole lot of Brent preaching goodness about how to use data to advance product quality.
August 26, 2016
AB Testing – Episode 43
Hey – audio is much better this time. Episode 43 includes discussions of test generation tools, learning, and whether or not Microsoft is a good example of advancing engineering. Don’t forget – use the promo code ABTESTING when you register for StarWest to get a $200 discount (or go to
August 17, 2016
AB Testing – Episode 42
First off, the audio on this episode has a few issues – we’ve taken steps to improve that in the future. The content includes Alan being Angry, the concept of “Feature Jam”, discussion about CDT, and a bit about data driven engineering.    
July 25, 2016
AB Testing – Episode 41
Topics this time include Performance Reviews, Alan’s trip to the Better Software conference, the changing role of Program Management at Microsoft, and a shot out to the folks at The Testing Show.
June 20, 2016
AB Testing – Episode 40
It’s an(other) all mailbag show. And it’s episode 40. And it was recorded on National Donut Day – what else could you ask for?
June 07, 2016
AB Testing – Episode 39
Hey everyone… This time around, we talk a bit about Alan’s trip to Waterloo, and then we deep dive into Agile’s “longstanding and ongoing attack on the testing role” As you can expect, it’s a fun one.
May 20, 2016
AB Testing – Episode 38
An almost all-mailbag show, Alan and Brent discuss planning semesters vs. milestones, v-teams, and managing teams vs. managing features.
May 04, 2016
AB Testing – Episode 37
Live, on the air, we reply to a tweet that kicks off And then Brent (inspired by Patrick Prill  aka @testpappy) asks Alan to define Quality, Testing, Test Cases, Testers, Integration Testing, and a few other terms that are difficult to define for most of the three, including (especially?) Alan.
April 18, 2016
AB Testing – Episode 36
It’s been two years, and 36 episodes…of the same stuff. Yet people still listen. This episode includes some discussion of shipping every day (or at least more often than once a week), open space work areas, and what could be our own AB Testing record for tangents within a podcast!
April 10, 2016
AB Testing – Episode 35
We’re baaaaack. This episode covers the one time when Alan was a developer, Ryan Reynolds as Deadpool, balancing quality, collective ownership, and developer owned quality. All in less than an hour.
March 15, 2016
AB Testing – Episode 34
Another AB Testing podcast in the books (and on the web). This time, we discuss the nature of bugs, why we don’t want our testers back; and in the mailbag session, discuss our role in backlog grooming and planning.
February 12, 2016
AB Testing – Episode 33
Happy new year. This blab-fest covers Star Wars, Fallout 4, straw man arguments, even more on Combined (or Unified!) Engineering, and why smart people do stupid things!
January 12, 2016
AB Testing – Episode 32: 2015 Year in review and Predictions episode
It’s the 2015 recap show, where Brent and Alan reflect on 2015, and make some predictions for 2016…and somehow take nearly an hour to do so. Some links from the show Podcasts: This American Life – Episode 403: Nummi – Freakonomics Radio – Wait Wait Don’t Tell Me – Good Job Brain – Total Soccer […]
December 18, 2015
AB Testing – Episode 31
This episode includes a quick rant on tecking vs. chesting (or something like that), moving from predictive to adaptive, and the mailbag is back with a discussion on combined engineering.
December 05, 2015
AB Testing – Episode 30
This episode is all about “stuff Alan and Brent did” – or at least it started that way until we start rambling. Quick recaps of the Strata conference, Eurostar, Redfin, and Riot all find their way into our discussion.
November 20, 2015
AB Testing – Episode 29: LIVE from the Redfin HQ
From the offices of Redfin in Seattle, Washington, Brent and Alan deliver the first ever AB Testing roadshow. Sound isn’t fantastic, and the content, as usual, isn’t very original, but I’m sure one of us said something that will raise your eyebrows. Thanks to Evan and everyone else at Redfin for hosting us, feeding us […]
November 10, 2015
AB Testing – Episode 28
Yet another installment of AB Testing (including what I think is a world record for AB Testing release velocity). This episode includes another rant from Alan about dumb things people do at Microsoft, a long tangent on Alan’s music background that has no relevance whatsoever, and some discussion on upcoming public appearances from Brent and […]
September 25, 2015
AB Testing – Episode 27
In their patented pattern of pure randomness, Alan and Brent talk about the context-driven school of testing, the video music awards, upcoming test conferences, and pointy-haired managers! Errata: Lessons learned…came out in 2001 (not 2003 as Alan guessed on the show). A link to Ioana’s awesome CAST talk is here
September 04, 2015
AB Testing – Episode 26
Recorded in downtown Bellevue, WA…Alan and Brent talk about a talk – and Alan asks Brent questions from a sample ISTQB test.
August 10, 2015
AB Testing – Episode 25
Discussions include Agile, Alan’s new job, developer testing, and we go to the mail bag (maaaiilll baaaaaag) for a quick discussion of Tension Metrics.
July 22, 2015
AB Testing – Episode 24
A trip report from STAR Canada; some discussion of whether or not test is dead (or if being dead is dead), and a trip into the mailbag to talk about monolithic test systems. And Alan fixed the bug that was hiding the media bar on the blog site.
June 30, 2015
AB Testing – Episode 23
It’s Alan’s 20-year anniversary at Microsoft…and he has a new job. We also talk preview Alan’s talk on A/B (with the slash) testing at Better Software, and attempt to get through some mailbag items without getting too distracted (we failed).
June 05, 2015
AB Testing – Episode 22
In this episode: Iterative is not Agile, Brent’s version of the penny game, and a short discussion of AB Testing vanity metrics. …and probably other stuff too.
May 18, 2015
AB Testing – Episode 21: Dysfunctional Central Organizations
Rants and soapboxes are back. This episode also includes a discussion of central organizations, why they fail, and how they can work. And other fun stuff too.
May 04, 2015
AB Testing – Episode 20
It’s our twentieth podcast, AND – it’s our one year anniversary show. We talk about logical fallacies, generalists, and a whole lot of other random topics.
April 09, 2015
AB Testing – Episode 19
In this episode, Alan and Brent talk about dashboards, scorecards, buzzwords, soapboxes, and Alan’s appearance on another podcast. Also included is yet another installment of silly things people do at work to incur the wrath of Alan.
March 24, 2015
AB Testing – Episode 18
Brent has a new job! We talk about what Brent does, discuss interviews, and what test specialists actually (may?) do. And other stuff. Here’s the link for the Data Science in the Cloud e-book mentioned in the podcast.
March 02, 2015
AB Testing – Episode 17
Eventually….Brent and Alan discuss their thoughts on the future of the Program Manager role at Microsoft. The journey there, as always, is interesting and perhaps even entertaining.
February 05, 2015
AB Testing – Episode 16
We’re back for our first episode of 2015…where we talk about almost nothing, but do manage to discuss management styles, holocracy, and what we did with our holiday break.
January 20, 2015
AB Testing – Episode 15
In our last podcast of Twenty-Fourteen, Brent and I talk about Generalizing Specialists, Specializing Generalists, and weird things Alan sees at Microsoft (and other random topics that pop up along the way).   Happy Holidays everyone
December 19, 2014
AB Testing – Episode 14
We’re back! Topics this week include: Heroes vs. Teamwork Not Invented Here Syndrome Bean Counters The Mailbag And the completely bizarre story of Brent’s first days at Microsoft
December 09, 2014
AB Testing – Episode 13
If only you could hear what we left on the editing room floor… Instead, Alan and Brent talk about GTAC, meetings, and bring back the mailbag.
November 13, 2014
AB Testing – Episode 12
This episode includes discussions on peer feedback, the “Skill and the Will”, Alan’s favorite author, Steven Johnson, Not-Invented-Here, and learning organizations.
October 27, 2014
AB Testing – Episode 11
Alan and Brent are back. In this episode, they discuss interviews, data driven engineering, knowledge sharing and the usual grab bag of tangents and random topics related to software and software quality.
October 14, 2014
AB Testing – Episode 10
After a long break, Brent and Alan are back with Episode 10. This week, A & B talk about what changed at Microsoft this summer, what they think is going to change next, and talk a bit about the latest IOS bug. Note: the audio is a little wonky this week. Long story, but it […]
September 26, 2014
AB Testing – Episode 9
Last episode of the summer! Alan talks about git, roles of testers, and we talk about what may (or will) change while Alan is taking a long, long break.
July 09, 2014
AB Testing – Episode 8
And…we’re back. this episode covers Brent’s IRRs, Agile Testing, and, as usual, a wide variety of tangents and rants.
July 09, 2014
AB Testing – Episode 7
In this “all-mailbag” episode, we talk more about change (specifically some changes in engineering approaches we see at Microsoft, including changes in the way we lead and manage people), and a variety of related (and not-so-related) rants. Enjoy.
June 16, 2014
AB Testing – Episode 6
Alan and Brent are back. This time they talk about work item tracking, QOS & QOE, the World of Warcraft guide to running v-teams, and Alan’s upcoming extended vacation.
June 03, 2014
AB Testing – Episode 5
Brent and Alan are  back! In this episode, they discuss Alan’s trip to STAR East, the perils of automation, some ideas on leadership, and how not to tell your team that you’re “Agile”.
May 19, 2014
AB Testing – Episode 4: Leading Change, Learning Organizations, and Test Zombies
In this episode, Brent and Alan discuss Alan’s upcoming trip to the STAR East test conference, and a bit about using Kanban for chores. And then comes a question / statement about estimation that’s worth every minute.
April 28, 2014
AB Testing – Episode 3: Dashboards, Documentation, and Dumb manifestos
In this installment, Alan and Brent talk about Alan’s trip to Florida, data driven engineering, and estimation. And Brent claims that he “hates the Agile Manifesto”.  
April 14, 2014
AB Testing – Episode 2: What AND Why
In this episode, Brent and Alan talk about change management and the difference between Predictive, Iterative, and Adaptive software development models. And Testing!
April 07, 2014
AB Testing – Episode 1: Introductions
Episode 1 – Two Software Testing veterans from Microsoft decide to start podcasting.
April 03, 2014