Empirical Software Engineering Banter

Dr. Andrew Begel, researcher at Microsoft, and Dr. Paige Rodeghero, professor at Clemson University in the US, answer questions on their recent paper which describes an Autism Coding Camp (conducted at Clemson, see link below).

Paige and Andrew discuss the importance of communication and teamwork for neurodiverse and neurotypical developers and the extra challenges faced going online with this camp. We also explored ideas for more inclusive tools and learning environments, and how to improve onboarding and inclusion of all developers in software teams.

Dr. Michael Hilton, a professor at CMU, visits our course and gives a short presentation on promoting human flourishing through ethical software development. Michael describes challenges software engineers face to day with recognizing biases in machine learning models, ethical dilemmas designing humane software.  He also shares with us why being a teaching professor is the very best job in the world (hint: it is about his ability to shape the software developers of tomorrow).   

Dr. Daniel German, a professor at the University of Victoria, talks about his many years conducting empirical research in open source.   

Daniel shares how the practices and challenges in open source provided many rich opportunities for research that are also relevant for industry, such as code review practices, the use of git for distributed version control, and intellectual property practices. While at the same time there are important differences between open source and industry, such as volunteer open source developers have more agency and freedom than paid developers, and the perils of assuming open source are representative of industrial projects.  He also talks about how his research on fairness in Open Stack required qualitative research methods to understand how many developers perceived peer review as unfair and leverages a fairness theory that can be used to understand and address unfairness in reviews in general.   

Dr. Jenna Butler, a senior software engineer with Microsoft, visits our course on empirical software engineering and discusses what she learned through a diary study to understand early and ongoing reflections from developers as they transitioned to working from home during the pandemic. In her longitudinal diary study, Jenna asks about gratitude developers feel (e.g., many feel gratitude for being productive at work and to their colleagues) as well as challenges developers experience.  

Dr. Tom Zimmermann visits our course on empirical software engineering and discusses his many years of research on developer productivity with us.    

We discuss what makes for a productive day from a developer's perspective, how time spent programming relates to perceived developer productivity, how individual and team productivity may have to be balanced, the role work environment and team culture plays on productivity and what we can learn (and can't learn) from "fitbit" type data and surveys. Tom also shares some secrets of his research success and this started with his master's thesis.  

Dr. James (Jim) Herbsleb visits our course on empirical software engineering and shares how the theory he developed was used to measure social technical coordination in real world projects.  Borrowing from organizational behavioural theories, Jim also discusses how communication concerning sources of complexity is particularly challenging when there is also uncertainty forcing a need for direct communication. We also discuss the challenges faced with collaboration today, but also explore the opportunities that the transparency of social coding tools and the ability to learn from open source brings. Finally, we discuss future work to investigate which communication channels are needed for optimal coordination in a project and how there is an even bigger need to design online or virtual environments that can mimic the benefits that colocation (at least within 30 metres) has.   

Dr. Anna Filipova, a Senior Manager of Data Science at GitHub, describes the role of identity and social norms in building sustainable online communities and answers questions about how to create a shared group culture, how to hire online, and whether group churn may not always be a negative thing.   Anna also muses whether bots should be used to block negative behaviour or should they try to shape positive behaviour? And if blocking, should this be done a private or public way?      

In this class visit, Dr. Carolyn Seaman, professor at the University of Maryland Baltimore County, describes the power and drawbacks of using "war stories", a storytelling technique used as part of interviews to learn about what really happens with software documentation in practice.   Carolyn shares with us other advice on using qualitative methods:  "Just as you should never drink or swim alone, never do qualitative analysis alone"!  

Brian Houck, principal program manager at Microsoft, describes the Code Review experience in Azure, sharing with us how static analysis tools are used to shift problems left, how code review is pivotal in avoiding security vulnerabilities, how code review supports building community and the role it plays it onboarding developer to new teams. Brian talks about the unique challenges faced in Azure and Windows: issues of scale and churn across multiple platforms and languages, the need to support code review across team boundaries, and challenges finding experts in such a complex and large scale system.  He also talks about the future design of code review tools that must address these challenges and integrate with unique developer tools. In the following Q&A, Brian discusses dealing with biases in software practices, as well as why mixed research methods are needed to understand and support the challenges they face in his team.

Dr. Christian (Chris) Bird, software researcher at Microsoft Research, discusses his paper and recent insights about code review at Microsoft. Chris shares with us how this study was one of the first to use developer interviews to understand the motivations and benefits of doing code review that go beyond code quality.  He also discusses the success of CodeFlow, a grassroots code review tool that saw widespread adoption across Microsoft and is widely used today.  He further muses how code reviews could be even more efficient and productive, and describes novel ideas around guided code reviews that could help developers do code reviews while working from home, and also how questions asked during code reviews could be leveraged to help generate documentation.  

Dr. Jorge Aranda, software practitioner at Workday, discusses the "Secret Life of Bugs" paper from 2009.  Jorge shares with us how this landmark study informed our community about the limitations of data mining studies and he suggests future research to investigate how the coordination patterns he identified in his study may or may not hold today given modern communication channels and cloud based project deployment.   

Dr. Ali Mesbah, professor at UBC, visits our class and answers questions about his paper  (linked below) on how  test assertions are more effective than code and branch coverage for improving test suite effectiveness. Ali also mentions how machine machine generated assertions fare against human written assertions, how mutation testing is making its way into industry use, how being conservative in our claims about SE research may undersell the value of our research.  Our discussion also leads to some exciting ideas for future research in this area!  

Dr. Nicole Forsgren, VP of Research and Strategy at GitHub and author of "Accelerate:  a book on the Science of Lean Software and DevOps"  shares her journey from NLP and ethnographies of sys admins to DevOps and her studies of Team Performance. 

In the Q&A, she discusses how tradeoffs between speed and stability are not made by either high or low performing teams, but high performing teams do well with both. She also discusses impact of culture on DevOps, how high performing teams may not be as susceptible to burnout and how to think about individual developer productivity.   

At a workshop on Continuous Software Engineering, Dr. Eben Haber from Couchbase and author of  the book "Taming Information Technology", discussed his work and answered our class questions about his ethnographic studies of work practices within complex socio-technical systems, in particular the studies of system administrators.  

At a workshop on Continuous Software Engineering (part of a Course on Empirical Software Engineering at the University of Victoria), Dr. Chandra Maddila from Microsoft Research discusses AIOps with our class.  

Chandra answers questions about how AI and machine learning can improve developer productivity, and how techniques used in the Sankie platform could apply to other companies.    

In this week's recorded live lecture at a Senior Topics Course in Empirical Software Engineering at the University of Victoria, Dr. Elise Paradis demystifies the differences between objectivist deductive research and subjective inductive research, shares how theories help bridge the tension between engineering and science, and with the course participants discusses why we should do research that is both personally and internally coherent.   

Conducted live during a Senior Topics Course on Empirical Software Engineering at the University of Victoria, Sept 25th, 2020.  

Dr. Rashina Hoda presents a concise and clear overview of grounded theory in this recorded talk.  Her talk (after 36 or so minutes)  is followed by an interview (Q&A)  where Rashina answers, in a very thoughtful and clear way, practical questions about how she came to use grounded theory in her research, how to do grounded theory in a rigorous manner (and publish along the way), how to mitigate research biases, which coding tools she uses and why, and she shares advice with us on how to do grounded theory during this time of work/learn from home.