How lessons learned from journalism can help improving our software testing
You cannot not communicate and even the most introverted programmer or tester has to do it all the time. But sometimes we forget about our main communication partner.
Which in software testing is our system-under-test - and we should start to behave accordingly.
Testing is communication
Let’s see whether this is true, starting with the definition of communication from Oxford Languages:
“Communication is the imparting or exchanging of information by speaking, writing, or using some other medium.” - Sounds true to me, but let’s take a closer look.
The following picture shows the essentials of a communication, leaving away some of the “personal” problems like wrong assumptions, misunderstanding, miscommunication, unhealthy relation, and other tensions, as these are often the results of emotions between the communicators. (Which so far a system does not display. Therefore I may exclude these for now.).
So, if I need some information from a friend, for example where we are going to meet to have a beer, I would just ask him or her, preferably in a language he or she understands.
“¿Dónde nos encontramos para tomar una cerveza?”, no es una buena pregunta si mi amigo no habla español.”
Me, as a sender, would talk to my friend in a form he or she understands and most likely I will get some response, or scientifically spoken, feedback. In case my friend understood my question and answered correctly we now have an appointment for having a beer.
Software testing to me looks similar to the model in our picture.
We, as testers, want some information from our system under test. This could be the current quality level, some defect or, in case our test basis is not complete or in good shape, simply how it works. In order to get this information we must “ask” the system under test questions it understands and that fulfill our or some other stakeholders’ information needs. Normally these “questions” are our test case. No matter if preformulated or designed spontaneously as in an exploratory testing session.
The system under test then, hopefully, gives us feedback in form of expected or unexpected behavior, which we have to analyze and interpret, as our system is not as thoughtful as we testers are and does not always speak our language.
Thus, we are having a rudimentary communication with this system. (Although this might change in the future thinking of AI or robots.)
But it is some special kind of conversation. It is a, more or less, formal meeting of someone, the tester, interested in the other one, the system, asking lots of interesting questions in order to gather the desired information – it is an interview with the system.
If testing is communication - it is an interview
So if testing is like an interview – can we as testers profit from the lessons learned by professional journalists conducting interviews frequently?
I thought this would be interesting and I did some research on what tips journalists are giving each other on how to successfully execute interviews revealing interesting information.
And I really found some advice from this craft, that can be applied in the testing domain and perhaps help us either in reinforcing good practices already at our hands or adding a new perspective on our discipline.
Prepare carefully, familiarizing yourself with as much background as possible.
First, and like with almost all tasks at hand, preparation is everything.
Even without a special testing task at hand, make yourself familiar with your system, the domain, and your context in general, as a journalist checks his/her interview partner and background beforehand. Read, research, and keep your eyes open all the time.
It is straightforward to know which tasks and processes the anticipated users will use the system for – this is the basis for your test cases. But in addition you should find out how similar systems have implemented these tasks to identify areas of improvement or distinction, or imagine what can go wrong.
While this advice might still seem a normal task of our testing, the following one is not that obvious. Be always up-to-date on what is going on in the project, as recent events, incidents or the general progress can give you starting points and hints for experience-based testing techniques. As these techniques highly rely on the experience, skill(s) and intuition of a tester, an unblurred and wakeful look at the project and some additional research can, at least, compensate.
Prepare your goals ahead of time
 | The next one is about focus and direction - this is true for a successful interview as well as for a successful testing session. If you have no idea what you want to find out, you will never find it. Before you start testing you should prepare your goals for the testing session. This helps a lot to sustain your focus and will direct your testing towards success. |
Write down your questions (or test cases)
The last preparation step is to prepare the questions you want to ask your interview partner - your test cases. May it be a formal effort with pre-conditions, test steps, expected results and post-conditions filed in your test management tool, or a collection of test ideas and conditions prepared as a checklist or an exploratory testing charter. Every minute you put in the preparation upfront – is a minute more for test execution and observation. And it intensifies your, already mentioned, focus in the testing session. |  |
Ask questions that are relevant to the source and that induce the source to talk.
Work on your flow.
 | This is one of the most challenging and important skills for a journalist interviewer - to strike a balance between a conversation (which helps make your subject feel comfortable and aids openness) and getting the job done. As your subject is answering your question, be thinking about what you’ll ask next and why. This sounds a bit awkward for us. Making our system FEEL comfortable might be something not applicable for us as SYSTEM Interviewers. I first wanted to skip it, but came to the conclusion that this skill is really important for us as testers, too. I believe it is essential and challenging to bring some balance in our testing effort, for example we have to adjust or test intensity based on our findings. |
Sometimes we test shallow and broad to get an overview over the system and its quality. Then we concentrate on an area of high risk. One time we just test for coverage, ticking the passed test cases from our test suite and another time we dig deep to find the cause for a certain symptom or to analyze an interesting defect.
So for testers as for journalists it is a good idea to optimize the flow. Journalists for example start with questions to comfort their communication partner to build trust and keep the tougher ones for later when a connection is established. We as testers should do something similar and adjust our testing based on the situation in a playful manner.
Avoid obsessing
Testers like journalists do not conduct their interviews for their own sake, but to gather interesting information on their observation target and deliver the findings as information of interest to another group of interested recipients and stakeholders. Therefore they have to record it as accurately as possible. But we shouldn’t be obsessed with recording every detail, because then we might miss some interesting gem our interview partner, respectively the system under test offers us. Remember that the most interesting defects often have the slightest observable symptoms |  |
Be a little annoying. / Be a little sneaky.
 | And in order to find these we should use every trick we can think of. Journalists in this case try to be a little annoying or sneaky. Insisting on answers their interview partner isn’t willing to give. We as testers have other possibilities. Working with longer breaks than expected. Doing the unexpected but possible (and if something is possible some user will do it). Or force timing issues, or, or, or. The list is almost endless and you all know some special treatment to provoke your system to show some interesting reactions. So be like a journalist that wants to find out more - annoying and a bit sneaky. |
Ask open-ended questions, and allow them to drift away (a bit)
An interviewer gets the most interesting answers if he/she asks open-ended questions. Answers to these questions normally drift away a bit from the topic and reveal information that might be surprising, but at least more interesting than yes or no. In testing this is pretty much the same. Of course can tick off pass or fail observing expected and actual results and this gives some information about the state of the system. But the more interesting information is often revealed by test ideas like:
|  |
Listen and watch attentively.
Pay attention to the answers you get
 | For journalists it is important to pay attention to the answers he/she gets. And these answers are in human communication more than words. Think of facial expressions, a change in the tone of voice or other non-verbal communication. Testers do not have a human as their interview partner, but a system-under-test gives similar hints to us. A blink on the screen, a change in color or an error brings the system to an unforeseeable state. Leverage on such signs and dig deepe |
Listen to the little burst
All of us know the excitement when we feel that we have almost reached our goal - that is what I refer to as a little burst. And this is the time when good interviewers get into their top form digging deeper to the core of the interview. A good advice for us testers,too. As soon as you feel the little burst, concentrate and amplify your testing intensity - it will pay off! |  |
Retrospect and improve
 | Everyone with pride in his/her craftsmanship seeks ways to excel in the discipline. For this sake journalists observe themselves like rats in the lab - they record their interviews and transcribe them in whole to analyze the flow and identify techniques that let their counterparts open up. We as testers can do the same. Analyze our test cases and charters and take additional notes to find ways to improve and get more information from our systems. To be the best tester you can be, study yourself and let your failures and victories lead you to better testing and more interesting results. |
The four principles of interviews
The system you are testing is one of your main communication partners and you are normally having an interview with it – so leverage also on techniques from journalism to get the best out of your testing. At the end, I would like to summarize by quoting Stephen D. Isaacs, a prominent journalist and professor at Columbia University with his four principles of interviews:
|  |
- Listen and watch attentively.
- Establish a relationship with the source conducive to obtaining information.
The attentive reader (or tester)might have recognized that I didn’t have content for the last principle mentioned. This is because with the current systems under test we don’t need to empathize.
But this will change the latest in the moment we have to test this sympathetic guy here - and what would you first ask him?
