Tone of Voice Trumps Word Choice in Spousal Arguments

By Rick Nauert PhD

An interdisciplinary team has developed a computer algorithm that can predict, with high levels of accuracy, if your relationship with your spouse will improve or deteriorate.

University of Southern California researchers say the software is 79 percent accurate. In fact, the algorithm did a better job of predicting marital success of couples with serious marital issues than descriptions of the therapy sessions provided by relationship experts.

Study results are reported in the journal Proceedings of Interspeech.

Researchers recorded hundreds of conversations from more than 100 couples taken during marriage therapy sessions over two years, and then tracked their marital status for five years. Drs. Shrikanth Narayanan and Panayiotis Georgiou of the University of Southern California Viterbi School of Engineering with doctoral student M.D. Nasir and collaborator Dr. Brian Baucom of University of Utah lead the research effort.

From the data gathered, investigators then developed an algorithm that broke the recordings into acoustic features using speech-processing techniques. These included pitch, intensity, “jitter” and “shimmer” among many; things like tracking warbles in the voice that can indicate moments of high emotion.

“What you say is not the only thing that matters, it’s very important how you say it. Our study confirms that it holds for a couple’s relationship as well,” Nasir said.

Taken together, the vocal acoustic features offered the team’s program a proxy for the subject’s communicative state, and the changes to that state over the course of a single therapy and across therapy sessions.

The innovative research looked at vocal patterns and inflections over time.

That is, the vocal signatures were not analyzed in isolation; rather, the impact of one partner upon the other and the vocal tone was studied over multiple therapy sessions.

“It’s not just about studying your emotions,” Narayanan said. “It’s about studying the impact of what your partner says on your emotions.”

“Looking at one instance of a couple’s behavior limits our observational power,” Georgiou said.

“However, looking at multiple points in time and looking at both the individuals and the dynamics of the dyad can help identify trajectories of the their relationship. Sometimes those are for the best or sometimes they are towards relationship deterioration.”

The power of such methods is to help identify how domain experts can better advise couples towards improved relationships, Georgiou said.

“Psychological practitioners and researchers have long known that the way that partners talk about and discuss problems has important implications for the health of their relationships. However, the lack of efficient and reliable tools for measuring the important elements in those conversations has been a major impediment in their widespread clinical use.

“These findings represent a major step forward in making objective measurement of behavior practical and feasible for couple therapists,” Baucom said.

Once it was fine-tuned, the program was then tested against behavioral analyses made by human experts who studied the couples. Those behavioral codes (positive qualities like “acceptance” or negative qualities like “blame”).

The team found that studying voice directly — rather than the expert-created behavioral codes — offered a more accurate glimpse at a couple’s future.

Researchers now plan to use behavioral signal processing — a framework for computationally understanding human behavior — to improve the prediction of how effective treatments will be.

This will entail the analysis of how language (e.g., spoken words) and nonverbal information (e.g., body language) influence the effectiveness of therapy.

Source: University of Southern California/EurekAlert