Twitter photos may help detect users with depression, anxiety

Depression and anxiety can affect all aspects of a person’s life, even the types of images they share on social media, a new study from the University of Pennsylvania suggests.

“When people are out of words to explain how they feel, they use images,” said Sharath Guntuku, a coauthor of the study and a research scientist with Penn Medicine’s Center for Digital Health. “And some of those images are incredibly telling.”

Guntuku and his colleagues gathered thousands of tweets from about 4,000 Twitter users who agreed to share their data. They then analyzed the images users posted, as well as their profile pictures. Although previous studies have looked at identifying depression through social media text, this study focused solely on images. A subset of users also completed depression and anxiety surveys.

Using machine learning, researchers found that certain features in users’ photos could predict a higher depression or anxiety score. These included less vivid, dim, or grayscale photos; less symmetry; and less depth of field.

People with depression or anxiety were also less likely to have other people in their photos or to post images showing themselves doing recreational activities like sports, being outdoors, or traveling.

People with depression tend to withdraw from others and be less expressive, Guntuku said. That might be reflected in the photos.

All of the findings were stronger for identifying depression than anxiety. Researchers are not entirely sure why, but Guntuku suggested anxious people may feel more pressure to meet societal expectations of happiness. They might refrain from posting images that would suggest sadness or worry.

In general, researchers found few images with frowning or unhappy expressions, especially in profile pictures. Instead, people with symptoms of depression or anxiety were more likely to have straight faces.

“You might not feel happy,” Guntuku said, “but social desirability comes in, and you don’t want to post a sad face of yourself.”

The research, which will be presented at the International Conference on Web and Social Media in Germany next month, is still in very early stages, Guntuku said. The findings need to be tested with more users and on other social platforms before researchers can consider building a formal assessment tool.

While more than half of all tweets contain an image, platforms like Instagram and Snapchat are even more visual. Guntuku and his colleagues are currently working on another study comparing users’ Instagram posts with mental health diagnoses in their medical records. This will be a more telling study, as formal diagnoses provide a higher degree of accuracy than depression and anxiety surveys users complete themselves.

In the future, Guntuku sees machine learning algorithms being used to scan users’ social media feeds and present risk scores.

“We want to build a risk assessment score rather than bucketing someone as depressed or not,” he said. That determination should be made by a doctor.

But prescreening this way would save medical professionals time and identify more people in need of treatment.

National statistics show more than half of American adults and 80 percent of children with mental-health needs do not receive treatment. A tool like this could prioritize those at greatest risk — a crucial factor given the lack of mental health providers and appointment wait times that can stretch into weeks or months.

“Providers don’t have time to go through the entire social media feed, so it’s up to researchers to summarize this in an efficient manner,” Guntuku said.

While his research focuses on the potential helpful uses of social media, Guntuku knows many also see it as a culprit in poor mental health. Research on the issue is still divided. Some studies have shown that those who spend more time on social media are more likely to be depressed, while others indicate that people who are already depressed use social media more.

Twitter and Instagram have recently taken steps to address such concerns, piloting tests that hide the number of retweets or likes users receive. Supporters say this could reduce social media pressure and competition that harms self-esteem, especially for youth.

Guntuku said that might reduce the spread of certain types of images or messages, but the relationships between social media and mental health are complex.

“If you’re depressed, you’re likely to use technology in a certain way. And the more you use it in a certain way, the more it may impact your mental health,” he said.

He hopes his research can help find a way to break the cycle.