Abstract

Content and trigger warnings give information about the content of material prior to receiving it and are used by social media users to tag their content when discussing sensitive topics. Trigger warnings are known to yield benefits in terms of an increased individual agency to make an informed decision about engaging with content. At the same time, some studies contest the benefits of trigger warnings suggesting that they can induce anxiety and reinforce the traumatic experience of specific identities. Our study involves the analysis of the nature and implications of the usage of trigger warnings by social media users using empirical methods and machine learning. Further, we aim to study the community interactions associated with trigger warnings in online communities, precisely the diversity and content of responses and inter-user interactions. The domains of trigger warnings covered will include self-harm, drug abuse, suicide, and depression. The analysis of the above domains will assist in a better understanding of online behaviour associated with them and help in developing domain-specific datasets for further research.

The problem

Trigger warnings let readers decide whether to engage with distressing content, but the evidence on their effect is mixed: some studies find they increase agency, others that they centre trauma and can make things worse. There is no agreed taxonomy of what deserves a warning, and no exhaustive list of triggers.

Prior computational work on trigger warnings is limited to assigning warnings to fan fiction. We were not aware of any study of how social media users themselves tag content with trigger warnings, what those posts talk about, or how communities respond. This paper is an initial study of that question, using Reddit data across four sensitive domains.

Approach

  • Dataset. Using the PRAW client we collected posts from r/selfharm and r/SelfHarmScars (self-harm), r/SuicideWatch (suicide), r/Depression and r/MentalHealth (depression), and r/RedditorsInRecovery (drug abuse), from 2013 to 2022. A search phrase combines trigger-warning mentions (“trigger warning”, “tw”, “TW”, “Trigger Warning”) with a curated keyword list per community, matched in the title or body. The final dataset has 4,322 self-harm, 7,090 suicide, 11,606 depression, and 2,437 drug-abuse posts, with metadata such as score, comment count, flair, and NSFW status. HTML, URLs, emoticons, and special characters are removed.
  • Exploratory analysis. VADER sentiment over the compound score, a trend analysis of posting frequency by year, and KeyBERT keyphrase extraction (top 5 keywords per corpus by cosine similarity to the document embedding).
  • Topic modelling. BERTopic with MiniLM-L6 embeddings, UMAP, HDBSCAN, and c-TF-IDF, evaluated with UMass, C_V, and C_UCI coherence and with topic diversity and inverted rank-biased overlap (IRBO) through the OCTIS framework.
  • Intent classification. Link flair text (questions, advice requests, rants, and so on) is binned into semantically similar classes; untagged posts are labelled manually. Post embeddings from MiniLM-L6 are concatenated with metadata (comments, likes, score, upvote ratio) and fed to XGBoost and LightGBM classifiers under 10-fold stratified sampling with a 70:30 split.
Reddit posts PRAW, 2013-2022 "tw" + keyword 25,455 posts Clean + embed strip HTML, URLs, emoticons MiniLM-L6 (384-d) r/SuicideWatch r/Depression BERTopic UMAP + HDBSCAN + c-TF-IDF KeyBERT + VADER top-5 keyphrases, sentiment, trends + metadata comments, score, upvote ratio Topic modelling coherence, diversity Keyword analysis per community Intent classifier XGBoost / LightGBM F1 0.86-0.92 4 communities
Posts tagged with trigger warnings are pulled from six subreddits, cleaned and embedded with MiniLM-L6; the same embeddings drive BERTopic topic modelling, keyphrase and sentiment analysis, and, joined with post metadata, an intent classifier.

Example

Two posts from the paper’s appendix, printed there as the highest-similarity matches for the KeyBERT keywords of their community.

Input: post from the depression corpus

“I am tired of fighting. My depression, anxiety, PTSD, and being jobless are destroying me.”

Output: top KeyBERT keyword

ptsd (similarity 0.8209), the highest-ranked keyphrase for the depression community; VADER sentiment for this community is predominantly negative.

Input: post from the suicide corpus (trimmed)

“trigger warning for mentions of suicide and sexual assault possible spoilers for cyberpunk … im posting this here instead of on the cyberpunk subreddit because this has less to do with the game and more to do with my reaction to it …”

Output: top KeyBERT keyword

game (similarity 0.8591), the highest-ranked keyphrase for the suicide community, which the paper reads as users discussing narratives from games, films, and shows rather than only personal crises.

Results

  • Sentiment. The drug-abuse community has a strikingly high share of positive-sentiment posts, the depression community the highest share of negative sentiment, and the suicide community the highest share of neutral posts (often discussing narratives from films or laws rather than personal crises).
  • Trends. Trigger-warning posting rises sharply from 2019 onward in all four communities, coinciding with COVID-19 isolation.
  • Keyphrases. Top KeyBERT terms include “fun”, “anxiety”, “relapse”, and “blades” for self-harm; “ptsd”, “desperate”, “relapsed” for depression; “relapse”, “craving”, “withdrawal” for drug abuse; and “game”, “hurts”, “cut”, “urgent”, “help” for suicide.
Intent classification Accuracy Precision Recall F1
Self-harm, BERTopic + XGBoost 0.895 0.946 0.835 0.887
Self-harm, BERTopic + LightGBM 0.899 0.948 0.857 0.900
Suicide, BERTopic + XGBoost 0.920 0.944 0.890 0.916
Suicide, BERTopic + LightGBM 0.910 0.948 0.883 0.914
Depression, BERTopic + XGBoost 0.930 0.947 0.887 0.912
Depression, BERTopic + LightGBM 0.920 0.945 0.879 0.910
Drug abuse, BERTopic + XGBoost 0.890 0.930 0.796 0.857
Drug abuse, BERTopic + LightGBM 0.900 0.928 0.798 0.858

Source: Table 4 of the paper. Multi-class post-intent classification on held-out folds (10-fold stratified sampling, 70:30 split); bold marks the better model per corpus. Higher is better.

Topic model quality Depression Drug abuse Self-harm Suicide
UMass coherence -0.562 -0.693 -0.532 -0.631
C_V coherence 0.563 0.701 0.567 0.642
C_UCI coherence 0.547 0.701 0.511 0.621
Topic diversity 0.800 0.844 0.878 0.881
IRBO 0.821 0.854 0.882 0.884

Source: Tables 2 and 3 of the paper. BERTopic with MiniLM-L6 embeddings, evaluated through OCTIS. For C_V, C_UCI, diversity, and IRBO higher is better.

  • Topic quality. Drug abuse has the highest coherence (C_V 0.701, C_UCI 0.701), followed by suicide (C_V 0.642). Topic diversity is above 0.8 on every corpus, with IRBO between 0.821 and 0.884, so the topics are distinct and cover a wide range of themes.
  • Classification. BERTopic embeddings plus XGBoost or LightGBM reach accuracy of 0.89-0.93 and F1 of 0.86-0.92 across the four communities, with the two gradient-boosting models performing evenly.

Dataset

  Self-harm Suicide Depression Drug abuse
Posts 4,322 7,090 11,606 2,437
Average sentences per post 4.70 6.55 7.54 8.09
Mean proportion of unique tokens 0.653 0.651 0.509 0.562

Source: Table 1 of the paper. Posts collected from r/selfharm, r/SelfHarmScars, r/SuicideWatch, r/Depression, r/MentalHealth, and r/RedditorsInRecovery, 2013-2022, with metadata (score, comment count, flair, NSFW flag).

The paper states that the dataset will be released after acceptance. Future work named in the paper includes extending to Twitter, other languages, audio-visual modalities, and demographic analysis.

Resources