an evaluation of the Our minds matter program: assessing positive coping, peer support, resource awareness, and prosocial behavior

Shane McCarty, PHD & Samantha Burns

Summary of the Research

Our Minds Matter is an upstream suicide prevention model based on resiliency research and the power of peer-to-peer influence among adolescents. Through student-led activities and campaigns, OMM clubs aim to improve four outcome areas: 1) help-seeking behavior, 2) positive coping skills, 3) social connectedness and 4) pro-social skills. In this study, we surveyed OMM members and leaders at the start and end of the 2019 – 2020 school year. Participants scored higher at posttest on positive coping and perception of resource awareness in comparison to their pretest scores, but their scores on peer support and preventive prosocial behaviors did not improve. This study provides some evidence for the positive impact of the Our Minds Matter program on two of the four outcomes. These results should be interpreted with caution given the small sample size (n=50), limitations associated with the research design (e.g., no control group) and the context of the COVID-19 crisis during the second assessment period. The highlighted results of the research study are shown below.

 

Summary of the Methods and Results

Outcome #1: Positive Coping

Participants were prompted with the question: “How often do you use a positive coping skill when stressed?” The survey also included examples of positive coping strategies: Use art, yoga, meditation, exercise, listen to music, practice kindness, practice good sleeping, healthy eating and reasonable technology use. The use of positive coping strategies was measured using five response options: never used these skills, rarely used these skills, occasionally used these skills, frequently used these skills, and always used these skills.

 

plot of chunk sum likert coping plot

 

Outcome #2: Social Support

Participants were prompted with the following statement: “Think of your sense of connection to OMM peers.” Perceived social support from OMM members was measured on a seven-point likert scale: strongly disagree, moderately, disagree, slightly disagree, neither agree nor disagree, slightly agree, moderately agree, strongly agree.

plot of chunk sum likert support plot

 

Awareness of Mental Health Resources

Participants were prompted with the following statement: “Think of your awareness and confidence related to mental health resources at your school. Report how much you agree with the following statements.” The one resource item measured at pretest and posttest (for inclusion in this study) was: “I am aware of the mental health resources available online.” Perceived awareness of mental health resources was measured on a seven-point likert scale: strongly disagree, moderately, disagree, slightly disagree, neither agree nor disagree, slightly agree, moderately agree, strongly agree.

 

plot of chunk sum likert resources plot

 

Preventive Prosocial Behaviors

Students read the statement: “When I see friends who are struggling with mental health issues, I notice the signs and say something before it is a bigger problem or crisis.” Preventive Prosocial behavior was measured on a five-point scale (never, rarely, occasionally, frequently, and always).

 

plot of chunk sum likert prosocial plot

 

Review of the Results

As shown in the four plots above, participants scored higher at posttest on positive coping and perception of resource awareness in comparison to their pretest scores. To view the full research study evaluating the Our Minds Matter program, continue below.

 

 

The Research Study

 

Abstract

Our Minds Matter is an upstream suicide prevention model based on resiliency research and the power of peer-to-peer influence among adolescents. Through student-led activities and campaigns, OMM clubs aim to improve four outcome areas: 1) help-seeking behavior, 2) positive coping skills, 3) social connectedness and 4) pro-social skills. In this study, we evaluated the impact of the OMM program on these outcomes using a one-group, pretest-posttest design. Fifty high school students in the OMM program completed an online survey in the Fall of 2019 and Spring of 2020 (during COVID-19). Four one-tailed, paired-sample t-tests were conducted to test differences between pretest and posttest scores on the four outcomes. Participants scored higher at posttest on positive coping and perception of resource awareness in comparison to their pretest scores. The mean differences at pretest and posttest were not different for peer support and preventive prosocial behaviors. This study provides some evidence for the positive impact of the Our Minds Matter program on two of the four outcomes. These results should be interpreted with caution given the the small sample size (n=50), limitations associated with the research design (e.g., no control group) and the context of the COVID-19 crisis during the second assessment period.

 

Introduction

The Our Minds Matter (OMM) program empowers high school students to change their school culture around mental health. A typical high school OMM club has three to five engaged student leaders who host weekly, bi-weekly or monthly meetings where they facilitate activities and discussion among their peers (see OMM Activities). Clubs are overseen by a school faculty sponsor(s) who is a school psychologist, social worker, counselor and/or teacher. A strong membership base ranging from five to forty or more students participate in activities developed by mental health professionals around OMM’s four outcome areas: 1) help-seeking behavior, 2) positive coping skills, 3) social connectedness and 4) pro-social skills (see OMM Activities). To create school-wide impact, OMM leaders and members engage in direct dyadic prosocial behaviors to impact students at school and indirect school-wide prosocial behaviors to improve the school culture.

The Current Study

The current study assesses the impact of the Our Minds Matter program using a one-group pretest-posttest design.

Research Questions

 

Methods

Participants

OMM club members were invited to participate in a pretest survey on demographics and program outcomes from August 12th, 2019 to September 30th, 2019 and a posttest survey from May 1st, 2020 to June, 30th 2020. Two-hundred and twenty students completed the pretest surveys and 174 students completed the posttest survey. At pretest and posttest, students provided permission for the data to be used by the OMM research team to improve the program and evaluate the effects. Students also had the option to provide their email address, which was used for matching scores at pretest and posttest. Fifty participants were matched via email address to combine their pretest and posttest scores into a new dataset used for these analyses.

Measures

Outcome Variables

Positive Coping

The use of positive coping strategies was measured based on these five response options: never used these skills, rarely used these skills, occasionally used these skills, frequently used these skills, always used these skills. Participants were prompted with the question: “How often do you use a positive coping skill when stressed?” The survey also included examples of positive coping strategies: Use art, yoga, meditation, exercise, listen to music, practice kindness, practice good sleeping, healthy eating and reasonable technology use.

Social Support

Perceived social support from OMM members was measured on a seven-point likert scale: strongly disagree, moderately, disagree, slightly disagree, neither agree nor disagree, slightly agree, moderately agree, strongly agree. Participants were prompted with the following statement: “Think of your sense of connection to OMM peers.” The one peer support item measured at pretest and posttest (for inclusion in this study) was: “I feel connected to OMM members, who I can talk to if I have a problem.”

Awareness of Mental Health Resources

Perceived awareness of mental health resources was measured on a seven-point likert scale: strongly disagree, moderately, disagree, slightly disagree, neither agree nor disagree, slightly agree, moderately agree, strongly agree. Participants were prompted with the following statement: “Think of your awareness and confidence related to mental health resources at your school. Report how much you agree with the following statements.” The one resource item measured at pretest and posttest (for inclusion in this study) was: “I am aware of the mental health resources available online.”

Preventive Prosocial Behaviors

Preventive Prosocial behavior was measured on a five-point scale (never, rarely, occasionally, frequently, and always) based on the following statement: “When I see friends who are struggling with mental health issues, I notice the signs and say something before it is a bigger problem or crisis.”

Exploratory Variables

OMM Leader

OMM Leader was assessed by asking: Are you a club leader? The response options were: yes (1) or no (0). Respondents were labeled as “OMM Leader” or “OMM Member”.

School District

School District was assessed by asking: What school do you go to? The response options were a list of schools with OMM clubs. Respondents were labeled as “FCPS Schools” (1) or “Non-FCPS School” (0) based on what schools are in the FCPS school district.

Meetings

Seven_Meetings variable was assessed by asking: How many meetings did you attend? Respondents who attended 7 or more meetings for the year were coded as “1” and those who attended ‘less than 7 meetings’ were coded as “0”.

Statistical Analyses

The data was collected using Survey Monkey, cleaned on Microsoft Excel for Mac v16.40, and analyzed using 4.0.2 and RStudio v1.3.1056. The HTML output was created using “rmarkdown”, “markdown”, and “knitr” packages in RStudio along with two applications: LaTeXiT for Mac and XQuartz. The purpose of using the aforementioned software was to create a fully transparent and reproducible research study. The data file (.xlsx), R script (.R), R notebook (.Rmd), and related documents are available at the OMM project GitHub.

Four one-tail (i.e., pretest > postest) paired t-tests to test the hypotheses. The null hypothesis is no difference in mean scores between pretest and posttest (Ho = µ0 = 0). The alternative hypothesis is the mean pretest score is less than mean posttest score (Ho = µ0 < 0). To reduce the possibility of a Type I error, a Bonferroni correction was made, resulting in an alpha of .0125 for each of the four statistical tests.

 

Results

Participant Characteristics

School District

Forty-one of the fifty respondents (82%) attended a high school in Fairfax County Public Schools (FCPS).

 

Count of respondents for each school, categorized by FCPS district or non-FCPS schools

 

Race/Ethnicity and Gender

An exploration of race/ethnicity and gender shows a majority of participants identified as “white/caucasian/european” (n = 26) and cisgender female (n= 49), resulting in 50% of the sample identifying as both “white/caucasian/european” and “cisgender female”.

Percentage of each gender identity of respondents by racial/ethnic group with 98% of respondents identifying as Cisgender Female.

 

The highest percentage of OMM leaders joined the OMM club at their school in Fall 2018, whereas the highest percentage of OMM members joined at the start of Fall 2019.

 

Percentage of respondents for Member vs. OMM Leader by the semester they joined OMM.

 

Summary of Outcomes

Histograms

Histograms are provided for each of the eight outcome variables. Four of the outcomes are negatively skewed.

plot of chunk histogram outcomes

 

Table of Outcomes

MeanStd.DevMinMedianMaxN.ValidPct.Valid
Post.Coping4.04000.7273134.0550100
Post.Preventive.Prosocial4.02000.5886734.0550100
Post.Resources6.08000.8998946.0750100
Post.Support5.36001.3815125.5750100
Pre.Coping3.80000.7824634.0550100
Pre.Preventive.Prosocial3.91840.7313434.054998
Pre.Resources5.60001.3850516.0750100
Pre.Support5.64001.3815126.0750100

 

Correlations Among Outcomes

plot of chunk ggcorrmat all

 

Correlations Among Pretest Outcomes and Exploratory Variables

plot of chunk ggcorrmat pre

 

Correlations Among Posttest Outcomes and Exploratory Variables

plot of chunk corr post

Individual Outcomes

Outcome #1: Positive Coping

Three plots are used to depict the data for positive coping scores at pretest and posttest: barplot of likert responses, boxplot, and barplot of mean scores. Results of the paired t-test are also shown.

 

The plot depicts the percentage per response option at pretest and posttest. The percentage for the combined total of never and rarely is shown on the left side. The percentage for the combined total of frequently and always is on the right side.

 

 

The boxplot depicts the distribution and skewness of data, including the min, lower quartile, median, upper quartile, and max values.

The barplot depicts the mean of the outcome variable at pretest and posttest.

 

Analyses

A paired-samples t-test was conducted to compare healthy coping use at pretest and posttest. There was a significant difference in the scores at pretest (M= 3.80, SD= .782) compared to posttest (M= 4.04, SD= .727); t(49)= -2.370, p= .011.

## 
## 
## Table 2 
## 
## Means, standard deviations, and d-values with confidence intervals
##  
## 
##   Variable    M    SD   1            
##   1. pretest  3.80 0.78              
##                                      
##   2. posttest 4.04 0.73 0.32         
##                         [-0.08, 0.71]
##                                      
## 
## Note. M indicates mean. SD indicates standard deviation. d-values are estimates calculated using formulas 4.18 and 4.19
## from Borenstein, Hedges, Higgins, & Rothstein (2009). d-values not calculated if unequal variances prevented pooling.
## Values in square brackets indicate the 95% confidence interval for each d-value. 
## The confidence interval is a plausible range of population d-values 
## that could have caused the sample d-value (Cumming, 2014). 
## 
## t(49) = -2.37, p = .011, d = -0.34

 

Outcome #2: Support

Three plots are used to depict the data for peer support at pretest and posttest: barplot of likert responses, boxplot, and barplot of mean scores. Results of the paired t-test are also shown.

 

The plot depicts the percentage per response option at pretest and posttest. The percentage for the combined total of disagreement responses is shown on the left side. The percentage for the combined total of agreement responses is on the right side.

 

As shown above, 18% of respondents who “occasionally” use positive coping skills at pretest reported more use of positive coping skills at posttest.
As shown above, the percentage of respondents who had a strong or moderate agreement decreased from pretest to posttest and the percentage who “slightly agree” increased at posttest.

 

The boxplot depicts the distribution and skewness of data, including the min, lower quartile, median, upper quartile, and max values.

 

The barplot depicts the mean of the outcome variable at pretest and posttest.

 

Analyses

A paired-samples t-test was conducted to compare peer support at pretest and posttest. The pretest score (M= 5.64, SD= 1.382) was not significantly less than the score at posttest (M= 5.36, SD= 1.382); t(49)= 1.549, p= .936.

## 
## 
## Table 2 
## 
## Means, standard deviations, and d-values with confidence intervals
##  
## 
##   Variable    M    SD   1            
##   1. pretest  5.64 1.38              
##                                      
##   2. posttest 5.36 1.38 0.20         
##                         [-0.19, 0.60]
##                                      
## 
## Note. M indicates mean. SD indicates standard deviation. d-values are estimates calculated using formulas 4.18 and 4.19
## from Borenstein, Hedges, Higgins, & Rothstein (2009). d-values not calculated if unequal variances prevented pooling.
## Values in square brackets indicate the 95% confidence interval for each d-value. 
## The confidence interval is a plausible range of population d-values 
## that could have caused the sample d-value (Cumming, 2014). 
## 
## t(49) = 1.55, p = .936, d = 0.22

 

Outcome #3: Resources

Three plots are used to depict the data for resource awareness at pretest and posttest: barplot of likert responses, boxplot, and barplot of mean scores. Results of the paired t-test are also shown.

 

The plot depicts the percentage per response option at pretest and posttest. The percentage for the combined total of disagreement responses is shown on the left side. The percentage for the combined total of agreement responses is on the right side.

 

As shown above, 10% more respondents selected an agreement statement (slightly, moderately, strongly) at posttest in comparison to pretest. Also noteworthy, none of the respondents disagreed at posttest.

 

The boxplot depicts the distribution and skewness of data, including the min, lower quartile, median, upper quartile, and max values.