an evaluation of the Our minds matter program: assessing positive coping, peer support, resource awareness, and prosocial behavior
Shane McCarty, PHD & Samantha Burns
Summary of the Research
Our Minds Matter is an upstream suicide prevention model based on resiliency research and the power of peer-to-peer influence among adolescents. Through student-led activities and campaigns, OMM clubs aim to improve four outcome areas: 1) help-seeking behavior, 2) positive coping skills, 3) social connectedness and 4) pro-social skills. In this study, we surveyed OMM members and leaders at the start and end of the 2019 – 2020 school year. Participants scored higher at posttest on positive coping and perception of resource awareness in comparison to their pretest scores, but their scores on peer support and preventive prosocial behaviors did not improve. This study provides some evidence for the positive impact of the Our Minds Matter program on two of the four outcomes. These results should be interpreted with caution given the small sample size (n=50), limitations associated with the research design (e.g., no control group) and the context of the COVID-19 crisis during the second assessment period. The highlighted results of the research study are shown below.
Summary of the Methods and Results
Outcome #1: Positive Coping
Participants were prompted with the question: “How often do you use a positive coping skill when stressed?” The survey also included examples of positive coping strategies: Use art, yoga, meditation, exercise, listen to music, practice kindness, practice good sleeping, healthy eating and reasonable technology use. The use of positive coping strategies was measured using five response options: never used these skills, rarely used these skills, occasionally used these skills, frequently used these skills, and always used these skills.
Outcome #2: Social Support
Participants were prompted with the following statement: “Think of your sense of connection to OMM peers.” Perceived social support from OMM members was measured on a seven-point likert scale: strongly disagree, moderately, disagree, slightly disagree, neither agree nor disagree, slightly agree, moderately agree, strongly agree.
Awareness of Mental Health Resources
Participants were prompted with the following statement: “Think of your awareness and confidence related to mental health resources at your school. Report how much you agree with the following statements.” The one resource item measured at pretest and posttest (for inclusion in this study) was: “I am aware of the mental health resources available online.” Perceived awareness of mental health resources was measured on a seven-point likert scale: strongly disagree, moderately, disagree, slightly disagree, neither agree nor disagree, slightly agree, moderately agree, strongly agree.
Preventive Prosocial Behaviors
Students read the statement: “When I see friends who are struggling with mental health issues, I notice the signs and say something before it is a bigger problem or crisis.” Preventive Prosocial behavior was measured on a five-point scale (never, rarely, occasionally, frequently, and always).
Review of the Results
As shown in the four plots above, participants scored higher at posttest on positive coping and perception of resource awareness in comparison to their pretest scores. To view the full research study evaluating the Our Minds Matter program, continue below.
The Research Study
Abstract
Our Minds Matter is an upstream suicide prevention model based on resiliency research and the power of peer-to-peer influence among adolescents. Through student-led activities and campaigns, OMM clubs aim to improve four outcome areas: 1) help-seeking behavior, 2) positive coping skills, 3) social connectedness and 4) pro-social skills. In this study, we evaluated the impact of the OMM program on these outcomes using a one-group, pretest-posttest design. Fifty high school students in the OMM program completed an online survey in the Fall of 2019 and Spring of 2020 (during COVID-19). Four one-tailed, paired-sample t-tests were conducted to test differences between pretest and posttest scores on the four outcomes. Participants scored higher at posttest on positive coping and perception of resource awareness in comparison to their pretest scores. The mean differences at pretest and posttest were not different for peer support and preventive prosocial behaviors. This study provides some evidence for the positive impact of the Our Minds Matter program on two of the four outcomes. These results should be interpreted with caution given the the small sample size (n=50), limitations associated with the research design (e.g., no control group) and the context of the COVID-19 crisis during the second assessment period.
Introduction
The Our Minds Matter (OMM) program empowers high school students to change their school culture around mental health. A typical high school OMM club has three to five engaged student leaders who host weekly, bi-weekly or monthly meetings where they facilitate activities and discussion among their peers (see OMM Activities). Clubs are overseen by a school faculty sponsor(s) who is a school psychologist, social worker, counselor and/or teacher. A strong membership base ranging from five to forty or more students participate in activities developed by mental health professionals around OMM’s four outcome areas: 1) help-seeking behavior, 2) positive coping skills, 3) social connectedness and 4) pro-social skills (see OMM Activities). To create school-wide impact, OMM leaders and members engage in direct dyadic prosocial behaviors to impact students at school and indirect school-wide prosocial behaviors to improve the school culture.
The Current Study
The current study assesses the impact of the Our Minds Matter program using a one-group pretest-posttest design.
Research Questions
- Do OMM participants use more healthy coping skills under stress at post-test than pre-test?
- Do OMM participants perceive more social support at post-test than pre-test?
- Are OMM participants more aware of mental health resources online at post-test than pre-test?
- Do OMM participants perform more preventive prosocial behaviors at post-test than pre-test?
Methods
Participants
OMM club members were invited to participate in a pretest survey on demographics and program outcomes from August 12th, 2019 to September 30th, 2019 and a posttest survey from May 1st, 2020 to June, 30th 2020. Two-hundred and twenty students completed the pretest surveys and 174 students completed the posttest survey. At pretest and posttest, students provided permission for the data to be used by the OMM research team to improve the program and evaluate the effects. Students also had the option to provide their email address, which was used for matching scores at pretest and posttest. Fifty participants were matched via email address to combine their pretest and posttest scores into a new dataset used for these analyses.
Measures
Outcome Variables
Positive Coping
The use of positive coping strategies was measured based on these five response options: never used these skills, rarely used these skills, occasionally used these skills, frequently used these skills, always used these skills. Participants were prompted with the question: “How often do you use a positive coping skill when stressed?” The survey also included examples of positive coping strategies: Use art, yoga, meditation, exercise, listen to music, practice kindness, practice good sleeping, healthy eating and reasonable technology use.
Social Support
Perceived social support from OMM members was measured on a seven-point likert scale: strongly disagree, moderately, disagree, slightly disagree, neither agree nor disagree, slightly agree, moderately agree, strongly agree. Participants were prompted with the following statement: “Think of your sense of connection to OMM peers.” The one peer support item measured at pretest and posttest (for inclusion in this study) was: “I feel connected to OMM members, who I can talk to if I have a problem.”
Awareness of Mental Health Resources
Perceived awareness of mental health resources was measured on a seven-point likert scale: strongly disagree, moderately, disagree, slightly disagree, neither agree nor disagree, slightly agree, moderately agree, strongly agree. Participants were prompted with the following statement: “Think of your awareness and confidence related to mental health resources at your school. Report how much you agree with the following statements.” The one resource item measured at pretest and posttest (for inclusion in this study) was: “I am aware of the mental health resources available online.”
Preventive Prosocial Behaviors
Preventive Prosocial behavior was measured on a five-point scale (never, rarely, occasionally, frequently, and always) based on the following statement: “When I see friends who are struggling with mental health issues, I notice the signs and say something before it is a bigger problem or crisis.”
Exploratory Variables
OMM Leader
OMM Leader was assessed by asking: Are you a club leader? The response options were: yes (1) or no (0). Respondents were labeled as “OMM Leader” or “OMM Member”.
School District
School District was assessed by asking: What school do you go to? The response options were a list of schools with OMM clubs. Respondents were labeled as “FCPS Schools” (1) or “Non-FCPS School” (0) based on what schools are in the FCPS school district.
Meetings
Seven_Meetings variable was assessed by asking: How many meetings did you attend? Respondents who attended 7 or more meetings for the year were coded as “1” and those who attended ‘less than 7 meetings’ were coded as “0”.
Statistical Analyses
The data was collected using Survey Monkey, cleaned on Microsoft Excel for Mac v16.40, and analyzed using 4.0.2 and RStudio v1.3.1056. The HTML output was created using “rmarkdown”, “markdown”, and “knitr” packages in RStudio along with two applications: LaTeXiT for Mac and XQuartz. The purpose of using the aforementioned software was to create a fully transparent and reproducible research study. The data file (.xlsx), R script (.R), R notebook (.Rmd), and related documents are available at the OMM project GitHub.
Four one-tail (i.e., pretest > postest) paired t-tests to test the hypotheses. The null hypothesis is no difference in mean scores between pretest and posttest (Ho = µ0 = 0). The alternative hypothesis is the mean pretest score is less than mean posttest score (Ho = µ0 < 0). To reduce the possibility of a Type I error, a Bonferroni correction was made, resulting in an alpha of .0125 for each of the four statistical tests.
Results
Participant Characteristics
School District
Forty-one of the fifty respondents (82%) attended a high school in Fairfax County Public Schools (FCPS).
Race/Ethnicity and Gender
An exploration of race/ethnicity and gender shows a majority of participants identified as “white/caucasian/european” (n = 26) and cisgender female (n= 49), resulting in 50% of the sample identifying as both “white/caucasian/european” and “cisgender female”.
The highest percentage of OMM leaders joined the OMM club at their school in Fall 2018, whereas the highest percentage of OMM members joined at the start of Fall 2019.
Summary of Outcomes
Histograms
Histograms are provided for each of the eight outcome variables. Four of the outcomes are negatively skewed.
Table of Outcomes
| Mean | Std.Dev | Min | Median | Max | N.Valid | Pct.Valid | |
|---|---|---|---|---|---|---|---|
| Post.Coping | 4.0400 | 0.72731 | 3 | 4.0 | 5 | 50 | 100 |
| Post.Preventive.Prosocial | 4.0200 | 0.58867 | 3 | 4.0 | 5 | 50 | 100 |
| Post.Resources | 6.0800 | 0.89989 | 4 | 6.0 | 7 | 50 | 100 |
| Post.Support | 5.3600 | 1.38151 | 2 | 5.5 | 7 | 50 | 100 |
| Pre.Coping | 3.8000 | 0.78246 | 3 | 4.0 | 5 | 50 | 100 |
| Pre.Preventive.Prosocial | 3.9184 | 0.73134 | 3 | 4.0 | 5 | 49 | 98 |
| Pre.Resources | 5.6000 | 1.38505 | 1 | 6.0 | 7 | 50 | 100 |
| Pre.Support | 5.6400 | 1.38151 | 2 | 6.0 | 7 | 50 | 100 |
Correlations Among Outcomes
Correlations Among Pretest Outcomes and Exploratory Variables
Correlations Among Posttest Outcomes and Exploratory Variables
Individual Outcomes
Outcome #1: Positive Coping
Three plots are used to depict the data for positive coping scores at pretest and posttest: barplot of likert responses, boxplot, and barplot of mean scores. Results of the paired t-test are also shown.
Analyses
A paired-samples t-test was conducted to compare healthy coping use at pretest and posttest. There was a significant difference in the scores at pretest (M= 3.80, SD= .782) compared to posttest (M= 4.04, SD= .727); t(49)= -2.370, p= .011.
##
##
## Table 2
##
## Means, standard deviations, and d-values with confidence intervals
##
##
## Variable M SD 1
## 1. pretest 3.80 0.78
##
## 2. posttest 4.04 0.73 0.32
## [-0.08, 0.71]
##
##
## Note. M indicates mean. SD indicates standard deviation. d-values are estimates calculated using formulas 4.18 and 4.19
## from Borenstein, Hedges, Higgins, & Rothstein (2009). d-values not calculated if unequal variances prevented pooling.
## Values in square brackets indicate the 95% confidence interval for each d-value.
## The confidence interval is a plausible range of population d-values
## that could have caused the sample d-value (Cumming, 2014).
##
## t(49) = -2.37, p = .011, d = -0.34
Outcome #2: Support
Three plots are used to depict the data for peer support at pretest and posttest: barplot of likert responses, boxplot, and barplot of mean scores. Results of the paired t-test are also shown.
As shown above, 18% of respondents who “occasionally” use positive coping skills at pretest reported more use of positive coping skills at posttest.
As shown above, the percentage of respondents who had a strong or moderate agreement decreased from pretest to posttest and the percentage who “slightly agree” increased at posttest.
Analyses
A paired-samples t-test was conducted to compare peer support at pretest and posttest. The pretest score (M= 5.64, SD= 1.382) was not significantly less than the score at posttest (M= 5.36, SD= 1.382); t(49)= 1.549, p= .936.
##
##
## Table 2
##
## Means, standard deviations, and d-values with confidence intervals
##
##
## Variable M SD 1
## 1. pretest 5.64 1.38
##
## 2. posttest 5.36 1.38 0.20
## [-0.19, 0.60]
##
##
## Note. M indicates mean. SD indicates standard deviation. d-values are estimates calculated using formulas 4.18 and 4.19
## from Borenstein, Hedges, Higgins, & Rothstein (2009). d-values not calculated if unequal variances prevented pooling.
## Values in square brackets indicate the 95% confidence interval for each d-value.
## The confidence interval is a plausible range of population d-values
## that could have caused the sample d-value (Cumming, 2014).
##
## t(49) = 1.55, p = .936, d = 0.22
Outcome #3: Resources
Three plots are used to depict the data for resource awareness at pretest and posttest: barplot of likert responses, boxplot, and barplot of mean scores. Results of the paired t-test are also shown.
As shown above, 10% more respondents selected an agreement statement (slightly, moderately, strongly) at posttest in comparison to pretest. Also noteworthy, none of the respondents disagreed at posttest.
Analyses
A paired-samples t-test was conducted to compare resource awareness at pretest and posttest. There was a significant difference in the scores at pretest (M= 5.60, SD= 1.385) compared to posttest (M= 6.08, SD= .900); t(49)= -2.370, p= .011.
##
##
## Table 3
##
## Means, standard deviations, and d-values with confidence intervals
##
##
## Variable M SD 1
## 1. pretest 5.60 1.39
##
## 2. posttest 6.08 0.90 0.41
## [0.01, 0.81]
##
##
## Note. M indicates mean. SD indicates standard deviation. d-values are estimates calculated using formulas 4.18 and 4.19
## from Borenstein, Hedges, Higgins, & Rothstein (2009). d-values not calculated if unequal variances prevented pooling.
## Values in square brackets indicate the 95% confidence interval for each d-value.
## The confidence interval is a plausible range of population d-values
## that could have caused the sample d-value (Cumming, 2014).
##
## t(49) = -2.37, p = .011, d = -0.34
Outcome #4: Preventive Prosocial Behavior
Three plots are used to depict the data for preventive prosocial behaviors at pretest and posttest: barplot of likert responses, boxplot, and barplot of mean scores. Results of the paired t-test are also shown.
As shown above, 15% more respondents selected an agreement-related response at posttest in comparison to pretest.
Analyses
A paired-samples t-test was conducted to compare preventive prosocial behavior at pretest and posttest. There was no difference in the scores at pretest (M= 3.92, SD= .731) compared to posttest (M= 4.02, SD= .589); t(49)= -1.062, p= .147.
##
##
## Table 4
##
## Means, standard deviations, and d-values with confidence intervals
##
##
## Variable M SD 1
## 1. pretest 3.92 0.73
##
## 2. posttest 4.02 0.59 0.15
## [-0.24, 0.55]
##
##
## Note. M indicates mean. SD indicates standard deviation. d-values are estimates calculated using formulas 4.18 and 4.19
## from Borenstein, Hedges, Higgins, & Rothstein (2009). d-values not calculated if unequal variances prevented pooling.
## Values in square brackets indicate the 95% confidence interval for each d-value.
## The confidence interval is a plausible range of population d-values
## that could have caused the sample d-value (Cumming, 2014).
##
## t(48) = -1.06, p = .147, d = -0.15
Summary
The current study evaluated the impact of the Our Minds Matter program on four outcomes. OMM participants self-reported more healthy coping skills and more awareness of mental health resources at posttest compared to pretest. There was improvements in peer support and preventive prosocial behaviors among OMM members.
Limitations
The study was limited by a one-group pretest-posttest design. A future study should add a control group to enhance the validity and associated inferences.