Reaching Those At Risk for Psychiatric Disorders and Suicidal Ideation: Facebook Advertisements to Recruit Military Veterans

Background: Younger military veterans are at high risk for psychiatric disorders and suicide. Reaching and engaging veterans in mental health care and research is challenging. Social media platforms may be an effective channel to connect with veterans. Objective: This study tested the effectiveness of Facebook advertisements in reaching and recruiting Iraq and Afghanistan-era military veterans in a research study focused on mental health. Methods: Facebook ads requesting participation in an online health survey ran for six weeks in 2017. Ads varied imagery and headlines. Validated instruments were used to screen for psychiatric disorders and suicidality. Outcomes included impressions, click-through rate, survey completion


Introduction
Military veterans who served during the Iraq and Afghanistan conflicts are at an elevated risk for a number of psychiatric problems.In a Veterans Affairs (VA) sample, one in four were found to have at least one mental health diagnosis, mostly commonly posttraumatic stress disorder (PTSD), depression, and alcohol and substance use disorders [1].Heightened risk for suicide is also a major concern [2], with rates of suicide among veterans in the United States markedly higher than the general population, even after adjustment for age and gender differences [3].Despite this, approximately 40% of Iraq and Afghanistan veterans have never accessed VA health services [4], and even when these veterans are in VA care, they may not be more inclined to utilize mental health care if experiencing active suicidal ideation [5].
Preventing veteran suicide has been, and continues to be, a top priority.Nonetheless, in-person health care appointments pose a significant barrier to Iraq and Afghanistan era veterans who tend to be younger and more likely to be employed than other veterans [6,7].Other common barriers to formal help-seeking and treatment access, even in the presence of seemingly severe symptoms such as suicidal ideation or behaviors, include low perceived need [8][9][10], distance from health care facilities [11], and a desire to "handle the problem alone" [8].In a sample of veterans who died by suicide in Oregon, an estimated 78% had not accessed VA health services [12].
Traditional strategies for recruiting participants into research studies can suffer from narrow reach, geographical limitations, costliness, and time-intensiveness.By comparison, recruitment via social media platforms, especially Facebook, may be faster, cheaper, and easier than traditional methods [13][14][15].Among social media platforms, Facebook may be an especially important tool because it is the largest and most used, with a diverse base of users with detailed demographic profiles [13].Facebook users spend upwards of 50 minutes a day on the platform, and among online adults between the ages of 18 to 29, approximately 9 of 10 use Facebook [14,15].In recent years, many studies have examined Facebook advertisements (ads) as a recruitment method for research studies on mental or behavioral health.Facebook ads have been used effectively in populations including college students and young adults [16,17] and military veterans particularly those with risky drinking [20][21][22][23].Prior research has also suggested ad campaigns can achieve both "broad reach and targeted recruitment," and found ad costs to be manageable [18].Throughout this paper, we use the term "reach" to mean even a minimal amount of engagement with an ad, not necessarily engagement in health care, a definition that is consistent with the vein in which it is used in social media contexts.Nonetheless, much work remains in the development of best practices and evidence-based recruitment strategies on social media.Studies conducting experiments comparing particular Facebook advertising approaches, such as differing images or text are lacking [19,21,[25][26][27].One recent study by Pedersen and colleagues, which focused on recruiting spouses of heavy drinking service members and veterans, did systematically and sequentially test different ad features.It concluded that ads with text accentuating the US $120 financial incentive for study participation had a higher conversion to study participation at a lower per-participant cost [19].Examination of a spectrum of potential outcomes, ranging from general exposure (eg, "impressions") to initial interaction (eg, link clicks) to implementation (eg, enrollment in a research study) [20], would also be helpful in social media studies.
We conducted a study with three aims: (1) determine the feasibility of reaching Iraq and Afghanistan era military veterans through Facebook ads, (2) quantify the extent to which reached veterans are at risk for psychiatric problems, and (3) characterize how veterans utilize social media and interact with their social networks on Facebook.In this article, we focus on the first two aims.More specifically, we determined the recruitment of military veterans to a mental health focused research study, examined what ad features are most relevant to engaging veterans, and characterized what kinds of veterans are likely to be reached by the ads.As an exploratory study, we limited our a priori hypothesis to state that it would be feasible to recruit recent military veterans with probable mental health problems.

Participants
The target population for the survey was United States (US) military veterans of the Operation Enduring Freedom-Operation Iraqi Freedom (OEF-OIF) service era (September 2001-present), also referred to as Iraq and Afghanistan era veterans.To be eligible, individuals needed to be age 18 or older, and on active duty in the US Armed Forces after September 2001 but not presently.We excluded individuals who completed surveys in less than five minutes, had more than one survey response, or incorrectly answered a military-related "insider knowledge" question to reduce the chance of online survey misrepresentation [18,21].Survey completion was defined as those respondents who reached the end of the online survey and were not excluded based on the above quality control measures.

Advertisement Campaign
Facebook offers myriad options related to placement and targeting of ads; the same parameters were used for all ads.Ads were run simultaneously, to identical audiences, with the same ad budget, and for the same duration of time.Ads ran for a total of 45 days between January 13, 2017, and March 18, 2017, except for one ad that was briefly deactivated by Facebook for technical reasons.Ads were exclusively placed in the News Feed on computers and not mobile phones as the survey was developed for computer administration.Ads were optimized per Facebook's algorithm for clicks, meaning that ads were automatically shown to users whom Facebook anticipated would click at the highest rates, in a targeting process adjusted by actual clicks during the campaign.
Study ads were broadly targeted at Facebook users in the US of any age or gender who had at least one of a variety of veteran-related characteristics (eg, interest in "United States Armed Forces" or "Supporting Our Veterans" as determined by their Facebook profile).Text above the ad image indicated XSL • FO RenderX that veterans who served between 2001 and 2017 were needed for an "online health survey." We designed a total of 15 ad variations in a 3x5 factorial design, with 3 images (ie, a person taking a survey on a tablet device; a veteran with his family; and soldiers marching) varied against five headlines.Ad images are illustrated in Figure 1.Headlines were informed by empirical research in psychology and survey methodology as well as established principles in behavioral economics known to help nudge behavior [22].Specific approaches that can motivate research participation include: targeting feelings of altruism or a "warm-glow" effect [23], harnessing "psychological capital", which is closely correlated with a sense of empowerment [24,25], or using a statement of what others in similar situations do, also known as descriptive social norms [26].Providing financial incentives [27] or encouraging the sharing of content with social network members could also increase engagement [28].Based on these ideas, we crafted and used the following 5 ad headlines: 1. Altruism: "Will you help us improve care for veterans?" 2. Empowerment: "You can tell us how to design new health programs for veterans." 3. Social norms: "Hundreds of veterans are participating in this survey.Will you join them?" 4. Incentive: "You can win a new 7.9" 16 GB iPad Mini 4 with Retina Display!" 5. Sharing: "Will you share this with one veteran you know?" Ads were hosted by Oregon Health & Science University (OHSU) and linked to an online survey.To calculate survey participation and other outcomes by an ad, we constructed separate uniform resource locators (URLs) to the online survey for each ad.Prospective participants initially completed an online consent and eligibility screener.As an incentive for survey completion, we informed potential participants of an optional sweepstakes or lottery, in which two randomly selected survey participants who provided contact information would receive an iPad.Eligible, consented participants then completed the full online survey.Before and after survey, we provided all participants with a series of online, phone, and text messaging mental health treatment resources, including options for crisis situations, non-urgent treatment referral, and-as we would not be aware of the particular location of respondents-ways to locate local support and treatment resources.The institutional review board of OHSU approved all study procedures.

Measures
Our primary outcome was survey completion, which represents the highest level of engagement with a Facebook ad [20].As additional outcomes, we included measures of ad engagement that are automatically tabulated by Facebook for advertisers: • Impressions: the total number of times that the ad is presented to any Facebook user.
• Clicks: the number of times that a user clicks on the ad.
• Click-through rate (CTR): the number of clicks divided by impressions.
The online survey included a series of self-report questions to obtain background information about the sample including sociodemographic characteristics, military history, social media use, and interest in social media-based interventions.Using survey items from the National Survey of Veterans [29], we assessed the period of service (ie, "Have you ever served on active duty in the US Armed Forces?Active duty includes serving in the US Armed Forces as well as activation from the Reserves or National Guard." and " When did you serve on active duty in the US.Armed Forces?Mark all service eras that apply."),branch of service (ie, "In which branch or branches did you serve on active duty?") and deployment history (ie, "Did you deploy in support of Operation Enduring Freedom (OEF) or Operation Iraqi Freedom (OIF) or Operation New Dawn (OND)") Frequency of Facebook use was assessed by adapting previously validated survey items used by Pew Research [30].We used two additional items from the National Survey of Veterans to determine VA health service use (ie, "Have you ever been enrolled in VA health care?" and "In the past 12 months, did you use any VA health care services?")[29].Participants who responded "Don't know" to health service use questions were classified as not non-users.

RenderX
To screen for mental health problems, we employed reliable and valid self-report tools.For PTSD, we used the Primary Care PTSD Screen for DSM-5 (PC-PTSD), a five-item scale assessing past-month symptoms of a lifetime traumatic event.A score of three or higher on the PC-PTSD indicates a positive screen [31].For alcohol misuse, we used the Alcohol Use Disorders Identification Test Alcohol Consumption Questions (AUDIT-C), a three-item scale on frequency and intensity of drinking.An AUDIT-C score of four or higher for men, or three or higher for women, indicates a positive screen for problematic drinking [32].For major depression, we used the Patient Health Questionnaire-2 (PHQ-2), a two-item scale on anhedonia and depressed mood in the previous two weeks.A score of two or higher on the PHQ-2 indicates a positive screen [33].For suicidality, we used the Depressive Symptom Inventory Suicidality Subscale (DSI-SS), a four-item scale on suicidal ideation within the past two weeks [34].A score of two or higher on the DSI-SS indicates a positive screen in a population-based sample [35].

Statistical Analysis
Demographic variables were compared by ad text and image for participants in the analytic sample using Pearson's chi-square test, or ANOVA for age.All outcomes were modeled as negative binomial counts.The study design parameters of image, headline, and the interaction were included as independent factors.The model for clicks and CTR included an offset for the number of impressions; the model for reactions included an offset for the number of clicks.

Feasibility of Recruiting Military Veterans Through Facebook Advertisements
Over the 45 days of the advertising campaign (Figure 2), the Facebook ads produced 827,918 impressions, 9,527 clicks (CTR=9,527/827,918, 1.20%), and 1,787 reactions.There were 1,329 complete responses to the eligibility screener, of which 711 met eligibility criteria, and 605 completions of the online survey (ie, 605/711, 85% response rate).A total of 18 responses were excluded from the analysis based on quality control measures.Ten took less than 5 minutes, 2 claimed nonexistent pay grades, and 6 were duplicate responses.This left a final sample of 587 (ie, roughly 13 new participants each day).Total ad expenditure was US $11,427, yielding an average cost per analyzed survey of US $19.47.

Characteristics of Recruited Veterans
Characteristics of survey participants are described in Table 1.Their mean age was 40 years.A total of 81% (474/587) were male, and 81% (477/587) were white and non-Hispanic.In addition to serving during the Iraq and Afghanistan era, many also served during prior eras, particularly the Gulf War era (213/587, 36%).The majority (426/587, 73%) had been deployed in support of OEF-OIF.In this sample, the majority (326/587, 56%) were in the Army, as compared with approximately 36% among active-duty military personnel [36].

Associations Between Advertisement Characteristics and Demographic Characteristics
Gender of respondents varied by ad text, (χ 2 4 (N=583)=10.7,P=.03), with sharing and empowerment messages having a higher proportion of women.Age varied by text (F(2, 585)=11.84,P<.01) and image (F(2, 585)=10.09,P<.01), with the soldiers marching image and incentive text attracting the youngest respondents and the survey-taking image and altruism and social norms headlines attracting the oldest.Service era varied correspondingly, with the soldiers marching image (χ 2 2 (N=585)=9.90,P<.01) and incentive text (χ 2 4 , N=585)=22.03,P<.01) attracting a higher proportion of respondents who had only served during the OEF-OIF era.Race, ethnicity, education, military branch, and deployment to Iraq or Afghanistan did not vary significantly by ad text or image.

Variations in Advertisement Engagement and Cost
There was a main effect for ad image across impressions, CTR, and reactions, but not survey participation.In terms of both impressions and click-through rates, the soldiers marching image performed better than the survey-taking and family images (P<.001 for all comparisons).In addition, the soldiers marching image generated significantly more reactions than the survey-taking (P=.001) and family (P<.001)images.However, there were no significant differences by ad image in terms of survey participation.There was also a main effect for ad headline on ad engagement outcomes.Specifically, the sharing headline was associated with more impressions than the incentive (P=.045) and empowerment (P=.004) headlines; more reactions than the altruism (P=.004) and empowerment (P=.014) headlines; and higher survey participation than the social norms headline (P<.001).In addition, the social norms headline was associated with higher click-through rates than the incentive (P<.001), altruism (P<.001), and empowerment P=.001) headlines.

XSL • FO
Two of the 15 ad versions generated nearly half (285/587, 49%) of the participants (Table 2).These were the ads containing the image of soldiers marching with either the incentive or sharing headline.These two ad versions had significantly higher impressions (P<.001) and CTR (P<.001) than the other 13.Consequently, these two ads were most cost effective, at US $4.88 and US $5.90 per participant, respectively.Results were similar when examining individuals who completed the online eligibility screener, regardless of whether they were eligible or completed the full survey.

Veterans With Suicidal Ideation or Non-Enrolled in Veterans Affairs Health Care
Among veterans who completed the survey, the probability of suicidal ideation ranged from an estimated 15%-50% across the 15 ad variants, and the probability of not being enrolled in VA health care ranged from an estimated 18%-50% (Table 3).Recruitment of veterans with suicidal ideation was significantly higher for ads with the survey-taking image, as compared to the soldiers marching (P=.007) image.There were no statistically significant differences in recruitment of non-enrolled veterans by ad image or headline.

Key Findings
Our study demonstrated that Facebook ads are a potentially powerful tool to recruit research subjects.With the support of a single, half-time research assistant, we engaged veterans in enrollment in our online survey at a rapid clip (ie, nearly 100 participants per week).Though our click-through rate was similar to prior studies, our response rate was very high, which may have reflected ease of participation in this online survey.Average cost per participant was less than US $20, and our best-performing ads were dramatically cheaper, approximately US $5-6 per survey participant, a figure that compares very favorably with most prior studies [37,38].Facebook ads were further disseminated through social sharing, as is illustrated by the "likes", comments, and sharing of ads that we observed.This is a significant "externality" from a cost efficiency standpoint.
The feasibility of reaching and engaging younger veterans in research through this approach has important public health and clinical implications.We reached not only a relatively broad target population (ie, recent military veterans) but also were effective in engaging subpopulations that can be hard-to-reach and are of heightened interest.Being able to rapidly reach veterans who are experiencing current suicidal ideation and unengaged in VA health care-as we did-is a major challenge for the VA, health policy-makers and other stakeholders interested in improving veteran mental health outcomes.
It is worth emphasizing the high rate of detection of potentially serious psychiatric problems in this sample; we found high rates of screening positive for active PTSD (266/585, 45%), problematic drinking (243/584, 42%), major depression (164/586, 28%) and serious suicidal ideation (132/585, 23%).Facebook ads, together with other digital media advertising strategies that can support help-seeking (eg, Google AdWords) [39], may comprise critical tools in the design of effective campaigns for mental health treatment engagement.The major-and more imposing-next challenge is how to move individuals from endorsing their distress (online) to engaging in treatment in a health care or other therapeutic setting.
A critical novel component of this study was the use of an experiment, or as close to a "true experiment" as is possible in the Facebook advertising environment, to determine what ad features are most likely to result in engagement with the ad.One of the most intriguing novel findings here was that a headline encouraging users to share the ad resulted in better ad engagement."Sharing" is a request that is uniquely suited to the social media milieu, and also appeals to a military ethos of helping peers.Results also varied depending on the level of engagement being measured and target population.What works as a "hook" regarding the generating clicks may not translate into more active participation, as was similarly found in a study reaching concerned partners of heavy drinking service members and veterans [19].We found that for a more modest level of engagement (eg, impressions and clicks) with a broad spectrum of recent military veterans, using an image of soldiers or headlines containing a social norms message may be more XSL • FO RenderX effective.In contrast, if the goal is more proactive engagement (ie, survey completion) by individuals with active suicidality, an image of a person taking a survey may work better.One reason careful development and selection of image and text may be necessary to optimize ads for individuals with suicidal ideation could be related to cognitive differences in these individuals [40].There is, for instance, an emerging set of empirical studies showing attentional biases toward certain words [41,42].

Limitations and Future Directions
Our results should be considered in the context of several limitations.As participation in this study only involved a one-time online survey, it is not clear if the same strategies would be effective for treatment engagement [39], or engagement in research requiring a higher burden (eg, intervention or longitudinal cohort study).Also, it is possible that individuals perceived our ads in ways different than hypothesized (eg, the "survey-taking" image could have been perceived as that of "computer technology").Military culture may also impact response to advertisements.For example, our military-related ad images favored the army, which may have contributed a higher representation of them in the sample.Future research focused on testing the effectiveness of online ads should consider a qualitative component to gain more insight into differential ad performance.If future studies can confirm and further identify ad features that result in more response and engagement by veterans with suicidal ideation, there is significant potential for targeted interventions or campaigns to enhance outreach, health messaging, help-seeking, or other behaviors.

Conclusions
Taken together, our study demonstrates that Facebook ads are an effective medium for rapidly identifying, reaching, and recruiting recent military veterans, and can particularly help in reaching individuals who screen positive for current mental health problems.These results provide a foundation to inform efforts to engage veterans disconnected from the health care system or at risk for suicidal ideation.©Alan R Teo, Samuel BL Liebow, Benjamin Chan, Steven K Dobscha, Amanda L Graham.Originally published in JMIR Mental Health (http://mental.jmir.org),05.07.2018.This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/),which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in JMIR Mental Health, is properly cited.The complete bibliographic information, a link to the original publication on http://mental.jmir.org/,as well as this copyright and license information must be included.

Figure 1 .
Figure 1.Sample Facebook ads illustrating the three different ad images (Survey-Taking, Family, and Soldiers Marching).

Figure 2 .
Figure 2. Cumulative ad campaign outcomes over time.

Facebook use frequency, n (%)
Percentages do not add up to 100% because some respondents indicated multiple service eras; 60.5% (355) served only during the current era, September 2001 to present.cPatient Health Questionnaire-2 (PHQ-2) score ≥ 3. Due to missing item response, the total number of respondents for this scale was 586.Primary Care posttraumatic stress disorder (PTSD) Screen for DSM-5 (PC-PTSD-5) score ≥ 3. Due to missing item response, the total number of respondents for this scale was 502.e Alcohol Use Disorders Identification Test Alcohol Consumption Questions (AUDIT-C) score ≥ 4 (men) or ≥ 3 (women).Due to missing item response, the total number of respondents for this scale was 477.Depressive Symptom Inventory Suicidality Subscale (DSI-SS) score ≥ 2. Due to missing item response, the total number of respondents for this scale was 585.
a Percentages do not add up to 100% because some respondents (45/587, 7.7%) indicated multiple branches.b d f

Table 2 .
Matrix of 15 Facebook advertisement variants with outcomes for each advertisement.
a CTR: click-through rate.

Table 3 .
Predicted probabilities of suicide ideation and non-enrollment in Veterans Affairs Health Care among survey participants.Depressive Symptom Inventory Suicidality Subscale (DSI-SS) score ≥ 2. a