Schools Turn to Private Companies to Find Threats on Social Media

According to Politico Pro, searching for threats made to their schools online, districts across the country are spending thousands of dollars to contract with private companies to scan public social media posts.

Dozens of school administrators, school resource officers and police department staffers around the country receive daily alerts detailing violent words and phrases emanating from public social media posts in their geographic areas. Some colleges like the University of Virginia and the Massachusetts Institute of Technology also sign up.

But while experts agree that schools should be concerned about threats made on social media, some say the privacy concerns and the lack of research on the effectiveness of monitoring social media outweigh the benefits.

Schools saw a sharp uptick in the number of threats after the shooting of 17 people at a Parkland, Fla., high school in February. U.S. schools were barraged with an average of 59.4 threats and incidents of violence each school day for a month afterward, according to a report from the Educator's School Safety Network, an Ohio-based nonprofit.

A separate report released earlier this month by the Educator's School Safety Network found 3,380 reported threats in the 2017-18 school year, up 62 percent compared to the 2016-17 school year.

The most common way a school would receive a threat was through social media, with nearly 40 percent made over a platform.

The scanning technique for finding threats online is sophisticated. Using location data, a "geofence" a prevalent form of technology is placed around a school district. A software system then scans public social media platforms from a specific area to determine if any posts include language that matches up with a library of violent words and phrases.

If the algorithm determines the post might be directly related to a school district, an email or text alert is sent to designated officials who then decide whether action is required.

These alerts, sent through a software developed by a company called Social Sentinel, typically don't flag anything concerning. Posts saying "last night's football game was the bomb" or something about gun control that includes the phrase "shooting today" might make it into the report.

Administrators and police officers ignore these false positives with relief.

"Any time that we have the opportunity to avoid something just simply by monitoring public traffic as far as the internet is concerned, then I think we should do it," said Darrell Myers, superintendent of the Cleveland Independent School District in Cleveland, Texas, which enrolls 6,300 students and has been using Social Sentinel since March 2018.

Social Sentinel describes itself as a "social media scanning tool," and the company emphasizes that "it is not a surveillance tool, or a monitoring tool or an investigative tool." While Social Sentinel declined to provide specific numbers, it says it has clients in 30 states. Social Sentinel also works with sport and event arenas, but the company says the majority of its clients are schools.

Social Sentinel is one of a handful of companies that offer such social media scanning to schools. Others include Soter Technologies and Media Sonar.

School districts that have been satisfied say that using a service has offered them a much-needed helping hand to see what's posted on social media.

A week after the Parkland shooting, a high school student was arrested in Myers' school district for threatening violence through a video on social media. At the time, the district was in the process of starting its $18,000-a-year contract with Social Sentinel, Myers said.

Though a community member flagged the post to school officials, and the district encourages such "see something, say something" practices, Myers said he's happy to have Social Sentinel as an additional precaution.

Other school districts that use Social Sentinel agree.

"A school district superintendent or principal does not have the ability to monitor all of the social media posts within their school districts. This seemed like a very simple way to add one more preventative tool to our toolbox," said Tom Isaacs, superintendent of the Warren County Educational Service Center in Lebanon, Ohio. The center coordinates training and services among the county's eight independent school districts, all of which have been using Social Sentinel since March 2018.

School officials also said the software has helped them detect instances in which students on social media showed signs of distress or were threatening to harm themselves.

Terry McGinnis, director of administrative service for Dare County Schools in Nags Head, N.C., said the district has been using Social Sentinel since fall 2017 and has acted on two alerts. Both were times when a student conveyed distress on public social media posts that showed up in alerts.

"Prior to this, we didn't have a good handle on what was out there in social media affecting our schools," McGinnis said, "This has given us assurances."

Depending on the district, both school administrators and law enforcement officials receive alerts.

In Arlington, Mass., Chief of Police Fred Ryan receives Social Sentinel alerts along with school officials. The police department determines if an alert is actionable since it is a 24/7 operation, Ryan said, but whether his team or school officials will respond to it depends. For example, when an alert was received that suggested a student was suicidal, the department contacted school officials, who intervened.

Those who have discontinued social media scanning systems say they had a difficult time seeing how the service was effective.

Wilson County School District in Lebanon, Tenn. ended its contract with Social Sentinel after a year.

Instead, the district is opting to use Meltwater, a service that broadly scans the web for any mentions of the school in blog posts and news articles as well as in public social media posts. Jennifer Johnson, public information officer for the district, said Meltwater's service is more valuable because officials can get an idea of what's being said online.

Had they not found a company that offered broader services, the district would have likely stopped using a social media scanning service altogether, she added.

The Portage Police Department in Portage, Ind., set up Media Sonar to scan public social media posts coming from the areas around the school in 2016. Similar to Social Sentinel, the software put a geofence around a specific area to scan for certain keywords, though the police department came up with the keywords.

The department discontinued the service after a year of use because it was too costly, said Portage Police Chief Troy Williams. The department would have to pay $16,500 to renew an 18-month contract.

"We didn't get a ton of information from it. There wasn't the singular event that we prevented because of (Media Sonar)," Williams said.

For now, Williams said he is confident suspicious online posts will be flagged, given good relationships between school resource officers and students.

Additionally, Media Sonar's inability to access Facebook's data in its scanning software diminished the value the service could bring to the district, Williams said.

Media Sonar lost access to Facebook, Twitter and Instagram's data in 2016 because it was selling its services to police departments to help them track protesters using hashtags like #BlackLivesMatter, Vice's Motherboard reported in January 2017.

Another social media scanning company, Geofeedia, lost access to those platforms for similarly selling its services to police departments. After losing access, the company cut its staff in half, the Chicago Tribune reported in October 2016.

The shutdown of the two companies "didn't affect (Social Sentinel) because we were never built to be a surveillance tool," said Gary Margolis, founder and CEO of Social Sentinel. "We aren't a surveillance tool, we're not a monitoring tool, and we built a library that is very unique to us."

Margolis said the biggest challenge he's dealt with when it comes to privacy is dealing with the media "trying to frame us against the privacy issue."

But privacy advocates say that the Geofeedia and Media Sonar cases prove schools are entering rough waters when using social media scanning companies to comb through million of public social media posts.

Kade Crockford, director of the Technology for Liberty Program at ACLU Massachusetts, and Faiza Patel, co-director of the Brennan Center for Justice's national security program at New York University School of Law, agreed that they would consider any company that scans social media posts even if they are public posts as a means of surveillance.

"Just because it's public, doesn't mean it doesn't have privacy implications when you're aggregating huge amounts of data and you're developing conclusions and assumptions about people," Patel said.

Students could feel they are being watched on social media, potentially restraining their "freedom to explore lots of ideas and concepts," Patel said.

School safety experts are wary that social media scanning companies have actually been shown to be more effective than schools relying on their own communities to keep an eye on social media, saying it's too early to tell.

"It's been very difficult to prove that spending all this money on apps and monitoring and all that kind of stuff really changes much compared to investing in people and training and the things that are happening directly in the school," said Amy Klinger, director of programs for the Educator School Safety Network and associate professor of educational administration at Ashland University.

It takes a few years to figure out whether software meant to predict the behavior of people is effective, said Ryan Boyd, a post-doctoral fellow in psychology at the University of Texas at Austin. It's also difficult to determine the value of privacy versus the value of detecting a problem.

"It's hard to quantify that and say that one approach is better than not at all," Boyd said, though companies can probably "put a price tag on peace of mind."

But Margolis said he knows Social Sentinel works for clients because of calls he receives from school administrators saying they were able to intervene when someone was threatening self-harm or when the service caught plans of violence toward a school. Margolis did not disclose the identities of districts that avoided threats.

"We know our service works, we know it brings value, we know we do a lot of good stuff. But we can't always scream from the rooftops because of privacy," he said.

Even if the district acts because of a alert two or three times a year, "it's helping," McGinnis, in North Carolina, said. "The phrase we use here is 'peace of mind.'" 

By Lauren Aratani