October 31, 2019 … Beaumont, Texas
Beaumont Independent School District, in Texas, has approved a contract with an artificial intelligence-based safety management system to monitor student emails and Google Drive accounts in an effort to prevent bullying, violence and suicide.
The approval comes amid rising suicide and bullying rates, although digital-safety advocates say it also raises concerns over the privacy of students private information.
Bill McCullough, spokesman for the company, Gaggle, said the software constantly scans emails and files stored on accounts hosted and provided to students by the school.
“We are looking at the email, Google and Microsoft environments that schools issue to students,” McCullough said. “And we’re looking for kids who are in crisis, whether that’s bullying, anxiety, depression, suicide, drugs or alcohol violence from others or child pornography.”
Once offending content is detected, the system alerts a safety expert to analyze how critical the threat is.
“Our safety experts look at the context, determine the severity of the threat, and then work with the school district to get that child help before a crisis or a tragedy occurs,” McCullough said.
Gaggle staff would then be able to search the student’s system to “see if there’s a pattern.”
“If a child is a target of bullying, they could go backwards to see if this repeated over time between these two children and then notify the school that this is more than just one incident,” McCullough said. “So that way we can be more accurate with what the information we’re delivering to the school district.”
Access other than to research around the incident would not be possible.
Toni McPherson, director of information services and technology for Beaumont Independent School District, said the intent “is to protect our students from harmful content and help them learn to be responsible online.”
McPherson also said, in an emailed statement, that content flagged as dangerous “will be blocked from being shared to others and the Gaggle staff will immediately review it and determine the severity. In high-severity situations, the content will be sent to the appropriate district staff to be addressed.”
McCullough said districts, and not Gaggle, handle intervention because “schools, like in Beaumont, know their community, know their children and know how they want to use it. And so we let them make Gaggle what they need it to be.”
A recent report by the U.S. Centers for Disease Control and Prevention found that the suicide rate for persons between the age of 15 and 19 increased 76 percent from 2007 to 2017.
Despite that, and thousands of success stories cited by Gaggle, some advocates have called into question the possible ethical violations of student’s privacy rights by a software that constantly scans their correspondence.
Sophia Cope, a staff attorney for the Electronic Frontier Foundation, a nonprofit that advocates for civil liberties and technology, said the benefits of keeping students safe don’t necessarily outweigh the costs of violating their privacy.
“I do understand the impulse to want to intervene early, and try to stop students who are at risk for self-harm or harm to others,” Cope said. “But at the same time, I’m not sure violating their privacy is the best way to go about it.”
Cope said that implementing software like Gaggle could “actually disincentivize students from freely talking to each other over Gmail, causing them to self-censor.”
Other concerns include violations of the Family Education Rights Privacy Act (FERPA), although Gaggle touts strict compliance with the federal regulation under what is called the “school offical” exclusion, meaning that school districts give the company permission to handle students personally identifiable information.
BISD officials pointed to that exclusion when asked whether FERPA was being violated.
Parents can opt out of the program, but a district spokeswoman said they would have to also opt out of using school-provided technology.
McCullough agreed that students have a right to privacy, noting that while other similar automated safety companies monitor all communications, Gaggle focuses only on school-issued software, allowing students to freely express themselves through other mediums.
“We believe kids should have a way to communicate privately,” McCullough said. “So that’s why we do not look at their private email addresses, or private social media. We believe that that is where kids should have the right to express themselves privately.”
On school-issued software and property however, “schools have a moral and legal obligation to protect children,” he said. “That’s the reason why we put all of our concentration on that side of the fence,” he said.
Cope said that the understanding of legal protections like the Fourth Amendment right not to have unreasonable searches and seizures has not caught up to technology like AI — making regulation unclear for apps like Gaggle.
Beaumont is one of about 1,400 schools implementing Gaggle, but the company’s growth is being outpaced by skyrocketing rates of youth suicide and bullying, McCullough said, citing company estimates from last year that 52,000 children “had suicide ideation” — triple the year before.
“They might not have had clear plans, but they were certainly thinking about suicide,” he said. “ … I could tell you that while our sales have grown, they did not triple in one year to the next.”
While measuring the efficacy of suicide prevention is inexact, Gaggle said in a news release that 6,000 of the cases it found “were serious enough to merit immediate attention by the district, which resulted in more than 700 lives saved.”
The software will cost $88,785.50, according to a contract given to The Enterprise through a public information request, and it will be phased in over the next several months, McPherson said.
The program will be covered by federal funding, and will not come from the BISD general fund.
The approval was part of a larger package of software that included other IT purchases and funding for Communities in Schools, an in-school program that works with at-risk youth.
©2019 the Beaumont Enterprise (Beaumont, Texas). Distributed by Tribune Content Agency, LLC.
