Private Companies of Interest

[gfp927z]

Liminex - >>> Big Teacher Is Watching: How AI Spyware Took Over Schools


The pandemic caused schools to embrace laptops, tablets, Zoom, and an app called GoGuardian that tracks everything students (and, sometimes, parents) do online.


Bloomberg

October 28, 2021


https://www.bloomberg.com/news/features/2021-10-28/how-goguardian-ai-spyware-took-over-schools-student-devices-during-covid


At Pekin Community High School, the teachers are something close to omniscient. Education, even in-person education, is digital in the Covid-19 era, and staff members use a piece of software to watch everything students do on school-issued laptops and to keep them off banned websites. The kids are aware. “They pretty much know that they’re being monitored 24/7,” says Cynthia Hinderliter, head of technology at the school outside Peoria, Ill.

Still, class clowns persist. Hinderliter pulls up a detailed dashboard of student online activity, which reveals the identities of rule breakers. A yellow “EXPLICIT” label appears beside the name of a youngster who had typed “sexy girls” and “sugar daddy dating” into Google. Other students were searching YouTube for videos of a farming simulation game, guitar tutorials, and, for some reason, nursery rhymes about trucks. Another popular search: “How you bypass GoGuardian,” which is the name of the tracking software Pekin High uses. GoGuardian has been around since 2014, but the pandemic gave educators new reasons to adopt it. The software is quickly becoming almost as commonplace inside American classrooms as standardized tests.

Even as schools throughout the U.S. have opened back up for full in-person instruction, they’ve generally kept the technologies they picked up during the unending months of Zoom lessons. This includes laptops and tablets, which some schools were already using before Covid, as well as apps that allow students to attend classes while they are quarantining after an illness or a potential exposure and software to allow educators to keep tabs on them.

For kids that means their every keystroke, click, and search is recorded and analyzed by companies such as GoGuardian, which is based in Los Angeles. Its competitors include Gaggle.Net, Securly, and Bark Technologies. In addition to monitoring and website filtering, GoGuardian sells software tools for classroom management, video calling, and network security. Its most futuristic offering, Beacon, is an artificial intelligence feature the company claims can identify students who, based on their online behavior, are at risk of hurting themselves or others. Advait Shinde, GoGuardian’s chief executive officer, says his company plans to offer software for “classwork, homework, exams”—every aspect of primary and secondary school. “We haven’t fully realized what technology can do.” He imagines computer systems that recommend assignments and offer personalized curricula based on data GoGuardian collects. This summer he raised $200 million, valuing Liminex Inc. (the company’s official name) at more than $1 billion.

Earlier this year, GoGuardian struck deals with state education departments in Delaware and West Virginia, which will give every school in both states access to the software. New York City, the nation’s largest district, recently signed a similar contract with the company, bringing GoGuardian’s potential reach to more than 23 million students. Some parents and privacy groups find the use of tracking software inside classrooms to be disturbing, but many school administrators (and many parents, for that matter) seem to regard those concerns as overblown. Schoolkids already turn over personal data to big companies when they mindlessly scroll through TikTok and Instagram, and surveillance in schools has been around for as long as there have been hall monitors. “I’ve always felt that they’re already being tracked,” says Pekin High’s principal, Joel Schmieg. Administrators at another GoGuardian customer—Success Academy, the large New York charter school network renowned (or infamous) for harsh discipline—say they hear demands from parents for more internet filtering, not less.

Even if most educators believe that students should be monitored when they’re using computers, there’s far less clarity about what computers mean for the future of learning. For years, Silicon Valley has tried to leave its distinctive mark on education, but most attempts have failed. Massive open online courses, or MOOCs, haven’t been the game changer for higher education that futurists and TED talkers once described. Mark Zuckerberg’s $100 million effort to remake the school system in Newark, N.J., led to allegations of corruption along with limited evidence of improvement. AltSchool, an education company backed by Zuckerberg and top venture capital firms and founded by former Google executives, started closing its 3D-printer-equipped classrooms in 2017 and got out of the business of operating schools entirely in 2019.

Will this latest Silicon Valley foray into education end any differently? GoGuardian and similar apps were rushed into schools during the pandemic. Teachers like the apps, students accept them, and they even play in Peoria—or just outside it, anyway. But no one actually knows how well, or even if, these technologies work.

In 2013, President Barack Obama unveiled ConnectEd, an initiative calling on schools to embrace the internet era and to give students cheap computers. Soon, iPads and Chromebooks started pouring into classrooms, and Shinde, then a programmer at Google, sensed an opportunity. He quit his job and, working with two acquaintances, created a browser widget called Laptop Lookout. It was supposed to help schools hunt down kids’ lost devices. Their pitch didn’t take, but it gave him a new idea.

“This stuff is great and all,” Shinde recalls hearing when he talked to teachers. “But I’m having trouble keeping my kids from watching Netflix in class.” After unleashing the internet in classrooms, schools were desperate for ways to control it. The trio reworked their spiel into one for a web-censoring widget called GoGuardian. Rather than blocking entire websites—an approach that many corporate IT departments take to control what employees do on their work computers—the system they developed claimed to analyze text and pixels on each website to filter out only those deemed inappropriate or unwanted. So, in theory, teachers could send kids to Reddit and YouTube without worrying they might wander down distracting or darker paths. “You end up blocking much less,” Shinde says. Schools would pay a per-student fee (now around $5) for each GoGuardian product they used. GoGuardian raised $5 million in seed funding in 2015 and saw encouraging if unspectacular growth in its early years. In 2018 private equity firm Sumeru Equity Partners LP bought most of the company for an undisclosed price, and Shinde’s co-founders left, leaving him in charge.

Shinde has no background in education, nor, at age 31, is he a parent—“not yet,” he says. But he’s won fans in the field simply by being more agreeable than the average tech bro. Many Silicon Valley executives selling products to schools “model themselves after Steve Jobs: ‘I know best,’?” says Tony Miller, a deputy secretary of education under Obama. Shinde met Miller in 2018 and recruited him to GoGuardian’s board. Shinde “had a real vision for improving education that was less about technology and more about its benefit,” Miller says.

The pandemic gave Shinde’s pitch urgency. He sweetened the deal by offering GoGuardian software for free on a trial basis to any school that wanted it. Teachers can see students browsing the web in real time. “It’s like the teacher is there with them,” says Marshall Beyer, education technology coordinator for the Turlock Unified School District in California’s Central Valley, where GoGuardian tracks roughly 14,000 students. If a kid is playing a video game instead of working, the teacher can commandeer the screen and force-quit the game. There are Reddit threads from teachers cataloging student Googling, ranging from the banal (photos of potatoes) to the discomfiting (“cat poop” and “how much does horse semen cost”).

More than 1,000 schools signed up for free trials in the spring of 2020, and some of those paid for accounts at the end of the school year. In some cases new customers used federal Covid relief funds for the purchase. That fueled whiplash growth for GoGuardian: Since March 2020 the company has added 253 employees, bringing its total to about 480, and Shinde acquired two other ed-tech startups.

The schools—not GoGuardian—decide what to block on kids’ computers, though this isn’t always clear to pupils. Precocious seventh graders once emailed Shinde to complain that their district in New Jersey had blocked the New York Times but let right-wing outlets such as Breitbart News slide. The seventh graders blamed Shinde. He set up a video call to explain to the students that GoGuardian didn’t make filtering decisions and suggested they lobby their administrators. He hasn’t heard from them since. “From our perspective,” he says, “we’re not in a position to say what’s right or wrong.”

Pekin High was flooded with alerts and “kids searching for problematic stuff,” Hinderliter says. “Problematic would be porn”

Pekin is a blue-collar town. Its public high school sits next to a strip mall with a Pizza Hut and a nail salon and is 1 mile from a park that claims to have the “world’s greatest sundial.” (It isn’t the tallest sundial in the world, or even in the country, but it does have informative plaques about the solar system, and there’s a lagoon nearby.)

Pekin High signed up for GoGuardian three years ago, after doling out bare-bones Acer Chromebooks to its 1,800 students. Every summer the school gives graduates a chance to keep their computers (for $25 each) and orders a fresh batch of 500. With the help of a few members of the school staff, Hinderliter sets up each laptop, which comes equipped with only a web browser, adding some library and math programs, and a software widget for recording online lessons.

On the computer at her desk, nestled in a corner of the school library, Hinderliter is already logged in to GoGuardian. She opens a dashboard labeled “Fleet” and pulls up a digital map, revealing the location of each laptop. “This one says that it’s 6 miles away,” she says, pointing to the screen. It’s summer vacation, so the student is online at home. “That’s very useful when we have a stolen Chromebook.”

It’s taken a while for Hinderliter and her team to figure out what to do with all the intel pouring in. After first introducing GoGuardian, Pekin High was flooded with alerts and “kids searching for problematic stuff,” she says. “Problematic would be porn.” Her philosophy is fairly liberal. A few questionable Google queries won’t result in any discipline, though she calls students to her desk to warn them not to cross lines on a school laptop. “We don’t have the time or the staff to monitor every little thing they do,” she says.

Repeat offenders are put in what she calls “the penalty box”—a virtual form of detention that locks them out of anything unrelated to schoolwork. Hinderliter clicks on the search history of the sophomore recently busted for Googling “sexy girls,” unfurling a string of time stamps and other naughty terms. Hinderliter is letting it go for now but is keeping an eye on him in case he continues to look for adult content. “For an entire semester, we might have a kid in the penalty box,” she says. Hinderliter says she also has to remind at least one student each year not to send love letters through school email accounts.

For teachers, GoGuardian’s classroom tools are the major attraction. Riley Faille, a Pekin math teacher, says she logs in to GoGuardian from the laptop on her desk when she’s teaching so she can limit what students have open on their screens to only what she wants—a calculator or exam, for instance. During study halls she uses the software to watch what the students are doing. If someone strays from homework, she’ll open a tab on their screen and type GoDoYourWork.com. “I like stuff like that,” she says. She considers it a gentle way to remind them “Oh, I shouldn’t be doing this right now.”

At the end of each class, Faille gets a report with pie charts summarizing what students were doing on their laptops, which helps her see if they missed something while she was teaching. But on days when students are home because they’re sick or quarantining, she finds GoGuardian less useful. Last year, Pekin split kids into two cohorts, with each coming in every other day. On work-from-home days, students were free to set their own schedules. When kids were in class in person, Faille blocked sites such as YouTube, but she couldn’t limit YouTube usage for students at home because they might need access to online videos for their Spanish or French assignments.

No matter where kids are, Hinderliter watches GoGuardian closely every day. Almost half of Pekin’s high schoolers receive free or reduced lunch, and many take after-school jobs. During the pandemic, Hinderliter watched bills get paid and unemployment forms get filled out on student laptops, knowing that her kids’ Chromebooks were often the only computers at home.

Sometimes she saw more troubling indicators. Around dinnertime on a Thursday earlier this year, Hinderliter received a panicked alert. A student had Googled “How to tell your girlfriend if you’re suicidal.” Hinderliter and three other administrators sprang to action in a text thread, so that a school official could call the student’s home and alert a guidance counselor. Hinderliter declines to share details, other than to say that the student was ultimately all right.

Schmieg, the principal, says GoGuardian can help catch a slip in a student’s mental health well before teachers, counselors, or loved ones might notice. He says he has called parents, finding them entirely unaware that their child is in distress. “That’s happened multiple times in the past year,” he recalls.

This school year, Pekin began using Beacon, which automates the process of alerting administrators about students the software deems most at risk. Schmieg says he received more than 40 alerts in the first week on the service. With each alert, the software created a report of the student’s online activity, so he and school staff wouldn’t have to dig through their search history to try to figure out what was going on. “I already love it,” he says.

GoGuardian began testing Beacon back in 2018, but, like most of what the company offers, the service exploded in popularity after schools closed in March 2020. Last year, Clark County, an enormous district that includes Las Vegas, started using Beacon and saw more than 7,300 alerts, acting on almost a third of those. GoGuardian has promoted the new service as a way to combat the psychological strain students have faced during the pandemic. Shinde says the company “could be in a position to shine a flashlight on this issue.”

But some parents and privacy groups consider Beacon to be emblematic of larger problems with GoGuardian’s approach, which they characterize as unnecessary snooping that might actually be contributing to the psychological strain it’s supposed to mitigate. “We were teenagers, right?” says Geoff Shandler, a parent in Montclair, N.J., whose school system tested out GoGuardian. “You shut your door when you’re in your room. I don’t think anyone liked the idea of being watched and tracked.”

Shandler is a book editor, with three children in public schools (ages 8, 14, and 16). In February, in a weekly note to families, his district’s superintendent said Montclair was using GoGuardian in several classrooms. Parents in the left-leaning suburb freaked, which prompted local news coverage and an online petition. Part of the concern came because the district said GoGuardian’s filtering would be active whenever kids were logged in to their school accounts—whether that’s on school laptops or computers at home. (GoGuardian says schools choose whether to turn on the function that allows filtering on home computers and declined to share how many have opted to do so.) In Berkeley the public school district paused its rollout of GoGuardian last year after a similar outcry. Students “feel like they don’t have any space to themselves,” a sophomore complained to the school paper.

In October a trio of Democratic senators sent letters to GoGuardian and three of its competitors chiding them for surveilling students and “compounding racial disparities” that already exist in school discipline. “We need to protect our students from the long-lasting and harmful effects these invasions of privacy may have,” Massachusetts Senator Elizabeth Warren wrote in a statement to Bloomberg Businessweek. GoGuardian said in a statement that the company cares “deeply about keeping students safe and protecting their privacy.”

The company gives schools handouts that purport to explain how the technology works. Montclair’s schools tried addressing parents’ concerns, but Shandler found the information confusing. He felt that GoGuardian’s claims, particularly those about identifying students in danger, lacked supporting evidence. In a message to parents, Montclair Superintendent Jonathan Ponds wrote, “It may be helpful to know that 10,000 other schools, including our neighbors?…?use GoGuardian.”

Critics describe GoGuardian’s technology as a black box. Jonathan Singer, the former president of the American Association of Suicidology, says he worries that schools are deploying tools such as GoGuardian’s without empirical evidence of their impact on mental health or an understanding of the algorithms at play. “There has to be some public involvement in what the software is doing,” he says. While developing Beacon, a GoGuardian spokesman said the company had spoken with several experts, including Singer. Although Singer acknowledges he walked the company’s staff through his research, he says that was the extent of his involvement with Beacon. “I don’t really even know how it works.”

Schools, swamped by debates about reopening and testing and mask mandates, haven’t had much time to address these questions. As Delaware prepares to introduce GoGuardian into its schools, Alyssa Moore, the state official coordinating the rollout, says the education department hasn’t rigorously tested Beacon’s effectiveness and may not have a chance to prior to offering it to schools. “I need to get this system up and running before I do anything else,” she says.

When a school sets up Beacon, GoGuardian’s machine-learning models crunch everything students type and visit. According to Shinde, these systems can detect whether a post has “general suicide ideation” or is a true “high-stakes alert.” GoGuardian has a support team available around-the-clock to sift through machine alerts on behalf of schools that don’t have the staff or the inclination to do it themselves, giving the company more responsibility over how incidents are handled. When staffers review an alert and deem it serious, they call the school. Shinde says GoGuardian’s goal is to make sure a school is aware of the issue, but “ultimately that’s it. We want them to take the next action.” GoGuardian declined to say how many people are on the team, calling that figure proprietary. The company says it changes the number of its round-the-clock staff based on demand.

GoGuardian, like many other tech companies, declines to disclose the specifics of how its algorithms make determinations. Shinde says GoGuardian complies with all federal laws on student data and privacy. If police are brought in to investigate an alert, GoGuardian allows schools to decide what data to share. And the company contends that Beacon is effective. “We have customers who literally say, ‘There’s kids alive today that wouldn’t be alive without a tool like Beacon,’?” Shinde says. “So from an evidence perspective, that’s what we truly hang our hat on.”

Some of GoGuardian’s competitors are going further, selling services that scrape student searches, keystrokes, and social media posts in an attempt to read their minds and ferret out feelings of anger or harm. GoGuardian so far has steered clear of this type of so-called sentiment analysis. Having a computer play child psychologist isn’t realistic, Shinde says. Technology just doesn’t work well enough to do that reliably. But, he adds, “we’re certainly open to expanding to it one day.”

<<<