BROKEN CODE

Things ramped up once Trump tweeted, on December 19, that there would be a "big protest in D.C. on January 6th. Be there, will be wild!" The tweet blindsided the Kremers and White House staff alike, and it didn't unite anyone so much as ramp up the ante of the squabbling. The stakes were getting bigger, and anyone who was anyone in the world of right-wing influencers wanted a piece of the action.

The seventy-three-year-old heiress to the Publix supermarket fortune donated $3 million in total to the effort, some of it going to [Alex] Jones, some to Trump advisor Roger Stone, but a big chunk—$1.25 million—going to Charlie Kirk, the founder of the conservative student group Turning Point USA, to "deploy social media influencers to Washington" and "educate millions." When investigators later confronted Kirk with documents showing he had billed the heiress for $600,000 of buses that were never chartered, he responded by invoking his Fifth Amendment right against self-incrimination.

This band of self-described patriots came together at a unique moment in American history, with social media coming fully into its power, and they had a ready audience in the social-media-obsessed president in the White House. Trump wanted the likes of Alexander and Jones to speak at his January 6 event, according to texts sent by one of his aides, Katrina Pierson. Or as she put it in a text to Kylie Kremer: "He likes the crazies." There was a nominal division between the "crazies" on one side and the Kremers on the other, But all were coming together to make sure January 6 would be unforgettable.

"I mean, there were so many things that were being said or pushed out via social media that were just concerning," Kylie Kremer told investigators, while defending the decision to maintain a loose alliance with what she variously called mercenary, larcenous, and quite possibly mentally ill social media activists who were posting about civil war, 1776, and their willingness to die for liberty. "It took all of us getting the messaging out to get all the people that came to DC," she said.

Pierson's "crazies" were in fact the luminaries of Zuckerberg's Fifth Estate.

– Pages 231-232

The story of how Facebook's defenses were so soundly defeated by such a hapless crew begins right after the Stop the Steal group's takedown, and it starts right at the top.

Although Facebook had vaguely alleged that it had taken down the group because of prohibited content, the truth was that the group hadn't violated Facebook's rules against incitement to violence, and the platform had no policy forbidding false claims of election fraud. Based on the group's obvious malignancy, however, Bickert and Facebook's Content Policy team had declared a "spirit of the policy" violation, a rare but not unheard-of designation that boiled down to "because we say so."

Zuckerberg had accepted the deletion under emergency circumstances, but he didn't want the Stop the Steal group's removal to become a precedent for a backdoor ban on false election claims. During the run-up to Election Day, Facebook had removed only lies about the actual voting process—stuff like "Democrats vote on Wednesday" and "People with outstanding parking tickets can't go to the polls." Noting the thin distinction between the claim that votes wouldn't be counted and that they wouldn't be counted accurately, Chakrabarti had pushed to take at least some action against baseless election fraud claims.

Civic hadn't won that fight, but with the Stop the Steal group spawning dozens of similarly named copycats—some of which also accrued six-figure memberships—the threat of further organized election delegitimization efforts was obvious.

Barred from shutting down the new entities, Civic assigned staff to at least study them. Staff also began tracking top delegitimization posts, which were earning tens of millions of views, for what one document described as "situational awareness." A later analysis found that as much as 70 percent of Stop the Steal content was coming from known "low news ecosystem quality" pages, the commercially driven publishers that Facebook's News Feed integrity staffers had been trying to fight for years.

Civic had prominent allies in this push for intelligence gathering about these groups, if not for their outright removal. Facebook had officially banned QAnon conspiracy networks and militia groups earlier in the year, and Brian Fishman, Facebook's counterterrorism chief, pointed to data showing that Stop the Steal was being heavily driven by the same users enthralled by fantasies of violent insurrection.

"They stood up next to folks that we knew had a track record of violence," Fishman later explained of Stop the Steal.

But Zuckerberg overruled both Facebook's Civic team and its head of counterterrorism. Shortly after the Associated Press called the presidential election for Joe Biden on November 7—the traditional marker for the race being definitively over—Molly Cutler assembled roughly fifteen executives that had been responsible for the company's election preparation. Citing orders from Zuckerberg, she said the election delegitimization monitoring was to immediately stop.

– Pages 233-234