Most of us desire to be in a position to speak our minds onlineto be heard by our friends and talk (back) to your opponents. Simultaneously, we dont desire to be subjected to speech that’s inappropriate or crosses a line. Technology companies address this conundrum by setting standards free of charge speech, a practice protected under federal law. They hire in-house moderators to look at individual bits of content and take them off if posts violate predefined rules set by the platforms.
The approach clearly has problems: harassment, misinformation about topics like public health, and false descriptions of legitimate elections run rampant. But even though content moderation were implemented perfectly, it could still miss a complete host of conditions that tend to be portrayed as moderation problems but are really not. To handle those non-speech issues, we are in need of a fresh strategy: treat social media marketing companies as potential polluters of the social fabric, and directly measure and mitigate the consequences their choices have on human populations. Which means establishing an insurance plan frameworkperhaps through something comparable to an Environmental Protection Agency or Food and Drug Administration for social mediathat may be used to identify and measure the societal harms generated by these platforms. If those harms persist, that group could possibly be endowed having the ability to enforce those policies. But to transcend the limitations of content moderation, such regulation would need to be motivated by clear evidence and also have a demonstrable effect on the issues it purports to resolve.
Moderation (whether automated or human) could work with what we call acute harms: those caused directly by individual bits of content. But we are in need of this new approach because there’s also a bunch of structural problemsissues such as for example discrimination, reductions in mental health, and declining civic trustthat manifest in broad ways over the product instead of through anybody little bit of content. A famous exemplory case of this type of structural issue is Facebooks 2012 emotional contagion experiment, which showed that users affect (their mood as measured by their behavior on the platform) shifted measurably based on which version of the merchandise they were subjected to.
In the blowback that ensued following the results became public, Facebook (now Meta) ended this kind of deliberate experimentation. But because they stopped measuring such effects will not mean product decisions dont continue steadily to keep these things.
Structural problems are direct outcomes of product choices. Product managers at technology companies like Facebook, YouTube, and TikTok are incentivized to target overwhelmingly on maximizing time and engagement on the platforms. And experimentation continues to be quite definitely alive there: nearly every product change is deployed to small test audiences via randomized controlled trials. To assess progress, companies implement rigorous management processes to foster their central missions (referred to as Objectives and Key Results, or OKRs), even using these outcomes to find out bonuses and promotions. The duty for addressing the results of product decisions is frequently positioned on other teams which are usually downstream and also have less authority to handle root causes. Those teams are usually capable of giving an answer to acute harmsbut often cannot address problems due to the merchandise themselves.
With attention and focus, this same product development structure could possibly be considered the question of societal harms. Consider Frances Haugens congressional testimony this past year, alongside media revelations about Facebooks alleged effect on the mental health of teens. Facebook responded to criticism by explaining that it had studied whether teens felt that the merchandise had a poor influence on their mental health insurance and whether that perception caused them to utilize the merchandise less, rather than if the product actually had a negative effect. As the response could have addressed that one controversy, it illustrated a study aiming directly at the question of mental healthrather than its effect on user engagementwould not be considered a big stretch.
Incorporating evaluations of systemic harm wont be easy. We’d have to straighten out what we are able to actually measure rigorously and systematically, what we’d require of companies, and what issues to prioritize in virtually any such assessments.
Companies could implement protocols themselves, but their financial interests all too often run counter to meaningful limitations on product development and growth. That the truth is a typical case for regulation that operates with respect to the general public. Whether by way of a new legal mandate from the Federal Trade Commission or harm mitigation guidelines from the new governmental agency, the regulators job is always to use technology companies product development teams to create implementable protocols measurable through the course of product development to assess meaningful signals of harm.
That approach may sound cumbersome, but adding these kinds of protocols ought to be straightforward for the biggest companies (the only real ones to which regulation should apply), since they have previously built randomized controlled trials to their development process to measure their efficacy. The more time-consuming and complex part will be defining the standards; the specific execution of the testing wouldn’t normally require regulatory participation at all. It could only require asking diagnostic questions alongside normal growth-related questions and making that data accessible to external reviewers. Our forthcoming paper at the 2022 ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization will explain this process in greater detail and outline how it might effectively be established.
When products that reach tens of millions are tested for his or her capability to boost engagement, companies would have to make sure that those productsat least in aggregatealso follow a dont make the issue worse principle. As time passes, more aggressive standards could possibly be established to roll back existing ramifications of already-approved products.
There are various methods that could be ideal for this kind of process. Included in these are protocols just like the photographic affect meter, which includes been used diagnostically to assess how contact with services and products affects mood. Technology platforms already are using surveys to assess product changes; according to reporters Cecilia Kang and Sheera Frankel, Mark Zuckerberg talks about survey-based growth metrics for some every product decision, the outcomes which were section of his choice to roll back the nicer version of Facebooks news feed algorithm following the 2020 election.
It could be reasonable to ask if the technology industry sees this process as possible and whether companies would fight it. While any potential regulation might engender this type of response, we’ve received positive feedback from early conversations concerning this frameworkperhaps because under our approach, most product decisions would pass muster. (Causing measureable harms of the type described this is a high bar, one which most product choices would clear.) And unlike other proposals, this plan sidesteps direct regulation of speech, at the very least beyond your most acute cases.
Simultaneously, we dont need to await regulators to do this. Companies could readily implement these methods by themselves. Establishing the case for change, however, is difficult without starting to collect the type of high-quality data were describing here. That’s because one cannot prove the existence of these kinds of harms without real-time measurement, developing a chicken-and-egg challenge. Proactively monitoring structural harms wont resolve platforms content issues. Nonetheless it could allow us to meaningfully and continuously verify if the public interest has been subverted.
THE UNITED STATES Environmental Protection Agency can be an apt analogy. The initial reason for the agency had not been to legislate environmental policy, but to enact standards and protocols in order that policies with actionable outcomes could possibly be made. From that time of view, the EPAs lasting impact had not been to solve environmental policy debates (it hasnt), but to create them possible. Likewise, the initial step for fixing social media marketing would be to create the infrastructure that well need to be able to examine outcomes in speech, mental well-being, and civic rely upon real-time. Without that, we are prevented from addressing some of the most pressing problems these platforms create.
Nathaniel Lubin is really a fellow at the Digital Life Initiative at Cornell Tech and the former director of any office of Digital Strategy at the White House under President Barack Obama. Thomas Krendl Gilbert is really a postdoctoral fellow at Cornell Tech and received an interdisciplinary PhD in machine ethics and epistemology at UC Berkeley.