Social Reproduction and Social Credit Apparatuses

Dystopia is a peculiar concept. Like dark matter or ‘the market,’ dystopias are generally difficult to find in the wild even while there are always ‘future dystopias’ looming on the horizon. While many places on earth may already deserve the title – Congolese cobalt mines, the Gaza Strip, secret Chicago prisons – our interpretation of dystopia is chiefly one based on temporality. Things are changing, and we fear the worst will happen. Or, what’s happening is bad, but don’t worry – it gets worse.

The types of futuristic societies envisioned by Reijers and van ‘t Klooster provide outwardly opposing views. But whether dystopian or not, the ‘social’ of the social credit system under debate remains largely untroubled. For Reijers, it’s simply an instrumentalised ‘social order’ within China; for van ‘t Klooster, it’s an ideal market with potential for just consequences. The social serves as a placeholder for something more, an unstated premise that hides in plain sight.

In the case of China, the dystopian algorithmic future of this placeholder is seasoned by Orientalist appreciation. The widely cited Bonaparte quotation ‘China is a sleeping lion. Let her sleep, for when she wakes she will shake the world’ locates this futurity on a terrain of awe-inspired fear. In a 2014 speech in Paris, even Chinese President Xi Jinping recited the passage, marshalling the implied danger that establishes China as an imminent, not-to-be-messed-with seismic force to an apprehensive French audience.

And for the West, with the narrativised help of Black Mirror’s ‘Nosedive’, China’s Social Credit System supplies the ‘ur text’ of the intersection between technology and dystopia for our contemporary era. But while social credit systems of any form are appalling for many reasons, I want to resist relying on a good vs. bad framing without taking a breath to assess a presupposition. What do all social credit systems purportedly want; what is the ‘social’ in social credit? I answer with conventional wisdom from digital studies, science and technology studies, and critical theory: to reproduce the social, just as asymmetric, biased, and discriminating as it was when it was born.

Social, So What?

I am no expert of China. And while I am a professor who writes and teaches on digital technology and cultural studies, my interest in the issue of China’s Social Credit System is one that doesn’t necessarily fit the typical Orwellian shade of authoritarianism. I propose we briefly remove China from our analysis and think about ‘social credit system’ in the abstract, alleviating some of the more problematic points of cross-cultural pressure – like racial othering and cultural fetishization – normally assigned to China by a Western audience. In doing so, I structure my perspective in an Althusserian form, concerned not only with the power to make ‘good citizens’, but also the process by which that power is constructed and maintained.

Abstracting a social credit system allows us to ask more general questions. Most important: what in the world is the ‘social’? Any social system, credit or not, needs to reproduce itself. For Althusser, the social conditions of life are reproduced – just like labour power – through repression and/or ideological interpellation. While Althusser’s perspective has been long-troubled by scholars, it remains a useful analytic of the social that refuses a vague, neutral origin story. Rather, the social is a constantly reorganised definition of community that always exploits some for the sake of others. Far from Hobbesian or Lockean mythologies of harmonious nation-building, Althusser’s formulation apprehends empowered difference, i.e., those with power vs. those without, as its structural a priori.

But social credit systems in general are more intimate than some rudimentary ideological relation. Reijers explains that such a system ‘extends the notion of a permanent record tied to a person to a great many kinds of everyday behaviours’, a tying that combines singular institutions of power (the police, education, the media) into one agglomerated ideological social factory. This definition of society routes through the calculations of biopolitics and statistics, scarred and disfigured by the founding limitations of datafication.

Limits of Datafication

One limitation of datafication: as much as dystopian imaginations tend to universalise their anxieties onto all aspects of life, there can be no completeness of surveillance and control. From Chinese Social Credit, to Facebook’s ‘Trustworthiness’ scale, and to even Pinochet’s ‘Directory of Commercial Information’, all these systems rely on complex, but still incomplete, methods of deducing an individual’s value to a society. Subjects are not interpreted, as is, but forced to prefigure their actions based on how they are datafied: smiling at a stranger won’t raise your score while buying vegetables will.

In this case, civic obligations are configured by the recognised perimeter of datafication, a street light effect of virtue where civic virtue that is untranslatable into data gets demoted or ignored. While virtue itself can never be separated from existing reward structures in society (in the US, ‘I Voted’ stickers, not votes, operate as the real designation of virtue), an ideal of virtue configured according to data and its algorithmic outcome pursue a value structure according to what is interpreted to be measured as virtuous, not what may be socially deemed as virtuous. The social is reproduced in intermediary, data-based, form.

Another limit of datafication: that social credit systems rely exclusively on data means subjects are not just seen by power but seen by power through data. The production and reproduction of the social is how data produces and reproduces its datafied subjects. Yet instances of datafication are exceedingly social – and I do not mean this in a good way. Just like the social reproduces the social, readers interested in questions of digital technology may be familiar with the fact that algorithmic processing of data also inevitably reproduces existing biases and oppressions. Facial recognition systems are notoriously racist and transphobic. Algorithmic hiring systems redouble sexism. Quantified self-tracking systems further alienate and dispossess users with disabilities.

Datafication is thus an intensely political process, and that ‘any behaviour that can be reliably measured is a potential input for a social credit rating’, citing van ‘t Klooster, should worry us. What it means to ‘reliably measure’ is not merely a question of scientific method as measurement itself can only exist within the realm of politics. We see this in China. While the persecuted Uyghur ethnic group in the country’s western Xinjiang region are certainly repressed by security forces, the Uyghurs – their faces and names – are also studied in ways that may technologically improve that repression. Just like how the post-9/11 US push to hire more Arabic translators allowed for increased surveillance and thus measurement of Arab-speaking populations, measurement is not just using a ruler to assess the length of an object. It is an intensely political act, an act where to ‘reliably measure’ this social in unqualified form is, at face value, unjust. If an algorithmic social credit system might lead to dystopia, I would not really blame the algorithm but the ‘topia’ it comes from.

Qualifications for the Future

Ironic for an argument of abstraction, I end with an appeal for qualification. Much like an unqualified ‘citizen’ or ‘civic virtue’ – which strategically positions the rights and obligations of citizen away from its lived, differential reality, and thus affirms an ideal that likely will not apply in practice – the qualified examples of the US and China show us that not all types of people are citizens in the same way. The ideal of the nation, and the society that grounds that ideal, is itself exclusionary. While Arab people in the US are formally citizens, their enacted practice of citizenship is ontologically conditioned through the lens of white supremacy. The same, but in distinct ways, with China and its Uyghur population. When datafication reproduces the social, it is precisely this relationship of marginalization that is intentionally and unintentionally datafied.

We need to both qualify subjects of surveillance and the underlying assumptions of that surveillance. Neither capitalism nor socialism was discussed in Reijers or van ‘t Klooster – only an unqualified market – in debates about the efficacy of social credit systems for enforcing authoritarianism or producing justice. I return to the origin question: what is the ‘social’ of social credit? At the end of the day, these systems are not currently for the demos. The social here is deployed as a source of information, turned against the demos by forces hegemonically empowered to maintain ideological control. If there is to be any justice, I would most intently ask: how do we make these systems about us, a social credit system democratically operated – not a social credit by a socialist social credit system?