Untrustworthy: Social Credit Isn’t What You Think It Is
Technology is redefining citizenship, governance,
privacy, and individuality in China and elsewhere. We face challenges of
securing personal data, limiting collection and use of online information, and
controlling the spread of misinformation. Many of these issues may be global,
but China is different. It lacks a healthy civil society or free media to
foster debate and draw attention to potential abuses, it lacks independent
courts that might check police power, and it has a government that views
individual rights as inherently subordinate to obligations to maintain social
order and stability.
This means there will be fewer brakes on government
experimentation with new technologies in the pursuit of stability and order.
There will be less resistance to invasive data collection, wrongful arrests
from facial recognition misidentifications, or over-inclusive profiling of
potential criminals by opaque algorithms. Stories
from Xinjiang already confirm these fears.
Despite these differences, or because of them, there
can be great comparative value for democracies in watching China’s integration
of technology, governance, and society, but meaningful comparison requires
accurate understanding. Social credit should not be used as a stand-in for all
Chinese government uses of new data technology, nor should China be used solely
as a foil for our fears about technology’s creeping reach into privacy and
In his blog post, Reijers defines the
essential characteristics of a social credit system as including tracking and
surveillance, scoring, and gamification through rewards and punishments. He further distinguishes ‘social credit
systems’ from other recordkeeping or ratings schemes based both on the breadth
of inputs (including ‘a great many
kinds of everyday behaviours’) and the ‘integral character’ of the output ‘score’, which creates a
generalised, holistic profile assessment connecting seemingly unrelated
These criteria highlight core concerns about how
information from our rapidly expanding digital presences might be used for
control by corporations and governments, but they are neither closely derived
from Chinese law nor reflective of its practice.
How Social Credit Works
Generally speaking, China’s social credit system includes three discrete pillars: 1) A financial credit reporting component similar to that found in many nations, which seeks to evaluate one’s ability to repay loans; 2) an administrative enforcement component involving a series of industry blacklists and joint punishment agreements for those who violate laws and regulations; and 3) a moral education component that focuses on teaching values of honesty and integrity.1) While there is some overlap between these areas, in that say, a conviction for fraud would be relevant to all three, they have different goals, methods, and even terminology, and should be considered individually.
Consider the inputs of surveillance and behaviour tracking. China is widely regarded as a surveillance state, and its law enforcement has sweeping investigative powers with minimal external restraints. Social credit could be said to facilitate tracking by consolidating existing information records under citizen and organization identification numbers, and publicly disclosing them for limited time, but it does not really expand the scope of government data collection. It is instead concerned with public, government-created records such as for administrative punishments, permits, or licenses that reflect natural persons’ and organizations’ compliance with legal and contractual obligations in social and economic activity.
As for Reijers’ expected output of holistic scoring or ranking, neither the 5-year planning
document initiating the establishment of China’s social credit system nor any
other higher-level social credit legal authority mentions a social credit score. A few dozen regions have created
local ‘scoring’ systems for ‘personal integrity’ (the language of the moral
education component) but point deductions in these systems result from violations
of legal obligations, not an integrated behavioural assessment. Moreover, the
rewards and punishments are nominal in comparison to existing punishments for
those same violations. The criminal penalty for drunk driving, for example, is
likely of more concern than that the conviction will lower one’s score. Even
the most prominent point systems have so little impact on employment,
education, and daily life that most
citizens remain totally unaware of them.
Within certain fields and industries, regulatory
agencies reward consistent compliance with facilitated and priority services
(‘green channels’) while violations can result in heightened scrutiny or
restrictions. This type of ‘ranking’ is industry-specific rather than
generalised to other areas, meaning a company viewed as a compliant or
non-compliant ‘food producer’ would only face consequences in the area of food
Entities with more serious violations may be blacklisted by industry authorities, which does result in ‘joint-enforcement’ penalties imposed by other regulatory bodies in unrelated fields, based on inter-agency agreements.2) These punishments often include consideration of the blacklisting when applying for professional credentials and licenses, and limits on bidding in government projects or receiving public funding. Blacklisting by the FDA for selling tainted foodstuffs, for example, can impact applications to run a securities trading business.
More exotic penalties, such as restrictions on purchase of airplane tickets and other expensive goods, are essentially unique to the court systems’ blacklist for those refusing to perform on valid court judgments. The appropriate scope and legality of such restrictions, and how long they should be enforced, remain hotly debated topics in Chinese legal circles.
In short, it is questionable whether China’s emerging
social credit system described above would even qualify as ‘social credit’
under Reijers’ criteria.
A well-informed citizen is a vigilant citizen
Posing questions about how technology can be used to
shape citizens, and change what it means to be a citizen, is of critical and
immediate importance, but using China as a blank slate on which we project
hypotheticals causes more confusion than clarity. It can distract us from more
pressing concerns regarding China, technology or both.
As to China, the distraction may be direct, in
shifting our focus from other crises like that in Xinjiang, or indirect,
creating confusion as to the true causes for concern about social credit. If
social credit is largely a mechanism for enforcing laws, is the real issue the
enforcement or the unjust laws such as those that limit speech? Is the real
concern China’s actual social credit system, or a society perceived as so
undemocratic that it could not meaningfully resist the imposition of a system
like the one that Reijers envisions?
For understanding our own situation, overstating China’s use of technology can be just as harmful as underestimating its use by our own governments and corporations. Because China is often held up as embodying one extreme on a spectrum, misrepresentations of its law and practice can shift the boundaries of the global discussion, making anything less invasive seem acceptable by comparison. This can lead to complacency as technology shapes our societies back home. If we only envision a caricature of China, with citizens labouring under an all-seeing algorithm, we ignore how subtle, and even comfortable, a control-based model can be – we lower our guard to more realistic threats to freedom and autonomy that surveillance capitalism is bringing to us all.
关于加强和规范守信联合激励和失信联合惩戒对象名单管理工作的指导意见. Translation: ‘Guiding Opinions on Strengthening and Regulating Efforts on the Management of Lists of Persons Receiving Joint Incentives for Trustworthiness or Joint Punishment for Untrustworthiness’ available here.
Daum, Jeremy: Untrustworthy: Social Credit Isn’t What You Think It Is, VerfBlog, 2019/6/27, https://verfassungsblog.de/untrustworthy-social-credit-isnt-what-you-think-it-is/, DOI: 10.17176/20190627-112616-0.