Abstract
This chapter explores personalization and its connection to the philosophical concept of the person, arguing that a deeper understanding of the human person and a good society is essential for ethical personalization. Insights from artificial intelligence (AI), philosophy, law, and more are employed to examine personalization technology. The authors present a unified view of personalization as automated control of human environments through digital platforms and new forms of AI, while also illustrating how platforms can use personalization to control and modify persons' behavior. The ethical implications of these capabilities are discussed in relation to concepts of personhood to autonomy, privacy, and self-determination within European AI and data protection law. Tentative principles are proposed to better align personalization technology with democratic values, and future trends in personalization for business and public policy are considered. Overall, the chapter seeks to uncover unresolved tensions among philosophical, technological, and economic viewpoints of personalization.
Top1. Introduction
The creeping rise of government and corporate surveillance. The viral spread of misinformation and conspiracy theories. The growing ideological polarization of politics and society. The profit-driven promotion of addictive social media behaviors. The increasing concern that algorithmic discrimination reproduces social, historical, and economic inequities. What do all of these have in common? They are all, at least in part, believed to stem from applications of artificial intelligence (AI). In particular, these ethically, socially, and personally troubling effects are believed to stem from an increasingly widespread form of AI-based technology: personalization.
At the same time, however, many applications of personalization seem intuitively beneficial. Personalized medicine promises to ease human suffering and extend our lifespan by providing more effective, precise, and evidence-based treatments of medical conditions based on our unique genetic profiles and biomarkers (Jameson & Longo, 2015). Personalized behavioral interventions promise to help cure us of our bad habits, improve our physical fitness, and promote our psychological well-being. Personalized education promises to provide customized sequences of exercises to help us master new skills and domains of knowledge. Positive use cases for personalization abound, and more remain to be discovered, particularly in the realm of health, public policy, and government. Legal scholars have even suggested that law itself would benefit from becoming more personalized1 (Ben-Shahar & Porat, 2021).
Personalization is a multi-faceted concept often used synonymously with customization, user modeling, behavioral profiling, algorithmic selection, computational advertising or actuarial prediction, depending on the field of study. But surprisingly, very little of the extant research on personalization views it from a humanistic angle or relates it to any philosophical concept of the person. We believe this is a major oversight because cogently articulating why a particular application of personalization is good or bad generally requires justifying one’s evaluation through the concepts of a particular ethical theory (Gal et al., 2022). To date, however, most academic discussions of personalization tend to treat its technical foundations separately from its ethical implications. For instance, personalization researchers have generally focused on how it can reduce information overload and search costs, improve decision-making, and boost the user experience (Häubl & Trifts, 2000). The dominant view of personalization seems to be that it is an AI-driven process that permits better preference matching, reduces cognitive load, and makes performing digital tasks more convenient (Aguirre et al., 2015).
Personalization is also a major source of business value. Personalization is a competitive advantage and means to greater customer satisfaction, loyalty, and profits (Murthi & Sarkar, 2003). Indeed, the technological evolution of personalization really begins with the emergence of a new business model: the platform-based business. Personalization offers firms new dynamic capabilities for not only responding to changes in fast-changing digital environments, but importantly, for actively creating change in digital environments in pursuit of longer-term business goals. Technology firms such as Alphabet, Facebook (Meta), and Amazon derive much of their business revenue from the ownership and control of highly popular digital platforms offering vast ecosystems of digitally-connected products and services (Van Dijck et al., 2018). These companies have consistently proven their ability to find creative ways of monetizing user activity data (Wang et al., 2022; Zuboff, 2019). Consequently, AI-centric, platform-based companies have the motivation, financial resources, and data science talent to steer the future of personalization. But can we trust them with the keys?
Key Terms in this Chapter
Consequentialism: A system of ethical thought associated with utilitarianism in which the rightness or wrongness of an action derives from its external consequences (e.g., increased net utility), rather than its internal motivations. Consequentialism describes how artificial agents reason about the value of actions.
Control: A key concept in engineered systems that relies on the notion of feedback. A controller senses the state of a system, compares it against the desired or reference behavior, computes corrective actions, and intervenes in the system to effect the desired change. Some control approaches require a model of the underlying system dynamics, while others, such as those based on reinforcement learning, can learn a model of the system while interacting with it.
Reinforcement Learning: An approach to AI influenced by psychology, animal learning, neuroscience, and control theory that studies how artificial agents interacting with an environment learn to accumulate reward by solving sequential decision-making problems under uncertainty, delayed action-outcome pairings, and using evaluative feedback.
Personalization: Personalization is the goal-directed process of taking actions, making recommendations or decisions, or allocating resources on the basis of measured states of humans, the digital environment, and their interactions, and possibly adaptively modifying these actions after observing feedback about their consequences.
Algorithmic Behavior Modification (BMOD): Any algorithmic action, manipulation, or intervention on digital platforms intended to impact user behavior.
Autonomy: The (idealized) capacity of a person to be self-lawgiving and self-determining in choosing one's ends or goals. A key precondition for informed consent, autonomy is freedom from both internal (e.g., addictions, false beliefs) and external forms of control (e.g., physical force, threat).
Platform: Both a digital infrastructure and business model based on designing digital multi-sided markets for the systematic collection, algorithmic processing, circulation, and monetization of user data.
Value Alignment: The task of creating artificial agents that behave in accordance with users' or designers' intentions and ethical values.