![](\"https:\/\/theresearchpapers.org\/archive\/wp-content\/uploads\/2021\/08\/4faf35712538cc501be4bfd809c2c4ee.gif\")
<\/p>\n<\/p>\n
This is an edited version of my notes for the lecture of my ongoing research on Beauty and the Machine (on machine learning and aesthetics) at the Critical Studies Department of the Sandberg Instituut.<\/em><\/p>\n Last year, during the series that I did on the artifice of intelligence here at critical studies, I opened the last lecture of the series discussing Karl Marx so in the interest of continuity, I shall recall this text briefly to bring us back to today or at least to attempt to draw a continuity of my research projects<\/p>\n Between the years of 1857 and 1858 in Fundamentals of Political Economy Criticism, a lengthy and morose unfinished manuscript, Karl Marx wrote:<\/p>\n As long as the means of labour remains a means of labour in the proper sense of the term, such as it is directly, historically, adopted by capital and included in its realization process, it undergoes a merely formal modification, by appearing now as a means of labour not only in regard to its material side, but also at the same time as a particular mode of the presence of capital, determined by its total process — as fixed capital. But, once adopted into the production process of capital, the means of labour passes through different metamorphoses, whose culmination is the machine, or rather, an automatic system of machinery (system of machinery: the automatic one is merely its most complete, most adequate form, and alone transforms machinery into a system), set in motion by an automaton, a moving power that moves itself; this automaton consisting of numerous mechanical and intellectual organs, so that the workers themselves are cast merely as its conscious linkages. In the machine, and even more in machinery as an automatic system, the use value, i.e. the material quality of the means of labour, is transformed into an existence adequate to fixed capital and to capital as such; and the form in which it was adopted into the production process of capital, the direct means of labour, is superseded by a form posited by capital itself and corresponding to it. In no way does the machine appear as the individual worker’s means of labour.\u00a0<\/p><\/blockquote>\n And I want to stress this part:<\/p>\n \u201cthe means of labour passes through different metamorphoses, whose culmination is the machine, or rather, an automatic system of machinery (system of machinery: the automatic one is merely its most complete, most adequate form, and alone transforms machinery into a system), set in motion by an automaton, a moving power that moves itself\u201d<\/p>\n an automaton, a moving power that moves itself.<\/strong>\u00a0<\/p>\n Given that scholars\u00a0often describe the Fundamentals as the rough draft of\u00a0Das Kapital, I believe it wouldn\u2019t be a stretch to say that automation was already a preoccupation in Marx theories of labor and capital. And not just any automation but the kind of automatic process that does not require human intervention in order to be completed. Moreover, he continues elaborating on the quality of this machine:\u00a0<\/p>\n \u201cRather, it is the machine which possesses skill and strength in place of the worker, [it is the machine that] is itself the virtuoso, with a soul of its own in the mechanical laws acting through it; and it consumes coal or oil just as the worker consumes food to keep up its perpetual motion.\u201d<\/p><\/blockquote>\n Now, there is a lot going on here: Marx creates a sort of \u201cworkflow\u201d through which he describes how the means of labour go through a metamorphoses so radical, so unprecedented that they become a machine that does not require human intervention. In turn, this machine is described as \u201chaving a soul\u201d.\u00a0<\/p>\n in his conception of capital, Marx talks about a perpetual drive to accumulation. In this conception, he envisions capital accumulation\u00a0as the operation through which profits are reinvested into the economy, increasing the total quantity of\u00a0capital (investments as further tool of creation of capital). Capital, then defined essentially as economic or commercial asset value that is used by capitalists to obtain even more value. This accumulation would, of course, impact material conditions and I want to briefly borrow from \u201cThe Social Thought of Karl Marx\u201d to get into the heart of this conception of accumulation of capital: materialist theory holds that humans and their interactions are intrinsically organic, physical, and temporal. This means that all human activities and all human societies can be analyzed according to humans\u2019 organic, physical, and temporal characteristics<\/em>. But, is this conception of sociality still relevant when we are in the age of cognitive machines? How can we situate these machines that Marx anticipated as \u201chaving a soul\u201d within this continuum of accumulation of capital?\u00a0<\/p>\n What are these \u201ccognitive machines\u201d? Rather than resort to academic definitions, I\u2019d rather we focus on what the owners of the means of producing foresight<\/em> have to say about cognitive machines:<\/a>\u00a0<\/p>\n Cognitive machines \u2013 these are industry-specific solutions, which are based on a number of core technologies \u2013 machine learning, natural speech processing, image recognition \u2013 and suitable infrastructure \u2013 cloud computing, the Internet of things, big data \u2013 and have innovative abilities, which can reasonably be described as higher level cognitive processes. According to a current study their use promises productivity increases of up to forty per cent. Yingxu Wang, Professor of Cognitive Informatics at the University of Calgary, identifies processes of understanding, learning and problem-solving, decision-making, planning, designing and pattern matching, in particular, as \u201ccognitive\u201d. The terms make clear that cognitive machines are \u2013 at least according to their potential \u2013 not only purely passive tools but self-willed agents, which have \u201cartificial intelligence\u201d and are able to penetrate deep into the fabric of business and society with their actions.\u00a0<\/p><\/blockquote>\n The latest progress in AI is essentially based on learning algorithms, which generate forms of artificial intelligence on the basis of huge quantities of data, making the ability to learn a fundamental product component. We must distinguish between two phases of learning. First, the training phase: during training, the product acquires the critical skills. This is followed by the application phase: the product improves while it is being used. Again, there are two parts to this. Firstly, context learning: the product adapts to the user and the actual context in which the product is being used.<\/p><\/blockquote>\n …<\/p><\/blockquote>\n Products, which are constantly improving and learning from the experiences of all the people using these products, are products whose capabilities will no longer be determined by the production run but by software updates. Consequently, it is precisely this software and not hardware that will be the critical value driver<\/strong> \u2013 regardless of whether we are speaking of wind turbines, air-conditioning systems or smartphones.<\/p><\/blockquote>\n \u00a0technologies used in affective computing will also allow machines to interpret emotional states correctly and react appropriately to the situation on the basis of the sound of voice and the user\u2019s facial expression.<\/p>\n Affective computing<\/strong>: the study and development of systems and devices that can recognize, interpret, process, and simulate human affects.<\/p>\n Manfred Edward Clynes was an Austrian scientist and inventor (who, incidentally, is credited for creating the word cyborg) who developed a way to measure neurophysiology responses to emotions. MIT\u2019s Affective computer lab has created numerous datasets of what they consider the 8 basic human emotions: These\u00a0eight\u00a0states (from the Clynes sentograph protocol) are: neutral, anger, hate, grief, love, romantic love, joy, and reverence.<\/p>\n From MIT\u2019s project page<\/a>:<\/p>\n This was the first data set generated as part of the MIT Affective Computing Group’s research. The research question motivating the collection of this particular data set was: Will physiological signals exhibit characteristic patterns when a person experiences different kinds of emotional feelings? We wanted to know if patterns could be found day-in day-out, for a single individual, that could distinguish a set of eight affective states (in contrast with prior emotion-physiology research, which focused on averaging results of lots of people over a single session of less than an hour.) We wanted to determine if it might be possible to build a wearable computer system that could learn how to discriminate an individual’s affective patterns, based on skin-surface sensing. We did build such a system, which attained 81% classification accuracy among the eight states studied for this data set, separating emotions not only on arousal but also on valence.<\/p><\/blockquote>\n The data set consists of measurements of four physiological signals and eight affective states, taken once a day, in a session lasting around 25 minutes, for over twenty days of recordings from an individual trying to keep every other aspect of the measurements constant (time of day, electrode placement, eliciting procedure, etc.) The four physiological signals are: blood volume pulse, electromyogram, respiration and skin conductance.<\/p>\n These datasets are now used to train algorithms for the recognition of affect.<\/p>\n trying to understand how cultural analytics would operate vis a vis Barthes\u2019 semiotic codes<\/a>, particularly in regards to teaching machines to identify highly subjective codes (not even necessarily shared among humans and so heavily dependant on culture\/ethnicity\/ gender etc). To go back to the previous point about MIT\u2019s \u201cbasic emotions\u201d, how are machines trained to distinguish the racial differences in neurological responses (for example, Black men have marked differences in blood pressure in relation to white men or white women; when these neurological responses are measured, which values are taken into account?)<\/p>\n The political implications of teaching these cultural specific aesthetics and affect to machines: Consider how, for example, Instagram\u2019s algorithm \u201crecommends\u201d certain images in detriment of others, how the algorithm deems a photo to be \u201cbeautiful\u201d and worth promoting etc<\/p>\n I am interested not only in how these decisions are made (ie a programer \u201ctaught\u201d the machine to aggregate specific types of data obviously) but also the taxonomies that went into play for this process and the type of aesthetic choices that are made to create the taxonomies. Not only how \u201cbeauty\u201d or \u201cugliness\u201d are coded into the machine but the cultural aspects of defining these categories.<\/p>\n On the politics of ugliness<\/a>, Sara Rodrigues et al 2018:<\/p>\n ugliness seems to emerge as a property or attribute of places and bodies rather than as a process that relies on an unjust distribution of value and power in relation to the workings of gender, ability, race, class, beauty norms, body size, health, sexuality, and age.\u00a0<\/p><\/blockquote>\n We position ugliness politically, rather than purely aesthetically, tracing its intersections with discourses, practices, and institutions of power.\u00a0<\/p><\/blockquote>\n Throughout, we focus on literature that is adept at exploring the politics behind the operations of ugliness\u2014that is, work within the fields of feminist theory, critical disability studies, sexuality studies, cultural studies, postcolonial literatures, and critical race studies.\u00a0<\/p><\/blockquote>\n We begin by exploring ugliness as a form of visual injustice, focusing in particular on how ugliness affects relating and how spatio-temporalities are organized to expunge bodies deemed ugly. Following this, we explore the materialization of ugliness through and on bodies as well as in representations.\u00a0<\/p><\/blockquote>\n Theoretical and scholarly work on ugliness has developed along two tracks. The first mostly developed within philosophy, elaborating upon ugliness as an aesthetic category opposed to beauty. This body of work suggests that ugliness is the direct opposite of beauty and that the two qualities are properties of seeing.\u00a0<\/p><\/blockquote>\n Such a view tends to naturalize ugliness as a property of objects, people, places, and of the technology of sight. The philosophical engagement with ugliness also binds its analysis to the examination of art and literature. We are interested in disassociating our project from the philosophy of aesthetics to pursue a more politicized understanding of ugliness and to consider how categories of ugliness are interlaced with and deeply underwritten by ability, age, gender, race, class, body size, health, and sexuality.\u00a0<\/p><\/blockquote>\n Critical disability studies theorist Tobin Siebers argues that while philosophy has sought to disembody aesthetics, aesthetics is inherently political and embodied. Tracing the notion and discipline of \u201caesthetics\u201d to eighteenth- century philosopher Alexander Baumgarten, Siebers asserts that \u201caesthetics […] posits the human body and its affective relation to other bodies as foundational to the appearance of the beautiful\u201d\u00a0<\/p>\n Borrowing from Rodrigues text, I invite you to politicise aesthetics and to critically examine every Instagram algorithm recommendation.\u00a0<\/p>\n Ed Finn, founding director of the Center for Science and the Imagination at Arizona State University:<\/a><\/p>\n Today, we experience art in collaboration with these algorithms. How can we disentangle the book critic, say, from the highly personalised algorithms managing her notes, communications, browsing history and filtered feeds on Facebook and Instagram? She exemplifies what philosophers call the extended mind, meaning that her memories, thoughts and perceptions extend beyond her body to algorithmically mediated objects, databases and networks. Without this externalised thinking apparatus, she is not the same critic she would be otherwise.<\/p><\/blockquote>\n if the algorithm is an externalised thinking apparatus, basically, teaching the algorithm to see means teaching the algorithm capitalist value: this is worth seeing, this is worth ignoring, this is beautiful, this is ugly, this thing needs to be protected or cherished, this can be discarded, this person is ugly, this other person is non compliant etc etc.\u00a0<\/p>\n It is here that I unequivocally observe that the algorithm itself becomes a key organisational structure: what I mean is, the taxonomies that determine what we see have been drawn from our own cultural values (and the concomitantly perceived as lacking in value) but at the same time, they operate as a super structure that perpetuates these values unexamined and offer a bureaucratic organisation for our aesthetics choices. \u201cthis is pretty, look at it\u201d says the algorithm and this statement does not exist in isolation: it exists as part of more than 500 years of intergenerational utterances of \u201cthis is pretty\u201d or \u201cthis is ugly\u201d.<\/p>\n The algorithm, as organisational structure also becomes a structure of administration: \u201cthis is pretty, look at it\u201d, functions in an attention economy where influencers and marketing organisations earn a living producing aesthetically pleasing content, as a form of administration of the capitalist value.<\/p>\n of course, the algorithm as organisational and administrative structure is evident in fields such as healthcare (for example, where the algorithm \u201cdenies\u201d treatment to a patient based on certain data points such as financial situation, life expectancy, age, etc). However, it is not merely an evidently biopolitical interface for the administration of life but a totallizing structure that informs our leisure, social relations, love life or art appreciation.<\/p>\n \u00a0A quick recap is in order: this current research is a continuation of last year\u2019s project about \u201cthe coloniality of the algorithm\u201d that situated Linnaean taxonomies at the heart of both colonial history and our contemporary uses of technology.<\/p>\n Thinking of how colonisers, upon meeting native americans or Black African women superimposed their aesthetic values on their bodies. It wasn\u2019t long ago that beauty contests were reserved to white women only. From the Smithsonian Magazine<\/a> Archive (emphasis mine):<\/p>\n \u00a0Since its inception, the pageant has evolved in some ways and not so much in others. The talent competition was introduced in 1938 so that perhaps the young women could be judged on more than just their appearance, but with that small bit of progress came regression. That same year, the pageant chose to limit eligibility to single, never-married women between the ages of 18 and 28. The kind of beauty the pageant wanted to reward was very specific and very narrow\u2014that of the demure, slender-but-not-too-thin woman, the girl next door with a bright white smile, a flirtatious but not overly coquettish manner, smart but not too smart, certainly heterosexual. There was even a \u201cRule 7,\u201d abandoned in 1940, that stated that Miss America contestants had to be \u201cof good health and of the white race.\u201d\u00a0<\/strong><\/p><\/blockquote>\n \u00a0I would add: it is not merely that the people who make programming decisions for algorithms are biased but that the building blocks of these technologies are based on data fields and taxonomies that were, from their inception, a project of exclusion. It is not just that humans enter their own biases into algorithms but that the entire field of data collection is a perpetuation of colonial biases that have never really gone away.<\/p>\n So, in this context, I am also interested in the idea of the simulacrum: how do machines learn what is real vs what is simulacra? (Side note for Blade Runner\u2019s machines themselves being the simulacra)\u00a0particularly because this distinction anchors our notions of purity and \u201crealness\u201d. Through the simulacrum, we expose the mechanisms through which we judge something pure or impure, fake or real.\u00a0<\/p>\n \u00a0Entrupy\u2019s CTO and co-founder, Ashlesh Sharma, completed a PhD at NYU specializing in computer vision, which allows computers to capture an object\u2019s microscopic surface data. Sharma saw the potential to use this technology to map what Entrupy calls the \u201cgenome of physical objects.\u201d<\/p>\n \u201cThe genome of physical objects\u201d<\/strong><\/p>\n Interesting to me is that the founders published a paper with the title \u201cThe Fake vs Real Goods Problem: Microscopy and Machine Learning to the Rescue<\/a>\u201d and that machines are being trained to detect simulacra at microscopic level. From the paper (again, emphasis mine):<\/p>\n we introduce a new mechanism that uses machine learning algorithms on microscopic images of physical objects to distinguish between genuine and counterfeit versions of the same product<\/strong>. The underlying principle of our system stems from the idea that microscopic characteristics in a genuine product or a class of products (corresponding to the same larger product line), exhibit inherent similarities that can be used to distinguish these products from their corresponding counterfeit versions.<\/p><\/blockquote>\n thinking of Deleuze\u2019s Difference and Repetition, particularly \u201csimulacra as the avenue by which an accepted ideal or “privileged position” could be “challenged and overturned” and simulacra as “those systems in which different relates to different by means of difference itself\u201d. Especially in terms of class issues surrounding counterfeit goods, how if the simulacra is \u201ctoo good\u201d, it shatters the class signifier through which \u201cdifferent relates to different\u201d (ie rich signaling their wealth to fellow rich). But, on the other hand, if the simulacrum can only be detected at microscopic level, how much of a simulacrum is it (at least in terms of delivering the class signifier its meant to deliver)? ie if the copy cannot be detected with the naked eye, if it looks authentic and serves the function for which it was created, then what is real?<\/p>\n Deleuze: \u201cThe simulacrum is not just a copy, but that which overturns all copies by also overturning the models\u201d<\/p>\n and\u00a0<\/p>\n \u201cDoes this not mean that simulacra provide the means of challenging both the notion of the copy and that of the model?\u201d<\/p>\n Again, from the paper (emphasis mine)<\/p>\n In the counterfeiting industry, most of the counterfeits are manufactured or built without paying attention to the microscopic details of the object<\/strong>. Even if microscopic details are observed, manufacturing objects at a micron or nano-level precision is both hard and expensive. This destroys the economies of scale in which the counterfeiting industry thrives. Hence we use microscopic images to analyze the physical objects.<\/p><\/blockquote>\n The authors of the paper describe this process as “feature extraction”.<\/p>\n \u201cFeature extraction\u201d, I want to put a finger in this notion of extraction because if the cognitive machine is good at something, it is precisely this.<\/p>\n In \u201cSettler Colonialism as Structure\u201d, Evelyn Nakano Glenn\u00a0writes:<\/p>\n In classic colonialism, the object is to exploit not only natural resources but also human resources. Native inhabitants represent a cheap labor source that can be harnessed to produce goods and extract materials for export to the metropole. They also serve as consumers, expanding the market for goods produced by the metropole and its other colonies. Goods and raw materials, like colonists, follow a circular path in classic colonialism.<\/p><\/blockquote>\n Extractivist practices have been extended to include not merely the physical (ie natural resources) but also data and intellectual property.\u00a0<\/p>\n In that sense, the algorithm serves a double function:<\/p>\n it extracts data<\/p>\n it protects capital (by detecting the simulacrum)<\/p>\n also, worth noting that the simulacrum \u201cdevalues\u201d the original by allowing access to the aesthetics of the real to the class of people who wouldn\u2019t otherwise be able to afford it. while \u201cthe fake\u201d could be seen as having a democratising effect, the algorithm is deployed to maintain the strict class divisions that only money can transcend. in that sense, the simulacrum devalues and corrupts the aesthetics of a certain class even if simulacrum and real are identical to the naked eye. I would go as far as saying that the class division is then maintained at microscopic level.<\/p>\n and on the issue of class, \u00a0a rhetorical question I\u2019ve been asking for a good while now: who owns the robots? of course I am talking about algorithm ownership as the new means of production (the hardware itself being easy to reproduce\/ scalable enough that it is not out of reach etc). With the algorithm, of course: the database, the information itself. once extractivism runs its course, the progression to the intangible, attuned to a world running out of physical resources to extract. Production then not just as a physical task but the body as data (what we consume\/ health metrics\/ movement\/ genetic information\/ preferences etc)<\/p>\n And again, I insist on robot ownership as a metaphor for the way these cognitive machines, this superstructure of administration and organisation operate because I want to bring attention to the several issues that permeate my research, namely:<\/p>\n the algorithm as an epistemic and pedagogical interface (we teach the algorithm to see and, in turn, the algorithm teaches us what is worth seeing, a perpetual motion of both teaching and learning the same desire(s); an ouroboros of aesthetics and capitalist value)<\/p>\n but also, as capitalism reaches resource scarcity, we shift the ownership from material resources to ownership of knowledge itself. To quote Yann Moulier Boutang, the French philosopher who coined the term, Cognitive capitalism is not only a type of accumulation oriented towards the valorisation of knowledge and innovation. It is also a new mode of capitalist production<\/em>.\u00a0<\/p>\n \u00a0Matteo Pasquinelli in \u201cThe Eye of the Algorithm: Cognitive Anthropocene and the Making of the World Brain<\/a>\u201d<\/p>\n Cognitive capitalism and cognitive Anthropocene<\/p>\n From an epistemological point of view, it is not arbitrary to establish a parallelism between the protocols used to intercept and \u2018forecast\u2019 anti-social behaviours and terrorism and the protocols used to intercept and \u2018forecast\u2019 the anomalies of climate change.<\/p><\/blockquote>\n it is not an extrapolation to add that these same protocols are equally deployed to forecast aesthetic choices. Appreciation of beauty (or ugliness) is not merely the realm of humans.<\/p>\n In a study published last year<\/a>, researchers from Cornell University\u00a0examined racial bias\u00a0on the 25 highest grossing dating apps in the US. They found race frequently played a role in how matches were found. Nineteen of the apps requested users to input their own race or ethnicity; 11 collected users\u2019 preferred ethnicity in a potential partner, and 17 allowed users to filter others by ethnicity.<\/p>\n Even the way the food we eat looks like is influenced by algorithms: the pervasive nature of Instagram has meant that restaurants and cafes are creating dishes to cater specifically to instagram users so that the photos get more recommendations, increasing the exposure of the cafe to attract more customers.<\/p>\n I took this photo in Lisbon last weekend. This cafe offering a menu identical to the menus found in any other city the world over. \u201coptimized\u201d not only for tourism but for a specific kind of tourist that seeks to photograph their \u201cfashionable\u201d food. Cognitive capitalism requires a portable, global aesthetic that can easily be reproduced everywhere. The english language is not the only hegemonic factor in these developments, a uniform look facilitates data aggregation as well. It reduces the number of specificities and variables that would have to be indexed. Even though pancakes are consumed in different forms the world over, the pancakes optimised for Instagram are of a specific type: American, piled on top of one another, dripping with syrup etc. Restaurants change their interior decoration to be more appealing for Instagram: white marble tables, sleek surfaces.\u00a0<\/p>\n \u00a0In Sao Paulo there is even an ice cream shop that was created with the express purpose of selling ice creams and providing backgrounds that would look good on Instagram. I took these photos this past summer and I didn\u2019t even capture all the aesthetic possibilities that the place offered to customers. There were more backgrounds and more props available. You\u2019d notice the bright colours which are also not gratuitous: apparently, Instagram is optimised for bright colors.<\/p>\n![](\"https:\/\/theresearchpapers.org\/archive\/wp-content\/uploads\/2021\/08\/fa4e788f0399ab4f62d23db1c6a738a6.gif\")
<\/p>\n![](\"https:\/\/theresearchpapers.org\/archive\/wp-content\/uploads\/2021\/08\/a8f65bd6939491fa86584d8bd646f443.gif\")
<\/p>\n![](\"https:\/\/theresearchpapers.org\/archive\/wp-content\/uploads\/2025\/04\/ec7cff5919dfca0e039cf0bd37efc427.png\")
<\/p>\n![](\"https:\/\/theresearchpapers.org\/archive\/wp-content\/uploads\/2025\/04\/7a224ff40079ea0ecf7e122050e398fc.png\")
<\/p>\n![](\"https:\/\/theresearchpapers.org\/archive\/wp-content\/uploads\/2025\/04\/5acd033ab1e0e8b0ae554300f27d6340.png\")
<\/p>\n![](\"https:\/\/theresearchpapers.org\/archive\/wp-content\/uploads\/2025\/04\/7b0cf38bca65ed5e5fa8e7e8ada262b5.png\")
<\/p>\n![](\"https:\/\/theresearchpapers.org\/archive\/wp-content\/uploads\/2025\/04\/1b4eacca86b5d7f69b5701a7d9b03d43.png\")
<\/p>\n![](\"https:\/\/theresearchpapers.org\/archive\/wp-content\/uploads\/2025\/04\/e8169c14ffba2e75e813b963a31cb4ac.png\")
<\/p>\n![](\"https:\/\/theresearchpapers.org\/archive\/wp-content\/uploads\/2025\/04\/e4679175387ead84bf38ecead9b66324.jpeg\")
<\/p>\n![](\"https:\/\/theresearchpapers.org\/archive\/wp-content\/uploads\/2025\/04\/2d31fec24201384fd4cd39916deeb13e.png\")
<\/p>\n![](\"https:\/\/theresearchpapers.org\/archive\/wp-content\/uploads\/2025\/04\/3f71c50eacaedc32b8fcfbd427761bc9.jpeg\")
<\/p>\n![](\"https:\/\/theresearchpapers.org\/archive\/wp-content\/uploads\/2025\/04\/73355d7303f2d0dcc4dabf6eeaf82090.jpeg\")
<\/p>\n![](\"https:\/\/theresearchpapers.org\/archive\/wp-content\/uploads\/2025\/04\/f0402f4fa8586d3a4373fe9d745107ed.jpeg\")
<\/p>\n