Goggle Capitalism - On yer face and in yer space!

The new techno-goggles are about looking into (and manipulating) you.

Like many of my age and social background my life has been successfully colonized by Apple Inc of Cupertino, California. Depressingly, I may spend more time in a typical day touching Apple products (laptop, iPhone) than physically touching or connecting with human beings or the material of the natural world. I’m far from an extreme case - I  don’t wear an Apple watch. I never succumbed to the iPad.  Now Tim Cook’s Cupertino crew want people like me (and you) to  increase our corporate-connection time  by physically strapping the latest Apple product onto the top half of our face.

I’m talking about the Apple Vision Pro headset - a sort of tricked-out pair of techno ski-goggles that promise to immerse the wearer in a new paradigm of ‘spatial computing’. With goggles on, the ubiquitous screen that’s crept into every corner of our lives will now expand to fill our peripheral vision too. Instead of typing or scrolling on glass or plastic we will finger-click or eye-roll our way around a floating digital universe overlaid on the real world in ‘augmented reality’. Thankfully at almost 4000 US dollars per pair I don’t expect anyone in my immediate vicinity to be buying one right away  - but reporting suggests that a cheaper less ‘pro’ version is on the way soon too and that there may be 20 million users within 5 years. Moreover the arrival of  Apple’s goggles has intensified the race for other tech giants such as Meta, Google and Microsoft to also land their  ‘headset’ devices on your face as soon as they can. Ultimately the expectation is that the goggles themselves  will shrink - first to glasses and then one day to invisible ‘smart’ contact lenses. One way or another there is a pair of techno goggles coming soon to a face near you.

What should we make of this coming gogglification trend? Predictably and refreshingly there have been a lot of critical takes on this strapped-on vision of the future. We have after all had decades of cyberpunk sci-fi movies to  think about it. Others have eloquently expressed how strapping yet another technological layer between humans in our communication is unpleasant to consider when we are already aggressively isolated and distracted by our smartphones, zoom rooms and social media feeds. Many young children are  growing up to viscerally detest or unhealthily desire the metal and glass totem that routinely places itself between them and their parents. Adults in turn bemoan how the same devices  steal their adolescents, partners or friends away from being present with them. I know I’m as guilty as everyone else. Our family professes to keep screens somewhat at bay. We even have occasional “phone-free” holidays - but its depressing how unusual and exotic even that is now considered. Its not original to point out that Apple and their ilk daily atomize us all even while they pretend to connect us.

It would be nice to think that the great handing-out of the goggles in the coming few years will prove to be the line too far in the sand - the moment the tech takeover of our intimate lives became too crass and unbearable. Maybe a “goggle-free” movement will arise - inspiring the masses to cast off not just the glasses themselves but all of the distorting lenses and false visions the tech industry has pasted over our perception. But despite a generalized angst about screens and big tech, I don’t see much evidence of that happening yet - not while our addicted fingers are still double tapping and swiping away at lit up metal and glass.

Like others, I’ve been watching the advance of the techno-goggles since  Google Glass flopped a decade ago (and we all got to taunt nerd-wearers as ‘glassholes’). I thought I kind of knew what was coming when I heard of Apple Vision Pro and didn’t pay the launch much mind. When belatedly I did pay attention though I realized there was one set of details in this device that I hadn’t previously grasped and which  has set all my alarms going: namely facial recognition.

As with all digital devices the new techno-goggles are sold as a way for you the user to better see into and manipulate digital worlds. However, just like our smartphones,  they are of course a two-way window - enabling  two-way manipulation too. That is,  they also let digital giants look deeper into your soul to find ways to capture it and nudge it.  Specifically the new Apple goggles come equipped with multiple cameras and sensors that not only continually scan your eyes in order to allow your eye movements to control cursors on the ‘screen’ but also scan your face in order to record the twitching of your muscles, the shape of your smile, the raising of your heart rate, the furrowing of your brow, the biting of your lip. Apple is quite open about this intimate personal surveillance . It even promotes two features that take advantage of the ongoing surveillance of your face and of the ocular gateways to your soul: On the front of the Vision Pro it projects a digital real time rendering of your eyes (called ‘Eyesight’) so that others can interact with you in a more natural way. It also maps and then reproduces all the small movements of your face to create a digital avatar of you that interacts in real time with other users in ‘FaceTime’ calls. So they see you as if they were viewing your face goggle-free.

But of course Apple isn’t just capturing your face and eyes to let others see you better - it also intends to see you more deeply too. Back in 2021 the Wall Street journal revealed that Apple was using iPhones for facial recognition to detect mood and emotion (particularly depression)  - leading a field known as ‘Emotion AI’ or ‘Affective Computing’.  Consider the incredible volume of  information you give away all day long  about your emotions (fears, joys, desires, uncertainties, triggers and more) through the medium of your face and your eyes. When we watch another person closely for any length of time we start to learn what excites or surprises them by the growing of their pupils or narrowing of their eyes. We learn what they might be trying to hide by the quivering of a lip. We see what is a real smile and what is a fake, forced smile. I am someone with a particularly expressive face who is not very good at hiding my emotional responses but with powerful AI to assist them, even the most stoic or talented actors when goggle-wearing will likely give away continual cues to their inner state that can be captured, decoded, catalogued and  capitalized on by the worlds wealthiest tech company.

Don’t just take my word for it. Follow Sterling Crispin, Neurotechnology Prototyping Researcher at Apple who claims he was instrumental in the development of the Apple Vision Pro. He claims that his part of the project was to enable “mind reading” via AI (that is ‘Emotion AI’) : “Generally as a whole, a lot of the work I did involved detecting the mental state of users based on data from their body and brain when they were in immersive experiences,” an article reports him saying going on to explain how these states would be measured by eye movements, electrical activity in the brain, heartbeat and rhythm, muscle activity, blood density in the brain, blood pressure, skin conductance, and more. Using this data, “AI models are trying to predict if you are feeling curious, mind wandering, scared, paying attention, remembering a past experience, or some other cognitive state,”

So if you watch a movie or some other piece of entertainment on the Apple Vision Pro, Apple may be able to scrutinize your every emotional micro-reaction every microsecond of that movie. After a few movies, if the Emotion AI researchers are right,  Apple may know better than you do exactly which triggers, colours, movements, music make you swoon, cry, laugh or feel angry. That sort of emotional metadata would be  gold to an advertiser who wants to develop targeted content to nudge the right mark to eat a processed food product, to download their app or to vote for their candidate. Of course it may not be so straightforward. Emotion AI as a field is hyped to be a multi-billion dollar field already but has its critics who point out how bad algorithms are at really understanding emotions : “Efforts to simply ‘read out’ people’s internal states from an analysis of their facial movements alone, without considering various aspects of context, are at best incomplete and at worst entirely lack validity, no matter how sophisticated the computational algorithms,” stated one 2019 paper on the topic.  But that doesn’t stop companies such as Apple or Amazon digging deeper into Emotion AI to gather incredible amounts of intimate data.

Luckily, (maybe???), for the last few years Apple has sold itself to consumers as a leader in offering privacy for such data - promising to not read our emails, share our uploaded photos or record our conversations - nor to sell our data to third parties - except when gathered through third party apps. But what about this new firehose of  rich personal metadata about to be streaming off of tech goggles? Are they going to resist commercializing it? Just by tracking my eyes and the twitching of my face muscles while I watch a movie, Apple could have me marked as a big softy who gets teary when he sees animals hurt or they may figure that my friend is a tough as nails masochist who grins at situations where ‘woke’ people look stupid. Someone is going to want to sell those particular “emotional profiles” to vegan alt-protein brands  or to right wing think tanks who would love to access our attention to  further nudge us towards their commercial or political  interests. If Apple somehow resists selling our emotion-profiles, its unlikely that all of Meta, Microsoft, Google, Amazon or a shady army of intermediate data traders will be so restrained. Remember how Cambridge Analytica used metadata from Facebook to  create personal profiles of millions of people to help elect Trump, secure Brexit and prop up  other dictators? Imagine if Cambridge Analytica had enjoyed an always-on pipeline scanning millions of faces?

And it’s not just our face and our souls that Apple is gaining intimate access to. It turns out that our living spaces too will be mapped. If I understand correctly Apple’s Vision Pro apparently utilizes a 3D camera and software that digitally  maps the space in which it operates so that it can properly project images in mid air and with correct sound. An article in UX design points out “the potential misuse of 3D mapping technology in a policing society is alarming. Detailed maps of private spaces could become tools of surveillance, used by authorities under the guise of public safety. But is safety worth the cost of our privacy?”  Such maps may also be excellent cues for nudging and manipulating users . What can an AI understand about us by mapping and watching our homes, workplaces or other spaces. Can those maps be mashed together to give metadata about trends that will sold to advertisers (about interior design? energy use? etc?).

The same report in UX Design also points out that the close intimate surveillance data streaming off of headsets like the Apple Vision Pro will be used to create synthetic data - such as deepfake videos - which imitate and steal from our lives to profit digital companies:

“By providing a detailed map of your face, you could inadvertently be contributing to the creation of synthetic data that could be used to impersonate you or violate your privacy in other ways. But who owns the digital representation of your face? And what happens if this data falls into the wrong hands? Do you have any rights regarding your face data and does anyone protect that?”

Probably this is just the tip of the iceberg. The key point here  is we shouldn’t look at the approaching goggles as something the eager wearer puts on to look into and access the digital world. In the coming years we will increasingly find they are something the tech industry aggressively affixes to our faces to look into us and to access us.