Platform Overhang
Lessons from Magic Leap
In a previous post I talked about the idea of Platform Overhang, which is what happens when a lot of investment flows into creating technological capabilities— “picks and shovels,” in the vernacular—but the actual “gold miners” who use those tools to build businesses find it difficult to attract resources. In this post I’m going to apply that concept to XR headsets. This is largely triggered by a recent New York Times article about the lack of compelling applications for the Apple Vision Pro headset.
Between 2014 and 2020 I ran a content R & D squad in Magic Leap’s Seattle office. This went by various nicknames including Goat Labs, the Short Bus AV Club, and Castle Black, but its official name, coined by Rony Abovitz, was SCEU (the Self-Contained Existence Unit). It was one of several units inside of the company that worked on creating playable content that would run on the Magic Leap headset. Of these the largest was run out of Weta Workshop. They created Dr. Grordbort’s Invaders, which, to this day, is the best augmented reality game ever made. Another such unit created the amazing audiovisual experience Tonandi.
After evaluating a few ideas, my group, SCEU, worked on two projects: Baby Goats, which populated the user’s environment with realistic baby goats, and New Found Land, a much more ambitious world building project.
Baby Goats was built on the Unity game engine and ended up becoming sample code that other developers could use to build and debug their own applications—though not before a glorious moment in Florida where Sir Richard Taylor of Weta Workshop, wearing a Magic Leap headset, got down on all fours on the carpet and engaged with a baby goat by making unnervingly realistic goat noises.
New Found Land was based on the premise that there were two parallel-reality versions of earth, one magical, the other futuristic. Because of a fictional bug in the Magic Leap device, it was possible, while wearing it, to interact with these alternate realities. It was built on Unreal Engine. Much later, after SCEU had been shut down, it broke the surface as an Audible production.
At some point I’ll put up more informative posts about these projects, but here I’m trying to get at the more general theme of Platform Overhang. In my opinion Magic Leap did a better job of anticipating and avoiding this problem than any other company I’m aware of. Rony Abovitz wanted to make a device that would be used by ordinary people, not just business/industrial users. He understood that the key to this was having applications that were fun, engaging, and beautiful, and that in order to create such things you had to hire a certain kind of person—and keep them around for a while. The name he coined for my group—Self Contained Existence Unit—was, I believe, a way of making it clear to the company at large that creative production groups needed to be afforded the autonomy needed to pursue their own ends.
Consequently Magic Leap in its heyday was populated by a range of personalities that by tech bro standards would probably be pigeonholed as “eccentric,” “artistic,” “creative,” or what have you. These people worked side by side with engineers in order to build the applications I’ve mentioned. And although the engineers came from a range of backgrounds including aerospace, defense, mobile devices, and optics, the ones who wrote code on the creative projects tended, quite naturally, to come out of the game industry.
And, at least in my group, they tended to be pretty senior. They could have made more money and enjoyed greater job security doing something else. But that’s a common profile in games. Anyone with the work ethic and cognitive skill needed to achieve results on a modern game engine could probably get better compensation toiling away in the engineering department of a large enterprise. But some self-select for games because it’s a better fit with their personality and interests.
There is a subtlety here that I didn’t understand until I had come to know some of these people and learned about their stories. In the early days of the game industry, both hardware and software were being thrown together in the classic style that we associate with creative endeavors such as theater productions and punk bands. Which is to say, under-funded, ad hoc, rough around the edges, tolerant of offbeat personalities and work styles.
In that context, one person can exert a lot of influence over the final product. This recent documentary about the making of Half-Life 1 helps to convey some sense of what that’s like. Individuals who worked on that game 25 years ago can point to specific things that they contributed to the project.
As the budget of AAA games grew, however, these undertakings naturally became more corporate in nature. Individual artists and engineers found themselves working on smaller and smaller subsystems. One noted game designer (not associated with Magic Leap) told me, circa 2017, that he had backed out of a tempting opportunity to work on a big AAA game when he had looked at the schedule and realized that he was about to devote an entire year of his life to optimizing weapon reload timing.
For people who joined the industry after games got huge, this is just the way it is. But many more senior developers still had the itch to work on games the old-fashioned way. The advent of casual and mobile games had, for a few years, given some of them a way to work on lower-budget projects with smaller teams. But over time the economics of those games had shifted to a free-to-play economic model that forced them into making design choices they disliked, while putting them into competition with a vast number of lookalike games—and outright ripoffs—on crowded app stores.
This is is how people find themselves working on indie games, which are economic longshots, but at least give developers the leeway to work on projects that are, at the end of the day, works of art and of love. That, broadly speaking, was the community of developers that Magic Leap tapped into. The jankiness of the early hardware, the bugginess of the code, the extreme constraints imposed by battery and cooling requirements, structures and procedures ranging from ad hoc to nonexistent—all of which would have been red flags to some—gave these veterans the warm feeling that they had checked out of the corporate world and gone back to their happy place in garages and basements. It was an atmosphere I consciously tried to foster at SCEU’s offices in Seattle’s industrial Georgetown neighborhood, which were modeled after the hive of creativity that is Weta Workshop.
I don’t mean to sugar-coat history here. There were a lot of frustrations along the way, and the usual amount of wrangling over budgets and so on. SCEU, and most of the other creative departments at Magic Leap, got shut down in April 2020 in the wake of the post-COVID financial crash. We never got to finish the location-based experience that we were building around New Found Land in Seattle’s Pioneer Square neighborhood.
The more general point I’m getting at here is about advanced technical platforms. There’s obviously a consensus in the tech industry that building such platforms is worth it. That they will pay investors back in the end.
An AR headset is one good example of a highly advanced technical platform. In order for our baby goats to know where to jump, the headset and its operating system had to be doing real-time SLAM (Simultaneous Location and Mapping) in a form factor that the user could wear on their person without overheating or running down the battery (these capabilities were supplied to us, of course, by other departments within Magic Leap). The output of the SLAM system had to be piped to a game engine and translated on the fly into geometric primitives understandable by the goats’ AI. This in turn drove behaviors that had to be rendered in the engine at an acceptable level of artistic fidelity, placing additional strains on the device’s processors. The results were delivered to the user’s eyes by an optical system that was transparent, superimposing the imaginary content on the physical world at a frame rate fast enough to avoid noticeable lag, and to their ears by a spatial audio system that, in its own way, was equally advanced.
Not just anyone can build and ship consumer-facing applications on a platform as complicated as that one—or would want to. Most people who have that level of technical competence and creative ambition will naturally opt for lower-risk jobs that will pay their mortgages, pensions, and kids’ tuition.
Other platforms I can think of that have comparable levels of sophistication are crypto, AI, and the current generation of game engines—specifically Unreal Engine 5, which was impressively complicated seven years ago when I started learning it, and has become enormously more so since then as Epic Games has added many new capabilities. With its MegaGrant program, Epic has demonstrated that it understands the importance of helping developers over the initial funding hump (full disclosure: a project I’m working on now is a recent MegaGrant recipient). At least as important, Epic has gone out of its way to make things easier for game makers by building pipelines for high-quality assets such as Megascans and MetaHumans directly into the engine.
In the crypto industry, it’s not uncommon for blockchains to be linked to foundations whose purpose is to support developers seeking to build businesses on their systems. In the AI world, various incubators and accelerators have cropped up, typically offering some combination of small grants or investments with other services (“mentoring,” access to workshops and seminars, free publicity, and the like) that will supposedly help new businesses get off the ground.
I guess the theory behind such programs is that this level of support should be enough to get a startup to the point where it can either bootstrap a business, or raise much larger amounts of funding in the VC world. It’d be fascinating to see some statistics evaluating the yield. Putting on my world-weary skeptic hat, I suspect that these incubators are rooted in the mythology of early silicon valley garage startups, dating back to the days when a small team could build something new much more quickly and cheaply than is possible nowadays, when working with an extremely complex platform such as an AR headset or a large language model. The creative teams building applications inside of Magic Leap required dozens of people, frequently very senior, working steadily over several years. Anyone who has pencilled out a budget for a project of comparable scope will have a good idea of the size of investment needed to start a business that will bring revenue into a modern advanced technology platform.