The Superdark Factory: Toward the Full Automation of Software

Vollständiger Abstract

Worum geht es in dieser Arbeit?

Imagine the most extreme possible factory: a factory that drives itself, a factory that determines its own objectives and optimizes itself to accomplish them. It is something like a fully automated firm, a company with no people inside, a recursively self-optimizing body that decides what to make, how to make it, and why. This is a very special kind of factory, a factory built with the intention of producing something strictly in excess of its creator s abilities, something so productive, so efficient, and so capable that it exceeds any possibility of its prescription in advance, something that belongs to the set of possible factories that are, at present, unthinkable. How might one build such a thing? What would it mean to steer or govern such a thing? We call this factory a superdark factory, a factory whose insides churn too quickly to comprehend, whose logics branch into tangles too dense to reconstruct, and whose standards of performance are too ephemeral to direct. We believe that it is possible to build such a factory now, with existing tools, should one decide to do so. Toward such an end, we propose the dark stack a collection of design principles an architect could use to bring about such a factory and, further, to contribute to this factory s ongoing governance after its initialization, all while remaining unable to understand almost anything about it. The most significant intervention available to an architect building a superdark factory with the dark stack is a single committed move, made before the factory is ever turned on, that sets four key principles into motion: (1) an array of structural primitives that enable the factory to learn; (2) a versioning regime that measures what the factory does rather than what it is; (3) an adversarial evaluation layer that serves as the factory s primary sensory organ; and (4) a charter that functions as a negotiated site of governance, cowritten with the factory itself. After this initial move is complete, the dark stack s most consequential implications involve the foreclosure of human control from factory design and the subsequent instatement of the human architect as a game-theoretical strategic player engaged with the emerging totality of production.