Humanity Unleashed is developing a novel method of sociotechnical cooperative function: an artificial intelligence system capable of processing input from human groups of any size, enabling transparent, fair, and effective governance and compensation based on individual actions and the group’s values.

This system, called Social Environment Design, or SED, merges policy design with reinforcement learning and computational social choice to allow for preference aggregation and counterfactual testing at any scale. SED allows organizations and individuals to contribute to public goods with a high surety of future reward and rewards earlier, more committed, and more impactful contributors proportionate to the value of their contributions in a transparent and inclusive process. Humanity Unleashed is incorporating the SED framework into its operation and is actively supporting research on the extension of SED to enable AI-assisted governance and public goods incentivisation at scale.

3.1 Prevents an AI Arms Race

The SED framework is well-suited to most stakeholder management problems, including international negotiations, owing to its ability to elicit inputs from participants, rapidly iterate through simulated environments, and learn from human feedback throughout the process. This makes it a valuable tool for assisting complex negotiations such as those that would be needed to coordinate international agreements on AI development, as well as other types of negotiations and joint action problems that are proving intractable to current systems.

Critically, SED does not require AI capacities that are at the forefront of anticipated future development, such as future versions of “foundation” AI models like OpenAI’s GPT 4 and Anthropic’s Claude 3.5.   Only modest advancement of these models is needed in order for SED to deliver significant value, and that value will improve considerably as these capacities are refined and expanded in provably safe ways narrowly applicable to the multi-agent optimization problems SED is designed to address.

Thus, SED will not increase the risk of an AI capacity arms race or of the emergence of unaligned ASI and will likely reduce these and other human coordination risks.

3.2 Aligns to Humanity While Bootstrapping Itself

SED is designed to incentivize its own construction, as the existence of a system with its features is necessary for all other public goods to be valued and incentivized. Once the SED system is operational, it will begin to value all socially and environmentally beneficial contributions, and, based on the values decided upon by the community, begin incentivizing action toward those as well.

Along the way, the AI engine of SED aligns to human interests through its feedback process. A Voting Mechanism ensures we all get a say in how it operates and allows for radically new ways for humans to cooperate with one another and decide on collective courses of action. With such an AI to assist, these actions can be simulated and confirmed to be effective, widely supported, and conducive to the flourishing of humans and of all life before action is taken in the real world.

3.3 Protects Humanity’s Highest Values

SED is designed such that it cannot be co-opted by its creators or by a malicious majority at any point in its development. To this end, it has several core values seeded within it. Functionally, systems that hold opposing values are competitors, and systems with the same values are alternative implementations that may be more or less capable. Such value-aligned alternative systems (we hope for many to emerge, but we are not currently aware of any) should ultimately subsume or be subsumed by SED based upon the specific features, efficacy and traction they attain.

In recognition of the diverse nature of humanity, we have made the core values of SED as open-ended as possible while ensuring that we minimize the most egregious potentials for tyranny of the majority within the system. Our guiding principle is selfless love and care for one another, and we operationalize this as an expansion on the United Nations International Bill of Human Rights and Sustainable Development Goals. We inherit all rights and goals from these documents as a starting point for valuing actions within the system. These values can be updated over time through the SED voting mechanism to maintain their relevance. However, the following core values are immutable:

• Equality and Cohesion - All lives have equal and intrinsic value. Being human must matter more than any specific attributes of that human, such as race, class, gender, nationality, etc. A baseline level of support must be offered to all humans. (See UDHR Articles 1, 2, 22, 23, and 25)

  
  

• Merit - People who contribute to the welfare of others must be rewarded beyond this baseline proportionate to their contributions as imputed by the values of the group, however marginal reward must decrease as total reward grows. (See SDG 10)

  
  

• Sustainability - The world we construct toward must be capable of being sustained over time. The human population must operate in balance with our natural systems, and these systems must be protected in perpetuity for the benefit of all. (See UN Resolution A/76/L.75)

The world is still very far off from these rights being secured for all humanity. The goal of SED is to bridge this gap.