As a follow up to last week's article about the Biden executive order on artificial intelligence (AI), this is a brief overview of one of its provisions that has proven to be controversial -- namely, the additional scrutiny that the government plans for AI models and computing clusters over a certain size. The order states that the Secretary of Commerce shall require that entities acquiring, developing, training, or using such models must report their activities to the government, including results of red-team testing intended to discover flaws and vulnerabilities of these systems.
The current size thresholds are:
(i) any model that was trained using a quantity of computing power greater than 1026 integer or floating-point operations, or using primarily biological sequence data and using a quantity of computing power greater than 1023 integer or floating-point operations; and
(ii) any computing cluster that has a set of machines physically co-located in a single datacenter, transitively connected by data center networking of over 100 Gbit/s, and having a theoretical maximum computing capacity of 1020 integer or floating-point operations per second for training AI.
For sake of comparison, today's high-end PCs typically can maintain a few trillion floating-point operations per second (teraflops), which is around 1012 floating-point operations. The NVIDIA DGX H100 server, which is specifically designed for supporting AI uses, is rated at 32 petaflops, which is around 1016 floating-point operations. Thus, to reach the 1020 floating-point operation threshold, one would need about 10000 DGX H100s. At a price of about $400,000 each, it would require exceptionally deep pockets to even consider building such a system, and this does not even take into account the cost of electrical power.[1]
According to some reports, OpenAI's GPT-3 was trained on a cluster that has about 800 petaflops of computing power (just under 1018 floating-point operations per second).[2] Hitting the regulatory threshold would require a system with about 100 times more capacity. OpenAI did not release the technical details of its training cluster for GPT-4 but did indicate that the training cost the company over $100 million.[3]
It is safe to say that only a handful of non-governmental entities currently have the ability to train models that would trigger the reporting required by the order. But, if anything, we have learned over the last several decades that the amount of computing power available per dollar increases by an order of magnitude every few years. Thus, the 1020 floating-point operation threshold may be within reach of many entities in the near future.
Despite the call for regulation coming from OpenAI and other leaders in generative AI, there has been some pushback. So far, this is largely originating with the open source community and venture capitalists.
Researcher Andrew Ng is concerned that regulations based on model size are an overreaction to the dangers of AI, would stifle innovation, and would advantage the incumbent players.[4] Researcher Yann LeCun, with a number of other tech executives and venture capitalists, sent a letter to President Biden arguing that the order would result in AI being dominated by a few well-heeled companies.[5] The letter also urges the president to consider the benefits that open source AI models can bring to the public by increasing marketplace competition while opening development and use of the technology to academia and individuals.
The argument for taking a lighter regulatory touch on open source AI models is that they foster collaboration, transparency, and security by democratizing the use of AI. An open source mantra is that "given enough eyeballs, all bugs are shallow." This can be interpreted to mean that the world's programming community being able to inspect these models would increase their reliability and usefulness while reducing harm. Yet, by their very nature, open source initiatives are unlikely to have the resources to engage with regulators. Furthermore, the concern over the most powerful AI in the world being black boxes understood and controlled by only a handful of people is very real.
While open source has proven to be an incredibly powerful tool for advancing technology and bringing it to the masses -- note the rise of Linux over the last 30 years -- a blind adherence to its principles ignores some obvious risks. Bad actors of all stripes could obtain and customize generative AI models at a low cost, potentially removing any guardrails and using them to produce an unlimited amount of misinformation, deepfakes, and cyberattacks, or even to design novel bioweapons. There is a big difference between open sourcing an operating system or web server versus open sourcing a large language model with even a fraction of GPT-4's capabilities.
At this juncture, we appear to be between the devil and the deep blue sea. The question comes down to who can be trusted more -- a handful of wealthy tech firms, the governments of the world, or the public.
The order draws some lines in the sand, but is still a thoughtful response to an emerging technology that has the potential to be simultaneously useful and dangerous. And unlike many regulatory efforts, the Biden administration is not waiting for a disaster to happen before putting a framework into place. One thing that we do know about the generative AI space is that size matters -- scaling up models can produce emergent capabilities. Given this, focusing initial scrutiny on models above a certain size is not a bad idea.
[1] This is just a back of the envelope calculation based on currently available information. The costs of the NVIDIA and other hardware is currently in flux due to market demand.
[2] https://www.linkedin.com/pulse/mind-boggling-processing-power-cost-behind-chat-gpt-what-thakur/
[3] https://www.wired.com/story/openai-ceo-sam-altman-the-age-of-giant-ai-models-is-already-over/
[4] https://www.cnbc.com/2023/11/02/biden-ai-executive-order-industry-civil-rights-labor-groups-react.html
[5] https://twitter.com/ylecun/status/1720547401075675613
[View source.]
