In its yearly report of the global economy’s most compelling facts and figures, the Atlantic Council includes specs on Generative AI among its 26 highlights, as described by Giulia Fanti, assistant professor of electrical and computer engineering at Carnegie Mellon University, a nonresident senior fellow at the Council’s GeoEconomics Center, and also a Principal Investigator on cybersecurity for the C3.ai Digital Transformation Institute.

Fanti brings to light the staggering parameters in state-of-the-art Large Language Models (LLMs), followed by her explanation.

This year, generative artificial intelligence (AI) captured the public’s imagination with its ability to generate photorealistic images, videos, audio, and text. Many believe that models such as GPT-4, PaLM 2, Llama 2, and Mistral will revolutionize how humans interact with computers for government services, education, and enterprise settings, to name a few. However, the amazing capabilities of generative models come at a cost.

Today, the leading models are growing quickly in size (as measured by their number of parameters, the values that control LLMs’ behavior). This matters because larger models are more expensive to train and more expensive to use once trained. For example, the Llama 2 (70B) model has 70 billion parameters and required a staggering 1.7 million graphics processing unit (GPU) hours, or the equivalent of almost two hundred years, to train. (This was sped up in practice by using these resources in parallel.)

The geoeconomic implications of these trends are likely to become more severe in the coming years. To train or host these models, organizations will need access to data centers with many GPUs. Moreover, due to data use and data locality restrictions in many regions, such data centers may need to be local. However, data centers are distributed inequitably across the world, with the vast majority of data centers located in the United States and Europe. This is likely to lead to a massive disparity in the ability to train, use, and benefit from generative AI.

Read the full story here.

Atlantic Council image: Mark Schiefelbein/Pool via REUTERS