The rise and importance of Secret Congress


Lessons From the World’s Two Experiments in AI Governance

Artificial intelligence (AI) is both omnipresent and conceptually slippery, making it notoriously hard to regulate. Fortunately for the rest of the world, two major experiments in the design of AI governance are currently playing out in Europe and China. The European Union (EU) is racing to pass its draft Artificial Intelligence Act, a sweeping piece of legislation intended to govern nearly all uses of AI. Meanwhile, China is rolling out a series of regulations targeting specific types of algorithms and AI capabilities. For the [host of countries](https://oecd.ai/en/dashboards/policy-instruments/Emerging_technology_regulation) starting their own AI governance initiatives, learning from the successes and failures of these two initial efforts to govern AI will be crucial.

When policymakers sit down to develop a serious legislative response to AI, the first fundamental question they face is whether to take a more “horizontal” or “vertical” approach. In a horizontal approach, regulators create one comprehensive regulation that covers the many impacts AI can have. In a vertical strategy, policymakers take a bespoke approach, creating different regulations to target different applications or types of AI.

Neither the EU nor China is taking a purely horizontal or vertical approach to governing AI. But the EU’s AI Act leans horizontal and China’s algorithm regulations incline vertically. By digging into these two experiments in AI governance, policymakers can begin to draw out lessons for their own regulatory approaches.

### **The EU’s Approach**

The EU’s approach to AI governance centers on one central piece of legislation. At its core, the AI Act groups AI applications into four risk categories, each of which is governed by a predefined set of regulatory tools. Applications deemed to pose an “unacceptable risk” (such as social scoring and certain types of biometrics) are banned. “High risk” applications that pose a threat to safety or fundamental rights (think law enforcement or hiring procedures) are subject to certain pre- and post-market requirements. Applications seen as “limited risk” (emotion detection and chatbots, for instance) face only transparency requirements. The majority of AI uses are classified as “minimal risk” and subject only to voluntary measures.

The AI Act vaguely defines “essential requirements” for each risk tier, placing different constraints on each category. The easiest way for developers to satisfy these mandates will be for them to adhere to technical standards that are being formulated by European standards-setting bodies. This makes technical standards a key piece of the AI Act: they are where the general provisions described in legislation are translated into precise requirements for AI systems. Once in force, years of work by courts, national regulators, and the technical standards bodies will clarify precisely how the AI Act will apply in different contexts.

In effect, the AI Act uses a single piece of horizontal legislation to fix the broad scope of what applications of AI are to be regulated, while allowing domain- and context-aware bodies like courts, standards bodies, and developers to determine exact parameters and compliance strategies. Furthering its ability to act in more context-specific ways, the EU is also pairing the requirements in the AI Act with co-regulatory strategies such as regulatory sandboxes, an [updated liability policy](https://ec.europa.eu/commission/presscorner/detail/en/ip_22_5807) to deal with the challenges of AI, and associated legislation focused on [data](https://ec.europa.eu/commission/presscorner/detail/en/ip_22_1113), [market structures](https://ec.europa.eu/info/strategy/priorities-2019-2024/europe-fit-digital-age/digital-markets-act-ensuring-fair-and-open-digital-markets_en), and [online platforms](https://digital-strategy.ec.europa.eu/en/policies/digital-services-act-package).

This framework strikes a balance between the dual imperatives of providing predictability and keeping pace with AI developments. Its risk-based approach allows regulators to slot new application areas into existing risk categories as AI’s uses evolve, providing a balance between flexibility and regulatory certainty. Meanwhile, the AI Act’s definition of relatively flexible essential requirements also alleviates the key precision challenge posed by purely horizontal frameworks, allowing compliance strategies to be flexible across sectors and as technology evolves.

But the EU’s broadly horizontal approach faces several risks that other countries should watch closely. Individual regulators tasked with enforcing requirements might differ in their interpretations or capacity to regulate, undermining the key capacity and harmonization benefits of a horizontal approach. Another factor is whether the proposed central and horizontal European AI Office will be effective in supplementing the capacity of national and sectoral regulators. Delegating the creation of precise technical requirements to expert standard-setting bodies allows more technical expertise and precision to be channeled into precise requirements. However, the standards process has historically been driven by industry, and it will be a challenge to ensure governments and the public have a meaningful seat at the table.

### **China’s Approach**

Over the past year, China has rolled out some of the world’s first nationally binding regulations targeting algorithms and AI. It has taken a fundamentally vertical approach: picking specific algorithm applications and writing regulations that address their deployment in certain contexts. The first two regulations in this camp targeted recommendation algorithms and deep synthesis technology, also known as generative AI.

The recommendation algorithm regulation focused on its use in disseminating information, as well as setting prices and dispatching workers. It required that algorithm providers “vigorously disseminate positive energy” and avoid price discrimination or [overly demanding workloads](https://carnegieendowment.org/2022/11/02/how-food-delivery-workers-shaped-chinese-algorithm-regulations-pub-88310) for delivery drivers. The second regulation targeted deep synthesis algorithms that use training data to generate new content, such as deepfakes. The regulation again focused on concerns around “harmful information,” but it also required providers to obtain consent from individuals if their images or voices are manipulated by the technology. These application-specific requirements stem from the vertical nature of the regulations. Neither of these requirements would make sense when applied to the other set of applications or to ones that would be covered by a horizontal regulation.

Yet China’s regulations do contain a horizontal element: they create certain regulatory tools that can be applied horizontally across several different vertical regulations. A prime example of this is the algorithm registry (算法备案系统, literally “algorithm filing system”). The registry was created by the recommendation algorithm regulation and reaffirmed by the deep synthesis regulation, both of which require developers to register their algorithms. It acts as a central database for Chinese officials to [gather information on algorithms](https://carnegieendowment.org/2022/11/02/how-food-delivery-workers-shaped-chinese-algorithm-regulations-pub-88310), such as their sources for training data and potential security risks. As such, the registry also serves as a vehicle for regulators to learn more about how AI is being built and deployed—a key goal of sectoral regulators around the globe.

Looking ahead, the registry may continue to act as a horizontal tool to gather similar information on algorithms that fall under a variety of vertical regulations. It could be adapted to require different kinds of information depending on the algorithm’s application, or it could simply provide a uniform baseline of information required on all algorithms subject to regulation.

China’s approach allows it to more precisely target regulatory requirements to specific technical capabilities. In many cases, this approach might risk rules falling behind quickly evolving technology. In China’s AI regulation, however, some requirements are defined so vaguely that they effectively function to shift power from technology companies to government regulators, which can wield their newfound regulatory power to force any changes on companies that they wish. China’s vertical regulations could also end up as building blocks for a more comprehensive AI governance regime, a pattern that [played out](https://www.researchgate.net/publication/324137548_Historicizing_Internet_Regulation_in_China_A_Meta-Analysis_of_Chinese_Internet_Policies_1994-2017) in Chinese internet governance prior to the country’s Cybersecurity Law. But until that time, the main risk of such an approach is the creation of a patchwork of regulations that are, collectively, poorly considered and expensive to comply with.

### **Lessons**

The EU’s and China’s approaches both have their critics. Business groups argue that the EU’s broad approach will stifle innovation, and analysts correctly assert that China’s targeted regulations will be used to tighten information controls. But by taking a step back to look at their fundamental approaches to regulating AI, policymakers can draw lessons for countries and institutions around the world.

One central lesson is that neither approach can stand fully on its own. A purely horizontal regulatory approach will not be able to set out meaningfully specific requirements for all applications of AI. Conversely, creating a grab bag of freestanding vertical regulations targeting each new application of AI will likely create a compliance mess for regulators and companies alike.

The most effective approaches incorporate both horizontal and vertical elements. Regulators taking a horizontal approach benefit from deferring the nitty-gritty work of creating specific compliance requirements to more vertically oriented organizations like sectoral regulators, standards bodies, or in some cases courts. Likewise, governments taking a vertical approach can create horizontal regulatory tools and resources that can be used across a variety of application-specific laws, reducing the burden on regulators and the unpredictability for businesses.

We can see versions of this in both the EU and Chinese approaches. While the EU AI Act creates four risk tiers and lays out broad requirements for each one, the [heavy lifting](https://www.lawfareblog.com/eus-ai-act-barreling-toward-ai-standards-do-not-exist) of articulating specific compliance thresholds will be done by Europe’s main standardization bodies. And while China has issued vertical regulations targeted at recommendation engines and generative AI, those regulations lean on horizontal tools like the algorithm registry in order to bring coherence across them.

Beyond this overarching need for supplementing both approaches, horizontal and vertical regulatory regimes each present different pros and cons. The choice of which regulatory regime will work best often comes down to the structure and culture of each government. How nimble or gridlocked are a country’s legislative bodies? How empowered and coordinated are its sectoral regulators? The answers to these questions inform which approach will work best.

Horizontal approaches provide predictability to developers and businesses by laying out a fixed set of governance tools. Despite the diversity of AI applications, the risks they pose often involve similar themes of transparency, robustness, and accountability, and horizontal strategies can help governments concentrate limited resources on these repeating themes. Horizontal approaches can also reduce the chance of regulatory gaps emerging when overtaxed sector-specific regulators lack the capacity to consider new technologies such as AI.

But governments will need a few characteristics in order to reap the benefits of horizontal approaches. Their legislative bodies will need the ability to amend or add to its main horizontal regulation in order to keep pace with the technology. A static horizontal regulation for a fast-evolving technology is unlikely to hold up. In addition, when a horizontal regulation delegates the setting of precise compliance requirements to other institutions, those institutions need to be able to resist industry capture. For example, if industry-dominated standards bodies set weak standards for compliance, the regulation itself becomes weak.

Vertical AI regulations have the benefit of being tailored to mitigate the specific harms posed by different AI applications. Though more limited in their ultimate ambition, these individual regulations can tackle existing harms without the added burden of being generalizable to all applications of the technology. As regulators and legislators learn which interventions prove successful, they can incorporate those tools or requirements into future legislation targeting other harms. And if this piecemeal approach ultimately proves insufficient, they can bring these lessons to bear on shaping a more horizontal regulatory regime.

But this vertical approach requires a certain level of legislative and interagency coordination to minimize costs for both regulators and businesses. If agencies do not coordinate to build common regulatory tools, they risk reinventing the wheel each time a new department is tasked with regulating a specific application of AI. Similarly, if vertical regulations don’t come paired with the necessary technical and financial resources, sectoral regulators will be outmatched in their efforts to meaningfully constrain businesses applying AI.

The United States’ political structure has produced a motley set of forays into AI regulation. These efforts have brought about a unique blend of horizontal and vertical elements. The past two presidential administrations have laid out their own politically oriented guiding principles in the form of former president Donald Trump’s [guidance on AI regulation](https://www.whitehouse.gov/wp-content/uploads/2020/11/M-21-06.pdf) and President Joe Biden’s [Blueprint for an AI Bill of Rights](https://www.whitehouse.gov/ostp/ai-bill-of-rights/). These have served as horizontal guidance meant to inform vertical [sectoral](https://www.consumerfinance.gov/compliance/circulars/circular-2022-03-adverse-action-notification-requirements-in-connection-with-credit-decisions-based-on-complex-algorithms/) [regulators](https://www.ada.gov/resources/ai-guidance/) in the executive branch. Both horizontal, principle-based approaches attempt to harmonize high-level approaches to AI regulations while preserving flexibility to adapt to individual settings. However, if not coupled with resources and pressure to actually perform this sector-specific adaptation, this approach runs the risk of resulting in no meaningfully binding regulation at all.

In addition, regulation that relies on federal department rulemaking or rule interpretation runs the risk of being reversed during the next administration, [as happened with explanation requirements](https://www.paceanalyticsllc.com/post/cfpb-brewing-algorithmic-storm) for algorithms used for credit decisions between the Trump and Biden administrations. Meanwhile, bipartisan majorities in Congress have funded AI research efforts and best-practice guidelines such as the recent [AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework) (RMF) from the National Institute of Standards and Technology. The RMF is nonbinding by design, but it does bear similarities to the kinds of horizontal tools that could be applied to supporting future vertical regulations.

There’s no single formula for AI regulation that can be applied across countries, but blends of horizontal and vertical elements used in the EU’s and China’s AI regulations offer a menu of policy options. By adapting lessons learned from these initial approaches to their unique contexts, other countries can take meaningful steps toward reducing the harms produced by AI systems.


Transformative AI and Compute - Reading List

Transformative AI and Compute
=============================

Reading list

Updated every so often by [Lennart Heim](https://www.google.com/url?q=http://heim.xyz/about&sa=D&source=editors&ust=1692801441270235&usg=AOvVaw01oTwqySEmHDE9LVoplS08) ([lennart@heim.xyz](mailto:lennart@heim.xyz))

(with lots of help and summaries from Konstantin Pilz)

Last updated May 2023

By MidJourney

---

There could be significant progress in AI this century, leading to what has been described as [transformative AI](https://www.google.com/url?q=https://www.openphilanthropy.org/research/some-background-on-our-views-regarding-advanced-artificial-intelligence/&sa=D&source=editors&ust=1692801441271606&usg=AOvVaw1uNHJ7iNa7UU3rFg9Ega91) or even [Artificial General Intelligence](https://www.google.com/url?q=https://en.wikipedia.org/wiki/Artificial_general_intelligence&sa=D&source=editors&ust=1692801441271868&usg=AOvVaw36rX5kgYKUA31EK88zmZQ-) (AGI). This would involve substantial risks, as capable AI systems may be hard to control and could have devastating effects on society if used by malicious actors. Potentially, advanced forms of AI could constitute an [existential risk](https://www.google.com/url?q=https://futureoflife.org/existential-risk/existential-risk/&sa=D&source=editors&ust=1692801441272257&usg=AOvVaw31L7nZMYVCdzQ8TA6yeuZZ) to humanity. The topic, hence, deserves much more attention, both the technical aspects of how to create safe systems ([AI alignment](https://www.google.com/url?q=https://en.wikipedia.org/wiki/AI_alignment&sa=D&source=editors&ust=1692801441272574&usg=AOvVaw3qra7ais1qFHXWBWSVhGPZ)) as well as their governance ([AI governance](https://www.google.com/url?q=https://www.allandafoe.com/opportunity&sa=D&source=editors&ust=1692801441272790&usg=AOvVaw25WTRWnQfWMcHflKoZyVHp)).

Computational resources (compute) refers to the computational infrastructure required to run AI training and inference and is, therefore, a necessity for AI.

Due to its unique properties (quantifiability, rivalrousness, excludability) and state of affairs, such as a concentrated supply chain, compute may be one of the most promising nodes to steer the development of beneficial and safe AI.

I put readings into the following buckets:

1. [Compute in the AI Production Function](#h.22di12jwyi2m): Why compute matters for AI
* Compute is one of the key drivers of AI progress.
2. [Compute Supply Landscape](#h.2syusscpwpi9): How chips are produced and used
3. [Compute Governance](#h.t5ffwi6bvd65): How can we govern compute to achieve beneficial AI outcomes?
* Using compute as a governance node by either (a) monitoring, (b) restricting, or (c) promoting access.
4. [Compute-Based Transformative AI Forecasting](#h.7hhqfxns4gt4)
* How much compute we might need to achieve certain transformative and potentially dangerous capabilities
* How much compute we will have in the future and which computing paradigms will dominate
5. [Others](#h.xsewbkbnuqo5): Books, research questions, related topics, newsletters, podcasts, career advice, forecasting

This reading list is not meant as “read all of this in this order”, rather “here are some selected papers and articles on topics related to AI and compute”.

---

Overview
--------

Pieces that cover all of the domains below:

* Podcast: [Lennart Heim on the compute governance era and what has to come after](https://www.google.com/url?q=https://80000hours.org/podcast/episodes/lennart-heim-compute-governance/&sa=D&source=editors&ust=1692801441276306&usg=AOvVaw1GwLGXzX0l2CHi5Rdzz0l3), 80,000 Hours Podcast, 2023
* A broad overview over the motivation behind compute governance, what has recently happened and many other commonly asked questions on the topic.
* [Transformative AI and Compute [Summary]](https://www.google.com/url?q=https://www.lesswrong.com/s/bJi3hd8E8qjBeHz9Z/p/XJYdnHQqpengWn3xb&sa=D&source=editors&ust=1692801441276806&usg=AOvVaw2GfFbkMxO6u1v5crCf79rI) (11 min), Heim, 2021
* Next to algorithms and data, computational resources (compute) are a major input into AI advancement. With AI systems doubling in compute usage approximately every six months, scaling compute has arguably been a main driver of recent progress. Understanding this input can thus help us forecast future developments and quantify the progress and capabilities of AI systems. Because compute has unique properties and states of affairs, it may present a promising opportunity for the long-term governance of AI. This area is called compute governance.
* [Computational Power and the Social Impact of Artificial Intelligence](https://www.google.com/url?q=https://arxiv.org/abs/1803.08971&sa=D&source=editors&ust=1692801441277295&usg=AOvVaw1kxpGxDlMf4NzFgMHs9Eep), Hwang, 2018
* Some early work on the connection of compute and AI. Provides a good overview but is a bit outdated.
* Virtual Talk: [Introduction to Compute Governance](https://www.google.com/url?q=https://youtu.be/jJH8ApqyOOY&sa=D&source=editors&ust=1692801441277682&usg=AOvVaw2rbFD9vrhwrMnM0LAvcC7U), Heim, 2023
* In this talk, I present the idea of using computational resources (short compute) as a node for AI governance. First, I will start by talking about recent events in compute and AI and how they relate to compute governance. I will then discuss the unique properties and state of compute affairs that make it a particularly governable node for AI governance and how this relates to the compute supply chain and other concepts. Subsequently, we will explore the idea of hardware-enabled mechanisms and how they can be used for providing assurances and other AI governance goals. Lastly, I will present our policy work and close with a summary.
* Discussion with Lennart Heim (that’s me) on the AI Triad and Compute Governance on the Future of Life Podcast, 2023
* [Lennart Heim on the AI Triad: Compute, Data, and Algorithms](https://www.google.com/url?q=https://www.youtube.com/watch?v%3D7EwAdTqGgWM&sa=D&source=editors&ust=1692801441278334&usg=AOvVaw0XxsKEGMH27ZLpyzhbXDrk)
* Topics include: How we can forecast AI progress by researching AI hardware, The AI triad, Modern chip production, Forecasting AI with compute, Running out of data?, Three eras of AI training, Next chip paradigm, AI takeoff speeds
* [Lennart Heim on Compute Governance](https://www.google.com/url?q=https://www.youtube.com/watch?v%3DiCxJUDDvq94&sa=D&source=editors&ust=1692801441278770&usg=AOvVaw03ozhpPvSmRbNhX6fVgnGi)
* Options for governing the compute used by AI labs and potential problems with this approach to AI safety.
* Topics include: AI risk, Why focus on compute?, Monitoring compute, Restricting compute, Subsidising compute, Compute as a bottleneck, US and China, Unintended consequences, Will AI be like nuclear energy?

---

Table of Contents
-----------------

[Transformative AI and Compute](#h.kimhqj72mew4)

[Overview](#h.9trcejpb8x1n)

[Table of Contents](#h.22di12jwyi2m)

[1 Compute in the AI Production Function](#h.s5hq40c8pr92)

[1.1 Compute as an input to AI systems](#h.tgcys8tn942p)

[1.2 Compute usage trends in AI](#h.gzv3uu6061zn)

[1.3 Scaling laws](#h.gflipzqh65sy)

[1.4 Compute efficiency](#h.7aeqiuilwyrl)

[1.5 How to use compute efficiently](#h.3wdvox4mfvs2)

[2 Compute Supply Landscape](#h.2syusscpwpi9)

[2.1 Semiconductor supply chain](#h.9ecvgm78asj)

[2.2 High-Performance Computing (HPC) and AI](#h.185afhum7c48)

[2.3 Trends in Computing Hardware](#h.y1e7mj6k73ja)

[2.4 Where’s all the compute?](#h.1x1pbwavllod)

[2.4.1 China](#h.o6aeyc2ys270)

[2.5 What’s next in computing?](#h.e6noruurtq8a)

[3 Compute Governance](#h.t5ffwi6bvd65)

[3.1 Motivation](#h.wf421w5z1mnh)

[3.2 Existing policy work relevant to Compute Governance](#h.z7b7bc7twjl6)

[3.3 Promoting Access through Compute Funds](#h.no113fj7uphi)

[3.4 Measuring compute](#h.r83uhndpil4w)

[3.4.1 Monitoring compute](#h.vsh72z9wij2u)

[3.4.2 Verification](#h.36snplbynf3y)

[3.5 Export Restrictions](#h.6pkyhhw52t9d)

[3.5.1 US Export Restrictions from October 2022](#h.ref8pof5hkub)

[3.6 Verifying Compute Use](#h.xgb1st6hccai)

[3.7 Hardware Security](#h.dftvm228q1l1)

[3.8 Compute Providers](#h.jan0g0ta6enb)

[3.9 Data centers](#h.ieh7jwrze8v4)

[3.10 Corporate Compute Governance](#h.ljof0274a16m)

[4 Compute-Based Transformative AI Forecasting](#h.7hhqfxns4gt4)

[4.1 TAI forecasting](#h.nlviro7sij7e)

[4.2 Forecasting compute](#h.tr6sokx3h33x)

[4.2.1 Interpreting AI compute trends and the limits](#h.qcrmpx5j4e9)

[4.3 Takeoff scenarios and compute](#h.gsbwm4n9430r)

[4.4 Brain Comparisons and analogies](#h.21ejpooteh6i)

[5 Others](#h.xsewbkbnuqo5)

[5.1 Books](#h.ocp44viuiy24)

[5.2 Research Questions](#h.qulocncg05fo)

[5.3 AI chip architecture](#h.u364u9pwzaa3)

[5.4 Compute and carbon emissions](#h.p0uphk49hxzx)

[5.5 Podcast Episodes](#h.yvllopc0tcwb)

[5.6 Career advice](#h.otzba399qdrv)

[5.7 Newsletters](#h.614otttfgt7j)

[5.8 Youtube Channels](#h.9g7comv75fsm)

[5.9 Forecasting Platforms](#h.49xb9mo0efoo)

[5.10 Bonus: Information/Compute Security](#h.ytq01wbuu2lk)

---

1 Compute in the AI Production Function
---------------------------------------

Pieces that cover the role of compute in developing more powerful AI systems

* [AI Chips: What They Are and Why They Matter](https://www.google.com/url?q=https://cset.georgetown.edu/publication/ai-chips-what-they-are-and-why-they-matter/&sa=D&source=editors&ust=1692801441287600&usg=AOvVaw34UewntkV_Z0z5Vy-IEwuv), CSET, Khan, 2020
* Modern AI training requires specialized hardware. Those AI chips outperform general-purpose chips (CPUs) in speed by ten to a thousand times for machine learning applications and are thus much more energy efficient. Due to their high cost, training a large model can cost up to $1m. The main advantages of AI chips over CPUs are parallel computing and faster memory access. The report further outlines types of AI Chips and current trends.

### 1.1 Compute as an input to AI systems

* [The AI Triad and What It Means for National Security Strategy](https://www.google.com/url?q=https://cset.georgetown.edu/publication/the-ai-triad-and-what-it-means-for-national-security-strategy/&sa=D&source=editors&ust=1692801441288780&usg=AOvVaw0heUFlawS4UHfZnhzwAyD9), CSET, Buchanan, 2020
* “Three components make deep learning happen: data, algorithms, and computing power. Together, I call these components the AI triad.”

### 1.2 Compute usage trends in AI

* [Compute Trends Across Three Eras of Machine Learning](https://www.google.com/url?q=https://arxiv.org/abs/2202.05924&sa=D&source=editors&ust=1692801441289481&usg=AOvVaw05eAMEto_qaz3nKW4aNZ6l), Sevilla et al., 2022
* [Summary on Alignment Forum](https://www.google.com/url?q=https://www.alignmentforum.org/posts/XKtybmbjhC6mXDm5z/compute-trends-across-three-eras-of-machine-learning&sa=D&source=editors&ust=1692801441289901&usg=AOvVaw1aA5ZR1TXsQ_dM-TvHQtic)
* [Twitter summary](https://www.google.com/url?q=https://twitter.com/ohlennart/status/1493521176286609412&sa=D&source=editors&ust=1692801441290215&usg=AOvVaw0qRy3of8DbhlfvRt7diXHJ)
* Database:[Parameter, Compute and Data Trends in Machine Learning](https://www.google.com/url?q=https://docs.google.com/spreadsheets/d/1AAIebjNsnJj_uKALHbXNfn3_YsT6sHXtCU0q7OIPuc4/edit%23gid%3D0&sa=D&source=editors&ust=1692801441290662&usg=AOvVaw2a6RowlecHEkKZjmdint2w)
* [AI and Compute](https://www.google.com/url?q=https://openai.com/blog/ai-and-compute/&sa=D&source=editors&ust=1692801441290947&usg=AOvVaw0IUC4j-TcotVKc5esH2CDw), OpenAI, Amodei & Hernandez, 2020
* [Compute Trends — Comparison to OpenAI's AI and Compute](https://www.google.com/url?q=https://www.lesswrong.com/posts/sDiGGhpw7Evw7zdR4/compute-trends-comparison-to-openai-s-ai-and-compute&sa=D&source=editors&ust=1692801441291329&usg=AOvVaw3-XfB62NUvpQINm53RYkId), LessWrong, Heim et al., 2022
* Compares the two analyses above

### 1.3 Scaling laws

Exploring the connection of compute and capabilities of AI systems.

* [Scaling Hypothesis](https://www.google.com/url?q=https://www.gwern.net/Scaling-hypothesis&sa=D&source=editors&ust=1692801441292252&usg=AOvVaw3iBkrIQS39eWS90VlKZImf) Gwern.net
* Introduction to the Scaling Hypothesis, that AI systems will continue to get more powerful through increasing their compute input
* [Scaling Laws for Neural Language Models](https://www.google.com/url?q=http://arxiv.org/abs/2001.08361&sa=D&source=editors&ust=1692801441292719&usg=AOvVaw3oPAiBqNq-6ZoH5C1V9Ai7), Kaplan et al. (OpenAI), 2020
* Studies the optimal allocation of a fixed compute budget and derives laws for compute scaling.
* [Training Compute-Optimal Large Language Models](https://www.google.com/url?q=http://arxiv.org/abs/2203.15556&sa=D&source=editors&ust=1692801441293184&usg=AOvVaw2Qd_8o6b9QJGFsEJqHcBjU), Hofman et al. (DeepMind), 2022
* Update towards data being relatively more important than Kaplan et al, 2020 above found
* [New Scaling Laws for Large Language Models,](https://www.google.com/url?q=https://www.lesswrong.com/posts/midXmMb2Xg37F2Kgn/new-scaling-laws-for-large-language-models&sa=D&source=editors&ust=1692801441293663&usg=AOvVaw3lc9S9I1ERPgIIaGpsRx6c) LessWrong, 1a3orn, 2022
* Summary and comparing the papers above
* [Chinchillas Wild Implications](https://www.google.com/url?q=https://www.lesswrong.com/posts/6Fpvch8RR29qLEWNH/chinchilla-s-wild-implications&sa=D&source=editors&ust=1692801441294007&usg=AOvVaw2FbDiwjSlmqPv-ifSMNJj3), LessWrong, nostalgebraist, 2022
* The new scaling laws proposed by DeepMind are a major update toward compute being relatively less important and data potentially being a bottleneck for building bigger systems.
* [Extrapolating GPT-N performance - AI Alignment Forum](https://www.google.com/url?q=https://www.alignmentforum.org/posts/k2SNji3jXaLGhBeYP/extrapolating-gpt-n-performance&sa=D&source=editors&ust=1692801441294410&usg=AOvVaw0nBoAvwhc2ZLxb-2AWlMp-), Finnveden, 2020
* Scaling laws for other domains
* [Scaling Laws for Autoregressive Generative Modeling](https://www.google.com/url?q=https://arxiv.org/abs/2010.14701&sa=D&source=editors&ust=1692801441294771&usg=AOvVaw3UietEpbx56Z2TUwPXcI7K), OpenAI, Henighan et al., 2020
* [Will we run out of ML data? Evidence from projecting dataset size trends](https://www.google.com/url?q=https://epochai.org/blog/will-we-run-out-of-ml-data-evidence-from-projecting-dataset&sa=D&source=editors&ust=1692801441295115&usg=AOvVaw0fuywgnQs6iHaZAO1lmuna), Epoch, Villalobos et al., 2022

### 1.4 Compute efficiency

Algorithmic improvements allow us to achieve similar capabilities for less compute. Therefore, monitoring them informs the prioritization of compute.

* [AI and Efficiency](https://www.google.com/url?q=https://openai.com/blog/ai-and-efficiency/&sa=D&source=editors&ust=1692801441295866&usg=AOvVaw3j2wKswmU4h1wIndBt0wmn), OpenAI, Hernandez & Brown, 2020
* [Revisiting algorithmic progress](https://www.google.com/url?q=https://epochai.org/blog/revisiting-algorithmic-progress&sa=D&source=editors&ust=1692801441296242&usg=AOvVaw24NlVU1by04TVHRJtLoBAy), Epoch, Erdil & Besiroglu, 2022

### 1.5 How to use compute efficiently

* [OpenAI overview of how conversational models are made](https://www.google.com/url?q=https://build.microsoft.com/en-US/sessions/db3f4859-cd30-4445-a0cd-553c3304f8e2&sa=D&source=editors&ust=1692801441296764&usg=AOvVaw3yK7YKUFD8QaEYkImJWane) (Video, 42 min), Andrey Karpathy

2 Compute Supply Landscape
--------------------------

* [Supply Chain Explorer](https://www.google.com/url?q=https://chipexplorer.eto.tech/&sa=D&source=editors&ust=1692801441297300&usg=AOvVaw0Rc7Ub3Y5k-t0G8CNi1LOz), CSET, Emerging Technology Observatory, 2022
* Interactive summary of the compute supply chain, great to get an appreciation for its complexity and learn about specific inputs
* [A blog post about the main takeaways](https://www.google.com/url?q=https://eto.tech/blog/five-takeaways-chip-supply-chain/&sa=D&source=editors&ust=1692801441297701&usg=AOvVaw3lmeEkyc5cfyQU4OcMZVlO)
* [The Geopolitics Of AI Chips Will Define The Future Of AI](https://www.google.com/url?q=https://www.forbes.com/sites/robtoews/2023/05/07/the-geopolitics-of-ai-chips-will-define-the-future-of-ai/?sh%3D319d39095c5c&sa=D&source=editors&ust=1692801441298030&usg=AOvVaw1m3c7otXpMRH_yh5K-jpQj), Forbes, Rob Toews

### 2.1 Semiconductor supply chain

With semiconductors, I usually refer to the production of chips — most notably the semiconductor supply chain.

* [Quick introduction graphics to the semiconductor industry](https://www.google.com/url?q=https://twitter.com/Invesquotes/status/1516429054018310157&sa=D&source=editors&ust=1692801441298611&usg=AOvVaw1LWXME07a2JT7I7AGKOoDm), Twitter, Leandro, 2022
* The first page covers types of chips and how they are made, the second continues with the production, discusses Moore's law, and gives a brief introduction to the industry's history
* [The Huge Endeavor to Produce a Tiny Microchip](https://www.google.com/url?q=https://www.nytimes.com/2022/04/08/technology/intel-chip-shortage.html&sa=D&source=editors&ust=1692801441298976&usg=AOvVaw15pfu2xpdIxgloquJL82QO), NYT, Clark, 2022
* Short introduction to chip production with a lot of photos. Fabs (factories producing chips) are sophisticated facilities with large cleanrooms with controlled conditions for producing microscopically precise chips. Construction costs billions and needs various specialized equipment.
* [The Semiconductor Supply Chain](https://www.google.com/url?q=https://cset.georgetown.edu/publication/the-semiconductor-supply-chain/&sa=D&source=editors&ust=1692801441299464&usg=AOvVaw10t6s6hcRLx1G6mflWa7kN), CSET, Khan, 2020
* Summarizes each component of the semiconductor supply chain and where the United States and its allies possess the greatest leverage
* [China, Semiconductors, and the Push for Independence - Part 1](https://www.google.com/url?q=https://lillianli.substack.com/p/china-semiconductors-and-the-push?s%3Dr&sa=D&source=editors&ust=1692801441299929&usg=AOvVaw0vYITP7v4M7JVsPErtibdq), Substack, Li, 2021
* [Decoupling in Strategic Technologies](https://www.google.com/url?q=https://cset.georgetown.edu/publication/decoupling-in-strategic-technologies/&sa=D&source=editors&ust=1692801441300240&usg=AOvVaw0ln8BEPP6Wi7v5d7upn3TD), CSET, Hwang & Weinstein, 2022
* [ASML's Secret: An exclusive view from inside the global semiconductor giant | VPRO Documentary](https://www.google.com/url?q=https://www.youtube.com/watch?v%3DzQu_TMgHO98%26t%3D3s&sa=D&source=editors&ust=1692801441300553&usg=AOvVaw1NmJTvlnrbgeBudwN5e5oS), Documentary, 2022
* ASML is the only company worldwide capable of producing the lithography machines needed to fabricate advanced chips at scale. They continuously invest in developing the next generation of lithography machines, that could be used to make chips that are even more efficient.
* [How Taiwan became the indispensable economy](https://www.google.com/url?q=https://asia.nikkei.com/static/vdata/infographics/taiwan-economy/&sa=D&source=editors&ust=1692801441300978&usg=AOvVaw318vESngoIooeauDegdNdX), Nikkei Asia, 2023
* [Mapping the Semiconductor Supply Chain: The Critical Role of the Indo-Pacific Region](https://www.google.com/url?q=https://www.csis.org/analysis/mapping-semiconductor-supply-chain-critical-role-indo-pacific-region&sa=D&source=editors&ust=1692801441301433&usg=AOvVaw0MqW928KqHwUyQhC4Q2lKH), CSET, Thadani & Allen, 2023

### 2.2 High-Performance Computing (HPC) and AI

* [Reinventing High-Performance Computing: Challenges and Opportunities](https://www.google.com/url?q=http://arxiv.org/abs/2203.02544&sa=D&source=editors&ust=1692801441301954&usg=AOvVaw203YcVV7xWmv0hoe0xV48Z), Reed et al., 2022
* [Twitter summary](https://www.google.com/url?q=https://twitter.com/ohlennart/status/1518656432438226947&sa=D&source=editors&ust=1692801441302196&usg=AOvVaw05SPn2ImEceA3GRlc64MPv)
* I think this paper provides a decent overview of HPC and upcoming challenges (among others for AI).
* [A Roadmap for Big Model](https://www.google.com/url?q=https://arxiv.org/abs/2203.14101&sa=D&source=editors&ust=1692801441302496&usg=AOvVaw1MzzIiTO3uHzt-joXE1ThZ) (withdrawn, older version [here](https://www.google.com/url?q=https://arxiv.org/abs/2203.14101v1&sa=D&source=editors&ust=1692801441302682&usg=AOvVaw2lagusKpB4Vwqnry7ss6BX)), Yuan et al, 2022
* Paper from China laying out a roadmap for creating big models. [Section 4](https://www.google.com/url?q=https://arxiv.org/pdf/2203.14101v3.pdf%23section.4&sa=D&source=editors&ust=1692801441303009&usg=AOvVaw0bnqHfgPTr_NjAY3bM-uyL) lays out some important considerations for computing and training such “big models”. While this paper has been accused of plagiarism, I still think it provides a decent overview of the challenges for AI compute.
* [Exascale Computing Technology Challenges](https://www.google.com/url?q=http://link.springer.com/10.1007/978-3-642-19328-6_1&sa=D&source=editors&ust=1692801441303440&usg=AOvVaw2U1u0K15bvAF60C3C78RpX), Shalf et al., 2011
* As clock speed increases become more difficult due to physical limits of shrinking transistor size, parallelizing on the chip could deliver further performance gains. The paper explores connected challenges.

### 2.3 Trends in Computing Hardware

* [Moore's Law, AI, and the pace of progress](https://www.google.com/url?q=https://www.lesswrong.com/posts/aNAFrGbzXddQBMDqh/moore-s-law-ai-and-the-pace-of-progress&sa=D&source=editors&ust=1692801441303935&usg=AOvVaw17V9PegawyysS4gwyTzATx), LessWrong, Veedrac, 2021
* Some insights into recent hardware trends and why they (don’t) matter for AI.
* [The big question of how small chips can get | Financial Times](https://www.google.com/url?q=https://www.ft.com/content/fbf52ede-e1a9-4797-9e52-3f80a7d855d9&sa=D&source=editors&ust=1692801441304341&usg=AOvVaw2qZN_bQimDvTpNnC6C1jl-), Bradshaw & Gross, 2023
* Benchmarks (not necessarily a “reading” but rather a resource)
* Benchmarks are useful to measure progress in computing hardware. Find below some commonly used benchmarks for HPC and ML-specific workloads. Note that [various](https://www.google.com/url?q=https://www.lesswrong.com/posts/G4KHuYC3pHry6yMhi/compute-research-questions-and-metrics-transformative-ai-and%23B__Metrics&sa=D&source=editors&ust=1692801441304888&usg=AOvVaw0rwmqvUzjJt1Byilkuf_Go) [caveats apply](https://www.google.com/url?q=https://www.lesswrong.com/posts/G4KHuYC3pHry6yMhi/compute-research-questions-and-metrics-transformative-ai-and%23B__Metrics&sa=D&source=editors&ust=1692801441305156&usg=AOvVaw07FIvAzdxkubFWVdHejGhb).
* [TOP500](https://www.google.com/url?q=https://www.top500.org/&sa=D&source=editors&ust=1692801441305459&usg=AOvVaw0eqbHff6JZ0DGXmDEgwmah) / Graph500 / Green500
* The TOP500 benchmark is not an ML benchmark and uses different computation schemes (FP64 etc.). Some of them have GPUs, some don’t. One should expect new systems to have GPUs and be usable for AI workloads.
* [ML Commons Training HPC](https://www.google.com/url?q=https://mlcommons.org/en/training-hpc-20/&sa=D&source=editors&ust=1692801441305868&usg=AOvVaw2UXzleS9ZFV8koSETxotsi)
* Benchmark suite for ML training
* [Trends in GPU price-performance](https://www.google.com/url?q=https://epochai.org/blog/trends-in-gpu-price-performance&sa=D&source=editors&ust=1692801441306361&usg=AOvVaw3SG5IB3AN_ImdIH6UntyWV), Epoch, Hobbhahn & Besiroglu, 2022
* [Predicting GPU performance](https://www.google.com/url?q=https://epochai.org/blog/predicting-gpu-performance&sa=D&source=editors&ust=1692801441306660&usg=AOvVaw1miODWqy0AVqwyOcEh8tmw), Epoch, Hobbhahn & Besiroglu, 2022

### 2.4 Where’s all the compute?

* [The world's distribution of computation (initial findings) - Machine Intelligence Research Institute](https://www.google.com/url?q=https://intelligence.org/2014/02/28/the-worlds-distribution-of-computation-initial-findings/&sa=D&source=editors&ust=1692801441307112&usg=AOvVaw22PJL9GLhopDGRlt-bsiqj), MIRI, Naik, 2014
* [Note that I would be interested in an updated investigation of this. Got a couple of ideas on how to go about it.]

#### 2.4.1 China

* [White Paper on China's Computing Power](https://www.google.com/url?q=https://chinai.substack.com/p/chinai-159-white-paper-on-chinas&sa=D&source=editors&ust=1692801441307750&usg=AOvVaw0SWpgb9kfIkQ8tqxI5XMRd), ChinAI, 2023

### 2.5 What’s next in computing?

* [There's plenty of room at the Top: What will drive computer performance after Moore's law?](https://www.google.com/url?q=https://www.science.org/doi/10.1126/science.aam9744&sa=D&source=editors&ust=1692801441308314&usg=AOvVaw3j5DlNXOfyZs9YNZ4jd3Rx), Leiserson et al. 2020
* Although Moore’s law has been slowing down in the last years, further advances in computing could be gained through better algorithms, code and specialized hardware.
* [The Hardware Lottery](https://www.google.com/url?q=http://arxiv.org/abs/2009.06489&sa=D&source=editors&ust=1692801441308692&usg=AOvVaw3b3UPXsCwCHsKY2psuMDqM), Hooker, 2020
* Our current hardware paradigms are dictating which AI innovations succeed.
* [The decline of computers as a general purpose technology,](https://www.google.com/url?q=https://doi.org/10.1145/3430936&sa=D&source=editors&ust=1692801441309082&usg=AOvVaw1GjSHG6-5eQF2se7Reg6n9) Communications of the ACM, Thompson, 2021
* We will see more specialized chips in the future — first specialized ones for AI and then even for different AI workloads (such as inference and training).
* [Beyond CMOS: the Future of Semiconductors](https://www.google.com/url?q=https://irds.ieee.org/home/what-is-beyond-cmos&sa=D&source=editors&ust=1692801441309497&usg=AOvVaw2DvsBYrio_-tY9kAhgNN_f), IEEE IRDS, 2018
* [The Accelerator Wall: Limits of Chip Specialization](https://www.google.com/url?q=https://parallel.princeton.edu/papers/wall-hpca19.pdf&sa=D&source=editors&ust=1692801441309831&usg=AOvVaw3qsEUoBpZXyZ9hXH1mEyX1), Fuchs & Wntzlaff, 2019

3 Compute Governance
--------------------

The previous reading layed out the technical foundation of AI and compute. Compute Governance focuses on compute as a governance lever by either monitoring, restricting, or promoting access.

### 3.1 Motivation

* [AI Governance: Opportunity and Theory of Impac](https://www.google.com/url?q=https://www.allandafoe.com/opportunity&sa=D&source=editors&ust=1692801441310603&usg=AOvVaw1kOKc2NaeP3qIDG_qV5k29)t, Allen Dafoe (Centre for the Governance of AI), 2020
* General motivation for governing advancing AI systems to limit risks to society coming from different aspects of the technology
* Section 6 in [Compute Governance and Conclusions - Transformative AI and Compute [3/4]](https://www.google.com/url?q=https://www.lesswrong.com/s/bJi3hd8E8qjBeHz9Z/p/M3xpp7CZ2JaSafDJB&sa=D&source=editors&ust=1692801441311206&usg=AOvVaw2cqPBa8yriOGVDNpvLVJvV), Heim, 2021
* Compute is a uniquely governable input into AI systems because it is specialized, expensive, highly centralized and requires physical space and energy to be deployed. This means restrictions on compute can be enforced more easily than restrictions on data or algorithms.
* [Compute and Antitrust](https://www.google.com/url?q=https://verfassungsblog.de/compute-and-antitrust/&sa=D&source=editors&ust=1692801441311704&usg=AOvVaw1gIi58wQAZTP-0QodZjuC0), Verfassungsblog, Haydn Belfield & Shin-Shin Hua, 2022
* We argue that the antitrust and regulatory literature to date has failed to pay sufficient attention to compute, despite compute being a key input to AI progress and services, the potentially substantial market power of companies in the supply chain, and the advantages of compute as a ‘unit’ of regulation in terms of detection and remedies.
* [Compute Accounting Principles Can Help Reduce AI Risks](https://www.google.com/url?q=https://techpolicy.press/compute-accounting-principles-can-help-reduce-ai-risks/&sa=D&source=editors&ust=1692801441312202&usg=AOvVaw1x5n1omCmEpcQhL5l5E1LV), Jackson et al., 2023
* [Arms Control for Artificial Intelligence - Texas National Security Review](https://www.google.com/url?q=https://tnsr.org/2023/05/arms-control-for-artificial-intelligence/&sa=D&source=editors&ust=1692801441312623&usg=AOvVaw3hr-E_75xhByj8N9mWoy1i), Lamberth & Scharre, 2023

### 3.2 Existing policy work relevant to Compute Governance

* [UK’s Future of Compute Review](https://www.google.com/url?q=https://www.gov.uk/government/publications/future-of-compute-review&sa=D&source=editors&ust=1692801441313200&usg=AOvVaw0imD8hCDC9fGLioNrpH6Mi) (2022)
* The UK government calls for proposals to establish a national compute strategy
* [Future of compute review - submission of evidence](https://www.google.com/url?q=https://www.longtermresilience.org/post/future-of-compute-review-submission-of-evidence&sa=D&source=editors&ust=1692801441313698&usg=AOvVaw3c8NhdvXHt901tkac-fHQi), CLTR, Whittlestone et al., 2022
* Suggests enabling more access to compute for academia and beginning to monitor (AI) compute.
* [GovAI Response to the UK Future of Compute Review - Call for Evidence](https://www.google.com/url?q=https://www.governance.ai/research-paper/future-of-compute-review-call-for-evidence&sa=D&source=editors&ust=1692801441314286&usg=AOvVaw1D1RjNVMh7yjJaoV44Rm4F), Heim & Anderljung, 2022
* Suggests structured access to AI compute favoring research focused on interpretability and safety and ensuring that irresponsible actors do not get access.
* [Response to the UK’s Future of Compute Review: A missed opportunity to lead in compute governance](https://www.google.com/url?q=https://www.governance.ai/research-paper/response-to-the-uks-future-of-compute-review&sa=D&source=editors&ust=1692801441314786&usg=AOvVaw2EpH5Y6UfOQOu8TNZ-T0bJ), CLTR & GovAI, and others, 2022
* [US National AI Research Resource (NAIRR)](https://www.google.com/url?q=https://www.federalregister.gov/documents/2022/05/25/2022-11223/request-for-information-rfi-on-implementing-initial-findings-and-recommendations-of-the-national&sa=D&source=editors&ust=1692801441315284&usg=AOvVaw2hO6uYQSgWPeKn04ldJ_r6)[Request for Information](https://www.google.com/url?q=https://www.federalregister.gov/documents/2022/05/25/2022-11223/request-for-information-rfi-on-implementing-initial-findings-and-recommendations-of-the-national&sa=D&source=editors&ust=1692801441315557&usg=AOvVaw2mZ70U4aV08pG1LTGgJ9ZC) (2022)
* Relevant to any discussion around giving academics more compute and setting up (inter-)national computing centers.
* [Submission to the Request for Information (RFI) on Implementing Initial Findings and Recommendations of the NAIRR Task Force](https://www.google.com/url?q=https://www.governance.ai/research-paper/submission-nairr-task-force&sa=D&source=editors&ust=1692801441315949&usg=AOvVaw0D56maJR34LxgREopXIItA), GovAI, Heim & Anderljung, 2022
* Suggests giving academia access to pre-trained models via an API while ensuring research is conscious of safety concerns and favoring actors doing safety-relevant research.
* [A blueprint for building national compute capacity for artificial intelligence](https://www.google.com/url?q=https://oecd.ai/en/compute-report&sa=D&source=editors&ust=1692801441316415&usg=AOvVaw3uwX_hRuuhWyk3KRfEpodH), OECD AI, 2023

### 3.3 Promoting Access through Compute Funds

As machine learning models get increasingly larger, research requires access to large amounts of compute. Consequently, there have been calls to establish compute funds to provide subsidized access for academia.

* [White Paper | Building a National AI Research Resource](https://www.google.com/url?q=https://hai.stanford.edu/white-paper-building-national-ai-research-resource&sa=D&source=editors&ust=1692801441317277&usg=AOvVaw08-D3nRsKPW9VuYyPTcRAW), Stanford HAI, Ho et al., 2021
* Proposes a US national project to give academia access to compute to ensure it can work on state-of-the-art AI models.
* [Compute Funds and Pre-trained Model](https://www.google.com/url?q=https://www.governance.ai/post/compute-funds-and-pre-trained-models&sa=D&source=editors&ust=1692801441317734&usg=AOvVaw2ohvH9HDzfOEuT_QHmP_2x)[s](https://www.google.com/url?q=https://www.governance.ai/post/compute-funds-and-pre-trained-models&sa=D&source=editors&ust=1692801441317956&usg=AOvVaw25YCgCS3nfU7Rki9gVqeMO), GovAI, Anderljung et al., 2022
* Suggests giving academia access to large AI models via structured access
* [Twitter summary](https://www.google.com/url?q=https://twitter.com/ohlennart/status/1513572670109061124&sa=D&source=editors&ust=1692801441318468&usg=AOvVaw1tBYAN-qXd8KR3z8hIgjNK)
* Based on the above recommendation, we submitted to the US National AI Research Resources (NAIRR) task force (see section above)

### 3.4 Measuring compute

Measuring compute is crucial for understanding the AI capabilities of relevant actors. It’s also a prerequisite to regulating or allocating compute.

* [Large-scale computing: the case for greater UK coordination](https://www.google.com/url?q=https://www.gov.uk/government/publications/large-scale-computing-the-case-for-greater-uk-coordination&sa=D&source=editors&ust=1692801441319425&usg=AOvVaw2Sa9_U3hLoxrEAmg5CWxqp), UK Government, 2021
* Describes the economic value of computing infrastructure and suggests more public investment
* Measuring national compute capacity for Artificial Intelligence (AI): existing measurement tools and preliminary findings, OECD, 2022 (not declassified yet)
* Related blog posts
* [Measuring compute capacity: a critical step to capturing AI's full economic potential](https://www.google.com/url?q=https://oecd.ai/en/wonk/ai-compute-capacity&sa=D&source=editors&ust=1692801441320164&usg=AOvVaw20cbKQWPN3CTCR9yzsfYjs), OECD.AI, Strier et al., 2022
* Most countries don’t know how much compute they have. As compute is a key driver for innovation, they will need to measure it to make informed decisions. To stay competitive, countries will need to invest into national compute resources.
* [Reducing the carbon emissions of AI](https://www.google.com/url?q=https://oecd.ai/en/wonk/reducing-ai-carbon-emissions&sa=D&source=editors&ust=1692801441320575&usg=AOvVaw2hsHqD2rpi2xyE2onGMaZ-), OECD.AI, Patterson, 2022
* Suggests reducing carbon emissions of AI training by supporting further efficiency gains in algorithms and hardware and incentivizing the use of green energy.

#### 3.4.1 Monitoring compute

* [Why and How Governments Should Monitor AI Development](https://www.google.com/url?q=http://arxiv.org/abs/2108.12427&sa=D&source=editors&ust=1692801441321314&usg=AOvVaw3B046bFoXQGS6tHqC3gu-z), Whittlestone & Clark, 2021
* Governments already face challenges in relation to governing AI. These will get more severe as systems advance. To make informed decisions and notice trends early, they should set up mechanisms for monitoring different aspects of AI developments as well as their impacts.
* Pieces mentioning and discussing compute monitoring
* [Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims](https://www.google.com/url?q=https://arxiv.org/abs/2004.07213&sa=D&source=editors&ust=1692801441321770&usg=AOvVaw39gLe1IkYk62VvKodBFhFS), Brundage et al. (OpenAI), 2020
* 4.2: The lack of standards is an obstacle to more widespread reporting of compute used for training AI systems. AI labs should start reporting their system specifications in more detail to encourage a wide adoption of the procedure.
* [Open Problems in Cooperative AI](https://www.google.com/url?q=https://arxiv.org/abs/2012.08630&sa=D&source=editors&ust=1692801441322178&usg=AOvVaw36U4LFxtO8h2GdCZrfy0hP), DeepMind, Dafoe et al., 2020

#### 3.4.2 Verification

Verifying and proving the amount of compute one has.

* [Joint Verification Experiments](https://www.google.com/url?q=https://www.armscontrolwonk.com/archive/402990/joint-verification-experiments/&sa=D&source=editors&ust=1692801441322743&usg=AOvVaw3svbPiTQ5XEqvhDvglMssW), Arms Control Wonk, Krepon, 2011
* [requires explanation]

### 3.5 Export Restrictions

For the moment, this is mostly focusing on the semiconductor supply chain.

* All hardware and compute publications by CSET:
* [Banned in DC - Examining Government Approaches to Foreign Technology Threats](https://www.google.com/url?q=https://cset.georgetown.edu/publications/?fwp_topic%3Dhardware-and-compute&sa=D&source=editors&ust=1692801441323629&usg=AOvVaw2EGuLHgN3383yvhY-p19Bs), CSET, Corrigan et al., 2022
* [Securing Semiconductor Supply Chains](https://www.google.com/url?q=https://cset.georgetown.edu/publication/securing-semiconductor-supply-chains/&sa=D&source=editors&ust=1692801441323920&usg=AOvVaw3bdj7ocmRUJedDG5R9Buzy), CSET, Khan, 2021
* [Slowing Moore's Law: How It Could Happen](https://www.google.com/url?q=https://www.gwern.net/Slowing-Moores-Law&sa=D&source=editors&ust=1692801441324240&usg=AOvVaw1B-d0xGtsTia-5ayZS0Lry), Gwern, Branwen, 2017
* [Digitizing Export Controls: A Trade Compliance Technology Stack?](https://www.google.com/url?q=https://www.csis.org/analysis/digitizing-export-controls-trade-compliance-technology-stack&sa=D&source=editors&ust=1692801441324526&usg=AOvVaw2p0l3H03B8BGD4zErznqmx), Center for Strategic and International Studies (CSIS), Reinsch & Benson, 2021
* [Microsoft and OpenAI’s comment to BIS](https://www.google.com/url?q=https://www.regulations.gov/comment/BIS-2020-0029-0056&sa=D&source=editors&ust=1692801441324756&usg=AOvVaw23dizkMWT4K_KlIBdNrWdx), O’Neal & Clark, 2020
* advocates for “smart” export controls
* [CoCom’s daughter?](https://www.google.com/url?q=https://cset.georgetown.edu/article/cocoms-daughter/&sa=D&source=editors&ust=1692801441325063&usg=AOvVaw3fgPSiI7dv74h2F05Vy1iJ), CSET, Weinstein & Wolf, 2022 discusses the potential for new multilateral export controls.
* A[ssociated presentation](https://www.google.com/url?q=https://www.youtube.com/watch?v%3DajC5yIoFZFE&sa=D&source=editors&ust=1692801441325450&usg=AOvVaw21iRlENQp1bDOfZ-RYkY1n)

#### 3.5.1 US Export Restrictions from October 2022

The Biden administration restricted the export of advanced AI hardware and tools and software required to build it to China (and Russia) in September 2022. ([Here's the official PDF.](https://www.google.com/url?q=https://t.co/sVENwdho1h&sa=D&source=editors&ust=1692801441325887&usg=AOvVaw2xlAVuDiRhJKGNdTQzvXTZ))

* [Choking Off China’s Access to the Future of AI](https://www.google.com/url?q=https://www.csis.org/analysis/choking-chinas-access-future-ai&sa=D&source=editors&ust=1692801441326250&usg=AOvVaw32gSfumyZZR_SQ16q5OqAs), CSIS, Allen, 2022
* Media summaries:
* SemiAnalysis: [China and USA Are Officially At Economic War – Technology Restriction Overview](https://www.google.com/url?q=https://www.semianalysis.com/p/china-and-usa-are-officially-at-economic&sa=D&source=editors&ust=1692801441326589&usg=AOvVaw1PR53EzfxeN_TqaMYvhweF)
* NYT: [Biden Administration Clamps Down on China’s Access to Chip Technology](https://www.google.com/url?q=https://www.nytimes.com/2022/10/07/business/economy/biden-chip-technology.html&sa=D&source=editors&ust=1692801441326935&usg=AOvVaw2lVyHMaE5ysC9qEImrScNy)
* South China Morning Post: [Tech war: US chip restrictions could cost 0.6 per cent of China’s GDP and weigh on yuan, according to Barclays report](https://www.google.com/url?q=https://www.scmp.com/tech/big-tech/article/3199860/tech-war-us-chip-restrictions-could-cost-06-cent-chinas-gdp-and-weigh-yuan-according-barclays-report&sa=D&source=editors&ust=1692801441327257&usg=AOvVaw3HknnjvrLO-iy6pBNE2VP8)
* [Freeze-in-Place: The Impact of US Tech Controls on China](https://www.google.com/url?q=https://rhg.com/research/freeze-in-place/&sa=D&source=editors&ust=1692801441327470&usg=AOvVaw0Bt5xuQsbG27ZuQYU0Es36), Rhodium Group, Goujon et al., 2022
* [New Chip Export Controls and the Sullivan Tech Doctrine with Kevin Wolf](https://www.google.com/url?q=https://www.chinatalk.media/p/new-chip-export-controls-explained&sa=D&source=editors&ust=1692801441327806&usg=AOvVaw2me9iMEOGarJPePUrBiNVa), China Talk, Schneider and Zhang, 2022[A Bombshell For U.S.-China Tech Relations](https://www.google.com/url?q=https://www.noemamag.com/a-bombshell-for-u-s-china-tech-relations/&sa=D&source=editors&ust=1692801441327993&usg=AOvVaw0qdIXbC5srCcue-iqi-Tt5), NOEMA, Jordan Schneider, 2022
* [The AI Lockout](https://www.google.com/url?q=https://www.thewirechina.com/2023/03/12/the-ai-lockout-nvidia-china/&sa=D&source=editors&ust=1692801441328242&usg=AOvVaw34eMibS-KSgrSU637cOag0), The Wire China, 2023 ([pdf version](https://www.google.com/url?q=https://www.thewirechina.com/wp-content/uploads/2023/03/The-AI-Lockout.pdf&sa=D&source=editors&ust=1692801441328495&usg=AOvVaw2JHlN85IxVAN7pRBmnt7HX))
* [Japan and the Netherlands Announce Plans for New Export Controls on Semiconductor Equipment](https://www.google.com/url?q=https://www.csis.org/analysis/japan-and-netherlands-announce-plans-new-export-controls-semiconductor-equipment&sa=D&source=editors&ust=1692801441328791&usg=AOvVaw3jxdvZi0aGBPk4cmRFPWrE), CSIS, Allen & Benson, 2023

### 3.6 Verifying Compute Use

* [What does it take to catch a Chinchilla? Verifying Rules on Large-Scale Neural Network Training via Compute Monitoring](https://www.google.com/url?q=https://arxiv.org/abs/2303.11341&sa=D&source=editors&ust=1692801441329171&usg=AOvVaw1jM-iedXEFRbc3I7wIepLr), Yondav Shavit, 2023
* Hardware-based mechanisms on AI accelerators could periodically log hashed snapshots of the model weights in their memory. These could be used to verify claims about training specifications, allowing governments or international organisations to enforce rules on responsible AI development.
* One of the most thorough proposals of a compute monitoring regime

### 3.7 Hardware Security

* [AI Verification: Mechanisms to Ensure AI Arms Control Compliance](https://www.google.com/url?q=https://cset.georgetown.edu/publication/ai-verification/&sa=D&source=editors&ust=1692801441329780&usg=AOvVaw1NtVd8CjbzWWBThYGSSNkL), CSET, Mittelsteadt, 2021
* The Malicious Use of Artificial Intelligence: [Forecasting, Prevention, and Mitigation](https://www.google.com/url?q=https://maliciousaireport.com/&sa=D&source=editors&ust=1692801441330028&usg=AOvVaw0nzmXj4DJL0haxy_-2vZYS), maliciousaireport.com, Brundage et al., 2018
* See secure hardware recommendations
* [AI Research Considerations for Human Existential Safety](https://www.google.com/url?q=https://arxiv.org/abs/2006.04948&sa=D&source=editors&ust=1692801441330362&usg=AOvVaw0stjFmRq6jeYQN6HdUgiKr), Critch & Krueger, 2022
* See secure hardware recommendations
* [Structured Access: An Emerging Paradigm for Safe AI Deployment](https://www.google.com/url?q=https://arxiv.org/abs/2201.05159&sa=D&source=editors&ust=1692801441330702&usg=AOvVaw1jIvfvg58170Ne2v2XlWZg), Shevlane, 2022
* See discussion of hardware-level mechanisms to control access to models
* Some specific technical papers on the above mechanisms:
* [Hardware-Assisted Intellectual Property Protection of Deep Learning Models](https://www.google.com/url?q=https://ieeexplore.ieee.org/document/9218651&sa=D&source=editors&ust=1692801441331137&usg=AOvVaw2c-D8IoYq-khqLS8t6EXUt), IEEE, Chakraborty et al., 2020
* [Deep-Lock: Secure Authorization for Deep Neural Networks](https://www.google.com/url?q=https://arxiv.org/abs/2008.05966&sa=D&source=editors&ust=1692801441331390&usg=AOvVaw0UZR__QXmKasUFTJPJoMKp), Alam et al., 2020
* [AdvParams: An Active DNN Intellectual Property Protection Technique via Adversarial Perturbation Based Parameter Encryption](https://www.google.com/url?q=https://arxiv.org/abs/2105.13697&sa=D&source=editors&ust=1692801441331620&usg=AOvVaw0jwiKTZVpLqqUXBO5_vIey), Xue et al., 2021
* [Enabling IC Traceability via Blockchain Pegged to Embedded PUF](https://www.google.com/url?q=https://dl.acm.org/doi/fullHtml/10.1145/3315669&sa=D&source=editors&ust=1692801441331842&usg=AOvVaw1oeXSqu-FuwUnlp1MfPsqn), Islam & Kundu, 2019

### 3.8 Compute Providers

Compute providers are at the end of the supply chain and provide access to compute as a service. How could they be governed?

* [Structured access: an emerging paradigm for safe AI deployment](https://www.google.com/url?q=https://arxiv.org/abs/2201.05159&sa=D&source=editors&ust=1692801441332404&usg=AOvVaw3vA69qLsbmc56Hm8xrPgBw), Shevlane
* [Blog post](https://www.google.com/url?q=https://www.governance.ai/post/sharing-powerful-ai-models&sa=D&source=editors&ust=1692801441332645&usg=AOvVaw2NQyZYcBmw2KK2siRmv0pA) summarizing it

### 3.9 Data centers

* [Compute at scale: A broad investigation into the data center industry](https://www.google.com/url?q=https://www.konstantinpilz.com/data-centers/report&sa=D&source=editors&ust=1692801441333007&usg=AOvVaw3aqOBFByorj9VexrRWcAZV), Pilz & Heim, 2023
* Provides an overview of how the industry works, what its key inputs are and draws some conclusion for AI governance, emphasizing data centers’ importance in the AI compute supply chain and their role in training and deploying advanced AI systems. It outlines key features of data centers, such as power consumption, cooling requirements, and infrastructure, while also discussing market size and growth, major players, and potential bottlenecks.
* See [An assessment of data center infrastructure's role in AI governance](https://www.google.com/url?q=https://www.konstantinpilz.com/data-centers/assessment&sa=D&source=editors&ust=1692801441333383&usg=AOvVaw2ATQ_sqmROgM3CIO5pIwn2), Pilz, 2023 for an interpretation of the key findings and some helpful models for how to think about data centers.

### 3.10 Corporate Compute Governance

What could actors providing compute, such as AI hardware companies, do for AI safety?

* [soon]

4 Compute-Based Transformative AI Forecasting
---------------------------------------------

### 4.1 TAI forecasting

* [Updates and Lessons from AI Forecasting](https://www.google.com/url?q=https://bounded-regret.ghost.io/ai-forecasting/&sa=D&source=editors&ust=1692801441334325&usg=AOvVaw1sEKzKEP3qUohA07hXq6i6), Steinhardt, 2021
* [More Is Different for AI](https://www.google.com/url?q=https://www.alignmentforum.org/s/4aARF2ZoBpFZAhbbe&sa=D&source=editors&ust=1692801441334577&usg=AOvVaw3XoI-6cbx1L06gPynkbx5p), Alignment Forum, Steinhardt, 2022
* [AI Timelines: Where the Arguments, and the "Experts," Stand](https://www.google.com/url?q=https://www.cold-takes.com/where-ai-forecasting-stands-today/&sa=D&source=editors&ust=1692801441334831&usg=AOvVaw3WEvV0e38TR9LPvwmLbzzS), Cold Takes, Karnofsky, 2021
* Overview of current attempts at AI forecasting
* [Grokking “Forecasting TAI with biological anchors”](https://www.google.com/url?q=https://www.lesswrong.com/posts/wgio8E758y9XWsi8j/grokking-forecasting-tai-with-biological-anchors&sa=D&source=editors&ust=1692801441335170&usg=AOvVaw18vJtPS4_ITunXMKhHK1HX), LessWrong, Ho, 2022
* Well written and visualized summary of bio-anchors report

### 4.2 Forecasting compute

Forecasting how much compute we might need and how much we will have.

* [Draft report on AI timelines](https://www.google.com/url?q=https://www.alignmentforum.org/posts/KrJfoZzpSDpnrv9va/draft-report-on-ai-timelines&sa=D&source=editors&ust=1692801441335681&usg=AOvVaw3T57-IVLrT2FK_SaDxfuDP) Alignment Forum, Cotra, 2021
* Split into two aspects: (I) how much compute we might need based on biological anchors, and (II) how much compute might be available.
* [Forecasting Compute - Transformative AI and Compute [2/4]](https://www.google.com/url?q=https://forum.effectivealtruism.org/posts/YNB39RyJ7iAQKGJvq/forecasting-compute-transformative-ai-and-compute-2-4&sa=D&source=editors&ust=1692801441336077&usg=AOvVaw29ghrnEcgk-AZ4nqY-8VjO), EA Forum, Heim, 2021
* [Projecting compute trends in Machine Learning](https://www.google.com/url?q=https://www.alignmentforum.org/posts/3dBtgKCkJh5yCHbag/projecting-compute-trends-in-machine-learning-2&sa=D&source=editors&ust=1692801441336357&usg=AOvVaw3yvpwmSnoMIo2Ip9dhmf72), Epoch, AI Alignment Forum, Besiroglu et al., 2022
* [Compute Trends Across Three Eras of Machine Learning](https://www.google.com/url?q=https://arxiv.org/abs/2202.05924&sa=D&source=editors&ust=1692801441336562&usg=AOvVaw0fcMXAs5ESAs2O_fRCsTYj), Sevilla et al., 2022

#### 4.2.1 Interpreting AI compute trends and the limits

* [Interpreting AI compute trends](https://www.google.com/url?q=https://aiimpacts.org/interpreting-ai-compute-trends/&sa=D&source=editors&ust=1692801441336903&usg=AOvVaw20mxKEH1yfkmAzmzDBSa0i), AI Impacts, Carey, 2018
* [Reinterpreting “AI and Compute”](https://www.google.com/url?q=https://aiimpacts.org/reinterpreting-ai-and-compute/&sa=D&source=editors&ust=1692801441337168&usg=AOvVaw3QkbgA8kq4bQ5JGQU9LC0S), AI Impacts, Garfinkel, 2018
* [The Computational Limits of Deep Learning](https://www.google.com/url?q=http://arxiv.org/abs/2007.05558&sa=D&source=editors&ust=1692801441337418&usg=AOvVaw3SY-A6F3jvSqLb3UTBYSm_), Thompson, 2020
* I don’t agree with their exact numbers and how they draw the comparison, however, I think the main message still holds.
* [How Much Longer Can Computing Power Drive Artificial Intelligence Progress?](https://www.google.com/url?q=https://cset.georgetown.edu/publication/ai-and-compute/&sa=D&source=editors&ust=1692801441337724&usg=AOvVaw1K44QKE1wsJPBLc_JMFCGr), CSET, Lohn & Musser, 2022
* Note that their model does assume that computing cost stays constant. Also, they assume a 3.4-months doubling time which is not up to date anymore.
* [TODO: link my draft on “This can’t go on”]

### 4.3 Takeoff scenarios and compute

* [Inference cost limits the impact of ever larger models](https://www.google.com/url?q=https://www.lesswrong.com/posts/zTDkhm6yFq6edhZ7L/inference-cost-limits-the-impact-of-ever-larger-models&sa=D&source=editors&ust=1692801441338377&usg=AOvVaw22NAZ3Km021HeIDTp6r_-z), LessWrong, Mindermann, 2021
* Definition of [Hardware overhang](https://www.google.com/url?q=https://aiimpacts.org/hardware-overhang/&sa=D&source=editors&ust=1692801441338702&usg=AOvVaw39-P9_HybAHZvG1qzHk2Pf) on AI Impacts

### 4.4 Brain Comparisons and analogies

* [Brain Efficiency: Much More than You Wanted to Know](https://www.google.com/url?q=https://www.lesswrong.com/posts/xwBuoE9p8GE7RAuhd&sa=D&source=editors&ust=1692801441339206&usg=AOvVaw04rvcW8LPMyulO-kBNzfow), LessWrong, Canell, 2022
* [How Much Computational Power Does It Take to Match the Human Brain](https://www.google.com/url?q=https://www.openphilanthropy.org/brain-computation-report&sa=D&source=editors&ust=1692801441339462&usg=AOvVaw15q0GBC2a-Wdnr4aKDWsJ0), Open Philanthropy, Carlsmith, 2021

5 Others
--------

Anything that doesn’t fit into the above categories.

### 5.1 Books

* [Chip War](https://www.google.com/url?q=https://www.christophermiller.net/semiconductors-1&sa=D&source=editors&ust=1692801441340059&usg=AOvVaw26_81S8jWnNVyLIi0rG1CE), Christopher Miller, 2022
* Gives a broad overview of the history of semiconductors and the conflicts around them, shedding light on many contemporary issues and the geopolitics of chips.

### 5.2 Research Questions

* [Some AI Governance Research Ideas](https://www.google.com/url?q=https://docs.google.com/document/d/13LJhP3ksrcEBKxYFG5GkJaC2UoxHKUYAHCRdRlpePEc/edit%23heading%3Dh.759074zafivf&sa=D&source=editors&ust=1692801441340611&usg=AOvVaw2JGyN5_Ft5mu_QcXyiVBt5), Anderljung & Carlier, 2021
* [AI Governance Needs Technical Work](https://www.google.com/url?q=https://forum.effectivealtruism.org/posts/BJtekdKrAufyKhBGw/ai-governance-needs-technical-work&sa=D&source=editors&ust=1692801441340880&usg=AOvVaw27zhZSLajuAOClXzj6XO-E), EA Forum, Baker, 2022
* [TODO: add updated doc]

### 5.3 AI chip architecture

If you want to have a more in-depth understanding of AI accelerators, here’s a great series. My best guess is that this is probably not relevant for most projects. However, some understanding might be helpful in categorizing new innovations.

* [AI Accelerators — Part I: Intro](https://www.google.com/url?q=https://medium.com/@adi.fu7/ai-accelerators-part-i-intro-822c2cdb4ca4&sa=D&source=editors&ust=1692801441341603&usg=AOvVaw0MXGLnDOgDYJydvr8V6bF0), Medium, Fuchs, 2022.
* Youtube interview and summary: [All about AI Accelerators: GPU, TPU, Dataflow, Near-Memory, Optical, Neuromorphic & more (w/ Author)](https://www.google.com/url?q=https://www.youtube.com/watch?v%3DVQoyypYTz2U%26t%3D4s&sa=D&source=editors&ust=1692801441341857&usg=AOvVaw3fAYygodDN-v7AnAq5Ct1-)
* [Tutorial on Hardware Accelerators for Deep Neural Networks](https://www.google.com/url?q=http://eyeriss.mit.edu/tutorial.html&sa=D&source=editors&ust=1692801441342074&usg=AOvVaw1-xtAK1GHmJ6ZPYIe_yv1M), by Emer et al., MIT/ Nvidia
* Memory [to explain]
* …
* Interconnect [to explain]
* [An Overview of Efficient Interconnection Networks for Deep Neural Network Accelerators](https://www.google.com/url?q=https://ieeexplore.ieee.org/document/9189825&sa=D&source=editors&ust=1692801441342519&usg=AOvVaw3ozo5E78WydOjq2qG-AwOY), IEEE, Nabavinejad et al., 2020

### 5.4 Compute and carbon emissions

When talking about compute many people are concerned about carbon footprints. Unfortunately, there have been some wrong numbers going around, and I think this aspect is less important than often claimed. Still, knowing a bit about it is useful. Next to this, it provides some insights into the economics and distribution of AI workloads in data centers.

* [The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink](https://www.google.com/url?q=https://arxiv.org/abs/2204.05149&sa=D&source=editors&ust=1692801441343061&usg=AOvVaw2oOBZxrDWRD1EYr5xEb1sT), Paterson et al., 2022
* Predecessor paper: [Carbon Emissions and Large Neural Network Training](https://www.google.com/url?q=https://arxiv.org/abs/2104.10350&sa=D&source=editors&ust=1692801441343340&usg=AOvVaw2kCnqNQGdRgyKv-6K-wR7E), Patterson et al., 2021

### 5.5 Podcast Episodes

* [Danny Hernandez on forecasting and the drivers of AI progress](https://www.google.com/url?q=https://80000hours.org/podcast/episodes/danny-hernandez-forecasting-ai-progress/&sa=D&source=editors&ust=1692801441343826&usg=AOvVaw14J_QnMIdv_1MkZ2MYu5TY), 80,000 Hours
* [TSMC Deep Dive with Asionanometry](https://www.google.com/url?q=https://pca.st/t3mxn9kd&sa=D&source=editors&ust=1692801441344126&usg=AOvVaw35n-e3D6NSglzT8syIFJG_), Compounding Curiosity, 2022
* Some interesting insights into Taiwan and its TSMC cult.
* [The Asianometry Podcast](https://www.google.com/url?q=https://pca.st/oe7bbyya&sa=D&source=editors&ust=1692801441344461&usg=AOvVaw3mDgfRbtRsejfr_6ClXix8) (audio version of YouTube channel linked below)
* [Semiconductors: Everything You Wanted to Know](https://www.google.com/url?q=https://www.youtube.com/watch?v%3Dr6NUO_bymuA&sa=D&source=editors&ust=1692801441344756&usg=AOvVaw2gQ4h9Yx8QVS6tbJNpjRcX)
* [Markus Anderljung on AI Policy](https://www.google.com/url?q=https://theinsideview.ai/markus&sa=D&source=editors&ust=1692801441345024&usg=AOvVaw1BL3pFUo5yNKKxYCl3dwV9), The Inside View, 2022
* Covers some of our work at GovAI on compute governance.
* [Lennart Heim (that’s me!) on AI Compute 101: The Geopolitics of Giant models](https://www.google.com/url?q=https://pca.st/jjcewr1h&sa=D&source=editors&ust=1692801441345434&usg=AOvVaw04ss4OkKEgPf7WloWBZoi2), ChinaTalk, 2023
* [AI Compute 101: The Geopolitics of GPUs](https://www.google.com/url?q=https://www.chinatalk.media/p/ai-compute-101-the-geopolitics-of&sa=D&source=editors&ust=1692801441345670&usg=AOvVaw3topdyQd4bpqOOYg3vCMZW)
* [Compute and the Future of US-China Relations](https://www.google.com/url?q=https://www.chinatalk.media/p/compute-and-the-future-of-us-china&sa=D&source=editors&ust=1692801441345912&usg=AOvVaw2N5MNjWW2Ml3-6V_Ty7zf7)
* [How the rise of AI will affect Taiwan's semiconductor industry](https://www.google.com/url?q=https://pca.st/rucx4j03&sa=D&source=editors&ust=1692801441346157&usg=AOvVaw0yByU7QeHJ8vfW8jjKKyev), Startup Island, 2023
* Good summary of current trends in semiconductors, with strong focus on Taiwan
* [add ChinaTalk episodes covering recent export controls]

### 5.6 Career advice

* [What does it mean to become an expert in AI Hardware?](https://www.google.com/url?q=https://forum.effectivealtruism.org/posts/HrS2pXQ3zuTwr2SKS/what-does-it-mean-to-become-an-expert-in-ai-hardware-1&sa=D&source=editors&ust=1692801441346855&usg=AOvVaw3PsahM9PG6zolAmA34k1eM)[,](https://www.google.com/url?q=https://forum.effectivealtruism.org/posts/HrS2pXQ3zuTwr2SKS/what-does-it-mean-to-become-an-expert-in-ai-hardware-1&sa=D&source=editors&ust=1692801441347093&usg=AOvVaw0mNiRhgVoWB0TW5nnPGy8l) EA Forum, Phenicie 2021
* [Expert in AI hardware - Career review](https://www.google.com/url?q=https://80000hours.org/career-reviews/become-an-expert-in-ai-hardware/&sa=D&source=editors&ust=1692801441347472&usg=AOvVaw0f9HCBmmtsknFnMJk6ZeSq), 80,000 Hours, 2021

### 5.7 Newsletters

* [SemiAnalysis](https://www.google.com/url?q=https://semianalysis.substack.com/&sa=D&source=editors&ust=1692801441347978&usg=AOvVaw09RlTs0cif9lEDMzzlWKhU)
* [The Asianometry Newsletter](https://www.google.com/url?q=https://asianometry.substack.com/&sa=D&source=editors&ust=1692801441348206&usg=AOvVaw1d59rFaxSXEjrWb_2DJRZV)
* [Fabricated Knowledge](https://www.google.com/url?q=https://www.fabricatedknowledge.com/&sa=D&source=editors&ust=1692801441348478&usg=AOvVaw1fnZQIUgLg5YDIaJ7uAQgB)
* [Semi-Literate](https://www.google.com/url?q=https://semiliterate.substack.com&sa=D&source=editors&ust=1692801441348731&usg=AOvVaw1NH6ObcZ85asvWLE888hOI)
* Sometimes touches on hardware/compute
* [ChinAI Newsletter](https://www.google.com/url?q=https://chinai.substack.com/&sa=D&source=editors&ust=1692801441349022&usg=AOvVaw1RyWyum6i7QcQwNjwff2_Y)
* [CSET Newsletter](https://www.google.com/url?q=https://www.google.com/url?q%3Dhttps://georgetown.us20.list-manage.com/subscribe?u%253D248119f7a6386759c3a4f7155%2526id%253Dfcbacf8c3e%26sa%3DD%26source%3Ddocs%26ust%3D1657978953602252%26usg%3DAOvVaw1ebo1VKe2x9jPaZAUbUJcM&sa=D&source=editors&ust=1692801441349360&usg=AOvVaw2GJvPiy1HKUyMMAzu7pAI9)

### 5.8 Youtube Channels

* [TechTechPotato](https://www.google.com/url?q=https://www.youtube.com/c/TechTechPotato&sa=D&source=editors&ust=1692801441349753&usg=AOvVaw2JvWtoSwKXg2-nQtPEC2G6)
* [Asianometry](https://www.google.com/url?q=https://www.youtube.com/c/Asianometry&sa=D&source=editors&ust=1692801441350038&usg=AOvVaw0rYxj8k95Bo2FSDTFdKx1c)

### 5.9 Forecasting Platforms

* [INFER Pub has many forecasting questions on microelectronics](https://www.google.com/url?q=https://www.infer-pub.com/challenges/28-will-the-u-s-regain-and-retain-a-two-generation-lead-in-microelectronic-technology&sa=D&source=editors&ust=1692801441350532&usg=AOvVaw0jH4Om5joRUkseQdVVDWRC) in regards to China and the US
* Metaculus has many questions [related to compute](https://www.google.com/url?q=https://www.metaculus.com/questions/?order_by%3D-rank%26main-feed%26search%3DFLOP&sa=D&source=editors&ust=1692801441350875&usg=AOvVaw3DGQ-D3ZMBzt9x53AemEFk) and [TAI timelines](https://www.google.com/url?q=https://www.metaculus.com/project/ai-fortified-essay-contest/&sa=D&source=editors&ust=1692801441351119&usg=AOvVaw1aRCt8wgoBEAI_nTlj9hSG)

### 5.10 Bonus: Information/Compute Security

I think information security is related to compute governance. To put it simply, we won’t be able to restrict access to compute if you can just hack into the cluster and misuse it.

* [Information security in high-impact areas - Career review](https://www.google.com/url?q=https://80000hours.org/career-reviews/information-security/&sa=D&source=editors&ust=1692801441351830&usg=AOvVaw3cCef1snUNGmJi8VhxkCTD), 80,000 Hours, 2023
* [Information security considerations for AI and the long term future](https://www.google.com/url?q=https://forum.effectivealtruism.org/posts/WqQDCCLWbYfFRwubf/information-security-considerations-for-ai-and-the-long-term&sa=D&source=editors&ust=1692801441352234&usg=AOvVaw0TzmpahVadVJcKlFO4Didp)[,](https://www.google.com/url?q=https://forum.effectivealtruism.org/posts/WqQDCCLWbYfFRwubf/information-security-considerations-for-ai-and-the-long-term&sa=D&source=editors&ust=1692801441352465&usg=AOvVaw37qDCvVUwuZBTvbSSIIqCB) EA Forum, Ladish & Heim, 2022
* Podcast: [Nova DasSarma on why information security may be critical to the safe development of AI systems](https://www.google.com/url?q=https://80000hours.org/podcast/episodes/nova-dassarma-information-security-and-ai-systems/&sa=D&source=editors&ust=1692801441352923&usg=AOvVaw19XQ6i-UzXmAUc_UWSO8Y_)[,](https://www.google.com/url?q=https://80000hours.org/podcast/episodes/nova-dassarma-information-security-and-ai-systems/&sa=D&source=editors&ust=1692801441353134&usg=AOvVaw03TnyfGulnylpdVThWKvyI) 80,000 Hours, 2022


Will we run out of ML data? Evidence from projecting dataset size trends

Pablo Villalobos
Jaime Sevilla
Lennart Heim
Tamay Besiroglu
Marius Hobbhahn
Anson Ho


Winning the Tech Talent Competition