itch.io

This is a nicely written proposal that was interesting to read. I like the emphasis on prevention rather than reaction. It does not seem to me that it directly targets "pausing/slowing down the progress of AGI" though. It is true that implementing this proposal would indirectly slow down the progress due to administrative overhead, but it doesn't attempt to address issue directly. In fact, slowing down is even mentioned in "considerations" almost as a downside.

Interesting proposal for sure, I wrote down a few comments when reading, mainly for the "Stakeholder Management (Differential Incentives)" part:

1. The second paragraph of "AGI Companies" seems to focus mostly on US companies.
2. Something to consider for the measures proposed for academia: this might drive academia away from the topic altogether, which does not seem ideal. Competition for funding is already taking a toll on researchers' time and well-being, adding another layer of bureaucracy (even if the goal is a good one) could be the last straw. Corporations have financial incentives to follow regulations, but academic researchers do not (apart from earning a living, which they could do more lucratively elsewhere anyway).
3. For the "non-research" part,  at which point does of the supply chain would one draw the line? For example, NVIDIA manufactures their chips through TSMC, TSMC uses lithography machines made by ASML, and ASML has 5000 suppliers according to their website (https://www.asml.com/en/company/sustainability/responsible-supply-chain).

The proposal is nicely written and the concept of generating potential future scenarios using AI (and attaching costs/fines to them) sounds interesting to me in principle. However, it seems to me that the proof of concept shows that this cannot work (yet, and we might be far away from it working). As the authors mention (to their credit), the generated numbers seem arbitrary (I did not exactly follow why the relative numbers are interesting if the absolute numbers are arbitrary) and the text of the post-mortem analysis in Figure 4 is quite generic and contains little information (i.e., it could apply to almost any input).