Judge feedback

Judge feedback is anonymous.

• Good to see someone exploring ICE in the context of sandwiching! Quick comment on the Future Chatbot Goal - "Then a group of sustainability experts will determine company sustainability based on their morals with their expert knowledge to improve the model" -- in the classic sandwiching paradigm the expert knowledge is used to determine whether the a, but not to improve the model, as that would be 'cheating' -- see for example John Wentworth's comment at the end of https://www.alignmentforum.org/posts/PZtsoaoSLpKjjbMqM/the-case-for-aligning-narrowly-superhuman-models : "...(Though I also expect that a lot of people will read the description of the sandwich problem and fail to understand the requirement which makes it interesting - namely, that the experts be completely and totally absent from the training process, and in particular no data from experts should be involved in the training process.) If I thought I had a generalizable method capable of solving the sandwich problem, it would probably be the highest-priority thing on my agenda." and Ajeya's reply: "Thanks for the comment! Just want to explicitly pull out and endorse this part: the experts be completely and totally absent from the training process, and in particular no data from the experts should be involved in the training process I should have emphasized that more in the original post as a major goal. I think you might be right that it will be hard to solve the "sandwich" problem without conceptual progress, but I also think that attempts to solve the sandwich problem could directly spur that progress (not just reveal the need for it, but also take steps toward finding actual algorithms in the course of doing one of the sandwich problems)..."