This is a cool project. It seems very important to understand how dropout affects superposition as many models we are interested in were trained with dropout.

I really like the last set of graphs (feature norm bar graphs) and I think they show that dropout reduces superposition. Your explanation that "features in superposition are exponentially more likely to be perturbed in at least one of their dimensions by dropout" seems plausible.

I'm not sure the first graph (kurtosis) is valid. You say:

"The kurtosis plot across feature sparsity for a model that does not use dropout shows that superposition starts occurring rapidly as feature sparsity increases"

Kurtosis measures the degree to which the model has a privileged basis (which dropout certainly increases). But privileged bases and superposition are different things. As shown in the Toy Models paper, you can have superposition with or without a privileged basis.