I'll tweak how often you can have a scriptorium. Did you stack them and sold them all at once?
Rather than tweaking how often it shows up (because it’s one of the more fun events to find), consider changing how scrolls stack to make them less exponential.
Right now, if you have six scrolls of -0.1s, and sell them, you’ll get -0.1s, -0.2s, -0.4s, -0.8s, -1.6s, -3.2s, for a total of -6.3s. That’s fun, but pretty broken. ;)
As much fun as it was to play with, and as much as I hate to suggest nerfing it, you might consider instead capping it, so that using a scroll of cooldown when you have other scrolls of cooldown will upgrade the other scrolls as if they were bumped uncommon->epic->legendary, with a hard-cap at legendary (so a legendary scroll doesn’t get any better). That would provide some benefit to saving them up (which compensates for having to hold off on using them and hope for more scrolls, which requires luck), but would keep it from being exponential.
I didn't stack All of them, I just kinda used one when I needed a little power bump for the next level. Then as I got towards like level 8 I sold them all.
I'm not a game balancing expert, so my suggestion on a fix wouldn't really be worth much. But at least at first glance, not upgrading scrolls with scrolls as you mentioned below seems like it would work at least. Although the guy below's suggestion sounds pretty interesting too, pending some number crunching investigation probably.