Interesting. I use ApplyModifiedProperties in my own rich text editor and have no performance issues whatsoever even with very large amounts of text.
I built this system a while ago (and in a hurry), but I recall running into some major issue - not sure if it was a performance issue, but looking at my code I see I have delta checks and dirty flags for the text and comments saying to only call ApplyModifiedProperties when necessary.
I took a brief glance at your editor and it seems like you might be saving a lot of different properties in OnInspectorGUI and I don't see anything that checks if calling ApplyModifiedProperties is actually necessary (though I think it should just return if there's no changes, but according to your testing it seems like just the call itself is the issue). This would mean you're calling ApplyModifiedProperties every frame while the editor is visible (unless I'm mistaken).
I did try adding a HasModifiedProperties check before your ApplyModifiedProperties calls, but it didn't help - maybe it's just the number of properties you're trying to save every frame (or maybe HasModifiedProperties has the same issue). I can confirm, however that commenting those calls out solves the performance problem.