Everything you describes is exactly contrary to how it's meant to be used. Yes you can, but that's not how you're supposed to. If it was, then all the Unity tutorials, documentation, and features would be working towards that kind of workflow and they absolutely do not. The whole point of having an Editor is so that you don't have to do what you're describing.
I have to wonder, if you're going to such an extreme as to not even use GameObjects, why you would use Unity at all and not a simpler engine, or write your own?