Thanks for the playtesting video; it was very enlightening to see how it went and what caused the wrong kind of confusion! That helped a lot to understand the issues.
Yes, you are correct. I am actually planning to reskin the game with UI and symbol improvements, and some text additions. I will still keep the faction mechanics and strategies partially hidden (it is part of the puzzle to decipher what they do), but currently, wrong things are causing confusion, like "what happens in battles", "how to move units", and "what these are". The mystery should be at a deeper level, not at the surface.
Indeed, this game couldn't be "wordless", but the jam restriction pushed it towards elegance, where it could be "almost" wordless; by contrast, games in the 4X/strategy genres usually have very large amounts of text and symbols.
Thanks for the video!