Hey again,
It's been really inspiring and educational for me going through your code, thank you for making it available. I have a couple more things to add after looking at it.
You're using averaged bone rotation.z as one of the inputs, but despite how it appears in the inspector, transform.rotation is a Quaternion which gives you some issues... Unity will automatically convert the rotation (0,0,1,0) to (1,0,0,0) and this rare, sudden spike just adds to the internal weirdness of quaternion, eg. z is exactly the same at 135 and -135 degrees, and flips negative at around 120 degrees - so the input is very inconsistent. The fact that creatures can overcome this garbled input is testament to how great neural networks really are.
A simple way to sanitise rotation is using two separate inputs - ie averaging the 2D vectors along the length of each bone, so instead of using rotation.z, if you use the local transform.up you'll get a couple of nice normalised sine waves for the "upness" and "rightness" of them, which should average out well and give your creatures far more meaningful input overall.
The other thing is muscle scaling - for me designing creatures was a bit of a struggle, I want to add rigid parts, eg triangles for body and feet, but the extra bone and joint weight means it often struggles to move at all, so I end up adding more muscles to nearby bones to compensate, which make the NN more complicated and slower than it needs to be. One solution would be to scale the force of each muscle by creatureWeight / totalMuscles.
Anyway thanks again for making this and for posting the code. :D