I think the rotational input may have been at least partially responsible for that random thrashing, I modified it to add the transform.up of each bone and use a normalised (x,y) as two inputs and have not seen the random explosions very often since. Currently using 2100 strength (up from 1500) rather than scaling on the fly, 2100 reduces the need for additional muscles, which may also be reducing the glitches. I've tested out some other changes too which have been effective, but as you mention, these would certainly break existing creature minds. First, I changed all the sigmoids to tanh and got rid of the scaling on the output as the whole thing now works in the -1/+1 range, as opposed to weights flipping a 0-1 range. This seems to make the initial creatures a lot more active - every muscle is doing something - which usually affects the first generations pretty favourably. I've also started normalising the inputs to +/-1 eg: number of points on ground I calculate as (((float) count / joints.Count) - 0.5) * 2
At this point I will probably start building my own version from scratch soon, there's a lot of things I want to change and add, going full 3D with hinge, ball and rotator joints for a start, and things like hearts which generate a pulse as a sine wave, or touch sensors for discrete and selective input on what is on the ground, or a head bone that provides a single source for height and velocity, rather than an aggregation.... so many ideas! :)