Evolution

Create creatures and let them evolve to see how they master various tasks. · By Keiwan

Add to collection

Simulation Community Devlog

Evolution community

Somthing seems off in the pick best creature algorithm. (BUG, maybe.)

A topic by Zypher created Oct 12, 2017 Views: 1,656 Replies: 10

Viewing posts 1 to 7

Zypher6 years ago

After some extensive testing I've seen several very strange 'best' creature choices.

I'm running batches of 15 at a time but I am gonna use 2 as an example.

2 bidpeds, one falls flat on it's face from the start, the other makes it several 'step' before falling flat on it's face about a screen widths away.

The sim picks the one that fell first and made it the LEAST distance as the best creature, from that point on all creatures fall flat on thier face and the sim is pretty much ruined. (This is in the running test btw.)

Sooo, why did it pick the one that fell first and didn't make it as far? Could it be tied into top speed, it fell faster so that might be doing it?

All in all, somthing seems way off if thats the case. Distance should be paramount in a distance test.

It could aso be some kind of random bug, though I encounter it often enough for me to think it's not totally 'random', And instead mayhaps a logistics error.

(This is also easily repeated on the climbing challenge, somtimes it picks critters that fall down the stairs at high speed vs the critters that stick the landing but arent climbing yet.)

KeiwanDeveloper6 years ago

This is the current fitness function for the running task:

fitness = Mathf.Clamp(creature.GetXPosition() / (MAX_DISTANCE * SimulationTime), 0f, 1f);

The fitness is directly proportional to the horizontal distance that the creature has travelled, the top speed has no effect.

I'm going to look into this to see if I can reproduce this the next time I have time to work on this project again.

Tossi6 years ago

I think I am experiencing the same thing. I have made a little video about it here:

Also if you wish to download the creature from that video and see for yourself how the game behaves you can get it here: https://drive.google.com/open?id=18IafZLqzAKWo3M3TC0MKXQBzBdkblKnM

KeiwanDeveloper6 years ago

(I commented this underneath the video already but I'm reposting it here so other people can read it as well. The comment might have also landed in your YouTube spam filter.)

Thanks a lot for the video! There is certainly a bug in here, but it's not really what seems to be the 'obvious' reason. In fact, I tried to solve this bug alone for a couple of days when I had time to work on the 2.0 update but I didn't manage to figure out what's going on.

That being said, I did a ton of testing and know certain things that are definitely not causing the bug. In short the problem is this: Sometimes the creatures in the "Best of" scene don't behave the same way they did during the actual simulation, even though they have the exact same brain as the best creature of that previous simulated generation.

I understand that when you (or anybody else) looks at the behaviour the first thing to think is: „The simulation clearly doesn’t pick the best one / It picks one of the bad ones.“ A lot of people say that. The real problem is that this is not the case (It would be really nice if it was because that would have been easy to fix).

This is definitely not the case though. The simulation consistently picks the creature with the highest fitness score of the generation as the best creature and stores that brain (in chromosome form, so as an encoded string). The fitness scores are also assigned properly. This is why 1) the creatures evolve at all and 2) the reason why you were seeing relatively high fitness scores (~17%) being displayed in the best creatures scene, while the „best creature“ was just falling over. It’s because the same creature (with the same brain) performed well and actually got 17% fitness during the simulation. It just for some reason doesn’t want to move when you put it in the „best creatures scene“. <- This is the really annoying bug that I’ve already tried to fix for days, but the problem is that I have no idea why this might be happening at all, so I can’t even consistently reproduce it.

As you have probably also noticed this doesn’t happen all the time. Most of the time the best creatures perform exactly the same as they did during the simulation and everything is fine. But that just makes it harder for me to find out the reason behind the bug. So if anybody can find out how to consistently reproduce this issue (instead of just randomly witnessing it) that would be of great help!

Another way to tell that the best creature is actually behaving differently (compared to when it was simulating) is that sometimes when this bug happens (not in this video) you can see that all of the creatures during the simulation actually walk - some better, some worse, but all of them walk - whereas the bugged out best creature just wiggles or falls over. This shows that this isn’t a matter of the wrong creature being picked but it’s actually a change in behaviour of the best creature (without any apparent reason).

That’s as much as I can say about this bug. The thing to take away from this is: While this bug is highly annoying and definitely something I’d love to be able to finally fix one day, it doesn’t affect the outcome of the simulation at all. It’s something that purely happens on the „best of“ scene.

Now you might ask yourself, well then why did the fitness score randomly drop from 17 to 5%?

This could have a lot of different reasons so I can’t say for sure why that is, but I’m pretty sure it’s unrelated to the above bug. Not only does the fitness drop „randomly“, it also goes back up as you can even see in this video. Additionally, the fitness neither drops exactly when the best creature bug happens for the first time, nor does it drop only when the bug occurs. Sometimes you can see these fitness fluctuations between generations without the bug even happening at all.

My current guess is that the creature brains have reached a point where you have (sort of) two groups of creatures. The ones that are doing pretty well at walking and the ones that can barely walk at all. Now when the next generation is created you can end up mating good and bad creatures which could cause the new creatures to receive one part of the bad creature’s brain and another part of the good creature’s brain. This could result in a bunch of creature that can’t walk at all (just like we’re seeing it in the video). However, the brain information of the previously good creatures is not fully gone; it’s still in the brains of the current generation. Therefore, during the next mating process you can still end up with creatures that receive both good brain parts and then „know“ how to walk again.

This is just my guess as to what might be happening here - especially since it seems to be happening in cycles and the good behaviour always keeps coming back after a certain number of generation after the fitness drops. The important thing is that this is unrelated to the bug described above and most likely not a bug itself.

That being said this bipedal creature is pretty amazing when it walks. Would you be okay with it if I record a little gif of one of them and add it to the collection at keiwando.com/evolution-gifs ? I would link back to this video of yours or another video/gif you might want to make yourself with this creature design.

Tossi6 years ago (7 edits)

Feel free to make gif of it or anything you want, I really don't mind. I thank you for your detailed explanation but there are still some things that I do not understand.

Isn't the program supposed to preserve the best creature and so the drop in fitness shouldn't happen because of that?

Isn't the simulation deterministic so the same creature with the same brain should always behave exactly the same way?

Maybe the best creature is picked but for some reason some other creature is being displayed?

Also shoudn't the worst creatures be erased from the population and so there shouldn't be such a big gap between the best and worst creature?

You said that the bug doesn't happen often but for me it happens all the time. As you can see in the video the "best of" screen never shows anything other than a creature falling on it's face.

Are you the one who wrote the code behind the neural network or are you using some libraries for that?

And one last thing. Is there any chance of you releasing the source code to public?

BTW If you so like funny creatures here is one on wheels:

Hmm, may it be that the reason for the bug is that, you are reseting physical structure of creature but not it's brain at the start of "the best" simulation? Same reason could be the cause for that fitness drop.

KeiwanDeveloper6 years ago

"Isn't the program supposed to preserve the best creature and so the drop in fitness shouldn't happen because of that?“

Yes, the best creature’s brain will be included without any modifications in the next generation if you have the setting enabled (I’ve also tested and verified this). You’re right, this should theoretically prevent a drastic fitness drop. Maybe this is also related to the behaviour change bug.

"Isn't the simulation deterministic so the same creature with the same brain should always behave exactly the same way?“

The neural network is fully deterministic so yes, the idea is that the same brain should always result in the same behaviour. Apparently though, this doesn’t always happen, for which I have no explanation. The only part of the full code that isn’t deterministic is the mutation and mating logic (the creatures that get picked are based on weighted randomization), but this has nothing to do with the brain during the simulation.

The part that is not under my control is Unity’s physics system, which could have something to do with the issue.

"Maybe the best creature is picked but for some reason some other creature is being displayed?“

The way that the best creatures logic works is the following: At the end of every generation I store the encoded brain (as a string) of the best creature in a list. When a „Best creature“ needs to be displayed (either manually or automatically), I pick the respective encoded brain, instantiate a new copy of the body, decode the brain string (chromosome) into an actual neural network and apply that to the body. (I also verified that the creatures always get the same brain that the best one of the generation had - even when the „Best creature“ doesn’t move, it has the exact same brain as when it used to be moving)

"Also shouldn't the worst creatures be erased from the population and so there shouldn't be such a big gap between the best and worst creature?“

The worst ones don’t get completely erased but they have a very small chance of passing on their genes to the next generation. It can also always happen that you mate two good creatures and then mutate a section of their brain that then renders the whole brain useless and results in a really bad creature.

"You said that the bug doesn't happen often but for me it happens all the time. As you can see in the video the "best of" screen never shows anything other than a creature falling on it's face.“

I mean it doesn’t happen that often in different simulations. Once it happens in a certain simulation then it seems to persist for the rest of that simulation without going back to normal.

"Are you the one who wrote the code behind the neural network or are you using some libraries for that?“

Yes, I wrote the network (and everything else) myself without using any libraries. The whole point of this project to me was to learn the basics of neural networks and evolutionary algorithms and have something to apply that knowledge to and playing around with.

"And one last thing. Is there any chance of you releasing the source code to public?“

I get asked that quite often and I’m actually surprised that nobody just tries to search for it, because you could easily find it in about 20 seconds. The source code is and has always been on my GitHub. https://github.com/keiwando/evolution

The project does not have an open source license, so the regular copyright stuff applies. You can obviously read through the code or play around with it for personal use though.

"Hmm, may it be that the reason for the bug is that, you are reseting physical structure of creature but not it's brain at the start of "the best" simulation? Same reason could be the cause for that fitness drop.“

The brains / neural networks have no state associated with them that would need to be reset. The networks always get encoded into strings (which is just an encoding of the weight matrices), then these strings are used for mutation and reproduction and then the new strings are decoded into networks to be applied to the new set of copies of the body.

DonaldKronos5 years ago

"The part that is not under my control is Unity’s physics system, which could have something to do with the issue."

Yes, Unity's physics engine is not deterministic. That would be the source of the behavioral not being deterministic since the behavior is dependent not only on the neural network but on the inputs to the neural network which in this case may vary from one run to another due to the physic engine not being deterministic. Personally, I like it that way, because it gives us a look at how the neural network reacts, rather than simply how it once reacted.

DonaldKronos5 years ago(+1)

According to what I gather from the discussion at https://stackoverflow.com/questions/41063608/why-are-computer-game-physics-engin... it appears that Unity would be deterministic within a given installation if time is calculated within the program without reference to real time. That is, because background processes may cause the computer used to take a different amount of time to do the same thing on different occasions, synchronizing Unity to the actual time can cause velocity calculations to give different results in otherwise identical situations.

DonaldKronos5 years ago(+1)

Here's another article on the subject, which goes into more detail. https://forum.unity.com/threads/the-truth-about-fixedupdate.231637/

Tossi6 years ago (10 edits)

Big thanks for your detailed answers, they explain a lot. Also thanks for the link to the code. I will try to have a look into it though I am not sure if I will be able to do anything with it without frying my brain.

Now, I would like to further elaborate on the reason for the bugs. Though I guess you might have already thought about what I am going to say.

A lot of my creatures suffered from this bug but I also have a creature that seems to be immune to it. That immune creature is the one on wheels, let's call it "wheeler". What is the difference between wheeler and other creatures? When the wheeler "walks", its physical state eventually returns to the arrangement that it was initially in and it keeps periodically returning to that arrangement as it "walks". On the other hand the creatures that suffer from the bug have, at the start, a very unique physical arrangement that they never ever return to as they walk. That leads me to the conclusion that the bug must be hiding somewhere in the creation/initialization of creatures.

EDIT: So I downloaded the source, installed Unity, opened the project, pressed play, got notified of 58 errors, got headache :D

EDIT2: Managed to make it work! Hmm, when loading in my old creature it doesn't work as it was working before, is the source updated with some new code that makes old brains incompatible or something?

EDIT3: Ok, I noticed things are even weirder. I launch the game inside unity, load my evolved bipedal and it doesn't want to walk at all. I build an exe, load my bipedal with that exe and the creature walks as it should. I can't figure out what's going on.

EDIT4: As far as I understand the code "thinking" happens once per frame. Frames though are generated at different intervals as FPS is something that is constantly fluctuating. Maybe this causes different behaviours at different times and thus generates those bugs?

EDIT5: After changing some "Update"s to "FixedUpdate"s it seems like simulation got a bit more consistent but I am not 100% sure just yet.

Zypher5 years ago

My gosh this blew up right after I stopped checking it. THANKS GUYS! I really apreciate the work to slove this problem. And I'm glad I could bring it to attention.

I absolutly LOVE simulators of any kind, and would LOVE to play around with this more. That "bug" was preventing me from reaching goals though.

Seems like you guys really put some work into this, but after rereading I can't tell if any soulution was found? Tossi, or Keiwan If a fix was found do you think you could possibly post it at the bottom here in case anyone else in the future runs into this?

itch.io

Evolution

Somthing seems off in the pick best creature algorithm. (BUG, maybe.)