Posted August 15, 2022 by Lyaaaaaaaaaaaaaaa
#v1.4.0-BETA.1
Hello, today I’m releasing an important patch to the 1.4.0-BETA.
The most important part is the “low memory mode”. This is an option used on the server to reduce the RAM used by the CPU. This is an experimental feature that’s why I’m only releasing it in the BETA branch for now.
It divides by ~2 the RAM used for the AIs. Moreover, it’s not even slowing down the generation! See the result of my experiment at the end of this article.
(The game should tell you by itself to run the installer)
I ran multiples generation with gpt2-large. With my hardware running the AI I got the following results:
As you can see the generation time on my device is almost the same. The good new is we don’t have any lose of performances.
Moreover, I checked the RAM used for a few models:
While playing the server might use a few more hundred MB