itch.io

Gday'

I believe that the cause of this softlocking is that the app is waiting for a response from the server. I should certainly add a timeout, but the real issue here is that the models are massive and slow. I tried this 2GB model https://huggingface.co/lmg-anon/vntl-gemma2-2b-gguf and while it works just fine for one or two text lines, it quickly becomes very slow and sometimes fails to return English output. I only have an old 980Ti to play with, so it is pretty much unrealistic for me to self-host LLM's.

What one can do in this case is to add a max_tokens value so that you can never select so much text that the server becomes unresponsive, but instead returns before the entire translation is finished. (The key max_tokens may be called something else for different setups)

Side note;
I noticed that numbers are currently treated as strings in the JSON body. I have fixed it, but the full 0.4.9 version won't be released yet for a few hours. If you want to try it out before that, you'll have to go to user/appdata/roaming/GameTranslate/config/configuration.toml and remove the max_tokens quotes around the number. I'd recommend setting it to 128 just to make sure it works.