itch.io

Been testing out GPT-4-x-Alpaca 13b with oobabooga UI. Fits in 11 gigs of VRAM locally, can alternatively load on CPU, has reasonable 3-5 second response times (GPU tested), exposes a local API for use, and handles prompts similarly to ChatGPT. Might be a nice option in the future, if we could somehow point the game to a different endpoint API.

GPT4All runs on a macbook air CPU. So easy a moldy potato can do it. Quality of responses would suffer somewhat. I'd be much more interested in GPT-J-6b or other open models that are used in KoboldAI, as they are much more open-ended and unbound.

I love this so much. I've been tinkering with it each day. This is what I dream of every NPC in games of the future to be like. You've got the first sparks of something really special here.