Post by Invisible Games in Elixir comments

Viewing post in Elixir comments

You are welcome, bro, but there's no way to toggle it off, because it's very important to keep the character's behaviour consistent, it tracks her attire, mood, feelings etc. and acts as her long term memory. I'll improve this system in upcoming months.

But even in its current state, the summarization takes very little time if you run ollama on Google colab with the colab ollama script included in the app.

The setup process is as simple as clicking a few buttons. Just check the short video guide in the setup instructions folder.

Have a nice day.

Invisible Games34 days ago (12 edits)

Detailed Explanation of Summarization Process:

Elixir’s memory system is designed to be lightweight enough to run on low-end hardware, but its speed relies entirely on your Ollama server's processing power.

Running Ollama locally without a dedicated GPU (at least 8GB VRAM) will cause slowdowns. That’s exactly why the app includes a setup to run the Ollama server on Google Colab, taking advantage of their T4 GPUs for incredibly fast processing. Summarizing your entire chat history into the character's long-term memory within ~ 30 seconds.

This step is essential: since fluffy/l3-8b-stheno-v3.2 (q4_k_m) has an 8k context limit, we must compress past conversations to keep the story moving without breaking the context limit.

itch.io

Viewing post in Elixir comments