Detailed Explanation of Summarization Process:
Elixir’s memory system is designed to be lightweight enough to run on low-end hardware, but its speed relies entirely on your Ollama server's processing power.
Running Ollama locally without a dedicated GPU (at least 8GB VRAM) will cause slowdowns. That’s exactly why the app includes a setup to run the Ollama server on Google Colab, taking advantage of their T4 GPUs for incredibly fast processing. Summarizing your entire chat history into the character's long-term memory within ~ 30 seconds.
This step is essential: since fluffy/l3-8b-stheno-v3.2 (q4_k_m) has an 8k context limit, we must compress past conversations to keep the story moving without breaking the context limit.