I'm very impressed by how well optimized this demo is and how responsive the LLM integration can be, it's a bit of a technical hurdle with how the api is handled and i don't see a "green confirmation" over the url like the instructions claim, but it works. I have it hooked up to TabbyAPI running an EXL2 of a 12b, on just 16gb of vram, and it runs like a dream. I am really looking forward to full release.
psychxklx
1
Posts
A member registered Sep 19, 2020