irate944

joined 6 months ago
[–] irate944@piefed.social 5 points 1 month ago

That is a bit too much for your hardware, even the Q4_0. You needed a smaller version (26B likely would suit you better. It would be faster and is a MoE)

[–] irate944@piefed.social 19 points 1 month ago

Creeps love an always-on camera that you can wear on your face so that it's not obvious that you're filming?

Who could've seen that coming

[–] irate944@piefed.social 11 points 1 month ago (2 children)

How much RAM do you have and which version of the model did you run?

Local LLMs can be just as fast as long your device clears the requirements. If you noticed a huge difference, there’s a really good chance that you tried to use a model that requires more RAM than you have

[–] irate944@piefed.social 110 points 2 months ago* (last edited 2 months ago) (3 children)

It’s not an hallucination, it just found search results from a better dimension