Has anyone used AnythingLLM?

AI Machine Learning
1

AnythingLLM is a free LLM run locally on you computer. There are a variety of open-source LLM models available. AnythingLLM recommended Qwen 3 Vision 4B Instruct for my hardware but Ollama and a wide variety of other LLMs are available.

I just downloaded it and would be happy to share my experience for those interested. It is easy to download and does not need a lot of resources. So almost a no-brainer to try and see if there is a use-case. Now to see how well it works. I am interested in seeing if it can replace NotebookLM when security is an issue, and maybe save on token-use with paid models.

Any tips from those already using it would be greatly appreciated.

2 Likes
2

I use LM studio and ollama. Both let you run local llm. Ollama is headless and lm studio is a nice ui. I currently use Qwen and gemma locally. I have a mac studio, but will probably get an even better one since the cloud models still blow my local ones out of the water. Locally I am currently limited to low context windows and moe models. I think i will need at least 128gb ram for my new one. 36gb is so weak.

1 Like