TinyLLM Trials: Can We Run LLMs Locally?
Ran TinyLLM on an M1 Mac:
- Quantized model = 4GB
- Great for offline QA tasks
- Too slow for real chat UX
Still… seeing it run without the cloud is magical.
This post is licensed under CC BY 4.0 by the author.
Ran TinyLLM on an M1 Mac:
Still… seeing it run without the cloud is magical.