Post

TinyLLM Trials: Can We Run LLMs Locally?

Ran TinyLLM on an M1 Mac:

  • Quantized model = 4GB
  • Great for offline QA tasks
  • Too slow for real chat UX

Still… seeing it run without the cloud is magical.

This post is licensed under CC BY 4.0 by the author.