Introduction

In Generative AI systems, most models are accessed through APIs. However, running models locally is becoming increasingly important for privacy, cost efficiency, and offline usage.

To understand how local inference works, I ran a Large Language Model (LLM) on my system using llama.cpp.

In this session, I focused …