Local LLM Fine-Tuning on a Mac with MLX and LoRA: A Guide Without a Cloud GPU

A small language model can be fine-tuned directly on a Mac with an Apple Silicon chip, without renting a cloud GPU. The MLX framework and LoRA technique help make this possible. However, the task promises more than is fair: MLX doesn’t “crush” PyTorch, and fine-tuning won’t reliably teach the model to “never suggest React.” This guide shows what you can realistically achieve, how to prepare data from your projects, and how to start training.

What’s Verified and What Isn’t

• “MLX crushes PyTorch”: an unsubstantiated claim. Comparisons depend on the model, chip, and settings, and I haven’t found any clear, universally valid evidence to support it. You can compare them yourself on your own Mac.
• Copying between RAM and VRAM: On Apple Silicon, memory is shared between the CPU and GPU, so “classic” copying to separate graphics memory doesn’t exist there in that sense. Even PyTorch with the MPS backend doesn’t start with this disadvantage. The advantage of MLX is that it was designed from the ground up for this memory model and works with arrays without transfers between devices.
• “End of 2026” trend: I haven’t found any evidence of a massive shift by developers toward local tuning.
• Chips: MLX works on all Apple Silicon chips (M1 and newer). It does not work on Macs with Intel processors.

What Is Unified Memory?

On a typical PC, the processor and graphics card have separate memory. Data must be transferred between them. With Apple Silicon chips, both the CPU and GPU have access to the same memory. For model fine-tuning, this means that the limit is the machine’s total memory, not the graphics card’s individual capacity. For smaller models (in the billions of parameters), this is sufficient for reasonable experiments. According to available guides, the recommended minimum is 16 GB, though 32 GB is more comfortable.

What is LoRA?

With full fine-tuning, you would adjust all of the model’s weights. LoRA freezes the original weights and adds small trainable “adapters” to selected layers. Only a fraction of the parameters (often less than one percent) are trained, so both memory requirements and the size of the resulting model are significantly reduced. An adapter is a small file that can be loaded alongside the base model or merged with it. The QLoRA variant also uses a quantized (compressed) base model, which saves memory.

What optimization can and cannot do

• It can: adjust the style, output format, and common patterns (such as naming conventions, file structure, and a preference for clean DOM code).
• It cannot reliably: add new knowledge or guarantee that the model will never suggest React. A smaller model may still hallucinate.
• A cheaper alternative: if the goal is simply to ensure the model doesn’t use a framework, a system prompt or a file with project rules is often sufficient. Fine-tuning makes sense if you want the style baked into the model or if you want to run everything locally.

Step 1: Installation

python3 -m venv .venv
source .venv/bin/activate
pip install -U mlx-lm
# ak trénovanie nahlási chýbajúce závislosti:
pip install -U "mlx-lm[train]"

Step 2: Dataset from your own projects

The most important part is data quality. The model learns from examples, so you need “task and code” pairs. The script below will scan the folder containing your projects, skip files that look like secrets, and create files train.jsonl and valid.jsonl in chat message format. The prompt is derived from the opening comment in the file or from its filename. This description is only rough, so the result won’t be perfect. The better you write the prompts, the better the model you’ll get.

import json
import random
import re
from pathlib import Path

SRC = Path("~/projekty").expanduser()
OUT = Path("data")
OUT.mkdir(exist_ok=True)

EXT = {".js", ".html", ".css"}
SKIP_DIRS = {"node_modules", ".git", "dist", "build", "vendor"}
MIN_CHARS, MAX_CHARS = 200, 6000
SYSTEM = ("Si programátor, ktorý píše čistý Vanilla JavaScript a Tailwind CSS "
          "bez frameworkov. Odpovedaj iba kódom.")

SECRET = re.compile(
    r"(api[_-]?key|secret|token|password)s*[:=]s*['"][^'"]+['"]", re.I
)


def describe(path: Path, code: str) -> str:
    m = re.match(r"s*(?:<!--|/*|//)s*(.+?)s*(?:-->|*/)?s*(?:n|$)", code)
    return m.group(1) if m else f"súbor {path.name}"


rows = []
for p in SRC.rglob("*"):
    if not p.is_file() or p.suffix not in EXT:
        continue
    if any(part in SKIP_DIRS for part in p.parts):
        continue
    code = p.read_text(encoding="utf-8", errors="ignore")
    if not (MIN_CHARS < len(code) < MAX_CHARS) or SECRET.search(code):
        continue
    rows.append({"messages": [
        {"role": "system", "content": SYSTEM},
        {"role": "user", "content": f"Napíš kód: {describe(p, code)}"},
        {"role": "assistant", "content": code},
    ]})

random.seed(42)
random.shuffle(rows)
n_valid = max(1, len(rows) // 10)
valid, train = rows[:n_valid], rows[n_valid:]

for name, part in (("train", train), ("valid", valid)):
    with open(OUT / f"{name}.jsonl", "w", encoding="utf-8") as f:
        for r in part:
            f.write(json.dumps(r, ensure_ascii=False) + "n")

print(f"Trénovacích: {len(train)}, validačných: {len(valid)}")

Be careful with the data: use only code that you own or have the rights to. The script filters out only obvious secrets, so be sure to review the data yourself before training the model. A few high-quality examples are better than a lot of chaotic ones.

Step 3: Training

mlx_lm.lora 
  --model mlx-community/Llama-3.2-3B-Instruct-4bit 
  --train 
  --data ./data 
  --iters 600 
  --learning-rate 1e-4 
  --mask-prompt

The option --mask-prompt is set so that the error is calculated only from the assistant’s response, not from the prompt. The model in the example is for demonstration purposes only. Check the license of the selected model and whether it is available in MLX format. Memory requirements and processing time depend on the model size and the length of the examples. If you run out of memory, shorten the examples or reduce the model size. Adapters are saved to the folder adapters.

Step 4: Try it out and compare

from mlx_lm import load, generate

MODEL = "mlx-community/Llama-3.2-3B-Instruct-4bit"

def ask(adapter_path, task):
    model, tokenizer = load(MODEL, adapter_path=adapter_path)
    messages = [
        {"role": "system", "content": "Si programátor, ktorý píše čistý Vanilla JavaScript a Tailwind CSS bez frameworkov. Odpovedaj iba kódom."},
        {"role": "user", "content": task},
    ]
    prompt = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    return generate(model, tokenizer, prompt=prompt, max_tokens=500)

task = "Napíš kód: tlačidlo, ktoré prepína tmavý režim"
print("--- základný model ---")
print(ask(None, task))
print("--- s adaptérom ---")
print(ask("adapters", task))

Test it on tasks that the model didn’t see during training. Check whether it adheres to the required style and whether it still provides the framework. If you want to get rid of a separate adapter, you can merge it into the model: mlx_lm.fuse --model … --adapter-path adapters --save-path fused-model.

Common Mistakes

• Overfitting: The model memorizes your data and stops generalizing. Monitor the error on the validation data and avoid running too many unnecessary iterations.
• Insufficient or chaotic data: the result will be just as chaotic.
• Forgotten system prompt: When using the model, use the same message format as during training.
• Expecting a miracle: a 3-billion-parameter model won’t compete with large models—you’ll just fine-tune it to your style.

What to Take Away From This

Local fine-tuning of a small model on a Mac is accessible and instructive. The real-world input is the dataset, not the framework. When it comes to code style preferences, try the system prompt first, and save the fine-tuning for when you want a local model with your style built right in.