DEV Community: Erik Bagdaer

Our watchdog restarted a healthy agent 24 times a day for months, and its own log said everything was fine

Erik Bagdaer — Thu, 24 Sep 2026 10:47:48 +0000

Disclosure first: I work on macyou.co, which rents Apple Silicon machines, so I have a stake in where people run things. Everything below is from our own install and from a harness anyone can run without a paid key, both open.

We have an agent that has run unattended since May: a gateway, a model behind it, a vector memory, about seventy scheduled jobs. Ask me whether it works and I would have said yes without hesitating.

Then I measured it, and found it restarts every hour, on the hour, and has been doing that for months.

The number, and why the first version of this number was wrong

I started where everyone starts, with the log. The watchdog writes a line whenever it decides something, and counting those lines gave 2,688 kickstarts over 126 days, about 21 a day. I nearly published that as "the watchdog saved the agent 2,688 times".

It would have been wrong twice.

Wrong once because log lines are not restarts. A sampler that recorded the gateway's process age every hour for three and a half days puts the real figure at exactly 24 restarts a day, while the log was writing 96 kickstart lines a day over the same window. Four lines per restart, which nobody would guess from reading the log.

Wrong twice, and this is the part worth your time, because the restarts are not rescues. Here are the intervals between restarts, reconstructed from process age:

3603.2 s mean, 0.65 s standard deviation, range 3602 to 3606, over 79 consecutive gaps

That is not a process falling over. A crashing process does not crash on a metronome with sub second jitter. Something restarts it every hour by construction, and it took measuring the process to see it, because the log describes its own decisions in a vocabulary that sounds like rescue.

What the watchdog actually checks

Reading its source settles it. The check is: has a line tagged [telegram] appeared in the log in the last thirty minutes? If not, and the Telegram API is reachable, the gateway is declared hung and kickstarted. There is even a comment above the threshold warning that idle long polling produces no log lines, so the value must be generous.

It is not generous enough. In quiet hours nothing arrives for thirty minutes, the check cannot tell a hung poller from a quiet one, and a healthy process gets killed. Of the last two hundred decisions in the log, two hundred were "Bot API reachable, gateway telegram stale, kickstart".

So the real lesson is not that agents are fragile. It is that a health check which measures traffic will restart you precisely when nothing is happening, and it will look like diligence in the log while it does it.

The fix is a heartbeat the work writes, not one the traffic writes. Have the agent touch a file when it finishes a unit of work, have the watchdog restart on that file going stale, and idleness stops being indistinguishable from death. Our kit does it that way now, which is easy to say after four months of doing it the other way.

While I was in there: what the frameworks do when things go wrong

Same harness, different question. One task for each framework: call a tool until a counter reaches three, then stop. Same mock endpoint, same step limit of twelve. The endpoint can be told to return 500, return 429, or reply with something that is not JSON. The tool can be told to fail every time. No paid keys, so all of it is reproducible for nothing.

Versions on 2026-09-20: langgraph 1.2.11, openai-agents 0.22.3, crewai 1.15.22.

What happens	LangGraph	OpenAI Agents SDK	CrewAI
Nothing wrong	done in 4 model calls	done in 4	done in 4
Tool fails every time	stops at once, 1 call spent	runs to the cap, 12 calls	13 calls, then a validation error
Endpoint returns 500 three times	raises, process exits	raises, process exits	retries and finishes, 7 calls
Endpoint returns 429 three times	raises, process exits	raises, process exits	retries and finishes, 7 calls
Reply is not valid JSON	raises	raises	recovers and finishes
Prompt tokens for identical work	428	472	802
State survives the process dying	yes, sqlite checkpointer, one line	yes, SQLiteSession, one line	partly, memory is optional and needs an embedder

Toughness inverts, which is the useful part. Against a flaky endpoint CrewAI is the survivor, because litellm retries underneath it, while the other two let the exception out and your unattended process exits. Against a tool that keeps failing it reverses: LangGraph stops after one model call, the Agents SDK spends its entire step budget discovering the tool is still broken, twelve calls per cycle for as long as the backend is down.

Neither is wrong. But combine that table with the story above and you can see the shape of the thing: a supervisor that restarts too eagerly, plus a framework that retries too eagerly, is a machine for spending money at four in the morning while every log says healthy.

One aside for private deployments: the Agents SDK ships traces to a hosted endpoint by default. In my runs it surfaced as a 401.

What to take from this

Measure the process, not its opinion of itself. Every number in the first draft of this post came from a log written by the component being judged, and every one of them was wrong in a direction that flattered the setup.

Write the heartbeat from the work. If your liveness signal is traffic, your agent will be restarted hardest at night, when nothing is wrong.

Count what a restart costs before deciding it is harmless. Ours takes seconds and nobody noticed for four months, which is exactly why nobody looked.

The lab, a watchdog that keys on progress, a proxy that caps the daily spend and writes a ledger, and launchd and systemd templates are here, MIT, four tests, no keys needed:

https://github.com/bagdaer1/agent-uptime-kit

Run python lab/drive.py and you get the table above with your own versions. If your framework behaves differently, that is the interesting case and I would like the numbers.

The part where I am not neutral

Where an agent lives matters more than what it is written in. A laptop sleeps and stops mid task. A serverless function dies at its timeout with no state. A Linux VPS is the right answer for most agents and it is what I would tell a friend to rent. A dedicated Mac earns its keep in three cases: the agent needs a local model in unified memory, it has to drive macOS or build for Apple platforms, or the token bill has passed the rent.

We rent the third thing. The full numbers, including what a four month old install looks like on disk, are at https://macyou.co/agents.

Best Hardware for Local LLMs in 2026: Mac vs NVIDIA vs AMD, One Formula for Every Row

Erik Bagdaer — Thu, 17 Sep 2026 15:39:06 +0000

Originally published on macyou.co, where the tables update as specs change. Disclosure up front: I run Macyou, we rent dedicated Apple Silicon Macs. That is exactly why every number below is computed the same way for NVIDIA, AMD and Apple, and why the section on where a Mac is the wrong purchase is the longest one.

The honest short version, and it is not the one you would expect from a company that rents Macs: if your model fits in 32 GB, buy an NVIDIA card. An RTX 5090 moves 1,792 GB/s against 546 for an M4 Max. Above 32 GB the question changes completely, because speed stops mattering when the model will not load at all, and that is where unified memory earns its price.

The market has split into three shapes

Fast and small. Discrete GPUs put GDDR7 behind a wide bus and reach 936 to 1,792 GB/s. Consumer cards stop at 24 or 32 GB, so they hold roughly 32B at Q4. Two cards double the capacity on paper but split the model across a bus, which is a different and worse thing than one pool.

Big and slow. The 128 GB unified memory boxes, the NVIDIA DGX Spark (GB10) and the AMD Strix Halo systems, use LPDDR5x. They hold a 123B model at 273 and 256 GB/s respectively, between a quarter and a seventh of a modern GPU.

Big and fast, at a price. Apple's Ultra tier is the only consumer hardware that is both. The M5 Ultra addresses 512 GB at 1,200 GB/s. NVIDIA's answer in that quadrant is the RTX PRO 6000 at 96 GB and 1,597 GB/s, a workstation part with a workstation price.

The comparison that surprises people

The DGX Spark costs $3,999 and has 273 GB/s of memory bandwidth. The M4 Pro, a two year old mid tier chip in a Mac mini, has 273 GB/s. Same number. The M4 Max has 546 GB/s at the same 128 GB, twice the Spark, and the M5 Ultra is 4.4 times it. If you are buying a 128 GB box for generation speed, make that comparison first. If you are buying it for CUDA, prompt processing or fine tuning, none of this applies and the Spark is a reasonable purchase.

Everything, at Q4_K_M

Tokens per second for generation, computed identically for every row from memory bandwidth and model size. "no" means the model does not fit. Memory is the largest configuration available.

Hardware	Memory	Bandwidth	8B	32B	70B	123B	Price
RTX 3090	24 GB	936 GB/s	105	no	no	no	~$1,300 used
RTX 4090	24 GB	1,008 GB/s	109	no	no	no	varies
RTX 5090	32 GB	1,792 GB/s	146	62	no	no	varies
RTX PRO 6000 Blackwell	96 GB	1,597 GB/s	139	57	30.1	18.2	varies
DGX Spark (GB10)	128 GB	273 GB/s	42.9	12	5.7	3.3	~$3,999
AMD Strix Halo (Ryzen AI Max+ 395)	128 GB	256 GB/s	40.7	11.3	5.4	3.1	~$2,000
Apple M5 Ultra	512 GB	1,200 GB/s	120	45.5	23.3	13.9	varies
Apple M5 Max	128 GB	614 GB/s	80	25.5	12.5	7.3	varies
Apple M5 Pro	64 GB	307 GB/s	47.3	13.4	6.4	no	varies
Apple M6	32 GB	153 GB/s	26	6.9	no	no	varies
Apple M4 Max (40-core GPU)	128 GB	546 GB/s	74	22.9	11.2	6.5	varies
Apple M4 Pro	64 GB	273 GB/s	42.9	12	5.7	no	varies
Apple M3 Ultra	512 GB	819 GB/s	97	32.9	16.4	9.7	varies

Where a Mac is the wrong purchase

Anything that fits in 24 or 32 GB. A used RTX 3090 at around $1,300 runs an 8B model at 105 tokens/sec. The base M4 does 21. The 3090 runs out of memory before 32B, but a 5090 holds it and still runs it at 62 tokens/sec, against 12 on an M4 Pro.

Fine tuning. MLX is good and improving, but the ecosystem assumes CUDA. You will hit repositories that will not run without porting, and the time spent porting is usually worth more than the hardware difference.

Long prompts with short answers. Prompt processing is compute bound, not bandwidth bound, and NVIDIA hardware is substantially better at it. The Spark processes prompts two to three times faster than a Strix Halo box despite nearly identical generation speed.

Serving many users at once. These figures are single stream. Batched serving is where GPU compute and stacks like vLLM pull away, and none of the unified memory machines, Apple included, are the right tool.

Where a Mac is the right purchase

Models between 32 and 128 GB, where you still want speed. This is the gap the GPUs leave. A 70B at Q4 will not load on a 5090; on a 128 GB box it runs at about 5.7 tokens/sec; on an M4 Max it runs at 11.2.

The very large models, at all. Above 128 GB the field empties out. Apple's Ultra tier at 256 and 512 GB is most of what is left short of datacenter hardware.

Machines that have to be quiet, small and always on. A Mac mini idles at a few watts and is silent. A 5090 draws hundreds of watts under load and needs a case and a power supply built around it.

Where the newest Apple chips land

Chip	Bandwidth	Max memory	Versus the DGX Spark
M5 Ultra	1,200 GB/s	512 GB	4.4x bandwidth
M5 Max	614 GB/s	128 GB	2.25x bandwidth
M5 Pro	307 GB/s	64 GB	1.12x bandwidth
M6	153 GB/s	32 GB	0.56x bandwidth

Read the M6 row before drawing the wrong conclusion from the others. Apple's entry desktop chip sits at 153 GB/s, below both 128 GB boxes, so the boxes are not simply slower than Macs. The crossover is the M5 Pro, the first chip in a Mac mini to edge past them, by twelve percent. What separates the classes is the top, not the entry point: Apple keeps scaling to 614 and 1,200 GB/s while the boxes stop at 273. Their argument was never speed. It is 128 GB for around $2,000, and against that specific number Apple has nothing to offer.

How these numbers were produced, and what is wrong with them

Every speed above comes from one formula applied to every row: seconds per token equals weight size divided by (bandwidth times 0.915), plus 3.9 ms of fixed overhead. The two constants were fitted by least squares to six models measured on a base M4 and reproduce all six within 4%. The calculator runs it for any model and chip.

The obvious objection, stated before someone else does: the overhead constant was measured on Apple hardware running Ollama. Applying it to CUDA and ROCm assumes a comparable per token cost, which I have not verified. It is not a rounding error: without it the formula claims a 5090 does over 300 tokens/sec on an 8B model, which nobody observes; with it, 146, which matches what people report. If you have measured numbers on a 5090 or a Spark, I would like to check the model against them. The HN thread is the place.

Everything else is a published specification with a source link on the original page. Street prices move constantly; treat the price column as an order of magnitude.

Run Local LLMs on a Mac in 2026: Which Chip Runs Which Model, and Why Bandwidth Beats Cores

Erik Bagdaer — Thu, 10 Sep 2026 14:03:09 +0000

Originally published on the Macyou blog. Disclosure up front: I run Macyou - we rent dedicated Apple Silicon Macs for AI. This post is about the hardware math, which is the same whether the Mac is on your desk or in a rack.

The short answer: any Apple Silicon Mac with 16 GB of unified memory runs 7B-14B models well, a 64 GB M4 Pro runs 70B-class models at usable speeds, and a 256 GB Mac Studio M3 Ultra runs 200B-class models that no single consumer GPU can hold. Generation speed is set almost entirely by memory bandwidth, so the chip tier matters more than the year.

Which Mac runs which model

The rule that decides everything: a model has to fit in unified memory at the quantization you choose, with a few gigabytes left for the context window and the OS. Once it fits, tokens per second scale with memory bandwidth. The M4 row below is measured (methodology and raw JSON, CC BY 4.0); the other rows are bandwidth-scaled estimates until measured columns land.

Mac	Bandwidth	Comfortable models (Q4)	What to expect
M4 Mac mini, 16 GB	120 GB/s	3B-14B	Measured: Llama 3.2 3B 46.7 tok/s, Llama 3.1 8B 21.2, Qwen 2.5 14B 11.7
M4 Mac mini, 24-32 GB	120 GB/s	14B comfortably, 32B at the edge	Same speeds as 16 GB - extra memory buys model size, not tok/s
M4 Pro Mac mini, 48-64 GB	273 GB/s	32B comfortably, 70B Q4 at 64 GB	~2x the base M4 at equal model size; 70B at ~5-6 tok/s
M4 Max Mac Studio, 128 GB	546 GB/s	70B Q8, 123B Q4	~4x the base M4; 70B Q4 in the low teens tok/s
M3 Ultra Mac Studio, 256 GB	819 GB/s	200B-class Q4, 70B FP16	Largest single-box option; 405B still needs clustering

The RAM math in one line

Weights in GB = parameters (billions) x bits per weight / 8, plus about 15% runtime overhead, plus context. At Q4_K_M (~4.85 bits) an 8B model is ~4.9 GB of weights and ~6 GB in memory; a 32B model is ~20 GB; a 70B model is ~42 GB; a 123B model is ~75 GB. Q8_0 roughly doubles those numbers, FP16 roughly quadruples them. Add 1-4 GB for a 32K context, more for 128K.

That is why 16 GB tops out at 14B, 64 GB is the 70B threshold, and 128 GB is where 100B+ dense models become practical.

Why speed is about bandwidth, not cores

Generating one token means reading every active weight once. A 4.9 GB model on a 120 GB/s bus can therefore never exceed ~24 tok/s - and we measured 21.2, about 87% of the theoretical ceiling. The same arithmetic predicts the M4 Pro (273 GB/s) at ~2x and the M4 Max (546 GB/s) at ~4x for the same model, which matches what the community consistently reports.

Two consequences:

More GPU cores on the same bandwidth barely help generation.
Mixture-of-experts models only read their active experts per token, so they run far faster than their parameter count suggests - a 35B-A3B MoE can out-run a dense 8B.

Prompt processing is the exception: it is compute-bound, and it varies about 2x between model families at equal size. We measured Qwen 2.5 7B at 1,130 prompt tok/s against Llama 3.1 8B at 587. If your workload is long-prompt/short-answer (RAG, classification), that gap matters more than generation speed.

Ollama vs LM Studio vs MLX vs llama.cpp

Ollama - the default for anything headless or scripted: one command to pull a model, a REST API on port 11434, and an OpenAI-compatible endpoint. Runs llama.cpp underneath.
LM Studio - the best GUI: model browser, chat window, and a local server that speaks the OpenAI API. Same engine class as Ollama, so same speeds.
MLX - Apple's own array framework. Fastest on some models and the natural choice for fine-tuning on a Mac; it's a Python library, not an app.
llama.cpp - the raw engine when you want every flag, the newest quant formats, or a C/C++ embed.

Five-minute setup with Ollama

brew install ollama
ollama serve &
ollama pull llama3.1:8b
ollama run llama3.1:8b "Explain unified memory in two sentences."

# OpenAI-compatible endpoint for your existing code:
curl http://localhost:11434/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model":"llama3.1:8b","messages":[{"role":"user","content":"Hi"}]}'

Point any OpenAI SDK at http://localhost:11434/v1 with a dummy API key and it works. Before pulling a bigger model, do the RAM math - a 70B build on a 16 GB machine downloads 40 GB and then fails to load.

Common mistakes

Buying GPU cores instead of memory. A 24 GB Mac with more GPU cores runs the same 8B model no faster than a 16 GB one on the same chip; the next tier of bandwidth is what changes speed.
Ignoring the context window. A model that "fits" with 1 GB to spare will swap and crawl the moment you paste a long document. Leave 2-4 GB free, more for RAG.
Running production on a laptop. Thermal throttling, sleep, and a residential uplink turn a 21 tok/s machine into an unreliable one. Anything that needs to be up 24/7 belongs on a desktop-class Mac with a real network connection - yours or rented.

When a cloud Mac makes more sense than buying

If you need a local-class model available around the clock, reachable by your team or your app, the honest comparison is not laptop vs cloud but buying a Mac mini vs renting one. Owning wins on raw hardware cost over three years; renting wins the moment you count hosting, a static IP, power, remote-access setup, and being your own on-call. That's the niche we built Macyou for - a dedicated M4 Mac mini from $79/mo with Ollama pre-installed and the OpenAI endpoint already exposed, up to a 256 GB M3 Ultra.

Per-model requirements, quant tables, and the cheapest chip that fits each model are in our model guides. Questions about a specific model or Mac - ask in the comments, I'll answer with numbers where we have them.