FlatClaw, Private AI Platform
Build recipes

What we actually run, and what it costs.

Two model classes, each with a local build you can order today and the cloud node that runs the same checkpoint, side by side. Parts carry live Newegg prices. Cloud nodes carry indicative list prices, the same class on Azure, AWS, Google Cloud or our reference lane. The recipes are the configurations behind them and the lessons that came with them.

Exploded view of the FlatClaw inference workstation beside an eight-GPU server
loading prices…
Flagship class

Gemma 4 31B at FP8, 256K context

One 96 GB card locally, one H100 in the cloud. The model every FlatClaw tenant runs today.

96 GB
VRAM, one card
FP8
native on Blackwell / Hopper
256K
context served
8–12
concurrent streams
1
card, one process

Local

The one-card inference workstation

One 96 GB card holds Gemma 4 31B at FP8 with the full 256K window, exactly as the hosted tenant runs it; the rest of the box is an ordinary high-end desktop platform. Every part is sold by Newegg today at the price shown.

  • GPU
    NVIDIA RTX PRO 6000 Blackwell Max-Q, 96 GB
    …
    96 GB of GDDR7 holds the 31B at FP8 with the full 256K window on one card, exactly how the hosted tenant runs it. 300 W, a quiet blower, display outputs. The 600 W Workstation Edition is a few percent faster when Newegg has it in stock; today it does not. The passive Server Edition is wrong for a tower.
    Newegg
  • CPU
    AMD Ryzen 9 9900X, 12 cores
    …
    Feeds the card over PCIe 5.0 and runs the OS, SGLang, the gateway and ingestion. Twelve cores is plenty; the 9950X only pays off for CPU-heavy ingestion.
    Newegg
  • Mainboard
    ASUS ProArt X870E-Creator WiFi
    …
    Two PCIe 5.0 x16 slots (x8/x8) so a second card can follow later, 10 GbE + 2.5 GbE, four M.2, USB4: workstation-grade AM5.
    Newegg
  • Memory
    Corsair Vengeance 64 GB (2 × 32 GB) DDR5-6000 CL30
    …
    Two DIMMs keep AM5 at full speed; 64 GB carries the OS, Docker, the gateway and ingestion with headroom, because the model lives in VRAM. DDR5 has roughly quadrupled in price this year; 96 GB kits are not worth it for this box.
    Newegg
  • Power
    Corsair HX1200i (2025), 1200 W, 80+ Platinum, ATX 3.1
    …
    A native 12V-2×6 lead for the Pro 6000 and wide margin over the ~500 W peak of card plus platform; the same unit carries the 600 W Workstation Edition if you swap in later.
    Newegg
  • CPU cooler
    Corsair Nautilus II 360 RS, 360 mm AIO
    …
    Holds the 120 W 9900X while the GPU dumps heat into the case; fits the North XL's top mount.
    Newegg
  • Case
    Fractal Design North XL, Mesh, charcoal and walnut
    …
    A tower that looks like furniture: 413 mm of GPU room, a 360 mm top radiator, a mesh side feeding the blower. Mesh, not glass.
    Newegg
  • Storage
    Samsung 9100 PRO 4 TB, NVMe Gen5
    …
    OS plus the model library (~1 TB system, ~3 TB weights); the board takes Gen5 natively.
    Newegg
  • All-in · 8 parts · 0 live, 8 unpriced
    $0
# Serve
python -m sglang.launch_server --model-path google/gemma-4-31b-it --quantization fp8 --tp 1 --context-length 262144 --tool-call-parser gemma4 --reasoning-parser gemma4 --port 30000 --mem-fraction-static 0.90
  • ▸Peak draw is the card's 300 W plus about 200 W of platform: ~500 W, about 4 A at 120 V on an ordinary circuit. Quiet enough for a desk.
  • ▸Ubuntu 24.04 Server with NVIDIA R570+ / CUDA 12.8+ (sm_120). The card has display outputs for a monitor when you want one and runs headless otherwise. Serve from containers and hold driver updates before demo days.
  • ▸One SGLang process on the one card behind one OpenAI-compatible endpoint, the same launch line as the hosted tenant. The model never reloads.
  • ▸Run the OpenClaw gateway and the Portal on the same box for a fully local setup, or point a separate control machine at its endpoint; both work.

Cloud

The same checkpoint in a tenancy on the cloud the customer already runs: Azure, AWS, Google Cloud, or our reference lane.

Recommended
GPU node
1 × H100 80 GB · 26 vCPU · 234 GB RAM on the reference plan
$2,000/mo
$2.74/hr warm
AzureNC H100 v5AWSp5Google CloudA3Northflanknf-gpu-h100-80-1g
  • About $2,000 a month held warm 24/7 including the bundled CPU and RAM.
  • Weights are fetched from the weights volume over the tenancy's own network at boot; nothing leaves the tenancy.
Control plane
4 vCPU · 16 GB · the Portal plus one OpenClaw gateway per user
$144/mo
~$0.20/hr
AzureD4s v5AWSm6i.xlargeGoogle Cloudn2-standard-4Northflanknf-compute-400-16
  • Two or three users at 0.6–1.2 GB each with room to spare. Every tenant has this service next to its GPU, on any cloud's small VM or container plan.
Weights volume
200 GB of storage plus a small CPU pod that serves it over HTTP
$30/mo
~$0.04/hr
AzureBlob + blobfuse2AWSS3Google CloudGCSNorthflankvolume + CPU pod
  • Staged once; the GPU node fetches the weights at boot because GPU plans rarely mount a volume directly.
All-in
About $2,200 a month per tenant held warm: the GPU node, the control plane and the weights volume, on whichever cloud the customer already runs.

Indicative list prices held warm 24/7 on our reference lane; the same classes on Azure, AWS and Google Cloud land in the same band on reserved terms, and bare metal amortizes lower.

  • ▸Flat per-tenant rate, not per-token metering: the GPU line dominates and everything else is under $200.
  • ▸Capacity is peak concurrent sessions, not head count: one H100 sustains 8–12 concurrent streaming chats at FP8 with first tokens in 1–2 s.
Download this sheet as a PDF →
Frontier class

GLM-5.2 at FP8 on eight 96 GB cards, or eight B200s in the cloud

Only about 40B parameters are active per token, so the weights have to be held, not computed. The FP8 checkpoint is 744 GB: eight 96 GB cards in a 4U server locally, eight B200s in the cloud. A two-card workstation runs a 4-bit copy.

744B
total params, MoE
40B
active per token
768 GB
VRAM, eight cards
FP8
the checkpoint, unquantized
1M
context tokens

Local

The FP8 server: eight cards in 4U (chosen)

Eight 96 GB Blackwell cards in a 4U chassis hold the 744 GB FP8 checkpoint in VRAM, the same way the cloud node does and with the same launch line. Six cards (576 GB) cannot hold it; eight is the minimum, and the remaining ~24 GB carries an FP8 KV cache for a handful of long conversations at a time. Every part is sold by Newegg today.

  • Chassis
    ASUS ESC8000A-E13-32W, 4U 8-GPU server barebone (dual EPYC 9005)
    …
    Eight double-width GPU bays with the airflow and redundant 3 kW supplies to match; board, backplane and power are included. In stock at Newegg; the E12 generation is not.
    Newegg
  • CPU × 2
    AMD EPYC 9355, 32 cores (5th Gen, SP5) × 2
    …
    Two sockets for 24 memory channels and 256 PCIe 5.0 lanes feeding eight x16 cards; the GPUs do the work. The 48- and 64-core parts are out of stock and unnecessary here.
    Newegg
  • Memory
    Kingston Server Premier 64 GB DDR5-5600 ECC RDIMM × 12 = 768 GB × 12
    …
    768 GB matches the VRAM, so the whole checkpoint can stage through system memory; one DIMM on six of the twelve channels per socket. Memory is where the money went this year.
    Newegg
  • GPU × 8
    NVIDIA RTX PRO 6000 Blackwell Max-Q, 96 GB × 8
    …
    8 × 96 GB = 768 GB, FP8 weights in VRAM. The 300 W blower was made for dense boxes; the passive Server Edition is on Newegg only through a marketplace seller at $30,000, so the Max-Q it is.
    Newegg
  • Storage
    Samsung 9100 PRO 4 TB, NVMe Gen5 × 4 × 4
    …
    The FP8 checkpoint alone is 744 GB; four drives hold it plus two quant tiers and the BF16 shards for conversion.
    Newegg
  • All-in · 5 parts · 0 live, 5 unpriced
    $0
# Launch
python -m sglang.launch_server --model-path zai-org/GLM-5.2 --tp 8 --quantization fp8 --enable-moe-ep --kv-cache-dtype fp8_e5m2 --context-length 131072 --port 8000
  • ▸8 × 300 W plus two 280 W CPUs and the platform: about 3.2 kW, a dedicated 240 V circuit and a room that can shed the heat.
  • ▸Same shape as the cloud node, which is the point: the launch line is identical, only the owner changes. Per-stream speed follows memory bandwidth, GDDR7 at 1.8 TB/s per card against the B200's 8 TB/s, so the cloud node decodes faster; this box is FP8 parity you own.
  • ▸Six cards (576 GB) cannot hold the FP8 checkpoint at all, and eight leave about 24 GB for KV. For FP8 at full concurrency and the 1M window, the cloud node's 1,536 GB is the shape.

Cloud

The same checkpoint in a tenancy on the cloud the customer already runs. The fast node and the economy node are both eight-GPU machines; the control plane is twice the flagship tenant's.

Recommended
GPU node, fast
8 × B200 180 GB · 1,536 GB
$34,300/mo
$47/hr warm
AzureND GB200 v6AWSp6Google CloudA4Northflanknf-gpu-b200-180-8g
  • HBM3e at about 8 TB/s per card, roughly twice the H200, so decode per stream is about twice as fast; FP8 weights and an FP8 KV cache with room for the full 1M-token window.
  • SGLang --tp 8 --enable-moe-ep; a 1.5 TB or larger weights volume, fetched by the node at boot (744 GB).
GPU node, economy
8 × H200 141 GB · 1,128 GB
$18,300/mo
$25/hr warm
Spot: $14.6/hr, about $10,700/mo when the capacity is there.
AzureND H200 v5AWSp5eGoogle CloudA3 UltraNorthflanknf-gpu-h200-141-8g
  • The same checkpoint and launch line at about half the decode speed; context up to 256K with KV room.
Control plane
8 vCPU · 32 GB · the Portal, one OpenClaw gateway per user and the failover router
$288/mo
~$0.40/hr
AzureD8s v5AWSm6i.2xlargeGoogle Cloudn2-standard-8Northflanknf-compute-800-32
  • Twice the flagship tenant's control plane: the frontier class carries more users and longer sessions, and the router that fails over between the fast node, the economy node and a quantized fallback runs here too. Room for eight to ten users at 1–1.2 GB each.
Weights volume
1.5 TB or larger, plus a small CPU pod that serves it
$60/mo
~$0.08/hr
AzureBlob + blobfuse2AWSS3Google CloudGCSNorthflankvolume + CPU pod
  • The GPU node fetches the checkpoint at boot; an Ampere node (A100) has no FP8 and would need a quantized copy.
All-in
About $34,700 a month held warm on the B200 node and about $18,700 on the H200 node, control plane and weights volume included; spot on H200 about $10,900. Schedule the frontier node for live load and keep a smaller class as the floor.

Indicative list prices held warm 24/7 on our reference lane; the same classes on Azure, AWS and Google Cloud land in the same band on reserved terms, and bare metal amortizes lower.

  • ▸Cost reality: this is a different tier. The B200 node is about seventeen times the 31B tenant held warm, so the frontier class is scheduled for live load and scaled to zero between sessions.
  • ▸Per GPU-hour at reference list prices: H100 $2.74 · RTX PRO 6000 96 GB $3.00 · H200 ~$3.14 · B200 ~$5.87.
  • ▸Routing is unchanged: OpenClaw points at one endpoint; a LiteLLM or vLLM router in the same tenancy adds failover to the economy node, a quantized fallback or a second region.

The quant ladder — what fits where

TierFootprintFits
BF16 full precision~1,500 GBcloud only, multi-node
FP8 (the prod checkpoint)~744 GBeight 96 GB cards locally (short context) · the cloud node with KV room
8-bit810 GBGPU server or cloud
5-bit570 GB512 GB RAM + 192 GB VRAM, tight
4-bit (UD-Q4_K_XL), the workstation372–475 GBthe 512 GB two-card box, near-lossless
3-bit290–360 GBa 384 GB box
2-bit (UD-IQ2_M)239 GBa 256 GB box; ~82% top-1 at 84% smaller
1-bit223 GBtight budgets; the biggest quality hit
Download this sheet as a PDF →