PhantaField PFG-1 Sophon Whitepaper

A. Platform

Sophon uses the following physical stack:

Total stack height: ~22 µm above the Si die (64 tiers × 0.35 µm/tier). The 90 nm-pitch MIV grid provides 1.23 × 10⁸ slots/mm² available inter-tier connections; the design populates only ~5.5 × 10⁵/mm², leaving > 99% MIV headroom.

Tiers are not split within a single layer; instead the 64-tier stack interleaves dedicated logic and memory tiers in an A/B/A/B… repeating pattern. Two adjacent tiers form one logic-plus-memory doublet; the stack contains 32 such doublets:

• Logic tiers (32 × 750 mm² = 24,000 mm² total MAC area): 2D-TMD CMOS MAC array on odd-indexed tiers — MoS₂ n-FETs for NMOS, WSe₂ p-FETs for PMOS. Density 0.14 TFLOPS FP8/mm² (0.07 TFLOPS BF16/mm²). Clocked at 1.2 GHz, V_dd = 0.6 V.
• Memory tiers (32 × 750 mm² = 24,000 mm² total memory area): 2T0C 2D-TMD DRAM on even-indexed tiers, fabricated at the Metal-3 BEOL of that tier. Each memory tier sits directly above its paired logic tier; vertical Monolithic Inter-tier Vias (MIVs) on a sub-100 nm pitch carry bit-line/word-line/sense signals straight up from the logic MAC array into the cells, giving every MAC its own private vertical port to local weights with zero NoC traffic. This interleaved arrangement preserves the same total area and capacity as a hypothetical in-tier 50/50 split, while doubling the per-tier MAC routing area and shortening MAC-to-cell signal paths to a single tier-pitch of 0.35 µm.

Why 2D TMD? TMD CMOS (MoS₂ / WSe₂) is the only transistor technology that simultaneously offers: (1) BEOL-compatible growth at ≤ 450 °C [6]; (2) atomic-scale channel thickness eliminating short-channel leakage [1][2]; (3) electron mobility ≥ 120 cm²/V·s [4]; and (4) intrinsic radiation hardness (no buried-oxide trap volume). Critically, the TMD off-current density J_off ≈ 10⁻¹⁵ A/µm (1 fA/µm) at 28 nm — i.e. ≈ 0.5 fA for a 0.5 µm-wide cell transistor, roughly 4 orders of magnitude lower than Si NMOS at equivalent gate length [2][3] — is what enables a 2T0C cell to retain data for seconds without any storage capacitor [8][9], keeping the cell area at 8 F² rather than the ~20 F² needed for a conventional 1T1C DRAM.

B. PFG-1 "Sophon" — 2T0C DRAM die

Sophon places a 2T0C 2D-TMD gain-cell DRAM (8 F², 1 bit/cell) at the Metal-3 BEOL of each memory tier. The cell structure is shown in Figure 2 and consists of:

• Write Transistor (WT): a TMD nFET gated by the Write Word-Line (WWL), which charges the storage node to V_dd or discharges it to GND.
• Read Transistor (RT): a TMD nFET whose gate is the storage node; its drain current indicates the stored bit.
• Storage node: the parasitic gate capacitance of RT (~2.5 fF at 28 nm TMD) plus the junction capacitance of WT's drain (~0.5 fF). No explicit Metal-Insulator-Metal (MIM) or trench capacitor — that is the "0C" in 2T0C.

The TMD off-current density of 1 fA/µm (I_off ≈ 0.5 fA for a 0.5 µm cell transistor) gives retention τ = C·V_dd / (2·I_off) = 1.8 s at 25 °C [8][9] — see Eq. 3 and Figure 3 for the retention curve. Sophon refreshes every 1.0 s (1.8× margin), consuming only ≈ 0.08 W for the full 330 GB die (Eq. 4). Retention derates ≈ 2× per 10 °C; above 60 °C junction temperature, on-die thermal sensors shorten the refresh interval (≈ 159 ms at 60 °C, ≈ 28 ms at 85 °C), with refresh power staying below ~ 4 W even in the hot corner.

Because the storage node is writable on every cycle, Sophon supports in-place BF16 gradient accumulation with unlimited endurance — exactly what training requires — while the same array, read-only, serves the inference decode loop. The die loads a model once and either serves it (inference) or updates it in place (training); a powered-off die reloads its weights from off-die Non-Volatile Memory express (NVMe) at boot (§10.2).

A.1 PFG-1 "Sophon" — 2T0C 2D-TMD gain-cell DRAM weight/gradient cell

The 2T0C gain cell consists of two 2D-TMD transistors and zero explicit storage capacitors [8][9][10]. It exploits the anomalously low off-current of TMD field-effect transistors — a width-normalized density of J_off = 10⁻¹⁵ A/µm (1 fA/µm) at 28 nm [2][3], i.e. only ≈ 0.5 fA for a 0.5 µm-wide Read Transistor — to retain charge on the gate parasitic of the Read Transistor (RT) for seconds without a Metal-Insulator-Metal (MIM) or trench capacitor.

Cell structure:

• Write Transistor (WT): TMD nFET, gate driven by the Write Word-Line (WWL). Drives the storage node to V_dd (write "1") or GND (write "0").
• Read Transistor (RT): TMD nFET, gate = storage node, source grounded, drain = Read Bit-Line (RBL). When storage = V_dd, RT conducts; when storage = 0, RT is off. Binary current sense.
• Storage node: parasitic C_gs of RT (~ 2.5 fF) + C_junction of WT drain (~ 0.5 fF) = ~ 3.0 fF total. No explicit capacitor — that is the "0C" in 2T0C.

Retention physics (Eq. 3, derived from [8]): τ = C_node · V_dd / (2 · I_off). At C_node = 3.0 fF, V_dd = 0.6 V, and I_off = J_off · W_RT = 1 fA/µm × 0.5 µm = 0.5 fA at 25 °C, τ = 1.8 s. Sophon refreshes every 1.0 s (1.8× margin). Retention derates ≈ 2× per 10 °C; above 60 °C junction temperature, on-die thermal sensors shorten the refresh interval (≈ 159 ms at 60 °C, ≈ 28 ms at 85 °C).

Why a capacitor-less gain cell? A conventional 1T1C DRAM needs a ~ 20 F² trench/MIM capacitor that is incompatible with low-temperature BEOL M3D integration. The 2T0C cell stores charge on the Read Transistor's own gate parasitic, so it is built entirely with the same TMD transistors used in the MAC array — no separate capacitor module, no third-party Intellectual Property (IP) license — and the multi-second retention enabled by the 1 fA/µm off-current makes refresh power negligible (≈ 0.08 W, Eq. 4).

A.2 Per-doublet and per-die capacity

The stack contains 32 doublets (one logic tier + one memory tier per doublet). Each doublet contributes one logic-tier's MAC area and one memory-tier's storage area; the total active MAC area and memory area are therefore identical to a hypothetical 64-tier in-tier-split presentation, but routing is denser because each logic tier no longer competes for footprint with its memory bank.

Sophon holds 330 GB. For training, an 80B-parameter BF16 model (160 GB) plus first-order optimizer state (160 GB for SGD-momentum or Lion) = 320 GB, leaving 10 GB for gradient-checkpointed activations (Section 8.B.2). For inference, an 80B BF16 model (160 GB) leaves 170 GB free, or an 80B FP8 model (80 GB) leaves 250 GB free for an extended Key-Value (KV) cache or a co-resident draft model (Section 8.A).

B.1 Weight bandwidth (memory → local MAC)

Each BF16 MAC reads 16 bits from the DRAM bank directly above its tile at 30 fJ/bit with 3 ns latency. The bit-serial multiply runs at the 500 MHz wordline rate over 16 cycles for BF16 (8 cycles in FP8 inference mode); the per-column sense amplifier produces a 1-bit partial product per cycle that feeds an 8-level binary adder tree. A 4-stage pipeline hides DRAM latency.

Sophon delivers 4.20 PB/s of aggregate weight bandwidth in either datatype — the byte-rate of weight consumption is the same: 2 bytes/BF16-MAC at 2,100 TFLOPS, or 1 byte/FP8-MAC at 4,200 TFLOPS, both producing 4.20 PB/s. This bandwidth is in-tile and never crosses the Network-on-Chip (NoC).

Why is weight bandwidth independent of datatype and of capacity? In a Compute-In-Memory architecture, weight bandwidth is set by the MAC array's weight-consumption rate, which is intrinsic to the logic tiers, while capacity is set by the memory-tier areal density (110.0 Mb/mm² for 2T0C DRAM, §3.A). Because every weight is physically co-located with the MAC that consumes it, there is no shared bus whose width would scale with total stored bytes or with bit-depth: a higher-bit datatype simply reads more bits per MAC at a proportionally lower MAC rate. The bandwidth equality is therefore a direct consequence of BW = (bytes per MAC) × (MAC rate) being identical for both modes (1 B × 4,200 TFLOPS = 2 B × 2,100 TFLOPS = 4.20 PB/s).

B.2 Gradient bandwidth (training write path)

During the backward pass, accumulated gradients are written back to the DRAM bank at 20 fJ/bit:

Inference uses the read path only and incurs none of this write power.

B.3 Activation bandwidth (per-tile SRAM scratchpad)

Activations occupy a small per-tile SRAM scratchpad (SPM) (5% of tier area, ~37.5 mm²/tier, ~0.7 GB/tier):

• Per-tier activation bandwidth: ~11,000 GB/s aggregated
• Total activation bandwidth: ~700 TB/s

B.4 NoC bandwidth (inter-tile)

A 2-D mesh NoC routes activations and control. Each tier has its own mesh; vertical MIVs carry inter-layer activations.

B.5 Bandwidth summary

Sophon provides ~ 525× more weight bandwidth than a conventional HBM3e-based GPU (4,200 TB/s vs 8.0 TB/s for an 8-stack HBM3e package on NVIDIA B300 — unchanged from B200/B100 [16][18]) — because that bandwidth is intrinsic to the storage location, not a separate interconnect. Figure 4 plots the comparison.

C.1 Energy per MAC operation

Convention note: throughout this paper, "2,100 TFLOPS BF16" and "4,200 TFLOPS FP8" count each multiply-accumulate (MAC) as 2 floating-point operations (one mul + one add) [16]. Energies tabulated below are stated per MAC (per weight processed), so per-FLOP figures are half the listed values. The chip-power calculations in §C.3 use the per-FLOP convention to align with the TFLOPS rates.

Architecture note: Sophon uses pure digital Compute-In-Memory (CIM). Each tile contains a per-column sense amplifier feeding an 8-level binary adder tree that produces the partial sum for one row of a 256×256 weight subarray. All multiply-accumulate arithmetic is performed in the binary domain with full deterministic 16-bit (BF16) or 8-bit (FP8) precision — see §3.D for the digital-CIM tile walkthrough and §3.D.2 for why this choice constrains throughput as 1/N in the dense-decode regime.

C.2 Static and refresh power

Sophon's near-zero idle is an operational advantage: an 80B model loaded into Sophon waits for requests at ~ 2–3 W. An equivalent HBM3e-based GPU (e.g. NVIDIA B300) holds its 288 GB memory subsystem in self-refresh at ~ 10–15 W. With the 2D-TMD off-current at 1 fA/µm (I_off ≈ 0.5 fA per cell), the 2T0C retention time rises to 1.8 s and the array needs only a 1 Hz refresh, costing ≈ 0.08 W. A nominal 1 W allowance is carried below to cover warm steady-state operation; refresh is no longer a meaningful component of the power budget.

C.3 Active power by phase

C.4 Efficiency comparison

Sophon achieves ~ 2.1× higher TFLOPS/W than B300 at BF16 training (using the time-averaged training power figure of 564 W vs. B300's ~ 1,400 W TDP), because the digital adder tree keeps per-MAC energy low in both forward and backward passes and the 1 fA/µm off-current removes the former 264 W refresh tax. For inference, FP8-mode decode at 16.3 mJ/token is ~ 390× lower energy per token than B300 at low batch, where B300 is HBM-bandwidth-bound. Idle power is ~ 3 W, vs. ~ 10–15 W for B300's 288 GB HBM3e subsystem in self-refresh.

D.1 Tile geometry

Each Sophon tile is a 256×256 DRAM subarray with co-located digital MAC circuitry. The activation is bit-serialized — broadcast as sequential 1-bit wavefronts across the 256 wordlines at the 500 MHz tile clock (16 wavefronts for BF16, 8 for FP8). Each bit-cycle fires one row, producing 256 1-bit partial products that flow into a per-column sense amplifier, then into a tile-wide 8-level binary adder tree.

In FP8 inference mode the same tile geometry runs an 8-cycle bit-serial activation (vs 16 for BF16), doubling the MAC rate to 4,200 TFLOPS FP8.

D.2 Why digital CIM still scales as 1/N

A common misconception about CIM is that "all the math happens in parallel inside the memory, so model size shouldn't matter." This is true for weight transport, but not for MAC execution. A dense N-parameter transformer requires exactly 2N FLOPs per output token at batch size 1 — a mathematical requirement that no architecture can shortcut without changing the model.

For Sophon FP8 inference at 2,100 TMAC/s aggregate:

The slope is strictly inverse to N because each weight stored in the DRAM array participates in exactly one MAC per token, and the aggregate MAC ceiling is fixed by the tile count.

D.3 What CIM eliminates vs. what it preserves

Both fall as 1/N — only the absolute curve height differs. Sophon sits ~ 131× above B300 on the FP8-mode decode tokens/s curve because (a) zero weight-transport overhead (B300 decode at low batch is HBM-bandwidth-bound at the same 8 TB/s), (b) lower energy per MAC, and (c) sufficient peak MAC throughput — even though B300's raw peak FP8 FLOPS per die is somewhat higher.

D.4 What WOULD break 1/N — and what we picked

Three architectural or algorithmic paths can break the dense-decode 1/N curve:

1. Per-cell dedicated MAC units — give each of the 80 × 10⁹ cells its own dedicated MAC. Cells become ~ 7× larger; memory density drops sharply; 99% of MAC units idle on any given clock. Rejected: trades capacity for parallelism that cannot be sustained at constant utilization.

2. Speculative decoding — run a small draft model ahead, verify with the large model. Effective speedup of ~ 2.5× when the draft (1 B parameters, ~ 1.4% of Sophon's MAC budget) co-resides on the same die. Selected as Sophon's default inference deployment mode — see §8.A.6.

3. MoE (Mixture-of-Experts) and INT4 quantization — reduce the effective N that the MAC array sees. MoE shrinks active N by 4–10× (e.g., DeepSeek-V3 671 B → 37 B active); INT4 halves the cycle count by halving activation bit-depth. Both supported as first-class workloads, with combined effective throughput documented in §8.A.6.

The combination of (2) and (3) yields ~ 5× effective inference throughput improvement over the raw FP8 dense baseline on a single Sophon die.

Figure 4 plots the weight bandwidth comparison. Figure 5 decomposes per-MAC energy by component. Figure 6 shows the resulting active-power breakdown by workload phase.

4.1 2T0C gain-cell DRAM

Setup: write 1 at t = 0; hold; read at t = 1.0 s.

The stored voltage at the 1.0 s refresh point (433 mV, a comfortable 133 mV above the V_dd/2 ≈ 300 mV sense threshold) confirms the 1.0 s refresh interval is safe at 25 °C — see Figure 3 for the time-domain retention envelope at multiple temperatures. Retention scales ≈ 2× per 10 °C (Arrhenius); at 85 °C, τ falls to ≈ 28 ms, so the on-die controller shortens the interval to ≈ 20 ms (50 Hz) — a refresh cost of only ~ 4 W, with no dedicated high-power "fast-refresh" mode required.

4.2 Latch sense-amplifier

Binary current sense: a single latch fired against a fixed mid-point reference. The 1-bit output drives directly into the per-tile binary adder tree.

4.3 Thermal RC

34-node thermal network solved at DC for peak training power injection (749 W backward pass). Stack ΔT remains sub-Kelvin; package resistance dominates (see Section 5).

Steady-state at design power

All operating points — including the 100% fwd+bwd peak (1,362 W → 94.1 °C) — stay below T_jmax = 105 °C on a standard liquid cold plate, because eliminating the 264 W refresh tax (via the 1 fA/µm off-current) lowers every power point by ~ 263 W.

Key results

• The intrinsic stack ΔT is negligible (≤ 1.2 K at any tier count and any power level in this study), because each tier is only 0.35 µm thick and the Cu-MIV network conducts heat efficiently.
• The package thermal resistance R_pkg is the dominant bottleneck — not the M3D stack itself.
• Inference (235 W FP8 decode, 427 W FP8 peak burst) is comfortably within both liquid cold-plate and air-cooled envelopes; T_j ≤ 47 °C at decode and ≤ 96 °C even under 1U-server air cooling — a major operational advantage of the digital-CIM design.
• Training time-average (564 W) gives T_j = 53.6 °C under liquid cooling — comfortably below T_jmax and within the 2T0C retention model (τ = 1.8 s at 25 °C, ≈ 159 ms at 60 °C). Because the 1 fA/µm off-current makes refresh negligible (≈ 0.08 W at 1 Hz), the on-die controller simply shortens the refresh interval as T_j rises (≈ 20 ms at 85 °C, costing only ~ 4 W) — there is no longer a large "fast-refresh" power penalty.
• The peak fwd+bwd burst (1,362 W → 94.1 °C) stays within T_jmax on a standard liquid cold-plate; sustained 100% fwd+bwd duty is supported without microfluidic cooling.

Maximum sustained power vs. cooling technology

Inference (235 W FP8 decode, 427 W peak) fits comfortably within liquid cold-plate limits and is within striking distance of standard air cooling at decode — the chip can operate without any liquid plumbing in edge-inference deployments at moderately reduced clock rates. The training time-average (564 W) also fits liquid cold-plate with wide margin, and even the fwd+bwd 100%-duty peak (1,362 W → 94 °C) stays within T_jmax on a standard liquid cold plate — a direct benefit of removing the 264 W refresh overhead.

PFG-1 "Sophon" Roadmap (2T0C DRAM)

Comparison with HBM roadmap (8-stack package)

Sophon widens its capacity lead against HBM every generation. More importantly, the bandwidth lead is already insurmountable: 4.20 PB/s vs. HBM3e's 9.6 TB/s (8-stack package) — a ~ 440× gap that no interposer-based approach can close.

Total Cost of Ownership (TCO) and Bill of Materials (BOM)

Comparison vs. B300 (Blackwell Ultra) + HBM3e

A B300 SXM ships with 288 GB HBM3e at ≈ 8 TB/s. To match Sophon's 330 GB capacity requires a 1.15× capacity multiplier. The matched-bandwidth scaling is far harder: 8 HBM3e stacks deliver ~ 8 TB/s (unchanged from B200/B100), vs. Sophon's 4,200 TB/s in-tile — a ~ 525× gap that cannot be closed at any price point within the interposer paradigm.

Total Cost of Ownership (TCO) over 3-year datacenter deployment

The table below uses a representative production-server duty cycle, a Power Usage Effectiveness (PUE) of 1.5, and a $0.10/kWh electricity tariff — yielding an effective $0.15/kWh after datacenter cooling and distribution overhead. Numbers are per single die over 3 years (26,280 hours).

Sophon's TCO advantage comes from two compounding effects:

1. Hardware cost: ~ 7.2× cheaper BOM than a B300.
2. Idle + active energy: at ~ 3 W idle vs. B300's ~ 10–15 W memory-idle, and 235 W FP8 decode vs. B300's ~ 1,400 W TDP, Sophon spends a small fraction of B300's combined idle+active energy budget. For training, with refresh eliminated by the 1 fA/µm off-current, Sophon draws a 564 W training average (vs. a B300's ~ 1,400 W TDP) and idles at ~ 3 W (vs. B300's ~ 10–15 W memory-idle). It completes the same training work in roughly 7.5× fewer die-seconds per token, and on an energy-per-trained-token basis is ~ 21× more efficient than B300 (Section 8.B.5).

Tier 1 — Column-Redundancy Repair (Yield Recovery)

Each 2D-TMD CIM tile is provisioned with 4 spare columns per 256-column bank (~1.6% column-area overhead). Wafer-level Automated Optical Inspection (AOI) identifies defective bitlines; a one-time electrical fuse (e-fuse) map reroutes those columns to spares before Known-Good-Die (KGD) selection. This converts the majority of single-column faults — typically the dominant failure mode in M3D via layers — into repaired working dies.

Tier 2 — Tile-Level Disaggregation (Partial-Good Harvesting)

Dies that fail Tier 1 repair due to clustered multi-column faults are evaluated at the tile granularity (each die contains 576 tiles, §3.D). A die with ≤ 10% tile failures (~58 tiles) is re-characterised and deployed at reduced capacity:

Grade-B and Grade-C dies are targeted at edge-inference and MoE partial-expert deployments where capacity headroom exceeds strict density requirements. Modelling of the negative-binomial defect distribution (α = 3) indicates that ~18% of otherwise-scrapped dies qualify for Grade-B or Grade-C harvest.

Tier 3 — Known-Good-Die (KGD) Burn-In Protocol

All KGD candidates (full and partial-good) undergo a 24-hour elevated-voltage burn-in at V_DD + 10% and T_junction = 85 °C to screen infant-mortality failures — primarily 2T0C retention outliers. Post burn-in, full parametric re-test confirms:

• 2T0C retention τ ≥ 1.0 s at 25 °C; ≥ 15 ms at 85 °C
• Leakage I_off per device ≤ 2 fA/µm at 85 °C
• Sense-margin window ≥ 130 mV at the 1.0 s refresh point

Field return data from analogous 28 nm BEOL products places the post-burn-in Annualised Failure Rate (AFR) below 0.1% per die-year — consistent with the mission-life assumptions in §5 (Thermal) and §7 (TCO).

Combined Yield & Cost Impact

The Tier 1 + Tier 2 combined uplift reduces the effective cost per working die by ~29–30%, tightening the BOM advantage over B300-class HBM3e systems from ~50% to a ~58–62% realised advantage after accounting for the area and test cost overheads.

Note on M3D-specific defect modes. The dominant yield detractor in the 64-tier 2D-TMD M3D stack is not planar Si lithography (which is mature at 28 nm) but rather Monolithic Inter-tier Via (MIV) open/short defects at the ~0.2 µm via pitch. Tier 1 column redundancy is specifically architected to absorb MIV-induced single-bitline opens — the most frequent M3D failure signature observed in imec SCALE 2024 demonstration vehicles [7]. Tier 2 tile harvesting addresses clustered MIV fault regions that escape column repair, which are typically correlated with local TMD grain boundary density gradients from CVD non-uniformity.

8.1 Die stack overview

8.A. Inference

Sophon serves inference on the same silicon it trains on. The MAC array supports both native BF16 (the training datatype) and an FP8 inference mode (4,200 TFLOPS / 8,400 INT8 TOPS); FP8 is the recommended serving mode because it doubles decode throughput, halves energy/token, and frees capacity. The model loads once and serves indefinitely; a powered-off die reloads from NVMe at boot (§10.2).

8.B. Training

8.C. Train-then-serve system view

Because inference and training run on the same die, a production AI cluster is built from a single Sophon Stock-Keeping Unit (SKU) and repartitioned by software:

This flow lets a single fleet elastically shift dies between training and serving without any hardware swap: the same silicon that trained a model can serve it (BF16 directly, or FP8 after a one-step quantization), and dies can be re-tasked from serving back to fine-tuning as demand shifts. The only operational discipline DRAM imposes is volatility management — weights are checkpointed to NVMe and reloaded at boot (§10.2); there is no non-volatile "model resident across power-off" property, but in a continuously-powered datacenter the ~ 3 W idle makes keeping a model resident essentially free.

9.1 Minimal single-event target — the capacitor-less cell

In a conventional 1T1C DRAM, the bit lives as charge on a deep-trench or stacked capacitor of tens of femtofarads; the capacitor and its substrate collection volume present a large sensitive cross-section, and a single ionizing strike that collects enough charge flips the bit [31]. The 2T0C gain cell eliminates the capacitor entirely: state is held on the ~ 3.0 fF parasitic node (C_gs of the read transistor plus the write transistor's junction) confined to a sub-micron footprint at the Metal-3 BEOL — far above the silicon substrate. The radiation target area per bit shrinks by orders of magnitude relative to a capacitor cell, and with it the single-event upset (SEU) cross-section of the 330 GB array.

9.2 Channel on dielectric — no substrate damage path, no lattice cascade

The 2D-TMD channel is grown on amorphous dielectric, not on a bulk semiconductor. This removes the two dominant radiation-degradation mechanisms of silicon devices at the root. First, there is no substrate beneath the active channel to accumulate displacement damage: the lattice-disorder-induced leakage paths, charge-funneling collection, and parasitic latch-up structures of bulk CMOS simply do not exist in the upper tiers [32]. Second, displacement damage in the channel itself is bounded by geometry: an energetic particle traversing a three-atom-thick sheet can at most knock individual atoms out of the monolayer, producing an isolated point defect. There is no three-dimensional volume in which a collision cascade can develop, so the surrounding covalently bonded lattice remains crystalline and the transistor continues to operate — in contrast to bulk silicon, where a single primary knock-on atom displaces thousands of lattice atoms [33].

These mechanisms are not merely theoretical. 2D-material devices have shown negligible performance change after γ-ray, proton, and electron irradiation at space-relevant doses [34], and a wafer-scale monolayer MoS₂ RF system has operated in low Earth orbit for nine months with a bit error rate below 10⁻⁸ — with a predicted lifetime of ~ 271 years even in geosynchronous-orbit flux [35]. Combined with the total-ionizing-dose immunity noted in §1 (no buried-oxide trap vulnerability) and the seconds-scale refresh that bounds any transient corruption window, these properties make the platform a natural fit for satellite inference payloads. Formal SEE characterization of the full Sophon stack for LEO/MEO flux environments remains a qualification milestone (§10.3).

10.1 Validation status

10.2 Risks

1. 2T0C retention temperature derating. At junction T > 60 °C, I_off increases (≈ 2× per 10 °C in TMD), reducing τ from 1.8 s (25 °C) to ≈ 28 ms at 85 °C. Mitigated by an on-die thermal sensor that shortens the refresh interval (e.g. ≈ 20 ms at 85 °C). Power overhead: only ~ 4 W even in the hot corner — because the 1 fA/µm off-current keeps baseline refresh at ≈ 0.08 W, temperature derating no longer carries a large power penalty.
2. Refresh power under training load. At nominal training, refresh draws ≈ 0.08 W (1 Hz) — less than 0.02% of the 564 W average training power, and effectively negligible. This is the decisive benefit of the 1 fA/µm TMD off-current: the former 264 W refresh tax (~ 32% of power) is eliminated, and no bank-level power gating is required to manage refresh.
3. Gradient write bandwidth. Backward pass writes one BF16 gradient per active MAC, drawing 370 W at 55% utilization and 672 W at 100% utilization. The TMD write transistor has demonstrated > 10¹⁵ write cycles in laboratory tests, but production qualification at full training duty cycle is pending.
4. Optimizer state capacity. 80B BF16 training with full Adam requires 480 GB (weights + first moment + second moment). Sophon at 330 GB supports SGD with momentum or Lion (320 GB total). A scaled 96-tier Sophon (Section 6) reaches ~ 495 GB and accommodates full Adam.
5. Power-off model loss. As with all DRAM, Sophon loses its contents on power-off. Production flows must checkpoint to off-die NVMe at standard intervals and reload from NVMe at boot; the ~ 3 W idle means a resident model can simply be kept powered between requests.
6. 2T0C + M3D thermal budget. All BEOL steps must remain ≤ 450 °C. PhantaField Phase 1 tapeout validates co-integration of the 2T0C DRAM module with the TMD MAC stack.
7. Wafer-scale TMD uniformity. NanoGalaxy™ MOCVD qualification in progress; wafer-scale uniformity governs both MAC yield and 2T0C retention spread.

10.3 Future work

• 96-tier variant. 96 tiers × 330/64 GB/tier ≈ 495 GB — fits full Adam optimizer for 80B BF16 training with 15 GB activation headroom, and serves a 480 GB FP8 inference model.
• Radiation-hardness qualification. The TMD stack benefits from intrinsic Total Ionizing Dose (TID) immunity. Early Single Event Effect (SEE) data needed for Low Earth Orbit (LEO) and Medium Earth Orbit (MEO) deployment.
• 1 GHz bit-serial mode. A speculative PFG-1 Rev 1.5 running the bit-serial activation broadcast at 1 GHz instead of 500 MHz would push Sophon to 4,200 TFLOPS BF16 / 8,400 TFLOPS FP8 per die at the same per-MAC energy, contingent on adder-tree timing closure at the higher rate. This is the headline ceiling for the Sophon scaling roadmap (§6).
• Optical I/O (PFG-2). Co-packaged silicon-photonics (SiPh) optics for inter-die NVLink replacement, eliminating the 1.8 TB/s conventional interconnect bottleneck in multi-die training clusters.
• Non-volatile companion tier. An optional embedded non-volatile tier (for example a thin RRAM or MRAM snapshot layer) could checkpoint weights on-die for instant warm-restart, removing the NVMe reload latency — evaluated for a future revision.

A. Device physics — 2D Transition Metal Dichalcogenide (TMD) transistors

[1] Radisavljevic, B., et al. "Single-layer MoS₂ transistors." Nature Nanotechnology 6, 147–150 (2011). DOI: 10.1038/nnano.2010.279. https://doi.org/10.1038/nnano.2010.279 → Source for MoS₂ baseline mobility (~ 200 cm²/V·s), I_on/I_off > 10⁸.

[2] Liu, Y., Duan, X., Shin, H.-J., et al. "Promises and prospects of two-dimensional transistors." Nature 591, 43–53 (2021). DOI: 10.1038/s41586-021-03339-z. https://doi.org/10.1038/s41586-021-03339-z → Source for TMD I_off density ≈ 10⁻¹⁵ A/µm (1 fA/µm) at 28 nm gate length; comparative tables of MoS₂ vs Si scaling.

[3] Lan, H.-Y., et al. "Dual-Gate Synthetic MoS₂ MOSFETs with 4.56 µS/µm g_m, 320 µA/µm I_d at 1 V V_d." IEDM 2022 Technical Digest, paper 7.3. IEEE. https://ieeexplore.ieee.org/document/10019462 → Source for TMD nFET drive current, sub-threshold slope (~ 75 mV/dec), V_dd = 0.6 V operation.

[4] Sebastian, A., et al. "Benchmarking monolayer MoS₂ and WS₂ field-effect transistors." Nature Communications 12, 693 (2021). DOI: 10.1038/s41467-020-20732-w. https://doi.org/10.1038/s41467-020-20732-w → WSe₂/WS₂ p-FET hole mobilities (60–120 cm²/V·s); CMOS-pair benchmarking.

B. Compute-In-Memory and Monolithic 3D (M3D) integration

[5] Shulaker, M. M., et al. "Three-dimensional integration of nanotechnologies for computing and data storage on a single chip." Nature 547, 74–78 (2017). DOI: 10.1038/nature22994. https://doi.org/10.1038/nature22994 → M3D nanosheet proof-of-concept; demonstrates low-temperature BEOL stacking compatible with this paper's TMD M3D approach.

[6] Vinet, M., et al. (CEA-Leti). "Monolithic 3D Integration: A Powerful Alternative to Classical 2D Scaling." IEEE S3S Conference 2014. https://ieeexplore.ieee.org/document/7028181 → Established M3D thermal budget constraints (≤ 450 °C BEOL ceiling) cited in §2.A.

[7] imec. "SCALE-3D: Scaling roadmap for monolithic 3D integration." imec Technology Forum 2024. https://www.imec-int.com/en/articles/monolithic-3d-integration → MIV (Monolithic Inter-tier Via) pitch (~ 90 nm) and density (~ 10⁸/mm²) used in §2.A.

C. 2T0C gain-cell DRAM

[8] Belmonte, A., et al. (imec). "Capacitor-less, Long-Retention (>400 s) DRAM Cell Paving the Way Towards Low-Power and High-Density Monolithic 3D DRAM." IEDM 2020, paper 28.2. https://ieeexplore.ieee.org/document/9372074 → Imec 2T0C IGZO-channel demonstration; establishes 2T0C feasibility and validates closed-form retention model τ = C·V/(2·I_off) used in §4.1.

[9] Liu, X., et al. "A 2T0C DRAM Based on Amorphous In-Ga-Zn-O Thin Film Transistors with Retention Time Larger Than 400 s." IEEE Electron Device Letters 41(8), 1184–1187 (2020). https://ieeexplore.ieee.org/document/9118898 → Independent confirmation of long-retention 2T0C; basis for TMD adaptation in this paper.

[10] Wu, F., et al. "Vertically Stacked Multilayer Heterostructures for 2T0C DRAM." Nature Electronics 5, 519–526 (2022). DOI: 10.1038/s41928-022-00807-w. https://doi.org/10.1038/s41928-022-00807-w → 2D-material-based 2T0C with sub-µm² cells; closest published analogue to the Sophon cell.

D. Energy and computation models

[11] Horowitz, M. "Computing's energy problem (and what we can do about it)." ISSCC 2014 Keynote. IEEE. https://ieeexplore.ieee.org/document/6757323 → Source for the per-operation energy model (FP add ~ 0.4 pJ @ 45 nm, scaling by V_dd²); the TMD MAC energy in §C.1 is computed by scaling this with V_dd² ratio and 0.85× TMD device factor (from [3]).

[12] Jouppi, N. P., et al. "Ten Lessons From Three Generations Shaped Google's TPUv4i." ISCA 2021. https://ieeexplore.ieee.org/document/9499913 → Industrial benchmark for tile-array CIM energy per MAC and utilization figures (55% sustained, 75% peak).

[13] Patterson, D., et al. "Carbon Emissions and Large Neural Network Training." arXiv:2104.10350 (2021). https://arxiv.org/abs/2104.10350 → Source for the "6 × N_params FLOPs per training token" estimator and per-token energy framework used in §8.B.3.

[14] Kaplan, J., et al. "Scaling Laws for Neural Language Models." arXiv:2001.08361 (2020). https://arxiv.org/abs/2001.08361 → Source for the 2 × N_params FLOPs per inference token estimator used in §8.A.3.

[15] Hoffmann, J., et al. (Chinchilla). "Training Compute-Optimal Large Language Models." arXiv:2203.15556 (2022). https://arxiv.org/abs/2203.15556 → Source for the 1T–15T training-token range used in §8.B.3 cluster sizing.

E. Comparison hardware

[16] NVIDIA Corporation. NVIDIA Blackwell Ultra (GB300 NVL72) Architecture Technical Brief (2025). https://www.nvidia.com/en-us/data-center/gb300-nvl72/ → Source for B300 (Blackwell Ultra) per-GPU specs: ≈ 5,000 TFLOPS dense FP8 (10,000 sparse), ≈ 10,000 TFLOPS dense FP4 (15,000 sparse, 1.5× the FP4 of B200), ≈ 2,500 TFLOPS dense BF16, 288 GB HBM3e (1.5× B200), ≈ 8.0 TB/s memory bandwidth per GPU, ≈ 1,400 W TDP, TSMC 4NP dual-die (208 B transistors), fifth-generation NVLink at 1.8 TB/s bidirectional.

[17] NVIDIA Corporation. NVIDIA GB300 NVL72 / DGX GB300 Platform Specifications (2025). https://www.nvidia.com/en-us/data-center/dgx-gb300/ → SKU details for B300/GB300/GB300 NVL72; hyperscaler street pricing (≈ $50k–$70k per B300 SXM module class); power reference (B300 SXM ≈ 1,400 W TDP).

[18] JEDEC Solid State Technology Association. JESD238A: HBM3 Standard (2023). https://www.jedec.org/standards-documents/docs/jesd238a → HBM3/HBM3e bandwidth per stack (~ 1.2 TB/s); 8-stack package = 9.6 TB/s reference.

[19] JEDEC. Roadmap: HBM4 and HBM5 — preliminary specifications. https://www.jedec.org/news/pressreleases → Source for HBM4/HBM4e/HBM5/HBM5e roadmap capacity figures used in §6.

F. Thermal model

[20] Pop, E. "Energy Dissipation and Transport in Nanoscale Devices." Nano Research 3, 147–169 (2010). DOI: 10.1007/s12274-010-1019-z. https://doi.org/10.1007/s12274-010-1019-z → Source for BEOL effective thermal conductivity baseline (k_BEOL ≈ 2.0 W/m·K).

[21] Mahajan, R., et al. (Intel). "Cooling a Microprocessor Chip." Proceedings of the IEEE 94(8), 1476–1486 (2006). https://ieeexplore.ieee.org/document/1683998 → Source for liquid cold-plate package thermal resistance (R_pkg ≈ 0.05 K/W).

[22] Bar-Cohen, A., et al. "Embedded Cooling for Wide Bandgap Power Amplifiers." IEEE Trans. Components, Packaging and Manufacturing Tech. 5(9), 1226–1239 (2015). https://ieeexplore.ieee.org/document/7173025 → Source for microfluidic R_pkg ≈ 0.02 K/W; two-phase immersion ≈ 0.01 K/W envelope.

G. Economics and yield

[23] Cunningham, J. A. "The Use and Evaluation of Yield Models in Integrated Circuit Manufacturing." IEEE Trans. Semiconductor Manufacturing 3(2), 60–71 (1990). https://ieeexplore.ieee.org/document/55438 → Negative-binomial yield model with clustering parameter α = 3; basis for the 49.5% base yield in §7.

[24] Stapper, C. H. "Modeling of Defects in Integrated Circuit Photolithographic Patterns." IBM Journal of R&D 28(4), 461–475 (1984). https://ieeexplore.ieee.org/document/5390244 → Murphy yield model used as cross-check (51.2% for A·D₀ = 0.75) in the audit calculations.

[25] TechInsights. 28 nm Foundry Wafer Cost Analysis, 2025–2026 Update. TechInsights subscription report; public summary: https://www.techinsights.com/wafer-cost-analysis → Source for the $3,500 28 nm 12-inch wafer cost.

[26] U.S. Energy Information Administration. Average Industrial Electricity Price, 2025. https://www.eia.gov/electricity/monthly/ → Source for the $0.10/kWh industrial tariff baseline used in TCO (§7).

[27] Uptime Institute. Global Data Center Survey 2024 — PUE Trends. https://uptimeinstitute.com/resources/research/global-data-center-survey-2024 → Source for the PUE = 1.5 assumption (industry median for liquid-cooled facilities).

I. Workload-level accelerators

[29] Leviathan, Y., Kalman, M., Matias, Y. "Fast Inference from Transformers via Speculative Decoding." ICML 2023. https://arxiv.org/abs/2211.17192 → Source for the speculative-decoding speedup model, k = 4 draft length, 70% token-acceptance rate baseline used in §8.A.6 and Eq. 17.

[30] Lin, J., et al. "AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration." MLSys 2024. https://arxiv.org/abs/2306.00978 → Source for INT4 weight-only quantization quality bounds (≤ 1–2 perplexity points vs FP8 on 70B-class instruction-tuned models) used in §8.A.6 and Eq. 17.

J. Radiation effects & space deployment

[31] Baumann, R. C. "Radiation-induced soft errors in advanced semiconductor technologies." IEEE Transactions on Device and Materials Reliability 5(3), 305–316 (2005). DOI: 10.1109/TDMR.2005.853449. https://doi.org/10.1109/TDMR.2005.853449 → Source for the single-event-upset mechanism: charge collection onto storage nodes, and the dependence of SEU cross-section on sensitive-node volume, used in §9.1.

[32] Schwank, J. R., Ferlet-Cavrois, V., Shaneyfelt, M. R., Paillet, P., Dodd, P. E. "Radiation effects in SOI technologies." IEEE Transactions on Nuclear Science 50(3), 522–538 (2003). DOI: 10.1109/TNS.2003.812930. https://ieeexplore.ieee.org/document/1208574 → Source for dielectric isolation effects: reduced charge-collection volume, elimination of substrate funneling, and latch-up immunity of devices isolated from the bulk substrate, used in §9.2.

[33] Komsa, H.-P., Kotakoski, J., Kurasch, S., Lehtinen, O., Kaiser, U., Krasheninnikov, A. V. "Two-dimensional transition metal dichalcogenides under electron irradiation: defect production and doping." Physical Review Letters 109, 035503 (2012). DOI: 10.1103/PhysRevLett.109.035503. https://doi.org/10.1103/PhysRevLett.109.035503 → Source for displacement-threshold energies in TMD monolayers and the isolated-point-vacancy character of irradiation damage in atomically thin sheets, used in §9.2.

[34] Vogl, T., Sripathy, K., Sharma, A., et al. "Radiation tolerance of two-dimensional material-based devices for space applications." Nature Communications 10, 1202 (2019). DOI: 10.1038/s41467-019-09219-5. https://doi.org/10.1038/s41467-019-09219-5 → Demonstrates negligible performance change in 2D-material devices after γ-ray, proton, and electron irradiation at space-relevant doses, used in §9.

[35] Zhu, L., et al. "Radiation-tolerant atomic-layer-scale RF system for spaceborne communication." Nature 650, 346–352 (2026). DOI: 10.1038/s41586-025-10027-9. https://www.nature.com/articles/s41586-025-10027-9 → On-orbit demonstration: a wafer-scale monolayer MoS₂ RF transmit/receive system operated at ~ 517 km LEO for 9 months with bit error rate < 10⁻⁸, with a predicted ~ 271-year lifetime in GEO flux, used in §9.

Eq. 1 — Planar memory density

where F² is the cell footprint in lithographic squares (8 for the 2T0C DRAM cell), F_nm is the half-pitch in nm (28 nm baseline), p is the periphery overhead fraction (0.45 for DRAM), and b is bits per cell (1 for 2T0C). The 10¹² factor converts nm² to mm².

Worked example — Sophon 2T0C DRAM: D = 10¹² / (8 × 28² × 1.45) × 1 = 110.0 Mb/mm². Source for cell: [10] (analogous 2D-material 2T0C); validated by [8][9].

Eq. 2 — Per-die capacity

where A_mem-tier is the full footprint of one memory tier (750 mm²) and N_mem-tiers = 32. The 64-tier stack interleaves dedicated logic and memory tiers (32 of each); only the 32 memory tiers contribute to capacity.

Sophon: C = (110.0 × 750 × 32) / 8000 = 330.2 GB (rounded to 330 GB).

Eq. 3 — 2T0C retention time

The factor of 2 reflects the sense margin: data is reliably recovered while the stored voltage remains above V_dd/2. Source: [8] (closed-form derivation); [9] (empirical confirmation).

Worked example: C_node = 3.0 fF (sum of C_gs,RT ≈ 2.5 fF + C_j,WT ≈ 0.5 fF), V_dd = 0.6 V. The off-current is specified as a width-normalized density J_off = 10⁻¹⁵ A/µm = 1 fA/µm for the 2D-TMD nFET [2][3]; with a Read-Transistor channel width W_RT = 0.5 µm the absolute leakage is I_off = J_off · W_RT = 0.5 fA (5 × 10⁻¹⁶ A) at 25 °C: τ = (3.0 × 10⁻¹⁵ × 0.6) / (2 × 5 × 10⁻¹⁶) = 1.8 s at 25 °C.

This is ≈ 4,800× longer than a 1T1C DRAM cell and reflects the exceptional sub-threshold off-state of the atomically-thin TMD channel (I_on/I_off > 10⁸, sub-threshold slope ≈ 75 mV/dec). Retention derates with junction temperature at ≈ 2× per 10 °C (Arrhenius): τ ≈ 159 ms at 60 °C and ≈ 28 ms at 85 °C.

Eq. 4 — Refresh power

with C_bits = capacity in bits, f_refresh = 1 / T_refresh.

Sophon: at 25 °C the retention τ = 1.8 s (Eq. 3) permits a relaxed refresh interval of T_refresh = 1.0 s (1.8× margin). P = (330 × 8 × 10⁹ bits) × (1 / 1.0 Hz) × (30 × 10⁻¹⁵ J/bit) = 0.079 W — effectively negligible. This is the decisive consequence of the 1 fA/µm off-current: refresh power drops by ≈ 3,300× relative to a conventional gain cell. The on-die controller scales the interval with junction temperature (Eq. 3 derating); even in the worst hot corner the refresh cost stays small — at 85 °C a 20 ms interval (50 Hz) gives P_refresh ≈ 4.0 W, and at a 105 °C excursion a 5 ms interval (200 Hz) gives ≈ 15.8 W. A nominal 1 W refresh allowance is carried in the power budget (§5) to cover warm steady-state operation with margin.

Eq. 5 — Per-MAC energy decomposition

Total energy per MAC operation is the sum of memory access and compute. Sophon uses pure digital CIM (binary sense amplifier + adder tree per column-group.

Sophon BF16 forward MAC: E = (30 fJ/bit × 16 bits) + E_{adder-tree,BF16} = 0.480 pJ + 0.140 pJ = 0.620 pJ/MAC.

Sophon BF16 backward MAC: add gradient write E_write = 20 fJ/bit × 16 bits = 0.320 pJ → 0.940 pJ/MAC total per weight per training step.

Sophon FP8 inference MAC: E = (30 fJ/bit × 8 bits) + E_{adder-tree,FP8} = 0.240 pJ + 0.070 pJ = 0.310 pJ/MAC.

E_{adder-tree,FP8} is computed from per-bit binary adder energy in 28 nm CMOS at 0.6 V [11] scaled to 2D-TMD: 8 fJ × 8 levels × 0.85 ≈ 0.054 pJ; with sign-bit and mantissa pipeline overhead the effective figure is 0.070 pJ/MAC. The BF16 adder-tree figure (0.140 pJ) is twice the FP8 figure because the bit-serial activation broadcast runs for 16 cycles instead of 8. The fully digital adder tree is the primary energy improvement of the digital-CIM architecture.

Eq. 6 — Active chip power

where R_op is the peak operation rate (FLOPS), u is the utilization fraction, E_{per op} is per-FLOP energy (half of per-MAC energy, since 1 MAC = 2 FLOPs).

Sophon FP8 decode (55% util.): P = 4,200 × 10¹² × 0.55 × (0.310 / 2) × 10⁻¹² + ~75 W overhead = ≈ 235 W (matches §C.3 table: DRAM read 138 W + digital MAC 81 W + NoC 13 W + static 2 W).

Sophon BF16 forward (55% util.): P = 2,100 × 10¹² × 0.55 × (0.620 / 2) × 10⁻¹² + ~1 W refresh + 20 W NoC + 2 W static = ≈ 379 W (the negligible refresh term, vs. the former 264 W, is the dominant change).

Sophon backward (55% util.): + gradient write power 2,100 × 10¹² × 0.55 × (0.320 / 2) × 10⁻¹² = + 185 W extra at FLOP rate, or 370 W at MAC rate. The §C.3 table uses 370 W → ≈ 749 W total.

Utilization 55% is from TPUv4i sustained workload data [12]; peak 100% used for thermal worst-case.

Eq. 7 — Inference throughput (decode)

From Kaplan et al. [14]:

Sophon 80B FP8 decode: tokens/s = (4,200 × 10¹² × 0.55) / (2 × 80 × 10⁹) = 14,438 tokens/s. Sophon 80B BF16 decode: tokens/s = (2,100 × 10¹² × 0.55) / (2 × 80 × 10⁹) = 7,219 tokens/s.

Eq. 8 — Training throughput

From Patterson et al. [13]:

The factor 6 (vs. 2 for inference) accounts for forward (2N) + backward (4N) compute.

Sophon 80B BF16: tokens/s = (2,100 × 10¹² × 0.55) / (6 × 80 × 10⁹) = 2,406 tokens/s/die.

Eq. 9 — Cluster training time

Examples:

• 256-die cluster, 1T tokens: 10¹² / (256 × 2406 × 86400) ≈ 18.8 days.
• 1,024-die cluster, 1T tokens: ≈ 4.7 days.
• 1,024-die cluster, 15T (Llama-3-class [15]): ≈ 70.6 days.

Eq. 10 — Energy per token

Sophon FP8 decode: E = 235 W / 14,438 tokens/s = 16.3 mJ/token. Sophon BF16 decode: E = 379 W / 7,219 tokens/s = 52.5 mJ/token. Sophon training (time-avg fwd + bwd): E = 564 W / 2,406 tokens/s = 0.234 J/token.

Eq. 11 — Yield (negative binomial with defect clustering)

Source: Cunningham [23]. A = 7.5 cm² die area, D₀ = 0.1 defect/cm² (mature 28 nm), α = 3 (typical clustering).

Y = (1 + 0.75/3)⁻³ = 0.495 → 49.5% base wafer yield.

Cross-check with Murphy/Stapper [24]: Y = ((1 − exp(−2·AD₀)) / (2·AD₀))² = 0.512 → 51.2%. The negative-binomial result is used as the conservative estimate.

Eq. 12 — M3D stack yield

With Y_tier = 0.997 (3 σ M3D process control achievable per imec [7]): Y_stack = 0.997⁶⁴ = 0.825.

Combined yield (base × stack): 0.495 × 0.825 = 0.408 → 40.8% final die yield used in the BOM calculation (§7).

Eq. 13 — BOM per die

Sophon: BOM = ($51 + 64 × $52) / 0.408 + $60 + $0 + $25 = $8,273 + $85 = $8,358.

Wafer cost from [25]; tier adder estimated from per-tier mask + processing economics in [7][5].

Eq. 14 — 3-year TCO

with 26,280 hours = 3 years × 8,760 h/year, PUE = 1.5 [27], c_kWh = $0.10/kWh [26]. P_avg is the duty-weighted average power.

Sophon inference (30% busy FP8 decode, 70% idle): P_avg = 0.30 × 235 + 0.70 × 3 = 72.6 W → energy 1,908 kWh × $0.15 = $286 → TCO = $8,358 + $286 = $8,644.

B300 same duty: P_avg = 0.30 × 1,400 + 0.70 × 200 = 560 W → energy 14,717 kWh × $0.15 = $2,208 → TCO = $16,375 + $2,208 = $18,583.

Sophon training (50% busy training, 50% idle): P_avg = 0.50 × 564 + 0.50 × 3 = 283.5 W → energy 7,450 kWh × $0.15 = $1,118 → TCO = $8,358 + $1,118 = $9,476.

Eq. 15 — Effective vertical thermal conductivity (BEOL stack)

Parallel-conduction model with Cu fill fraction φ_Cu = 0.06 (Monolithic Inter-tier Via density × via cross-section / total area), k_Cu = 380 W/m·K, k_BEOL = 2.0 W/m·K [20]:

k_eff = 0.06 × 380 + 0.94 × 2.0 = 24.7 W/m·K.

Eq. 16 — Steady-state junction temperature

R_stack = (N_tiers × t_tier) / (k_eff × A_die) is the M3D stack resistance; R_pkg is the package-to-coolant resistance from [21][22].

Sophon FP8 decode (235 W, liquid R_pkg = 0.05 K/W): R_stack = (64 × 0.35 × 10⁻⁶) / (24.7 × 7.5 × 10⁻⁴) = 0.00121 K/W (negligible) → T_j = 25 + 235 × 0.0512 = 37.0 °C.

Sophon training avg. (564 W): T_j = 25 + 564 × 0.0512 = 53.9 °C (well below T_jmax = 105 °C).

Eq. 17 — Effective decode throughput with workload-level accelerators

The raw dense FP8 baseline of Eq. 7 can be multiplied by three orthogonal workload-level accelerators on a single Sophon die. Let s be the speculative-decoding multiplier, q be the quantization multiplier, and N_active / N_total be the MoE sparsity ratio. The effective decode throughput becomes:

with assumed multiplier values supported by published technique benchmarks:

• s = 2.5 for speculative decoding with a 1 B-parameter draft model co-resident on the same die (k = 4 candidates, 70% mean acceptance per token; the draft consumes ~ 1.4% of the MAC budget) [29].
• q = 2.0 for INT4 weight quantization vs. FP8 (halves the bit-serial activation cycle count without changing the underlying MAC accuracy by more than 1–2 perplexity points on 80B-class instruction-tuned models) [30].
• N_active / N_total ∈ [0.05, 0.30] for production MoE configurations (Mixtral, DeepSeek-V3, frontier-MoE estimates).

Worked example — Sophon 80B dense, INT4 + speculative (FP8 mode): tokens/s = (4,200 × 10¹² × 0.55 × 2.5 × 2.0) / (2 × 80 × 10⁹) = 72,188 tokens/s/die = ~ 5× raw FP8 baseline.

Worked example — Sophon DeepSeek-V3 MoE (671 B total / 37 B active), FP8 dense weights: tokens/s = (4,200 × 10¹² × 0.55) / (2 × 37 × 10⁹) = 31,216 tokens/s/die = ~ 18× the equivalent 671 B dense decode rate.

Note that the three multipliers do not all compose additively in every regime: speculative decoding's effective speedup depends on the small-model draft accuracy (which itself depends on the deployment domain), and the q = 2 INT4 multiplier and the MoE sparsity multiplier compose only when the model architecture supports both jointly. The benchmark table in §8.A.6 enumerates the realistic combinations.