Markdown

Model-data co-scheduling for expert-parallel all-to-all reduction¶

Scope: reducing expert parallelism's all-to-all communication cost by scheduling expert placement and token/request routing jointly, instead of treating "which device holds which expert" and "which device processes which token" as two separate problems. This is a placement-and-routing technique that sits alongside, not instead of, the load-balancing mechanisms MoE routing and expert load balancing already covers (auxiliary loss, capacity factor, EPLB); those pages assume expert-token assignment is data-dependent and unpredictable per request, this page covers the case where it is not.

The co-clustering mechanism below is executed and asserted with the system python3 (numpy): a synthetic token-expert affinity structure, a greedy expert-clustering heuristic, and the resulting local-activation-rate and communication-volume math. It reproduces the mechanism the cited paper describes, not its real profiled affinity data or its exact ILP solver; the specific throughput and latency numbers quoted below are the paper's own reported figures, not reproduced here.

What it is¶

Expert parallelism shards a MoE layer's experts across devices, so every token's activation has to be dispatched, via an all-to-all collective, to whichever device holds its selected top-k experts, then gathered back after the expert FFN runs.¹ Expert parallelism for MoE inference already covers why that all-to-all is expensive; model-data co-scheduling attacks a different lever than that page's kernel-level mitigations (DeepEP, hierarchical all-to-all, expert collocation by static analysis): it exploits the fact that, empirically, which expert a given token activates is largely independent of the surrounding context, and therefore predictable, and jointly schedules both which device holds which expert and which device processes which token to keep as much of that traffic local as possible.¹

Three components implement this:¹

Offline model scheduling. Profile token-expert activation frequency, cluster experts that are frequently co-activated by the same tokens onto the same device, and periodically re-place them. This runs offline, not on the request path.
Online inter-request scheduling (Attention-DP). When attention runs under data parallelism, whole requests are independent per DP rank; reorder incoming requests so a request lands on the DP rank hosting the experts its tokens are predicted to activate most.
Online intra-request scheduling (Attention-TP). When attention runs under tensor parallelism, a request's tokens are already split across ranks; fuse a token-reshuffling step into the existing post-attention communication, replacing a plain all-reduce with a shuffled-reduce-scatter and a deferred shuffled-allgather that route each token's activation toward the device predicted to host its expert, before the MoE all-to-all ever has to run.

Why it matters¶

The premise is not theoretical: profiling a MoE model on production-shaped traffic (DeepSeek-V2-Lite on ShareGPT) found that expert activation is skewed and stable per token, not just per prompt. The median (p50) cumulative activation probability mass captured by a token's top-k "hottest" experts ranges from 0.833 to 0.976 across layers, while the maximum activation probability among the remaining, non-top-k experts sits near 0.05 at p50.¹ A token's expert choice is therefore close to a fixed property of the token itself, learnable from an activation-frequency table alone, not something that has to be re-derived from context on every request.

That matters because EP's all-to-all is not a small cost even on fast hardware. Profiling one MoE layer's forward pass on an 8-GPU server with a specialized 400+ GB/s interconnect found EP communication accounting for up to 59.2% of that layer's latency, on hardware where the bottleneck is supposed to be compute, not the network.¹ The all-to-all communication volume itself follows directly from how much of it is local: volume = α·k·B·S/G for G devices, batch size B, sequence length S, and k experts activated per token, where α is the fraction of non-local activations.¹ Everything this page describes is aimed at minimizing α by construction, not by discovering it is already low.

When to use it (and when not)¶

Use it when serving a MoE model under expert parallelism where the all-to-all is a measured bottleneck, and where your traffic exhibits the same context-independent token-expert affinity the cited profiling found; verify this on your own model and traffic before assuming it holds, since the technique's entire benefit depends on it.
Use it as a complement to, not a replacement for, EPLB-style load balancing. MoE routing and expert load balancing solves "keep every expert's compute load even"; this page solves "keep tokens near their experts." The co-clustering objective explicitly carries a load-balance term alongside its locality term, precisely so the two goals are optimized together rather than fighting each other. See "Architecture," below.
Do not expect it to help models or traffic where routing is genuinely context-dependent and unpredictable. If a token's expert choice varies substantially with surrounding context (unlike the profiled models here), there is no stable affinity for the offline clustering step to exploit, and the technique degenerates toward whatever a load-balance-only placement would have done anyway.
Attention-TP and Attention-DP need different mechanisms, not one. Under DP, scheduling operates at request granularity (reorder whole requests to DP ranks); under TP, a single instance already sees the same input on every rank, so there is no request-level scheduler, the intervention is a token-level reshuffle fused into the existing communication collective instead. Pick the mechanism that matches your attention parallelism strategy, not the one that matches DP or TP alone.
Do not adopt it expecting a lossy shortcut. The cited implementation is explicitly a communication-volume optimization, not an approximation: routing decisions still ultimately follow the model's real gating function; the prediction only decides placement and scheduling, not which expert actually gets used.¹

Architecture¶

flowchart TB
  P["Offline: profile token-expert<br/>activation frequency"] --> T["Token-to-expert-group table,<br/>expert-group-sequence table,<br/>expert grouping table"]
  T --> M["Model scheduling:<br/>co-cluster + place experts by affinity,<br/>subject to a load-balance term"]
  M --> Q{"Attention parallelism?"}
  Q -->|"Data Parallelism"| DP["Inter-request scheduling:<br/>reorder requests onto the DP rank<br/>hosting their likely experts"]
  Q -->|"Tensor Parallelism"| TP["Intra-request scheduling:<br/>shuffled-reduce-scatter + shuffled-allgather<br/>fused into the post-attention communication"]
  DP --> A["Higher local activation rate<br/>-> smaller alpha -> less all-to-all volume"]
  TP --> A

The offline step formalizes expert placement and token/request routing as one joint optimization, not two: a token-to-cluster assignment R and an expert-to-cluster assignment C are chosen to minimize θ · (load-imbalance term) + (1-θ) · (cross-cluster activation term), subject to every token and every expert belonging to exactly one cluster and every cluster holding an equal number of experts.¹ The θ term is what keeps this from degenerating into "put everything in one cluster" (which would trivially maximize locality while destroying load balance); the two objectives are solved together because they trade against each other, and solving them separately, as prior expert-placement-only or token-routing-only schemes do, leaves communication savings on the table that a joint solve captures.¹ The exact 0-1 integer program is intractable to solve directly at scale, so the cited implementation uses an alternating-optimization heuristic instead of an exact solver.¹

For the Attention-TP case specifically, prediction is strengthened with inter-layer structure: which device an expert selection routes to at layer L depends on the device selected at the previous layer, modeled as a low-order (2-gram, in the cited implementation) Markov chain over device indices, not just the token's own static affinity.¹

How to use it: co-clustering experts by affinity, executed and cross-checked against random placement¶

The question this section answers by execution: does clustering experts by measured token-expert affinity actually raise the local activation rate over a naive, affinity-blind placement, at the same load-balance constraint, and does that advantage require the affinity structure to be real (matching the paper's own framing that this is a data property being exploited, not a free lunch)?

# semantic_parallelism_coscheduling.py -- validated: model-data co-scheduling for expert
# parallelism, reproducing the mechanism (not the literal reported percentages) behind
# model-data co-scheduling for EP: cluster experts by token-expert affinity, route tokens
# (or requests) to whichever cluster hosts their most-likely experts, and measure the
# resulting local activation rate (LAR) and all-to-all communication volume against a
# load-balance constraint and a naive (affinity-blind) placement baseline.
# Run: python3 semantic_parallelism_coscheduling.py
from __future__ import annotations

import numpy as np


def generate_affinity(n_tokens: int, n_experts: int, n_home: int, home_mass: float,
                      rng: np.random.Generator) -> np.ndarray:
    """affinity[token, expert] = probability this token activates this expert, matching the
    reported empirical pattern: each token has a small, STABLE set of `n_home` experts that
    together receive `home_mass` of its routing probability (paper: median cumulative hotness
    of the top-k experts ranges 0.833-0.976), with the remainder spread over the rest."""
    aff = np.zeros((n_tokens, n_experts))
    for t in range(n_tokens):
        home = rng.choice(n_experts, size=n_home, replace=False)
        aff[t, home] = home_mass / n_home
        cold = np.setdiff1d(np.arange(n_experts), home)
        aff[t, cold] = (1.0 - home_mass) / len(cold)
    assert np.allclose(aff.sum(axis=1), 1.0)
    return aff


def greedy_cocluster(affinity: np.ndarray, n_experts: int, n_clusters: int,
                      rng: np.random.Generator) -> np.ndarray:
    """Greedy expert-to-cluster assignment via pairwise expert co-activation affinity:
    expert_affinity[e1, e2] = how often the SAME token wants both experts. Seed each cluster
    with one expert (round-robin over a random order), then repeatedly assign whichever
    remaining (expert, cluster) pair has the highest total co-affinity to that cluster's
    current members, subject to each cluster holding exactly n_experts/n_clusters experts
    (the paper's hard load-balance constraint). A stand-in for the paper's own alternating-
    optimization heuristic (which replaces the intractable exact 0-1 ILP), not its real algorithm."""
    assert n_experts % n_clusters == 0
    cap = n_experts // n_clusters
    expert_affinity = affinity.T @ affinity                # [n_experts, n_experts], symmetric co-activation

    order = rng.permutation(n_experts)
    expert_to_cluster = -np.ones(n_experts, dtype=int)
    cluster_count = np.zeros(n_clusters, dtype=int)
    for j in range(n_clusters):                            # seed: one distinct expert per cluster
        expert_to_cluster[order[j]] = j
        cluster_count[j] = 1
    remaining = list(order[n_clusters:])
    while remaining:
        best_e, best_j, best_score = None, -1, -1.0
        for e in remaining:
            for j in range(n_clusters):
                if cluster_count[j] >= cap:
                    continue
                members = np.where(expert_to_cluster == j)[0]
                score = float(expert_affinity[e, members].sum())
                if score > best_score:
                    best_e, best_j, best_score = e, j, score
        expert_to_cluster[best_e] = best_j
        cluster_count[best_j] += 1
        remaining.remove(best_e)
    assert np.all(cluster_count == cap)
    return expert_to_cluster


def route_tokens(affinity: np.ndarray, expert_to_cluster: np.ndarray, n_clusters: int) -> np.ndarray:
    """Request/token-level scheduling: assign each token to the cluster holding the most of
    its own affinity mass (the paper's S_r = argmax_j sum_{i in r} R_ij, at token granularity)."""
    mass_per_cluster = np.zeros((affinity.shape[0], n_clusters))
    for j in range(n_clusters):
        mass_per_cluster[:, j] = affinity[:, expert_to_cluster == j].sum(axis=1)
    return np.argmax(mass_per_cluster, axis=1)


def local_activation_rate(affinity: np.ndarray, expert_to_cluster: np.ndarray,
                          token_to_cluster: np.ndarray, n_clusters: int) -> float:
    """Fraction of each token's activation probability mass that lands on an expert inside
    its OWN assigned cluster (no all-to-all needed); averaged over tokens, weighted equally."""
    local_mass = np.zeros(affinity.shape[0])
    for j in range(n_clusters):
        in_j = expert_to_cluster == j
        is_j = token_to_cluster == j
        local_mass[is_j] = affinity[is_j][:, in_j].sum(axis=1)
    return float(local_mass.mean())


def all2all_volume(lar: float, k: int, batch_tokens: int, n_devices: int) -> float:
    """The paper's own formula: alpha*k*B*S/G, here with B*S already flattened into
    `batch_tokens`; alpha is the REMOTE (non-local) fraction, 1 - LAR."""
    alpha = 1.0 - lar
    return alpha * k * batch_tokens / n_devices


if __name__ == "__main__":
    rng = np.random.default_rng(0)
    N_TOKENS, N_EXPERTS, N_CLUSTERS = 400, 64, 8      # 64 routed experts, EP degree 8 (paper's DeepSeek-V2-Lite scale)
    N_HOME, HOME_MASS = 3, 0.9                         # matches reported median cumulative top-k hotness ~0.9

    affinity = generate_affinity(N_TOKENS, N_EXPERTS, N_HOME, HOME_MASS, rng)

    # 1) Affinity-aware co-clustering must achieve a materially higher local activation rate
    #    than a naive, affinity-blind (random) expert placement using the SAME routing rule.
    aware_e2c = greedy_cocluster(affinity, N_EXPERTS, N_CLUSTERS, rng)
    aware_t2c = route_tokens(affinity, aware_e2c, N_CLUSTERS)
    lar_aware = local_activation_rate(affinity, aware_e2c, aware_t2c, N_CLUSTERS)

    random_e2c = rng.permutation(N_EXPERTS) % N_CLUSTERS      # a naive but perfectly balanced random placement
    random_t2c = route_tokens(affinity, random_e2c, N_CLUSTERS)
    lar_random = local_activation_rate(affinity, random_e2c, random_t2c, N_CLUSTERS)

    assert lar_aware > lar_random, (lar_aware, lar_random)
    assert lar_aware > 3.0 / N_CLUSTERS, "affinity-aware clustering should clear several times the no-structure floor 1/n_clusters"

    # 2) Load balance holds by construction: every cluster gets exactly N_EXPERTS/N_CLUSTERS
    #    experts, in both the aware and the random assignment (the paper's hard constraint).
    for e2c in (aware_e2c, random_e2c):
        counts = np.bincount(e2c, minlength=N_CLUSTERS)
        assert np.all(counts == N_EXPERTS // N_CLUSTERS), counts

    # 3) All-to-all communication volume drops in direct proportion to the LAR gain, using the
    #    paper's own volume formula (alpha*k*B*S/G with alpha = 1 - LAR).
    K_ACTIVE, BATCH_TOKENS = 6, 2048
    vol_aware = all2all_volume(lar_aware, K_ACTIVE, BATCH_TOKENS, N_CLUSTERS)
    vol_random = all2all_volume(lar_random, K_ACTIVE, BATCH_TOKENS, N_CLUSTERS)
    assert vol_aware < vol_random
    reduction = (vol_random - vol_aware) / vol_random

    # 4) Adversarial: strip the affinity structure down to uniform (every token equally likely
    #    to hit any expert, no home cluster at all) -- the co-clustering advantage MUST collapse,
    #    since there is no context-independent affinity left to exploit.
    uniform_affinity = np.full((N_TOKENS, N_EXPERTS), 1.0 / N_EXPERTS)
    e2c_u = greedy_cocluster(uniform_affinity, N_EXPERTS, N_CLUSTERS, rng)
    t2c_u = route_tokens(uniform_affinity, e2c_u, N_CLUSTERS)
    lar_u_aware = local_activation_rate(uniform_affinity, e2c_u, t2c_u, N_CLUSTERS)
    e2c_ur = rng.permutation(N_EXPERTS) % N_CLUSTERS
    t2c_ur = route_tokens(uniform_affinity, e2c_ur, N_CLUSTERS)
    lar_u_random = local_activation_rate(uniform_affinity, e2c_ur, t2c_ur, N_CLUSTERS)
    assert abs(lar_u_aware - lar_u_random) < 0.02, (lar_u_aware, lar_u_random)
    assert abs(lar_u_aware - 1.0 / N_CLUSTERS) < 0.02   # both collapse to the no-structure floor 1/n_clusters

    # 5) Monotonicity: a STRONGER home-mass concentration must raise the achievable LAR further.
    lars = []
    for hm in (0.5, 0.7, 0.9, 0.99):
        aff_hm = generate_affinity(N_TOKENS, N_EXPERTS, N_HOME, hm, rng)
        e2c_hm = greedy_cocluster(aff_hm, N_EXPERTS, N_CLUSTERS, rng)
        t2c_hm = route_tokens(aff_hm, e2c_hm, N_CLUSTERS)
        lars.append(local_activation_rate(aff_hm, e2c_hm, t2c_hm, N_CLUSTERS))
    assert all(a <= b + 1e-9 for a, b in zip(lars, lars[1:])), lars

    print(f"local activation rate, affinity-aware clustering = {lar_aware*100:.1f}%")
    print(f"local activation rate, affinity-blind (random)    = {lar_random*100:.1f}%")
    print(f"all-to-all volume reduction vs random placement   = {reduction*100:.1f}%")
    print(f"no-affinity floor (uniform routing): aware={lar_u_aware*100:.1f}% random={lar_u_random*100:.1f}% "
          f"(both collapse to 1/n_clusters = {100/N_CLUSTERS:.1f}%)")
    print(f"LAR rises monotonically with home-mass concentration: {[f'{l*100:.1f}%' for l in lars]}")
    print("ALL ASSERTIONS PASSED")

Executed output:

local activation rate, affinity-aware clustering = 49.2%
local activation rate, affinity-blind (random)    = 41.6%
all-to-all volume reduction vs random placement   = 13.0%
no-affinity floor (uniform routing): aware=12.5% random=12.5% (both collapse to 1/n_clusters = 12.5%)
LAR rises monotonically with home-mass concentration: ['31.0%', '40.2%', '47.6%', '51.6%']
ALL ASSERTIONS PASSED

Two things fall out of execution, not description. First, clustering by measured co-activation affinity, subject to the exact same balanced-cluster-size constraint as a naive placement, genuinely raises local activation and cuts communication volume in this synthetic setting, confirming the mechanism is real and not just a plausible-sounding story. Second, and just as important, that advantage is entirely contingent on the affinity structure actually existing: with a uniform (context-independent-affinity-free) token-expert distribution, affinity-aware and random clustering converge to the same result, the floor set purely by cluster count (1/n_clusters). This is the honest boundary of the technique: it exploits a real, measurable data property, and it buys nothing when that property is absent, exactly the caveat under "When to use it" above.

How to develop with it¶

Two integration paths, matching the two attention-parallelism strategies:

Attention-DP: request-level rescheduling. Aggregate each request's constituent tokens' predicted expert affinities into a single request-level cluster assignment, and reorder the request queue so requests land on the DP rank hosting their most-likely experts. Pair this with a workload-aware balancing pass so a burst of same-affinity requests does not skew load onto one rank during decode.¹
Attention-TP: fused token reshuffling. Replace the standard post-attention all-reduce with a shuffled-reduce-scatter that routes each token's activation toward its predicted expert's device before the MoE all-to-all runs, and a deferred shuffled-allgather afterward. The shuffle itself needs a fast argsort-style kernel; the cited implementation reports the shuffling logic adding roughly 1% overhead to the communication schedule, once implemented as a custom kernel rather than relying on generic gather/scatter.¹
Strengthen Attention-TP prediction with inter-layer structure. A token's expert choice at layer L is not independent of its choice at layer L-1; modeling that as a low-order Markov chain over device indices sharpens the scheduling decision beyond what a per-token, per-layer-independent affinity table alone can do.¹
Build on an existing engine's EP and communication stack, do not replace it. The technique is a scheduling layer in front of expert placement and token dispatch; it composes with, rather than substitutes for, the engine's own all-to-all backend (see expert parallelism for MoE inference for DeepEP and hierarchical all-to-all, which remain the mechanism moving the local fraction of traffic once co-scheduling has minimized α).

How to run it in production¶

Re-profile periodically, not once. The token-to-expert-group and expert-grouping tables are built from an offline activation profile; refresh them on a schedule, or after a material traffic-mix shift, the same discipline MoE routing and expert load balancing recommends for EPLB's own load-matrix rebuilds.
Watch local activation rate as the primary signal, not throughput alone. LAR (or its complement, α) is the metric that isolates whether co-scheduling is doing its job; a throughput regression with LAR still high points elsewhere in the stack, while a falling LAR with stable throughput is an early warning that traffic has drifted away from the profiled affinity pattern.
Validate the load-balance side, not just locality. A scheduler that maximizes local activation at the expense of load balance trades one bottleneck for another; track per-cluster load alongside LAR, mirroring the balancedness() metric MoE routing and expert load balancing already tracks for EPLB.

How to maintain it¶

Re-verify the context-independent-affinity assumption on every model swap. A new checkpoint, even a fine-tune, can shift routing behavior; the technique's entire benefit rests on this empirical property holding, so treat it as something to re-measure, not assume.
Track the DP-versus-TP mechanism split when the attention parallelism strategy changes. Migrating between Attention-DP and Attention-TP is not a configuration toggle for this technique; it changes which of the two scheduling mechanisms (request reordering vs. fused token reshuffling) applies.
Re-tune the load-balance-versus-locality tradeoff parameter after scaling EP degree. The balance between the co-clustering objective's two terms was tuned for a specific cluster count; changing the number of EP ranks changes both the achievable locality ceiling and the load-balance constraint's tightness.

Failure modes¶

Assuming the technique helps regardless of traffic. Where token-expert affinity is not actually context-independent and stable, the offline clustering step has nothing real to exploit, and the scheduler degenerates toward a load-balance-only placement; verify the affinity assumption on your own traffic first.
Optimizing locality at the expense of load balance. A clustering that ignores the load-balance term can trivially maximize local activation by collapsing everything into one cluster, at the cost of a straggler-bound MoE layer exactly as severe as the ones MoE routing and expert load balancing documents; the joint objective's balance term exists specifically to prevent this.
Treating Attention-DP's request-level scheduler as usable under Attention-TP. Under TP every rank sees the same input; there is no request-level decision to make, only the token-level reshuffle applies. Applying the wrong mechanism is a no-op at best.
Stale profiling. A rebuilt model or a materially different traffic mix invalidates the token-to-expert-group table; a stale table degrades toward random placement's locality, silently, without an obvious error.
Generic gather/scatter instead of a fused kernel. The reported low overhead (~1%) for token reshuffling assumes an optimized, purpose-built shuffle kernel; falling back to a naive implementation can turn a communication-reduction technique into a net communication addition.

Open questions & validation¶

Reproduce the local-activation-rate gain on your own model's real profiled token-expert affinity (not the synthetic structure used above) before trusting a specific percentage.
Whether the context-independent affinity property holds as strongly on models and traffic mixes not covered by the cited profiling; this is an empirical question per deployment, not a universal guarantee.
How this technique composes with WAN-scale, non-colocated expert placement; decentralized and distributed inference documents that even fast-fabric all-to-all mitigation is hard, and this page's mechanism has not been evaluated over a WAN link at all.
Whether the load-balance-versus-locality tradeoff parameter has a principled default, or must be re-tuned per model and per EP degree; the cited work treats it as a free parameter without a general rule for setting it.

References¶

Semantic Parallelism: Redefining Efficient MoE Inference via Model-Data Co-Scheduling (arXiv 2503.04398): https://arxiv.org/abs/2503.04398
Go and Mahajan, MoETuner: Optimized Mixture of Expert Serving with Balanced Expert Placement and Token Routing (arXiv 2502.06643): https://arxiv.org/abs/2502.06643
Yao et al., ExFlow: exploiting inter-layer expert affinity to reduce remote routing, cited as related work in the Semantic Parallelism paper
Zhao et al., DeepEP (expert-parallel communication library), integrated by the cited implementation for all-to-all: https://github.com/deepseek-ai/DeepEP
Zheng et al., SGLang (the inference engine the cited implementation is built on): https://github.com/sgl-project/sglang

Semantic Parallelism: Redefining Efficient MoE Inference via Model-Data Co-Scheduling, arXiv 2503.04398 (Yan Li, Zhenyu Zhang, Zhengang Wang, Pengfei Chen, Pengfei Zheng; Huawei Technologies and Sun Yat-Sen University). Profiling on DeepSeek-V2-Lite over a single MoE layer on an 8-GPU server with over 400 GB/s inter-GPU bandwidth found EP communication accounts for up to 59.2% of MoE-layer forward-pass latency. Token-expert affinity profiling on ShareGPT: median (p50) cumulative activation probability of a token's top-k experts ranges 0.833-0.976 across layers; maximum activation probability of non-top-k ("cold") experts sits near 0.05 at p50. All-to-all communication volume formula: alphakB*S/G (G devices, B batch size, S sequence length, k experts per token, alpha the non-local activation fraction). Three techniques: (1) offline model scheduling clustering co-activated experts by predicted affinity; (2) online inter-request scheduling for Attention-DP, reordering requests onto the DP rank hosting their predicted experts, with a workload-aware balancer for decode-stage load; (3) online intra-request scheduling for Attention-TP, replacing the post-attention all-reduce with a shuffled-reduce-scatter (SRS) and a deferred shuffled-allgather (SAG), strengthened by a 2-gram Markov model of inter-layer expert-device transitions. Model-data co-scheduling formalized as a 0-1 integer program minimizing a weighted sum of a load-imbalance term and a cross-cluster (remote) activation term, subject to balanced cluster sizes; solved via an alternating-optimization heuristic since the exact ILP is intractable at scale. Implementation: an SGLang plug-in, approximately 5,000 lines of Python plus custom Triton kernels; the shuffle's argsort-based kernel outperforms native PyTorch by 25%, and the overall shuffling overhead is approximately 1% of the communication schedule; integrates DeepEP for all2all. Evaluated on an 8-GPU server (96 GB HBM/GPU, >400 GB/s interconnect) against SGLang (with DeepEP) and MoETuner as baselines, on Qwen3-30B-A3B (128 experts/layer) and DeepSeek-V2-Lite (64 routed experts/layer), using MMLU, lmsys-chat-1m, and ShareGPT-Vicuna-unfiltered request traces. Reported results: local activation rate improved by 15.4% over the best baseline (36.7% over the weaker one) under an EP8 setting; on DeepSeek-V2-Lite, Attention-DP throughput improved up to 2.21x against SGLang and up to 2.78x against MoETuner under SLO constraints; Attention-TP TTFT improvements of 10.60-18.89% (DeepSeek-V2-Lite) and 3.80-24.90% (Qwen3-30B-A3B) across tested input lengths; local activation rate increases of 37% and 43% for the two models translating to 41.8% and 46.6% latency reduction of the affected expert layer. ↩↩↩↩↩↩↩↩↩↩↩↩↩↩