Kun Li

Zexuan Qiu

Tianhua Zhang

Irwin King

Helen Meng

Abstract

Sequential memory agents process long documents by reading chunks one after another while maintaining a compact memory state, coupling document traversal to reasoning depth. This coupling introduces sensitivity to evidence placement and ties inference latency linearly to document length. We introduce ParSer, which decouples reading from reasoning. A bank of lightweight subagents each bound to a single chunk read the entire document in parallel, while a lead agent reasons in depth through iterative scatter–gather rounds: at each round it broadcasts a query to all subagents, aggregates the returned evidence, and formulates a deeper follow-up query conditioned on what has been found so far. This decoupled design concentrates all learnable behavior in the lead agent, which is optimized with reinforcement learning, while the subagents remain frozen off-the-shelf models. On multi-hop QA with contexts ranging from 7K to 896K tokens, ParSer with a 4B backbone outperforms the strongest sequential memory baseline by 5.7 points on average and by 12.0 points at 896K tokens. Scaling to a 9B backbone, ParSer surpasses DeepSeek-V4-Pro by 6.3 points. Controlled experiments confirm that ParSer is robust to perturbations in evidence position, order, and distance, conditions that cause large accuracy swings in sequential methods, while reducing inference latency by up to $11\times$.

{li.kun, qzexuan, thzhang}@link.cuhk.edu.hk

1 Introduction

Reasoning over long documents is a core capability for large language models, yet remains challenging: in tasks such as multi-document QA or legal analysis, the evidence for a single question can be scattered across hundreds of thousands of tokens. Despite context windows now reaching a million tokens or more (DeepSeek-AI, 2026), model accuracy degrades as context grows, a phenomenon known as context rot (Hong et al., 2025). One well-documented manifestation is positional bias: evidence placed away from the boundaries of the input is systematically ignored (Liu et al., 2024). We observe the same on multi-hop QA, where direct full-context answering drops by tens of points as documents lengthen from $7$K to $896$K tokens, even for million-token models. Since accuracy degrades even when the window is far from full, the bottleneck is no longer capacity; the open question is how to select what to read and in what order to reason about it.

One response to the selection problem is the sequential memory paradigm: the document is read chunk by chunk, and at each step the agent compresses the current chunk together with its previous memory into an updated memory. The final answer is generated from this memory alone. MemAgent (Yu et al., 2026) and its follow-ups (Shi et al., 2026; Sheng et al., 2026) train this recurrent workflow end-to-end with reinforcement learning, handling documents of millions of tokens within a small context window. These methods address a genuinely hard problem: bounded working memory, arbitrary input length, and end-to-end trainability.

However, these memory-based methods share a structural commitment: the document is traversed exactly once, in order. This single-pass recurrent structure imposes two constraints: evidence must be evaluated through a repeatedly compressed prefix state, and every chunk update depends on the preceding one. The first leads to a sensitivity to evidence placement: because each chunk is compressed before the rest of the document has been seen, the agent judges relevance under a strict information deficit. In particular, its accuracy could be affected by the absolute position of evidence, the logical order among evidence pieces, and the distance between them. This sensitivity is most damaging in multi-hop reasoning, where the answer depends on scattered evidence whose relevance emerges only incrementally. The second leads to inference latency: since each chunk update depends on the output of the previous one, the $T$ steps form an irreducibly sequential chain whose wall-clock cost grows linearly with document length, regardless of available parallelism. Subsequent work in this sequential memory paradigm has largely been a sequence of patches to these two symptoms, often trading one for the other. Shi et al. (2026) adds a callback module that retrieves earlier memory states to counter position bias, at the price of extra retrieval on the sequential path; Sheng et al. (2026) adds gates that skip evidence-free chunks to save computation, but the sequential chain remains intact because the agent must still scan up to the last required evidence.

The order in which a long document is read is imposed by the document; the order in which a question is reasoned about is imposed by the question. Sequential memory agents let the first drive the second. We introduce ParSer (Parallel Reading, Sequential Reasoning), which decouples the two orders. Instead of tying sequential depth to document length, ParSer reads all chunks in parallel and reserves sequential computation only for the reasoning the question demands. To achieve this decoupling, ParSer assigns reading and reasoning to two separate roles. A lead agent never sees a single raw document token; it reasons about the question in a ReAct-style (Yao et al., 2023) loop of interleaved thinking and action. A bank of lightweight subagents (one per chunk) read only their own chunk and extract evidence for a given query. At each round the lead agent scatters a query to all subagents in parallel, then gathers their findings and decides what to ask next, iterating until it can commit to an answer.

This scatter–gather architecture changes the dependency structure of long-document processing. Sequential memory requires $T$ dependent updates in sequence; ParSer replaces this with $K$ scatter–gather rounds where all chunk-level calls execute in parallel, so latency scales with reasoning depth rather than document length. Furthermore, every chunk is re-read under a freshly formulated query at each round, eliminating the chunk-level position bias inherent in sequential traversal. Because no chunk is permanently discarded, the lead agent can condition each new query on previously discovered evidence, which is essential for multi-hop reasoning where a single static query cannot identify downstream hops (Zhou et al., 2024; Zhao et al., 2024; Xu et al., 2026). Finally, this decoupled design simplifies training: we train only the lead agent with reinforcement learning using the verifiable outcome reward (Shao et al., 2024), while the subagents remain frozen. Since each subagent’s task is simple (locate evidence for a pointed query in a short span), an off-the-shelf model suffices.

We evaluate ParSer on multi-hop long-context question answering, including the in-distribution HotpotQA (Yang et al., 2018) and the out-of-distribution 2WikiMultiHopQA (Ho et al., 2020), with context ranging from $7$K to $896$K tokens. On HotpotQA, ParSer with 4B and 9B backbones achieves average accuracies of $84.6\%$ and $86.8\%$ respectively, outperforming the strongest sequential memory baseline by $5.7$ and $6.7$ percentage points; at the longest setting ($896$K tokens) the gaps widen to $12.0$ and $9.9$ percentage points, as sequential methods degrade sharply with length while ParSer remains stable. Scaling to a 9B backbone, ParSer achieves an average of $86.8\%$, surpassing DeepSeek-V4-Pro (DeepSeek-AI, 2026), which natively supports a one-million-token context, by $6.3$ percentage points. Controlled experiments that independently perturb the absolute position, the logical order, and the relative distance of evidence within context confirm the source of this stability: sequential memory agents exhibit large accuracy swings as any of these factors changes, whereas ParSer remains nearly flat across all three conditions. On inference latency, parallel reading yields an $11\times$ reduction at $896$K tokens under single concurrency ($876$s vs. $78$s per sample relative to MemAgent) and maintains a $1.7\times$ advantage under a concurrency of $16$ ($102$s vs. $59$s).

2 Related Work

2.1 Long-Context LLMs

Supporting million-token contexts efficiently has driven two complementary lines of architectural work. Positional interpolation rescales rotary embeddings so a model trained on short sequences extrapolates to far longer ones (Chen et al., 2023b; Peng et al., 2024; Ding et al., 2024). A separate family of attention mechanisms attacks the quadratic cost that dominates at long context: sparse attention attends only to a learned subset of relevant tokens per query (DeepSeek-AI, 2025c; DeepSeek-AI, 2025b; MiniMax, 2026), and linear-attention variants reduce the complexity to linear in sequence length (Kimi Team, 2026). Yet a longer window does not by itself yield better use of that window. Models systematically underuse evidence placed in the middle of their input (Liu et al., 2024), and accuracy degrades as the input grows even when the nominal window is far from full, a phenomenon documented as context rot (Hong et al., 2025). A prominent response sidesteps the window limit altogether by reading the document in chunks while maintaining a compact textual memory that is repeatedly rewritten. Training-free methods precompute and then navigate such a memory. ReadAgent (Lee et al., 2024) uses gist lookup and MemWalker (Chen et al., 2023a) uses a summary tree, while Chain-of-Agents (Zhang et al., 2024) assigns one chunk per worker but passes a single message sequentially down the chain. MemAgent (Yu et al., 2026) instead trains this recurrent read-and-compress workflow end-to-end with reinforcement learning. ReMemR1 (Shi et al., 2026) adds a callback that revisits earlier memory states, and GRU-Mem (Sheng et al., 2026) introduces gated updates with an early-exit mechanism. These methods share one commitment: processing $T$ chunks requires $T$ dependent steps. Three consequences follow. Latency grows linearly with document length, a fixed-capacity memory must irreversibly decide what to retain before downstream relevance can be known, and the outcome depends on the order in which evidence is encountered (Gupta et al., 2026). §5.1 confirms that these are measurable biases with respect to evidence position, order, and separation.

2.2 Parallel Reading via Orchestrator–Worker Architectures

Reading chunks independently and aggregating their results is the natural parallel alternative to a sequential memory. Map-reduce pipelines such as LLM$\times$MapReduce (Zhou et al., 2024) and ToM (Guo et al., 2025) explore this direction, but most are single-shot: the query sent to each chunk is fixed before any chunk is read. This cannot handle multi-hop questions, where later hops are not recognizable until earlier ones are found (Xu et al., 2026). LongAgent (Zhao et al., 2024) and XpandA (Xiao et al., 2025) pair a leader with per-chunk agents over multiple rounds, but coordinate through hand-specified protocols rather than a learned policy. A separate group achieves parallelism inside the model by encoding chunks independently and fusing them at the attention level (Ratner et al., 2023; Merth et al., 2024; Ma et al., 2025; Yang et al., 2025; Yen et al., 2024), but these are query-agnostic, single-round, and require architecture surgery. Structurally, a leader dispatching subtasks to workers that each run in an isolated context window is by now a common pattern in agentic systems (Anthropic, 2025), since a worker’s intermediate tokens never occupy the leader’s context. A growing line of work trains only this orchestrator while keeping the workers frozen (Hu et al., 2025; Dang et al., 2025), a design echoed by commercial agent swarms that optimize the scheduler alone (Moonshot AI, 2025). ParSer is the long-context instantiation of this design. Its subagents are bound to a disjoint partition of the input, so coverage is guaranteed by construction and the lead agent’s task reduces to query formulation and aggregation. This structure makes freezing the subagents viable (§5.4) and keeps training cost independent of document length.

3 Method

3.1 Problem Formulation

For the task of long-context question answering (QA), an agent is required to give an answer $𝒂$ to the question $𝒒$, conditioned on a corresponding long document $𝑫$. The document $𝑫$ can be extremely long, such as hundreds of thousands of tokens or even more. Typically, due to the limited LLM context window, $𝑫$ is split into a set of fixed-size chunks $\left\{{𝒅}_0,{𝒅}_1,\dots ,{𝒅}_{T-1}\right\}$ for processing. To answer the question $𝒒$, the agent needs to accurately locate and then reason over a few pieces of key evidence, which are sparsely distributed within $𝑫$.

As illustrated in the upper panel of Figure 1, sequential memory methods (e.g., MemAgent, ReMemR1) formulate long-context reasoning as a sequential, recurrent, and chunk-by-chunk process: throughout the entire reasoning process, the agent maintains a textual memory, which stores key summaries of the chunks. The update operation relies on the question, the previous memory, and the current chunk to produce the updated memory. Consequently, the update operations for chunk ${𝒅}_t$ with $0<t<T$ must execute sequentially.

3.2 Workflow: Parallel Reading, Sequential Reasoning

Sequential memory turns document length into dependency depth: processing ${𝒅}_t$ cannot start until the memory from ${𝒅}_{\phantom{}<t}$ has been written. ParSer instead turns document length into parallel width. It assigns one subagent to each chunk and organizes their interaction with a lead agent through repeated Scatter-Gather rounds (the lower panel of Figure 1; Algorithm 1). At round $k$, the lead agent scatters one or more focused queries; all $T$ subagents inspect their respective chunks concurrently, and their local findings are gathered as the observation $ℛ$ for the lead agent. Each round therefore covers the entire document in parallel: increasing $T$ adds parallel readers rather than dependent reading steps.

媒体内容 · 前往原文查看

Parallel chunk readers. Each subagent is bound to one chunk and, given a lead-agent query, returns a finding grounded only in that chunk. Most chunks are irrelevant to a given query, so a subagent may abstain; abstentions are dropped during gathering. Appendix E.2 shows the subagent prompt. All $T$ subagents run concurrently at every round, so every chunk is read symmetrically under the same query: none is privileged by its index, and none is permanently discarded after a single pass. Binding each subagent to a short chunk also keeps its effective context compact, mitigating the context rot issue that arises as input length grows. Together with question decomposition, this makes the subagent’s reading task simpler; we thus let the subagents run in non-thinking mode. We deploy the subagents with SGLang (Zheng et al., 2024) and achieve parallel execution across all subagents through concurrent request dispatching.

Question-driven reasoner. The lead agent controls this parallel reading. It takes as input $𝒒$, but never $𝑫$ or any chunk ${𝒅}_t$, so it reasons about the question rather than the document. Appendix E.1 gives the lead-agent prompt. The lead agent conducts this reasoning in a multi-step ReAct (Yao et al., 2023) loop of interleaved thinking and action—after thinking, it either performs a scatter–gather operation or commits to a final answer. Queries in the latest round are conditioned on the reasoning history including previously gathered findings. Only this reasoning process is sequential; document-wide reading remains parallel in every round. The loop terminates when the lead agent answers or reaches the maximum number of rounds. The number of rounds actually executed, $K$, is determined by the reasoning hops required by $𝒒$ rather than the number of chunks $T$.

Parallel reading yields two immediate consequences. First, every chunk is inspected under the same query in the same round and can be revisited under a newly formulated query. Access to evidence is therefore symmetric with respect to chunk position. Second, parallel reading removes document coverage from the sequential critical path: the dependent reasoning depth is $K$ rounds rather than $T$ chunks, and long documents satisfy $T\gg K$, leading to lower wall-clock latency. On the other hand, parallelizing the readers raises two natural concerns: whether independent chunk reading undermines cross-chunk dependencies, and whether repeatedly querying all chunks incurs excessive computation.

Adaptive parallel reading across rounds. Although processing chunks independently prevents each subagent from observing relations that span multiple chunks, ParSer does not ask subagents to solve the original multi-hop question. The lead agent decomposes it into specific queries whose relevant findings can typically be established independently within individual chunks. Instead, dependencies that span chunks at the task level are carried across reasoning rounds: findings gathered at round $k$ enter the lead agent’s context and condition the query at round $k+1$. Cross-chunk dependencies are thus resolved through successive parallel reading rounds. ParSer relocates such composition from document-ordered memory propagation to the question-driven query chain.

Efficiency through sparsity. The same decomposition also makes subagent communication sparse. For each query, only a small number of subagents return findings, while most emit only a short abstention and their responses are dropped before findings aggregation. Sequential memory methods, in contrast, generate a memory update (with hundreds or thousands of tokens) after every chunk. While multi-round reading may preserve or increase prefill computation (depending on whether KV-cache reuse is available), ParSer reduces decoding computation through substantially fewer generated tokens. Appendix A gives the computation and latency analysis.

Taken together, ParSer separates query-conditioned local reading from evidence-conditioned global reasoning. Cross-chunk dependencies are composed through successive reasoning rounds without reintroducing excessive computation. Crucially, this design converts document length from sequential depth into parallel width: the critical path scales with reasoning complexity rather than document length.

3.3 Optimization: Agentic Reinforcement Learning

The decoupling of reading from reasoning also determines what we train. Each subagent locates evidence for a pointed query in a short chunk—a task simple enough that a frozen off-the-shelf model already suffices. What still has to be learned is the lead agent’s policy: how to determine the next action based on reasoning history. We therefore train only the lead agent and keep the subagents frozen.

We optimize the lead agent with Reinforcement Learning with Verifiable Reward (RLVR; DeepSeek-AI 2025a). The reward is a binary exact-match score $r(𝒒,𝒚)=\text{EM}({𝒂}_{pred},{𝒂}_{gold})$, where ${𝒂}_{pred}$ is the answer extracted from the reasoning trajectory $𝒚$ and ${𝒂}_{gold}$ is the ground truth. We do not use format rewards, as the lead agent uses the backbone’s native multi-turn tool-calling format.

With this reward, we use Group Relative Policy Optimization (GRPO; Shao et al. 2024) to maximize

${𝒥}_{\mathrm{GRPO}}\left(\theta \right)={𝔼}_{\left\{y{}_i\right\}{}_{i=1}{}^G\sim \pi {}_{\text{old}}(\cdot \mid x;𝒮(𝑫\left)\right)}[\frac{1}{G}\sum _{i=1}^G\frac{1}{\left|y_i\right|}\sum _{t=1}^{\left|y_i\right|}\min ({\rho }_{i,t}\left(\theta \right){\hat{A}}_{i,t},\mathrm{clip}({\rho }_{i,t}\left(\theta \right),1-ϵ,1+ϵ){\hat{A}}_{i,t})-\beta {𝔻}_{\mathrm{KL}}[{\pi }_{\theta }\parallel {\pi }_{\text{ref}}]]$

(1)

where ${\rho }_{i,t}\left(\theta \right)=\frac{{\pi }_{\theta }(y_{i,t}\mid x,y_{i,\phantom{}<t};𝒮\left(𝑫\right))}{{\pi }_{\text{old}}(y_{i,t}\mid x,y_{i,\phantom{}<t};𝒮\left(𝑫\right))}$ is the importance ratio, $𝒮\left(𝑫\right)$ denotes the frozen subagents bound to the chunks of $𝑫$, $ϵ$ is the PPO clipping hyperparameter, $\beta$ is the KL regularization coefficient, and ${\hat{A}}_{i,t}$ denotes the advantage computed from the relative rewards of outputs in each group. We mask observation tokens, the findings gathered from $𝒮\left(𝑫\right)$, so the policy gradient is applied only to tokens generated by the lead agent.

4 Experiments

4.1 Implementation

Following prior work (Yu et al., 2026; Shi et al., 2026), we utilize multi-hop long-context question answering tasks for our training. We synthesized $\mathrm{32,768}$ training samples from the HotpotQA (Yang et al., 2018) dataset by following Yu et al. (2026)’s recipe, and each synthetic sample has a context composed of 200 paragraphs, with a total token length of $\phantom{}\sim 28$K tokens. More details of sample synthesis can be found in Appendix C.

We choose Qwen3.5-4B and Qwen3.5-9B (Qwen, 2026) as backbone models. During training, we impose an upper bound of 9 total turns, i.e., $K\le 9$; each turn is capped at 2048 tokens. The document is chunked into at most 512 tokens per chunk. The subagents use a temperature of 0.7 and a 512-token generation budget. To streamline the aggregation of findings from subagents, we instruct the subagents to structure their output in JSON format. At inference, we increase the cap of $K$ to 12 and the chunk size to $\mathrm{4,096}$ tokens¹¹ 1 To pursue faster training, we intentionally select a smaller chunk size for the training phase, despite the resulting mismatch with inference-time chunk sizes. As given in Equation (2), the prefill cost $𝒪\left(n^2/c\right)$ decreases as the number of chunks $c$ grows..

We optimize the lead agent using a learning rate of $1e-6$, a mini-batch size of 128, and 70 warm-up steps. We apply a PPO clipping with $ϵ=0.2$, and KL regularization with $\beta =1e-3$. The group size of rollouts is set to 5. We train ParSer on top of VERL (Sheng et al., 2025) framework with Megatron backend training and SGLang (Zheng et al., 2024) rollout service. For superior training efficiency, we adopt a fully asynchronous RL setting: all training runs on 6 H100 GPUs; 4 GPUs serve rollouts and 2 GPUs train the actor. We use 10 additional H100 GPUs to deploy subagents with SGLang. We let Qwen3.5-4B serve as the subagent for both 4B and 9B lead agents. More details of training and inference are in Appendix E.4 and E.6, respectively.

4.2 Baselines & Evaluation

We compare our method against two categories of baselines: (1) Long-context LLMs, including Qwen3.5 (Qwen, 2026) and DeepSeek-V4-Pro (DeepSeek-AI, 2026), which take the question together with the entire associated document as input to generate direct answers. DeepSeek-V4-Pro (preview, 2026-04-24) supports 1M-token context, and we set its reasoning effort mode as Max, the highest level of reasoning effort of this model. For Qwen3.5, we use the models’ default configuration without context extension for evaluations within 262K tokens; furthermore, to support evaluations beyond the models’ native 262K-token context window, we apply YaRN (Peng et al., 2024)-based RoPE scaling with a factor of 4.0, extending its context window to approximately 1M tokens. The temperature for them is set to $0$. (2) Sequential memory agents, such as MemAgent (Yu et al., 2026) and ReMemR1 (Shi et al., 2026). To ensure a fair comparison, we reimplement the sequential memory agent baselines under configurations identical to our approach, covering training data and backbone models, while reusing the training hyperparameters in the original papers.

For evaluation, we use the in-distribution HotpotQA (Yang et al., 2018) and the out-of-distribution 2WikiMultiHopQA (Ho et al., 2020). We use the HotpotQA test samples released by Yu et al. (2026), and regenerate the 2WikiMultiHopQA ones with Shi et al. (2026)’s public script (Appendix C). To enable comprehensive evaluations across diverse document lengths, the documents of the test samples have lengths ranging from $7$K to $896$K tokens. Following Yu et al. (2026) and Shi et al. (2026), we report Sub_EM as the evaluation metric. For each training method, we select the checkpoint that achieves the best in-distribution overall performance and report the average score over 3 runs.

4.3 Main Results

媒体内容 · 前往原文查看

(a) Accuracy on HotpotQA (In-Distribution)
Backbone	Method	# Paragraphs (Total Length)								Avg.
		50 (7K)	100 (14K)	200 (28K)	400 (56K)	800 (112K)	1600 (224K)	3200 (448K)	6400 (896K)
DeepSeek-V4-Pro	Full-context (non-think)	78.12	77.34	76.56	79.69	77.34	75.78	73.44	62.50	75.10
	Full-context (think-max)	82.03	82.81	80.47	80.47	80.47	81.25	77.34	78.91	80.47
Qwen3.5-4B	Full-context (non-think)	75.78	75.78	71.88	74.22	67.97	61.72	53.13	34.38	64.36
	Full-context (think)	80.47	78.91	78.12	76.56	74.22	60.94	46.09	31.25	65.82
	MemAgent	81.25	81.51	83.59	77.86	77.86	79.69	73.96	72.92	78.58
	ReMemR1	82.03	79.95	82.03	78.91	77.86	79.95	77.08	73.44	78.91
	ParSer	85.68	84.64	83.60	85.68	85.42	83.07	83.07	85.42	84.57
Qwen3.5-9B	Full-context (non-think)	75.00	72.66	72.66	71.88	70.31	65.62	58.59	47.66	66.80
	Full-context (think)	77.34	74.22	78.91	76.56	77.34	65.62	53.91	46.88	68.85
	MemAgent	81.77	80.73	82.03	79.95	79.95	81.25	79.69	75.00	80.05
	ReMemR1	81.25	79.69	77.60	78.39	78.13	78.13	77.86	76.04	78.39
	ParSer	86.72	88.80	87.24	85.68	86.46	86.72	86.72	85.94	86.79
(b) Accuracy on 2WikiMultiHopQA (Out-of-Distribution)
Backbone	Method	# Paragraphs (Total Length)								Avg.
		50 (7K)	100 (14K)	200 (28K)	400 (56K)	800 (112K)	1600 (224K)	3200 (448K)	6400 (896K)
DeepSeek-V4-Pro	Full-context (non-think)	89.84	86.72	87.50	87.50	87.50	77.34	78.91	68.75	83.01
	Full-context (think-max)	90.62	89.06	92.19	90.62	92.19	87.50	82.81	80.47	88.18
Qwen3.5-4B	Full-context (non-think)	85.94	85.94	82.81	78.12	74.22	69.53	66.41	46.88	73.73
	Full-context (think)	88.28	88.28	88.28	83.59	82.81	70.31	63.28	39.06	75.49
	MemAgent	67.45	73.18	69.01	61.46	54.17	54.43	60.16	45.05	60.61
	ReMemR1	88.54	86.85	83.20	83.07	76.83	67.71	72.40	60.68	77.41
	ParSer	86.98	84.64	88.28	87.76	87.50	85.94	88.28	86.98	87.04
Qwen3.5-9B	Full-context (non-think)	80.47	82.81	81.25	80.47	74.22	78.91	61.72	50.00	73.73
	Full-context (think)	88.28	89.06	92.19	85.16	89.06	82.81	57.81	52.34	79.59
	MemAgent	80.73	82.03	78.91	75.26	70.57	65.63	75.78	60.94	73.73
	ReMemR1	84.12	87.50	77.87	83.59	76.30	74.74	79.43	70.57	79.27
	ParSer	87.76	88.80	88.80	89.06	87.24	89.58	88.54	88.02	88.48

Table 1 presents all methods’ results on two benchmarks. ParSer consistently demonstrates higher accuracy than all other baselines across all subsets on HotpotQA and long-document subsets (with $\phantom{}\ge 1600$ paragraphs) on 2WikiMultiHopQA. For the three long-context LLMs, the baselines that incorporate full documents as input mostly suffer rapid performance degradation as the document length increases. While those sequential memory agent methods can alleviate this issue to some extent by storing salient information in a memory buffer, ParSer exhibits nearly zero performance variation across different document lengths on both benchmarks, achieving the best performance consistency.

Among the training-based methods, MemAgent and ReMemR1 perform far worse under out-of-distribution conditions than under in-distribution scenarios; in contrast, ParSer maintains favorable performance on out-of-distribution cases. We attribute this robustness to our decoupling of reading from reasoning: only the lead agent’s reasoning policy is trained, while the subagents responsible for document reading remain frozen. Because the lead agent never directly sees the document, its training develops general question-reasoning capabilities rather than learning to generate document-specific summaries, as memory-based baselines do. This decoupled reading-and-reasoning strategy thereby helps mitigate overfitting to the training documents.

5 Analysis

We compare ParSer with and without RL in Appendix B, confirming that RL is an effective complement to the ParSer workflow. In this section, we also conduct experiments to investigate the following research questions: (1) Why does ParSer’s parallel paradigm outperform memory-based approaches? (2) How efficient is ParSer in terms of inference latency? (3) How do subagent model size and document chunking affect ParSer? (4) Can ParSer accommodate alternative subagent implementations? All experiments in this section are conducted with Qwen3.5-4B, unless otherwise stated.

5.1 Why P AR S ER Outperforms Sequential Memory Agents

To understand why ParSer outperforms sequential memory agents, we construct three controlled evaluations that vary complementary aspects of evidence distribution: absolute position, logical order, and relative distance. Rather than comparing performance across methods, we focus on each method’s sensitivity to perturbations applied to these three dimensions.

媒体内容 · 前往原文查看

Evidence Position Control

We manipulate the absolute position of supporting evidence within long documents. Specifically, for each of 512 test questions sampled from HotpotQA, we place all evidence-bearing paragraphs at randomly sampled positions within the $\left[st,st+10\right]$ percentile range of an 894K-token document, where $st$ ranges from 0 to 90 in increments of 10. The distractor paragraphs and their positions remain identical across variants. Figure 2(a) compares the three methods across the resulting position-controlled test sets. While ParSer remains stable across evidence locations, MemAgent suffers a pronounced performance drop when the supporting evidence lies between the 50th and 70th percentiles of the document. MemAgent sequentially compresses document chunks into a fixed-capacity memory, making evidence availability dependent on where the evidence enters the memory-update sequence. Specifically, when all pieces of evidence appear near the beginning, MemAgent can aggregate them and derive an answer early; evidence near the end undergoes few subsequent memory updates. Evidence in the middle is therefore more susceptible to being overwritten. ReMemR1 partially mitigates this positional sensitivity by retrieving information from earlier memory states through its callback module. In contrast, ParSer gives every chunk symmetric access to the same query in each parallel reading round, making evidence retrieval independent of absolute document position.

Evidence Order Control

We sample 512 two-hop bridge-comparison questions from 2WikiMultiHopQA²² 2 We use 2WikiMultiHopQA because, unlike HotpotQA, it provides ground-truth annotations of the logical order among pieces of supporting evidence.. In this question type, the supporting evidence forms a logical reasoning chain in which later hops depend on entities identified in earlier hops. For example, answering the question in Figure 10 requires first identifying the director of each film and then comparing the directors’ dates of death. For each question, we construct two 894K-token documents containing the same set of paragraphs, with all distractor paragraphs kept in the same positions. The documents differ only in the relative order of their evidence-bearing paragraphs: one follows the logical dependency order, whereas the other reverses it. Figure 2(b) compares performance under the two evidence orders. Both MemAgent and ReMemR1 suffer substantial performance degradation when the order is reversed, whereas ParSer remains stable. In sequential memory agents, when an evidence paragraph appears before the prerequisite evidence needed to recognize its relevance, it may be omitted from memory or evicted before that relevance becomes apparent (see a MemAgent example in Appendix F.3). By contrast, ParSer revisits all chunks under queries conditioned on previously gathered findings, allowing the lead agent to follow the question’s logical dependencies independently of the evidence’s physical order in the document.

Evidence Distance Control

We further investigate how the distance between pieces of supporting evidence affects performance. We sample 512 questions requiring two pieces of supporting evidence from 2WikiMultiHopQA. For each question, we vary their separation by inserting different numbers of distractor paragraphs between the two evidence-bearing paragraphs. To reduce confounding factors identified in the two preceding experiments, we place the two evidence paragraphs following their logical order and pad a fixed set of $\mathrm{1,600}$ paragraphs both before the first evidence paragraph and after the final evidence paragraph. As shown in Figure 2(c), the performance of MemAgent and ReMemR1 deteriorates as the number of middle paragraphs increases, whereas ParSer remains stable. In sequential memory agents, the first piece of evidence must survive an increasing number of memory updates before the second is encountered, making it more likely to be evicted from the fixed-capacity memory given a large number of middle paragraphs. In ParSer, the two evidence-bearing chunks can instead be examined independently during parallel reading, and their findings can be composed by the lead agent across reasoning rounds; the required reasoning path therefore does not grow with their physical distance.

Taken together, these experiments expose three manifestations of the same structural bottleneck in sequential memory agents. All three failure modes arise from capacity-limited, document-ordered recurrent compression. In contrast, at each reasoning round, ParSer allows all subagents to inspect their respective chunks concurrently under the same question-conditioned query and to revisit them as the query evolves. This position-symmetric access decouples logical reasoning from the physical position, order, and distance of evidence, accounting for ParSer’s robustness across all three controls.

5.2 Inference Latency

媒体内容 · 前往原文查看

Concurrency	Method	# Paragraphs (Total Length)
		50 (7K)	100 (14K)	200 (28K)	400 (56K)	800 (112K)	1600 (224K)	3200 (448K)	6400 (896K)
1	Full-context (non-think)	1.16	1.86	2.11	3.52	6.25	18.36	51.86	185.42
	Full-context (think)	12.33	17.78	18.93	31.89	42.73	68.19	162.97	375.32
	MemAgent	10.56	15.99	30.40	58.49	112.59	222.86	438.61	876.20
	ParSer	5.33	5.84	6.83	8.71	13.30	22.43	42.19	78.22
16	Full-context (non-think)	0.63	0.91	1.11	2.55	9.46	28.00	94.22	–
	Full-context (think)	2.00	2.84	4.05	8.70	21.37	53.95	177.01	–
	MemAgent	1.49	2.28	3.76	7.13	13.61	26.24	51.80	101.94
	ParSer	0.87	1.09	1.83	3.30	7.38	15.24	31.31	58.86
32	Full-context (non-think)	0.35	0.65	1.07	3.85	9.23	27.98	93.45	–
	Full-context (think)	1.70	2.25	3.39	9.32	20.64	54.16	191.91	–
	MemAgent	1.21	1.63	2.76	5.14	9.76	19.61	37.50	74.96
	ParSer	0.64	0.99	1.89	3.66	7.36	15.40	30.77	58.76

Table 2 reports three methods’ amortized wall-clock inference time across all test subsets of HotpotQA with concurrency of 1, 16, and 32³³ 3 We exclude ReMemR1 from this analysis because it equips MemAgent with an extra retrieval module, which theoretically introduces higher inference latency than MemAgent.. Each entry is the total subset wall-clock time divided by the size of the subset (128), rather than the end-to-end latency of an individual request (which would typically increase under higher concurrency due to contention). Models are deployed with SGLang: we allocate one NVIDIA H100 GPU for Full-context and MemAgent, while ParSer employs two GPUs, one H100 dedicated to all subagents and one RTX3090 for the lead agent. For ParSer, the lead agent and subagents within a single inference instance run in an alternating fashion, as the lead agent has to await outputs from all subagents. Comparisons between our two-GPU ParSer and single-GPU baselines are thus valid.

Across all concurrency settings, Full-context (non-thinking) achieves the lowest latency on short-document subsets because it processes the input in a single pass and generates only a short output. As document length increases, however, its single-pass processing becomes increasingly costly due to the quadratic complexity of attention module. Under high concurrency, limited GPU memory even prevents it from processing 896K-token inputs. In contrast, MemAgent and ParSer process documents in fixed-length chunks, enabling them to handle longer documents within limited GPU memory.

媒体内容 · 前往原文查看

Figure 3 compares the inference step counts of MemAgent and ParSer, which help explain their latency trends. At a concurrency of 1, MemAgent’s amortized latency grows linearly with document length because its number of inference steps is proportional to the number of document chunks. In contrast, ParSer processes document chunks in parallel, reducing the number of sequential inference steps to the number of reasoning hops required to solve each question itself. Because the same set of questions is used across all subsets, the number of reasoning hops—and hence the inference step count of ParSer —remains nearly constant as document length increases. Consequently, ParSer achieves substantially lower amortized latency than MemAgent, with an order-of-magnitude advantage on long-document subsets. Under multi-concurrency requests, a more practical condition, MemAgent benefits from batched processing and narrows the latency gap. Nevertheless, ParSer consistently maintains lower amortized time across the evaluated concurrency levels. Appendix A explains why this advantage becomes less pronounced at higher concurrency.

5.3 Effect of Subagent

媒体内容 · 前往原文查看

Subagent	# Paragraphs (Total Length)								Avg.
	50 (7K)	100 (14K)	200 (28K)	400 (56K)	800 (112K)	1600 (224K)	3200 (448K)	6400 (896K)
Qwen3.5-2B	82.55	84.38	82.29	84.11	77.87	76.56	69.01	69.27	78.26
Qwen3.5-4B (default)	85.68	84.64	83.60	85.68	85.42	83.07	83.07	85.42	84.57
Qwen3.5-9B	82.81	85.16	87.50	84.38	84.38	85.16	82.81	85.94	84.77

Subagent Size

In addition to Qwen3.5-4B used by default, we add 2B and 9B models as alternative subagents without retraining the lead agents. According to Table 3, performance improves when replacing the 2B subagent with the 4B version, and then saturates when employing an even larger subagent (9B). We attribute this saturation to the simplification of the subagents’ task. After question decomposition and document chunking, each subagent only needs to answer a focused query over a short context, for which the 4B model already provides sufficient capacity. This highlights the deployment efficiency of ParSer via the adoption of lightweight subagents.

媒体内容 · 前往原文查看

Chunk size	# Paragraphs (Total Length)								Avg.
	50 (7K)	100 (14K)	200 (28K)	400 (56K)	800 (112K)	1600 (224K)	3200 (448K)	6400 (896K)
4096	85.68	84.64	83.60	85.68	85.42	83.07	83.07	85.42	84.57
Full	83.85	84.64	82.29	83.86	74.48	71.10	56.77	53.13	73.76
16,384	82.03	82.29	85.94	82.81	82.81	83.86	85.42	83.59	83.59
65,536	84.64	82.03	80.73	80.73	78.91	80.99	83.59	83.08	81.84
131,072	83.07	83.59	80.99	80.21	75.78	80.21	79.69	79.17	80.34

Chunk Size

The subagents of ParSer use the same working mode as the Full-context (non-thinking) baseline introduced in §4.2, but receive inputs at a different document granularity (document chunk vs. full document). Thus, the improvement of ParSer over Full-context (non-thinking) in Table 1 can be attributed to both the lead agent’s guidance and the chunk-level document decomposition. To isolate their respective contributions, we introduce an ablated ParSer variant without chunking, which employs a single subagent fed with the full document. Table 4 reports a notable performance drop upon the removal of chunking, particularly on long-document subsets. We also add variants with intermediate chunk sizes and observe that performance drops as chunk size increases. This suggests that key information contained in longer input is more difficult for LLMs to capture than in shorter input, consistent with the context rot phenomenon observed in prior work (Hong et al., 2025; Liu et al., 2024). To mitigate this issue, ParSer instantiates multiple subagents, and each subagent processes only a short chunk of the document.

5.4 Compatibility with Alternative Subagent Implementations

媒体内容 · 前往原文查看

Agent	# Paragraphs (Total Length)								Avg.
	50 (7K)	100 (14K)	200 (28K)	400 (56K)	800 (112K)	1600 (224K)	3200 (448K)	6400 (896K)
DCI agent	73.96	75.78	74.74	76.56	74.48	75.52	77.60	76.30	75.61
Lead agent + DCI subagents	86.72	88.28	89.06	86.72	85.94	80.47	81.25	78.91	84.67
Full-context (think)	80.47	78.91	78.12	76.56	74.22	60.94	46.09	31.25	65.82
Lead agent + thinking subagents	85.16	86.72	87.50	82.03	85.94	85.94	87.50	83.59	85.55

ParSer defaults to direct-answer subagents. We investigate whether the lead agent is compatible with alternative subagent implementations without retraining. We add two additional subagent variants: the thinking subagents and the DCI subagents. DCI (Direct Corpus Interaction; Li et al. 2026, Salemi et al. 2026, Sen et al. 2026) refers to an agentic search paradigm that enables LLMs to directly query raw text corpora via composable Unix shell tools like rg and grep for fine-grained lexical matching and multi-step evidence collection. We follow the implementation of Li et al. (2026) to build our DCI subagents, whose implementation details are presented in Appendix D.2. As shown in Table 5, both variants outperform their corresponding standalone baselines: ParSer with thinking subagents improves over Full-context (think), while ParSer with DCI subagents improves over the standalone DCI agent. These results suggest that the lead agent can effectively coordinate different subagent implementations.

6 Conclusion

We presented ParSer, a scatter–gather architecture for long-context reasoning that fundamentally decouples reading from reasoning. Instead of relying on a sequential traversal, ParSer distributes document chunks to a bank of lightweight subagents that read in parallel. Meanwhile, a central lead agent iteratively scatters queries, gathers the returned evidence, and refines its search over multiple rounds. This decoupled design dictates an efficient training paradigm: because locating evidence in short chunks requires minimal adaptation, the subagents stay frozen, and only the lead agent’s reasoning policy is optimized with reinforcement learning from a verifiable outcome reward alone. On HotpotQA and 2WikiMultiHopQA with contexts from $7$K to $896$K tokens, ParSer with 4B and 9B backbones outperforms the strongest sequential memory baselines by $5.7$ and $6.7$ points on average. The performance gap remarkably widens to $12.0$ and $9.9$ points at the extreme $896$K setting, all while cutting inference latency by up to $11\times$. Its stable length-invariant accuracy and its absolute robustness against changes in evidence position, order, and distance confirm our core insight: effective long-context reasoning requires dynamically refining queries across the full text rather than relying on capacity-limited sequential memory updates.

References

• Anthropic (2025) Anthropic How we built our multi-agent research system. Note: Anthropic Engineering Blog External Links: Link Cited by: §2.2.
• Chen et al. (2023a) H. Chen, R. Pasunuru, J. Weston, and A. Celikyilmaz Walking down the memory maze: beyond context limit through interactive reading. External Links: 2310.05029, Link Cited by: §2.1.
• Chen et al. (2023b) S. Chen, S. Wong, L. Chen, and Y. Tian Extending context window of large language models via positional interpolation. External Links: 2306.15595, Link Cited by: §2.1.
• Dang et al. (2025) Y. Dang, C. Qian, X. Luo, J. Fan, Z. Xie, R. Shi, W. Chen, C. Yang, X. Che, Y. Tian, X. Xiong, L. Han, Z. Liu, and M. Sun Multi-agent collaboration via evolving orchestration. In Advances in Neural Information Processing Systems (NeurIPS), Note: arXiv:2505.19591 Cited by: §2.2.
• DeepSeek-AI (2025a) DeepSeek-AI DeepSeek-r1: incentivizing reasoning capability in llms via reinforcement learning. External Links: 2501.12948, Link Cited by: §3.3.
• DeepSeek-AI (2025b) DeepSeek-AI DeepSeek-v3.2: pushing the frontier of open large language models. External Links: 2512.02556, Link Cited by: §2.1.
• DeepSeek-AI (2025c) DeepSeek-AI Native sparse attention: hardware-aligned and natively trainable sparse attention. External Links: 2502.11089, Link Cited by: §2.1.
• DeepSeek-AI (2026) DeepSeek-AI DeepSeek-v4: towards highly efficient million-token context intelligence. Cited by: §D.1, §1, §1, §4.2.
• Ding et al. (2024) Y. Ding, L. L. Zhang, C. Zhang, Y. Xu, N. Shang, J. Xu, F. Yang, and M. Yang LongRoPE: extending LLM context window beyond 2 million tokens. In Proceedings of the 41st International Conference on Machine Learning, Cited by: §2.1.
• Guo et al. (2025) J. Guo, Z. Li, J. Wu, Q. Wang, Y. Li, L. Zhang, H. Zhao, and Y. Yang ToM: leveraging tree-oriented mapreduce for long-context reasoning in large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Note: arXiv:2511.00489 Cited by: §2.2.
• Gupta et al. (2026) N. Gupta, V. Singh, A. Iyer, K. Shiragur, P. Grover, R. B. Bairi, R. Maiti, S. Damle, S. M. Gupta, R. Maurya, and V. D. C Chow-liu ordering for long-context reasoning in chain-of-agents. External Links: 2603.09835, Link Cited by: §2.1.
• Ho et al. (2020) X. Ho, A. Duong Nguyen, S. Sugawara, and A. Aizawa Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps. In Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Spain (Online), pp. 6609–6625. External Links: Link Cited by: Table 7, Table 7, §C.2, §1, §4.2, Table 1, Table 1.
• Hong et al. (2025) K. Hong, A. Troynikov, and J. Huber Context rot: how increasing input tokens impacts llm performance. Technical report Chroma. External Links: Link Cited by: §1, §2.1, §5.3.
• Hsieh et al. (2024) C. Hsieh, S. Sun, S. Kriman, S. Acharya, D. Rekesh, F. Jia, Y. Zhang, and B. Ginsburg RULER: what’s the real context size of your long-context language models?. In First Conference on Language Modeling (COLM), Note: arXiv:2404.06654 Cited by: §C.1.
• Hu et al. (2025) M. Hu, Y. Zhou, W. Fan, Y. Nie, B. Xia, T. Sun, Z. Ye, Z. Jin, Y. Li, Q. Chen, Z. Zhang, Y. Wang, Q. Ye, B. Ghanem, P. Luo, and G. Li OWL: optimized workforce learning for general multi-agent assistance in real-world task automation. External Links: 2505.23885, Link Cited by: §2.2.
• Jin et al. (2024) J. Jin, Y. Zhu, X. Yang, C. Zhang, and Z. Dou FlashRAG: a modular toolkit for efficient retrieval-augmented generation research. CoRR abs/2405.13576. External Links: Link, 2405.13576 Cited by: §C.2.
• Kimi Team (2026) Kimi Team Kimi k3: open frontier intelligence. External Links: 2607.24653, Link Cited by: §2.1.
• Lee et al. (2024) K. Lee, X. Chen, H. Furuta, J. Canny, and I. Fischer A human-inspired reading agent with gist memory of very long contexts. In International Conference on Machine Learning (ICML), Note: arXiv:2402.09727 Cited by: §2.1.
• Li et al. (2025) X. Li, Z. Yu, Z. Zhang, X. Chen, Z. Zhang, Y. Zhuang, N. Sadagopan, and A. Beniwal When thinking fails: the pitfalls of reasoning for instruction-following in LLMs. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, External Links: Link Cited by: §D.1.
• Li et al. (2026) Z. Li, H. Zhang, C. Wei, P. Lu, P. Nie, Y. Lu, Y. Bai, S. Feng, H. Zhu, M. Zhong, Y. Zhang, J. Xie, Y. Choi, J. Zou, J. Han, W. Chen, J. Lin, D. Jiang, and Y. Zhang Beyond semantic similarity: rethinking retrieval for agentic search via direct corpus interaction. arXiv preprint arXiv:2605.05242. Cited by: §D.2, §D.2, §5.4.
• Liu et al. (2024) N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang Lost in the middle: how language models use long contexts. Transactions of the Association for Computational Linguistics 12, pp. 157–173. External Links: Link, Document Cited by: §1, §2.1, §5.3.
• Ma et al. (2025) D. Ma, Y. Wang, and L. Tian Block-attention for efficient prefilling. In The Thirteenth International Conference on Learning Representations, Note: arXiv:2409.15355 Cited by: §2.2.
• Merth et al. (2024) T. Merth, Q. Fu, M. Rastegari, and M. Najibi Superposition prompting: improving and accelerating retrieval-augmented generation. In International Conference on Machine Learning (ICML), Note: arXiv:2404.06910 Cited by: §2.2.
• MiniMax (2026) MiniMax MiniMax sparse attention. External Links: 2606.13392, Link Cited by: §2.1.
• Moonshot AI (2025) Moonshot AI Kimi k2.5: visual agentic intelligence. Note: Technical Blog External Links: Link Cited by: §2.2.
• Peng et al. (2024) B. Peng, J. Quesnelle, H. Fan, and E. Shippole YaRN: efficient context window extension of large language models. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §D.1, §2.1, §4.2.
• Qwen (2026) Qwen Qwen3.5: accelerating productivity with native multimodal agents. External Links: Link Cited by: §C.1, §D.1, §D.3, §4.1, §4.2.
• Ratner et al. (2023) N. Ratner, Y. Levine, Y. Belinkov, O. Ram, I. Magar, O. Abend, E. Karpas, A. Shashua, K. Leyton-Brown, and Y. Shoham Parallel context windows for large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Note: arXiv:2212.10947 Cited by: §2.2.
• Salemi et al. (2026) A. Salemi, C. Zeng, A. Nijasure, J. Chung, R. Rahimi, F. Diaz, and H. Zamani GrepSeek: training search agents for direct corpus interaction. External Links: 2605.29307, Link Cited by: §D.2, §5.4.
• Sen et al. (2026) S. Sen, A. Kasturi, E. Lumer, A. Gulati, and V. K. Subbiah Is grep all you need? how agent harnesses reshape agentic search. External Links: 2605.15184, Link Cited by: §D.2, §5.4.
• Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. K. Li, Y. Wu, and D. Guo DeepSeekMath: pushing the limits of mathematical reasoning in open language models. External Links: 2402.03300, Link Cited by: §1, §3.3.
• Sheng et al. (2025) G. Sheng, C. Zhang, Z. Ye, X. Wu, W. Zhang, R. Zhang, Y. Peng, H. Lin, and C. Wu HybridFlow: a flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, EuroSys ’25, New York, NY, USA, pp. 1279–1297. External Links: ISBN 9798400711961, Link, Document Cited by: §D.3, §4.1.
• Sheng et al. (2026) L. Sheng, Y. Zhang, W. Ma, Y. Shi, T. Huang, X. Wang, A. Zhang, K. Shen, and T. Chua When to memorize and when to stop: gated recurrent memory for long-context reasoning. External Links: 2602.10560, Link Cited by: §1, §1, §2.1.
• Shi et al. (2026) Y. Shi, Y. Chen, S. Wang, S. Li, H. Cai, Q. GU, X. Wang, and A. Zhang Look back to reason forward: revisitable memory for long-context LLM agents. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §C.1, §C.2, §D.3, §E.6, §1, §1, §2.1, §4.1, §4.2, §4.2.
• Vaswani et al. (2017) A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems, NIPS’17, Red Hook, NY, USA, pp. 6000–6010. External Links: ISBN 9781510860964 Cited by: Appendix A.
• Xiao et al. (2025) S. Xiao, Z. Lin, W. Gao, H. Chen, and Y. Zhang Long context scaling: divide and conquer via multi-agent question-driven collaboration. External Links: 2505.20625 Cited by: §2.2.
• Xu et al. (2026) Z. Xu, S. Zhu, J. Wang, J. Wang, B. Athiwaratkun, C. Wang, J. Zou, and C. Zhang When does divide and conquer work for long context llm? a noise decomposition framework. In The Fourteenth International Conference on Learning Representations, Note: arXiv:2506.16411 Cited by: §1, §2.2.
• Yang et al. (2025) X. Yang, T. Chen, and B. Chen APE: faster and longer context-augmented generation via adaptive parallel encoding. In The Thirteenth International Conference on Learning Representations, Note: arXiv:2502.05431 Cited by: §2.2.
• Yang et al. (2018) Z. Yang, P. Qi, S. Zhang, Y. Bengio, W. W. Cohen, R. Salakhutdinov, and C. D. Manning HotpotQA: a dataset for diverse, explainable multi-hop question answering. In Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: Table 7, Table 7, §C.1, §1, §4.1, §4.2, Table 1, Table 1.
• Yao et al. (2023) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y. Cao ReAct: synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, External Links: Link Cited by: §1, §3.2.
• Yen et al. (2024) H. Yen, T. Gao, and D. Chen Long-context language modeling with parallel context encoding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, Note: arXiv:2402.16617 Cited by: §2.2.
• Yu et al. (2026) H. Yu, T. Chen, J. Feng, J. Chen, W. Dai, Q. Yu, Y. Zhang, W. Ma, J. Liu, M. Wang, and H. Zhou MemAgent: reshaping long-context LLM with multi-conv RL-based memory agent. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: §C.1, §C.1, §C.2, §D.3, §E.6, §1, §2.1, §4.1, §4.2, §4.2.
• Zhang et al. (2024) Y. Zhang, R. Sun, Y. Chen, T. Pfister, R. Zhang, and S. Ö. Arik Chain of agents: large language models collaborating on long-context tasks. In Advances in Neural Information Processing Systems (NeurIPS), Note: arXiv:2406.02818 Cited by: §2.1.
• Zhao et al. (2024) J. Zhao, C. Zu, H. Xu, Y. Lu, W. He, Y. Ding, T. Gui, Q. Zhang, and X. Huang LongAgent: scaling language models to 128k context through multi-agent collaboration. External Links: 2402.11550, Link Cited by: §1, §2.2.
• Zheng et al. (2024) L. Zheng, L. Yin, Z. Xie, C. Sun, J. Huang, C. H. Yu, S. Cao, C. Kozyrakis, I. Stoica, J. E. Gonzalez, C. Barrett, and Y. Sheng SGLang: efficient execution of structured language model programs. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, External Links: Link Cited by: §3.2, §4.1.
• Zhou et al. (2024) Z. Zhou, C. Li, X. Chen, S. Wang, Y. Chao, Z. Li, H. Wang, R. An, Q. Shi, Z. Tan, X. Han, X. Shi, Z. Liu, and M. Sun LLM$\times$mapreduce: simplified long-sequence processing using large language models. External Links: 2410.09342, Link Cited by: §1, §2.2.

Appendix A Time Complexity Analysis

We analyze the time complexity of Full-context (direct answering given the full document), MemAgent, and ParSer. ReMemR1 operates in a pipeline highly similar to MemAgent, so they have the same level of time complexity. Given a question and a document with $n$ tokens, Full-context and MemAgent generate responses with $r_F$ and $r_M$ tokens, respectively. For ParSer, the lead agent and all subagents generate $r_L$ and $r_S$ tokens in total, respectively. For MemAgent and ParSer, each document is split into $c$ chunks. As documents have significantly more tokens than the concatenation of task instructions and questions, the effective input sequence length for all three approaches can be approximated as $n$. To simplify the complexity derivation, we assume the GPU memory capacity is sufficient to hold the entire contextual KV cache, thereby eliminating recomputation of context’s key-value representations.

媒体内容 · 前往原文查看

	# Paragraphs (Total Length)
	50 (7K)	100 (14K)	200 (28K)	400 (56K)	800 (112K)	1600 (224K)	3200 (448K)	6400 (896K)
MemAgent   (# tokens)	2211	3379	6353	12334	23805	47174	93806	187058
ParSer   (# tokens by Lead + Subagents)	534+312	533+348	525+379	485+392	523+474	522+515	546+586	541+804
ParSer/MemAgent   (Decode Computation)	13.9%	10.6%	6.1%	3.2%	2.0%	1.09%	0.63%	0.42%

Prefill

Owing to the inherent properties of the attention mechanism in Transformer architectures (Vaswani et al., 2017), the Full-context baseline exhibits the highest prefill-phase time complexity $𝒪\left(n^2\right)$, which scales quadratically with the input sequence length. Through the adoption of chunk-level input, MemAgent and ParSer reduce the prefill-phase time complexity as

$c\cdot 𝒪\left({\left(\frac{n}{c}\right)}^2\right)=𝒪\left(\frac{n^2}{c}\right),$

(2)

where each of the $c$ chunks contains $n/c$ tokens and is encoded independently.

Decode

Under cached autoregressive decoding, each newly generated token attends to all preceding input and output tokens. Thus, decoding $r_F$ tokens from the $n$-token Full-context input has time complexity

$𝒪\left(\sum _{t=1}^{r_F}\left(n+t\right)\right)=𝒪\left(n{r}_F+r_F^2\right).$

(3)

Based on the operation pipeline of MemAgent, for each chunk, the model generates an average of $r_M/c$ tokens for memory update. Its decoding complexity is therefore

$c\cdot 𝒪\left(\frac{n}{c}\frac{r_M}{c}+{\left(\frac{r_M}{c}\right)}^2\right)=𝒪\left(\frac{n{r}_M+r_M^2}{c}\right).$

(4)

For ParSer, let $K$ denote the number of lead-agent rounds and $r_S^{\prime }$ the average number of tokens generated by each of the $c$ subagents per round. Thus, the total number of tokens generated by all subagents is $r_S=Kc{r}_S^{\prime }$. In each round, the $c$ subagents independently decode over their respective $n/c$-token chunks. Their aggregate decoding computation is

$Kc\cdot 𝒪\left(\frac{n}{c}{r}_S^{\prime }+{\left(r_S^{\prime }\right)}^2\right)=𝒪\left(Kn{r}_S^{\prime }+Kc{\left(r_S^{\prime }\right)}^2\right).$

(5)

The lead agent receives an observation of $r_S$ tokens generated by subagents and generates $r_L$ tokens. Its decoding complexity is $𝒪\left(r_S{r}_L+r_L^2\right)$. Consequently, the total decoding computation of ParSer is

$𝒪\left(Kn{r}_S^{\prime }+Kc{\left(r_S^{\prime }\right)}^2+r_S{r}_L+r_L^2\right).$

(6)

Using the total subagent output $r_S=Kc{r}_S^{\prime }$, this complexity can equivalently be written as

$𝒪\left(\frac{n{r}_S}{c}+\frac{r_S^2}{Kc}+r_S{r}_L+r_L^2\right).$

(7)

When the $c$ subagents are executed in parallel, their decoding contribution to wall-clock latency is reduced by a factor of $c$. The corresponding latency is

$𝒪\left(\frac{n{r}_S}{c^2}+\frac{r_S^2}{K{c}^2}+r_S{r}_L+r_L^2\right).$

(8)

Based on the decoding time-complexity formulas in (4) and (7), we compute the ratio of ParSer to MemAgent decoding computation under each document length; the resulting ratios are reported in the last row of Table 6. The large gap arises mainly from the disparity in total generated tokens: MemAgent produces a memory of about 1K tokens for every chunk, whereas each ParSer subagent emits only a short finding or an “Unknown” response for its chunk—and the latter dominates in most cases. As a result, MemAgent’s output length grows roughly with the number of chunks, while ParSer’s remains comparatively small and stable, yielding substantially lower decoding complexity. Moreover, ParSer exposes parallelism across the $c$ independent subagents. With sufficient hardware resources, this reduces the subagent contribution to wall-clock decoding latency from $𝒪\left(n{r}_S/c+r_S^2/\left(Kc\right)\right)$ to $𝒪\left(n{r}_S/c^2+r_S^2/\left(K{c}^2\right)\right)$, while the lead-agent terms remain sequential.

Note that this analysis assumes that GPU memory is sufficient to retain the KV caches for all encoded chunks. When this assumption does not hold, especially at high request concurrency, subagents’ chunk KV caches may be evicted. The subagents must then re-prefill their chunks in each query round, increasing the aggregate prefill complexity from $𝒪\left(n^2/c\right)$ to $𝒪\left(K{n}^2/c\right)$. With $K=4$, as measured in §5.2, this is four times the prefill computation of MemAgent. In contrast, MemAgent updates memory by sequentially scanning chunks and does not revisit previously processed chunks; thus, it is unaffected by this KV-cache retention constraint. This explains why ParSer’s latency advantage is less pronounced at higher concurrency, as shown in §5.2.

Appendix B Additional Experiments

B.1 Efficacy of Reinforcement Learning

Table 7 compares MemAgent, ReMemR1, and ParSer with and without RL training. Within the ParSer group, RL brings a clear gain: average HotpotQA accuracy rises from $74.42$ to $84.57$ on Qwen3.5-4B and from $76.24$ to $86.79$ on Qwen3.5-9B, with a smaller but consistent lift on out-of-distribution 2WikiMultiHopQA ($\phantom{}\to 87.04$ and $\phantom{}\to 88.48$). These gaps confirm that RL is an effective complement to ParSer.

Even without RL, ParSer already outperforms MemAgent and ReMemR1 by a wide margin and remains essentially flat as document length grows, whereas the sequential memory agents degrade sharply. We attribute this to a closer match between ParSer and the pretrained model: the lead agent follows the model’s native multi-turn tool-calling template, a format the backbone is already trained to follow, whereas sequential memory agents impose a custom recurrent memory-update interface that the pretrained checkpoint has never seen.

媒体内容 · 前往原文查看

(a) Accuracy on HotpotQA (In-Distribution)
Backbone	Method		# Paragraphs (Total Length)								Avg.
		50 (7K)	100 (14K)	200 (28K)	400 (56K)	800 (112K)	1600 (224K)	3200 (448K)	6400 (896K)
Qwen3.5-4B	MemAgent	w/o RL	69.53	62.76	58.86	50.78	40.62	32.29	25.78	19.79	45.05
		w/ RL	81.25	81.51	83.59	77.86	77.86	79.69	73.96	72.92	78.58
	ReMemR1	w/o RL	66.41	63.28	59.90	48.44	44.53	34.38	27.87	20.58	45.67
		w/ RL	82.03	79.95	82.03	78.91	77.86	79.95	77.08	73.44	78.91
	ParSer	w/o RL	72.92	76.04	73.96	74.48	76.04	73.96	75.00	72.92	74.42
		w/ RL	85.68	84.64	83.60	85.68	85.42	83.07	83.07	85.42	84.57
Qwen3.5-9B	MemAgent	w/o RL	68.49	61.20	53.64	52.08	47.14	38.80	26.82	26.30	46.81
		w/ RL	81.77	80.73	82.03	79.95	79.95	81.25	79.69	75.00	80.05
	ReMemR1	w/o RL	61.20	53.64	46.10	46.35	42.71	39.58	40.62	32.03	45.28
		w/ RL	81.25	79.69	77.60	78.39	78.13	78.13	77.86	76.04	78.39
	ParSer	w/o RL	77.60	74.22	74.48	74.74	79.43	77.34	76.56	75.52	76.24
		w/ RL	86.72	88.80	87.24	85.68	86.46	86.72	86.72	85.94	86.79
(b) Accuracy on 2WikiMultiHopQA (Out-of-Distribution)
Backbone	Method		# Paragraphs (Total Length)								Avg.
		50 (7K)	100 (14K)	200 (28K)	400 (56K)	800 (112K)	1600 (224K)	3200 (448K)	6400 (896K)
Qwen3.5-4B	MemAgent	w/o RL	72.92	72.66	62.76	59.63	46.61	44.01	42.71	34.90	54.53
		w/ RL	67.45	73.18	69.01	61.46	54.17	54.43	60.16	45.05	60.61
	ReMemR1	w/o RL	77.86	71.88	60.68	56.25	52.08	46.35	45.31	36.98	55.93
		w/ RL	88.54	86.85	83.20	83.07	76.83	67.71	72.40	60.68	77.41
	ParSer	w/o RL	82.29	84.38	85.68	84.90	80.99	81.51	83.33	79.69	82.85
		w/ RL	86.98	84.64	88.28	87.76	87.50	85.94	88.28	86.98	87.04
Qwen3.5-9B	MemAgent	w/o RL	74.48	69.79	60.94	60.42	50.26	44.53	44.79	31.51	54.59
		w/ RL	80.73	82.03	78.91	75.26	70.57	65.63	75.78	60.94	73.73
	ReMemR1	w/o RL	77.34	71.09	61.20	63.28	55.21	50.52	48.96	40.89	58.56
		w/ RL	84.12	87.50	77.87	83.59	76.30	74.74	79.43	70.57	79.27
	ParSer	w/o RL	84.37	86.46	86.98	83.07	83.59	83.86	85.94	84.64	84.86
		w/ RL	87.76	88.80	88.80	89.06	87.24	89.58	88.54	88.02	88.48

Appendix C Datasets

C.1 Training Data Construction

We construct the training set by following Stage I of MemAgent (Yu et al., 2026); ReMemR1 (Shi et al., 2026) adopts the same Stage I recipe. We do not use MemAgent’s Stage II data.

Concretely, we start from HotpotQA (Yang et al., 2018) training questions and retain each question’s supporting Wikipedia articles as gold evidence. Following the RULER (Hsieh et al., 2024)-style packing used by MemAgent (Yu et al., 2026), we pad each sample with distractor articles sampled from the same HotpotQA corpus until the context contains $200$ paragraphs ($\phantom{}\sim 28$K tokens), then shuffle the paragraph order with a fixed random seed. To remove questions that are already solvable from parametric knowledge alone, we query Qwen3.5-9B (Qwen, 2026) in the non-thinking mode without providing any document context, sample Best-of-3 responses, and discard any question for which the model achieves a $100\%$ score under the rule-based verifier (substring / boxed-answer matching as in MemAgent). $\mathrm{41,027}$ HotpotQA training examples are processed through this pipeline; we take the first $\mathrm{32,768}$ remaining samples as our RL training set.

C.2 Evaluation Data Construction

HotpotQA (in-distribution).

We directly reuse the long-context HotpotQA evaluation sets released by MemAgent (Yu et al., 2026)⁴⁴ 4 https://github.com/BytedTsinghua-SIA/MemAgent. MemAgent synthesizes $128$ questions from the HotpotQA validation split with the same packing recipe as training, then provides each question with contexts of $L\in \left\{\mathrm{50,100,200,400,800},1600,3200,6400\right\}$ paragraphs (approximately $7$K–$896$K tokens), reusing the same question indices across all length settings.

2WikiMultiHopQA (out-of-distribution).

For out-of-distribution evaluation we use 2WikiMultiHopQA (Ho et al., 2020). Because ReMemR1 (Shi et al., 2026) does not release the constructed test files, we regenerate them with the authors’ public data-processing script⁵⁵ 5 https://github.com/syr-cn/ReMemR1. The script loads 2WikiMultiHopQA from FlashRAG (Jin et al., 2024), retains supporting-fact evidence for each question, pads the context with random distractor paragraphs to the same $L$ grid as above, shuffles paragraphs and keeps $128$ samples per length setting.

Table 8 summarizes the training and evaluation data. For each evaluation benchmark, the notation $128\times 8$ indicates $128$ shared questions evaluated under the eight paragraph-count settings listed in the last column.

媒体内容 · 前往原文查看

Split	# Samples	# Paragraphs per sample
Training (HotpotQA)	$\mathrm{32,768}$	$200$
Eval (HotpotQA)	$128\times 8$	$\left\{\mathrm{50,100,200,400,800},1600,3200,6400\right\}$
Eval (2WikiMultiHopQA)	$128\times 8$	$\left\{\mathrm{50,100,200,400,800},1600,3200,6400\right\}$

Appendix D Baseline Implementation Details

D.1 Full-Context Answering

Full-context answering feeds the question together with the entire associated document into a single LLM call and asks the model to answer directly. We evaluate Qwen3.5 (Qwen, 2026) under this protocol (both thinking and non-thinking modes). For inputs within the model’s native $262$K-token window we use the default configuration; for longer documents we apply YaRN (Peng et al., 2024)-based RoPE scaling with a factor of $4.0$, extending the effective context to about $1$M tokens. Generation uses temperature $0$. DeepSeek-V4-Pro (DeepSeek-AI, 2026) natively supports a $1$M-token context window; we evaluate it under the same full-context input format with its reasoning effort set to Max.

Each sample is answered in two turns: the user prompt in the first turn provides the full document and the question, and the models perform inference optionally with thinking; the second turn is a short follow-up that asks for a concise final answer only. We adopt this protocol because thinking models fail to strictly follow format instructions after finishing their reasoning more frequently than non-thinking models (e.g., wrapping the final answer in designated tags) (Li et al., 2025); the issue becomes more pronounced after lengthy deliberation over long documents. In a preliminary experiment that required placing the final answer inside <answer> and </answer>, DeepSeek-V4-Pro under think-max violated the format instruction on $9.47\%$ of in-distribution evaluation cases. We therefore separate reasoning from answer presentation in a two-turn format.

D.2 Direct Corpus Interaction

We reimplement Direct Corpus Interaction (DCI; Li et al. 2026; Salemi et al. 2026; Sen et al. 2026) as a single-agent long-document QA baseline. Unlike other approaches that ingest document chunks into the context, DCI keeps the corpus outside the conversation and lets the model inspect it only through local shell tools.

We mainly follow the implementation of DCI in Li et al. (2026)⁶⁶ 6 https://github.com/DCI-Agent/DCI-Agent-Lite. The agent is equipped with two native tools: (i) “read”, which returns a line-numbered slice of a corpus file with a default window of $200$ lines; and (ii) “bash”, which executes a read-only shell command in the corpus directory (primarily rg, together with ordinary inspection utilities such as ls, find, head, and wc). Parallel tool calls within a single model turn are allowed. Bash execution is confined to a sandbox. Tool observations are truncated to at most $\mathrm{20,000}$ characters (read keeps the head; bash keeps the tail). When the cumulative size of tool results exceeds $\mathrm{240,000}$ characters, we apply the zero-LLM L3 history compaction of Li et al. (2026), replacing older tool results with a short placeholder while retaining the most recent $12$ results.

Inference proceeds in a multi-turn ReAct-style loop with a maximum of $48$ turns. We enable the model’s thinking mode, set temperature to $0$, and cap each generation at $\mathrm{8,192}$ tokens; length-truncated turns receive a continuation observation and continue. When the model stops without tool calls, its response is taken as the final answer. If the turn budget is exhausted before a usable answer appears, we append a tool-free finalize prompt that asks the model to emit a concise answer from evidence already present in the trajectory. All DCI baselines are served with SGLang under the same Qwen3.5 backbone family as ParSer.

D.3 MemAgent and ReMemR1

We reproduce MemAgent (Yu et al., 2026) and ReMemR1 (Shi et al., 2026) from the authors’ open-source repositories, adapting them only as needed to support Qwen3.5 (Qwen, 2026) by upgrading the underlying VERL (Sheng et al., 2025) dependency. We largely reuse the original training hyperparameters; the only intentional change is reducing the rollout group size from $16$ to $8$ (versus $5$ for ParSer) to control training cost. Models are trained on $8$ NVIDIA H100 GPUs for $250$ steps, and a checkpoint is saved every 10 steps. Evaluation likewise follows the authors’ released evaluation code. We refer readers to the original papers and code repositories for further implementation details.

Appendix E P AR S ER ’s Implementation Details

E.1 Lead Agent Prompt

The system prompt contains tool call function usage and QA task instructions. We use the default tool call function template of Qwen3.5, which the models readily follow.

E.2 Subagent Prompt

E.3 Overall Workflow

Algorithm 1 shows the overall workflow of ParSer. If a lead-agent generation contains neither a parseable query_agents tool call nor an <answer> block, the environment does not terminate the trajectory. Instead, it appends InvalidActionHint as the observation and continues the ReAct loop, prompting the lead agent to retry with a well-formed action. The hint text is:

媒体内容 · 前往原文查看

1: Question $𝒒$, document chunks ${\left\{{𝒅}_i\right\}}_{i=0}^{T-1}$, maximum number of steps $K$

2: Final prediction $𝒚$

3: $\mathscr{H}\leftarrow \left[\text{SystemPrompt},\text{UserPrompt}\left(𝒒\right)\right]$ $⊳$ initialize the lead agent’s reasoning trajectory

4: for $t=1$ to $K$ do

5:   $({𝒛}_t,{𝒜}_t)\leftarrow \text{LeadAgent}\left(\mathscr{H}\right)$ $⊳$ ${𝒛}_t$: thinking content; ${𝒜}_t$: actions of subagent calling/answer

6:   if $\text{ExtractAnswer}\left({𝒜}_t\right)\ne \varnothing$ then

7:    return $\text{ExtractAnswer}\left({𝒜}_t\right)$

8:   else if $\text{ExtractQueries}\left({𝒜}_t\right)\ne \varnothing$ then

9:    for all $u\in \text{ExtractQueries}\left({𝒜}_t\right)$ do

10:      parallel for $i=0$ to $T-1$ do $⊳$ Scatter phase: process all chunks concurrently

11:      $o_i\leftarrow \text{Subagent}(u,{𝒅}_i)$

12:      end parallel for

13:      ${ℛ}_u\leftarrow \text{Gather}\left({\left\{o_i\right\}}_{i=0}^{T-1}\right)$ $⊳$ Gather phase: aggregate local findings

14:    end for

15:    $\mathscr{H}\leftarrow \mathscr{H}\cup \left\{({𝒛}_t,{𝒜}_t,{\left\{{ℛ}_u\right\}}_{u\in \text{ExtractQueries}\left({𝒜}_t\right)})\right\}$ $⊳$ append current turn’s generation and observations to $\mathscr{H}$

16:   else

17:    $\mathscr{H}\leftarrow \mathscr{H}\cup \left\{({𝒛}_t,{𝒜}_t,\text{InvalidActionHint})\right\}$ $⊳$ observation flags an invalid generation

18:    continue

19:   end if

20: end for

21: return $\text{ExtractAnswer}\left({𝒜}_t\right)$

E.4 Training Setup

Environment and deployment.

We train ParSer with VERL’s Megatron backend. We adopt the fully asynchronous RL training architecture,⁷⁷ 7 https://verl.readthedocs.io/en/latest/advance/fully_async.html which decouples policy updating and rollout onto separate GPUs and allows both stages to run continuously. Beyond higher training throughput, this setup yields an additional practical benefit in our training: subagents continue serving rollout requests while the lead agent policy is being updated, rather than remaining idle during the update. We use six NVIDIA H100 GPUs for the trainer: two GPUs for policy updates and four GPUs for SGLang-based lead agent rollouts. We deploy the subagents with a separate SGLang Model Gateway⁸⁸ 8 https://docs.sglang.io/docs/advanced_features/sgl_model_gateway cluster on 10 additional NVIDIA H100 GPUs.

We optimize KV cache reuse for faster subagent inference. According to §3.2, across all lead-agent turns, despite having different queries, each subagent is persistently assigned a fixed document chunk. Note that the subagent prompt is ordered as fixed instructions $\to$ assigned document chunk $\to$ current query. Consequently, when the lead agent issues a new query in a later turn, requests sent to the same chunk differ only in the query prompt and subsequent text; the instructions and the chunk, which constitute most of the prompt, remain an identical prefix. During the first such request, SGLang computes the prefix’s KV states and stores them in its Radix Cache. For subsequent requests with that prefix, the server can retrieve the cached KV states and prefill only the new query suffix, avoiding repeated computation over the long chunk.

This reuse requires sending a request to an instance that already holds the relevant cached prefix. We therefore configure the SGLang Router with the cache_aware policy. For each incoming subagent request, the router compares its prompt prefix with the prefixes cached by its serving instances and preferentially routes the request to the instance with the longest match. The router thus preserves cache locality across lead agent turns, while the Radix Cache performs the KV reuse within the selected instance. This combination reduces redundant chunk-prefill computation and accelerates subagent inference.

Training hyperparameters.

Table 9 lists the 4B-model configuration. We optimize the lead agent with GRPO, use five rollouts per prompt, and set the learning rate to $1\times {10}^{-6}$. The lead agent operates for at most 9 turns; each subagent reads a 512-token chunk and generates at most 512 tokens per query.

媒体内容 · 前往原文查看

Category	Hyperparameter	Value
Actor optimization	Optimizer	Adam
	Learning rate; warmup steps; schedule	$1\times {10}^{-6}$; $70$; constant
	Adam $({\beta }_1,{\beta }_2)$; weight decay	$(0.9,0.999)$; $0.01$
	Gradient clipping	$1.0$
	PPO mini-batch	$128$
	PPO clip range; entropy coefficient	$[0.2,0.2]$; $0$
Megatron backend	Precision	bfloat16
	Tensor / pipeline / context / expert parallelism	$1/1/1/1$
	Parameter / gradient / optimizer offload	enabled / enabled / enabled
Asynchronous training	Rollout engine; mode	SGLang; asynchronous
	Rollout GPUs; policy-update GPUs	$4$; $2$
	Rollout GPU memory utilization	$0.8$
	Staleness threshold; parameter-sync interval	$2$; $4$ steps
	Dynamic sampling	enabled
	Checkpoint interval	$10$ steps
Lead agent	Rollouts per prompt	$5$
	Sampling temperature; top-$p$; top-$k$	$1.0$; $1.0$; $-1$
	Maximum turns	$9$
	Maximum generation per turn	$\mathrm{2,048}$ tokens
Subagents	Temperature; output length	$0.7$; $512$ tokens

Training for 180 steps takes about 312 hours for the 4B model and 400 hours for the 9B model.

E.5 Training Dynamics

Figure 4 summarizes the training dynamics of the 4B and 9B lead agents. Comparing the averages over the first and last ten steps, the training reward increases from $44.9\%$ to $83.2\%$ for the 4B model and from $49.8\%$ to $83.3\%$ for the 9B model. The two models reach similar rewards through different interaction dynamics. The mean number of turns increases from $4.65$ to $5.04$ for the 4B model, whereas it decreases from $4.85$ to $4.12$ for the 9B model. Meanwhile, the mean length of lead-agent responses grows from $1.36$K to $2.40$K tokens for 4B and from $1.47$K to $2.48$K tokens for 9B. For the 9B run, where the corresponding communication statistics were logged, subagent queries per turn increase steadily from $1.06$ to $1.62$. Notably, our reward neither penalizes the number of interaction turns nor explicitly encourages issuing multiple subagent queries in parallel. The increase therefore indicates an emergent strategy: the lead agent learns to identify queries without direct dependencies and place them in the same turn for parallel execution, rather than executing them sequentially across turns (see the example in Figure 5).

The lead agents also maintain reliable action formatting from the beginning of training: over the first ten steps, the tool-call format error ratio is only $0.22\%$ for 4B and $0.66\%$ for 9B, and it subsequently approaches zero. This reliability is achieved without a format reward because we use the backbone’s native multi-turn tool-calling format, which the backbone has already been trained to follow. Finally, the absolute log-perplexity difference between the rollout and actor policies remains on the order of ${10}^{-3}$ throughout training (at most $1.41\times {10}^{-3}$ across both runs). Because rollout generation and policy optimization proceed concurrently, a trajectory may be generated by a rollout worker whose policy parameters lag behind the current actor by several updates. The consistently small discrepancy shows that this staleness causes only a minor shift in the token probabilities assigned to collected trajectories, thereby limiting the off-policy mismatch introduced by fully asynchronous training.

E.6 Inference

Inference follows the same process as training. We serve the lead agent with a local SGLang offline engine on the trained checkpoint, and deploy subagents following the training setup described above.

For evaluation, documents are greedily packed into chunks of at most $\mathrm{4,096}$ tokens. The lead agent runs in thinking mode at temperature $0$ for up to $12$ turns and generates at most $\mathrm{2,048}$ tokens per turn. Subagents use temperature $0.7$ and a $512$-token generation budget. Relative to training, evaluation uses larger chunks ($\mathrm{4,096}$ vs. $512$ tokens) and a higher turn budget ($12$ vs. $9$). We intentionally adopt a smaller chunk size during training because finer chunking reduces prefill cost: as given in Equation (2), the prefill complexity $𝒪\left(n^2/c\right)$ decreases as the number of chunks $c$ grows, which speeds up training.

Following MemAgent (Yu et al., 2026) and ReMemR1 (Shi et al., 2026), we report Sub_EM: after standard answer normalization (lowercasing, removing articles and punctuation), a prediction is counted as correct if either the prediction or a ground-truth answer is a substring of the other.⁹⁹ 9 Strictly speaking, Sub_EM should be a unidirectional match in which the ground-truth answer is a substring of the model prediction; MemAgent’s paper also describes the metric in this sense. However, the open-source evaluation code of MemAgent and ReMemR1 implements a bidirectional variant. We follow the latter definition.

Appendix F Case Study

F.1 Success Case

Figure 5 shows a representative success trajectory. The lead agent decomposes the comparative question into two parallel query_agents broadcasts, gathers birth-date evidence from different subagents, and synthesizes the final answer in a second reasoning step. Figure 6 shows a two-hop director lookup. The lead agent first recovers that I Want Someone to Eat Cheese With was directed by Jeff Garlin, then queries his birth date. Figure 7 shows a comparative multi-hop success trajectory. The lead agent broadcasts parallel director queries for Everything’s Ducky and Karthika, then parallel death-date queries, and selects the film whose director died earlier. Figure 8 shows a successful disambiguation trajectory. The lead agent first finds that L’Oiseau Blanc was flown by Charles Nungesser and François Coli, then issues parallel queries and selects Nungesser as the French ace pilot and adventurer.

F.2 Failure Case

We illustrate a failure caused by a misleading subagent finding with the question “Who is the husband of Princess Elene of Georgia?” The displayed trajectory contains two relevant findings. One correctly identifies the target as the daughter of Heraclius II of Georgia and the mother of Solomon II of Imereti. A second finding establishes that Solomon II was born to Prince Archil of Imereti and his wife Helene (an alternative name for Elene), the daughter of Heraclius II, which therefore supports the correct answer, Prince Archil of Imereti. However, a subagent assigned to a chunk about Grand Duchess Elena Vladimirovna of Russia returns the statement “Her husband was Prince Nicholas of Greece and Denmark.” Since the subagent only observes its local chunk, it incorrectly treats the similarly named Elena in that chunk as the Elene mentioned in the question. The lead agent subsequently accepts this short, apparently direct answer and outputs Prince Nicholas of Greece and Denmark, despite the contradictory identity-grounded evidence from another subagent.

This example exposes a failure pattern caused by context isolation between the lead agent and subagents. Each subagent receives only the lead agent’s query and a chunk, without the reasoning trajectory. When the query is underspecified or ambiguous, as in this case, the subagent may not recover the lead agent’s current objective and can return an unintended conclusion based solely on its local chunk. Conversely, the lead agent has no access to the subagent’s source reference and thus cannot directly verify whether the returned conclusion is grounded in a relevant reference. It may consequently over-trust an erroneous subagent conclusion. This behavior is occasional: in other cases, the lead agent issues additional queries that are more specific and reconciles evidence from multiple subagents before answering. The present failure occurs when that cross-validation process is not triggered or does not override the misleading local finding.

F.3 Comparison against MemAgent on a Reverse-evidence Case

Figure 10 shows a question–document sample from the reverse-evidence setting. The question is a comparative two-hop query: identify the directors of Everything’s Ducky and Karthika, then select the film whose director died earlier. The four supporting paragraphs are embedded in a 6,400-paragraph document. Director death dates appear first (paragraphs 483 and 3911), while the film–director mappings appear later (paragraphs 5527 and 5631), i.e., in reverse logical order.

Figure 11 shows MemAgent’s sequential memory updates on this sample. When the agent encounters paragraph 483 (M. Krishnan Nair, died 2001) and later paragraph 3911 (Don Taylor, died 1998), it does not write either person’s identity or death date into memory. At those steps the two names have not yet been linked to the films in the question and are not written into the memory. Instead, the memory keeps recording directors of other films mentioned in the incoming chunks. Only after paragraphs 5527 and 5631 does the memory record that Karthika was directed by M. Krishnan Nair and Everything’s Ducky by Don Taylor. By then the death dates have already been dropped. The final memory therefore contains the two director names without their death dates, and MemAgent cannot complete the comparison. This trajectory shows that MemAgent’s reasoning is heavily driven by the document: the order and surface content of incoming chunks determine what is written into memory.

By contrast, Figure 7 shows ParSer on the same question. ParSer reasons from the question, issuing multi-round queries that execute in parallel over the full document, and is therefore completely insensitive to the order of evidence.

媒体内容 · 前往原文查看

媒体内容 · 前往原文查看

媒体内容 · 前往原文查看

媒体内容 · 前往原文查看

媒体内容 · 前往原文查看

媒体内容 · 前往原文查看

媒体内容 · 前往原文查看

媒体内容 · 前往原文查看