Spatial Atlas: Compute-Grounded Spatial Reasoning

Abstract

We describe compute-grounded reasoning (CGR), a design pattern in which code computes selected sub-problems from explicit intermediate representations before a language model answers. Spatial Atlas implements CGR as an Agent2Agent (A2A) server with a spatial question-answering handler and a machine-learning engineering handler. The spatial handler asks a language model to extract a scene graph, and code then fills in missing distances and checks the extracted safety rules. A separate benchmark driver can also run a strict metric bridge. It computes the gap for horizontal-gap questions from segmentation masks and a reconstructed point map, and it passes that gap to the answering model as a fact. The bridge returns a fixed unavailable answer when an evidence check fails, and it never falls back to model-estimated coordinates. The ML-engineering handler generates pipeline code and parses validation scores, and it caps the number of repair and refinement passes. Its code execution is off by default. The repository also provides four run modes that can write label-free journals, a shuffled-image control mapping, and journal validators that reject label-bearing fields. We report one private label-free operational run in which four paths each wrote eight prediction rows with zero retries. Labels stayed sealed and no score was computed, so this run establishes operational integrity only. We report no FieldWorkArena result, because the benchmark data were not accessible. We also omit every performance, latency, and resource-use number that lacks a reproducible run artifact.

How it works

Spatial Atlas follows a design pattern that the paper calls compute-grounded reasoning (CGR). When a sub-problem has a deterministic solution, code computes it first and passes the result to the language model as a fact. Figure 1 above shows how the system applies this pattern. Equation and table numbers on this page follow the paper, so some numbers are skipped.

One Agent2Agent (A2A) server, built on the official a2a-sdk, hosts a spatial question-answering handler and an ML-engineering handler. A domain classifier sends each task to one handler with four fixed rules, and no model call takes part in dispatch. Both handlers share a LiteLLM client, four model tiers, and usage tracking. Each call site names its tier in code, and no component routes calls between tiers at run time.

Table 1. This table lists the configured model tiers and their uses in the current code. It is a design table, and it reports no performance or resource-use result.

Tier	Model	Current use
Fast	GPT-4.1-mini	Self-grade, and entity phrases for the generic metric engine
Standard	GPT-4.1	MLE competition analysis
Strong	GPT-4.1	Scene-graph extraction, spatial answers, refinement, code generation, and code repair
Vision	GPT-4.1	Descriptions of images and video frames

The ML-engineering handler asks the Standard tier to analyze each competition and name a strategy template. The Strong tier then writes one pipeline script, repairs it after errors, and refines it. Generated-code execution is off by default. It needs an opt-in flag, an operator flag that attests an isolated and trusted worker, and authenticated server startup. After that authorization, the limits in Eq. (23) apply, with \(\nu\) as the best parsed validation score so far and \(\nu'\) as the revised score.

\[ \begin{aligned} & n_{\text{attempt}} \le 3, \qquad n_{\text{refine}} \le 2, \qquad t_{\text{run}} \le 600\ \text{s}, \\ & \text{accept } \nu' \iff \begin{cases} \nu' < \nu & \text{if the direction text contains a minimize keyword},\\ \nu' > \nu & \text{otherwise.} \end{cases} \end{aligned} \tag{23} \]

Here \(n_{\text{attempt}}\) counts the initial run and its repairs, \(n_{\text{refine}}\) counts the refinement runs, and \(t_{\text{run}}\) is the timeout of each run. These numbers are configured ceilings. None of them is an observed runtime, an observed attempt count, or a measured resource use.

Scene graph and computed facts

The default spatial path splits a question about an image into three stages, which are extraction, structuring, and computation. During extraction, the Vision tier describes the image. The Strong tier then reads the first 8,000 characters of those descriptions and returns a JSON list of entities, relations, zones, and safety rules. When its optional packages are installed, Florence-2 runs first and supplies object counts, keyword-based protective-equipment flags, and a caption.

The extracted entities form a spatial scene graph \(G = (V, \mathcal{E})\). Its vertices are entities, and its edges are spatial relations, as in Eqs. (2) and (3).

\[ v_i = \texttt{SpatialEntity}(\text{id}_i, \text{label}_i, \text{pos}_i, \text{attrs}_i, \text{zone}_i) \tag{2} \]

\[ e_{ij} = \texttt{SpatialRelation}(\text{subj}_i, \text{pred}_{ij}, \text{obj}_j, d_{ij}) \tag{3} \]

The position \(\text{pos}_i \in \mathbb{R}^2\) is an estimate that the Strong tier makes from the text descriptions. When the extraction supplies a distance, the graph keeps that model-estimated value. Otherwise, code computes the Euclidean distance between the two positions and rounds it to two decimals, as in Eq. (4).

\[ d_{ij} = \begin{cases} \hat{d}_{ij} & \text{if the extraction supplies a distance},\\ \operatorname{round}_2\bigl(\lVert \text{pos}_i - \text{pos}_j \rVert_2\bigr) & \text{if both positions exist},\\ \text{undefined} & \text{otherwise.} \end{cases} \tag{4} \]

The pipeline then checks the safety rules that the Strong tier extracted. A person-to-hazard relation is a violation when its distance falls below a threshold \(r_{\text{rule}}\) parsed from a rule, as in Eq. (6).

\[ \operatorname{viol}(e_{ij}) = [\ell_i \in \mathcal{P}] \wedge [\ell_j \in \mathcal{H}] \wedge [d_{ij} < r_{\text{rule}}] \tag{6} \]

The bracket \([\cdot]\) is 1 when its condition holds and 0 otherwise, and \(\ell_i\) is the label of entity \(i\). The labels are lowercased and must match a member of the person set \(\mathcal{P}\) or the hazard set \(\mathcal{H}\) exactly.

\[ \mathcal{P} = \{\text{worker}, \text{person}, \text{employee}\} \]

\[ \mathcal{H} = \{\text{forklift}, \text{machinery}, \text{crane}, \text{conveyor}, \text{vehicle}\} \]

The check also flags a worker whose extracted attributes mark missing protective equipment.

Finally, the code serializes the graph into a fact sheet, and the Strong tier answers from the question, the file descriptions, and that fact sheet. The fact sheet mixes model-estimated positions, model-supplied distances, and code-computed distances. The arithmetic is deterministic, but its inputs are not measurements, and deterministic arithmetic cannot correct a wrong geometric input.

The strict metric bridge

The strict metric bridge replaces the estimated coordinates with reconstructed geometry for one question type. It computes the horizontal gap between two objects from SAM 3 segmentation masks and a Depth Anything 3 point map, which it gets through SpatialClaw's tools. Its strict parser targets the horizontal-gap questions of Q-Spatial Bench. Only the frozen benchmark driver reaches this bridge, and an A2A request in metric mode uses a separate generic metric engine.

First, the bridge resizes each mask to the point-map grid and erodes it once with a disk of radius \(r_e\), as in Eq. (7). An empty eroded mask makes the pair invalid.

\[ \begin{aligned} r_e &= \Bigl\lceil \max\Bigl(\frac{w_m}{w_t},\, \frac{h_m}{h_t}\Bigr) \Bigr\rceil, \\ B_{r_e} &= \{(u, v) \in \mathbb{Z}^2 \mid u^2 + v^2 \le r_e^2\}, \\ E &= \tilde{M} \ominus B_{r_e}. \end{aligned} \tag{7} \]

Two eroded masks can still share pixels. A pair whose overlap coefficient \(\rho\) is 0.05 or more is rejected, and otherwise the shared pixels are removed from both masks, as in Eq. (8).

\[ \begin{aligned} & \rho(E_a, E_b) = \frac{\lvert E_a \cap E_b \rvert}{\min(\lvert E_a \rvert, \lvert E_b \rvert)}, \\ & \begin{cases} \text{reject the pair} & \text{if } \rho \ge 0.05,\\ E_k \leftarrow E_k \setminus (E_a \cap E_b) \text{ for } k \in \{a, b\} & \text{otherwise.} \end{cases} \end{aligned} \tag{8} \]

A pixel contributes only when its reconstructed point \(X(p)\) is finite and its confidence \(c(p)\) is finite and exceeds 0.3. The bridge bins the qualified points on the horizontal \(x\) and \(z\) axes into 5 mm voxels, and each occupied voxel keeps its highest-confidence point, as in Eqs. (9) and (10). The code assumes that the reconstruction is already metric and gravity-aligned, and it does not check either property. A tie goes to the lowest flat pixel index.

\[ \begin{aligned} Q(E) &= \{\, p \in E \mid X(p) \text{ finite},\ c(p) \text{ finite},\ c(p) > 0.3 \,\}, \\ \kappa(p) &= \Bigl(\Bigl\lfloor \frac{X_x(p)}{0.005} \Bigr\rfloor,\ \Bigl\lfloor \frac{X_z(p)}{0.005} \Bigr\rfloor\Bigr) \end{aligned} \tag{9} \]

\[ \pi(k) = \operatorname*{arg\,max}_{p \in Q(E),\ \kappa(p) = k} \bigl(c(p),\, -\operatorname{idx}(p)\bigr) \tag{10} \]

A mask with more than 50,000 voxel keys keeps the 50,000 keys with the smallest SHA-256 digests, so the retained subset does not depend on sampling order. After this cap, each mask must keep at least 32 keys, as in Eq. (11).

\[ \begin{aligned} K(E) &\leftarrow \begin{cases} \begin{array}[t]{l} \text{the 50,000 keys with the} \\ \text{smallest } \mathrm{SHA256}(k_x, k_z) \end{array} & \text{if } \lvert K(E) \rvert > 50{,}000,\\ K(E) & \text{otherwise,} \end{cases} \\ \lvert K(E) \rvert &\ge 32. \end{aligned} \tag{11} \]

For each representative point of one object, the bridge finds its nearest neighbor in the other object. It summarizes each directed distance set by its fifth percentile with linear interpolation, and the gap \(g\) is the smaller of the two directed values, as in Eq. (12). A gap that is not finite or lies outside the interval \((0, 20]\) m makes the pair invalid.

\[ \begin{aligned} \operatorname{nn}_{A \to B}(\mathbf{u}) &= \min_{\mathbf{w} \in R_B} \lVert \mathbf{u} - \mathbf{w} \rVert_2 \quad (\mathbf{u} \in R_A), \\ g &= \min\Bigl(\operatorname{pct}_{5}\bigl(\{\operatorname{nn}_{A \to B}(\mathbf{u})\}_{\mathbf{u} \in R_A}\bigr), \\ &\qquad\quad \operatorname{pct}_{5}\bigl(\{\operatorname{nn}_{B \to A}(\mathbf{w})\}_{\mathbf{w} \in R_B}\bigr)\Bigr), \\ 0 &< g \le 20\ \text{m}. \end{aligned} \tag{12} \]

At the 32-voxel minimum, the fifth percentile falls between the second and third smallest distances, so the single smallest distance in each direction does not set the gap. A stray reconstructed point close to the other object can still lower the gap, and its effect on gap error has not been evaluated. The symmetric minimum removes the dependence on which object is named first. For a repeated-instance question, an instance enters pairing only if its mask area is at least 0.30 of the largest instance area. The bridge then selects the valid pair with the smallest gap and breaks ties by the lowest mask indices, as in Eq. (13).

\[ \begin{aligned} \mathcal{I} &= \{\, i \mid \max_j \operatorname{area}_j > 0,\ \operatorname{area}_i \ge 0.30 \max_j \operatorname{area}_j \,\}, \\ (i^*, j^*) &= \operatorname*{arg\,min}_{(i, j)\ \text{valid}} \bigl(g_{ij},\, (i, j)\bigr). \end{aligned} \tag{13} \]

Equation (14) gives the outcome \(B\) of the strict bridge. The fact sheet prints \(g\) to six decimals, and the Strong tier writes the final answer from it. The Strong tier may refine that answer once, so the gap reaches the answer only through the model. No code checks that the final answer repeats \(g\). When an evidence check fails, the answer is the literal measurement unavailable, and the reasoner does not run. The bridge never substitutes the estimated-coordinate scene graph.

\[ B = \begin{cases} g_{i^* j^*} \text{ in meters, in the fact sheet} & \text{if a valid pair exists},\\ \mathtt{QSPATIAL\_UNAVAILABLE} & \text{if an evidence check fails},\\ \text{an error} & \begin{array}[t]{l} \text{if parsing, the manifest check,} \\ \text{or a service fails.} \end{array} \end{cases} \tag{14} \]

Reconstructed geometry is an estimate, and it is not ground truth. A wrong mask or a wrong scale that passes every check still produces a wrong gap. The 0.30 instance-area ratio was chosen by inspecting development cases. The strict parser was written against Q-Spatial Bench question text, so it is not a held-out component.

The controller and its one-refinement rule

The implemented controller answers once at the Strong tier, asks the Fast tier for one self-grade, and refines at most once, as in Eqs. (18) to (20).

\[ a_0 = \operatorname{Strong}\bigl(q,\, D_{:12000},\, F\bigr) \tag{18} \]

\[ \sigma = \begin{cases} \operatorname{parse}\bigl(\operatorname{Fast}(q,\, D_{:2000},\, a_0)\bigr) & \text{if the reply parses},\\ 0.5 & \text{otherwise} \end{cases} \tag{19} \]

\[ a = \begin{cases} \operatorname{Strong}\bigl(q,\, a_0,\, D_{:6000},\, F\bigr) & \text{if } n_{\text{refl}} > 0 \text{ and } \sigma < 0.6,\\ a_0 & \text{otherwise} \end{cases} \tag{20} \]

Here \(D_{:n}\) is the first \(n\) characters of the file descriptions, and \(F\) is the fact sheet. The self-grade \(\sigma\) is a self-reported heuristic. The code does not clip it, and it has not been shown to be calibrated, so it should not be read as a probability.

An unparseable self-grade defaults to 0.5, which lies below the threshold \(\tau = 0.6\), so an unparseable reply always triggers the refinement. The reflection setting \(n_{\text{refl}}\) has a default of 2, and it acts only as an on-off flag. The controller therefore refines at most once per task and never loops.

The refinement sees the question, the first answer, the first 6,000 characters of the descriptions, and the fact sheet. The first answer had already seen the first 12,000 characters, so reflection adds no new evidence. The effect of this refinement on accuracy and resource use has not been measured.

The repository also defines an expected-information-gain action selector and a tier router. No code in the repository calls either one, so the paper states the entropy-guided design as a proposal and reports no result for it.

What the V37 run establishes and what it does not

The paper reports one completed label-free operational run of the four-path design, called V37. It is an integrity check on the execution path, and it does not measure answer quality.

In the public driver, two options select a path, an engine and an image mode. The driver admits only four of the six combinations, as in Eq. (16).

\[ \begin{aligned} \mathcal{M} = \bigl\{ & (\text{scenegraph}, \text{correct}), \\ & (\text{scenegraph}, \text{question-only}), \\ & (\text{metric}, \text{correct}), \\ & (\text{metric}, \text{shuffled}) \bigr\} \end{aligned} \tag{16} \]

The shuffled-image mode requires the metric engine and a control mapping \(\phi\) over the set \(S\) of source image paths. The loader checks the size of the mapping, its bijectivity, the absence of fixed points, and the digest of every source and control image, as in Eq. (17).

\[ \begin{aligned} & \phi : S \to S, \qquad \lvert S \rvert = 84, \\ & \phi \text{ is a bijection}, \qquad \phi(s) \ne s \ \ \forall s \in S, \\ & \mathrm{SHA256}(s) = \delta_s \ \text{ and } \\ & \mathrm{SHA256}\bigl(\phi(s)\bigr) = \delta_{\phi(s)} \ \ \forall s \in S. \end{aligned} \tag{17} \]

The V37 record does not restate the mapping that the private run loaded, so the paper does not report its size.

The diagram shows four V37 paths, and each path writes eight label-free rows into the prescore journals. The terminal finalizer then writes the seal. The labels stay sealed, and no score is computed. — Figure 3. Four operational paths ran together in the V37 run. The paths were the scene-graph path, question-only baseline, correct-image metric path, and schema-matched shuffled-image metric control. Each path produced eight label-free prescore prediction rows, so the run wrote 32 rows with zero retries. The execution, warning-scan, restoration, cleanup, and terminal-seal gates passed. In the public driver, the shuffled-image mode swaps each attached image through a control mapping \(\phi\). That mapping must hold 84 pairs, be a bijection, and have no fixed point, and the loader checks the digest of every source and control image. The terminal finalizer ran once and wrote the smoke seal and the terminal result. Protected labels and ground truth remained sealed, and no score or paired comparison was computed, so the run establishes operational integrity only. The run is private, and the public repository cannot reproduce it.

Table 3. This table quotes the V37 evidence record, which asks writers to use this wording and no stronger interpretation. In item 9 of the list headed “V37 does not establish,” the paper writes “resource-use” in place of the record’s term.

V37 establishes

Four operational paths ran together.
The paths were the scene-graph path, question-only baseline, correct-image metric path, and schema-matched shuffled-image metric control.
Each path produced eight label-free prescore prediction rows.
The total was 32 rows.
The execution, warning-scan, restoration, cleanup, and terminal-seal gates passed.
The run used zero retries.
Protected labels and ground truth remained sealed.
No score or paired comparison was computed.
This establishes operational integrity only.

V37 does not establish

Accuracy
Superiority over another path or system
A causal or ablation effect
Statistical significance
An uncertainty interval
Calibration
Benchmark ranking
Production readiness
A resource-use or latency result
Native SpatialClaw persistent-kernel integration

Three distinctions matter for reading this result. The prediction journals record model outputs before scoring, and they do not show which evidence a given answer used. The terminal finalizer checks only lifecycle and evidence-integrity conditions, so answer correctness is outside its checks. The metric arms used SpatialClaw perception primitives through the strict bridge. V37 ran no native SpatialClaw persistent-kernel arm.

The public code does not settle whether the strict geometry step in the shuffled arm measured the control image or the source image. The paper therefore treats the shuffled arm as an executed control path. It does not treat that arm as a geometry control.

The table below lists the measured results that the paper reports. None of these numbers measures answer quality. The two test counts come from different suites, so they should not be compared with each other.

Result	Value	Scope
Operational paths that ran together in V37	4	Private V37 record, operational integrity only
Label-free prescore prediction rows	8 per path, 32 in total	Private V37 record, rows never scored
Retries	0	Private V37 record, one submission with no rollback
Bounded service logs that passed the warning scan	8 of 8	Private V37 record
Terminal finalizer invocations	1	Private V37 record
Frozen V37 control suite	350 tests plus 63 subtests passed	Private V37 record, a local regression pass and not benchmark accuracy
Public CI test command	323 passed, 1 deselected	A local run used Python 3.13. The deselected end-to-end test needs a GPU tool server and the dataset.
Line coverage	92.09%	This figure covers `src/fieldwork/perception.py` only. CI fails when that coverage drops below 90%.

Reproducibility

The public code is at github.com/arunshar/spatial-atlas-agent. It includes the agent, the four run modes, and their validators. The A2A server and task router are in src/server.py and src/agent.py, and the spatial path is in src/fieldwork/. The ML path is in src/mlebench/, and eval_bench.py is the benchmark driver that exposes the run modes.

The public repository cannot re-execute V37 by itself. The metric path needs an externally installed SpatialClaw package and a reachable GPU perception service. The gated benchmark images and the frozen control artifact are not bundled with the repository. The combined orchestration used for V37 ran in a separate private execution environment, and it is not included.

The paper reports no FieldWorkArena result, because the benchmark data were not accessible. It also omits every performance, latency, and resource-use number that lacks a reproducible run artifact. The next phase will unseal the protected labels under a frozen analysis plan and score the completed prediction journals. It will then report paired comparisons with uncertainty intervals.

BibTeX

@misc{sharmaspatialatlas,
  title        = {Spatial Atlas: Compute-Grounded Reasoning for Spatial-Aware Research Agent Benchmarks},
  author       = {Sharma, Arun},
  year         = {2026},
  eprint       = {2604.12102},
  archivePrefix = {arXiv},
  primaryClass = {cs.AI},
  url          = {https://arxiv.org/abs/2604.12102},
  note         = {arXiv preprint}
}

Acknowledgements

The metric path imports its segmentation and reconstruction tools from an externally installed copy of SpatialClaw by Cho et al. The SpatialClaw GPU service runs SAM 3 segmentation and Depth Anything 3 reconstruction. The repository does not vendor SpatialClaw or its GPU service.