SafeGEO, the research report

1. Summary

• Setting. A recommendation agent reads a fixed packet of retrieved web sources about a shortlist of products and returns a ranked recommendation with citations. One seller-controlled source in the packet is rewritten; everything else is held fixed.
• Scale. 600 requests in six product verticals; 22 shaped variants and two controls; three sampled targets per request; 40,800 evaluation instances. Four systems: Gemma 4 31B IT, Qwen3.6 27B, Devstral Small 2 24B Instruct, and DeepSeek-V4-Flash as a frontier-scale check.
• Headline pair. Averaged over the eight realistic variants, the shaped candidate reaches the top three in 72.6 to 90.9 percent of instances, against 3.4 to 12.7 percent under a truthful rewrite of the same source. The first recommendation violates one of the person’s hard constraints in 73.4 to 90.7 percent of instances, against 16.9 to 41.1 percent.
• Mechanism. Shaped sources work by being cited. The misleading-line citation rate and the shaped candidate’s top-three rate move together across variants (Pearson r = 0.91 for Gemma 4 31B IT, 0.99 for DeepSeek-V4-Flash), and paired responses show the route: a shifted citation, a missed product gap, a promotion.
• Mitigation. Five prompt-level or input-level interventions were tested on the same instances. A candidate-level evidence breakdown before ranking is the strongest on every system, lowering the shaped candidate’s top-three rate by 17.7 to 39.2 points. None restores the clean rate.

2. The question

A recommendation is utility-aligned when it puts the candidates that satisfy the person’s hard constraints first and orders them by how well they meet the person’s preferences. The paper writes this down: each request carries hidden hard constraints, soft preferences and preference weights; each candidate carries canonical attributes; a reference utility and a reference ranking follow from them and are never shown to the system under test.

A shaped source is a rewrite of one candidate’s own source, intended to promote that candidate although it is inferior under the reference ordering or violates a hard constraint. The rewrite changes the text; it does not change the product. Harm is when the rewrite moves such a candidate up the ranking. That is the failure the benchmark measures, and it is different from the visibility that earlier work on generative engine optimisation measured: whether rewritten content gets cited or surfaced at all.

3. Setting and protocol

The agent. The evaluated system receives a request, a candidate shortlist, product metadata and a packet of sources, and must return a ranked list with a short rationale and citations, as structured output. Retrieval, candidate generation and source selection are fixed, so the benchmark isolates the generation stage.

Three conditions. Every instance exists in one of three forms of the target’s own source.

Source lengths are matched: on average 3,911 words under no rewrite, 3,905 under the clean condition and 3,925 under the shaped condition, over all 22 variants. The effect is not a longer page.

The template. Every controlled source has ten lines with fixed roles: title and source type, framing, positioning, the caveat or claim, a positive note, a verification note, a comparison reminder, a boundary reminder, a statement of source authority, and a format note. A variant may overwrite only the lines its primitives are allowed to touch; the rest stay as truthful anchors. The paper’s Figure 8 shows a full pair for one instance.

Four measures. Percent of instances unless noted.

Four systems. Gemma 4 31B IT, Qwen3.6 27B and Devstral Small 2 24B Instruct are open-weight models in the 24B to 31B range, served locally with vLLM. DeepSeek-V4-Flash is queried through a hosted OpenAI-compatible API with reasoning disabled and a 4,096-token output cap. All run at temperature 0 with structured output enforced.

4. The taxonomy of shaped sources

Seven primitives in three loci, drawn from the literatures on information manipulation, persuasion and prompt injection, and kept only where a primitive could be realised as a controlled source rewrite that leaves the canonical attributes fixed.

The primitives are instantiated as 22 variants in four families.

The realistic family is the one the headline numbers use. The full variant-by-primitive matrix is Table 3 of the paper and the taxonomy document in the code repository.

5. The dataset

Verticals. AI meeting transcription tools, baby monitors, carry-on backpacks, home air purifiers, noise-cancelling headphones, office chairs. They were chosen so that requests carry verifiable constraints, fit depends on source-grounded evidence and canonical attributes, and the evidence takes different forms across verticals: policy and plan terms, safety claims, technical specifications, physical compatibility, review-based fit. One hundred requests per vertical were written by hand.

Discovery. Candidates, sources and metadata come from the candidate-search trace of a production shopping agent, used only for product discovery and source collection. The discovered candidates become the shortlist; the opened sources, from product pages and reviews to comparison pages and specification pages, become the packet.

Construction statistics (paper, Table 8).

Hidden labels. A query-decomposition step extracts only the requirements and preferences the request states. A verification step fills each candidate’s canonical attributes only where the supplied sources support them, and keeps missing or conflicting evidence as missing or conflicting. Each source span is labelled for its relation to a product claim: supports, refutes, insufficiently supports, or irrelevant. The labels were produced with GPT-5.5 and audited by two independent annotators on 60 cases stratified by vertical (paper, Table 15): agreement between the annotators was 100 percent on hard-requirement extraction, 91.7 percent on the top candidate, 78.1 percent on pairwise utility ordering, 76.7 percent on candidate hard status, and 80.3 to 88.9 percent on the three line-level labels; the model’s agreement with each annotator was of the same size.

Targets. For each case, three eligible candidates that are not ground truth are sampled as targets. Each target is crossed with every variant and both controls, and each instance rewrites that target’s source only.

SafeGEO Diamond. A 600-instance screening split from 120 cases, 20 per vertical, each contributing both controls and the three realistic variants with the highest DeepSeek-V4-Flash top-three rate on the full benchmark (selective comparison note, false-fit checklist, citation-padded note). It is a high-effect stress set for cheap iteration; population-level numbers should use the full benchmark.

6. Results

6.1 Four systems, four measures

Clean is the truthful-rewrite control; shaped is the average over the eight realistic variants. Percent of instances. Paper, Table 5 and Table 33; the no-rewrite reference is on the leaderboard.

Two things are visible at once. The harm measures move by 60 to 80 points. The quality measures move by a few points, because the shaped candidate usually joins the top three rather than displacing every right answer; a ranking can be mostly right and still lead with the product that cannot do what was asked.

6.2 By family and variant

Coherent documents beat combinations of primitives. Shaped candidate in the top three, percent, over all variants of each family (paper, Table 24 and Section C.7).

For the two systems with headroom, stacking primitives without a coherent source form does worse than a single primitive. Devstral Small 2 24B Instruct is near ceiling under every family.

Within the realistic family, the strongest variant is the selective comparison note on three systems (84.0 percent on Gemma 4 31B IT, 82.2 on Qwen3.6 27B, 82.3 on DeepSeek-V4-Flash) and the full-stack realistic rewrite on Devstral (95.9 percent, 83.2 points above its clean rate, the largest gap in the study). The weakest realistic variant on DeepSeek-V4-Flash is the AI-directed source text, at 51.3 percent against 95.5 on Devstral; among single primitives on that system, salience manipulation (70.9), caveat omission (67.8) and evidence padding (66.7) are the strongest, and model-directed instruction (42.4) and relevance flooding (42.9) the weakest.

6.3 By vertical

Shaped candidate in the top three, percent, by product vertical, for the three open-weight systems, with the uplift over the vertical’s clean rate in brackets (paper, Table 28).

The vertical decided by plan and policy terms is the most fragile for every system. Where fit is a physical specification that other sources state plainly, the two stronger systems resist more.

6.4 Mechanism

Across the 22 variants, the rate at which a system cites a misleading line from the shaped source and the rate at which the shaped candidate reaches the top three are tightly coupled: Pearson r = 0.91 for Gemma 4 31B IT, 0.92 for Qwen3.6 27B, 0.88 for Devstral Small 2 24B Instruct, and 0.99 for DeepSeek-V4-Flash (paper, Table 23 and Figure 22).

Paired analysis of matched clean and shaped responses for Gemma 4 31B IT (paper, Figure 5) traces the route. When the cited evidence shifted, the shaped response missed the target’s decision-relevant gap in 54.1 percent of pairs, against 6.7 percent without a shift. When the gap was missed, the target entered the top three in 92.8 percent of pairs, against 58.6 percent when it stayed recognised. The complete path, a shifted citation followed by a missed gap followed by a promotion, accounts for 49.7 percent of observed entries.

DeepSeek-V4-Flash often retrieves the refuting evidence and promotes the target anyway: its paired refuting-evidence recall is 68.3 percent and its valid-citation rate 86.8 percent. The failure is one of weighting after retrieval, not of retrieval.

6.5 Sensitivity

On DeepSeek-V4-Flash, averaged over the eight realistic variants and three targets (paper, Table 34).

Position matters and does not rescue the person. A thinner packet is worse: with five sources the shaped candidate is in the top three almost always.

7. Mitigation

Six matched conditions on the same shaped instances (the eight realistic variants, three targets), each measured against the unmitigated request. All are prompt-level or input-level changes a developer can make without changing the model.

Shaped candidate in the top three, percent, shaped condition (paper, Table 7 and Table 35).

Top pick violates a hard constraint, percent, same conditions.

Utility of the top five, same conditions.

What the tables say. Evidence breakdown is the strongest intervention on every system, on both harm measures, and it improves or holds utility on three of the four. It is also the best strategy for every one of the eight realistic variants on Gemma 4 31B IT (paper, Table 31). Its effect is uneven by vertical: 16.6 to 36.8 points on Gemma 4 31B IT, 19.9 to 57.8 on Qwen3.6 27B, 6.2 to 27.1 on Devstral Small 2 24B Instruct (results file main_models_evidence_breakdown_by_vertical.csv).

The defensive prompt works mainly against the two variants that carry explicit model-facing text: on Gemma 4 31B IT it lowers the top-three rate by 37.9 points for the AI-directed source text and 36.8 for the full-stack realistic rewrite, and by 6 to 12 points for the other six (paper, Figure 7). Instruction filtering barely moves any variant, because most strong variants contain no instruction. Rationale elicitation without an evidence check raises the top-three rate on Qwen3.6 27B and Devstral and costs 28.8 points of utility on Gemma 4 31B IT; context balancing is uneven and slightly raises the rate for the false-fit checklist.

After the best intervention the shaped candidate still reaches the top three in 39.1 to 73.2 percent of instances, far above every clean rate. Mitigation narrows the gap and does not close it.

8. What this release does not measure

• The sources are text. Real pages carry images, video, tables, layout, badges, ratings and structured metadata, and a rewrite can shape those.
• The agent answers once. Real agents ask clarifying questions, update preferences and use memory, and those turns may change vulnerability in either direction.
• Three open-weight systems of similar size and one frontier-scale system through an API are enough to show the pattern, not to rank the field.
• The clean baseline is itself a rewrite; the untouched original is also reported and behaves the same.
• The domain is products, where a constraint can be checked against a specification. The other domains of Humanity’s Last Choice have softer constraints, and the failure may look different there.

The paper’s ethics statement applies to everything derived from it: released examples use synthetic or de-identified products, shaped transformations are presented as controlled evaluation conditions and not as instructions, and system assessments should consider whether recommendations preserve the person’s constraints and avoid amplifying unsupported claims, alongside visibility and ranking.

9. Artifacts

from datasets import load_dataset
visible = load_dataset("wieeii/SafeGEO", "visible", split="test")          # model-facing inputs
labels  = load_dataset("wieeii/SafeGEO", "labels",  split="test")          # hidden benchmark reference
diamond = load_dataset("wieeii/SafeGEO", "diamond_visible", split="test")  # the screening split

10. How to cite

@misc{wen2026safegeo,
  title     = {SafeGEO: Understanding Generative Engine Optimization Risks in Recommendation Agents},
  author    = {Wen, Qianfeng and Liu, Yifan Simon and Liu, Xin and Jiao, Difan and Yang, Blair and Wu, Junda and Tang, Zhenwei},
  year      = {2026},
  eprint    = {2606.28356},
  archivePrefix = {arXiv},
  primaryClass  = {cs.IR},
  url       = {https://arxiv.org/abs/2606.28356v2}
}

11. Provenance

Results use the paper’s arXiv v2, updated 4 September 2026, and the aggregate CSV files in the code repository. The paper references are Tables 5, 6, 7, 8, 15, 22, 23, 24, 28, 31, 33, 34 and 35; Figures 5, 6 and 7; and Section C.7. Where our terminology differs, the paper’s term is given in brackets the first time. Send corrections to info@humanitylastchoice.org. Benchmark numbers are maintained in leaderboard.json and carried through to the site and figures.

• 2026-09-04. First report, based on arXiv v2.