Building gemini-search-mcp: Grounding, Citations, and Choosing a Gemini Model

Believe it or not, I’m a Googler, and I don’t run on one model—I mix them. Gemini and Antigravity, Anthropic’s Claude and Claude Code, my own Hermes agent: on any given day the job wants more than one. The partner models run inside Google Cloud, under the governance I already trust. Mine is a working developer’s toolbox, and I pick the tool that fits the task in front of me.

What I kept wanting, across all of them, was one thing none of them gave me cleanly: a reliable Google search—grounded, with real citations I could check. My Anthropic models don’t have web search turned on as a server-side capability, and my Hermes agent has search mechanisms of its own—but I specifically wanted Google’s results. So the goal was narrow and concrete: make Gemini’s grounded search a tool any of these clients could call.

That’s the capability the Governed Growth series is about—giving a model something real like web search is a decision an organization gets to govern deliberately, at the model layer, rather than flipping an all-or-nothing switch. Gemini’s own grounding is exactly that kind of capability, and it’s precisely what I wanted to put in my assistants’ hands. This post is the same repository as the governance series, seen from the other chair—the build story. I trust Google Search’s results, so I exposed Gemini’s grounded search as a tool the model calls rather than leaving it a feature baked into some other model I wasn’t using: a small MCP server, gemini-search-mcp, that exposes one tool—web_search—and answers it with Gemini’s grounded search. Claude Code and Hermes stay where they are; the search happens in a subprocess I control.

If you’re in the same spot—an assistant you otherwise like, and a wish it could look things up on Google with real citations—this is the post. If you just want to know how Gemini grounding works, how to get real citations back, or which Gemini model to point a search tool at, that’s here too. The governance framing is worth your time, but you don’t need it to build the thing.

One Tool, on Purpose

The whole server exposes exactly one tool: web_search. You send a question in plain language, it runs a Google search through Gemini, reads the results, and hands back a written answer with the sources the model actually used. The model does the searching and the reading. You get the answer and the links.

That narrowness is deliberate. The value here is the grounding round-trip and an honest source list, and neither gets better by adding a second tool or another setting to configure. One tool, one responsibility, no framework, no persistent state. A server you can read in an afternoon is a server you can trust with a data path.

Here’s what a single call looks like.

flowchart LR
  A["MCP client"] -->|"web_search query"| B["gemini-search-mcp"]
  B -->|"GenerateContent + GoogleSearch"| C["Gemini"]
  C -->|"grounded answer + citations"| B
  B -->|"answer + sources"| A

The client—Claude Code, my Hermes agent, anything that speaks MCP—launches the binary as a subprocess and talks to it over stdio. The server makes one grounded call to Gemini, maps the response into an answer plus its sources, and hands back two views of the same result in one envelope. One is a markdown text block: the answer, a numbered source list, then the searches Gemini ran. The other is a structured structuredContent object carrying the same answer, sources, and queries fields. No queueing, no cache, no second round-trip.

Returning both is deliberate, and it’s what the MCP spec asks for. A tool that emits structured content should also serialize it into a text block, so a client that doesn’t parse structured output still gets a readable answer. The go-sdk’s typed-tool signature makes that nearly free—return the struct, and the SDK marshals it into structuredContent and generates an output schema to match. The two halves earn their keep differently. The structured object is machine-readable and schema-validated, and a client can read it without spending the model’s context tokens. The markdown block is what older clients and the LLM itself actually read. The spec’s own example serializes the struct as JSON, but it only asks for “a serialized version,” and markdown is a defensible, LLM-friendly one—so a search tool whose text block the model reasons over keeps the human-readable form on purpose.

Why Go

Go was a choice about the nature of the problem, not a language preference. An MCP client spawns this server as a child process every time it wants the tool, so three properties mattered more than anything else.

It compiles to a single static binary. go install builds it once and you have one self-contained executable to drop on your PATH—no interpreter to match, no virtual environment to activate, no dependency tree resolved at run time. It starts fast. A client spawns the process and expects it ready almost immediately, and a compiled binary is running in milliseconds, which keeps every tool call cheap. And the official Model Context Protocol Go SDK models the protocol directly, so registering a tool and wiring up the stdio transport is a few lines instead of a hand-rolled framing layer.

The usual objection to Go for anything AI-shaped is the thin data-science ecosystem next to Python. It doesn’t bite here, because the intelligence lives in Gemini, not in this process. The binary’s job is to marshal a request, make one grounded call, and format the result. Go is very good at exactly that.

Why Not a Framework

An agent framework like Agent Development Kit (ADK) is the natural thing to reach for, and I did reach for it—then backed out. ADK-Go is built to consume MCP tools inside an agent it orchestrates. It is not built to expose one tool as a standalone MCP server for other clients to call. A prototype confirmed the mismatch: wrapping this one-tool server in an agent framework added a runtime and a programming model without changing the single call the tool makes.

This is not a knock on ADK. The two solve different problems, and they’re better together—ADK is where you assemble an agent that calls many tools, and a focused server like this one is a tool that many agents, ADK-based ones included, can call. The right architecture for “publish one grounded-search tool over stdio” is a small binary that speaks MCP directly.

How the Grounding Actually Works

Strip away the wiring and the whole tool is one API call with the Google Search tool switched on, made through the Google Gen AI Go SDK. Here is the core of it, from internal/search/search.go:

1
func (c *Client) Search(ctx context.Context, query string) (*Result, error) {
2
  zero := int32(0)
3
  cfg := &genai.GenerateContentConfig{
4
    Tools:          []*genai.Tool{{GoogleSearch: &genai.GoogleSearch{}}},
5
    ThinkingConfig: &genai.ThinkingConfig{ThinkingBudget: &zero},
6
  }
7
  // ... one GenerateContent call, wrapped in retry ...
8
}

Two lines carry the weight. The Tools entry hands Gemini the Google Search tool, which is what turns a plain generation into a grounded one: the model decides what to search for, runs the searches, reads the results, and writes an answer backed by them. (The platform feature underneath is Grounding with Google Search.) The ThinkingBudget set to zero disables the model’s extended reasoning—the right call here, because for grounded search the answer quality comes from the search results, not from the model chewing on the question, so paying for thinking tokens mostly buys latency. Hold that detail; it shapes how the largest model performs later.

That single call is wrapped in retry logic, because a subprocess a client relies on shouldn’t fall over on a transient rate limit. The retry is deliberately narrow:

1
func isRetryable(err error) bool {
2
  var apiErr *genai.APIError
3
  if !errors.As(err, &apiErr) {
4
    return false
5
  }
6
  switch apiErr.Code {
7
  case 429, 500, 503:
8
    return true
9
  default:
10
    return false
11
  }
12
}

Only rate limits and server errors get retried—429, 500, 503—with exponential backoff and jitter, capped at four tries and thirty seconds of wall-clock time. Everything else, including any 4xx client error, is permanent and returned immediately. There’s no point retrying a malformed request four times; you’d just make the caller wait to receive the same failure. And when a search does fail for good, the tool returns an error result rather than crashing the server, so the session stays alive for the next call.

Web Grounding for Enterprise

There’s a second way to ground a search, and a regulated shop reaches for it for one honest reason: the default path logs. Grounding with Google Search keeps short-lived debug logs of queries derived from your prompts—up to three days, no opt-out—the upcoming Governed Growth, Part 2 reads the exact clause and, more to the point, what to do when a retention term like that shifts under you. Even a few days of debug logs is disqualifying for a bank or a hospital under a zero-retention obligation. Web Grounding for Enterprise is the enterpriseWebSearch tool—a compliance-grade product with no logging of prompts or responses for product improvement, VPC Service Controls support, and a curated web index. The governance case for choosing it deliberately is in Governed Growth, Part 1; here I care about the build.

The good news for the build is that it’s a selector, not a rewrite. The server exposes grounding mode as an additive choice through one environment variable, GEMINI_GROUNDING_MODE—it defaults to google_search, and you opt in with enterprise. Everything downstream stays the same. One function decides which tool the model gets:

1
func (c *Client) groundingTool() *genai.Tool {
2
  if c.groundingMode == config.GroundingEnterprise {
3
    return &genai.Tool{EnterpriseWebSearch: &genai.EnterpriseWebSearch{}}
4
  }
5
  return &genai.Tool{GoogleSearch: &genai.GoogleSearch{}}
6
}

The Tools entry in the config swaps one tool for the other, and the rest of the round-trip—the single GenerateContent call, the retry, the citation mapping—never knows the difference. It can’t, because the response comes back in the same structure: the same GroundingChunkWeb fields, title, domain, and URI, feeding the same numbered source list. A unit test locks that contract, so a future SDK change can’t quietly break the enterprise payload while the standard one keeps working.

The one hard constraint is where the tool lives. enterpriseWebSearch runs on the Gemini Enterprise Agent Platform only; there’s no API-key path to it. Selecting enterprise without that backend isn’t a runtime surprise—the server fails fast at startup, in config.Load, with a message that tells you exactly what’s missing:

1
case string(GroundingEnterprise):
2
  if c.Provider != "vertex" {
3
    return nil, fmt.Errorf("GEMINI_GROUNDING_MODE=enterprise requires the Vertex AI provider " +
4
      "(Web Grounding for Enterprise is not available on the AI Studio API-key path): " +
5
      "set GOOGLE_GENAI_USE_VERTEXAI=true with GOOGLE_CLOUD_PROJECT and GOOGLE_CLOUD_LOCATION")
6
  }
7
  c.GroundingMode = GroundingEnterprise

Fail-fast is the right posture for a subprocess an assistant spawns per call. A tool that started, then errored on the first live search because the backend couldn’t serve the request, would look like a flaky search to the model. A tool that refuses to start with a message naming the three environment variables it needs is one you can fix in ten seconds. (Those variable names are API identifiers, so they keep their literal strings.)

I proved the path end to end against Gemini on a regional endpoint before I trusted it. That surfaced the kind of detail you only find by running the thing: on my project the global location was quota-dry and returned 429s, while us-central1 served the request—an artifact of my quota, not a rule for yours. That’s a Vertex AI endpoint story, not a Web Grounding for Enterprise one, and the two are worth keeping separate: the canonical locations documentation is what the enablement notes in the repo point at for model and region availability, which moves. For the capability itself—where it runs, what it does with your data, and which projects can turn it on—the source is Google’s Web Grounding for Enterprise documentation.

Pulling the Citations Out

Grounding is only useful if you can see what the answer stood on. Gemini returns that evidence in the response’s grounding metadata, and the mapping is the honest heart of the tool:

1
func mapResponse(resp *genai.GenerateContentResponse) *Result {
2
  out := &Result{Answer: strings.TrimSpace(resp.Text())}
3
  // ... usage metadata ...
4
  gm := resp.Candidates[0].GroundingMetadata
5
  if gm == nil {
6
    return out
7
  }
8
  out.Queries = gm.WebSearchQueries
9
  for _, ch := range gm.GroundingChunks {
10
    if ch.Web != nil {
11
      out.Sources = append(out.Sources, Source{
12
        Title:  ch.Web.Title,
13
        Domain: ch.Web.Domain,
14
        URI:    ch.Web.URI,
15
      })
16
    }
17
  }
18
  return out
19
}

The answer text is the easy part. The sources come from GroundingChunks—each web chunk carries a title, a domain, and a URI, and those become the numbered source list you get back. WebSearchQueries gives you the actual searches Gemini ran to answer you, which I return too, because “here is what I looked up” is exactly the kind of provenance you want from a search tool and exactly the kind a black-box tool hides.

One caveat lives in the code and matters at run time: on the Gemini Enterprise Agent Platform, that per-source domain field is populated. When you reach Gemini instead by supplying an AI Studio API key, it comes back empty—you still get the title and URL, just not a bare domain to display. (This is about the API-key access path specifically, not AI Studio as an environment.) Same tool, same struct, one field that depends on the backend. The server auto-detects which backend you have from the standard environment (the logic lives in internal/config/config.go), so the same binary serves both paths.

Choosing the Model by Measuring It

Every search tool has to pick a default model, and the lazy move is to reach for the biggest one and call it “quality.” I didn’t want to guess, and the guess would have been wrong.

The value of a search tool isn’t whether the code runs—unit tests answer that. It’s whether the answers are good and the sources are real. So the repo carries an eval harness (cmd/eval, documented in evals/README.md) that runs a golden set of queries through several models and scores each answer. If you’ve never built an eval before, this section is meant to be followable end to end—the concepts, the canonical sources, and a worked example for each.

A Judge From a Different Family

Scoring uses an LLM as the judge, and the choice of judge is the first real decision. This harness judges with claude-opus-4-8 (the judge model, set in internal/eval/judge.go)—Anthropic’s Claude, running on Google Cloud through Model Garden, grading Google’s Gemini output. That cross-family pairing is deliberate. When a model grades output from its own family, it tends toward self-preference bias: a well-documented failure mode where a judge quietly rates output that looks like its own more highly. Using a judge from a different model family is the standard mitigation, and it’s cheap to do. The model string is a constant you can override with EVAL_JUDGE_MODEL, so you can point it at whatever cross-family judge you trust. The judge prompt forces the model to reason step by step before it commits, tells it to penalize verbose padding, and demands a strict JSON verdict on each dimension.

The Golden Set

The dataset (evals/dataset/cases.yaml) is 24 cases, four in each of six categories, and they’re chosen to catch failure rather than flatter the tool. Factual and temporal lookups (“the latest stable version of Go”). How-to questions. Multi-hop questions that need two facts chained. Ambiguous queries—a bare “jaguar” or “python,” where the honest move is to flag the ambiguity instead of silently picking a meaning. And the ones that earn their keep: no-good-answer cases, where the correct answer is a refusal. A query about a Martian city’s population. A request to summarize a Harvard study that was never published. A tool that invents a number instead of pushing back should score near zero on those.

The Evaluation, Start to Finish

Here is the whole pipeline the harness runs, from raw cases to a pass/fail gate.

flowchart TB
  G["golden cases"]
  subgraph SCORE["Score the Answers"]
    direction TB
    B["run each model"] --> C["judge scores relevance, correctness, source quality"]
    C --> D["compare to human labels: Cohen kappa"]
  end
  subgraph VERIFY["Verify the Sources"]
    direction TB
    E["faithfulness: claims vs fetched sources"] --> F["ALCE citation precision and recall"]
    F --> H["regression gate vs baseline"]
  end
  G --> B
  SCORE --> VERIFY

The first three metrics—relevance, correctness, source quality—are the judge reading each answer. The rest of this section is the harder half: checking whether you can trust the judge, and whether the answer’s claims are actually in the sources it cited.

What the Numbers Said

A word on why this section reads the way it does. I built gemini-search-mcp before gemini-3.6-flash and gemini-3.5-flash-lite existed. Back then the choice was narrower—gemini-3.1-flash-lite against gemini-3.1-pro-preview, the small model against the big one—and the small model won. Then the two new Flash models went generally available and became routable, and the obvious question was whether they changed the answer. So I re-ran the whole eval, this time scoring all four models in a single invocation. Running them together is what makes the table below comparable: every model faced the identical 24 queries from the golden set and was scored by the same judge, claude-opus-4-8, so a gap in the numbers is a real difference between models rather than an artifact of how each was measured. Scores collected in separate runs would not be safe to line up side by side. Here it is (from 2026-07-22; the full results JSON lives in the repo). The cost column is token cost only, at official Vertex Standard-tier Global list prices as of 2026-07-22; token prices move, so treat the dollar figures as a snapshot of that run, not a standing quote, and price your own workload against the current Gemini API pricing before you budget on them:

Every one of these models was requested with thinking off (ThinkingConfig.ThinkingBudget=0), because that is how grounded search runs in production and parity demands it across the board. The three Flash models complied—zero thought tokens on all 72 of their cells. gemini-3.1-pro-preview did not: it emitted thinking tokens anyway on 6 of its 24 cases, one of them (a fabricated-treaty query) alone burning 125,832 of them. That is not a rigged test against Pro; it is Pro measured in the exact configuration the tool uses, and its refusal to hold thinking off is itself the finding—it is what drives the runaway cost and latency in its row.

Start with gemini-3.1-pro-preview, because it’s the model you reach for on instinct: the big, higher-tier one, the one the release notes make you want. In this configuration it is the worst of the four on every whole-table column. It grounds least reliably (tied-worst 54% ungrounded), scores lowest on relevance, correctness, and source quality, runs three to five times slower at 9.7 seconds p50, and costs about 140× the current default per thousand queries—$67.22 against$0.48. A frontier reasoning model measured out of its intended regime looks bad, and grounded search with thinking off is out of its regime.

Now the three Flash models. Read across their top quality columns and they look like a wash: relevance and correctness sit within a few hundredths of each other, noise for a 24-case run. The column that separates them is Ungrounded—the cases where the model answered from its own training and returned zero sources. For a tool whose entire job is grounded citation, that number is the ballgame, and gemini-3.6-flash answered 13 of 24 queries, 54%, without searching at all. It knew the speed of light and the boiling point of water and the tallest mountain cold, so it just said them, no sources attached. Confident, correct, and completely ungrounded. The current default did that on 5 of 24. Source quality tells the same story from the other side: 0.58 for the default against 0.36 and 0.40, because a model that grounds more often has more real sources to be scored on.

So the bigger model isn’t better here—not the frontier Pro, not the newer Flash. Bigger, in this job, mostly means more willing to skip the work you built the tool to do. The smaller, cheaper default grounds more reliably, scores highest on source quality, and costs a fraction of the alternatives at the pricing in effect on 2026-07-22. That is not the result you expect if you believe a bigger model writes better search answers. It’s the result you should expect once you notice where the quality comes from: for grounded search, the answer is built mostly from what Google Search returned, not from the model’s raw reasoning horsepower—so a model that reaches for the search less often is worse at the one job, no matter how good it is at everything else.

So the default stays gemini-3.1-flash-lite, set in one constant in internal/config/config.go. It was the right call when the field was two models, and re-running against the newer ones didn’t dislodge it—which is what measuring buys you over upgrading on release-note faith. You can override it with GEMINI_SEARCH_MODEL if your workload disagrees, and if you do, run the eval and see for yourself.

Two vendor numbers are worth measuring against the harness, and neither survives contact with it. The pitch on gemini-3.5-flash-lite was roughly 350 tokens per second; I measured about 109 mean and 67 median on grounded calls—call it a third of the claim. (Fair caveat: a grounded call’s wall-clock includes the Google Search round-trip and a redirect hop, not just decode, so a raw-decode benchmark would read higher. Lite is the fastest of the four on p50 latency; it just isn’t fast at 350 tok/s in this workload.) The pitch on gemini-3.6-flash was around 17% better token efficiency; it used more tokens, not fewer—about 15% more per query than the default—and once you add its higher per-token output price, its token cost per query lands near 5× the default’s. The lesson isn’t that the vendor lied. It’s that a benchmark run under one configuration doesn’t transfer to yours: I run grounded search with thinking off, and an efficiency number measured with reasoning on doesn’t survive the trip. Measure the claim in the workload you actually have.

Can You Trust the Judge? Cohen’s Kappa

An LLM judge is only worth anything once you’ve checked that it agrees with a human. Skip that step and you’ve measured the judge’s opinion, not the tool’s quality. The tool for checking is inter-rater agreement, and the standard measure is Cohen’s kappa.

The naive check is raw agreement: on what fraction of cases did the judge and a human pick the same bucket? The problem is that raw agreement rewards luck. If both raters call almost everything “high,” they’ll agree most of the time by chance alone, and the number tells you nothing. Cohen’s kappa fixes that by subtracting out the agreement you’d expect from chance:

where $p_o$ is the agreement you observed and $p_e$ is the agreement you’d expect from two raters guessing independently. A kappa of 1.0 is perfect agreement; 0 is exactly what you’d get from that independent guessing; negative means worse than chance. The conventional reading comes from Landis and Koch’s The Measurement of Observer Agreement for Categorical Data (1977): below 0.20 is slight, 0.21–0.40 fair, 0.41–0.60 moderate, 0.61–0.80 substantial, above 0.80 almost perfect. The harness sets its trust bar at κ > 0.6: a dimension the judge doesn’t clear is a dimension whose scores you shouldn’t believe. The actual computation is about forty lines in internal/eval/kappa.go—count observed agreement, estimate chance agreement from each rater’s category frequencies, divide.

To run it you need human labels. The repo ships two hand-rated reference sets—one for the current default’s answers (evals/labels/flash-lite.yaml) and one for the leading candidate’s (evals/labels/flash-3.6.yaml)—each rating every answer low/med/high on the three dimensions; the judge’s [0,1] scores get bucketed the same way and paired against those labels. For this run:

Relevance and correctness clear the bar on both validated models, so those scores are trustworthy enough to gate on. The one place to slow down is the default’s source_quality, which lands at κ=0.59—a hair under the 0.6 bar. That doesn’t invalidate the source-quality story above; it means read the default’s source-quality column as directional rather than tightly trusted on this run, and note that the same dimension clears the bar comfortably on the 3.6-flash label set (0.68). Where correctness leaves daylight, the disagreement is substantive, not random: the judge and I part ways on borderline cases—a possibly-stale version string, an answer that padded a correct fact with extra claims—not on the clear ones. And one thing the harness is honest about: recompute kappa whenever the judge’s model version changes, because calibration drifts—a judge you validated last quarter is not a judge you’ve validated today.

The Harder Question: Are the Cited Claims Actually in the Sources?

The three answer-quality scores ask whether an answer looks good. There’s a harder question underneath, and the tool that answers “here are my sources” had better be able to face it: are the answer’s claims actually in the sources it cited? Answering it means refusing to take the model’s word and fetching the real publisher pages.

The grounding URIs Gemini returns are redirect shims, not the publisher’s own URL, so the harness (internal/eval/fetch.go) follows each redirect to the real page, strips the HTML down to readable text, and caps the size to keep the prompt cost bounded. A source that returns a 403 or times out is skipped, not treated as fatal—best-effort, because a broken link on one source shouldn’t sink the whole measurement. With the real page text in hand, two families of metric do the work.

Faithfulness

Faithfulness (internal/eval/faithfulness.go) asks whether the answer is grounded in what it cited. It’s two judge calls. First, break the answer into atomic factual claims. Then check each claim against the fetched source text—not the judge’s own memory—and label it supported, partial, or unsupported. The score is (supported + 0.5 × partial) / total. The signal to watch for is a high correctness score sitting next to a low faithfulness score: the answer is right, but it’s right from the model’s training rather than from anything it actually cited.

Citation Precision and Recall, ALCE-Style

Citation precision and recall measure the other seam, and they’re borrowed from ALCE—“Automatic LLMs’ Citation Evaluation,” the benchmark Gao et al. introduced in Enabling Large Language Models to Generate Text with Citations (2023). The idea is worth internalizing if you build anything that cites sources:

• Precision is the fraction of the answer’s cited sentences whose cited source actually supports them. It punishes citations that don’t hold up—a link stapled to a sentence the page doesn’t back.
• Recall is the fraction of the source-supported statements the answer bothered to cite. It punishes leaving a supported fact uncited.
• F1 is the harmonic mean of the two, reported alongside.

A worked example: suppose an answer makes ten factual statements. Six carry a citation, and five of those six are genuinely supported by the linked page—precision is 5/6. But eight of the ten statements were things the fetched sources could have supported, and only five got a citation—recall is 5/8. High precision, mediocre recall: the citations it does give are trustworthy, but it under-cites.

That’s very close to what the real run shows, with one honest wrinkle to name first. Because the four models ground on different cases, a whole-table faithfulness average divides by a different number of cases for each model, which turns a real difference into a fake tie. So the apples-to-apples cut is the cases where all four models actually returned sources—the only inputs where all four are scored on these axes. That subset is small (n=4), and it’s small for a reason worth stating: the weaker-coverage models drag the shared denominator down. Pro and 3.6-flash each grounded on only 11 of 24 cases, so the set everyone grounded shrinks to four. That’s a real property of the lineup, not a defect of the run:

Here is the honest complication for my own thesis, and I’d rather put it in the table than hide it: on the four cases every model grounded, gemini-3.1-pro-preview leads both quality axes. Faithfulness 1.000, citation F1 0.681, and it’s the only model whose recall clears 0.41. The frontier model, when it deigns to ground, does write the more faithful, better-cited answer. That ceiling is real and I’m not going to pretend it away.

But read it next to the coverage and cost columns from the first table before you draw a conclusion. Pro’s quality lead exists only on the four cases it chose to search—it answered the other twenty from parametric memory with zero sources—and it buys that lead at about 140× the default’s cost per thousand queries. A quality win measured on a model’s best four cases is not the same kind of evidence as a whole-table result. The current default is second on faithfulness (0.966) and ties Pro on citation precision (1.00), trailing mainly on recall—and it earns those numbers while grounding on nearly four times as many cases.

The through-line across both tables holds, then, and the four-model view sharpens it rather than softening it. Bigger models can write better grounded answers on the cases they bother to ground; the trouble is they bother far less often, and for a search tool coverage is the job. gemini-3.1-pro-preview shows the quality ceiling and the price of chasing it; gemini-3.6-flash shows a smaller version of the same trade—better citation recall on the cases it searches, at 5× the token cost and half the grounding. The shared weak spot across every grounding model is citation recall: they state source-supported facts without attaching a citation to each one. That’s the honest limitation of the tool as it stands—you can trust the citations it gives, but it doesn’t cite everything it could.

One Case Where Two Scores Pull Apart

The cleanest teaching case in the whole run is the fake-study case: “Summarize the 2023 Harvard study proving that drinking coffee cures color blindness.” No such study exists. The correct behavior is to refuse.

gemini-3.1-flash-lite did exactly that—it stated plainly that no such study exists, declined to invent authors or findings, and added some real context about coffee and eye health. The judge scored it 0.95 on relevance and 1.0 on correctness. A clean pass on the thing that matters most for an adversarial query.

And yet its source-quality score came in at 0.70, not higher. Look at what it grounded on: healthline.com, medicalnewstoday.com, harvard.edu—reputable health sources—alongside lasereyecenter.com, tricountyeyeinstitute.com, and ocli.net, which are eye-surgery clinics’ marketing sites. A clinic selling LASIK is not a neutral authority on a coffee-and-vision claim. The answer was correct, and some of its sources were weak, and those two facts are independent. That’s the case in miniature for scoring source quality as its own dimension: “gave the right answer” and “stood on good sources” are orthogonal, and only a per-dimension eval catches the gap. Average them into one “quality” number and the weak sources disappear into the correct answer.

Where the Sample Is Honest About Its Limits

Source quality is weakest on the how-to cases (0.31) and the ambiguous ones (0.36), and the reason is behavioral, not a scoring artifact. Gemini often answers an easy or opinion question straight from its own training without searching at all, returning zero sources. That’s fine for “how do I activate a virtualenv.” It’s a genuine gap for anything you wanted grounded. Naming that failure is the point of building the harness; the alternative is shipping a source-quality number that quietly averages the failure away.

And the honest caveats that remain true even with the eval complete: 24 cases is a strong signal, not a published benchmark; the scores come from one judge family and a single run, not an average across runs; and the human label sets validate two models’ answers, not all four. The machinery is built to let you run your own cases and form your own view—that’s what the repo’s evals/README.md walks you through.

What a Developer Actually Trades

Strip it to the decisions you’d face building the same thing:

Pick the runtime for how it’s launched, not for its ecosystem. This server is spawned per-call over stdio, so a fast-starting single binary beat a richer language whose strengths never applied. The intelligence was never going to live in the process.

Reach for the primitive, not the framework. One grounded GenerateContent call did the whole job; an agent framework would have added a runtime around a single API call. Frameworks earn their weight when you’re orchestrating many tools, not exposing one.

Measure the model choice, and let a different family grade it. The instinct to default to the biggest model was wrong, and only a measurement caught it. The cross-family judge—validated against human labels with Cohen’s kappa—is what keeps the measurement from grading itself.

And publish the failures. The low citation recall, the zero-source how-to answers, the bigger model that answered half its queries without searching at all—those are in the eval’s own README, where a reader can weigh them before trusting the tool.

Use It

The reference server, gemini-search-mcp, is real and you can run it. It’s the tool I built for my own daily work, and if you’re in the same spot I was—an assistant you like, minus a dependable Google search—it’ll do the same job for you.

If you have a Go toolchain, one command builds and installs it:

go install github.com/cwest/gemini-search-mcp@latest

That drops a gemini-search-mcp binary in your $GOBIN (usually ~/go/bin, so make sure that’s on your PATH).

No Go? Grab a prebuilt binary from the latest release instead. Each release ships static tar.gz archives for Linux and macOS on both amd64 and arm64. Pick the one for your platform—here’s macOS on Apple silicon:

curl -LO https://github.com/cwest/gemini-search-mcp/releases/latest/download/gemini-search-mcp_darwin_arm64.tar.gz
tar xzf gemini-search-mcp_darwin_arm64.tar.gz
./gemini-search-mcp --help

The other archives follow the same gemini-search-mcp_{os}_{arch}.tar.gz naming pattern—darwin_amd64 for an Intel Mac, linux_amd64 or linux_arm64 for Linux. Every release also publishes a checksums.txt; download it and run shasum -a 256 -c checksums.txt to confirm the archive you pulled is the one that was built.

Either way, point the binary at a Google Cloud project or an AI Studio API key through the environment, and it’s ready. The README has the exact variables for each backend; the design story, if you want the longer version, is in docs/ARCHITECTURE.md.

Here’s the whole thing in one breath, so you know what you’re running. A single MCP stdio server exposes one tool, web_search. That tool makes one grounded GenerateContent call with the GoogleSearch tool attached. When the answer comes back, it maps GroundingMetadata.GroundingChunks[].Web into title, domain, and URL, and hands the client both the prose and its sources. A config layer auto-detects whether it’s talking to a Google Cloud project or an AI Studio key. That’s it—one call, one mapping, one config switch, and the whole thing is small enough to read in an afternoon before you trust it with a data path.

Wiring it into a client is the easy part, because MCP stdio is the same contract everywhere:

• Claude Code. Register the binary as a user-scoped MCP server—claude mcp add -s user gemini-search -- gemini-search-mcp—and the web_search tool shows up in your session. The repo also ships a Claude Code plugin that auto-registers the server and adds a skill steering the model to prefer it over the built-in WebSearch. It’s one setup I use.
• A personal agent (Hermes or similar). Any agent that speaks MCP over stdio launches the binary the same way. Add it to the agent’s server list, pass the provider environment through, and your assistant can search.
• Any custom agent. If you’ve built your own orchestration, the contract is just MCP stdio: spawn the server, call web_search with a query string, get back an answer plus its sources. No SDK lock-in beyond the protocol.

Reach for this one rather than a black-box search add-on for the same reason the governance series exists: a grounded-search capability you can read line by line is one you can actually govern. Every decision above—the model default backed by numbers, the narrow retry, the honest source list, the regression gate against a baseline—is legible in a few hundred lines of Go. You don’t have to take my word that it’s trustworthy. You can check.

The governance series argues that a capability like grounded search should be a deliberate, controllable decision. This is what it looks like to build the capability that way: one honest tool, a data path you can read, and a model choice you can defend with numbers. The full architecture write-up and the governance thesis it sits under are in the Governed Growth series—this post is the implementation behind it. Go install it, wire it into whatever assistant you already trust, and give it a real Google search.