# GenSig Research #1 — dataset & code

*How stable are AI brand recommendations? A reproducibility and fidelity study.*
GenSig, July 2026. Article: https://gensig.app/research/ai-brand-recommendations-study.html

This bundle is the complete, reproducible material behind the study: every raw
response, the frozen prompts, the brand dictionary, the runner and the analysis
code. Nothing is aggregated away — you can recompute every number in the article.

## What's inside

```
data/runs/              2,500 API responses (5 conditions × 5 prompts × 100 runs)
  bare.jsonl              gpt-4o-mini, default temperature
  bare_t0.jsonl           gpt-4o-mini, temperature 0
  persona.jsonl           gpt-4o-mini + "you answer someone in Argentina"
  search.jsonl            gpt-4o-mini-search-preview (built-in web search)
  grounded.jsonl          gemini-flash-latest + Google Search grounding
data/browser_anchor/    30 manual consumer-app sessions (ChatGPT web, Gemini web)
  all_passes_*.csv          raw hand-collected passes
  all_digested.jsonl        normalized anchor runs
harness/
  brands.py               the 25-brand alias dictionary + detect()
  runner.py               the resumable async runner that produced data/runs/
  analysis.py             all metrics (Wilson CI, Fleiss' kappa, Kendall's W,
                          top-3 Jaccard, Spearman rho) → results/
results/
  metrics.json            every computed statistic, per condition
  summary.md              human-readable results table
```

## Each run (one JSONL line)

```json
{"id": "...", "ts": "2026-07-23T...Z", "condition": "grounded",
 "engine": "gemini", "model": "gemini-flash-latest", "prompt": "P3",
 "temperature": null, "latency_s": 4.1,
 "answer": "full verbatim response text",
 "citations": ["cited domain/title", ...]}
```

`prompt` maps to the five frozen prompts in `harness/runner.py` (P1–P5).
Brand mentions are **not** pre-labeled in the raw data — they are derived at
analysis time by `harness/brands.py::detect()`, so you can inspect or change the
alias dictionary and recompute.

## Reproduce

```bash
pip install openai requests matplotlib
# recompute all metrics from the raw runs (no API calls, no keys needed):
python harness/analysis.py
# regenerate the raw runs yourself (needs your own OPENAI_API_KEY + GEMINI_API_KEY
# in a .env; append-only and resumable):
python harness/runner.py all
```

`analysis.py` reads only `data/runs/` — you can verify every figure and table in
the article offline, with no credentials.

## Scope & limitations (as stated in the article)

One category (investing platforms), one market (Argentina), one time window; the
consumer anchor is 15 runs per surface from a single account and IP. Brand
detection is dictionary-based (aliases frozen before running). Model versions:
gpt-4o-mini, gpt-4o-mini-search-preview, gemini-flash-latest, July 2026.

## License

Data and code released for research and verification. Attribution:
*GenSig Research #1, July 2026 (gensig.app).* Questions and replications:
support@gensig.app
