How to use the ChatGPT API to audit your own brand presence

TL;DR: Audit your brand's AI presence by sending a fixed prompt set to the ChatGPT API, logging what the model says (or doesn't) about you in category, comparison, and use-case questions, then scoring mention rate, position, sentiment, and factual accuracy. Run each prompt five times and average. You get a repeatable baseline, a competitive gap number, and a concrete list of content to fix.

Why does it matter what ChatGPT says about your brand?

AI assistants are a real discovery channel now. Perplexity reported roughly 780 million queries per month by late 2024, and OpenAI has said ChatGPT reaches more than 100 million weekly active users [1][6]. When someone asks "what's the best project management tool for remote teams" or "who makes reliable CRM software for small businesses," the answer the assistant gives shapes the buyer's shortlist before they ever open a browser tab.

SEO tells you where you rank on a results page. That model breaks down in generative AI. There's no page two. There are no ten blue links. The model names you or it doesn't.

A brand that sits at #1 on Google for a keyword can be completely missing from ChatGPT's answer to the same question. The two systems reward different signals, and you can't assume one predicts the other.

The good news: you can test your AI visibility today, systematically, using the same API that powers the product. No vendor required to start. You need an OpenAI API key, a little Python (or even a spreadsheet), and a prompt set built around the questions your buyers actually type.

For a wider view of what AI search means for discoverability, see our overview of ai search.

What does a ChatGPT API brand audit actually measure?

A good audit measures four things: mention rate, mention context, sentiment, and accuracy. Each one points to a different fix.

Mention rate is the simplest number. Out of 50 prompts a buyer might send, how many responses name your brand at all? Appear in 12 of 50 and your mention rate is 24%. There's no clean cross-industry benchmark yet (nobody has that data), so you benchmark against two or three direct competitors by running the same prompts and counting their appearances next to yours.

Mention context answers where you show up. First brand named? Second option? An "also worth a look" footnote at the bottom? Position matters a lot. Items named first or in the opening sentence of a recommendation pull disproportionate user attention and click-through intent [2].

Sentiment captures the language around your name: positive, neutral, or hedged. ChatGPT almost never trashes a brand outright, but it qualifies constantly ("X is good but has a steep learning curve," "Y is solid for enterprises but overkill for small teams"). Those qualifiers move conversion.

Accuracy is usually the most actionable one. Does the model describe your product correctly? Does it quote pricing you retired, miss a feature you shipped, or repeat a limitation you fixed? Wrong model knowledge is fixable, but only after you catch it.

These four map straight onto what you'd track in ai search visibility metrics.

How do I set up the ChatGPT API to run a brand audit?

You need an OpenAI API key from the platform dashboard. As of mid-2025, the cheapest capable model for this work is gpt-4o-mini, at $0.15 per million input tokens and $0.60 per million output tokens [3]. A 100-prompt audit with responses averaging 300 tokens each runs well under $1. Even gpt-4o at $2.50 per million input tokens keeps a 100-prompt audit under $5.

Here's the minimal Python setup:

import openai
import json

client = openai.OpenAI(api_key="YOUR_KEY")

def run_prompt(prompt_text, model="gpt-4o-mini", temperature=0.2):
    response = client.chat.completions.create(
        model=model,
        messages=[{"role": "user", "content": prompt_text}],
        temperature=temperature,
        max_tokens=500
    )
    return response.choices[0].message.content

Two settings carry the weight. Set temperature to 0.2, not 0. Zero makes the model maximally deterministic, which sounds ideal but hides edge cases where one token flip reorders a list. A low nonzero temperature gives you outputs closer to what real users get. Set max_tokens high enough to capture a full recommendation list; 400 to 600 usually covers it.

Run each prompt five times and aggregate. Outputs vary even at low temperature, and a single run can miss a brand that shows up in 3 of 5. Averaging across runs gives you a real mention rate instead of a one-shot snapshot.

Store every response in full. You want the raw text, more than a yes/no on whether you appeared. The qualifiers and factual claims you need for sentiment and accuracy scoring live in the prose.

ChatGPT API audit cost by model and prompt volume

| | | |---|---| | gpt-4o-mini, 50 prompts x5 runs | $0.11 | | gpt-4o-mini, 100 prompts x5 runs | $0.23 | | gpt-4o-mini, 150 prompts x5 runs | $0.34 | | gpt-4o, 50 prompts x5 runs | $1.88 | | gpt-4o, 100 prompts x5 runs | $3.75 | | gpt-4o, 150 prompts x5 runs | $5.63 |

Source: OpenAI API pricing, 2025

What prompts should I use to test brand visibility?

Your prompt set should span three types: category prompts, comparison prompts, and use-case prompts. A solid audit uses at least 15 to 20 prompts across all three.

Category prompts mirror how buyers open their research. Broad, no brand named:

• "What are the best email marketing tools for e-commerce?"
• "Which CRM software do most mid-sized B2B companies use?"
• "What accounting software is recommended for freelancers?"

Comparison prompts test whether you surface once buyers are deeper in the decision:

• "How does [Competitor A] compare to [Competitor B] for [use case]?"
• "What are the main alternatives to [market leader]?"

Missing from comparison prompts framed around your competitors tells you the training data doesn't tie you tightly to that category.

Use-case prompts are often the most revealing because they read like real buyer intent:

• "I run a 10-person marketing agency. What project management tool should I use?"
• "What's the best way to track customer support tickets for a SaaS startup?"

Add one more move: ask the model directly about your brand in a separate set. "What do you know about [Brand Name]?" and "What are the strengths and weaknesses of [Brand Name]?" These won't tell you how often you get recommended organically, but they surface factual errors fast.

For what kinds of queries AI engines prioritize, read about generative engine optimization.

How do I score and analyze the API responses?

Manual scoring works fine under 200 prompts. Build a spreadsheet with one row per prompt-run combination. Columns: prompt text, prompt category, raw response, brand mentioned (Y/N), mention position (1st, 2nd, 3rd+, not mentioned), sentiment qualifier (positive framing, neutral, hedged/negative), factual accuracy (accurate, minor error, significant error).

For larger sets, let the API score itself. Send the raw response back with a structured scoring prompt:

You are a brand analyst. Given this text, answer these questions in JSON:
1. Is [Brand Name] mentioned? (true/false)
2. If mentioned, what position is it in the recommendation list? (1, 2, 3, or "not a list")
3. What sentiment qualifier is used? ("positive", "neutral", "hedged")
4. Is any factual claim made about [Brand Name]? If so, is it accurate? ("accurate", "error", "none")

Text: [RESPONSE TEXT]

Meta-scoring works well, but it has its own error rate. Always hand-check a random 10% sample of the AI-scored rows to catch systematic misclassification.

Once the data is scored, lead with four numbers:

• Overall mention rate (% of prompts you appeared in)
• Category-level mention rate (you might be strong in use-case prompts but absent from category prompts, which is a real problem)
• Average mention position
• Accuracy error rate

Compare each against your top two competitors on the identical prompt set. The gap between your mention rate and theirs is your visibility deficit. The prompt categories where that gap is widest tell you exactly where to point your content.

What does low brand visibility in ChatGPT actually mean for your content strategy?

Absent from category prompts but present in use-case prompts usually means the model knows you from narrow contexts (a review, a comparison post, a Reddit thread) but doesn't tie you to the category at large. The fix is almost always more authoritative category-level content: definition pages, "what is X" explainers, industry guides that establish you as a source.

Always showing up in third position or lower points to a different issue. Competing brands carry more training signal: more mentions in high-authority publications, more citations in professional contexts, more consistent descriptions of what they do. The model learned a hierarchy, and your brand sits below.

Accuracy errors are the top priority every time. A model saying your product "doesn't integrate with Salesforce" when it has for two years, or quoting a pricing tier you killed, actively costs you deals when a buyer carries that into a sales call. You correct model knowledge by publishing clear, consistent, factual content in places the training pipeline sees: your own site, Wikipedia if you qualify, major publications, and structured data markup.

The link between content strategy and AI visibility is what ai seo covers in depth.

How is auditing with the ChatGPT API different from testing Perplexity or Gemini?

The difference is real, and you should audit across at least three AI surfaces if you have the time.

ChatGPT through the API uses a static knowledge cutoff for its base model. For gpt-4o, that cutoff is April 2024, though OpenAI updates models periodically [4]. Query the API directly with no browsing tool and you're seeing what the model learned in training, not live web results. That's the clean way to read baseline model knowledge about your brand, but it won't reflect last month's press or a page you shipped yesterday.

Perplexity retrieves live web results and synthesizes them [9]. Your visibility there depends more on current SEO and which sources Perplexity's retrieval favors.

Gemini sits between the two, with Google's search index feeding some responses [8].

To mirror what real users see in ChatGPT's consumer product (which now browses the web by default for many queries), you'd use the Responses API with the web search tool enabled. That raises cost and complexity and adds variability from live retrieval.

The cleanest place to start is the base API with no web search. It's reproducible, cheap, and it exposes the foundational model knowledge that carries into every AI surface. Add web-search testing as a second phase once you hold a baseline.

See ai-powered search features for how different AI systems retrieve and rank content.

How often should I re-run an API brand audit?

Quarterly is the floor for most brands. OpenAI ships new model versions several times a year, and each new model can carry a different knowledge distribution about your category and your name. A brand named consistently in gpt-4-turbo can fade in gpt-4o if the training mix shifted.

Run an extra audit any time you:

• Launch a major product or reposition the brand
• Land significant press coverage (to see whether it eventually reaches model knowledge)
• Notice a real shift in AI-referred traffic in analytics
• Move to a new primary keyword focus

The most useful version is a competitive audit: run your full prompt set, score your brand, then score your top two competitors on the same prompts. The relative gap beats your absolute rate. If you sit at 30% and a competitor sits at 55%, that gap is the story. If you're both stuck at 25%, that's a different problem (category-wide underrepresentation, sometimes fixable through industry-level content).

Log the model version with every run. The OpenAI API returns the model version in the response object. That versioning lets you separate "our content strategy is working" from "a new model trained on more data about us." Both are good news. They call for different next moves.

Can I automate a brand audit pipeline with the ChatGPT API?

Yes. For brands that want ongoing monitoring instead of point-in-time snapshots, automation earns its setup cost fast.

A basic pipeline: a cron job (daily or weekly) runs a fixed prompt set against the API, stores raw responses and scored outputs in a database, and drops a summary into a Slack channel or email. You track mention rate, position, and accuracy flags over time as a time series.

Prompt management is the tricky part. Your prompt set needs updates as your product changes and your category shifts. Build prompts as a versioned config file, not hardcoded strings, so you can revise them without touching your scoring logic.

One practical note on limits. OpenAI's rate limits vary by tier. The free tier caps at 3 requests per minute, which makes large-scale automation crawl. Paid tier 1 allows 500 requests per minute for gpt-4o-mini [3]. A 100-prompt audit run five times each is 500 calls. At tier 1, the whole run finishes in about a minute.

Tools that handle some of this off the shelf are worth pricing before you build. See our roundup of ai seo tools and the ai visibility tool options for what's on the market.

Spawned's own tooling automates multi-model prompt testing and tracks mention rate over time, which helps if you want the monitoring layer without building it yourself.

What content changes actually improve ChatGPT brand mentions?

This is where brand teams stall. They run the audit, find the gaps, then freeze on what to actually change. Here's what the research and practitioner evidence points to.

A 2024 analysis of AI-generated citations found that pages LLMs cite shared clear traits: they ran longer (about 1,447 words on average vs 833 for uncited pages), used more structured headers, packed specific factual claims with numbers, and skewed toward high-authority domains [5]. That study looked at Perplexity citations, but the pattern travels.

For ChatGPT's base model, training data exposure matters most. The model only knows what it trained on. Mentions in high-authority publications (major trade press, Wikipedia, well-trafficked review sites) carry more weight in training than your own blog posts. To get the model describing your product accurately, those descriptions need to exist in many places the training pipeline scraped, more than on your domain.

Five moves with the most evidence behind them:

1. Publish a clear, factual "what is [Brand]" page on your own site with accurate product descriptions, current integrations, and pricing ranges. Make unambiguous facts easy for scrapers to find.
2. Get listed on major review aggregators (G2, Capterra, Trustpilot, depending on category). These carry heavy training-data representation.
3. Earn coverage in category-defining publications. B2B SaaS might mean TechCrunch, VentureBeat, or vertical trade press. Consumer brands vary by category.
4. If you qualify for Wikipedia (significant coverage in reliable sources, notable by their standards), a Wikipedia article is one of the highest-signal training sources you can hold.
5. Build structured FAQ and comparison content that answers the exact questions buyers ask AI assistants. Training data is thick with Q&A-format content.

The generative engine optimization guide goes deeper on the signals that move AI mentions.

What are the limits of a DIY ChatGPT API audit?

A few honest limits worth stating up front.

First, the API gives you base model knowledge, not the full picture of ChatGPT with browsing on. The consumer product and the raw API can return materially different answers to the same query. Your audit covers one surface.

Second, there's no ground truth. You can measure whether ChatGPT names you, but you can't verify what share of real category queries actually go to ChatGPT vs Perplexity vs Gemini vs something else. AI search market share data is thin and contested [10]. You might be optimizing hard for a surface that holds 15% of AI search queries instead of 50%.

Third, training data is a black box. You observe outputs and infer what the model knows, but you can't inspect the training set to learn why you're underrepresented. The loop between publishing content and seeing it reflected in outputs runs slow (months to years, tied to training cycles) and unpredictable.

Fourth, prompt sensitivity is real. The same intent, phrased slightly differently, produces very different mention patterns. A brand can appear for "best project management software" but vanish for "top tools for managing remote teams" even though the intent is nearly identical. Your prompt set is a sample, not a census.

None of this makes the audit useless. It means you treat findings as directional signals, not precise measurements, and you run audits on a consistent cadence so trends outweigh any single snapshot.

For the broader context on tracking AI search performance, see ai search visibility metrics kpis.

How do I present API audit findings to my team or leadership?

Lead with the competitive gap, not your raw numbers. A mention rate of 28% means nothing on its own. A mention rate of 28% against a category leader at 61% and your nearest competitor at 44% means everything.

A one-page summary that works: current mention rate by prompt category, position distribution (what share of mentions land first vs second vs third), the top three accuracy errors found, and the two or three competitor gaps worth closing first.

For executives, anchor the findings to channel scale. At 100 million weekly ChatGPT users [1], even a 1% lift in mention rate across relevant queries is real top-of-funnel exposure. Don't overstate the line from AI mentions to revenue (that data is thin right now). The directional case is easy: buyers are asking AI assistants for recommendations, and brands that appear win consideration.

For content teams, the sharpest output is a list of specific prompts where you're absent but competitors appear, each mapped to a content gap. "We don't appear in 'best [category] tools for [use case]' prompts, Competitor A does, and we have no content addressing that use case" is a brief someone can act on tomorrow.

The scoring table from the audit doubles as your ongoing scorecard. Re-run quarterly, update the table, and the trend line tells a cleaner story than any single data point.

Sources

1. OpenAI, ChatGPT usage statistics
2. Stanford HAI, AI Index Report 2024
3. OpenAI API pricing page
4. OpenAI, model documentation
5. Search Engine Journal, coverage of LLM citation pattern analysis, 2024
6. Perplexity AI, company blog
7. OpenAI, usage policies
8. Google AI Studio, Gemini API documentation
9. Perplexity AI, developer documentation
10. Semrush, State of Search 2024