Back to Campaign QA

Getting started with AdOps Agent

A quick guide to planning and running QA on Yahoo DSP campaigns.

What this app does

AdOps Agent compares what is set up in OMS against what is live in Yahoo DSP. You describe what to check in plain English, the planner extracts a structured plan, and the validator runs the checks and reports pass / fail per entity.

The three tabs

  • Setup — Describe the QA you want and review the generated plan before anything runs.
  • Units in Progress — Watch validation run live, step by step.
  • Results — Review outcomes per entity. Use the Passed / Failed cards to filter, and drill into the specific failing checks.

Setting up a QA run

  1. Open the Setup tab (default view).
  2. In the planner chat on the left, describe what you want to QA. For example:
    Check all campaigns running for Wilhite Law Firm and confirm Yahoo matches OMS.
  3. The planner extracts your request and shows a draft plan in the Planned Entities panel on the right.
  4. Review the planned checks per entity. Refine by sending follow-up messages in the chat.
  5. When the plan looks right, review it and click Execute QA to run it against Yahoo DSP.

Refining a plan

Each message in the chat updates the plan. Some patterns the planner understands:

  • Add a check: Also include impression goal.
  • Exclude checks: Don't include AOS Order ID, Retail CPM, or Tier.
  • Narrow scope: Only check campaigns starting after 2026-05-01.
  • Re-include something: Actually include AOS Order ID again.
  • Attach a source: paste a Google Sheets URL or drop a CSV / XLSX into the chat for file-based QA.

Running the plan

When the plan looks right, click Execute QA in the approval bar. This launches browser automation that queries Yahoo DSP and compares it against OMS for real — there is no separate simulated or preview mode.

If you see Nothing to validate, every planned entity exists in OMS but hasn't been pushed to Yahoo yet. Execution stays disabled until you push those campaigns to Yahoo and re-plan.

Tips

  • The planner remembers prior turns in the conversation. You can correct or refine without restating everything.
  • If a campaign exists in OMS but hasn't been pushed to Yahoo yet, the planner flags it as "Not yet in Yahoo" and skips Yahoo validation for that entity.
  • For broad reviews ("All campaign details should match"), the planner auto-expands to all OMS detail fields, then filters out anything you've asked to exclude.
  • Click Reset in the chat to clear the planner state and start over.

Best practices for talking to the agent

A little structure in your request goes a long way. These habits help the agent stay focused, fast, and accurate:

  • Name exact entities. Give clear boundaries with a specific Advertiser name, Campaign name, or ID. This keeps the agent from running overly broad searches that can hit processing limits.
  • Use files as a source of truth. By default the agent compares Yahoo DSP against the internal OMS. You can attach a CSV, Excel file, or Google Sheet link and tell the agent to validate against that file instead.
  • Group your tasks. The agent understands bulk instructions — instead of listing every line item, ask it to check all line items in a campaign or hand it a range of IDs to process in parallel.
  • Mix different checks in one request. You can combine validation tasks in a single prompt — for example, check geo-targeting on one set of line items and audience targeting on another.
  • Lean on tolerances and thresholds. Timezone differences and buffer impressions mean exact matches aren't always possible. The system's tolerance settings (like an acceptable percentage variance for impression goals, or a +/- 1 day shift on flight dates) cut down on false positives and negatives.
  • Force checks for inactive campaigns. By default the agent skips campaigns that are "Canceled", "On Hold", or "Not yet in Yahoo". If you need to QA those, explicitly ask the agent to include them.

Prompt examples

Not sure how to phrase it? Here are a few prompts to borrow from:

  • Check a specific advertiser:
    Check Advertiser Mosquito Joe of Rhode Island and [campaign name] and just that it has all the line items set up correctly.
  • Split checks by line items:
    QA line items between 4000 and 4006 for Demo Advertiser. For 4000 and 4002 check geo-targeting and for 4003 to 4006 check audience targeting.
  • Check the most recent campaigns with specific parameters:
    Check the latest three campaigns for advertiser Menards that have all the correct details in Yahoo. Check Tactiq product impressions and flight dates.
  • Use an attached file as the source of truth:
    I will run campaigns in Yahoo against CSV file [attach file].