How to Evaluate an Agentic CEP: A Buyer's Framework
Vendors call everything "agentic" now. Here's a practical framework for testing whether a CEP actually is, before you sign.
·
Blogs
·
Every vendor calls their CEP "agentic" now. Most aren't.
Somewhere around 2025, "agentic" replaced "AI-powered" as the word every customer engagement platform put on its homepage. The problem is that the word got adopted faster than the architecture did. A platform that added a chatbot to its campaign builder and a platform that rebuilt its execution engine to receive goals instead of instructions will both tell you they're agentic in a sales call.
That's a problem if you're the one buying. You can't tell the difference from a deck. You can only tell from specific questions and a demo that's allowed to fail in front of you.
This is a framework for asking those questions. It builds on the four capabilities that define an agentic CEP: goal input, autonomous execution, proactive monitoring, and outcome learning. If you haven't read that piece, start there for the definition. This one is for the evaluation itself, the part that happens after the definition, when you're sitting across from a vendor and need to know what to actually ask.
The core question to ask every vendor
Cut through most of the noise with one question: "Walk me through what happens after I tell your platform a goal, without me configuring anything else."
Watch what happens next. A genuinely agentic platform walks you through audience selection, content generation, send timing, and monitoring, all of it initiated by the goal statement. A platform that's AI-assisted will pivot to a builder screen and start describing settings you need to choose. The tell isn't what they say. It's how fast they end up back at a configuration UI.
A useful follow-up: "What does the platform do on its own if this campaign underperforms next week, and I don't log in?" If the honest answer involves an alert email to a human, you're looking at monitoring, not agentic response.
A practical evaluation framework
Ask for the goal-input flow, not the automation flow
Vendors default to showing you automation, because automation demos well: triggers, branches, a clean flowchart. Ask specifically to skip that and see the goal-input flow instead. Something like "reduce week-one churn by 10%" typed into the platform, and then watch what it produces without further input. If there's no such flow, and every outcome still starts with a marketer picking a trigger and a segment, that's rules with a new coat of paint.
Test proactive monitoring in a live demo, not a slide
Ask the vendor to open a real account, ideally a trial environment connected to your own event data, and show you what the platform has flagged in the last 24 to 72 hours without a human asking it to look. Not a mockup. Not a case study screenshot. If the platform can't produce this in real time, it's not doing proactive monitoring, whatever the pitch deck says.
Ask what happens when a campaign underperforms
Get a specific, procedural answer, not a values statement. "The platform learns and improves" is not an answer. "The platform detects the drop in click-through against the campaign's stated goal within X hours, generates two alternative content variants, and either tests them automatically or surfaces them for one-click approval" is an answer. If the vendor can't get that specific, they haven't built the response logic. They've built the detection logic and are hoping you don't ask about the rest.
Check whether the "agentic" features shipped natively or via acquisition
Ask directly: was this feature built in-house, or did it arrive through an acquisition or a third-party AI partnership bolted onto the existing product? Neither answer should disqualify a vendor by itself, but the answer changes what you should test next. Acquired agentic layers often work well in isolation and poorly once you need them to talk to the rest of the platform, like segmentation built before the acquisition or a data model that wasn't designed with autonomous decisioning in mind.
Get a straight answer on data requirements
Agentic execution runs on real-time behavioral data. Ask exactly what data the platform needs, how fresh it needs to be, and what breaks if that data is delayed or incomplete. A vendor that answers this precisely, down to specific event types and latency tolerances, has almost certainly built and tested this in production. A vendor that answers in generalities, "any customer data works," probably hasn't stress-tested it against a real data pipeline yet.
Red flags that signal "agentic by acquisition," not by design
If "AI" only shows up in the campaign builder, and segmentation, sending, and reporting are still fully manual, the platform bolted AI onto one step instead of rebuilding the system around it.
If every demo runs in a pre-built sandbox with sample data and the vendor resists connecting a live account, that's often because the feature hasn't been tested much outside a controlled environment.
A team that has actually built agentic execution treats "agentic" and "AI-powered" as different claims, because they had to solve for the difference. A team using them interchangeably usually hasn't drawn that line internally either.
Proactive monitoring, outcome-based adjustment, and autonomous content generation are the three hardest capabilities to build. If two of the three show up as "coming next quarter" on the roadmap, you're buying a roadmap, not a product.
"We automated our win-back flow and saved 10 hours a week" is a fine result, but it describes automation, not autonomy. It doesn't describe a platform that built the win-back flow on its own because it noticed churn risk rising.
A sample evaluation scorecard
Use this during vendor calls. Score each row 1 to 3 based on how specific and demonstrable the answer is, not how confident the vendor sounds.
Capability | Question to ask | Strong answer sounds like | Weak answer sounds like |
Goal input | "Show me a campaign that started from a goal, not a trigger." | A live walkthrough from goal statement to executed campaign, no manual configuration in between | "You'd set that up in our campaign builder using these conditions" |
Autonomous execution | "What decisions does the platform make without approval?" | A specific list: audience refresh, send timing, content variant selection | "It can suggest those, and you approve them" |
Proactive monitoring | "What has it flagged in my account in the last 72 hours?" | A real, current, unsolicited flag, shown live | A case study screenshot from another customer |
Outcome learning | "What changed automatically after your last underperforming campaign?" | A specific adjustment: send time, channel, content variant, tied to a measured result | "The model gets smarter over time" |
Data requirements | "What breaks if my event data is delayed by an hour?" | A precise answer about latency tolerances and fallback behavior | "We work with whatever data you have" |
A platform built agentic by design, Sortment included, should get through all five rows with specifics in a single working session. If a vendor needs a follow-up call to answer more than one or two of them straight, that tells you something too.
Frequently asked questions
What questions should I ask a CEP vendor to test if it's really agentic?
Ask three things directly. First, what input does the platform take: a goal, or a rule set? Second, what happens automatically when a campaign underperforms, without a marketer opening a ticket? Third, can the platform surface an opportunity you didn't ask it to look for? If the answer to any of these is a workaround involving a professional services team or a roadmap item, the platform isn't there yet.
What's the difference between "AI-assisted" and "agentic" in a CEP?
AI-assisted means a human still initiates and configures every action, and AI helps with a piece of it, like writing subject lines or suggesting a segment. Agentic means the platform receives a goal and independently builds, executes, monitors, and adjusts the campaign toward it. Most vendors marketing "AI features" in 2026 are describing the first thing while using language that implies the second.
How can I tell if a vendor's agentic features were built natively or acquired?
Ask when the feature shipped relative to any acquisitions or partnerships the company has announced, and ask to see it work on a workflow that predates the acquisition. Agentic-by-acquisition features are usually bolted onto one part of the product (often campaign creation) rather than running through segmentation, execution, and monitoring as one system. If the "agent" only works in one module and the rest of the platform still runs on manual rule-building, that's your answer.
Is it worth switching to an agentic CEP if my team is small?
Often yes, and sometimes more than for a large team. Small marketing teams are usually the ones without spare headcount to manually monitor campaigns for underperformance or build audiences for every micro-segment. An agentic CEP does the work a second or third marketer would otherwise be hired to do. The caveat is data readiness, not team size: you need clean, real-time behavioral data flowing into the platform for any of this to function.
What should a strong live demo of agentic monitoring actually show?
It should show the platform noticing something without being told to look, live, on your own data if possible. For example, a cohort's engagement dropping over the last 72 hours, flagged with a recommended response, not a report generated after you asked a dashboard for it. If the "monitoring" demo is a screenshot of a report or a slide describing what the platform "can" do, that's not proactive monitoring, that's analytics with a new name.
How long should an agentic CEP evaluation take?
Plan for two to four weeks if you're doing it properly, including at least one live demo run against a real or realistic dataset, not a canned demo environment. Rushed evaluations are exactly how "AI-assisted" gets sold as "agentic," because there's no time to test the claims. If a vendor pushes hard for a same-week close, treat that as a data point on its own.
See also
What Is an Agentic CEP?
What Is an Agentic CEP?
What Is an Agentic CEP?
An agentic CEP receives goals, not instructions — then plans, executes, and monitors campaigns autonomously. Here's what that means in practice.
An agentic CEP receives goals, not instructions — then plans, executes, and monitors campaigns autonomously. Here's what that means in practice.
See what Sortment can do for your goals.
See what Sortment can do for your goals.
Book a 30-minute call. We'll show you how the pilot works with your data and your stack.
Book a 30-minute call. We'll show you how the pilot works with your data and your stack.
AGENTS
CASE STUDIES
RESOURCES
AGENTS
CASE STUDIES
RESOURCES
Every vendor calls their CEP "agentic" now. Most aren't.
Somewhere around 2025, "agentic" replaced "AI-powered" as the word every customer engagement platform put on its homepage. The problem is that the word got adopted faster than the architecture did. A platform that added a chatbot to its campaign builder and a platform that rebuilt its execution engine to receive goals instead of instructions will both tell you they're agentic in a sales call.
That's a problem if you're the one buying. You can't tell the difference from a deck. You can only tell from specific questions and a demo that's allowed to fail in front of you.
This is a framework for asking those questions. It builds on the four capabilities that define an agentic CEP: goal input, autonomous execution, proactive monitoring, and outcome learning. If you haven't read that piece, start there for the definition. This one is for the evaluation itself, the part that happens after the definition, when you're sitting across from a vendor and need to know what to actually ask.
The core question to ask every vendor
Cut through most of the noise with one question: "Walk me through what happens after I tell your platform a goal, without me configuring anything else."
Watch what happens next. A genuinely agentic platform walks you through audience selection, content generation, send timing, and monitoring, all of it initiated by the goal statement. A platform that's AI-assisted will pivot to a builder screen and start describing settings you need to choose. The tell isn't what they say. It's how fast they end up back at a configuration UI.
A useful follow-up: "What does the platform do on its own if this campaign underperforms next week, and I don't log in?" If the honest answer involves an alert email to a human, you're looking at monitoring, not agentic response.
A practical evaluation framework
Ask for the goal-input flow, not the automation flow
Vendors default to showing you automation, because automation demos well: triggers, branches, a clean flowchart. Ask specifically to skip that and see the goal-input flow instead. Something like "reduce week-one churn by 10%" typed into the platform, and then watch what it produces without further input. If there's no such flow, and every outcome still starts with a marketer picking a trigger and a segment, that's rules with a new coat of paint.
Test proactive monitoring in a live demo, not a slide
Ask the vendor to open a real account, ideally a trial environment connected to your own event data, and show you what the platform has flagged in the last 24 to 72 hours without a human asking it to look. Not a mockup. Not a case study screenshot. If the platform can't produce this in real time, it's not doing proactive monitoring, whatever the pitch deck says.
Ask what happens when a campaign underperforms
Get a specific, procedural answer, not a values statement. "The platform learns and improves" is not an answer. "The platform detects the drop in click-through against the campaign's stated goal within X hours, generates two alternative content variants, and either tests them automatically or surfaces them for one-click approval" is an answer. If the vendor can't get that specific, they haven't built the response logic. They've built the detection logic and are hoping you don't ask about the rest.
Check whether the "agentic" features shipped natively or via acquisition
Ask directly: was this feature built in-house, or did it arrive through an acquisition or a third-party AI partnership bolted onto the existing product? Neither answer should disqualify a vendor by itself, but the answer changes what you should test next. Acquired agentic layers often work well in isolation and poorly once you need them to talk to the rest of the platform, like segmentation built before the acquisition or a data model that wasn't designed with autonomous decisioning in mind.
Get a straight answer on data requirements
Agentic execution runs on real-time behavioral data. Ask exactly what data the platform needs, how fresh it needs to be, and what breaks if that data is delayed or incomplete. A vendor that answers this precisely, down to specific event types and latency tolerances, has almost certainly built and tested this in production. A vendor that answers in generalities, "any customer data works," probably hasn't stress-tested it against a real data pipeline yet.
Red flags that signal "agentic by acquisition," not by design
If "AI" only shows up in the campaign builder, and segmentation, sending, and reporting are still fully manual, the platform bolted AI onto one step instead of rebuilding the system around it.
If every demo runs in a pre-built sandbox with sample data and the vendor resists connecting a live account, that's often because the feature hasn't been tested much outside a controlled environment.
A team that has actually built agentic execution treats "agentic" and "AI-powered" as different claims, because they had to solve for the difference. A team using them interchangeably usually hasn't drawn that line internally either.
Proactive monitoring, outcome-based adjustment, and autonomous content generation are the three hardest capabilities to build. If two of the three show up as "coming next quarter" on the roadmap, you're buying a roadmap, not a product.
"We automated our win-back flow and saved 10 hours a week" is a fine result, but it describes automation, not autonomy. It doesn't describe a platform that built the win-back flow on its own because it noticed churn risk rising.
A sample evaluation scorecard
Use this during vendor calls. Score each row 1 to 3 based on how specific and demonstrable the answer is, not how confident the vendor sounds.
Capability | Question to ask | Strong answer sounds like | Weak answer sounds like |
Goal input | "Show me a campaign that started from a goal, not a trigger." | A live walkthrough from goal statement to executed campaign, no manual configuration in between | "You'd set that up in our campaign builder using these conditions" |
Autonomous execution | "What decisions does the platform make without approval?" | A specific list: audience refresh, send timing, content variant selection | "It can suggest those, and you approve them" |
Proactive monitoring | "What has it flagged in my account in the last 72 hours?" | A real, current, unsolicited flag, shown live | A case study screenshot from another customer |
Outcome learning | "What changed automatically after your last underperforming campaign?" | A specific adjustment: send time, channel, content variant, tied to a measured result | "The model gets smarter over time" |
Data requirements | "What breaks if my event data is delayed by an hour?" | A precise answer about latency tolerances and fallback behavior | "We work with whatever data you have" |
A platform built agentic by design, Sortment included, should get through all five rows with specifics in a single working session. If a vendor needs a follow-up call to answer more than one or two of them straight, that tells you something too.
Frequently asked questions
What questions should I ask a CEP vendor to test if it's really agentic?
Ask three things directly. First, what input does the platform take: a goal, or a rule set? Second, what happens automatically when a campaign underperforms, without a marketer opening a ticket? Third, can the platform surface an opportunity you didn't ask it to look for? If the answer to any of these is a workaround involving a professional services team or a roadmap item, the platform isn't there yet.
What's the difference between "AI-assisted" and "agentic" in a CEP?
AI-assisted means a human still initiates and configures every action, and AI helps with a piece of it, like writing subject lines or suggesting a segment. Agentic means the platform receives a goal and independently builds, executes, monitors, and adjusts the campaign toward it. Most vendors marketing "AI features" in 2026 are describing the first thing while using language that implies the second.
How can I tell if a vendor's agentic features were built natively or acquired?
Ask when the feature shipped relative to any acquisitions or partnerships the company has announced, and ask to see it work on a workflow that predates the acquisition. Agentic-by-acquisition features are usually bolted onto one part of the product (often campaign creation) rather than running through segmentation, execution, and monitoring as one system. If the "agent" only works in one module and the rest of the platform still runs on manual rule-building, that's your answer.
Is it worth switching to an agentic CEP if my team is small?
Often yes, and sometimes more than for a large team. Small marketing teams are usually the ones without spare headcount to manually monitor campaigns for underperformance or build audiences for every micro-segment. An agentic CEP does the work a second or third marketer would otherwise be hired to do. The caveat is data readiness, not team size: you need clean, real-time behavioral data flowing into the platform for any of this to function.
What should a strong live demo of agentic monitoring actually show?
It should show the platform noticing something without being told to look, live, on your own data if possible. For example, a cohort's engagement dropping over the last 72 hours, flagged with a recommended response, not a report generated after you asked a dashboard for it. If the "monitoring" demo is a screenshot of a report or a slide describing what the platform "can" do, that's not proactive monitoring, that's analytics with a new name.
How long should an agentic CEP evaluation take?
Plan for two to four weeks if you're doing it properly, including at least one live demo run against a real or realistic dataset, not a canned demo environment. Rushed evaluations are exactly how "AI-assisted" gets sold as "agentic," because there's no time to test the claims. If a vendor pushes hard for a same-week close, treat that as a data point on its own.