Ep. 4 – People : The Force Multiplier of the AI CoE

Part 3: Without the right people in the right seats, your CoE becomes a slide factory.

In Part 1, I made the case that an AI CoE = Platform + People + Patterns.
In Part 2, I broke down why Platform is the non-negotiable foundation.

Now let’s talk about the second multiplier: People.

The new era of Agentic AI requires new roles in the AI CoE that turn strategy into execution and help enable AI at scale and high velocity to address challenging business problems. But most organizations start with the wrong question.

They ask: “How many people do we need?”

They should ask: “What capabilities unlock velocity?”

The difference isn’t semantic. It’s the difference between building a team that governs versus one that accelerates.

Why People Matter More Than Headcount

You can buy all the infrastructure in the world, but without the right people in the right seats, your CoE becomes a slide factory. What separates a bottleneck CoE from a multiplier CoE isn’t just talent, it’s operating model.

So, what does the right operating model actually look like?

Building the Right AI CoE Team

An AI CoE isn’t just a governance body, it’s the engine room for enterprise AI adoption. To succeed, it needs more than good intentions. It needs authority, leadership, and the right people in the room.

It starts with executive sponsorship. Budget, credibility, and organizational authority all flow from the top. Without sponsorship, the CoE is just another advisory group with no teeth.

A steering committee with business and IT leaders, backed by regular reviews with sponsors, gives the CoE the power to drive meaningful change.

Leadership then becomes the linchpin. Many organizations are now formalizing this through roles like the Chief Artificial Intelligence Officer (CAIO) and Head of AI.

Top layer: Strategy and Sponsorship

• Executive Sponsor

  • Oversees: CAIO

  • Chairs: Steering Committee

• Steering Committee

  • Members: Business and IT leaders, monthly reviews

  • Role: Prioritize portfolio, remove blockers, approve policies

• Executive Sponsor chairs the Steering Committee and is accountable for decisions.

• The CAIO operationally leads it. That means shaping the agenda, preparing recommendations, and driving follow through after votes.

Chief Artificial Intelligence Officer (CAIO)

Provides enterprise-wide AI strategy and policy.

• Scope: Organization-wide, cross-functional, and long-term.

• Focus: Sets the AI vision and governance model. Aligns AI strategy with corporate objectives, risk posture, and responsible AI principles.

• Accountability: Reports directly to the CEO or CTO and owns AI transformation at the business level.

• Primary levers: Budget, partnerships, policy, ethical AI frameworks, and enterprise integration.

• Outcome: AI becomes part of the company’s operating DNA, not just a collection of isolated projects.

The CAIO shapes how the organization approaches AI and ensures it is deployed safely and strategically across every business unit.

Head of AI

Enables Execution and operational excellence.

• Scope: Focused on delivery, implementation, and governance of AI programs.

• Focus: Leads the teams that build and run AI systems model development, platform operations, CoE execution.

• Accountability: Typically reports to the CAIO, CTO, or CIO.

• Primary levers: Roadmaps, engineering alignment, platform health, project delivery, and compliance execution.

• Outcome: AI strategy turns into production systems that deliver measurable business impact.

The Head of AI translates strategy into execution, ensuring what the CAIO envisions actually gets built, deployed, and scaled responsibly.

With sponsorship and leadership in place, the next step is to assemble the team. And this is where clarity matters most not just hiring smart people, but defining the right roles that will shape how the CoE delivers value.

The Core Roles of a Modern AI CoE

Here’s the team I see as essential:

• AI Platform Product Manager
  Owns the enterprise AI platform like a product: roadmap, backlog, adoption. Without this role, the platform drifts into shelfware.

• Ethics/Responsible AI Lead:

  • Ensures all AI systems comply with ethical, regulatory, and internal governance standards.

  • Embeds Responsible AI principles transparency, reliability, fairness, privacy, accountability, and inclusiveness into the design and delivery process. This resource should be their go-to for every use case assessment. RAI Impact Assessment Template

• AI Solution Architects
  Translates business problems into production-ready AI workloads.

  • In the era of agentic AI, this means designing systems that can perceive, reason, and act autonomously, not just predict.

  • They architect AI Agents, design the agent orchestration layer and workflows, and ensure every new build leverage proven patterns instead of starting from scratch.

• AI Security engineer

  • A security practitioner who is able to apply appropriate security controls to harden AI applications from common attacks.

  • AI security engineers are also able to control the access of AI applications to organizational resources so that those applications only have access to appropriate data.

• FinOps Analysts
  AI spend is volatile. These analysts enforce cost visibility, quotas, and optimizations so “innovation at scale” doesn’t turn into “surprise cloud bill.”

• AI Quality and Evals Engineers

  A new but critical role: build evals-as-code pipelines to automate quality, compliance, safety, and performance checks.

• AI Factory Champs
  Embedded in business units, they extend the CoE’s reach. Think of them as distributed evangelists who bring ideas back and ship pilots on the golden paths.

  

Right Sizing Your Team

Not every organization needs all eight roles. The key is matching your team structure to your scale and maturity.

Here’s the minimal viable team for most organizations:

Minimal Team (6 roles):

1. CAIO (strategy)

2. Head of AI (execution)

3. AI Platform Product Manager (includes FinOps)

4. AI Solution Architect (includes agent patterns and evals)

5. AI Trust & Safety Lead (security + ethics + RAI)

6. AI Factory Champs (federated, can’t collapse)

Scale-Based Guidance:

• <500 employees: Collapse aggressively to 4-5 roles. Your CAIO might also be your Head of AI.

• 500-5000 employees: Keep 6-7 specialized roles. You need dedicated focus but can still consolidate adjacent functions.

• 5000+ employees: Keep all 8 roles. At this scale, specialization drives efficiency.

The one role you should NEVER collapse: AI Factory Champs. They’re your distribution layer. Without them, the CoE becomes an ivory tower that builds platforms nobody uses.

Which brings us to a question every enterprise eventually faces:

Centralized vs Federated

Every enterprise wrestle with this: Should the CoE be centralized, or should business units run their own AI experiments?

My experience: the winning model is federated with a strong center.

• Centralized platform and guardrails.

• Federated execution through champs and solution leads.

• Shared accountability on cost and compliance.

My View

People are the force multiplier. The platform provides the rails, but people decide whether the train moves or derails. Get the org design and roles wrong, and the CoE becomes a bottleneck. Get them right, and you unleash AI at enterprise scale.

This is Part 3 of a 4-part series. In Part 4, I’ll shift gears to the third multiplier: Patterns - the playbook for scaling AI safely and repeatedly.

Reference:

Microsoft Blog