Databricks Just Made AI Gateway the Control Plane for Agentic AI: What Data Engineers Need to Know

Unity AI Gateway brings MCP servers, on-behalf-of-user execution, and AI guardrails under Unity Catalog governance.

Databricks Just Made AI Gateway the Control Plane for Agentic AI: What Data Engineers Need to Know

Yesterday, Databricks changed the shape of AI governance at the Data + AI Summit 2026.

The announcement landed with significant impact. Unity AI Gateway is now generally available as part of Unity Catalog. It governs not just LLM endpoints anymore. It governs MCP servers, coding agents, and the new Genie One agentic coworker. It governs the tools your agents reach into when they act on behalf of your users.

If you have been wondering when the agentic AI hype would meet the same governance discipline we apply to data, the answer is now.

Unity Catalog is no longer just for tables. It is becoming the control plane for everything that touches your data, including the agents that act on it.

This shift matters more than the headline suggests. Let me walk you through why.

The Problem Unity AI Gateway Solves

If you have built anything with agents in the last year, you have probably hit the same wall.

Your agent needs to read a Jira ticket. It needs to search your Confluence space. It needs to pull a Salesforce record or open a GitHub pull request. Each of these connections requires its own auth setup, its own token rotation, its own audit trail.

Multiply that by ten tools and three environments, and you have a project that takes weeks before a single agent does anything useful.

Even worse, once the agent is working, you have a new problem. The agent runs with a service account. That service account has broad permissions. So your agent can technically read anything the service account can read, regardless of who is asking.

That is not governance. That is a security incident waiting to happen.

What Changed Yesterday

Unity AI Gateway brings three things together under one roof.

First, it manages connections to external systems through MCP servers. MCP is the Model Context Protocol, an open standard for how agents talk to tools. Databricks now lets you register MCP servers directly inside Unity Catalog, with managed OAuth handling the auth dance for you.

Second, it enforces guardrails at the gateway level. PII detection. Prompt injection prevention. Hallucination guards. Data exfiltration prevention. These run before the request reaches the model and before the response leaves the gateway.

Third, and this is the part that should make every data engineer pay attention, it supports on-behalf-of-user execution. When an agent calls a tool, it inherits the exact permissions of the person who asked the question.

If Sarah asks the agent to summarize Q4 sales, the agent sees only what Sarah is allowed to see. If David asks the same question, he might get a different answer, because his permissions are different.

That is the same principle Unity Catalog has applied to tables for years. Now it applies to agents.

How This Connects to What You Already Know

If you have worked through the Unity Catalog lesson in BricksNotes, this will feel familiar.

Three-level namespaces. Fine-grained access control. Audit logs. Lineage. These were the foundations Unity Catalog built for tables, views, and volumes. The same patterns now extend to AI artifacts like Agent Bricks and Genie Ontology.

A registered MCP server lives in a catalog. It has a schema. It has owners. It has grants. You can query its usage. You can revoke access. You can trace which agent called which tool on whose behalf.

This is governance that data engineers can reason about, because it uses concepts data engineers already understand.

A Practical Walkthrough

Let me show you what registering an external MCP server looks like in practice.

Imagine you want to give an agent access to your team's GitHub repository so it can summarize open pull requests. Before yesterday, you would have built a custom connector, stored a personal access token somewhere, and hoped nobody noticed.

Now the flow looks like this.

CREATE CONNECTION github_mcp
TYPE MCP_SERVER
OPTIONS (
  endpoint 'https://mcp.github.com/v1',
  auth_type 'OAUTH',
  scopes 'repo:read,pr:read'
);

This creates a connection object inside Unity Catalog. The OAuth flow is managed by the gateway. No tokens in your code. No tokens in your notebook. No tokens in a config file.

Next, you register the MCP server as a tool the agent can call.

GRANT USE CONNECTION ON CONNECTION github_mcp
TO `data_engineering_team`;

Now anyone in the data engineering team can build an agent that uses GitHub. But here is the key part. When that agent runs, it does not run as some shared service account. It runs as the user who invoked it.

If you do not have access to a private repository, neither does the agent acting on your behalf. The permissions are inherited automatically.

Deploying the agent itself is a few more lines.

from databricks.agents import deploy_agent

deploy_agent(
    name="github_summarizer",
    model_endpoint="databricks-claude-sonnet",
    tools=["main.tools.github_mcp"],
    guardrails={
        "pii_detection": True,
        "prompt_injection": "block",
        "data_exfiltration": "block"
    }
)

That is it. The agent is registered, governed, and ready to be called from a notebook, a workflow, or a downstream application.

The guardrails are not optional add-ons you bolt on later. They are part of the deployment contract.

Why This Matters for Your Career

Data engineers have spent the last decade learning how to govern data. We learned about lineage, access control, schema enforcement, and audit logs. We learned why these things matter when something goes wrong at 2am.

Agents are about to repeat that whole learning curve. Most teams will start by giving their agents wide permissions and figuring out the consequences later.

The teams that understand Unity Catalog already have a head start. They know how to think about catalogs and schemas. They know how to set grants. They know how to read audit logs.

If you understand how Unity Catalog governs a table, you already understand how Unity AI Gateway governs an agent. The mental model is the same.

The hardest part of agentic AI is not the AI. It is the governance. And governance is what data engineers do.

What to Watch Next

A few things to keep an eye on as this rolls out.

The list of pre-built MCP integrations is growing fast. GitHub, Glean, Atlassian, and Salesforce are confirmed. Expect ServiceNow, Notion, and Linear within the next few quarters.

On-behalf-of-user execution will reshape how teams think about service accounts. Many existing automations will need to be rebuilt with user-scoped permissions in mind.

The guardrails are evolving. Expect more granular controls around output filtering, response logging, and cost attribution per user and per agent.

Where to Start

If you want to be ready for this shift, start with the foundation.

Spend time with Unity Catalog. Understand how three-level namespaces work. Practice writing GRANT statements. Read the audit logs. Get comfortable with the idea that governance is not paperwork. It is architecture.

Then explore the Medallion Architecture lesson. Agents will eventually consume bronze, silver, and gold tables. The cleaner those layers, the better your agents will behave.

The teams that win the next chapter of data engineering will be the ones who treat agents the same way they treat pipelines. With structure. With governance. With patience.

Unity AI Gateway just made that possible.

Now the question is whether you will be ready when your team asks you to deploy your first governed agent.