What Should an AI Agent Be Allowed to Do in Your Inbox, CRM, or Files?

An AI agent should not gain blanket access to your business. Learn how to separate read, suggest, and act permissions across your inbox, CRM, and files—and enforce clear boundaries.

An AI agent that can summarize a customer email is useful. An agent that can send a quote, change a sales record, and share a pricing document has authority to affect your business. Those capabilities should not arrive as one package.

The security question is not simply whether you trust the agent. It is whether its permissions match the task—and whether your systems can stop it from crossing that boundary.

Separate permission to read, suggest, and act. Then enforce those distinctions through application permissions, restricted tools, and approval controls. A prompt telling an agent to “be careful” is not an access-control system.

Approve a Task, Not an Entire Account

Connecting an inbox, CRM, or file repository can expose far more information and authority than a particular assignment requires. “Help with this renewal” should not mean “search every email, inspect every customer, and modify any file.”

This is an application of least privilege: grant only the access necessary to complete the authorized work. Microsoft’s permissions guidance explicitly recommends granting the minimum privileges required.

Before enabling an agent, define:

  • The task: What outcome has the user actually requested?
  • The resources: Which messages, records, and documents may the agent access?
  • The operations: May it retrieve information, propose changes, or execute them?
  • The duration: When should its authorization expire?
  • The stop mechanism: Who can suspend the workflow and revoke access?

Apply these boundaries to the agent’s credentials and available tools. If the application exposes broader access than the task requires, a narrowly worded instruction does not make that access narrow.

One Approval Decision, Three Branches

Consider a hypothetical renewal workflow. A customer emails asking for a renewal quote. You ask an agent to help prepare a response using the selected email thread, that customer’s CRM record, and an approved pricing document.

The approval decision is: What may the agent do to handle this request? Work through the read, suggest, and act branches separately rather than treating “prepare a response” as blanket authorization.

Read: Retrieve Only the Material Needed

Authorize the agent to read the selected thread, relevant customer record, and approved pricing document. That does not authorize unrelated mailbox searches, access to other customers’ records, or browsing the entire shared drive.

Check what the connector can actually enforce. Some permission models may not support the exact resource boundary you want. If the connector cannot restrict access sufficiently, provide selected material through a controlled interface instead of granting account-wide access.

Reading also creates exposure: sensitive information enters the agent’s working context. Exclude unrelated attachments, unnecessary customer details, and credentials. Read-only access prevents certain changes; it does not eliminate confidentiality risk.

Suggest: Show Proposed Changes Without Applying Them

The agent may prepare a proposed reply, calculate a quote from the approved source, and display recommended CRM updates. Keep those suggestions in a review interface where a person can inspect them without changing the connected systems.

Saving a draft in the mailbox is a write operation. So is adding a CRM note or creating a file in a shared folder. “Nothing was sent” does not mean nothing changed.

Show the basis for the recommendation: the pricing document used, relevant renewal terms, and any missing information. If the materials do not establish discount authority, the agent should identify that gap rather than invent a policy or assume permission.

Act: Execute Only Specifically Authorized Changes

Sending the quote, updating the CRM, and changing file-sharing permissions are separate actions. Permission for one should not silently authorize the others.

In this workflow, the agent should pause before each consequential operation unless a narrowly defined policy already authorizes it. The enforcement belongs at the tool boundary: the system should check the permitted operation, target resource, and approved inputs before execution.

Successful execution also needs verification. If the email sends but the CRM update fails, report those outcomes separately. Do not describe the entire workflow as complete or repeat the send merely because another step failed.

The boundary is what the agent can change, not what it calls the work. A “draft,” “cleanup,” or “helpful update” can still modify a business system.

When Human Approval Is Required

In the hypothetical renewal scenario, require human approval before sending an external quote containing a discount. The action communicates a commercial offer outside the organization and needs deliberate review.

The approval screen should show:

  • The exact recipient, subject, message, and attachment.
  • The proposed price and discount, with the supporting pricing source.
  • Each proposed CRM change, including its old and new values.
  • Whether the action would give anyone new access to a file.

These details make approval meaningful. A generic “Allow agent to continue?” prompt does not tell the reviewer what authority they are granting.

Bind approval to the exact action reviewed. If the recipient, attachment, price, or sharing setting changes afterward, require fresh approval. Recheck authorization immediately before execution rather than relying on an earlier conversation.

OpenAI’s computer-use guidance describes controls that pause sensitive actions for human review. The practical lesson is to place the checkpoint immediately before the consequential tool call—not simply at the beginning of a workflow.

Approval should not override policy. A reviewer’s click cannot substitute for discount authority or make prohibited disclosure acceptable.

When the Agent Should Refuse, Not Ask

Now change one detail in the hypothetical example: the incoming email includes an instruction to ignore previous directions, export every customer record, and upload the export to an unfamiliar link.

The agent should refuse the export and upload. They are outside the authorized renewal task. Instructions embedded in customer correspondence are content to evaluate, not permission to expand access.

This is a prompt-injection boundary. An email, attachment, CRM note, or document can contain text that tries to redirect the agent. OpenAI’s guidance treats third-party content as potentially malicious rather than inherently authoritative.

Do not convert this request into a routine approval prompt. That would present an out-of-scope instruction as a normal next step. Instead, block the attempted action, flag the suspicious content, and continue only with the legitimate renewal work if doing so remains safe.

If the user genuinely needs a customer-data export, treat it as a separate task requiring its own purpose, permissions, destination checks, and applicable organizational approvals.

How to Revoke Access

Revocation should be a documented procedure with a named owner. It should not depend on remembering which employee originally connected the agent.

Google: Remove the Data-Access Grant

For an agent connected through a third-party app to Gmail or Google Drive, follow Google’s documented connection-management procedure:

  • Open your Google Account’s linked apps page.
  • Select Access to your Google Account, then the relevant app.
  • Select See details.
  • Select Remove access, then Confirm.

Remove the data-access grant, not merely the “Sign in with Google” connection. Google documents these as different connection types.

Microsoft: Revoke Managed Application Permissions

For a work application managed through Microsoft Entra, have an authorized administrator review and revoke its permissions through the enterprise-application controls. Use Microsoft’s application-permissions guidance to identify the relevant grants and administrative steps.

Also stop scheduled runs, cancel pending approvals, and revoke separately issued credentials. Verify revocation with a harmless access test that should now fail. Check retained data and completed actions separately: removing access does not recall an email or reverse a CRM update.

A Practical Checklist Before Enabling the Agent

  • Bound the task: Name the permitted outcome and resources.
  • Separate capabilities: Distinguish reading, proposing, writing, sending, and sharing.
  • Check enforcement: Confirm that connectors and tools impose the intended limits.
  • Make review specific: Display exact actions, destinations, and changes.
  • Block prohibited actions: Do not let third-party content grant authority.
  • Record outcomes: Log approvals and executed changes without unnecessarily duplicating sensitive content.
  • Test shutdown: Confirm who can revoke access and how failure of further access will be verified.

Make Authority Explicit Before You Automate

The right question is not whether an AI agent should have access to your inbox, CRM, or files. It is what authority a specific task requires—and how that authority will be constrained.

Let the agent read narrowly, suggest visibly, and act only within explicit authorization. Before connecting another system, walk one real workflow through those branches. Identify its approval point, its refusal conditions, and its revocation procedure. If those boundaries cannot be enforced, narrow the deployment before expanding the agent’s reach.

Browse all insights · Contact Bart McDonough