Skip to main content

Logs and Audit

This guide covers what is recorded where, and how you can get it out. There are four ways out: the admin console's screens, CSV, a SIEM and the system integration API.

Records and where to get them​

Names follow the screen. In the SIEM column, the value in parentheses is the identifier used in SIEM Integration.

RecordWhat it recordsAdmin console screenCSVSIEMSystem integration API
Audit LogsOperations performed in the admin console“Records and Audit” > “Operation Logs” > the “Audit Logs” tabNo limitAudit Logs (audit_logs)/v1/admin/audit-logs
Operation audit logsWhat users, agents and the system carried out, such as agents' tool calls, allows and denies by execution rules, and approvals. The text itself is not kept“Records and Audit” > “Operation Logs” > the “Operation audit logs” tab—Operation audit logs (agent_action_audit)/v1/admin/action-audit-logs
User action auditActions users performed in the chat. Message content is not recorded; IP addresses and user agents are stored as fingerprints. Only system administrators and auditors can see it“Records and Audit” > “Operation Logs” > the “User action audit” tabUp to 5,000User action audit (user_action_audit). Not sent by default/v1/admin/user-action-logs
AI Agent LLM UsageWhat agents had the AI think through (how a request was understood, guardrail decisions, a summary of the result). Personal information is masked before it is kept“Records and Audit” > “AI Agent LLM Usage”Up to 1,000—/v1/admin/llm-logs
Guardrail AlertsEvents detected by guardrails. The text itself is not kept, and nothing that identifies the user is shown on screen or in the API“Records and Audit” > “Guardrail Alerts”—Guardrail Alerts (guardrail_alerts)/v1/admin/guardrail-alerts
User EvaluationsReports users sent with “Report Issue” on an AI answer. The reason (six kinds), the user, the conversation and the message are kept“Records and Audit” > “User Evaluations”No limit——
Outbound Communication RecordsOutbound traffic that was allowed or blocked“Records and Audit” > “Outbound Communication Records”Up to 5,000Outbound Communication Records (egress_access). Not sent by default/v1/admin/access-logs
Authentication eventsAuthentication events such as sign-ins——Authentication events (event_entity)—
Inbound Connection RecordsConnections to the tenant——Inbound Connection Records (ingress_access)—
LLM Gateway recordsLLM Gateway calls, and operations run through the LLM Gateway“LLM Gateway Management” > “LLM Gateway Logs” and “LLM Gateway Operation Audit”Up to 1,000LLM Gateway Logs (llm_gateway_audit), LLM Gateway Operation Audit (llm_gateway_action_audit)/v1/admin/llm-gateway-logs (LLM Gateway Logs), /v1/admin/llm-gateway-action-logs (LLM Gateway Operation Audit)
Conversation ExportConversation text, attachment lists and metadata (conversations from project runs are excluded)“Record Retention and Delivery” > “Export and Legal Hold” > the “Conversation Export” tabJSON, up to 1,000 per export. Without a period, the last 30 days——

How to use the system integration API is in the System Integration API Reference.

What an audit log entry contains​

Audit Logs record each operation performed in the admin console. Each entry holds:

ItemDetails
Executed atWhen the operation was performed
WhoUser name, email address, user ID
Client informationIP address, user agent
TargetResource (the kind of thing operated on), resource ID, target email address
ActionWhat was done (see below)
Changed fieldsValues before and after

The main actions are:

GroupActions
Registration and changesCreate, Update, Delete, Enable, Disable, Import, Export
UsersApprove, Revoke approval, Suspend, Resume, Unlock, Reset Password
PrivilegesAdd Admin Privileges, Remove Admin Privileges
OtherTransfer owner, Regenerate secret, License Activation, License Refresh, Directory stop, Directory resume, Directory bulk add, Records officer appointed, Records officer removed, Passkey deleted, Delete MCP connection, MCP tool warning

“Export All” ignores the sorting and filters on screen and writes every audit log entry to CSV, newest first. The CSV can be downloaded again until the expiry shown on screen.

Choosing a record by what you want to know​

What you want to knowRecord to look at
Who changed an administrative setting, and whenAudit Logs
Whether outbound traffic was stopped, and where it wentOutbound Communication Records
Whether content passed to the AI was stoppedGuardrail Alerts (you cannot tell whose request it was)
Whether users reported a problem with an AI answerReports
What a user did in the chatUser action audit (you cannot see the text)
What an agent operated onOperation audit logs
Who signed in or failed to, and whenAuthentication events (send them to a SIEM to see them)

Sending to a SIEM​

Under “Record Retention and Delivery” > “SIEM Integration”, decide the destination and which records to send.

Destinations​

Choose the destination in “Delivery method”, then fill in the items for that destination.

Delivery methodReceiving end
HTTP push (generic)The ingestion URL provided by the receiver
Microsoft SentinelAzure Monitor Logs Ingestion API (data collection rule)
SplunkHTTP Event Collector (HEC)
AWS CloudWatch LogsA log group

HTTP push (generic)

ItemDetails
Destination URLThe ingestion URL provided by the receiver
Auth tokenSet it when the destination requires authentication
Token header nameAuthorization if left empty
Header value format{token} is replaced with the token (for example Bearer {token}). If empty, the token is used as it is
HMAC secretSet it only when the receiver verifies a signature

Microsoft Sentinel

ItemDetails
EndpointThe logsIngestion endpoint of the data collection rule (DCR)
DCR IDShown in the Azure portal, on the data collection rule's overview
Stream nameMust match the stream name declared in the data collection rule
Tenant ID, Client ID, Client secretThose of the Microsoft Entra ID application used to send. The application needs the Monitoring Metrics Publisher role
Azure cloudAzure public / Azure China (21Vianet) / Azure US Government. For the China and US Government clouds, where authentication happens changes

Splunk

ItemDetails
Destination URLThe HTTP Event Collector (HEC) collection URL
HEC tokenThe HEC token
IndexIf empty, events go to the token's default index
Source typeUsed for classification on the Splunk side; it becomes the sourcetype when searching

AWS CloudWatch Logs

ItemDetails
RegionThe region the log group is in
Log groupThe name of the destination log group. Sending fails if it does not exist
Log streamIf empty, the identifier of the kind of record (such as audit_logs) is used as the log stream name
Access key ID, Secret access keyThe AWS credentials used to send
Create the log group and log stream automaticallyNormally leave this off. With it on, you will not notice mistakes in the destination you entered

Rules when entering them

  • Use https for external destinations. Nothing is sent to an external destination over http (audit logs and tokens would travel unencrypted)
  • Secrets and tokens are not shown after saving. Saving with them empty keeps the current value. To remove one, use the delete action on screen and then save
  • Turning “SIEM integration” off keeps the settings
  • If another administrator saved first, the screen says so. Note down what you entered, reload the latest settings, and save again

Records to send​

Choose, per kind of record, whether to send it. These are the nine kinds in the SIEM column of the table above.

  • “User action audit” and “Outbound Communication Records” are not sent by default. To send them, select and turn them on
  • When SIEM integration starts, older records are not all sent retroactively. If you need past records, get them with CSV export on screen or the system integration API
  • Under “Delivery status” you can see, for each kind of record, when it was last sent, the latest count, the total and the latest error

Format and interval​

Every minute, records added since the previous run are sent. A record usually arrives about 1–2 minutes after it is created. Each run sends up to 500 records per kind (up to 2,500 for Inbound Connection Records); anything beyond that goes in the next run.

Each record is JSON in ECS (Elastic Common Schema) form. The time is in @timestamp, the operation in event.action and event.outcome, who in user.id and user.email, the source in source.ip, and so on.

DestinationBody
HTTP push (generic){"schema_version":"v1","source":"<kind of record>","rows":[records…]}, compressed with gzip (Content-Encoding: gzip). With an HMAC secret set, X-Timestamp and X-Signature are added (X-Signature is the hex HMAC-SHA256 of the X-Timestamp value, . and the compressed body joined together)
Microsoft SentinelA JSON array of rows. The columns are TimeGenerated, SourceName, EventId, EventKind, EventCategory, EventType, EventAction, EventOutcome, EventModule, EventProvider, EventReason, UserId, UserEmail, UserName, SourceIp, UserAgent and Document (the record itself). Declare these columns in the data collection rule's stream
SplunkEach record becomes {"event": record, "time": time it occurred, "sourcetype": …}, joined with newlines. With an index set, index is added too
AWS CloudWatch LogsEach record's JSON becomes one log event. The log event's time is when the record occurred

Retention​

Audit Logs, Operation audit logs, AI Agent LLM Usage, Outbound Communication Records and LLM Gateway records are kept for 24 months. To keep them longer, send them to a SIEM and keep them on your side.

Keeping and sharing conversations​

How far conversations can leave the tenant is decided under “Connections and Usage Scope” > “Feature permissions”, and when they are deleted under “Record Retention and Delivery” > “Export and Legal Hold” > the “Retention and Legal Hold” tab.

Shared links can be stopped in two places on the Feature permissions screen.

SettingWhat it stops
“Stop shared links” under “Shared links”Every shared link in the tenant. Links already created can no longer be opened. They can still be listed and deleted. Off by default
The “Sharing” featureOnly creating and editing shared links. Choose “Not allowed” or “Selected organizations and job types” to narrow who can use it
  • Saved changes take effect within 60 seconds
  • In tenants where shared links are turned off by the deployment settings, they cannot be turned on from this screen. The screen says so

Archiving and retention​

ItemDetails
Days before archivingDays before old conversations are moved to the archive. 30–3650 days. Leave blank for the default (deployment setting)
Retention after archivingDays to keep archived conversations. 30–3650 days. Leave blank for no limit. Once it passes, the conversation and its attachments are deleted and cannot be restored
  • Changes to the days take effect from the next archiving run
  • “Values in effect” on screen shows the days before archiving

For conversations and projects that must not be deleted, apply a “Legal hold”. Choose the target (conversation or project), enter its ID and press “Apply hold”. A project ID is a UUID; a conversation ID is up to 128 characters.

  • Conversations and projects under legal hold are not deleted by their owner, by user deletion, by archiving, or when retention expires
  • Placing a legal hold on a conversation that has been moved to the archive returns it to the user's conversation list
  • Changes take effect when you press the button and are recorded in the audit log