Cohere Opens Confidential Computing Beta for Its Model Vault AI Service

Cohere has opened a public beta of Confidential Computing for Model Vault, its managed service that runs AI models on dedicated infrastructure for enterprise customers. The company announced the beta on 22 October 2026. It runs on NVIDIA H100 GPUs on Microsoft Azure and targets material a company would rarely hand to an outside AI provider.[1]

Cohere says customers should be able to use managed AI without giving the operator access to plaintext data. Its examples include fraud investigations, privileged legal documents and proprietary source code.[1]

Encryption normally covers data in storage and in transit. Cohere says the new service also protects data in use, meaning while the model processes a request. Inference runs inside a hardware-backed confidential environment that spans the processor and the GPU. NVIDIA's confidential mode encrypts traffic between the processor and the GPU.[1][2]

A second layer protects the request itself. Cohere uses Oblivious HTTP to encrypt the application payload to a key that only the attested environment holds, so routing equipment passes along ciphertext it cannot read. Customers keep the same Model Vault API and request formats.[2]

The customer's software also checks the destination first. The environment produces signed evidence about its hardware, firmware and software, and the client compares that evidence with a policy the customer has approved. If the environment fails the policy, the client refuses to send the request.[1][2]

Cohere's engineering post separates evidence from policy. Evidence says what is running, and policy says whether the customer is willing to trust it. Cohere's chief technology officer, Phil Blunsom, says customers can verify the environment before sending data. We did not test these claims, and neither post cites an independent audit.[1][2]

Cohere's engineering post is direct about the limits. Physical and side-channel attacks remain outside the protection boundary. An operator who cannot read a workload can still stop it, so there is no availability guarantee. Prompt injection and other software flaws remain, and timing, message size and routing information stay visible.[2]

The API credential and model identifier also sit outside the encrypted payload. The post says the technology is not a data-residency tool and does not replace contracts or compliance reviews. It narrows what a customer must trust, yet the hardware, firmware, measured software and attestation service all remain trusted.[2]

Cohere says customers can create and manage encrypted vaults today. It is working toward NVIDIA B200 support on the way to general availability, with no date. It also intends to open-source the confidential inference stack, built on Kata Containers and Confidential Containers, so researchers can reproduce the measurements.[1][2]

Neither Cohere page names Azure regions, prices, performance overhead or customers. Microsoft and NVIDIA each supply a supporting statement, but no customer or independent tester appears. Until the open-source release and outside reviews exist, the security properties remain Cohere's description of a beta product.[1][2]