Azure Confidential VMs

Deploy Trustee outside an AMD SEV-SNP Azure CVM and run Propeller's guest components inside it, for encrypted workloads with real vTPM attestation.

Trustee runs outside the CVM, on a separate VM you control. The CoCo guest components — Attestation Agent, CoCo Keyprovider, Proplet — run inside an AMD SEV-SNP Azure confidential VM.

That split is the point. A CVM's threat model distrusts the machine hosting it, so the component that verifies evidence and releases decryption keys must live in its own trust domain. KBS inside the CVM would hand the workload the keys it is only supposed to receive after attestation.

The same guide lives in the repository at hal/azure-sev/README.md.

Architecture

Component	Runs on	Role
KBS (Key Broker Service)	Trustee host	Validates the attestation token, releases decryption keys
AS (Attestation Service)	Trustee host	Verifies TEE evidence (`az-snp-vtpm`)
RVPS (Reference Value Provider Service)	Trustee host	Reference values for verification
Attestation Agent (AA)	CVM	Collects evidence from the vTPM, performs the RCAR handshake
CoCo Keyprovider	CVM	Image key-wrap/unwrap keyprovider for `image-rs`
Proplet	CVM	Pulls and runs the encrypted workload
Wasmtime	CVM	Executes the decrypted WASM

Where to run Trustee

Trustee runs on a separate Azure VM you control — the Trustee host — never inside the CVM and never on the Azure hypervisor. The whole point of a CVM is that the machine hosting it (and its operator) is untrusted, so the component that verifies evidence and releases decryption keys must be its own trust domain. Put the Trustee host in the same virtual network as the CVM and have the CVM reach KBS over the private network; do not expose KBS publicly.

Placement	Correct?
A separate Azure VM in the same vNet (private IP)	Yes
Any machine or server you control, reachable from the CVM	Yes
Inside the CVM	No — the keys would live with the workload
The CVM's Azure host / hypervisor	No — you do not control it

Prerequisites

Trustee host — a plain Azure VM. Docker and Docker Compose installed. No confidential SKU needed; it is the relying party, not the workload.
CVM — an Azure subscription with quota for the DCasv5 family (usually 0 by default), Azure CLI 2.38.0+, an SSH key.
The CVM must reach the Trustee host on 8082 over the private network.

Both VMs go in one resource group. Set these once and keep the same shell:

RG=propeller-hal-rg
LOCATION=eastus

Provision the CVM

az login

# DCasv5 availability varies by region — check before creating:
#   az vm list-skus --location "$LOCATION" --size Standard_DC --output table
az group create --name "$RG" --location "$LOCATION"

az vm create \
  --resource-group "$RG" \
  --name propeller-amd-cvm \
  --size Standard_DC2as_v5 \
  --image "Canonical:ubuntu-26_04-lts:server-cvm:latest" \
  --admin-username propeller \
  --ssh-key-values ~/.ssh/id_ed25519.pub \
  --security-type ConfidentialVM \
  --enable-vtpm true \
  --enable-secure-boot true \
  --os-disk-security-encryption-type VMGuestStateOnly \
  --public-ip-sku Standard

The three security flags are required, not optional:

Flag	Why
`--security-type ConfidentialVM`	Selects AMD SEV-SNP. Without it the same SKU gives a regular VM and there is no evidence at all.
`--enable-vtpm true`	Gives the guest a vTPM. The `az-snp-vtpm` attester reads `/dev/tpm0` for evidence.
`--enable-secure-boot true`	Measures the boot chain into the vTPM, binding evidence to the image you intended to run.

Confirm the settings took effect:

CVM_IP=$(az vm list-ip-addresses -g "$RG" -n propeller-amd-cvm \
  --query "[0].virtualMachine.network.privateIpAddresses[0]" -o tsv)

az vm show -g "$RG" -n propeller-amd-cvm --query "securityProfile" -o json

{
  "securityType": "ConfidentialVM",
  "uefiSettings": {
    "secureBootEnabled": true,
    "vTpmEnabled": true
  }
}

Inside the guest, dmesg should report Detected confidential virtualization sev-snp and /dev/tpm0 should exist. The absence of /dev/sev* is normal — the vTPM is the evidence source.

Deploy Trustee

All of this runs on the Trustee host, never inside the CVM.

Provision the Trustee host

CVM_VNET=$(az network vnet list -g "$RG" --query "[0].name" -o tsv)
CVM_SUBNET=$(az network vnet subnet list -g "$RG" --vnet-name "$CVM_VNET" --query "[0].name" -o tsv)

az vm create \
  --resource-group "$RG" \
  --name propeller-trustee-host \
  --size Standard_D2s_v5 \
  --image Ubuntu2404 \
  --admin-username propeller \
  --ssh-key-values ~/.ssh/id_ed25519.pub \
  --vnet-name "$CVM_VNET" \
  --subnet "$CVM_SUBNET" \
  --public-ip-sku Standard

Record the addresses and allow the CVM to reach KBS:

CVM_IP=$(az vm list-ip-addresses -g "$RG" -n propeller-amd-cvm \
  --query "[0].virtualMachine.network.privateIpAddresses[0]" -o tsv)
TRUSTEE_IP=$(az vm list-ip-addresses -g "$RG" -n propeller-trustee-host \
  --query "[0].virtualMachine.network.privateIpAddresses[0]" -o tsv)

# The NSG already holds Azure's default-allow-ssh at priority 1000 in the
# Inbound direction, and priorities must be unique per direction — so use 1010.
az network nsg rule create \
  --resource-group "$RG" \
  --nsg-name propeller-trustee-hostNSG \
  --name allow-kbs-from-cvm \
  --priority 1010 \
  --direction Inbound \
  --access Allow \
  --protocol Tcp \
  --source-address-prefixes "$CVM_IP" \
  --destination-port-ranges 8082

export TRUSTEE_HOST=$TRUSTEE_IP
export CVM_HOST=$CVM_IP

az vm show does not return addresses. Use az vm list-ip-addresses, as above.

Start the stack

KBS runs over plain HTTP inside the vNet. The hop never leaves the private subnet, and the image key is additionally wrapped under an RCAR-derived session key.

Generate the KBS signing keys:

git clone https://github.com/confidential-containers/trustee
cd trustee

openssl genpkey -algorithm ed25519 > kbs/config/private.key
openssl pkey -in kbs/config/private.key -pubout -out kbs/config/public.pub

Set KBS to HTTP and publish it on 8082 (compose defaults to 8080):

awk '/^  kbs:/,/^  [a-z]/' docker-compose.yml | grep -A3 ports

Edit kbs/config/docker-compose/kbs-config.toml:

[http_server]
sockets = ["0.0.0.0:8080"]
insecure_http = true
private_key = "/opt/confidential-containers/kbs/user-keys/tls-key.pem"
certificate = "/opt/confidential-containers/kbs/user-keys/tls-cert.pem"
tls_profile = "intermediate"

sockets is the in-container bind address and stays 0.0.0.0:8080; the host port comes from the compose mapping. insecure_http = true makes KBS ignore the key/cert paths.

docker compose up -d
docker compose ps

Verify KBS answers on the private IP — never 0.0.0.0, which is a bind address, not a connect address:

curl -s -o /dev/null -w 'http=%{http_code}\n' http://${TRUSTEE_HOST}:8082/kbs/v0/resource-policy
# expect http=401 — a JSON auth error, not a connection error

Point KBS at the Attestation Service token CA

Easy to miss, and produces a confusing failure. Trustee's setup.sh creates two independent PKI hierarchies in the same directory:

File	Purpose
`ca-cert.pem`, `ca.key`	token-signing CA (`CN=KBS-compose-root`) used by the AS
`token.key`, `token-cert.pem`, `token-cert-chain.pem`	AS EAR-token signing key/cert
`tls-*.pem`	TLS endpoints, unrelated to attestation

KBS verifies the AS-issued token against [attestation_token].trusted_certs_paths. Point it at the token CA. The wrong CA gives:

TokenVerifierError: Cannot verify token: neither trusted jwk set nor trusted pem public key works
Failed to get KEK from KBS ... request unauthorized

Extract the token CA and repoint KBS:

cd ~/trustee/kbs/config/docker-compose

# the second certificate in the chain is the token-signing root
awk '/BEGIN CERTIFICATE/,/END CERTIFICATE/' token-cert-chain.pem \
  | awk 'BEGIN{n=0} /BEGIN CERTIFICATE/{n++} n==2' > token-ca-cert.pem

openssl verify -CAfile token-ca-cert.pem token-cert.pem   # expect: OK

cd ~/trustee
sed -i 's|trusted_certs_paths = \[.*\]|trusted_certs_paths = ["/opt/confidential-containers/kbs/user-keys/token-ca-cert.pem"]|' \
  kbs/config/docker-compose/kbs-config.toml

docker compose up -d --force-recreate kbs

Create and upload the image key

kbs-client needs Intel's SGX DCAP dev headers, pulled in transitively:

sudo apt-get install -y \
  build-essential pkg-config libssl-dev clang cmake jq unzip \
  libclang-dev libsgx-dcap-quote-verify-dev

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain stable
. "$HOME/.cargo/env"

openssl rand -base64 32 | tr -d '\n' > private_key
cargo build --release

./target/release/kbs-client \
  --url http://${TRUSTEE_HOST}:8082 \
  config set-resource \
  --resource-file private_key \
  --path default/key/propeller-addition

If apt cannot find libsgx-dcap-quote-verify-dev, add Intel's repository:

curl -fsSL https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key \
  | sudo gpg --dearmor -o /usr/share/keyrings/intel-sgx.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/intel-sgx.gpg] https://download.01.org/intel-sgx/sgx_repo/ubuntu $(lsb_release -cs) main" \
  | sudo tee /etc/apt/sources.list.d/intel-sgx.list
sudo apt-get update && sudo apt-get install -y libsgx-dcap-quote-verify-dev

Set the resource policy

./target/release/kbs-client \
  --url http://${TRUSTEE_HOST}:8082 \
  config set-resource-policy \
  --policy-file kbs/sample_policies/allow_all.rego

Tighten this once the flow works. The default AS attestation policy (default_cpu) understands Azure vTPM evidence.

Build the guest stack

Everything here runs inside the CVM.

ssh -i ~/.ssh/id_ed25519 propeller@$CVM_HOST

Dependencies

sudo apt-get update
sudo apt-get install -y \
  build-essential gcc make pkg-config libssl-dev openssl curl git \
  clang cmake jq unzip libtss2-dev tpm2-tools

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain stable
. "$HOME/.cargo/env"

WASMTIME_VERSION=v48.0.2
curl -L "https://github.com/bytecodealliance/wasmtime/releases/download/${WASMTIME_VERSION}/wasmtime-${WASMTIME_VERSION}-x86_64-linux.tar.xz" -o /tmp/wasmtime.tar.xz
tar -xf /tmp/wasmtime.tar.xz -C /tmp
sudo mv /tmp/wasmtime-${WASMTIME_VERSION}-x86_64-linux/wasmtime /usr/local/bin/
rm -rf /tmp/wasmtime*

Attestation Agent and CoCo Keyprovider

Build with the az-snp-vtpm attester only. Two details matter:

Do not use all-attesters. It enables the NVIDIA attester, whose build needs the NVIDIA C++ SDK and fails on nvat.h.
Do not combine tpm-attester with the Azure attesters. If both are present, detect_attestable_devices() adds a generic TPM as an additional device, and composite evidence then demands a quote from AK handle 0x81010002 — which Azure vTPMs do not have.

cd /tmp
git clone https://github.com/rodneyosodo/guest-components.git
cd guest-components
git checkout upstream-proplet

cd attestation-agent
make ATTESTER=az-snp-vtpm-attester ttrpc=false
sudo make install

cd coco_keyprovider
cargo build --release --target x86_64-unknown-linux-gnu \
  --no-default-features --features az-snp-vtpm-attester
sudo cp ../../target/x86_64-unknown-linux-gnu/release/coco_keyprovider /usr/local/bin/

Proplet

Proplet embeds the Attestation Agent but enables no attester by default, so encrypted tasks abort before decryption. Add the Azure attester in proplet/Cargo.toml:

attestation-agent = { git = "https://github.com/rodneyosodo/guest-components", branch = "enable-wasm-workloads", default-features = false, features = [
  "rust-crypto",
  "kbs",
  "az-snp-vtpm-attester",
] }
kbs_protocol = { git = "https://github.com/rodneyosodo/guest-components", branch = "enable-wasm-workloads", features = [
  "background_check",
  "rust-crypto",
  "az-snp-vtpm-attester",
], default-features = false }

git clone https://github.com/absmach/propeller.git ~/propeller
cd ~/propeller/proplet
cargo build --release
sudo cp target/release/proplet /usr/local/bin/

Attestation Agent config

The config must end in .toml (or .json) — the loader picks its parser from the filename, and anything else fails with is not of a supported file format:

sudo tee /etc/attestation-agent.toml >/dev/null <<EOF
[token_configs]

[token_configs.kbs]
url = "http://${TRUSTEE_HOST}:8082"

[eventlog_config]
init_pcr = 17
enable_eventlog = false

[log]
level = "info"
EOF

Services

sudo tee /etc/default/proplet >/dev/null <<EOF
PROPLET_LOG_LEVEL=info
PROPLET_INSTANCE_ID=$(cat /proc/sys/kernel/random/uuid)
PROPLET_TENANT_ID=${PROPLET_TENANT_ID}
PROPLET_ENTITY_ID=${PROPLET_ENTITY_ID}
PROPLET_API_KEY=${PROPLET_API_KEY}
PROPLET_CHANNEL_ID=${PROPLET_CHANNEL_ID}
PROPLET_MQTT_ADDRESS=${PROPLET_MQTT_ADDRESS}
PROPLET_MQTT_TIMEOUT=30
PROPLET_MQTT_QOS=2
PROPLET_EXTERNAL_WASM_RUNTIME=/usr/local/bin/wasmtime
PROPLET_HAL_ENABLED=true
PROPLET_KBS_URI=http://${TRUSTEE_HOST}:8082
PROPLET_AA_CONFIG_PATH=/etc/attestation-agent.toml
PROPLET_LAYER_STORE_PATH=/tmp/proplet/layers
EOF

Install the three units (AA on 127.0.0.1:50010, CoCo Keyprovider on 127.0.0.1:50011, then Proplet):

sudo mkdir -p /run/attestation-agent /run/coco-keyprovider /var/cache/wasmtime

sudo tee /etc/systemd/system/attestation-agent.service >/dev/null <<'EOF'
[Unit]
Description=Attestation Agent for Confidential Containers
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
ExecStartPre=/bin/mkdir -p /run/attestation-agent
ExecStart=/usr/local/bin/attestation-agent --attestation_sock 127.0.0.1:50010
Restart=on-failure
RestartSec=5s
Environment=RUST_LOG=info

[Install]
WantedBy=multi-user.target
EOF

sudo tee /etc/systemd/system/coco-keyprovider.service >/dev/null <<EOF
[Unit]
Description=CoCo Keyprovider for Confidential Containers
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
ExecStartPre=/bin/mkdir -p /run/coco-keyprovider
ExecStart=/usr/local/bin/coco_keyprovider --socket 127.0.0.1:50011 --kbs http://${TRUSTEE_HOST}:8082
Restart=on-failure
RestartSec=5s
Environment=RUST_LOG=info

[Install]
WantedBy=multi-user.target
EOF

sudo tee /etc/systemd/system/proplet.service >/dev/null <<'EOF'
[Unit]
Description=Proplet WebAssembly Workload Orchestrator
After=network-online.target attestation-agent.service coco-keyprovider.service
Wants=network-online.target
Requires=attestation-agent.service coco-keyprovider.service

[Service]
Type=simple
EnvironmentFile=/etc/default/proplet
Environment=WASMTIME_HOME=/var/lib/proplet
Environment=WASMTIME_CACHE_DIR=/var/cache/wasmtime
ExecStartPre=/bin/mkdir -p /var/lib/proplet/cache /var/cache/wasmtime
ExecStartPre=/bin/sh -c 'until nc -z 127.0.0.1 50010 && nc -z 127.0.0.1 50011; do sleep 1; done'
ExecStart=/usr/local/bin/proplet
Restart=on-failure
RestartSec=5s

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable --now attestation-agent coco-keyprovider proplet

proplet.service has Requires=coco-keyprovider.service, so stopping the keyprovider also stops proplet — restart both together.

Watch the services while a workload runs. A successful proplet start logs TEE runtime initialized successfully; when a task runs, the keyprovider logs an RCAR handshake and the AS logs Verifier/endorsement check passed. tee=AzSnpVtpm:

sudo journalctl -u attestation-agent -f
sudo journalctl -u coco-keyprovider -f
sudo journalctl -u proplet -f

Encrypt and publish a WASM image

These commands run on the Trustee host. Encrypt with the same key you uploaded to KBS:

wget https://github.com/tinygo-org/tinygo/releases/download/v0.42.0/tinygo_0.42.0_amd64.deb
sudo dpkg -i tinygo_0.42.0_amd64.deb
rm -f tinygo_0.42.0_amd64.deb

wget https://github.com/engineerd/wasm-to-oci/releases/download/v0.1.2/linux-amd64-wasm-to-oci
sudo mv linux-amd64-wasm-to-oci /usr/local/bin/wasm-to-oci
sudo chmod +x /usr/local/bin/wasm-to-oci

# Build a sample WASM (from the Propeller repo)
cd ~/propeller
GOOS=js GOARCH=wasm tinygo build -buildmode=c-shared -o build/addition.wasm -target wasi examples/addition/addition.go

wasm-to-oci push build/addition.wasm docker.io/<you>/tee-wasm-addition:latest --server docker.io

mkdir -p output
docker run \
  -v "$PWD/output:/output" \
  docker.io/rodneydav/coco-keyprovider:latest \
  /encrypt.sh \
  -k "$(cat ./private_key)" \
  -i kbs:///default/key/propeller-addition \
  -s docker://docker.io/<you>/tee-wasm-addition:latest \
  -d dir:/output

sudo apt install -y skopeo
skopeo login docker.io
skopeo copy dir:$(pwd)/output docker://<you>/tee-wasm-addition:encrypted

skopeo copy pushes directly. Do not follow it with docker push docker://... — docker push rejects the scheme with invalid reference format.

Copying blob ... skipped: already exists during the copy is normal — the plaintext layer is already in the registry from the wasm-to-oci push, and only the encrypted layer needs uploading. If skopeo copy prints Writing manifest to image destination, the push succeeded.

Run an encrypted workload

Two fields are easy to get wrong:

image_url must be a bare OCI reference — no docker:// scheme. That prefix belongs to skopeo/wasm-to-oci, not to task definitions. Passing it fails with Failed to parse image reference: invalid reference format before any attestation happens.
No file field for encrypted workloads.

{
  "name": "add",
  "image_url": "rodneydav/tee-wasm-addition:encrypted",
  "encrypted": true,
  "kbs_resource_path": "default/key/propeller-addition",
  "cli_args": ["--invoke", "add"],
  "inputs": [10, 20]
}

curl -s -X POST http://localhost:7070/tasks \
  -H 'Content-Type: application/json' \
  -d '{
    "name": "add",
    "image_url": "rodneydav/tee-wasm-addition:encrypted",
    "encrypted": true,
    "kbs_resource_path": "default/key/propeller-addition",
    "cli_args": ["--invoke", "add"],
    "inputs": [10, 20]
  }'

TID=<task id>
curl -s -X POST "http://localhost:7070/tasks/$TID/start"
sleep 15
curl -s "http://localhost:7070/tasks/$TID" | jq '{state, results, error}'

If the task shows no active proplets available when started, proplet is not connected — check systemctl is-active proplet first.

state: 3 with results: "30\n" is success — 10 + 20, decrypted and executed inside the CVM. The whole chain worked: the AA produced vTPM evidence, the AS verified it (Verifier/endorsement check passed. tee=AzSnpVtpm), KBS released the key, CoCo Keyprovider unwrapped it, and image-rs decrypted the layer.

Test the HAL interfaces directly

This is independent of Trustee and only exercises the WASM-facing HAL providers. It is useful to confirm the non-TEE interfaces work inside the CVM.

cd ~/propeller
rustup target add wasm32-wasip2
make hal-runner hal-test attestation-test

./build/hal-runner ./build/hal-test.wasm

Expected:

platform-info: type=AmdSev version=0.1.0
list-capabilities: 5 entries
random(32): <64 hex chars>
system-time: <seconds>s <nanoseconds>ns
sha256(hello)=2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824
generate-keypair: ok (pub=<n>B priv=<n>B)

./build/hal-runner ./build/attestation-test.wasm --function run-attestation

Expected, with a wasmhal build that includes the fix:

platform-info: type=AmdSev version=0.1.0
attestation: ok (evidence len=12316)
evidence: 7b2276657273696f6e223a312c2274706d5f71756f7465223a...

Cleanup

cd ~/trustee && docker compose down
az group delete --name propeller-hal-rg --yes --no-wait

Encrypted workloads — what runs once the key is released, and Trustee's TLS options
HAL — the local QEMU CVM path for hosts with TEE support
Proplet — PROPLET_TEE_ENABLED, PROPLET_KBS_URI, and the other TEE variables
GCP Intel TDX Confidential VMs — a managed CVM where the hypervisor exposes the TEE device nodes directly
GCP AMD SEV-SNP Confidential VMs — the same, with AMD SEV-SNP instead of Intel TDX
hal/azure-sev/README.md — this guide as plain Markdown
Azure confidential VM guest attestation — the Microsoft sample app

ArchitectureCopy Anchor Link

Where to run TrusteeCopy Anchor Link

PrerequisitesCopy Anchor Link

Provision the CVMCopy Anchor Link

Deploy TrusteeCopy Anchor Link

Provision the Trustee hostCopy Anchor Link

Start the stackCopy Anchor Link

Point KBS at the Attestation Service token CACopy Anchor Link

Create and upload the image keyCopy Anchor Link

Set the resource policyCopy Anchor Link

Build the guest stackCopy Anchor Link

DependenciesCopy Anchor Link

Attestation Agent and CoCo KeyproviderCopy Anchor Link

PropletCopy Anchor Link

Attestation Agent configCopy Anchor Link

ServicesCopy Anchor Link

Encrypt and publish a WASM imageCopy Anchor Link

Run an encrypted workloadCopy Anchor Link

Test the HAL interfaces directlyCopy Anchor Link

CleanupCopy Anchor Link

RelatedCopy Anchor Link

On this page