Enterprise AI Hub, Part 3: The Shared AI Services

With private corridors in place, we move the valuable tenants into the hub: the AI services worth sharing. The goal of this part is not just to deploy them, but to configure each one for safe, multi-tenant sharing — deployed once, consumed by many, governed centrally.

The problem: five services, five configurations

In the sprawl world, each team stands up its own Azure OpenAI and AI Search. Beyond the duplicated cost, the deeper issue is drift: one team enables content filtering, another doesn't; one disables keys, another emails them around; one pins a model version, another floats. There is no shared knowledge base, so the same documents get embedded five times into five indexes. The hub replaces this with a single, well-configured catalog of services.

Azure OpenAI — the shared model endpoint

This is the centerpiece. Three configuration choices turn a default Azure OpenAI resource into a platform-grade one: a custom subdomain (required for private endpoints and Entra auth), public access disabled, and — critically — local auth disabled so the only way to call it is with an Entra identity, never a key.

resource openai 'Microsoft.CognitiveServices/accounts@2024-10-01' = {
  name: 'oai-ai-hub'
  location: 'canadacentral'
  kind: 'OpenAI'
  sku: { name: 'S0' }
  properties: {
    customSubDomainName: 'oai-ai-hub'
    publicNetworkAccess: 'Disabled'   // reachable only via the private endpoint
    disableLocalAuth: true            // no API keys — Entra ID only (see Part 5)
  }
}

On top of the account you create model deployments. Using the GlobalStandard deployment type gives you a single, shared pool of throughput measured in tokens-per-minute (TPM). A practical starting allocation mirrors the reference pattern: 30K TPM for GPT-4o and 120K TPM for embeddings, shared across all spokes.

resource gpt4o 'Microsoft.CognitiveServices/accounts/deployments@2024-10-01' = {
  parent: openai
  name: 'gpt-4o'
  sku: { name: 'GlobalStandard', capacity: 30 }     // 30K TPM, shared
  properties: {
    model: { format: 'OpenAI', name: 'gpt-4o', version: '2024-08-06' }
  }
}

resource embeddings 'Microsoft.CognitiveServices/accounts/deployments@2024-10-01' = {
  parent: openai
  name: 'text-embedding-3-large'
  sku: { name: 'GlobalStandard', capacity: 120 }    // 120K TPM, shared
  properties: {
    model: { format: 'OpenAI', name: 'text-embedding-3-large', version: '1' }
  }
}

Azure AI Search — retrieval for production RAG

Azure AI Search is the hub's retrieval engine. Configure it for RBAC-only access (no admin/query keys), private networking, and semantic ranking. The shared service hosts separate indexes per spoke, which is how you get reuse of the engine without co-mingling each team's data.

resource search 'Microsoft.Search/searchServices@2024-06-01-preview' = {
  name: 'srch-ai-hub'
  location: location
  sku: { name: 'standard' }              // S1: vector + semantic
  properties: {
    publicNetworkAccess: 'disabled'
    disableLocalAuth: true               // RBAC only — no api-keys
    semanticSearch: 'standard'
  }
}

If you want the deep mechanics of how chunking, embeddings, and vector search actually work, I covered that end-to-end in my RAG Explained series — this hub is the production home for exactly that pipeline.

The data lake and the model registry

ADLS Gen2 (Storage with hierarchical namespace) is the platform's data plane. A clean container layout keeps the lifecycle obvious: raw for source documents, processed for cleaned/chunked text, embeddings for vector artifacts, and models for any fine-tuned or registered model files. Enforce TLS 1.2, disable account keys, and front it with the Blob and DFS private endpoints from Part 2.

The Azure Machine Learning workspace provides a central model registry — versioning, lineage, and staged promotion for any model the platform manages. The workspace itself is free; only compute is billed, so it's a low-cost way to bring discipline to model management across teams.

Key Vault — secrets done once, properly

Even in a keys-disabled world you still have secrets: certificates, connection strings for non-AAD systems, signing keys. Centralize them in one Key Vault configured for RBAC authorization (not legacy access policies) with purge protection on and public access off.

resource kv 'Microsoft.KeyVault/vaults@2023-07-01' = {
  name: 'kv-ai-hub'
  location: location
  properties: {
    sku: { family: 'A', name: 'standard' }
    tenantId: tenant().tenantId
    enableRbacAuthorization: true     // grant access with roles, not policies
    enablePurgeProtection: true
    publicNetworkAccess: 'Disabled'
  }
}

What this layer bought us

References

• What is Azure OpenAI Service?
• Azure OpenAI deployment types (Global Standard, etc.)
• Vector search in Azure AI Search
• Introduction to ADLS Gen2
• Key Vault RBAC authorization guide

What's next

We now have powerful shared services — and a shared-quota risk we deliberately left open. In Part 4 we put a single, governed front door in front of all of it: Azure API Management as the AI gateway. That one choke point is where per-spoke rate limiting, response caching, multi-region load balancing, and token metering all come to life.