Scaling and storage

These settings size the API and Brainstore compute for your workload. The defaults are suitable for most deployments. Tune them for high ingestion volume, heavy eval workloads, or cluster capacity constraints.

Configure AWS API ECS services

On AWS, Terraform module v6.0 introduces ECS as the target runtime for the API, split into three services by workload type. Each service scales automatically through Application Auto Scaling, adjusting its running task count between a minimum and maximum based on multiple metrics, including CPU target tracking and event loop timing. The module exposes optional variables to tune per-task size and task counts. The defaults are suitable for most deployments. See Upgrade to Terraform module v6 for the migration. 1024 CPU units equal 1 vCPU. braintrust-api handles general API traffic.

Variable	Default
`braintrust_api_cpu`	1024
`braintrust_api_memory`	8192
`braintrust_api_min_count`	3
`braintrust_api_max_count`	50

braintrust-api-ingest handles ingestion paths (/logs3, /logs3/overflow, /otel/v1/*).

Variable	Default
`braintrust_api_ingest_cpu`	1024
`braintrust_api_ingest_memory`	8192
`braintrust_api_ingest_min_count`	6
`braintrust_api_ingest_max_count`	200

braintrust-api-background handles background paths (evals, function invocation, proxy).

Variable	Default
`braintrust_api_background_cpu`	1024
`braintrust_api_background_memory`	8192
`braintrust_api_background_min_count`	3
`braintrust_api_background_max_count`	50

For example, to raise the ingestion service’s floor for a high-volume deployment:

braintrust_api_ingest_cpu = 2048
braintrust_api_ingest_min_count = 12

Raise braintrust_api_ingest_min_count for deployments with spiky ingestion volume or if you expect a launch that will suddenly increase log traffic, and braintrust_api_background_min_count for heavy eval workloads.

Configure Helm API workload isolation

This feature is available to Kubernetes deployments using Helm chart 6.13.0+. AWS ECS deployments use the Terraform-managed routing described above.

Setting api.workloadIsolation.enabled: true creates dedicated braintrust-api-ingest and braintrust-api-background Deployments and Services alongside the default braintrust-api pool. The pools share the same image and base configuration while allowing independent replica counts, resources, probes, rollout settings, and disruption budgets, so classified ingestion or background load does not consume the default API pool’s capacity. Your ingress or gateway must keep braintrust-api as its default backend and route the following paths to the specialized pools:

Pool	Paths
`braintrust-api-ingest`	`POST /logs3`, `POST /otel/v1/traces`, `POST /attachment`, `POST /attachment/status`
`braintrust-api-background`	`POST /v1/eval` (exact) and `POST /v1/eval/` (prefix), `POST /function/eval`, `POST /function/sandbox`, `POST /function/use`, `POST /function/invoke-async-batch`, `POST /function/insert-functions`, `POST /automation/logs/trigger`, and all methods for `/v1/proxy/chat/completions` and `/v1/proxy/responses`
`braintrust-api` (default)	All requests not matched by an ingest or background route

If your ingress cannot match requests by HTTP method (for example, GKE Ingress), route the listed paths for all methods instead. Pools use fixed replica counts by default. Configure them with api.replicas, api.workloadIsolation.ingest.replicas, and api.workloadIsolation.background.replicas. On GKE, you can scale each pool automatically instead. See Configure GKE API autoscaling.

Roll out workload isolation for an existing deployment

For an existing deployment, stage the rollout so you can verify each pool before routing traffic to it:

Create the pools without routing traffic to them or switching Brainstore’s internal AI proxy. Set api.workloadIsolation.brainstoreAiProxyToBackground: false and apply:
api: workloadIsolation: enabled: true brainstoreAiProxyToBackground: false
Verify the ingest and background pools are ready. Then update your ingress or gateway to route the classified paths to the new Services, and set brainstoreAiProxyToBackground: true to switch Brainstore’s internal AI proxy to the background pool:
api: workloadIsolation: enabled: true brainstoreAiProxyToBackground: true

To roll back, first return the classified paths and brainstoreAiProxyToBackground to the default API Service and verify it is serving them, then disable workload isolation in the chart. The chart cannot update an external ingress or gateway on its own, so disabling isolation before rerouting would leave the classified paths pointed at Services that no longer exist.

Integrate with the Istio VirtualService

When using the chart-managed Istio VirtualService, set virtualService.workloadIsolation.enabled: true after the isolated pools are healthy. The chart then renders the ingest and background route contract before your existing virtualService.http rules. The classified routes take precedence, so do not use virtualService.http to override a classified path.

Enable this only after the ingest and background pools are running and ready.

This option requires both virtualService.enabled: true and api.workloadIsolation.enabled: true. The chart fails to render if either is missing.

virtualService:
  workloadIsolation:
    enabled: true

Configuration reference

Key	Default	Description
`api.workloadIsolation.enabled`	`false`	Create dedicated ingest and background API pools.
`api.workloadIsolation.brainstoreAiProxyToBackground`	`true`	Route Brainstore’s internal AI proxy to the background pool. Set to `false` while staging pools for an existing deployment.
`api.workloadIsolation.ingest.name`	`braintrust-api-ingest`	Name of the ingest Deployment.
`api.workloadIsolation.ingest.replicas`	`3`	Number of ingest pod replicas.
`api.workloadIsolation.ingest.service.name`	`braintrust-api-ingest`	Name of the ingest Service.
`api.workloadIsolation.ingest.podDisruptionBudget.enabled`	`true`	Enable a PodDisruptionBudget for the ingest pool.
`api.workloadIsolation.ingest.podDisruptionBudget.minAvailable`	`1`	Minimum available ingest pods during voluntary disruptions.
`api.workloadIsolation.ingest.topologySpread.enabled`	`true`	Enable topology spread constraints for the ingest pool.
`api.workloadIsolation.background.name`	`braintrust-api-background`	Name of the background Deployment.
`api.workloadIsolation.background.replicas`	`3`	Number of background pod replicas.
`api.workloadIsolation.background.service.name`	`braintrust-api-background`	Name of the background Service.
`api.workloadIsolation.background.podDisruptionBudget.enabled`	`true`	Enable a PodDisruptionBudget for the background pool.
`api.workloadIsolation.background.podDisruptionBudget.minAvailable`	`1`	Minimum available background pods during voluntary disruptions.
`api.workloadIsolation.background.topologySpread.enabled`	`true`	Enable topology spread constraints for the background pool.
`virtualService.workloadIsolation.enabled`	`false`	Render the ingest and background route contract in the chart-managed VirtualService. Requires `virtualService.enabled: true` and `api.workloadIsolation.enabled: true`.

Default pool availability settings

Helm chart 6.13.0 or later also exposes availability settings for the default braintrust-api pool. The defaults preserve the chart’s previous behavior, so no configuration changes are required when upgrading. The ingest and background pools inherit these values as their base configuration.

Key	Default	Description
`api.strategy.type`	`RollingUpdate`	Rollout strategy for the API Deployment.
`api.strategy.rollingUpdate.maxSurge`	`100%`	Maximum number of extra pods created during a rollout.
`api.strategy.rollingUpdate.maxUnavailable`	`0`	Maximum number of pods that can be unavailable during a rollout.
`api.podDisruptionBudget.enabled`	`false`	Enable a PodDisruptionBudget for the default API pool.
`api.podDisruptionBudget.minAvailable`	`1`	Minimum available default-pool pods during voluntary disruptions.
`api.topologySpread.enabled`	`false`	Enable topology spread constraints for the default API pool.
`api.topologySpread.maxSkew`	`1`	Maximum pod-count skew across topology domains.
`api.topologySpread.topologyKey`	`topology.kubernetes.io/zone`	Node label that defines the topology domain.
`api.topologySpread.whenUnsatisfiable`	`ScheduleAnyway`	Whether the scheduler places pods when the constraint cannot be met.

Configure GKE API autoscaling

This feature is available to GKE deployments using Helm chart 6.15.0 or later. It is built on GKE’s AutoscalingMetric resource, which Google classifies as Preview (Pre-GA), so Google may change or discontinue it.

Setting api.autoscaling.enabled: true renders a HorizontalPodAutoscaler and an AutoscalingMetric for each API pool. Each pool then scales on three signals:

CPU utilization scoped to the api container, so sidecars and extraContainers do not skew the measurement.
Node.js event-loop utilization, as a ratio from 0 to 1.
Mean Node.js event-loop delay, in seconds.

These are the same three signals AWS uses through Application Auto Scaling. Default CPU (50%) and event-loop utilization (40%) match. Event-loop delay is the exception: AWS uses step scaling, which Kubernetes HPA does not support, so GKE uses target tracking at 50 ms instead. See Configure AWS API ECS services.

Prerequisites

Data plane v2.9.0 or later, which serves the Prometheus metrics endpoint on the API health server.
GKE 1.35.1-gke.1396000 or later.
The Performance HPA profile enabled on the cluster.
The Autoscaling API enabled on the cluster.
roles/autoscaling.metricsWriter granted to every node service account.
The Autoscaling API included in your service perimeter, if you use VPC Service Controls.

helm install and helm upgrade fail fast if the chart detects a missing prerequisite, rather than rendering a partial configuration:

Setting cloud to anything other than google fails, because autoscaling is supported only on GKE.
A cluster without the autoscaling.gke.io/v1beta1 API fails. Verify with kubectl api-resources | grep autoscalingmetric.

Enable autoscaling

api:
  autoscaling:
    enabled: true
    minReplicas: 4
    maxReplicas: 50

When autoscaling is enabled for a pool, the chart omits replicas from that pool’s Deployment and the HorizontalPodAutoscaler controls the replica count instead. The chart also sets ENABLE_PROMETHEUS_METRICS to true and exposes api.healthServer.port (8001 by default) on the api container so the AutoscalingMetric can scrape it. Account for that port in any network policy that restricts traffic to the API pods.

Override autoscaling per pool

With api.workloadIsolation.enabled: true, the ingest and background pools inherit every value under api.autoscaling and can override any of them under api.workloadIsolation.<pool>.autoscaling, including the metric targets and the scaling behavior:

api:
  autoscaling:
    enabled: true
    minReplicas: 3
    maxReplicas: 40
  workloadIsolation:
    enabled: true
    ingest:
      autoscaling:
        minReplicas: 3
        maxReplicas: 60
        eventLoopUtilization:
          targetAverageValue: "0.5"
        behavior:
          scaleUp:
            stabilizationWindowSeconds: 30
    background:
      autoscaling:
        minReplicas: 3
        maxReplicas: 50

This example turns on workload isolation at the same time as autoscaling, which suits a new deployment. On an existing deployment, stage the pools first. The example leaves api.workloadIsolation.brainstoreAiProxyToBackground at its default true, which switches Brainstore’s internal AI proxy to the background pool as soon as you apply it. See Roll out workload isolation for an existing deployment. Setting api.workloadIsolation.<pool>.autoscaling.enabled: false keeps that pool at a fixed replica count while its siblings continue to scale. That pool’s replicas value is honored again.

The Helm chart ships examples/google-api-isolation-autoscaling/values.yaml as a starting point for this pattern. Combine it with an Autopilot or Standard values file, which supply the rest of the configuration.

For background on the GKE resource behind this feature, see Google’s Expose custom metrics for autoscaling.

Configuration reference

Key	Default	Description
`api.autoscaling.enabled`	`false`	Autoscale every API pool. Requires `cloud: google`.
`api.autoscaling.minReplicas`	`4`	Minimum replicas for the default pool. Other pools inherit this value unless overridden.
`api.autoscaling.maxReplicas`	`50`	Maximum replicas for the default pool. Other pools inherit this value unless overridden.
`api.autoscaling.metricsPath`	`/metrics`	Prometheus metrics path on the API health server.
`api.autoscaling.cpu.targetAverageUtilization`	`50`	Target CPU utilization percentage for the `api` container.
`api.autoscaling.eventLoopUtilization.targetAverageValue`	`"0.4"`	Target event-loop utilization, as a ratio from 0 to 1. `"0.4"` is 40%.
`api.autoscaling.eventLoopDelayMean.targetAverageValue`	`"0.05"`	Target mean event-loop delay in seconds. `"0.05"` is 50 ms.
`api.autoscaling.behavior.scaleDown.stabilizationWindowSeconds`	`300`	Stabilization window before scaling down. Passed through to the HorizontalPodAutoscaler.
`api.autoscaling.behavior.scaleUp.stabilizationWindowSeconds`	`60`	Stabilization window before scaling up. Passed through to the HorizontalPodAutoscaler.
`api.annotations.hpa`	`{}`	Annotations added to each generated HorizontalPodAutoscaler.
`api.annotations.autoscalingMetric`	`{}`	Annotations added to each generated `AutoscalingMetric`.
`api.workloadIsolation.ingest.autoscaling.minReplicas`	`3`	Minimum replicas for the ingest pool.
`api.workloadIsolation.ingest.autoscaling.maxReplicas`	`200`	Maximum replicas for the ingest pool.
`api.workloadIsolation.background.autoscaling.minReplicas`	`3`	Minimum replicas for the background pool.
`api.workloadIsolation.background.autoscaling.maxReplicas`	`50`	Maximum replicas for the background pool.

Configure Brainstore fast readers

Fast readers are isolated Brainstore nodes dedicated to serving predictable UI queries (paginated viewers, span and trace lookups), preventing resource-intensive ad-hoc queries from making the UI unresponsive.

GCP and Azure: Fast readers are enabled by default starting in Helm chart v5.0.0. See the configuration reference below.
AWS: Fast readers are enabled by default (2 nodes) starting in Terraform module v5.5.0. On earlier module versions they are disabled by default. Set brainstore_fast_reader_instance_count in your Terraform configuration to control the node count, or set it to 0 to opt out (recommended for sandbox or non-production deployments).

Upgrading to Helm chart v5.0.0 from an earlier version automatically creates fast reader nodes. By default, 2 fast reader nodes are created with the same resource profile as standard reader nodes (CPU: 16, memory: 32Gi). Verify that your cluster has capacity for these additional nodes before upgrading.

If you have custom brainstore.readinessProbe overrides pointing to /status, remove them before upgrading to Helm chart v5.0.0+. The /status readiness endpoint has a bug where it never recovers after a failure, which can permanently mark Brainstore nodes as not ready. Remove any brainstore.readinessProbe or brainstore.fastreader.readinessProbe customizations and rely on the chart defaults.

Configuration reference

Fast readers are configured under the brainstore.fastreader key in your values.yaml. If you have customized brainstore.reader settings, mirror those customizations to brainstore.fastreader.

Key	Default	Description
`brainstore.fastreader.name`	`brainstore-fastreader`	Name of the Deployment, Service, and ConfigMap
`brainstore.fastreader.replicas`	`2`	Number of fast reader pod replicas
`brainstore.fastreader.service.port`	`4000`	Service port
`brainstore.fastreader.service.type`	`ClusterIP`	Kubernetes service type
`brainstore.fastreader.resources.requests.cpu`	`16`	CPU request
`brainstore.fastreader.resources.requests.memory`	`32Gi`	Memory request
`brainstore.fastreader.resources.limits.cpu`	`16`	CPU limit
`brainstore.fastreader.resources.limits.memory`	`32Gi`	Memory limit
`brainstore.fastreader.objectStoreCacheMemoryLimit`	`1Gi`	Object store memory cache limit
`brainstore.fastreader.objectStoreCacheFileSize`	`1000Gi`	Object store file cache size
`brainstore.fastreader.cacheDir`	`/mnt/tmp/brainstore`	Local cache mount path
`brainstore.fastreader.volume.size`	`""`	Ephemeral storage size; required for Azure ACS
`brainstore.fastreader.extraEnvVars`	`[]`	Additional environment variables
`brainstore.fastreader.nodeSelector`	`{}`	Node selector for scheduling
`brainstore.fastreader.tolerations`	`[]`	Pod tolerations
`brainstore.fastreader.affinity`	`{}`	Pod affinity rules

Azure
GCP Autopilot

Azure users must explicitly set brainstore.fastreader.volume.size when using Azure Container Storage (enableAzureContainerStorageDriver: true):

brainstore:
  fastreader:
    volume:
      size: "100Gi"

On GKE Autopilot, set brainstore.fastreader.volume.size to configure ephemeral storage requests. Match the resource profile of your standard reader nodes:

brainstore:
  fastreader:
    resources:
      requests:
        cpu: "16"
        memory: "32Gi"
      limits:
        cpu: "16"
        memory: "32Gi"
    objectStoreCacheFileSize: "900Gi"
    volume:
      size: "1000Gi"

Brainstore resource configuration

This section applies to GCP and Azure deployments using the Helm chart (v5.0.1+). AWS deployments manage Brainstore resources automatically.

Starting in Helm chart v5.0.1, the resources block for each Brainstore component (brainstore.reader, brainstore.writer, brainstore.fastreader) is passed through as-is to the Kubernetes pod spec. You can omit limits entirely, set them to {}, or supply any valid Kubernetes resource spec.

Omitting limits sets the pod QoS class to Burstable, which prevents CPU throttling and allows pods to use available node capacity. This can improve query performance on nodes with spare capacity, but increases the risk of resource contention if multiple pods compete for the same node.

# With limits (Guaranteed QoS — default)
brainstore:
  reader:
    resources:
      requests:
        cpu: "16"
        memory: "32Gi"
      limits:
        cpu: "16"
        memory: "32Gi"

# Without limits (Burstable QoS — allows burst CPU/memory)
brainstore:
  reader:
    resources:
      requests:
        cpu: "16"
        memory: "32Gi"
  writer:
    resources:
      requests:
        cpu: "4"
        memory: "16Gi"
  fastreader:
    resources:
      requests:
        cpu: "16"
        memory: "32Gi"

Auto-derived Brainstore environment variables

As of Helm chart v5.1.0, BRAINSTORE_RESPONSE_CACHE_URI and BRAINSTORE_CODE_BUNDLE_URI are automatically populated from your objectStorage configuration and do not need to be set manually. The chart derives these values as follows:

Cloud	Source key	`BRAINSTORE_RESPONSE_CACHE_URI`	`BRAINSTORE_CODE_BUNDLE_URI`
AWS	`objectStorage.aws.responseBucket` / `objectStorage.aws.codeBundleBucket`	`s3://<responseBucket>/brainstore-cache`	`s3://<codeBundleBucket>`
Azure	`objectStorage.azure.responseContainer` / `objectStorage.azure.codeBundleContainer`	`az://<responseContainer>/brainstore-cache`	`az://<codeBundleContainer>`
GCP	`objectStorage.google.apiBucket`	`gs://<apiBucket>/brainstore-cache`	`gs://<apiBucket>`

If you previously configured these via extraEnvVars, remove those overrides after upgrading to v5.1.0 to avoid conflicts.

Start

Instrument

Observe

Annotate

Evaluate

Deploy

Admin

Best practices

Configure AWS API ECS services

Configure Helm API workload isolation

Roll out workload isolation for an existing deployment

Integrate with the Istio VirtualService

Configuration reference

Default pool availability settings

Configure GKE API autoscaling

Prerequisites

Enable autoscaling

Override autoscaling per pool

Configuration reference

Configure Brainstore fast readers

Configuration reference

Brainstore resource configuration

Auto-derived Brainstore environment variables

​Configure AWS API ECS services

​Configure Helm API workload isolation

​Roll out workload isolation for an existing deployment

​Integrate with the Istio VirtualService

​Configuration reference

​Default pool availability settings

​Configure GKE API autoscaling

​Prerequisites

​Enable autoscaling

​Override autoscaling per pool

​Configuration reference

​Configure Brainstore fast readers

​Configuration reference

​Brainstore resource configuration

​Auto-derived Brainstore environment variables

Configure AWS API ECS services

Configure Helm API workload isolation

Roll out workload isolation for an existing deployment

Integrate with the Istio VirtualService

Configuration reference

Default pool availability settings

Configure GKE API autoscaling

Prerequisites

Enable autoscaling

Override autoscaling per pool

Configuration reference

Configure Brainstore fast readers

Configuration reference

Brainstore resource configuration

Auto-derived Brainstore environment variables