cloudzero-charts

CloudZero Agent Helm Chart

Contributor Covenant License GitHub release

A Helm chart for deploying Prometheus in agent mode to send cluster metrics to the CloudZero platform.

For the latest release, see Releases. You can also enable release notifications.

Prerequisites

For an optimal installation experience, we recommend the following:

Installation

Quick Start

⚠️ By default, this chart exports only the app.kubernetes.io/component label from pods and namespaces. No annotations are exported. While this provides a safe default for demo purposes, it may be insufficient for your organization.

Recommendations:

1. Add CloudZero Helm Repository

Refer to the helm repo documentation for command details. To use a beta version, refer to the beta installation document for the appropriate channel.

helm repo add cloudzero https://cloudzero.github.io/cloudzero-charts
helm repo update

2. Install Helm Chart

helm install <RELEASE_NAME> cloudzero/cloudzero-agent \
    --set apiKey=<CLOUDZERO_API_KEY> \
    --set clusterName=<CLUSTER_NAME> \
    --set-string cloudAccountId=<CLOUD_ACCOUNT_ID> \
    --set region=<REGION>

Advanced Install

The “Quick Start” option will cover most test and demo use cases, but may not be appropriate for a production deployment. This section provides configuration options to ensure production quality deployment.

Below is an example of a configuration file that one might use to configure some more advanced features of the chart.

# -- Account ID of the account the cluster is running in. This must be a string - even if it is a number in your system.
cloudAccountId: YOUR_CLOUD_ACCOUNT_ID
# -- Name of the clusters.
clusterName: YOUR_CLUSTER_NAME
# -- Region the cluster is running in.
region: YOUR_CLOUD_REGION
# -- CloudZero API key. Required if existingSecretName is null.
apiKey: YOUR_CLOUDZERO_API_KEY
# -- If set, the agent will use the API key in this Secret to authenticate with CloudZero. This may be preferable for users who would like to manage the CloudZero API key in a Secret external to this helm chart. See *Secret Management* below for details.
existingSecretName: YOUR_EXISTING_API_KEY_K8S_SECRET


# -- Configuration for managing the gathering of labels and annotations. See the below *Labels and Annotations* section for more details.
insightsController:
  # -- By default, a ValidatingAdmissionWebhook will be deployed that records all created labels and annotations
  enabled: true
  labels:
    # -- Determines whether the agent will gather labels from Kubernetes resources.
    enabled: true
    # -- This value MUST be set to a list of regular expressions which will be used to gather labels from pods, deployments, statefulsets, daemonsets, cronjobs, jobs, nodes, and namespaces
    patterns:
      - '^foo' # -- Match all labels whose key starts with "foo"
      - 'bar$' # -- Match all labels whose key ends with "bar"
    # -- Labels can be gathered from pods and namespaces by default. See the values.yaml for more options.
    resources:
      pods: true
      namespaces: true
      deployments: true
      statefulsets: true
      nodes: true
      jobs: true
      cronjobs: true
      daemonsets: true
  annotations:
    # -- By default, the gathering of annotations is not enabled. To enable, set this field to true
    enabled: false
    patterns:
      - '.*' # -- match all annotations. This is not recommended.
  tls:
    # -- If disabled, the insights controller will not mount a TLS certificate from a Secret, and the user is responsible for configuring a method of providing TLS information to the webhook-server container.
    enabled: true
    # -- If left as an empty string, the certificate will be generated by the chart. Otherwise, the provided value will be used.
    crt: ""
    # -- If left as an empty string, the certificate private key will be generated by the chart. Otherwise, the provided value will be used.
    key: ""
    secret:
      # -- If set to true, a Secret will be created to store the TLS certificate and key.
      create: true
      # -- If set, the Secret will be created with this name. Otherwise, a default name will be generated.
      name: ""
    # -- The following TLS certificate information is for a self signed certificate. It is used as a default value for the validating admission webhook and the webhook server.
    # -- This path determines the location within the container where the TLS certificate and key will be mounted.
    mountPath: /etc/certs
    # -- This is the caBundle used by the Validating Admission Webhook when sending requests to the webhook server. If left empty, the default self-signed certificate will be used.
    # Set this value to an empty string if using cert-manager to manage the certificate instead. Otherwise, set this to the base64 encoded caBundle of the desired certificate.
    caBundle: ""
    # -- If enabled, the certificate will be managed by cert-manager, which must already be present in the cluster.
    # If disabled, a default self-signed certificate will be used.
    useCertManager: false

Mandatory Values

There are several mandatory values that must be specified for the chart to install properly. Below are the required settings along with strategies for providing custom values during installation:

Key Type Default Description
cloudAccountId string nil Account ID in AWS or Subscription ID in Azure or Project Number in GCP where the cluster is running. Must be a string due to Helm limitations.
clusterName string nil Name of the cluster. Must be RFC 1123 compliant.
host string "api.cloudzero.com" CloudZero host to send metrics to.
apiKey string nil The CloudZero API key to use for exporting metrics. Only used if existingSecretName is not set.
existingSecretName string nil Name of the secret that contains the CloudZero API key. Required if not providing the API key via apiKey.
region string nil Region where the cluster is running (e.g., us-east-1, eastus). For more information, see AWS or Azure documentation.

It is recommended to use a values-override.yaml file for customizations. For details, refer to the official Helm documentation.

Labels and Annotations

⚠️ CloudZero supports a maximum of 300 labels for Kubernetes resources. Ensure you configure regex patterns to gather only the necessary labels/annotations. Additional labels after the first 300 are discarded.

By default, this chart exports pod and namespace labels with keys matching app.kubernetes.io/component, and no annotations. You can configure what labels and/or annotations are exported by following the steps in this section.

This chart allows the exporting of labels and annotations from the following resources:

The export of labels and annotations from a cluster can be turned on or off within the insightsController field. For example, the following enables exporting both labels and annotations from pods and namespaces:

insightsController:
  enabled: true
  labels:
    enabled: true
  annotations:
    enabled: true

It is recommended to supply a list of regexes to filter only the labels/annotations required:

insightsController:
  enabled: true
  labels:
    enabled: true
    patterns:
      - '^foo' # -- Match all labels whose key starts with "foo"
      - 'bar$' # -- Match all labels whose key ends with "bar"

Labels/annotations can also be gathered from more than just pods and namespaces. An example of gathering labels from all available resources would be:

insightsController:
  enabled: true
  labels:
    enabled: true
    resources:
      pods: true
      namespaces: true
      deployments: true
      statefulsets: true
      nodes: true
      jobs: true
      cronjobs: true
      daemonsets: true

Additional Notes:

Secret Management

The chart requires a CloudZero API key to send metric data. Admins can retrieve API keys here.

The API key is typically stored in a Secret in the cluster. The cloudzero-agent chart will create a Secret if the API key is provided via the apiKey argument. Alternatively, the API key can be stored in a Secret external to the chart; this Secret name would then be set as the existingSecretName argument. If creating a Secret external to the chart, ensure the Secret is in the same namespace as the chart and follows this format:

Example User-Created Secret Content

data:
  value: <API_KEY>

Example of creating a secret:

kubectl create secret -n example-namespace generic example-secret-name --from-literal=value=<example-api-key-value>

The secret can then be used with existingSecretName.

Memory Sizing

Please see the sizing guide in the docs directory.

Update Helm Chart

If you are updating an existing installation, pull the latest chart information:

helm repo update

Next, upgrade the installation to the latest chart version:

helm upgrade --install <RELEASE_NAME> cloudzero/cloudzero-agent -f configuration.example.yaml

Getting All Image References

A common customization may be to mirror the container images used in this chart into a private registry. To fetch all image references used in this chart, use the following commands:

CHART_VERSION=1.0.0        # Set this to the chart version for which container image references should be fetched
CHART_REPO=cloudzero       # Set this to the name of cloudzero helm repository.
helm template $CHART_REPO/cloudzero-agent --version $CHART_VERSION --set clusterName=foobar --set cloudAccountId=foobar --set region=foobar | grep -i image: | tr -d '"' | sort | uniq | awk '{print $NF}'

This will return the latest image references for that particular chart version.

Passing Values to Subcharts

Values can be passed to subcharts like kube-state-metrics by adding entries in values-override.yaml as per their specifications.

A common addition may be to pull the container images from custom image registries/repositories:

values-override.yaml

kubeStateMetrics:
  enabled: true
  image:
    registry: my-custom-registry.io
    repository: my-custom-kube-state-metrics/kube-state-metrics

Dependencies

Repository Name Version
https://prometheus-community.github.io/helm-charts kube-state-metrics 5.15.*

Note that while kube-state-metrics is listed as a dependency, it is referred to as cloudzero-state-metrics within the helm chart. This is to enforce the idea that this KSM deployment is used exclusively by the cloudzero-agent.

Enabling Release Notifications

To receive a notification when a new version of the chart is released, you can watch the repository:

  1. Navigate to the repository main page.
  2. Select Watch > Custom.
  3. Check the Releases box.
  4. Select Apply.

Useful References

This site is open source. Improve this page.