KafkaSchema

Prerequisites¶

• A Kubernetes cluster with the operator installed using helm, kubectl or kind (for local development).
• A Kubernetes Secret with an Aiven authentication token.

Required permissions¶

To create and manage this resource, you must have the appropriate roles or permissions. See the Aiven documentation for details on managing permissions.

This resource uses the following API operations, and for each operation, any of the listed permissions is sufficient:

Usage examples¶

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31

apiVersion: aiven.io/v1alpha1
kind: KafkaSchema
metadata:
  name: order-event-pinned
spec:
  authSecretRef:
    name: aiven-token
    key: token

  project: my-aiven-project
  serviceName: my-kafka
  subjectName: com.example.OrderEvent
  schemaType: PROTOBUF
  schema: |
    syntax = "proto3";
    import "common/money.proto";
    message OrderEvent {
      common.Money total = 1;
    }
  references:
    # Pin the referenced subject and version explicitly. Use this when the
    # referenced schema is managed outside this operator, or when you want
    # the dependent to keep pointing at a specific version regardless of
    # what the referent advances to.
    #
    # If the referent IS another KafkaSchema in the same namespace, prefer
    # the kafkaSchemaRef variant — see the with_refs example. That form
    # auto-propagates new versions instead of requiring manual updates here.
    - name: common/money.proto
      subject: common.Money
      version: 1

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46

apiVersion: aiven.io/v1alpha1
kind: KafkaSchema
metadata:
  name: money-type
spec:
  authSecretRef:
    name: aiven-token
    key: token

  project: my-aiven-project
  serviceName: my-kafka
  subjectName: common.Money
  schemaType: PROTOBUF
  schema: |
    syntax = "proto3";
    package common;
    message Money {
      string currency = 1;
      int64 amount_cents = 2;
    }
---
apiVersion: aiven.io/v1alpha1
kind: KafkaSchema
metadata:
  name: order-event
spec:
  authSecretRef:
    name: aiven-token
    key: token

  project: my-aiven-project
  serviceName: my-kafka
  subjectName: com.example.OrderEvent
  schemaType: PROTOBUF
  schema: |
    syntax = "proto3";
    import "common/money.proto";
    message OrderEvent {
      common.Money total = 1;
    }
  references:
    # Resolve subject and version from another KafkaSchema in the same namespace.
    # No need to hard-code the version - dependents pick up new versions automatically.
    - name: common/money.proto
      kafkaSchemaRef:
        name: money-type

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

apiVersion: aiven.io/v1alpha1
kind: KafkaSchema
metadata:
  name: my-schema
spec:
  authSecretRef:
    name: aiven-token
    key: token

  project: my-aiven-project
  serviceName: my-kafka
  subjectName: mny-subject
  compatibilityLevel: BACKWARD
  schema: |
    {
        "doc": "example_doc",
        "fields": [{
            "default": 5,
            "doc": "field_doc",
            "name": "field_name",
            "namespace": "field_namespace",
            "type": "int"
        }],
        "name": "example_name",
        "namespace": "example_namespace",
        "type": "record"
    }

Apply the resource with:

kubectl apply -f example.yaml

Verify the newly created KafkaSchema:

kubectl get kafkaschemas order-event-pinned

The output is similar to the following:

Name                  Service Name    Project             Subject                   Version      
order-event-pinned    my-kafka        my-aiven-project    com.example.OrderEvent    <version>    

KafkaSchema¶

KafkaSchema is the Schema for the kafkaschemas API.

Self-references (A -> A) are blocked at admission; transitive cycles (A -> B -> A) are not detected at admission time.

Deletion: the operator performs a soft delete followed by a hard delete on the subject. The subject disappears from the registry's listing, re-applying a KafkaSchema with the same subjectName after deletion starts a brand-new subject at version 1.

Required

• apiVersion (string). Value aiven.io/v1alpha1.
• kind (string). Value KafkaSchema.
• metadata (object). Data that identifies the object, including a name string and optional namespace.
• spec (object). KafkaSchemaSpec defines the desired state of KafkaSchema. See below for nested schema.

spec¶

Appears on KafkaSchema.

KafkaSchemaSpec defines the desired state of KafkaSchema.

Required

• project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.
• schema (string). Kafka Schema definition. Format depends on schemaType (AVRO/JSON/PROTOBUF).
• serviceName (string, Immutable, Pattern: ^[a-z][-a-z0-9]+$, MaxLength: 63). Specifies the name of the service that this resource belongs to.
• subjectName (string, Immutable). Kafka Schema Subject name.

Optional

• authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.
• compatibilityLevel (string, Enum: BACKWARD, BACKWARD_TRANSITIVE, FORWARD, FORWARD_TRANSITIVE, FULL, FULL_TRANSITIVE, NONE). Kafka Schemas compatibility level.
• references (array of objects, MaxItems: 100). Schema references for Protobuf or JSON schemas that import other schemas. References must form a directed acyclic graph (DAG); cycles are not allowed. See below for nested schema.
• schemaType (string, Enum: AVRO, JSON, PROTOBUF, Immutable). Schema type.

authSecretRef¶

Appears on spec.

Authentication reference to Aiven token in a secret.

Required

• key (string, MinLength: 1).
• name (string, MinLength: 1).

references¶

Appears on spec.

SchemaReference is a reference to another schema in the registry. Exactly one of {subject+version} or kafkaSchemaRef must be set.

Required

• name (string, MinLength: 1, MaxLength: 512). Name used to reference the schema (e.g., the import path in Protobuf).

Optional

• kafkaSchemaRef (object). Reference to another KafkaSchema resource in the same namespace. Mutually exclusive with subject/version.

  Cleanup order matters: delete the dependent before the referent. See below for nested schema. - subject (string, MinLength: 1, MaxLength: 512). Subject name of the referenced schema in the registry. Mutually exclusive with kafkaSchemaRef. - version (integer, Minimum: 1). Version of the referenced schema. Mutually exclusive with kafkaSchemaRef.

kafkaSchemaRef¶

Appears on spec.references.

Reference to another KafkaSchema resource in the same namespace. Mutually exclusive with subject/version.

Cleanup order matters: delete the dependent before the referent.

Required

• name (string, MinLength: 1, MaxLength: 253). Name of the KafkaSchema resource in the same namespace.