Flink

Prerequisites¶

• A Kubernetes cluster with the operator installed using helm, kubectl or kind (for local development).
• A Kubernetes Secret with an Aiven authentication token.

Required permissions¶

To create and manage this resource, you must have the appropriate roles or permissions. See the Aiven documentation for details on managing permissions. For more precise access control, use permissions instead of roles.

Permissions

All of these permissions are required to create and manage this resource.

• organization:projects:write
• project:services:read
• project:services:write
• service:configuration:write
• service:secrets:read

Usage example¶

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29

apiVersion: aiven.io/v1alpha1
kind: Flink
metadata:
  name: my-flink
spec:
  authSecretRef:
    name: aiven-token
    key: token

  connInfoSecretTarget:
    name: flink-secret
    annotations:
      foo: bar
    labels:
      baz: egg

  project: my-aiven-project
  cloudName: google-europe-west1
  plan: business-4

  maintenanceWindowDow: sunday
  maintenanceWindowTime: 11:00:00

  userConfig:
    number_of_task_slots: 10
    ip_filter:
      - network: 0.0.0.0/32
        description: whatever
      - network: 10.20.0.0/16

Apply the resource with:

kubectl apply -f example.yaml

Verify the newly created Flink:

kubectl get flinks my-flink

The output is similar to the following:

Name        Project             Region                 Plan          State      
my-flink    my-aiven-project    google-europe-west1    business-4    RUNNING    

To view the details of the Secret, use the following command:

kubectl describe secret flink-secret

You can use the jq to quickly decode the Secret:

kubectl get secret flink-secret -o json | jq '.data | map_values(@base64d)'

The output is similar to the following:

{
    "FLINK_HOST": "<secret>",
    "FLINK_PORT": "<secret>",
    "FLINK_USER": "<secret>",
    "FLINK_PASSWORD": "<secret>",
    "FLINK_URI": "<secret>",
    "FLINK_HOSTS": "<secret>",
}

Flink¶

Flink is the Schema for the flinks API.

Required

• apiVersion (string). Value aiven.io/v1alpha1.
• kind (string). Value Flink.
• metadata (object). Data that identifies the object, including a name string and optional namespace.
• spec (object). FlinkSpec defines the desired state of Flink. See below for nested schema.

spec¶

Appears on Flink.

FlinkSpec defines the desired state of Flink.

Required

• plan (string, MaxLength: 128). Subscription plan.
• project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.

Optional

• authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.
• cloudName (string, MaxLength: 256). Cloud the service runs in.
• connInfoSecretTarget (object). Secret configuration. See below for nested schema.
• connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.
• disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.
• maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.
• maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.
• powered (boolean, Default value: true). Determines the power state of the service. When true (default), the service is running. When false, the service is powered off. For more information please see Aiven documentation. Note that:
  • When set to false the annotation controllers.aiven.io/instance-is-running is also set to false.
  • Services cannot be created in a powered off state. The value is ignored during creation.
  • It is highly recommended to not run dependent resources when the service is powered off. Creating a new resource or updating an existing resource that depends on a powered off service will result in an error. Existing resources will need to be manually recreated after the service is powered on.
  • Existing secrets will not be updated or removed when the service is powered off.
  • For Kafka services with backups: Topic configuration, schemas and connectors are all backed up, but not the data in topics. All topic data is lost on power off.
  • For Kafka services without backups: Topic configurations including all topic data is lost on power off.
• projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.
• projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.
• serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.
• tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.
• technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.
• terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.
• userConfig (object). Cassandra specific user configuration options. See below for nested schema.

authSecretRef¶

Appears on spec.

Authentication reference to Aiven token in a secret.

Required

• key (string, MinLength: 1).
• name (string, MinLength: 1).

connInfoSecretTarget¶

Appears on spec.

Secret configuration.

Required

• name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.

Optional

• annotations (object, AdditionalProperties: string). Annotations added to the secret.
• labels (object, AdditionalProperties: string). Labels added to the secret.
• prefix (string). Prefix for the secret's keys. Added "as is" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.

projectVPCRef¶

Appears on spec.

ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.

Required

• name (string, MinLength: 1).

Optional

• namespace (string, MinLength: 1).

serviceIntegrations¶

Appears on spec.

Service integrations to specify when creating a service. Not applied after initial service creation.

Required

• integrationType (string, Enum: read_replica).
• sourceServiceName (string, MinLength: 1, MaxLength: 64).

technicalEmails¶

Appears on spec.

Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.

Required

• email (string). Email address.

userConfig¶

Appears on spec.

Cassandra specific user configuration options.

Optional

• additional_backup_regions (array of strings, MaxItems: 1). Deprecated. Additional Cloud Regions for Backup Replication.
• custom_code (boolean, Immutable). Enable to upload Custom JARs for Flink applications.
• flink_version (string, Immutable). Available versions: 1.16, 1.19, 1.20. Newer versions may also be available. Flink major version. Deprecated values: 1.16.
• ip_filter (array of objects, MaxItems: 8000). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.
• number_of_task_slots (integer, Minimum: 1, Maximum: 1024). Task slots per node. For a 3 node plan, total number of task slots is 3x this value.
• pekko_ask_timeout_s (integer, Minimum: 5, Maximum: 60). Timeout in seconds used for all futures and blocking Pekko requests.
• pekko_framesize_b (integer, Minimum: 1048576, Maximum: 52428800). Maximum size in bytes for messages exchanged between the JobManager and the TaskManagers.
• privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.
• public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.
• service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.
• static_ips (boolean). Use static public IP addresses.

ip_filter¶

Appears on spec.userConfig.

CIDR address block, either as a string, or in a dict with an optional description field.

Required

• network (string, MaxLength: 43). CIDR address block.

Optional

• description (string, MaxLength: 1024). Description for IP filter list entry.

privatelink_access¶

Appears on spec.userConfig.

Allow access to selected service components through Privatelink.

Optional

• flink (boolean). Enable flink.
• prometheus (boolean). Enable prometheus.

public_access¶

Appears on spec.userConfig.

Allow access to selected service ports from the public Internet.

Required

• flink (boolean). Allow clients to connect to flink from the public internet for service nodes that are in a project VPC or another type of private network.