Skip to content

ServiceIntegrationEndpoint

Prerequisites

  • A Kubernetes cluster with the operator installed using helm, kubectl or kind (for local development).
  • A Kubernetes Secret with an Aiven authentication token.

Required permissions

To create and manage this resource, you must have the appropriate roles or permissions. See the Aiven documentation for details on managing permissions.

This resource uses the following API operations, and for each operation, any of the listed permissions is sufficient:

Operation Permissions
ServiceIntegrationEndpointCreate project:integrations:write
ServiceIntegrationEndpointDelete project:integrations:write
ServiceIntegrationEndpointUpdate project:integrations:write

Usage examples

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
apiVersion: aiven.io/v1alpha1
kind: ServiceIntegrationEndpoint
metadata:
  name: my-service-integration-endpoint
spec:
  authSecretRef:
    name: aiven-token
    key: token

  project: aiven-project-name
  endpointName: my-autoscaler
  endpointType: autoscaler

  autoscaler:
    autoscaling:
      - type: autoscale_disk
        cap_gb: 100

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
apiVersion: aiven.io/v1alpha1
kind: ServiceIntegrationEndpoint
metadata:
  name: my-service-integration-endpoint
spec:
  authSecretRef:
    name: aiven-token
    key: token

  project: aiven-project-name
  endpointName: my-external-postgresql
  endpointType: external_postgresql

  externalPostgresql:
    username: username
    password: password
    host: example.example
    port: 5432
    ssl_mode: require

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
apiVersion: aiven.io/v1alpha1
kind: ServiceIntegrationEndpoint
metadata:
  name: my-service-integration-endpoint
spec:
  authSecretRef:
    name: aiven-token
    key: token

  project: aiven-project-name
  endpointName: my-external-schema-registry
  endpointType: external_schema_registry

  externalSchemaRegistry:
    url: https://schema-registry.example.com:8081
    authentication: basic
    basic_auth_username: username
    basic_auth_password: password

Apply the resource with:

kubectl apply -f example.yaml

Verify the newly created ServiceIntegrationEndpoint:

kubectl get serviceintegrationendpoints my-service-integration-endpoint

The output is similar to the following: 

Name                               Project               Endpoint Name    Endpoint Type    ID      
my-service-integration-endpoint    aiven-project-name    my-autoscaler    autoscaler       <id>

ServiceIntegrationEndpoint

ServiceIntegrationEndpoint is the Schema for the serviceintegrationendpoints API.

Required

  • apiVersion (string). Value aiven.io/v1alpha1.
  • kind (string). Value ServiceIntegrationEndpoint.
  • metadata (object). Data that identifies the object, including a name string and optional namespace.
  • spec (object). ServiceIntegrationEndpointSpec defines the desired state of ServiceIntegrationEndpoint. See below for nested schema.

spec

Appears on ServiceIntegrationEndpoint.

ServiceIntegrationEndpointSpec defines the desired state of ServiceIntegrationEndpoint.

Required

  • endpointType (string, Enum: autoscaler, datadog, external_aws_cloudwatch_logs, external_aws_cloudwatch_metrics, external_aws_s3, external_clickhouse, external_elasticsearch_logs, external_google_cloud_bigquery, external_google_cloud_logging, external_kafka, external_mysql, external_opensearch_logs, external_postgresql, external_redis, external_schema_registry, external_sumologic_logs, jolokia, prometheus, rsyslog, Immutable). Type of the service integration endpoint.
  • project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.

Optional

authSecretRef

Appears on spec.

Authentication reference to Aiven token in a secret.

Required

  • key (string, MinLength: 1).
  • name (string, MinLength: 1).

autoscaler

Appears on spec.

Autoscaler configuration values.

Required

  • autoscaling (array of objects, MaxItems: 64). Configure autoscaling thresholds for a service. See below for nested schema.

autoscaling

Appears on spec.autoscaler.

Autoscaling properties for a service.

Required

  • cap_gb (integer, Minimum: 50, Maximum: 9007199254740991). The maximum total disk size (in gb) to allow autoscaler to scale up to.
  • type (string, Enum: autoscale_disk). Type of autoscale event.

datadog

Appears on spec.

Datadog configuration values.

Required

  • datadog_api_key (string, Pattern: ^[A-Za-z0-9]{1,256}$, MinLength: 1, MaxLength: 256). Datadog API key.

Optional

  • datadog_tags (array of objects, MaxItems: 32). Custom tags provided by user. See below for nested schema.
  • disable_consumer_stats (boolean). Disable consumer group metrics.
  • extra_tags_prefix (string, Pattern: ^[A-Za-z0-9\-]{0,64}$, MinLength: 0, MaxLength: 64). Extra tags prefix. Defaults to aiven.
  • kafka_consumer_check_instances (integer, Minimum: 1, Maximum: 100). Number of separate instances to fetch kafka consumer statistics with.
  • kafka_consumer_stats_timeout (integer, Minimum: 2, Maximum: 300). Number of seconds that datadog will wait to get consumer statistics from brokers.
  • max_partition_contexts (integer, Minimum: 200, Maximum: 200000). Maximum number of partition contexts to send.
  • site (string, Enum: ap1.datadoghq.com, datadoghq.com, datadoghq.eu, ddog-gov.com, us3.datadoghq.com, us5.datadoghq.com). Datadog intake site. Defaults to datadoghq.com.

datadog_tags

Appears on spec.datadog.

Datadog tag defined by user.

Required

Optional

  • comment (string, MaxLength: 1024). Optional tag explanation.

externalAWSCloudwatchLogs

Appears on spec.

ExternalAwsCloudwatchLogs configuration values.

Required

  • access_key (string, MaxLength: 4096). AWS access key. Required permissions are logs:CreateLogGroup, logs:CreateLogStream, logs:PutLogEvents and logs:DescribeLogStreams.
  • region (string, MaxLength: 32). AWS region.
  • secret_key (string, MaxLength: 4096). AWS secret key.

Optional

  • log_group_name (string, Pattern: ^[\.\-_/#A-Za-z0-9]+$, MinLength: 1, MaxLength: 512). AWS CloudWatch log group name.

externalAWSCloudwatchMetrics

Appears on spec.

ExternalAwsCloudwatchMetrics configuration values.

Required

  • access_key (string, MaxLength: 4096). AWS access key. Required permissions are cloudwatch:PutMetricData.
  • namespace (string, MinLength: 1, MaxLength: 255). AWS CloudWatch Metrics Namespace.
  • region (string, MaxLength: 32). AWS region.
  • secret_key (string, MaxLength: 4096). AWS secret key.

externalElasticsearchLogs

Appears on spec.

ExternalElasticsearchLogs configuration values.

Required

  • index_prefix (string, Pattern: ^[a-z0-9][a-z0-9-_.]+$, MinLength: 1, MaxLength: 1000). Elasticsearch index prefix.
  • url (string, MinLength: 12, MaxLength: 2048). Elasticsearch connection URL.

Optional

  • ca (string, MaxLength: 16384). PEM encoded CA certificate.
  • index_days_max (integer, Minimum: 1, Maximum: 10000). Maximum number of days of logs to keep.
  • timeout (number, Minimum: 10, Maximum: 120). Elasticsearch request timeout limit.

externalGoogleCloudBigquery

Appears on spec.

ExternalGoogleCloudBigquery configuration values.

Required

externalGoogleCloudLogging

Appears on spec.

ExternalGoogleCloudLogging configuration values.

Required

externalKafka

Appears on spec.

ExternalKafka configuration values.

Required

  • bootstrap_servers (string, MinLength: 3, MaxLength: 256). Bootstrap servers.
  • security_protocol (string, Enum: PLAINTEXT, SASL_PLAINTEXT, SASL_SSL, SSL). Security protocol.

Optional

  • sasl_mechanism (string, Enum: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512). SASL mechanism used for connections to the Kafka server.
  • sasl_plain_password (string, MinLength: 1, MaxLength: 256). Password for SASL PLAIN mechanism in the Kafka server.
  • sasl_plain_username (string, MinLength: 1, MaxLength: 256). Username for SASL PLAIN mechanism in the Kafka server.
  • ssl_ca_cert (string, MaxLength: 16384). PEM-encoded CA certificate.
  • ssl_client_cert (string, MaxLength: 16384). PEM-encoded client certificate.
  • ssl_client_key (string, MaxLength: 16384). PEM-encoded client key.
  • ssl_endpoint_identification_algorithm (string, Enum: https). The endpoint identification algorithm to validate server hostname using server certificate.

externalOpensearchLogs

Appears on spec.

ExternalOpensearchLogs configuration values.

Required

  • index_prefix (string, Pattern: ^[a-z0-9][a-z0-9-_.]+$, MinLength: 1, MaxLength: 1000). OpenSearch index prefix.
  • url (string, MinLength: 12, MaxLength: 2048). OpenSearch connection URL.

Optional

  • ca (string, MaxLength: 16384). PEM encoded CA certificate.
  • index_days_max (integer, Minimum: 1, Maximum: 10000). Maximum number of days of logs to keep.
  • timeout (number, Minimum: 10, Maximum: 120). OpenSearch request timeout limit.

externalPostgresql

Appears on spec.

ExternalPostgresql configuration values.

Required

  • host (string, MaxLength: 255). Hostname or IP address of the server.
  • port (integer, Minimum: 1, Maximum: 65535). Port number of the server.
  • username (string, MaxLength: 256). User name.

Optional

  • default_database (string, Pattern: ^[_A-Za-z0-9][-_A-Za-z0-9]{0,62}$, MaxLength: 63). Default database.
  • password (string, MaxLength: 256). Password.
  • ssl_client_certificate (string, MaxLength: 16384). Client certificate.
  • ssl_client_key (string, MaxLength: 16384). Client key.
  • ssl_mode (string, Enum: prefer, require, verify-ca, verify-full). SSL mode to use for the connection. Please note that Aiven requires TLS for all connections to external PostgreSQL services. Deprecated values: prefer.
  • ssl_root_cert (string, MaxLength: 16384). SSL Root Cert.

externalSchemaRegistry

Appears on spec.

ExternalSchemaRegistry configuration values.

Required

  • authentication (string, Enum: basic, none). Authentication method.
  • url (string, MaxLength: 2048). Schema Registry URL.

Optional

jolokia

Appears on spec.

Jolokia configuration values.

Optional

  • basic_auth_password (string, MinLength: 8, MaxLength: 64). Jolokia basic authentication password.
  • basic_auth_username (string, Pattern: ^[a-z0-9\-@_]{5,32}$, MinLength: 5, MaxLength: 32). Jolokia basic authentication username.

prometheus

Appears on spec.

Prometheus configuration values.

Optional

  • basic_auth_password (string, MinLength: 8, MaxLength: 256). Prometheus basic authentication password.
  • basic_auth_username (string, Pattern: ^[a-z0-9\-@_]{5,32}$, MinLength: 5, MaxLength: 32). Prometheus basic authentication username.

rsyslog

Appears on spec.

Rsyslog configuration values.

Required

  • format (string, Enum: custom, rfc3164, rfc5424). Message format.
  • port (integer, Minimum: 1, Maximum: 65535). Rsyslog server port.
  • server (string, MinLength: 4, MaxLength: 255). Rsyslog server IP address or hostname.
  • tls (boolean). Require TLS.

Optional

  • ca (string, MaxLength: 16384). PEM encoded CA certificate.
  • cert (string, MaxLength: 16384). PEM encoded client certificate.
  • key (string, MaxLength: 16384). PEM encoded client key.
  • logline (string, Pattern: ^[ -~\t]+$, MinLength: 1, MaxLength: 512). Custom syslog message format.
  • max_message_size (integer, Minimum: 2048, Maximum: 2147483647). Rsyslog max message size.
  • sd (string, MaxLength: 1024). Structured data block for log message.