OpenSearch

Usage example¶

Example

apiVersion: aiven.io/v1alpha1
kind: OpenSearch
metadata:
  name: my-os
spec:
  authSecretRef:
    name: aiven-token
    key: token

  connInfoSecretTarget:
    name: os-secret
    prefix: MY_SECRET_PREFIX_
    annotations:
      foo: bar
    labels:
      baz: egg

  project: my-aiven-project
  cloudName: google-europe-west1
  plan: startup-4
  disk_space: 80GiB

  maintenanceWindowDow: friday
  maintenanceWindowTime: 23:00:00

Info

To create this resource, a Secret containing Aiven token must be created first.

Apply the resource with:

kubectl apply -f example.yaml

Verify the newly created OpenSearch:

kubectl get opensearches my-os

The output is similar to the following:

Name     Project             Region                 Plan         State      
my-os    my-aiven-project    google-europe-west1    startup-4    RUNNING

To view the details of the Secret, use the following command:

kubectl describe secret os-secret

You can use the jq to quickly decode the Secret:

kubectl get secret os-secret -o json | jq '.data | map_values(@base64d)'

The output is similar to the following:

{
    "OPENSEARCH_HOST": "<secret>",
    "OPENSEARCH_PORT": "<secret>",
    "OPENSEARCH_USER": "<secret>",
    "OPENSEARCH_PASSWORD": "<secret>",
}

OpenSearch¶

OpenSearch is the Schema for the opensearches API.

Exposes secret keys

OPENSEARCH_HOST, OPENSEARCH_PORT, OPENSEARCH_USER, OPENSEARCH_PASSWORD.

Required

apiVersion (string). Value aiven.io/v1alpha1.
kind (string). Value OpenSearch.
metadata (object). Data that identifies the object, including a name string and optional namespace.
spec (object). OpenSearchSpec defines the desired state of OpenSearch. See below for nested schema.

spec¶

Appears on OpenSearch.

OpenSearchSpec defines the desired state of OpenSearch.

Required

plan (string, MaxLength: 128). Subscription plan.
project (string, Immutable, Pattern: ^[a-zA-Z0-9_-]+$, MaxLength: 63). Identifies the project this resource belongs to.

Optional

authSecretRef (object). Authentication reference to Aiven token in a secret. See below for nested schema.
cloudName (string, MaxLength: 256). Cloud the service runs in.
connInfoSecretTarget (object). Secret configuration. See below for nested schema.
connInfoSecretTargetDisabled (boolean, Immutable). When true, the secret containing connection information will not be created, defaults to false. This field cannot be changed after resource creation.
disk_space (string, Pattern: (?i)^[1-9][0-9]*(GiB|G)?$). The disk space of the service, possible values depend on the service type, the cloud provider and the project. Reducing will result in the service re-balancing. The removal of this field does not change the value.
maintenanceWindowDow (string, Enum: monday, tuesday, wednesday, thursday, friday, saturday, sunday). Day of week when maintenance operations should be performed. One monday, tuesday, wednesday, etc.
maintenanceWindowTime (string, MaxLength: 8). Time of day when maintenance operations should be performed. UTC time in HH:mm:ss format.
projectVPCRef (object). ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically. See below for nested schema.
projectVpcId (string, MaxLength: 36). Identifier of the VPC the service should be in, if any.
serviceIntegrations (array of objects, Immutable, MaxItems: 1). Service integrations to specify when creating a service. Not applied after initial service creation. See below for nested schema.
tags (object, AdditionalProperties: string). Tags are key-value pairs that allow you to categorize services.
technicalEmails (array of objects, MaxItems: 10). Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability. See below for nested schema.
terminationProtection (boolean). Prevent service from being deleted. It is recommended to have this enabled for all services.
userConfig (object). OpenSearch specific user configuration options. See below for nested schema.

authSecretRef¶

Appears on spec.

Authentication reference to Aiven token in a secret.

Required

key (string, MinLength: 1).
name (string, MinLength: 1).

connInfoSecretTarget¶

Appears on spec.

Secret configuration.

Required

name (string, Immutable). Name of the secret resource to be created. By default, it is equal to the resource name.

Optional

annotations (object, AdditionalProperties: string). Annotations added to the secret.
labels (object, AdditionalProperties: string). Labels added to the secret.
prefix (string). Prefix for the secret's keys. Added "as is" without any transformations. By default, is equal to the kind name in uppercase + underscore, e.g. KAFKA_, REDIS_, etc.

projectVPCRef¶

Appears on spec.

ProjectVPCRef reference to ProjectVPC resource to use its ID as ProjectVPCID automatically.

Required

name (string, MinLength: 1).

Optional

namespace (string, MinLength: 1).

serviceIntegrations¶

Appears on spec.

Service integrations to specify when creating a service. Not applied after initial service creation.

Required

integrationType (string, Enum: read_replica).
sourceServiceName (string, MinLength: 1, MaxLength: 64).

technicalEmails¶

Appears on spec.

Defines the email addresses that will receive alerts about upcoming maintenance updates or warnings about service instability.

Required

email (string). Email address.

userConfig¶

Appears on spec.

OpenSearch specific user configuration options.

Optional

additional_backup_regions (array of strings, MaxItems: 1). Additional Cloud Regions for Backup Replication.
azure_migration (object). Azure migration settings. See below for nested schema.
custom_domain (string, MaxLength: 255). Serve the web frontend using a custom CNAME pointing to the Aiven DNS name.
disable_replication_factor_adjustment (boolean). Disable automatic replication factor adjustment for multi-node services. By default, Aiven ensures all indexes are replicated at least to two nodes. Note: Due to potential data loss in case of losing a service node, this setting can not be activated unless specifically allowed for the project.
gcs_migration (object). Google Cloud Storage migration settings. See below for nested schema.
index_patterns (array of objects, MaxItems: 512). Index patterns. See below for nested schema.
index_rollup (object). Index rollup settings. See below for nested schema.
index_template (object). Template settings for all new indexes. See below for nested schema.
ip_filter (array of objects, MaxItems: 1024). Allow incoming connections from CIDR address block, e.g. 10.20.0.0/16. See below for nested schema.
keep_index_refresh_interval (boolean). Aiven automation resets index.refresh_interval to default value for every index to be sure that indices are always visible to search. If it doesn't fit your case, you can disable this by setting up this flag to true.
max_index_count (integer, Minimum: 0). DEPRECATED: use index_patterns instead.
openid (object). OpenSearch OpenID Connect Configuration. See below for nested schema.
opensearch (object). OpenSearch settings. See below for nested schema.
opensearch_dashboards (object). OpenSearch Dashboards settings. See below for nested schema.
opensearch_version (string, Enum: 1, 2). OpenSearch major version.
private_access (object). Allow access to selected service ports from private networks. See below for nested schema.
privatelink_access (object). Allow access to selected service components through Privatelink. See below for nested schema.
project_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 63). Name of another project to fork a service from. This has effect only when a new service is being created.
public_access (object). Allow access to selected service ports from the public Internet. See below for nested schema.
recovery_basebackup_name (string, Pattern: ^[a-zA-Z0-9-_:.]+$, MaxLength: 128). Name of the basebackup to restore in forked service.
s3_migration (object). AWS S3 / AWS S3 compatible migration settings. See below for nested schema.
saml (object). OpenSearch SAML configuration. See below for nested schema.
service_log (boolean). Store logs for the service so that they are available in the HTTP API and console.
service_to_fork_from (string, Immutable, Pattern: ^[a-z][-a-z0-9]{0,63}$|^$, MaxLength: 64). Name of another service to fork from. This has effect only when a new service is being created.
static_ips (boolean). Use static public IP addresses.

azure_migration¶

Appears on spec.userConfig.

Azure migration settings.

Required

account (string, Pattern: ^[^\r\n]*$). Account name.
base_path (string, Pattern: ^[^\r\n]*$). The path to the repository data within its container. The value of this setting should not start or end with a /.
container (string, Pattern: ^[^\r\n]*$). Azure container name.
indices (string). A comma-delimited list of indices to restore from the snapshot. Multi-index syntax is supported.
snapshot_name (string, Pattern: ^[^\r\n]*$). The snapshot name to restore from.

Optional

chunk_size (string, Pattern: ^[^\r\n]*$). Big files can be broken down into chunks during snapshotting if needed. Should be the same as for the 3rd party repository.
compress (boolean). when set to true metadata files are stored in compressed format.
endpoint_suffix (string, Pattern: ^[^\r\n]*$). Defines the DNS suffix for Azure Storage endpoints.
include_aliases (boolean). Whether to restore aliases alongside their associated indexes. Default is true.
key (string, Pattern: ^[^\r\n]*$). Azure account secret key. One of key or sas_token should be specified.
restore_global_state (boolean). If true, restore the cluster state. Defaults to false.
sas_token (string, Pattern: ^[^\r\n]*$). A shared access signatures (SAS) token. One of key or sas_token should be specified.

gcs_migration¶

Appears on spec.userConfig.

Google Cloud Storage migration settings.

Required

base_path (string, Pattern: ^[^\r\n]*$). The path to the repository data within its container. The value of this setting should not start or end with a /.
bucket (string, Pattern: ^[^\r\n]*$). The path to the repository data within its container.
credentials (string, Pattern: ^[^\r\n]*$). Google Cloud Storage credentials file content.
indices (string). A comma-delimited list of indices to restore from the snapshot. Multi-index syntax is supported.
snapshot_name (string, Pattern: ^[^\r\n]*$). The snapshot name to restore from.

Optional

chunk_size (string, Pattern: ^[^\r\n]*$). Big files can be broken down into chunks during snapshotting if needed. Should be the same as for the 3rd party repository.
compress (boolean). when set to true metadata files are stored in compressed format.
include_aliases (boolean). Whether to restore aliases alongside their associated indexes. Default is true.
restore_global_state (boolean). If true, restore the cluster state. Defaults to false.

index_patterns¶

Appears on spec.userConfig.

Allows you to create glob style patterns and set a max number of indexes matching this pattern you want to keep. Creating indexes exceeding this value will cause the oldest one to get deleted. You could for example create a pattern looking like logs.? and then create index logs.1, logs.2 etc, it will delete logs.1 once you create logs.6. Do note logs.? does not apply to logs.10. Note: Setting max_index_count to 0 will do nothing and the pattern gets ignored.

Required

max_index_count (integer, Minimum: 0). Maximum number of indexes to keep.
pattern (string, Pattern: ^[A-Za-z0-9-_.*?]+$, MaxLength: 1024). fnmatch pattern.

Optional

sorting_algorithm (string, Enum: alphabetical, creation_date). Deletion sorting algorithm.

index_rollup¶

Appears on spec.userConfig.

Index rollup settings.

Optional

rollup_dashboards_enabled (boolean). Whether rollups are enabled in OpenSearch Dashboards. Defaults to true.
rollup_enabled (boolean). Whether the rollup plugin is enabled. Defaults to true.
rollup_search_backoff_count (integer, Minimum: 1). How many retries the plugin should attempt for failed rollup jobs. Defaults to 5.
rollup_search_backoff_millis (integer, Minimum: 1). The backoff time between retries for failed rollup jobs. Defaults to 1000ms.
rollup_search_search_all_jobs (boolean). Whether OpenSearch should return all jobs that match all specified search terms. If disabled, OpenSearch returns just one, as opposed to all, of the jobs that matches the search terms. Defaults to false.

index_template¶

Appears on spec.userConfig.

Template settings for all new indexes.

Optional

mapping_nested_objects_limit (integer, Minimum: 0, Maximum: 100000). The maximum number of nested JSON objects that a single document can contain across all nested types. This limit helps to prevent out of memory errors when a document contains too many nested objects. Default is 10000.
number_of_replicas (integer, Minimum: 0, Maximum: 29). The number of replicas each primary shard has.
number_of_shards (integer, Minimum: 1, Maximum: 1024). The number of primary shards that an index should have.

ip_filter¶

Appears on spec.userConfig.

CIDR address block, either as a string, or in a dict with an optional description field.

Required

network (string, MaxLength: 43). CIDR address block.

Optional

description (string, MaxLength: 1024). Description for IP filter list entry.

openid¶

Appears on spec.userConfig.

OpenSearch OpenID Connect Configuration.

Required

client_id (string, Pattern: ^[^\r\n]*$, MinLength: 1, MaxLength: 1024). The ID of the OpenID Connect client configured in your IdP. Required.
client_secret (string, Pattern: ^[^\r\n]*$, MinLength: 1, MaxLength: 1024). The client secret of the OpenID Connect client configured in your IdP. Required.
connect_url (string, Pattern: ^[^\r\n]*$, MaxLength: 2048). The URL of your IdP where the Security plugin can find the OpenID Connect metadata/configuration settings.
enabled (boolean). Enables or disables OpenID Connect authentication for OpenSearch. When enabled, users can authenticate using OpenID Connect with an Identity Provider.

Optional

header (string, Pattern: ^[^\r\n]*$, MinLength: 1, MaxLength: 1024). HTTP header name of the JWT token. Optional. Default is Authorization.
jwt_header (string, Pattern: ^[^\r\n]*$, MinLength: 1, MaxLength: 1024). The HTTP header that stores the token. Typically the Authorization header with the Bearer schema: Authorization: Bearer . Optional. Default is Authorization.
jwt_url_parameter (string, Pattern: ^[^\r\n]*$, MinLength: 1, MaxLength: 1024). If the token is not transmitted in the HTTP header, but as an URL parameter, define the name of the parameter here. Optional.
refresh_rate_limit_count (integer, Minimum: 10). The maximum number of unknown key IDs in the time frame. Default is 10. Optional.
refresh_rate_limit_time_window_ms (integer, Minimum: 10000). The time frame to use when checking the maximum number of unknown key IDs, in milliseconds. Optional.Default is 10000 (10 seconds).
roles_key (string, Pattern: ^[^\r\n]*$, MinLength: 1, MaxLength: 1024). The key in the JSON payload that stores the user’s roles. The value of this key must be a comma-separated list of roles. Required only if you want to use roles in the JWT.
scope (string, Pattern: ^[^\r\n]*$, MinLength: 1, MaxLength: 1024). The scope of the identity token issued by the IdP. Optional. Default is openid profile email address phone.
subject_key (string, Pattern: ^[^\r\n]*$, MinLength: 1, MaxLength: 1024). The key in the JSON payload that stores the user’s name. If not defined, the subject registered claim is used. Most IdP providers use the preferred_username claim. Optional.

opensearch¶

Appears on spec.userConfig.

OpenSearch settings.

Optional

action_auto_create_index_enabled (boolean). Explicitly allow or block automatic creation of indices. Defaults to true.
action_destructive_requires_name (boolean). Require explicit index names when deleting.
auth_failure_listeners (object). Opensearch Security Plugin Settings. See below for nested schema.
cluster_max_shards_per_node (integer, Minimum: 100, Maximum: 10000). Controls the number of shards allowed in the cluster per data node.
cluster_routing_allocation_node_concurrent_recoveries (integer, Minimum: 2, Maximum: 16). How many concurrent incoming/outgoing shard recoveries (normally replicas) are allowed to happen on a node. Defaults to node cpu count * 2.
email_sender_name (string, Pattern: ^[a-zA-Z0-9-_]+$, MaxLength: 40). Sender name placeholder to be used in Opensearch Dashboards and Opensearch keystore.
email_sender_password (string, Pattern: ^[^\x00-\x1F]+$, MaxLength: 1024). Sender password for Opensearch alerts to authenticate with SMTP server.
email_sender_username (string, Pattern: ^[^\x00-\x1F]+$, MaxLength: 320). Sender username for Opensearch alerts.
enable_security_audit (boolean). Enable/Disable security audit.
http_max_content_length (integer, Minimum: 1, Maximum: 2147483647). Maximum content length for HTTP requests to the OpenSearch HTTP API, in bytes.
http_max_header_size (integer, Minimum: 1024, Maximum: 262144). The max size of allowed headers, in bytes.
http_max_initial_line_length (integer, Minimum: 1024, Maximum: 65536). The max length of an HTTP URL, in bytes.
indices_fielddata_cache_size (integer, Minimum: 3, Maximum: 100). Relative amount. Maximum amount of heap memory used for field data cache. This is an expert setting; decreasing the value too much will increase overhead of loading field data; too much memory used for field data cache will decrease amount of heap available for other operations.
indices_memory_index_buffer_size (integer, Minimum: 3, Maximum: 40). Percentage value. Default is 10%. Total amount of heap used for indexing buffer, before writing segments to disk. This is an expert setting. Too low value will slow down indexing; too high value will increase indexing performance but causes performance issues for query performance.
indices_memory_max_index_buffer_size (integer, Minimum: 3, Maximum: 2048). Absolute value. Default is unbound. Doesn't work without indices.memory.index_buffer_size. Maximum amount of heap used for query cache, an absolute indices.memory.index_buffer_size maximum hard limit.
indices_memory_min_index_buffer_size (integer, Minimum: 3, Maximum: 2048). Absolute value. Default is 48mb. Doesn't work without indices.memory.index_buffer_size. Minimum amount of heap used for query cache, an absolute indices.memory.index_buffer_size minimal hard limit.
indices_queries_cache_size (integer, Minimum: 3, Maximum: 40). Percentage value. Default is 10%. Maximum amount of heap used for query cache. This is an expert setting. Too low value will decrease query performance and increase performance for other operations; too high value will cause issues with other OpenSearch functionality.
indices_query_bool_max_clause_count (integer, Minimum: 64, Maximum: 4096). Maximum number of clauses Lucene BooleanQuery can have. The default value (1024) is relatively high, and increasing it may cause performance issues. Investigate other approaches first before increasing this value.
indices_recovery_max_bytes_per_sec (integer, Minimum: 40, Maximum: 400). Limits total inbound and outbound recovery traffic for each node. Applies to both peer recoveries as well as snapshot recoveries (i.e., restores from a snapshot). Defaults to 40mb.
indices_recovery_max_concurrent_file_chunks (integer, Minimum: 2, Maximum: 5). Number of file chunks sent in parallel for each recovery. Defaults to 2.
ism_enabled (boolean). Specifies whether ISM is enabled or not.
ism_history_enabled (boolean). Specifies whether audit history is enabled or not. The logs from ISM are automatically indexed to a logs document.
ism_history_max_age (integer, Minimum: 1, Maximum: 2147483647). The maximum age before rolling over the audit history index in hours.
ism_history_max_docs (integer, Minimum: 1). The maximum number of documents before rolling over the audit history index.
ism_history_rollover_check_period (integer, Minimum: 1, Maximum: 2147483647). The time between rollover checks for the audit history index in hours.
ism_history_rollover_retention_period (integer, Minimum: 1, Maximum: 2147483647). How long audit history indices are kept in days.
knn_memory_circuit_breaker_enabled (boolean). Enable or disable KNN memory circuit breaker. Defaults to true.
knn_memory_circuit_breaker_limit (integer, Minimum: 3, Maximum: 100). Maximum amount of memory that can be used for KNN index. Defaults to 50% of the JVM heap size.
override_main_response_version (boolean). Compatibility mode sets OpenSearch to report its version as 7.10 so clients continue to work. Default is false.
plugins_alerting_filter_by_backend_roles (boolean). Enable or disable filtering of alerting by backend roles. Requires Security plugin. Defaults to false.
reindex_remote_whitelist (array of strings, MaxItems: 32). Whitelisted addresses for reindexing. Changing this value will cause all OpenSearch instances to restart.
script_max_compilations_rate (string, Pattern: ^[^\r\n]*$, MaxLength: 1024). Script compilation circuit breaker limits the number of inline script compilations within a period of time. Default is use-context.
search.insights.top_queries (object). See below for nested schema.
search_backpressure (object). Search Backpressure Settings. See below for nested schema.
search_max_buckets (integer, Minimum: 1, Maximum: 1000000). Maximum number of aggregation buckets allowed in a single response. OpenSearch default value is used when this is not defined.
shard_indexing_pressure (object). Shard indexing back pressure settings. See below for nested schema.
thread_pool_analyze_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.
thread_pool_analyze_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.
thread_pool_force_merge_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.
thread_pool_get_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.
thread_pool_get_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.
thread_pool_search_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.
thread_pool_search_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.
thread_pool_search_throttled_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.
thread_pool_search_throttled_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.
thread_pool_write_queue_size (integer, Minimum: 10, Maximum: 2000). Size for the thread pool queue. See documentation for exact details.
thread_pool_write_size (integer, Minimum: 1, Maximum: 128). Size for the thread pool. See documentation for exact details. Do note this may have maximum value depending on CPU count - value is automatically lowered if set to higher than maximum value.

auth_failure_listeners¶

Appears on spec.userConfig.opensearch.

Opensearch Security Plugin Settings.

Optional

internal_authentication_backend_limiting (object). See below for nested schema.
ip_rate_limiting (object). IP address rate limiting settings. See below for nested schema.

internal_authentication_backend_limiting¶

Appears on spec.userConfig.opensearch.auth_failure_listeners.

Optional

allowed_tries (integer, Minimum: 1, Maximum: 2147483647). The number of login attempts allowed before login is blocked.
authentication_backend (string, Enum: internal, MaxLength: 1024). internal_authentication_backend_limiting.authentication_backend.
block_expiry_seconds (integer, Minimum: 0, Maximum: 2147483647). The duration of time that login remains blocked after a failed login.
max_blocked_clients (integer, Minimum: 0, Maximum: 2147483647). internal_authentication_backend_limiting.max_blocked_clients.
max_tracked_clients (integer, Minimum: 0, Maximum: 2147483647). The maximum number of tracked IP addresses that have failed login.
time_window_seconds (integer, Minimum: 0, Maximum: 2147483647). The window of time in which the value for allowed_tries is enforced.
type (string, Enum: username, MaxLength: 1024). internal_authentication_backend_limiting.type.

ip_rate_limiting¶

Appears on spec.userConfig.opensearch.auth_failure_listeners.

IP address rate limiting settings.

Optional

allowed_tries (integer, Minimum: 1, Maximum: 2147483647). The number of login attempts allowed before login is blocked.
block_expiry_seconds (integer, Minimum: 0, Maximum: 36000). The duration of time that login remains blocked after a failed login.
max_blocked_clients (integer, Minimum: 0, Maximum: 2147483647). The maximum number of blocked IP addresses.
max_tracked_clients (integer, Minimum: 0, Maximum: 2147483647). The maximum number of tracked IP addresses that have failed login.
time_window_seconds (integer, Minimum: 0, Maximum: 36000). The window of time in which the value for allowed_tries is enforced.
type (string, Enum: ip, MaxLength: 1024). The type of rate limiting.

search.insights.top_queries¶

Appears on spec.userConfig.opensearch.

Optional

cpu (object). Top N queries monitoring by CPU. See below for nested schema.
latency (object). Top N queries monitoring by latency. See below for nested schema.
memory (object). Top N queries monitoring by memory. See below for nested schema.

cpu¶

Appears on spec.userConfig.opensearch.search.insights.top_queries.

Top N queries monitoring by CPU.

Optional

enabled (boolean). Enable or disable top N query monitoring by the metric.
top_n_size (integer, Minimum: 1). Specify the value of N for the top N queries by the metric.
window_size (string). The window size of the top N queries by the metric.

latency¶

Appears on spec.userConfig.opensearch.search.insights.top_queries.

Top N queries monitoring by latency.

Optional

enabled (boolean). Enable or disable top N query monitoring by the metric.
top_n_size (integer, Minimum: 1). Specify the value of N for the top N queries by the metric.
window_size (string). The window size of the top N queries by the metric.

memory¶

Appears on spec.userConfig.opensearch.search.insights.top_queries.

Top N queries monitoring by memory.

Optional

enabled (boolean). Enable or disable top N query monitoring by the metric.
top_n_size (integer, Minimum: 1). Specify the value of N for the top N queries by the metric.
window_size (string). The window size of the top N queries by the metric.

search_backpressure¶

Appears on spec.userConfig.opensearch.

Search Backpressure Settings.

Optional

mode (string, Enum: disabled, enforced, monitor_only). The search backpressure mode. Valid values are monitor_only, enforced, or disabled. Default is monitor_only.
node_duress (object). Node duress settings. See below for nested schema.
search_shard_task (object). Search shard settings. See below for nested schema.
search_task (object). Search task settings. See below for nested schema.

node_duress¶

Appears on spec.userConfig.opensearch.search_backpressure.

Node duress settings.

Optional

cpu_threshold (number, Minimum: 0, Maximum: 1). The CPU usage threshold (as a percentage) required for a node to be considered to be under duress. Default is 0.9.
heap_threshold (number, Minimum: 0, Maximum: 1). The heap usage threshold (as a percentage) required for a node to be considered to be under duress. Default is 0.7.
num_successive_breaches (integer, Minimum: 1). The number of successive limit breaches after which the node is considered to be under duress. Default is 3.

search_shard_task¶

Appears on spec.userConfig.opensearch.search_backpressure.

Search shard settings.

Optional

cancellation_burst (number, Minimum: 1). The maximum number of search tasks to cancel in a single iteration of the observer thread. Default is 10.0.
cancellation_rate (number, Minimum: 0). The maximum number of tasks to cancel per millisecond of elapsed time. Default is 0.003.
cancellation_ratio (number, Minimum: 0, Maximum: 1). The maximum number of tasks to cancel, as a percentage of successful task completions. Default is 0.1.
cpu_time_millis_threshold (integer, Minimum: 0). The CPU usage threshold (in milliseconds) required for a single search shard task before it is considered for cancellation. Default is 15000.
elapsed_time_millis_threshold (integer, Minimum: 0). The elapsed time threshold (in milliseconds) required for a single search shard task before it is considered for cancellation. Default is 30000.
heap_moving_average_window_size (integer, Minimum: 0). The number of previously completed search shard tasks to consider when calculating the rolling average of heap usage. Default is 100.
heap_percent_threshold (number, Minimum: 0, Maximum: 1). The heap usage threshold (as a percentage) required for a single search shard task before it is considered for cancellation. Default is 0.5.
heap_variance (number, Minimum: 0). The minimum variance required for a single search shard task’s heap usage compared to the rolling average of previously completed tasks before it is considered for cancellation. Default is 2.0.
total_heap_percent_threshold (number, Minimum: 0, Maximum: 1). The heap usage threshold (as a percentage) required for the sum of heap usages of all search shard tasks before cancellation is applied. Default is 0.5.

search_task¶

Appears on spec.userConfig.opensearch.search_backpressure.

Search task settings.

Optional

cancellation_burst (number, Minimum: 1). The maximum number of search tasks to cancel in a single iteration of the observer thread. Default is 5.0.
cancellation_rate (number, Minimum: 0). The maximum number of search tasks to cancel per millisecond of elapsed time. Default is 0.003.
cancellation_ratio (number, Minimum: 0, Maximum: 1). The maximum number of search tasks to cancel, as a percentage of successful search task completions. Default is 0.1.
cpu_time_millis_threshold (integer, Minimum: 0). The CPU usage threshold (in milliseconds) required for an individual parent task before it is considered for cancellation. Default is 30000.
elapsed_time_millis_threshold (integer, Minimum: 0). The elapsed time threshold (in milliseconds) required for an individual parent task before it is considered for cancellation. Default is 45000.
heap_moving_average_window_size (integer, Minimum: 0). The window size used to calculate the rolling average of the heap usage for the completed parent tasks. Default is 10.
heap_percent_threshold (number, Minimum: 0, Maximum: 1). The heap usage threshold (as a percentage) required for an individual parent task before it is considered for cancellation. Default is 0.2.
heap_variance (number, Minimum: 0). The heap usage variance required for an individual parent task before it is considered for cancellation. A task is considered for cancellation when taskHeapUsage is greater than or equal to heapUsageMovingAverage * variance. Default is 2.0.
total_heap_percent_threshold (number, Minimum: 0, Maximum: 1). The heap usage threshold (as a percentage) required for the sum of heap usages of all search tasks before cancellation is applied. Default is 0.5.

shard_indexing_pressure¶

Appears on spec.userConfig.opensearch.

Shard indexing back pressure settings.

Optional

enabled (boolean). Enable or disable shard indexing backpressure. Default is false.
enforced (boolean). Run shard indexing backpressure in shadow mode or enforced mode. In shadow mode (value set as false), shard indexing backpressure tracks all granular-level metrics, but it doesn’t actually reject any indexing requests. In enforced mode (value set as true), shard indexing backpressure rejects any requests to the cluster that might cause a dip in its performance. Default is false.
operating_factor (object). Operating factor. See below for nested schema.
primary_parameter (object). Primary parameter. See below for nested schema.

operating_factor¶

Appears on spec.userConfig.opensearch.shard_indexing_pressure.

Operating factor.

Optional

lower (number, Minimum: 0). Specify the lower occupancy limit of the allocated quota of memory for the shard. If the total memory usage of a shard is below this limit, shard indexing backpressure decreases the current allocated memory for that shard. Default is 0.75.
optimal (number, Minimum: 0). Specify the optimal occupancy of the allocated quota of memory for the shard. If the total memory usage of a shard is at this level, shard indexing backpressure doesn’t change the current allocated memory for that shard. Default is 0.85.
upper (number, Minimum: 0). Specify the upper occupancy limit of the allocated quota of memory for the shard. If the total memory usage of a shard is above this limit, shard indexing backpressure increases the current allocated memory for that shard. Default is 0.95.

primary_parameter¶

Appears on spec.userConfig.opensearch.shard_indexing_pressure.

Primary parameter.

Optional

node (object). See below for nested schema.
shard (object). See below for nested schema.

node¶

Appears on spec.userConfig.opensearch.shard_indexing_pressure.primary_parameter.

Required

soft_limit (number, Minimum: 0). Define the percentage of the node-level memory threshold that acts as a soft indicator for strain on a node. Default is 0.7.

shard¶

Appears on spec.userConfig.opensearch.shard_indexing_pressure.primary_parameter.

Required

min_limit (number, Minimum: 0). Specify the minimum assigned quota for a new shard in any role (coordinator, primary, or replica). Shard indexing backpressure increases or decreases this allocated quota based on the inflow of traffic for the shard. Default is 0.001.

opensearch_dashboards¶

Appears on spec.userConfig.

OpenSearch Dashboards settings.

Optional

enabled (boolean). Enable or disable OpenSearch Dashboards.
max_old_space_size (integer, Minimum: 64, Maximum: 2048). Limits the maximum amount of memory (in MiB) the OpenSearch Dashboards process can use. This sets the max_old_space_size option of the nodejs running the OpenSearch Dashboards. Note: the memory reserved by OpenSearch Dashboards is not available for OpenSearch.
opensearch_request_timeout (integer, Minimum: 5000, Maximum: 120000). Timeout in milliseconds for requests made by OpenSearch Dashboards towards OpenSearch.

private_access¶

Appears on spec.userConfig.

Allow access to selected service ports from private networks.

Optional

opensearch (boolean). Allow clients to connect to opensearch with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.
opensearch_dashboards (boolean). Allow clients to connect to opensearch_dashboards with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.
prometheus (boolean). Allow clients to connect to prometheus with a DNS name that always resolves to the service's private IP addresses. Only available in certain network locations.

privatelink_access¶

Appears on spec.userConfig.

Allow access to selected service components through Privatelink.

Optional

opensearch (boolean). Enable opensearch.
opensearch_dashboards (boolean). Enable opensearch_dashboards.
prometheus (boolean). Enable prometheus.

public_access¶

Appears on spec.userConfig.

Allow access to selected service ports from the public Internet.

Optional

opensearch (boolean). Allow clients to connect to opensearch from the public internet for service nodes that are in a project VPC or another type of private network.
opensearch_dashboards (boolean). Allow clients to connect to opensearch_dashboards from the public internet for service nodes that are in a project VPC or another type of private network.
prometheus (boolean). Allow clients to connect to prometheus from the public internet for service nodes that are in a project VPC or another type of private network.

s3_migration¶

Appears on spec.userConfig.

AWS S3 / AWS S3 compatible migration settings.

Required

access_key (string, Pattern: ^[^\r\n]*$). AWS Access key.
base_path (string, Pattern: ^[^\r\n]*$). The path to the repository data within its container. The value of this setting should not start or end with a /.
bucket (string, Pattern: ^[^\r\n]*$). S3 bucket name.
indices (string). A comma-delimited list of indices to restore from the snapshot. Multi-index syntax is supported.
region (string, Pattern: ^[^\r\n]*$). S3 region.
secret_key (string, Pattern: ^[^\r\n]*$). AWS secret key.
snapshot_name (string, Pattern: ^[^\r\n]*$). The snapshot name to restore from.

Optional

chunk_size (string, Pattern: ^[^\r\n]*$). Big files can be broken down into chunks during snapshotting if needed. Should be the same as for the 3rd party repository.
compress (boolean). when set to true metadata files are stored in compressed format.
endpoint (string, Pattern: ^[^\r\n]*$). The S3 service endpoint to connect to. If you are using an S3-compatible service then you should set this to the service’s endpoint.
include_aliases (boolean). Whether to restore aliases alongside their associated indexes. Default is true.
restore_global_state (boolean). If true, restore the cluster state. Defaults to false.
server_side_encryption (boolean). When set to true files are encrypted on server side.

saml¶

Appears on spec.userConfig.

OpenSearch SAML configuration.

Required

enabled (boolean). Enables or disables SAML-based authentication for OpenSearch. When enabled, users can authenticate using SAML with an Identity Provider.
idp_entity_id (string, Pattern: ^[^\r\n]*$, MinLength: 1, MaxLength: 1024). The unique identifier for the Identity Provider (IdP) entity that is used for SAML authentication. This value is typically provided by the IdP.
idp_metadata_url (string, Pattern: ^[^\r\n]*$, MinLength: 1, MaxLength: 2048). The URL of the SAML metadata for the Identity Provider (IdP). This is used to configure SAML-based authentication with the IdP.
sp_entity_id (string, Pattern: ^[^\r\n]*$, MinLength: 1, MaxLength: 1024). The unique identifier for the Service Provider (SP) entity that is used for SAML authentication. This value is typically provided by the SP.

Optional

idp_pemtrustedcas_content (string, MaxLength: 16384). This parameter specifies the PEM-encoded root certificate authority (CA) content for the SAML identity provider (IdP) server verification. The root CA content is used to verify the SSL/TLS certificate presented by the server.
roles_key (string, Pattern: ^[^\r\n]*$, MinLength: 1, MaxLength: 256). Optional. Specifies the attribute in the SAML response where role information is stored, if available. Role attributes are not required for SAML authentication, but can be included in SAML assertions by most Identity Providers (IdPs) to determine user access levels or permissions.
subject_key (string, Pattern: ^[^\r\n]*$, MinLength: 1, MaxLength: 256). Optional. Specifies the attribute in the SAML response where the subject identifier is stored. If not configured, the NameID attribute is used by default.