Skip to content

Migration path for configuration index names

The new product is probably not supposed to store its configuration in indices named searchguard* or .signals*.

We need to migrate the indices to the new index names while retaining the ability to have mixed clusters.

Strategy

Eliatra Suite plugin for OpenSearch

The OpenSearch plugin comes in two variants:

  • The transitional compatibility edition
  • The final edition

The compatibility edition will keep using the old Search Guard index names. Additionally, it offers protection for all indices with the name .eliatra_internal_*. Thus, it will also protect the newly migrated indices.

The final edition will use the new index names. The final edition will drop any possiblity to customize internal index names. For the final edition, we can also remove the double index name handling in ConfigurationRepository:

https://git.eliatra.com/eliatra-suite/eliatra-opensearch-plugin/-/blob/main/security-core/src/main/java/com/eliatra/suite/securityplus/configuration/ConfigurationRepository.java#L106

eliatra-suite-migration.sh

A new tool eliatra-suite-migration.sh is created. The tool shall have a similar architecture to sgctl/spctl:

  • It is a shaded, executable JAR
  • It has sub-commands (for now only one)
  • It is able to read the connection configuration from an existing sgctl configuration

This tool does the following for now:

  • Suspend watch execution by DELETE /_signals/admin/_active
  • Create the indices used by Eliatra Suite and copy all content from the legacy Search Guard indices to it.
  • Delete the legacy Search Guard indices and create aliases with the names of the Search Guard indices pointing towards the new indices.
  • Resume watch execution by PUT /_signals/admin/_active

This tool needs to be executed after all nodes are migrated to the Eliatra Suite compatibility edition, but before any node is migrated to the final edition.

As the index names are configurable, we might want to give the possibility to feed a elasticsearch.yml to the tool in order to pick up the customized index names.

We keep it separate from spctl because:

  • It has a one-off purpose
  • Due to the renamed APIs, spctl can be only used after the migration completed.

Potentially, we can later extend this tool if we find more functionality which can facilitate cluster migration.

Index Name Mapping

  • searchguard, .searchguard -> .eliatra_internal__security_plus_config

  • .searchguard_sessions -> .eliatra_internal__security_plus_sessions

  • .searchguard_config_vars -> .eliatra_internal__security_plus_config_vars

  • .searchguard_authtokens -> .eliatra_internal__security_plus_authtokens

  • .searchguard_config_history -> .eliatra_internal__security_plus_config_history

  • .signals_watches -> .eliatra_internal__alerting_plus_watches

  • .signals_watches_state -> .eliatra_internal__alerting_plus_watches_state

  • .signals_watches_trigger_state -> .eliatra_internal__alerting_plus_watches_trigger_state

  • .signals_accounts -> .eliatra_internal__alerting_plus_accounts

  • .signals_settings -> .eliatra_internal__alerting_plus_settings

  • .searchguard_resource_owner -> .eliatra_internal__resource_owner

  • .signals_log_<now/d> -> Note: This is not a protected index. Having an alias for this is not possible due to the date math. I would propose to just leave the old index as is. Any Eliatra Suite nodes will then log to the new index, which will be called .alerting_plus_log_<now/d>.

Edited by Łukasz Soszynski