[Tutorial] Elasticsearch Backup/Restore Using S3

juju
1

As a follow up to the previous elasticsearch post I felt it would be helpful to highlight a few useful elasticsearch lifecycle operations around backup/restore using S3.

Configure undocumented system property

To use s3 as a snapshot repository we need to apply an undocumented system property in the /etc/elasticsearc/jvm.options on all nodes in the cluster.

-Des.allow_insecure_settings=true

To apply this add the key=val to the file.

juju run --application elasticsearch "echo '-Des.allow_insecure_settings=true' >> /etc/elasticsearch/jvm.options"

Configure Elasticsearch to use the S3 repository backend

Set the elasticsearch-keystore values so we can backup/restore a snapshot using an s3 backend.

juju run --application elasticsearch \
    "export ES_PATH_CONF=/etc/elasticsearch && echo -n '<aws-access-key>' | /usr/share/elasticsearch/bin/elasticsearch-keystore add --stdin s3.client.default.access_key"

juju run --application elasticsearch \
    "export ES_PATH_CONF=/etc/elasticsearch && echo -n '<aws-secret-key>' | /usr/share/elasticsearch/bin/elasticsearch-keystore add --stdin s3.client.default.secret_key"

# Restart elasticsearch
juju run --application elasticsearch "service elasticsearch restart"

If you are working with a heterogeneous deploy this would need to be preformed on all node types:

# Master nodes
juju run --application es-master \
    "export ES_PATH_CONF=/etc/elasticsearch && echo -n '<aws-access-key>' | /usr/share/elasticsearch/bin/elasticsearch-keystore add --stdin s3.client.default.access_key"

juju run --application es-master \
    "export ES_PATH_CONF=/etc/elasticsearch && echo -n '<aws-secret-key>' | /usr/share/elasticsearch/bin/elasticsearch-keystore add --stdin s3.client.default.secret_key"

# Data nodes
juju run --application es-data \
    "export ES_PATH_CONF=/etc/elasticsearch && echo -n '<aws-access-key>' | /usr/share/elasticsearch/bin/elasticsearch-keystore add --stdin s3.client.default.access_key"

juju run --application es-data \
    "export ES_PATH_CONF=/etc/elasticsearch && echo -n '<aws-secret-key>' | /usr/share/elasticsearch/bin/elasticsearch-keystore add --stdin s3.client.default.secret_key"


# Restart elasticsearch on all node types
juju run --application es-master "service elasticsearch restart"
juju run --application es-data "service elasticsearch restart"

Setup the snapshot repository

To backup and index to an s3 location:

curl -X PUT "http://<elasticsearch-node-ip>:9200/_snapshot/<repository-name>" -H 'Content-Type: application/json' -d'
{
  "type": "s3",
  "settings": {
    "bucket": "<s3-bucket-name>",
    "base_path": "<s3-base-path>",
    "region": "<region-if-applicable>",
    "access_key": "<aws-access-key>",
    "secret_key": <aws-secret-key>"
  }
}'

Backup a snapshot of one or more indices to an S3 repository

curl -X PUT "http://<elasticsearch-node-ip>:9200/_snapshot/<repository-name>/<snapshot-name>" -H 'Content-Type: application/json' -d'
{
  "indices": "<indices-to-include-in-snapshot-separated-by-commas>",
  "ignore_unavailable": true,
  "include_global_state": false
}
'

Get the status of the snapshot operation

curl "http://<elasticsearch-node-ip>:9200/_cat/snapshots/<repository-name>"

Restore a snapshot of one or more indices from S3 repository

curl -X POST "http://<elasticsearch-node-ip>:9200/_snapshot/<repository-name>/<snapshot-name>/_restore" -H 'Content-Type: application/json' -d'
{
  "indices": "<indices-to-include-in-snapshot-restore-separated-by-commas>",
  "index_settings": {
    "index.number_of_replicas": 0
  },
  "ignore_index_settings": [
    "index.refresh_interval"
  ]
}
'

Modify index settings

Example of how to modify index settings.

curl -X PUT "http://<elasticsearch-node-ip>:9200/<index-name>/_settings" -H 'Content-Type: application/json' -d'
{
    "index" : {
        "number_of_replicas" : 1
    }
}'
3 Likes
2

Excellent post @jamesbeedy