In one of my previous post, I talked about how I set up Elasticsearch 2.3.5 on ECS. I got a comment in that post that prompted me to update the setup for Elasticsearch 5. It’s been awhile, but better late than never right? So I gave it a go! In this post I’ll like to share what I found in the process.
There were a couple of other configuration changes that were required to upgrade to 5.2.2 from 2.3.5 but they weren’t difficult, except one that may potentially deter you from using ECS with Elasticsearch 5, for the time being at least.
At this point, I’ll mention a caveat that will likely save you an hour of headache and trouble.
Long story short, You will need to SSH into the ECS instances to run the command on the parent to get past the error message below. I am not aware of any other solutions but if you do, feel free to let me know in the comments section below!
elasticsearch:5.2.2 max virtual memory areas vm.max_map_count  likely too low, increase to at least 
This docker-library/elasticsearch github issue suggests running
sudo sysctl -w vm.max_map_count=262144 or run the docker
container with the
--sysctl option to fix this problem.
However, because of how ECS implements the agents currently, many of the docker run options are not available. This is well-documented over at the amazon-ecs-agent Github repository so I won’t echo them here. But it does seem like there are a bunch of others who are encountering the same issue.
In my opinion, this makes the combination slightly less-than-ideal because the manual configuration work that is required on the EC2 instances takes away some of the benefits of implementing ElasticSearch in ECS.
If you’re okay with the manual configuration running that command on the instances, or for example, if you plan to provision a few instances and leave them there for awhile, then this hiccup would deal no damage.
Let’s forge on.
FROM elasticsearch:2.3 WORKDIR /usr/share/elasticsearch RUN bin/plugin install cloud-aws RUN bin/plugin install mobz/elasticsearch-head RUN bin/plugin install analysis-phonetic COPY elasticsearch.yml config/elasticsearch.yml COPY logging.yml config/logging.yml COPY elasticsearch-entrypoint.sh /docker-entrypoint.sh
And modified to:
FROM elasticsearch:5.2.2 COPY elasticsearch.yml config/elasticsearch.yml COPY logging.yml config/logging.yml COPY elasticsearch-entrypoint.sh /docker-entrypoint.sh RUN bin/elasticsearch-plugin install discovery-ec2
Notable changes include bumping the version and changing
cloud-aws plugin to
is the new plugin for the same purpose of node discovery in cloud environments.
File Descriptors and Ulimits
I needed to change the docker-compose file slightly to include the
ulimits. It is a new mandatory configuration item.
You can find out more in this documentation.
version: '2' services: data: build: ./docker-data/ volumes: - /usr/share/elasticsearch/data search: build: ./docker-elasticsearch/ volumes_from: - data ports: - "9200:9200" - "9300:9300" ulimits: nofile: soft: 65536 hard: 65536
plugin.mandatory: cloud-aws and
discovery.type: EC2 and
discovery.zen.ping.multicast.enabled: false has been
removed or modified to the following below.
script.inline: true bootstrap.memory_lock: false network.host: 0.0.0.0 network.publish_host: _ec2:privateIp_ discovery.zen.hosts_provider: ec2 discovery.ec2.groups: dockerecs
Task Definition / Heap Size
In Elasticsearch 5, the heap size is also a mandatory configuration. For this, I set it directly in ECS via
the JSON task definition. I had to set the
ES_JAVA_OPTS for it to work.
It isn’t a whole lot of changes but it did take some time googling each of the issues that came up as I tried to start
the services on ECS and also eventually had to SSH into the instance to set the
vm.max_map_count before I managed to
get the cluster up.
This is obviously less than ideal in a deployment process which otherwise could be full-automated. But if you’re still looking ahead to use ElasticSearch 5 in ECS, I hope the above steps serve you well!