ELK useful commands
Run ELK in docker:
Create docker-compose.yml
version: '2.2' services: es01: image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1 container_name: es01 environment: - node.name=es01 - cluster.name=es-docker-cluster - discovery.seed_hosts=es02,es03 - cluster.initial_master_nodes=es01,es02,es03 - bootstrap.memory_lock=true - "ES_JAVA_OPTS=-Xms512m -Xmx512m" ulimits: memlock: soft: -1 hard: -1 volumes: - data01:/usr/share/elasticsearch/data ports: - 9200:9200 networks: - elastic es02: image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1 container_name: es02 environment: - node.name=es02 - cluster.name=es-docker-cluster - discovery.seed_hosts=es01,es03 - cluster.initial_master_nodes=es01,es02,es03 - bootstrap.memory_lock=true - "ES_JAVA_OPTS=-Xms512m -Xmx512m" ulimits: memlock: soft: -1 hard: -1 volumes: - data02:/usr/share/elasticsearch/data networks: - elastic es03: image: docker.elastic.co/elasticsearch/elasticsearch:7.12.1 container_name: es03 environment: - node.name=es03 - cluster.name=es-docker-cluster - discovery.seed_hosts=es01,es02 - cluster.initial_master_nodes=es01,es02,es03 - bootstrap.memory_lock=true - "ES_JAVA_OPTS=-Xms512m -Xmx512m" ulimits: memlock: soft: -1 hard: -1 volumes: - data03:/usr/share/elasticsearch/data networks: - elastic kib01: image: docker.elastic.co/kibana/kibana:7.12.1 container_name: kib01 ports: - 5601:5601 environment: ELASTICSEARCH_URL: http://es01:9200 ELASTICSEARCH_HOSTS: '["http://es01:9200","http://es02:9200","http://es03:9200"]' networks: - elastic volumes: data01: driver: local data02: driver: local data03: driver: local networks: elastic: driver: bridge
Run
docker-compose up -d
Test
Open link http://localhost:5601 and see Kibana home page.
Stop
docker-compose down
Useful commands
Get Elasticsearch info: curl http://localhost:9200
{ "name" : "es01", "cluster_name" : "es-docker-cluster", "cluster_uuid" : "er6oQrBjSGqrUu3gW03Eug", "version" : { "number" : "7.12.1", "build_flavor" : "default", "build_type" : "docker", "build_hash" : "3186837139b9c6b6d23c3200870651f10d3343b7", "build_date" : "2021-04-20T20:56:39.040728659Z", "build_snapshot" : false, "lucene_version" : "8.8.0", "minimum_wire_compatibility_version" : "6.8.0", "minimum_index_compatibility_version" : "6.0.0-beta1" }, "tagline" : "You Know, for Search" }
Get Elasticsearch cluster health: curl http://localhost:9200/_cluster/health
{ cluster_name: "es-docker-cluster", status: "green", timed_out: false, number_of_nodes: 3, number_of_data_nodes: 3, active_primary_shards: 7, active_shards: 14, relocating_shards: 0, initializing_shards: 0, unassigned_shards: 0, delayed_unassigned_shards: 0, number_of_pending_tasks: 0, number_of_in_flight_fetch: 0, task_max_waiting_in_queue_millis: 0, active_shards_percent_as_number: 100 }
Get Elasticsearch nodes list: curl http://localhost:9200/_cat/nodes
172.18.0.2 19 96 30 0.40 1.34 0.72 cdfhilmrstw - es02 172.18.0.4 51 96 30 0.40 1.34 0.72 cdfhilmrstw - es01 172.18.0.3 61 96 30 0.40 1.34 0.72 cdfhilmrstw * es03
Get Elasticsearch nodes list: curl "http://localhost:9200/_cat/nodes?v" https://www.elastic.co/guide/en/elasticsearch/reference/current/cat-nodes.html https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-info.html
ip heap.percent ram.percent cpu load_1m load_5m load_15m node.role master name 172.18.0.2 44 96 21 0.10 0.84 0.62 cdfhilmrstw - es02 172.18.0.4 73 96 22 0.10 0.84 0.62 cdfhilmrstw - es01 172.18.0.3 29 96 22 0.10 0.84 0.62 cdfhilmrstw * es03
Get Elasticsearch indices list: curl "http://localhost:9200/_cat/indices?v"
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size green open .kibana_7.12.1_001 9QAcJsFqTn6CDkYUrMFPjQ 1 1 15 6 4.2mb 2.1mb green open .apm-custom-link w_a9w3D8SCaX0kSwfxicbQ 1 1 0 0 416b 208b green open .apm-agent-configuration KS98WCEWQWmS5P0I8v1tPg 1 1 0 0 416b 208b green open .kibana_task_manager_7.12.1_001 NoBVpx5tQkOIQXggLYFxRw 1 1 9 212 465.5kb 335.4kb green open .kibana-event-log-7.12.1-000001 E4TzA9wQSWus9H2hHN0Ogg 1 1 4 0 43.5kb 21.7kb green open .tasks ec2oTla2RMKkV5tKseh1vQ 1 1 4 0 42.6kb 21.3kb
Create an empty index: curl -X PUT http://localhost:9200/pages
{"acknowledged":true,"shards_acknowledged":true,"index":"pages"}
Get information about shards: curl "http://localhost:9200/_cat/shards?v"
index shard prirep state docs store ip node .kibana_7.12.1_001 0 p STARTED 17 2.1mb 172.18.0.4 es03 .kibana_7.12.1_001 0 r STARTED 17 2.1mb 172.18.0.2 es01 pages 0 p STARTED 0 208b 172.18.0.5 es02 pages 0 r STARTED 0 208b 172.18.0.4 es03 .kibana-event-log-7.12.1-000002 0 r STARTED 1 5.6kb 172.18.0.4 es03 .kibana-event-log-7.12.1-000002 0 p STARTED 1 5.6kb 172.18.0.2 es01 .apm-agent-configuration 0 r STARTED 0 208b 172.18.0.5 es02 .apm-agent-configuration 0 p STARTED 0 208b 172.18.0.2 es01 .ds-ilm-history-5-2021.07.06-000002 0 p STARTED 172.18.0.5 es02 .ds-ilm-history-5-2021.07.06-000002 0 r STARTED 172.18.0.2 es01 .tasks 0 p STARTED 10 55.9kb 172.18.0.5 es02 .tasks 0 r STARTED 10 55.9kb 172.18.0.4 es03 .ds-ilm-history-5-2021.05.21-000001 0 p STARTED 172.18.0.4 es03 .ds-ilm-history-5-2021.05.21-000001 0 r STARTED 172.18.0.2 es01 .apm-custom-link 0 p STARTED 0 208b 172.18.0.5 es02 .apm-custom-link 0 r STARTED 0 208b 172.18.0.4 es03 .kibana-event-log-7.12.1-000001 0 r STARTED 5 27.1kb 172.18.0.5 es02 .kibana-event-log-7.12.1-000001 0 p STARTED 5 27.1kb 172.18.0.2 es01 .kibana_task_manager_7.12.1_001 0 p STARTED 9 215.5kb 172.18.0.5 es02 .kibana_task_manager_7.12.1_001 0 r STARTED 9 312kb 172.18.0.4 es03
Delete index: curl -X DELETE http://localhost:9200/pages
{"acknowledged":true}
Create an empty index with parameters:
PUT /products
{
"settings": {
"number_of_shards": 2,
"number_of_replicas": 2
}
}
{ "acknowledged" : true, "shards_acknowledged" : true, "index" : "products" }
Managing documents
Insert document into the index:
POST /products/_doc
{
"name": "Product1",
"price": 10
}
{ "_index" : "products", "_type" : "_doc", "_id" : "b549e3oB0mLlmgW9c9jO", "_version" : 1, "result" : "created", "_shards" : { "total" : 3, "successful" : 3, "failed" : 0 }, "_seq_no" : 0, "_primary_term" : 1 }
Insert new document into the index with custom ID or replace it with new data by ID:
PUT /products/_doc/p200
{
"name": "Product2",
"price": 15
}
{ "_index" : "products", "_type" : "_doc", "_id" : "p200", "_version" : 1, "result" : "created", "_shards" : { "total" : 3, "successful" : 3, "failed" : 0 }, "_seq_no" : 0, "_primary_term" : 1 }
Get document by ID:
GET /products/_doc/p200
{ "_index" : "products", "_type" : "_doc", "_id" : "p200", "_version" : 1, "_seq_no" : 0, "_primary_term" : 1, "found" : true, "_source" : { "name" : "Product2", "price" : 15 } }
Update document by ID:
POST /products/_update/p200
{
"doc": {
"price": 19
}
}
{ "_index" : "products", "_type" : "_doc", "_id" : "p200", "_version" : 2, "result" : "updated", "_shards" : { "total" : 3, "successful" : 3, "failed" : 0 }, "_seq_no" : 1, "_primary_term" : 1 }
Delete document: curl -X DELETE http://localhost:9200/products/_doc/p200
{ "_index":"products", "_type":"_doc", "_id":"p200", "_version":3, "result":"deleted", "_shards":{ "total":3, "successful":3, "failed":0 }, "_seq_no":2, "_primary_term":1 }
Add comfortable view parameter: curl ... http://...?pretty - will add formatting