A Complete And Easy Guide To Check Elasticsearch

To poll Elasticsearch db status, we usually need to learn and try many many REST API. When we’re new, it would take quite a while to put all the pieces together.

As Ops/DevOps, we are usually more concerned about cluster health and data inside. Enclosed is A Simple Step-By-Step Guide to check elasticsearch. Cluster, Nodes, Shards, Indices, Documents And Beyond.

The Complete Guide To Query And Check Elasticsearch


Elasticsearch Cheat Sheet for impatient users:

What To Check Summary
elasticsearch-head[1] A web tool: $es_ip:$es_port/_plugin/head/
elasticdump Import and export tools
Get cluster health $es_ip:$es_port/_cluster/health?pretty
Get cluster setting $es_ip:$es_port/_cluster/settings?v
Get Index Setting $es_ip:$es_port/$es_name/_settings?v
List Nodes $es_ip:$es_port/_cat/nodes?v
List shards $es_ip:$es_port/_cat/shards?v
List indices $es_ip:$es_port/_cat/indices?v
Get allocation $es_ip:$es_port/_cat/allocation?v
Get Version $es_ip:$es_port
Version By CLI bin/elasticsearch –version
Indice Summary $es_ip:$es_port/$index/_count?pretty
Indice Stats $es_ip:$es_port/$index/_stats?pretty
Get all docs $es_ip:$es_port/$index/_search?pretty
Full Text Search $es_ip:$es_port/$index/_search?q=50
Search By field $es_ip:$es_port/$index/_search?q=f1:50
Search By 2 fields $es_ip:$es_port/$index/_search?q=”f1:v1&f2=v2″

1. Basic Intro About Elasticsearch

Generally speaking, elasticsearch is an efficient and API friendly search engine based on Lucene.

Some key concepts to know, before we dive deeper.

  • Cluster: a collection of ES nodes.
  • Nodes: server/instance stores data or coordinate with indexing and search capabilities.
  • Index: a collection of documents. Like tables in MySQL.
  • Type: To ensure optimal performance, we can define mappings for data types.
  • Document: a basic unit of information. Like record in SQL.
  • Shards: It allows us to horizontally split a big index, and store in multiple nodes.

2. GUI Tool: elasticsearch-head

elasticsearch-head is a web front end for browsing and interacting with an Elastic Search cluster[1]. It is pretty much like phpMyAdmin for mysql[2]. By default, elasticsearch doesn’t install this plugin.

Though its UI looks a bit primitive and less attracting, it’s certainly a useful tool for dummies like me. I frequently use the “Structured Query” menu to grab some random records, in order to understand unfamiliar ES indices.

http://localhost:9200/_plugin/head

Elasticsearch head plugin

3. Elasticsearch Global Info By API

Time for complicated, yet powerful REST API.

  • Check cluster health.
curl \
 http://localhost:9200/_cluster/health?pretty

Elasticsearch cluster health

Usually we pay attention to “status” field, which should be green. When it’s yellow, probably we have unassigned shards or unavailable nodes. When it’s red, all primary shards of certain indices are inactive. This is truely bad.

  • Check global setting and cluster stats[3].
# Get version
curl http://localhost:9200

# Get global settings
curl \
 http://localhost:9200/_cluster/settings

# Poll cluster stats
curl http://localhost:9200/_stats?pretty

Get version by CLI, without starting ES service.

bin/elasticsearch --version

  • List nodes and shards
# List nodes
curl http://localhost:9200/_cat/nodes?v

# List shards
curl http://localhost:9200/_cat/shards?v

  • Show elasticsearch slow query[4]: As DevOps/Ops, we definitely want to be on top of this. To enable this, make sure elasticsearch.yml is proplery configured. Tips: If you have problems to verify your setting about this, temporarily change thresholds to zero. Then every fetch operation should generate a slow log.

4. Check Elasticsearch Index By API

  • Show Basic Summary Of An Elasticsearch Index
curl \
http://localhost:9200/$index/_count?pretty

Summary Info For An Elasticsearch Index

“count” indicates how many records in the index. “_shards” shows how the index is horizontally split.

  • Check DB schema of An Index or A field.
# Get Index Mapping
curl \
localhost:9200/$index/_mapping?pretty

# Get Field Mapping
curl \
localhost:9200/$index/_mapping/$type

  • Indices verbose statistics.[5].
curl \
http://localhost:9200/$index/_stats

5. Check Documents Inside indices

  • Get all documents of an index. Be careful with big indices. You might get tons of output running this command.
curl \
http://localhost:9200/$index/_search?pretty

  • Full Text Search
# Get documents contains 50
curl \
http://localhost:9200/$index/_search?q=50

  • URI Search: filter by field.[6] URI search indicates parsing request parameters from URI.
# Get documents whose f1 is 50
curl \
http://localhost:9200/$index/_search?q=f1:50

  • Get documents with paging mechanism.[7]
# Get iterator, fetching 2 records each time
curl \
"localhost:9200/$index/_search?scroll=1m&size=2"\
-d '{
    "query" : {
        "match_all" : {}
    }
}'

# Get scroll id from previous command
# Then keep polling records of next page
curl 'localhost:9200/_search/scroll'  \
-d' {
    "scroll" : "1m",
    "scroll_id" : "INPUT_YOUR_SCROLL_ID"
}'
  • Advanced Search With Request Body.[8] This would fall into the realm of lucene now.
curl \
http://localhost:9200/$index/_search?pretty\
 -d '{
    "query": {
        "query_string":
        {"query": "(field1:value1)
            AND (field2:value2)"}
    }
}'

6. More Tools To Recommend

  • elasticdump is a 3rd tools to import and export elasticsearch indices[9]. Since it’s a npm module, we have to install nodejs/npm to use it. Frankly speaking, this is not my preference. And elasticsearch has built-in support, called snapshot[10].

    However I would still recommand elasticdump for small indices. Why? Snapshot a bit complicated than I would expect. And we have to configure elasticsearch.yml and restart es instances to take effect. Too intruding. Compared to elasticdump, I wish elasticsearch can have more native support in the future version.

More Reading:

Footnotes:

[3] www.elastic.co/guide/en/elasticsearch/reference/current/cluster-stats.html
[4] www.elastic.co/guide/en/elasticsearch/reference/current/index-modules-slowlog.html
[5] www.elastic.co/guide/en/elasticsearch/reference/current/indices-stats.html
[6] www.elastic.co/guide/en/elasticsearch/reference/current/search-uri-request.html
[7] www.elastic.co/guide/en/elasticsearch/reference/current/search-request-scroll.html
[8] www.elastic.co/guide/en/elasticsearch/reference/current/search-request-body.html
[10] www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html

Original URL: https://dennyzhang.com/query_elasticsearch

Connect with Denny In LinkedIn Or MailList

2 Responses to A Complete And Easy Guide To Check Elasticsearch

Leave a Reply

Your email address will not be published. Required fields are marked *