To poll Elasticsearch db status, we usually need to learn and try many many REST API. When we’re new, it would take quite a while to put all the pieces together.
As Ops/DevOps, we are usually more concerned about cluster health and data inside. Enclosed is A Simple Step-By-Step Guide to check elasticsearch. Cluster, Nodes, Shards, Indices, Documents And Beyond.
Generally speaking, elasticsearch is an efficient and API friendly search engine based on Lucene.
Some key concepts to know, before we dive deeper.
Cluster: a collection of ES nodes.
Nodes: server/instance stores data or coordinate with indexing and search capabilities.
Index: a collection of documents. Like tables in MySQL.
Type: To ensure optimal performance, we can define mappings for data types.
Document: a basic unit of information. Like record in SQL.
Shards: It allows us to horizontally split a big index, and store in multiple nodes.
1.2 2. GUI Tool: elasticsearch-head
elasticsearch-head is a web front end for browsing and interacting with an Elastic Search cluster[1]. It is pretty much like phpMyAdmin for mysql[2]. By default, elasticsearch doesn’t install this plugin.
Though its UI looks a bit primitive and less attracting, it’s certainly a useful tool for dummies like me. I frequently use the “Structured Query” menu to grab some random records, in order to understand unfamiliar ES indices.
Usually we pay attention to “status” field, which should be green. When it’s yellow, probably we have unassigned shards or unavailable nodes. When it’s red, all primary shards of certain indices are inactive. This is truely bad.
Check global setting and cluster stats[3].
# Get version
curl http://localhost:9200
# Get global settings
curl \
http://localhost:9200/_cluster/settings
# Poll cluster stats
curl http://localhost:9200/_stats?pretty
Get version by CLI, without starting ES service.
bin/elasticsearch --version
List nodes and shards
# List nodes
curl http://localhost:9200/_cat/nodes?v
# List shards
curl http://localhost:9200/_cat/shards?v
Show elasticsearch slow query[4]: As DevOps/Ops, we definitely want to be on top of this. To enable this, make sure elasticsearch.yml is proplery configured. Tips: If you have problems to verify your setting about this, temporarily change thresholds to zero. Then every fetch operation should generate a slow log.
1.4 4. Check Elasticsearch Index By API
Show Basic Summary Of An Elasticsearch Index
curl \
http://localhost:9200/$index/_count?pretty
“count” indicates how many records in the index. “shards” shows how the index is horizontally split.
Check DB schema of An Index or A field.
# Get Index Mapping
curl \
localhost:9200/$index/_mapping?pretty
# Get Field Mapping
curl \
localhost:9200/$index/_mapping/$type
Indices verbose statistics.[5].
curl \
http://localhost:9200/$index/_stats
1.5 5. Check Documents Inside indices
Get all documents of an index. Be careful with big indices. You might get tons of output running this command.
# Get documents contains 50
curl \
http://localhost:9200/$index/_search?q=50
URI Search: filter by field.[6] URI search indicates parsing request parameters from URI.
# Get documents whose f1 is 50
curl \
http://localhost:9200/$index/_search?q=f1:50
Get documents with paging mechanism.[7]
# Get iterator, fetching 2 records each time
curl \"localhost:9200/$index/_search?scroll=1m&size=2"\
-d '{ "query" : { "match_all" : {} }}'# Get scroll id from previous command# Then keep polling records of next page
curl 'localhost:9200/_search/scroll'\
-d' { "scroll" : "1m", "scroll_id" : "INPUT_YOUR_SCROLL_ID"}'
Advanced Search With Request Body.[8] This would fall into the realm of lucene now.
elasticdump is a 3rd tools to import and export elasticsearch indices[9]. Since it’s a npm module, we have to install nodejs/npm to use it. Frankly speaking, this is not my preference. And elasticsearch has built-in support, called snapshot[10].
However I would still recommand elasticdump for small indices. Why? Snapshot a bit complicated than I would expect. And we have to configure elasticsearch.yml and restart es instances to take effect. Too intruding. Compared to elasticdump, I wish elasticsearch can have more native support in the future version.
[…] extra wrapper layer. Let’s say, you need to trigger some elasticsearch actions by programming. You can either call elasticsearch management REST API directly by shell, or use […]
[…] extra wrapper layer. Let’s say, you need to trigger some elasticsearch actions by programming. You can either call elasticsearch management REST API directly by shell, or use […]
[…] A Complete And Easy Guide To Check Elasticsearch […]