如何使用 Kibana Console DSL 查詢語言 - 操作 Elasticsearc 資料結構(上)

在 Kibana Console 可以用來直接執行查詢檢索及操作數據功能，再輸入 Elasticsearch 查詢語言 DSL

在這裡會列出一些實用的 DSL CRUD 查詢語句：

(在這裡的 index 都以 poc-trylog 來做範例說明)

在開始前先複習一下 Elasticsearch 與關聯資料庫概念的類比：

DBMS	Elasticsearch
database	index
table	type (7.0 後預設為 _doc)
row	document
column	field
schema	mapping
SQL	DSL (Descriptor structure language)

(在 7.0 之前，同一個 index 可以創建多個類型，7.0 以後，一個索引只能創建一種類型 - _doc)

在這裡我們會以傳輸方法及命令進行說明：

後面的部分我們會在提到 Elasticsearch Analyzer 分詞器是什麼？實際範例演練

GET「 _cat 」查詢支援命令及索引

透過 _cat CVT2HUGO: 命令後，可以透過 ```Ctrl/Cmd CVT2HUGO: 可以用來查詢支援的命令

CVT2HUGO: + Enter``` 送出結果，接下來，分別說明一些常見的查詢方式。
GET _cat

也可列出目前的索引

GET _cat/indeces

輸出結果

green  open apm-7.9.1-span-000004                6lTXIJjISs6MYutfkYHrSQ 1 0  121742      0  27.9mb  27.9mb
green  open apm-7.9.1-metric-000002              JDKWgF3wS1qbqQcKPF-RxA 1 0       0      0    208b    208b
green  open apm-7.9.1-metric-000003              VMKH7L8VRaaGaUA7t2MHnA 1 0   60738      0  12.7mb  12.7mb

GET「 _cat/indexName/?v 」查看詳細內容

透過 ?v 可以用來查看該索引的詳細內容，例如，查看. users_logs 的詳細內容：

GET _cat/users_logs?v

GET [ indexName/_search ] 搜尋索引內容

透過 [indexName]/_search 可以搜尋索引內容

例如

GET poc-trylog/_search

輸出結果如下：

{
  "took" : 1, //搜尋所需要的時間
  "timed_out" : false,
  "_shards" : { //分片，在集群時使用
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 3,//命中，目前命中(預設為 10000，也就是最多查詢10000個結果)
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "poc-trylog",//索引名稱
        "_type" : "_doc",//索引類型
        "_id" : "PzpcmnYB0g5WnlnpsGRo", //索引ID
        "_score" : 1.0,//權重分數
        "_source" : { //數據本身
          "@version" : "1",
          "@timestamp" : "2020-12-25T14:45:33.542Z",
          "host" : "172.26.0.1",
          "user" : "adam",
          "headers" : {
            "http_accept" : "*/*",
            "content_length" : "96",
            "http_host" : "127.0.0.1:8080",
            "content_type" : "application/json",
            "request_path" : "/logstash-poc-test/1",
            "http_user_agent" : "curl/7.64.1",
            "http_version" : "HTTP/1.1",
            "request_method" : "PUT"
          },
          "post_date" : "2020-12-07T10:12:12",
          "message" : "trying out Elasticsearch"
        }
      },...
    ]
  }
}

GET [ indexName/_count ] 查詢 index 結果總數

透過 _count

CVT2HUGO: 可查詢 index 結果總數，例如：
GET poc-trylog/_count

得出結果

{
  "count" : 3,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  }
}

GET [ indexName/indices ] 查看所有索引

GET [ indexName/_doc/[id] ] 依照 ID 查詢數據

例如，透過以下方式查詢指定ID內容

GET poc-trylog/_doc/PzpcmnYB0g5WnlnpsGRo

輸出結果

{
  "_index" : "poc-trylog",
  "_type" : "_doc",
  "_id" : "PzpcmnYB0g5WnlnpsGRo",
  "_version" : 1,
  "_seq_no" : 0,
  "_primary_term" : 1,
  "found" : true,
  "_source" : {
    "@version" : "1",
    "@timestamp" : "2020-12-25T14:45:33.542Z",
    "host" : "172.26.0.1",
    "user" : "adam",
    "headers" : {
      "http_accept" : "*/*",
      "content_length" : "96",
      "http_host" : "127.0.0.1:8080",
      "content_type" : "application/json",
      "request_path" : "/logstash-poc-test/1",
      "http_user_agent" : "curl/7.64.1",
      "http_version" : "HTTP/1.1",
      "request_method" : "PUT"
    },
    "post_date" : "2020-12-07T10:12:12",
    "message" : "trying out Elasticsearch"
  }
}

POST [ indexName/_doc/[id] ] 增加一筆數據，若存在就覆蓋

可以透過 POST 方式來新增文檔內容，並且可指定ID，如果ID沒有指定就會由Elasticsearch 自動生成。

如果是已經存在的ID，則會覆蓋掉原有ID內容。

POST poc-trylog/_doc/1234
{ "user" : "brown", 
  "post_date" : "2020-12-26T10:12:12",
  "message" : "hello brown" 
}

輸出結果

{
  "_index" : "poc-trylog",
  "_type" : "_doc",
  "_id" : "1234",
  "_version" : 1,
  "result" : "created",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 3,
  "_primary_term" : 1
}

POST [ indexName/_create/[id] ] 增加一筆數據，若存在就拒絕

前面我們提到，透過 _doc 新增數據，若ID已經存在就會進行覆蓋。

這裡的 _create 也是新增內容，但id如果存在就會報錯。

POST poc-trylog/_create/1234
{ "user" : "brown", 
  "post_date" : "2020-12-26T10:12:12",
  "message" : "hello brown3" 
}

輸出內容

{
  "error" : {
    "root_cause" : [
      {
        "type" : "version_conflict_engine_exception",
        "reason" : "[1234]: version conflict, document already exists (current version [2])",
        "index_uuid" : "nCu2xyJfSlycXx4e3DzWxw",
        "shard" : "0",
        "index" : "poc-trylog"
      }
    ],
    "type" : "version_conflict_engine_exception",
    "reason" : "[1234]: version conflict, document already exists (current version [2])",
    "index_uuid" : "nCu2xyJfSlycXx4e3DzWxw",
    "shard" : "0",
    "index" : "poc-trylog"
  },
  "status" : 409
}

POST [ indexName/_update/[id] ] 更新數據

更新數據可用 _update 更新內容：

POST poc-trylog/_update/1234
{ "doc": {
  "message" : "hello brown6"
  }
}

輸出結果

{
  "_index" : "poc-trylog",
  "_type" : "_doc",
  "_id" : "1234",
  "_version" : 4,
  "result" : "updated",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 6,
  "_primary_term" : 1
}

PUT [ indexName/_doc/[id] ] 更新數據

與前面提到的 _update

CVT2HUGO: 一樣，可以針對指定ID更新內容
PUT poc-trylog/_doc/1234
{
  "doc": {
    "message" : "hello bb7"
  }
}

輸出結果

{
  "_index" : "poc-trylog",
  "_type" : "_doc",
  "_id" : "1234",
  "_version" : 5,
  "result" : "updated",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 7,
  "_primary_term" : 1
}

PUT [ indexName/_create/[id] ] 增加一筆數據，若存在就拒絕

與前面 __create

如果 ID 存在則拒絕新增：

CVT2HUGO: 一樣，可以指定ID新增數據，若沒有指定則由 Elasticsearch 自動生成，
PUT poc-trylog/_create/1234
{ "user" : "cayla", 
  "post_date" : "2020-12-26T10:12:13",
  "message" : "hello3344" 
}

輸出結果

{
  "error" : {
    "root_cause" : [
      {
        "type" : "version_conflict_engine_exception",
        "reason" : "[1234]: version conflict, document already exists (current version [5])",
        "index_uuid" : "nCu2xyJfSlycXx4e3DzWxw",
        "shard" : "0",
        "index" : "poc-trylog"
      }
    ],
    "type" : "version_conflict_engine_exception",
    "reason" : "[1234]: version conflict, document already exists (current version [5])",
    "index_uuid" : "nCu2xyJfSlycXx4e3DzWxw",
    "shard" : "0",
    "index" : "poc-trylog"
  },
  "status" : 409
}

DELETE [ indexName ] 刪除索引

透過 DELETE 方法可以刪除指定的索引

DELETE poc-trylog

DELETE [ _doc/[id] ] 刪除數據

透過 DELETE 方法可以刪除指定的 id

DELETE poc-trylog/_doc/11

輸出結果：

{
  "_index" : "poc-trylog",
  "_type" : "_doc",
  "_id" : "11",
  "_version" : 2,
  "result" : "deleted",
  "_shards" : {
    "total" : 2,
    "successful" : 1,
    "failed" : 0
  },
  "_seq_no" : 16,
  "_primary_term" : 1
}

POST [ indexName/_bulk ] 批量新增

透過 _bulk

CVT2HUGO: 批量新增方式
POST poc-trylog/_bulk
{"index":{"_id": 11}}
{"user":"a","post_date" : "2020-12-26T10:12:11","message" : "txt1" }
{"index":{"_id": 12}}
{"user":"b","post_date" : "2020-12-26T10:12:12","message" : "txt2" }
{"index":{"_id": 13}}
{"user":"c","post_date" : "2020-12-26T10:12:13","message" : "txt3" }
{"index":{"_id": 14}}
{"user":"d","post_date" : "2020-12-26T10:12:14","message" : "txt4" }

輸出結果

{
  "took" : 6,
  "errors" : false,
  "items" : [
    {
      "index" : {
        "_index" : "poc-trylog",
        "_type" : "_doc",
        "_id" : "11",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 12,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "poc-trylog",
        "_type" : "_doc",
        "_id" : "12",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 13,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "poc-trylog",
        "_type" : "_doc",
        "_id" : "13",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 14,
        "_primary_term" : 1,
        "status" : 201
      }
    },
    {
      "index" : {
        "_index" : "poc-trylog",
        "_type" : "_doc",
        "_id" : "14",
        "_version" : 1,
        "result" : "created",
        "_shards" : {
          "total" : 2,
          "successful" : 1,
          "failed" : 0
        },
        "_seq_no" : 15,
        "_primary_term" : 1,
        "status" : 201
      }
    }
  ]
}

GET [ mget ] 批量+跨索引查詢

mget 批量查詢，可以從不同索引取得數據：

GET _mget
{
	"docs": [
    {"_index": "poc-logs", "_id": 12},
    {"_index": "movies", "_id": 1},
    {"_index": "users", "_id": 2}
  ]
}

備註：Kibana console 快捷鍵參考

Ctrl/Cmd + I

針對目前查詢自動建立 indent

Auto indent current request
Ctrl/Cmd + /

開啟目前查詢相關的文件

Open documentation for current request
Ctrl + Space

開啟自動完成

Open Auto complete (even if not typing)
Ctrl/Cmd + Enter

送出查詢

Submit request
Ctrl/Cmd + Up/Down

跳到前一個或下一個查詢的開始或結束

Jump to the previous/next request start or end.
Ctrl/Cmd + Alt + L

搜集或擴大目前的規模

Collapse/expand current scope.
Ctrl/Cmd + Option + 0

Collapse all scopes but the current one. Expand by adding a shift.
Down arrow

Switch focus to auto-complete menu. Use arrows to further select a term
Enter/Tab

Select the currently selected or the top most term in auto-complete menu
Esc

關閉 auto-complete 選單

Close auto-complete menu

如何使用 Kibana Console DSL 查詢語言 - 操作 Elasticsearc 資料結構(上)