2. 程式人生 > >Elastic search 系統學習之四： 文件API

Elastic search 系統學習之四： 文件API

阿新 • • 發佈：2019-02-20

一、Index API

1、插入文件

curl -XPUT 'localhost:9200/twitter/tweet/1?pretty' -H 'Content-Type: application/json' -d'
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
'
或

http://localhost:5601/app/kibana#/dev_tools/console?load_from=https://www.elastic.co/guide/en/elasticsearch/reference/current/snippets/docs-index_/1.json

2、自動建立索引

索引操作自動建立索引

put操作可以手動建立新型別

動態建立索引禁止：

  action.auto_create_index: false 或 the cluster update settings API

動態對映型別禁止：

  index.mapper.dynamic： true

動態建立索引模式：

    action.auto_create_index:aaa*,-bbb*,+ccc*,-*

      +表示允許，-表示不允許

3、版本控制

curl -XPUT 'localhost:9200/twitter/tweet/1?version=2&pretty' -H 'Content-Type: application/json' -d'
{
    "message" : "elasticsearch now has versioning support, double cool!"
}
'
或
PUT twitter/tweet/1?version=2
{
    "message" : "elasticsearch now has versioning support, double cool!"
}

4、版本控制型別

internal，external or external_gt, external_gte

5、操作型別

op_type: create   存在就會建立失敗

PUT twitter/tweet/1?op_type=create
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"

}

或

curl -XPUT 'localhost:9200/twitter/tweet/1?op_type=create&pretty' -H 'Content-Type: application/json' -d'
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
'

6、自動產生id

POST twitter/tweet/
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}

或

curl -XPOST 'localhost:9200/twitter/tweet/?pretty' -H 'Content-Type: application/json' -d'
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
'

7、路由

POST /twitter/tweet?routing=kimchy&pretty
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}

或

curl -XPOST 'localhost:9200/twitter/tweet?routing=kimchy&pretty' -H 'Content-Type: application/json' -d'
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
'

文件根據routing值將文件路由到相應的分片

8、分散式

分片有主次之分

9、等待分片啟用

index.write.wait_for_active_shards: 2  寫操作會等待2個分片備份恢復才會返回或者超時

 及時設定為all,

也不一定能保證一定寫成功

判斷是否啟用是在各個備份寫入前判斷的

10、重新整理

       寫入後進行重新整理

11、等待更新noop update

12、超時

       PUT twitter/tweet/1?timeout=5m
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}

或

curl -XPUT 'localhost:9200/twitter/tweet/1?timeout=5m&pretty' -H 'Content-Type: application/json' -d'
{
    "user" : "kimchy",
    "post_date" : "2009-11-15T14:12:12",
    "message" : "trying out Elasticsearch"
}
'

二、獲取

1、查詢文件

GET twitter/tweet/1

或

curl -XGET 'localhost:9200/twitter/tweet/0?pretty'

文件存在查詢

HEAD /twitter/tweet/1

或

curl -XHEAD 'localhost:9200/twitter/tweet/0?pretty'

2、實時性

realtimeparameter: false 關閉實時性

預設，都是實時的

文件更新後沒有refresh，查詢會導致refresh，從而使文件可見

3、源過濾  官方教程有毛病，不配套

關閉源檢索？

GET twitter/tweet/0?_source=false

或

curl -XGET 'localhost:9200/twitter/tweet/0?_source=false&pretty'

獲取部分源資訊

GET twitter/tweet/1?_source=user&_source_exclude=messag*


4、儲存域資訊

設定counter域為非儲存域

PUT test
{
   "mappings": {
      "tweet": {
         "properties": {
            "counter": {
               "type": "integer",
               "store": false
            },
            "tags": {
               "type": "keyword",
               "store": true
            }
         }
      }
   }

}

或

curl -XPUT 'localhost:9200/test?pretty' -H 'Content-Type: application/json' -d'
{
   "mappings": {
      "tweet": {
         "properties": {
            "counter": {
               "type": "integer",
               "store": false
            },
            "tags": {
               "type": "keyword",
               "store": true
            }
         }
      }
   }
}
'


PUT test/tweet/1
{
    "counter" : 1,
    "tags" : ["red"]
}

或

curl -XPUT 'localhost:9200/test/tweet/1?pretty' -H 'Content-Type: application/json' -d'
{
    "counter" : 1,
    "tags" : ["red"]
}
'

GET /test/tweet/1?stored_fields=tags,counter

或

curl -XGET 'localhost:9200/test/tweet/1?stored_fields=tags,counter&pretty'

5、直接獲取_source

獲取文件source資訊

GET twitter/tweet/1/_source

或

curl -XGET 'localhost:9200/twitter/tweet/1/_source?pretty'

過濾source資訊

GET /twitter/tweet/1/_source?_source_include=user&_source_exclude=message'
或

curl -XGET 'localhost:9200/twitter/tweet/1/_source?_source_include=*.id&_source_exclude=entities'&pretty'

測試文件是否存在

HEAD twitter/tweet/1/_source

或

curl -XHEAD 'localhost:9200/twitter/tweet/1/_source?pretty'

6、路由

GET twitter/tweet/2?routing=user1

7、GET偏向

預設各個備份隨機提供服務

_primary， _local

8、重新整理

     refresh:true

9、分散式

10、版本控制

三、刪除API

1、操作

      GET twitter/tweet/1?routing=user1

2、版本控制

3、路由

      DELETE /twitter/tweet/1?routing=kimchy

4、自動建立索引

      如果刪除的檔案的索引不存在，delete操作會自動建立索引

5、等待啟用的分片

6、重新整理

7、超時

     DELETE /twitter/tweet/1?timeout=5m

     或

     curl -XDELETE 'localhost:9200/twitter/tweet/1?timeout=5m&pretty'


四、通過query刪除文件API

1、通過query刪除文件

POST twitter/_delete_by_query
{
  "query": {
    "match": {
      "message": "trying"
    }
  }

}

或

curl -XPOST 'localhost:9200/twitter/_delete_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "match": {
      "message": "some message"
    }
  }
}
'

2、過程

     獲得索引快照--------->刪除匹配文件   兩個步驟中間發生了文件更新，刪除會有版本衝突

3、刪除索引所有文件

POST twitter/tweet/_delete_by_query?conflicts=proceed
{
  "query": {
    "match_all": {}
  }

}

或

curl -XPOST 'localhost:9200/twitter/tweet/_delete_by_query?conflicts=proceed&pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "match_all": {}
  }
}
'

4、多索引清空

POST twitter,blog/tweet,post/_delete_by_query
{
  "query": {
    "match_all": {}
  }

}

或

curl -XPOST 'localhost:9200/twitter,blog/tweet,post/_delete_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "match_all": {}
  }
}
'

5、路由限定處理的分片

POST twitter/_delete_by_query?routing=1
{
  "query": {
    "range" : {
        "age" : {
           "gte" : 10
        }
    }
  }
}

或

curl -XPOST 'localhost:9200/twitter/_delete_by_query?routing=1&pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "range" : {
        "age" : {
           "gte" : 10
        }
    }
  }
}
'
6、限定滾動數

POST twitter/_delete_by_query?scroll_size=5000
{
  "query": {
    "term": {
      "user": "kimchy"
    }
  }

}

或

curl -XPOST 'localhost:9200/twitter/_delete_by_query?scroll_size=5000&pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "term": {
      "user": "kimchy"
    }
  }
}
'

7、URL引數

待補充

https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-delete-by-query.html


8、任務介面

獲取delete-by-query任務的狀態

GET _tasks?detailed=true&actions=*/delete/byquery
或

curl -XGET 'localhost:9200/_tasks?detailed=true&actions=*/delete/byquery&pretty'

根據任務id直接檢視任務

GET /_tasks/taskId:1

或

curl -XGET 'localhost:9200/_tasks/taskId:1?pretty'



9、取消delete-by-query任務

POST _tasks/task_id:1/_cancel

或

curl -XPOST 'localhost:9200/_tasks/task_id:1/_cancel?pretty'

10、rethrottling

POST _delete_by_query/task_id:1/_rethrottle?requests_per_second=-1

或

curl -XPOST 'localhost:9200/_delete_by_query/task_id:1/_rethrottle?requests_per_second=-1&pretty'

11、人工分片

POST twitter/_delete_by_query
{
  "slice": {
    "id": 0,
    "max": 2
  },
  "query": {
    "range": {
      "likes": {
        "lt": 10
      }
    }
  }
}

或

curl -XPOST 'localhost:9200/twitter/_delete_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "slice": {
    "id": 0,
    "max": 2
  },
  "query": {
    "range": {
      "likes": {
        "lt": 10
      }
    }
  }
}
'
curl -XPOST 'localhost:9200/twitter/_delete_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "slice": {
    "id": 1,
    "max": 2
  },
  "query": {
    "range": {
      "likes": {
        "lt": 10
      }
    }
  }
}
'

確認工作：

GET _refresh
POST twitter/_search?size=0&filter_path=hits.total
{
  "query": {
    "range": {
      "likes": {
        "lt": 10
      }
    }
  }
}

或

curl -XGET 'localhost:9200/_refresh?pretty'
curl -XPOST 'localhost:9200/twitter/_search?size=0&filter_path=hits.total&pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "range": {
      "likes": {
        "lt": 10
      }
    }
  }
}
'

12、自動分片

POST twitter/_delete_by_query?refresh&slices=5
{
  "query": {
    "range": {
      "likes": {
        "lt": 10
      }
    }
  }
}

或

curl -XPOST 'localhost:9200/twitter/_delete_by_query?refresh&slices=5&pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "range": {
      "likes": {
        "lt": 10
      }
    }
  }
}
'

確認：

POST twitter/_search?size=0&filter_path=hits.total
{
  "query": {
    "range": {
      "likes": {
        "lt": 10
      }
    }
  }
}


五、更新API

PUT /test1/type1/1
{
    "counter" : 1,
    "tags" : ["red"]
}

1、指令碼更新

POST test1/type1/1/_update
{
    "script" : {
        "source": "ctx._source.counter += params.count",
        "lang": "painless",
        "params" : {
            "count" : 4
        }
    }
}

或

curl -XPOST 'localhost:9200/test1/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "script" : {
        "source": "ctx._source.counter += params.count",
        "lang": "painless",
        "params" : {
            "count" : 4
        }
    }
}
'

新增一個tag

POST test1/type1/1/_update
{
    "script" : {
        "source": "ctx._source.tags.add(params.tag)",
        "lang": "painless",
        "params" : {
            "tag" : "blue"
        }
    }
}
或

curl -XPOST 'localhost:9200/test/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "script" : {
        "source": "ctx._source.tags.add(params.tag)",
        "lang": "painless",
        "params" : {
            "tag" : "blue"
        }
    }
}
'

新增一個新域

POST test1/type1/1/_update
{
    "script" : "ctx._source.new_field = 'value_of_new_field'"
}

或

curl -XPOST 'localhost:9200/test1/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "script" : "ctx._source.new_field = \u0027value_of_new_field\u0027"
}
'

刪除一個域

POST test1/type1/1/_update
{
    "script" : "ctx._source.remove('new_field')"
}

或

curl -XPOST 'localhost:9200/test/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "script" : "ctx._source.remove(\u0027new_field\u0027)"
}
'

條件刪除

POST test1/type1/1/_update
{
    "script" : {
        "source": "if (ctx._source.tags.contains(params.tag)) { ctx.op = 'delete' } else { ctx.op = 'none' }",
        "lang": "painless",
        "params" : {
            "tag" : "green"
        }
    }
}

或

curl -XPOST 'localhost:9200/test/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "script" : {
        "source": "if (ctx._source.tags.contains(params.tag)) { ctx.op = \u0027delete\u0027 } else { ctx.op = \u0027none\u0027 }",
        "lang": "painless",
        "params" : {
            "tag" : "green"
        }
    }
}
'

2、區域性文件更新

併入到文件中

POST test1/type1/1/_update
{
    "doc" : {
        "name" : "new_name"
    }

}

或

curl -XPOST 'localhost:9200/test1/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "doc" : {
        "name" : "new_name"
    }
}
'

3、刪除等待更新

不改變任何內容時候返回： result:noop

POST test1/type1/1/_update
{
    "doc" : {
        "name" : "new_name"
    }
}

或

curl -XPOST 'localhost:9200/test1/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "doc" : {
        "name" : "new_name"
    }
}
'

使noop行為失效

POST test1/type1/1/_update
{
    "doc" : {
        "name" : "new_name"
    },
    "detect_noop": false
}

或

curl -XPOST 'localhost:9200/test1/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "doc" : {
        "name" : "new_name"
    },
    "detect_noop": false
}
'

4、Upserts

存在就更新，文件不存在就將upsert中的欄位插入：

例如：

POST test1/type1/1/_update
{
    "script" : {
        "source": "ctx._source.counter += params.count",
        "lang": "painless",
        "params" : {
            "count" : 4
        }
    },
    "upsert" : {
        "counter" : 1
    }
}

或

curl -XPOST 'localhost:9200/test/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "script" : {
        "source": "ctx._source.counter += params.count",
        "lang": "painless",
        "params" : {
            "count" : 4
        }
    },
    "upsert" : {
        "counter" : 1
    }
}
'

5、通過指令碼upsert

POST /sessions/session/dh3sgudg8gsrgl/_update
{
    "scripted_upsert":true,
    "script" : {
        "id": "my_web_session_summariser",
        "params" : {
            "pageViewEvent" : {
                "url":"foo.com/bar",
                "response":404,
                "time":"2014-01-01 12:32"
            }
        }
    },
    "upsert" : {}
}

或

curl -XPOST 'localhost:9200/sessions/session/dh3sgudg8gsrgl/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "scripted_upsert":true,
    "script" : {
        "id": "my_web_session_summariser",
        "params" : {
            "pageViewEvent" : {
                "url":"foo.com/bar",
                "response":404,
                "time":"2014-01-01 12:32"
            }
        }
    },
    "upsert" : {}
}
'

6、doc_as_upsert

POST test1/type1/3/_update
{
    "doc" : {
        "name" : "new_name"
    },
    "doc_as_upsert" : true
}

或

curl -XPOST 'localhost:9200/test/type1/1/_update?pretty' -H 'Content-Type: application/json' -d'
{
    "doc" : {
        "name" : "new_name"
    },
    "doc_as_upsert" : true
}
'

六、通過query API更新

1、最簡的query 更新api

POST twitter/_update_by_query?conflicts=proceed

或

curl -XPOST 'localhost:9200/twitter/_update_by_query?conflicts=proceed&pretty'

POST twitter/tweet/_update_by_query?conflicts=proceed

或

curl -XPOST 'localhost:9200/twitter/tweet/_update_by_query?conflicts=proceed&pretty'

2、通過query限制範圍

POST twitter/_update_by_query?conflicts=proceed
{
  "query": {
    "term": {
      "user": "kimchy"
    }
  }
}

3、指令碼

POST twitter/_update_by_query
{
  "script": {
    "source": "ctx._source.likes++",
    "lang": "painless"
  },
  "query": {
    "term": {
      "user": "kimchy"
    }
  }
}

或

curl -XPOST 'localhost:9200/twitter/_update_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "script": {
    "source": "ctx._source.likes++",
    "lang": "painless"
  },
  "query": {
    "term": {
      "user": "kimchy"
    }
  }
}
'

更新欄位：

POST twitter/_update_by_query
{
  "script": {
    "source": "ctx._source.likes++",
    "lang": "painless"
  },
  "query": {
    "term": {
      "user": "kimchy"
    }
  }
}

或

curl -XPOST 'localhost:9200/twitter/_update_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "script": {
    "source": "ctx._source.likes++",
    "lang": "painless"
  },
  "query": {
    "term": {
      "user": "kimchy"
    }
  }
}
'

4、多索引操作

POST twitter,blog/tweet,post/_update_by_query

或

curl -XPOST 'localhost:9200/twitter,blog/tweet,post/_update_by_query?pretty'

POST twitter/_update_by_query?routing=1
或

curl -XPOST 'localhost:9200/twitter/_update_by_query?routing=1&pretty'


POST twitter/_update_by_query?scroll_size=100
或

curl -XPOST 'localhost:9200/twitter/_update_by_query?scroll_size=100&pretty'

5、接待節點屬性

PUT _ingest/pipeline/set-foo
{
  "description" : "sets foo",
  "processors" : [ {
      "set" : {
        "field": "foo",
        "value": "bar"
      }
  } ]
}
POST twitter/_update_by_query?pipeline=set-foo

或

curl -XPUT 'localhost:9200/_ingest/pipeline/set-foo?pretty' -H 'Content-Type: application/json' -d'
{
  "description" : "sets foo",
  "processors" : [ {
      "set" : {
        "field": "foo",
        "value": "bar"
      }
  } ]
}
'
curl -XPOST 'localhost:9200/twitter/_update_by_query?pipeline=set-foo&pretty'


6、URL引數

pretty

refresh

wait_for_completion

wait_for_active_shards

timeout

7、任務API執行任務

獲取所有執行update-by-query的請求的狀態

GET _tasks?detailed=true&actions=*byquery

或

curl -XGET 'localhost:9200/_tasks?detailed=true&actions=*byquery&pretty'


通過task id查詢task

GET /_tasks/taskId:1

或

curl -XGET 'localhost:9200/_tasks/taskId:1?pretty'

8、取消任務API

POST _tasks/task_id:1/_cancel

或

curl -XPOST 'localhost:9200/_tasks/task_id:1/_cancel?pretty'

9、Rethrottling

POST _update_by_query/task_id:1/_rethrottle?requests_per_second=-1
‘或’

POST _update_by_query/task_id:1/_rethrottle?requests_per_second=-1

10、manual slicing

POST twitter/_update_by_query
{
  "slice": {
    "id": 0,
    "max": 2
  },
  "script": {
    "source": "ctx._source['extra'] = 'test'"
  }
}
POST twitter/_update_by_query
{
  "slice": {
    "id": 1,
    "max": 2
  },
  "script": {
    "source": "ctx._source['extra'] = 'test'"
  }
}

或

curl -XPOST 'localhost:9200/twitter/_update_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "slice": {
    "id": 0,
    "max": 2
  },
  "script": {
    "source": "ctx._source[\u0027extra\u0027] = \u0027test\u0027"
  }
}
'
curl -XPOST 'localhost:9200/twitter/_update_by_query?pretty' -H 'Content-Type: application/json' -d'
{
  "slice": {
    "id": 1,
    "max": 2
  },
  "script": {
    "source": "ctx._source[\u0027extra\u0027] = \u0027test\u0027"
  }
}
'

確認：

GET _refresh
POST twitter/_search?size=0&q=extra:test&filter_path=hits.total

或

curl -XGET 'localhost:9200/_refresh?pretty'
curl -XPOST 'localhost:9200/twitter/_search?size=0&q=extra:test&filter_path=hits.total&pretty'

11、自動分片

POST twitter/_update_by_query?refresh&slices=5
{
  "script": {
    "source": "ctx._source['extra'] = 'test'"
  }
}

或

curl -XPOST 'localhost:9200/twitter/_update_by_query?refresh&slices=5&pretty' -H 'Content-Type: application/json' -d'
{
  "script": {
    "source": "ctx._source[\u0027extra\u0027] = \u0027test\u0027"
  }
}
'

確認：

POST twitter/_search?size=0&q=extra:test&filter_path=hits.total
或

curl -XPOST 'localhost:9200/twitter/_search?size=0&q=extra:test&filter_path=hits.total&pretty'

12、pick up新屬性

PUT test2
{
  "mappings": {
    "test": {
      "dynamic": false,   //只儲存在source中不進行索引
      "properties": {
        "text": {"type": "text"}
      }
    }
  }
}

POST test2/test?refresh
{
  "text": "words words",
  "flag": "bar"
}
POST test2/test?refresh
{
  "text": "words words",
  "flag": "foo"
}
PUT test2/_mapping/test    //新增新的flag域
{
  "properties": {
    "text": {"type": "text"},
    "flag": {"type": "text", "analyzer": "keyword"}
  }
}

或

curl -XPUT 'localhost:9200/test?pretty' -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "test": {
      "dynamic": false,   
      "properties": {
        "text": {"type": "text"}
      }
    }
  }
}
'
curl -XPOST 'localhost:9200/test/test?refresh&pretty' -H 'Content-Type: application/json' -d'
{
  "text": "words words",
  "flag": "bar"
}
'
curl -XPOST 'localhost:9200/test/test?refresh&pretty' -H 'Content-Type: application/json' -d'
{
  "text": "words words",
  "flag": "foo"
}
'
curl -XPUT 'localhost:9200/test/_mapping/test?pretty' -H 'Content-Type: application/json' -d'
{
  "properties": {
    "text": {"type": "text"},
    "flag": {"type": "text", "analyzer": "keyword"}
  }
}
'

搜尋無結果：

POST test2/_search?filter_path=hits.total
{
  "query": {
    "match": {
      "flag": "foo"
    }
  }
}

或

curl -XPOST 'localhost:9200/test2/_search?filter_path=hits.total&pretty' -H 'Content-Type: application/json' -d'
{
  "query": {
    "match": {
      "flag": "foo"
    }
  }
}
'

獲取新的對映：

POST test2/_update_by_query?refresh&conflicts=proceed
POST test2、_search?filter_path=hits.total
{
  "query": {
    "match": {
      "flag": "foo"
    }
  }
}

七、Multi Get API

1、獲取多文件API

GET /_mget
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1"
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2"
        }
    ]

}

或

curl -XGET 'localhost:9200/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1"
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2"
        }
    ]
}
'


GET /test/_mget
{
    "docs" : [
        {
            "_type" : "type",
            "_id" : "1"
        },
        {
            "_type" : "type",
            "_id" : "2"
        }
    ]
}

或

curl -XGET 'localhost:9200/test/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_type" : "type",
            "_id" : "1"
        },
        {
            "_type" : "type",
            "_id" : "2"
        }
    ]
}
'

GET /test/type/_mget
{
    "docs" : [
        {
            "_id" : "1"
        },
        {
            "_id" : "2"
        }
    ]
}

或

curl -XGET 'localhost:9200/test/type/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_id" : "1"
        },
        {
            "_id" : "2"
        }
    ]
}
'

GET /test/type/_mget
{
    "ids" : ["1", "2"]
}
或

curl -XGET 'localhost:9200/test/type/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "ids" : ["1", "2"]
}
'

2、可選型別

GET /test/_mget/
{
  "docs" : [
        {
            "_type":"typeA",
            "_id" : "1"
        },
        {
            "_type":"typeB",
            "_id" : "1"
        }
    ]
}

或

curl -XGET 'localhost:9200/test/_mget/?pretty' -H 'Content-Type: application/json' -d'
{
  "docs" : [
        {
            "_type":"typeA",
            "_id" : "1"
        },
        {
            "_type":"typeB",
            "_id" : "1"
        }
    ]
}
'

3、源過濾

GET /_mget
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1",
            "_source" : false
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2",
            "_source" : ["field3", "field4"]
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "3",
            "_source" : {
                "include": ["user"],
                "exclude": ["user.location"]
            }
        }
    ]
}

或

curl -XGET 'localhost:9200/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1",
            "_source" : false
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2",
            "_source" : ["field3", "field4"]
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "3",
            "_source" : {
                "include": ["user"],
                "exclude": ["user.location"]
            }
        }
    ]
}
'

4、域

GET /_mget
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1",
            "stored_fields" : ["field1", "field2"]
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2",
            "stored_fields" : ["field3", "field4"]
        }
    ]
}

或

curl -XGET 'localhost:9200/_mget?pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1",
            "stored_fields" : ["field1", "field2"]
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2",
            "stored_fields" : ["field3", "field4"]
        }
    ]
}
'

或

GET /_mget
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1",
            "stored_fields" : ["field1", "field2"]
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2",
            "stored_fields" : ["field3", "field4"]
        }
    ]
}

或

curl -XGET 'localhost:9200/test/type/_mget?stored_fields=field1,field2&pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_id" : "1"
        },
        {
            "_id" : "2",
            "stored_fields" : ["field3", "field4"]
        }
    ]
}
'

5、路由

GET /_mget?routing=key1
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1",
            "_routing" : "key2"
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2"
        }
    ]
}

或

curl -XGET 'localhost:9200/_mget?routing=key1&pretty' -H 'Content-Type: application/json' -d'
{
    "docs" : [
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "1",
            "_routing" : "key2"
        },
        {
            "_index" : "test",
            "_type" : "type",
            "_id" : "2"
        }
    ]
}
'

八、批量API

1、批量操作

lxq@slave-01:~/elastic-search/elasticsearch-6.0.0/data$ cat requests
{ "index" : { "_index" : "test5", "_type" : "type1", "_id" : "1" } }
{ "field1" : "value1" }
lxq@slave-01:~/elastic-search/elasticsearch-6.0.0/data$ sh curl.sh
{"took":131,"errors":false,"items":[{"index":{"_index":"test5","_type":"type1","_id":"1","_version":1,"result":"created","_shards":{"total":2,"successful":1,"failed":0},"_seq_no":0,"_primary_term":1,"status":201}}]}
curl.sh: 2: curl.sh: {took:7,: not found
lxq@slave-01:~/elastic-search/elasticsearch-6.0.0/data$ cat curl.sh
curl -u "elastic":"elastic" -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests";echo
{"took":7, "errors": false, "items":[{"index":{"_index":"test5","_type":"type1","_id":"1","_version":1,"result":"created","forced_refresh":false}}]}

POST _bulk
{ "index" : { "_index" : "test4", "_type" : "type1", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test4", "_type" : "type1", "_id" : "2" } }
{ "create" : { "_index" : "test4", "_type" : "type1", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "test4"} }
{ "doc" : {"field2" : "value2"} }

或

curl -XPOST 'localhost:9200/_bulk?pretty' -H 'Content-Type: application/json' -d'
{ "index" : { "_index" : "test4", "_type" : "type1", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test4", "_type" : "type1", "_id" : "2" } }
{ "create" : { "_index" : "test4", "_type" : "type1", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "test4"} }
{ "doc" : {"field2" : "value2"} }
'

2、版本控制

3、路由

4、等待啟用分片數

5、重新整理

6、更新

支援選項：doc(partial document),upsert,doc_as_upsert,script,params(for script),

          lang(for script) and_source

POST _bulk
{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_type" : "type1", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_type" : "type1", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}

或

curl -XPOST 'localhost:9200/_bulk?pretty' -H 'Content-Type: application/json' -d'
{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "script" : { "source": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_type" : "type1", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_type" : "type1", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}'

九、重新索引API

1、本地拷貝

目的：將文件從一個索引拷貝到另一個索引

POST _reindex
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter"
  }

}

或

curl -XPOST 'localhost:9200/_reindex?pretty' -H 'Content-Type: application/json' -d'
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter"
  }
}
'

直接覆蓋目標索引的同type和id的文件

POST _reindex
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter",
    "version_type": "internal"
  }
}

只覆蓋低版本的目標文件

POST _reindex
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter",
    "version_type": "external"
  }
}

只補充新文件

POST _reindex
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter",
    "op_type": "create"
  }
}

統計版本衝突次數

POST _reindex
{
  "conflicts": "proceed",
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter",
    "op_type": "create"
  }
}

拷貝query匹配的文件

POST _reindex
{
  "source": {
    "index": "twitter",
    "type": "tweet",
    "query": {
      "term": {
        "user": "kimchy"
      }
    }
  },
  "dest": {
    "index": "new_twitter"
  }
}

多索引拷貝

POST _reindex
{
  "source": {
    "index": ["twitter", "blog"],
    "type": ["tweet", "post"]
  },
  "dest": {
    "index": "all_together"
  }
}

文件數限定拷貝

POST _reindex
{
  "size": 1,
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter"
  }
}

源索引域限定拷貝

POST _reindex
{
  "source": {
    "index": "twitter",
    "_source": ["user", "tweet"]
  },
  "dest": {
    "index": "new_twitter"
  }
}

指令碼修改文件元資訊拷貝

POST _reindex
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter",
    "version_type": "external"
  },
  "script": {
    "source": "if (ctx._source.foo == 'bar') {ctx._version++; ctx._source.remove('foo')}",
    "lang": "painless"
  }
}

路由設定拷貝

POST _reindex
{
  "source": {
    "index": "source",
    "query": {
      "match": {
        "company": "cat"
      }
    }
  },
  "dest": {
    "index": "dest",
    "routing": "=cat"
  }
}

設定batches大小

POST _reindex
{
  "source": {
    "index": "source",
    "size": 100
  },
  "dest": {
    "index": "dest",
    "routing": "=cat"
  }
}

指定pipeline拷貝

POST _reindex
{
  "source": {
    "index": "source"
  },
  "dest": {
    "index": "dest",
    "pipeline": "some_ingest_pipeline"
  }
}

2、遠端拷貝

POST _reindex
{
  "source": {
    "remote": {
      "host": "http://otherhost:9200",
      "username": "user",
      "password": "pass"
    },
    "index": "source",
    "query": {
      "match": {
        "test": "data"
      }
    }
  },
  "dest": {
    "index": "dest"
  }
}

注意：如果需要使用者名稱和密碼，那麼host要用https

elasticsearch.yml要將遠端的ip配置到reindex.remote.whitelist

例如：

reindex.remote.whitelist： otherhost：9200, otherhost1:9200

如果遠端文件大，要設定小的batch

POST _reindex
{
  "source": {
    "remote": {
      "host": "http://otherhost:9200"
    },
    "index": "source",
    "size": 10,
    "query": {
      "match": {
        "test": "data"
      }
    }
  },
  "dest": {
    "index": "dest"
  }
}

設定超時時間

POST _reindex
{
  "source": {
    "remote": {
      "host": "http://otherhost:9200",
      "socket_timeout": "1m",
      "connect_timeout": "10s"
    },
    "index": "source",
    "query": {
      "match": {
        "test": "data"
      }
    }
  },
  "dest": {
    "index": "dest"
  }
}

3、URL引數

pretty，

refresh,

wait_for_completion,

wait_for_active_shards,

timeout,

requests_per_second.

4、Task API

獲取reindex的狀態

GET _tasks?detailed=true&actions=*reindex

或

curl -XGET 'localhost:9200/_tasks?detailed=true&actions=*reindex&pretty'

獲取單個任務的狀態

GET /_tasks/taskId:1

或

curl -XGET 'localhost:9200/_tasks/taskId:1?pretty'

取消任務

POST _tasks/task_id:1/_cancel

或

curl -XPOST 'localhost:9200/_tasks/task_id:1/_cancel?pretty'

5、rethrottling

POST _reindex/task_id:1/_rethrottle?requests_per_second=-1

或

curl -XPOST 'localhost:9200/_reindex/task_id:1/_rethrottle?requests_per_second=-1&pretty'

6、reindex並修改域名稱

POST reindex/test/1?refresh
{
  "text": "words words",
  "flag": "foo"
}

或

curl -XPOST 'localhost:9200/reindex/test/1?refresh&pretty' -H 'Content-Type: application/json' -d'
{
  "text": "words words",
  "flag": "foo"
}
'

POST _reindex
{
  "source": {
    "index": "reindex"
  },
  "dest": {
    "index": "reindex2"
  },
  "script": {
    "source": "ctx._source.tag = ctx._source.remove(\"flag\")"
  }
}

或

curl -XPOST 'localhost:9200/_reindex?pretty' -H 'Content-Type: application/json' -d'
{
  "source": {
    "index": "reindex"
  },
  "dest": {
    "index": "reindex2"
  },
  "script": {
    "source": "ctx._source.tag = ctx._source.remove(\"flag\")"
  }
}
'

7、手工分片

POST _reindex
{
  "source": {
    "index": "twitter",
    "slice": {
      "id": 0,
      "max": 2
    }
  },
  "dest": {
    "index": "new_twitter"
  }
}
POST _reindex
{
  "source": {
    "index": "twitter",
    "slice": {
      "id": 1,
      "max": 2
    }
  },
  "dest": {
    "index": "new_twitter"
  }
}

或

curl -XPOST 'localhost:9200/_reindex?pretty' -H 'Content-Type: application/json' -d'
{
  "source": {
    "index": "twitter",
    "slice": {
      "id": 0,
      "max": 2
    }
  },
  "dest": {
    "index": "new_twitter"
  }
}
'
curl -XPOST 'localhost:9200/_reindex?pretty' -H 'Content-Type: application/json' -d'
{
  "source": {
    "index": "twitter",
    "slice": {
      "id": 1,
      "max": 2
    }
  },
  "dest": {
    "index": "new_twitter"
  }
}
'


驗證

GET _refresh
POST new_twitter/_search?size=0&filter_path=hits.total

或

curl -XGET 'localhost:9200/_refresh?pretty'
curl -XPOST 'localhost:9200/new_twitter/_search?size=0&filter_path=hits.total&pretty'


8、自動切片

POST _reindex?slices=5&refresh
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter"
  }
}

或

curl -XPOST 'localhost:9200/_reindex?slices=5&refresh&pretty' -H 'Content-Type: application/json' -d'
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter"
  }
}
'
驗證

POST new_twitter/_search?size=0&filter_path=hits.total

或

curl -XPOST 'localhost:9200/new_twitter/_search?size=0&filter_path=hits.total&pretty'

9、picking the number of slices

10、reindex daily indices

11、extracting a random subset of an index

十、詞向量

GET /bank/account/1/_termvectors

或

curl -XGET 'localhost:9200/bank/account/1/_termvectors?pretty'

GET /bank/account/1/_termvectors?fields=address

或

curl -XGET 'localhost:9200/bank/account/1/_termvectors?fields=address&pretty'

十一、多詞向量API

POST /_mtermvectors
{
   "docs": [
      {
         "_index": "twitter",
         "_type": "tweet",
         "_id": "2",
         "term_statistics": true
      },
      {
         "_index": "twitter",
         "_type": "tweet",
         "_id": "1",
         "fields": [
            "message"
         ]
      }
   ]
}

或

curl -XPOST 'localhost:9200/_mtermvectors?pretty' -H 'Content-Type: application/json' -d'
{
   "docs": [
      {
         "_index": "twitter",
         "_type": "tweet",
         "_id": "2",
         "term_statistics": true
      },
      {
         "_index": "twitter",
         "_type": "tweet",
         "_id": "1",
         "fields": [
            "message"
         ]
      }
   ]
}
'

POST /twitter/_mtermvectors
{
   "docs": [
      {
         "_type": "tweet",
         "_id": "2",
         "fields": [
            "message"
         ],
         "term_statistics": true
      },
      {
         "_type": "tweet",
         "_id": "1"
      }
   ]
}


相關推薦

Elastic search 系統學習之四： 文件API

一、Index API 1、插入文件 curl -XPUT 'localhost:9200/twitter/tweet/1?pretty' -H 'Content-Type: application/json' -d' {     "user" : "kimchy",   

Linux學習之三：文件夾系統的結構和相對（絕對）路徑

sharp 二進制 沒有 數據 csharp pan 用戶 ont 臨時 理解每個目錄的作用 bin 　　二進制文件 boot　　 系統的啟動文件、內核 dev　　 設備文件 etc　　 配置文件 home　　用戶的家目錄 lib　　　 鏈接庫文件　 l

Python學習之路：文件操作

append 句柄 enc pre light logs utf highlight 追加內容 文件基本操作： 打開、讀取、追加、關閉 #data = open("yesterday",encoding="utf-8").read() #打開並讀取文件 f=open("y

Python學習之路：文件操作之增刪改查

打印 odin day 打開 aps 之前 編碼 數據 adl f = open("yesterday","r",encoding="utf-8") #print(f.read()) #for i in range(5): # print(f.readline())

seL4微核心學習之四：系統呼叫

seL4系統呼叫主要有以下八個： seL4 Send()： 通過已被命名的cap傳遞訊息，然後允許程式繼續，如果呼叫這個cap的是endpoint，且沒有receiver接收訊息，sender將會

Linux系統學習之 一：新手必須掌握的Linux命令1

網絡設備 agents daemon gen boot 3.0 media license bubuko 2018-10-03 16:04:12 一、常用系統工作命令 1、wget 命令 作用：用於在終端中下載網絡文件。 格式：wget [參數] 下載地址 參數及作用

Docker學習之四：使用docker安裝mysql，碰到了一個啟動的坑

Docker學習之四：使用docker安裝mysql，碰到了一個啟動的坑 第一步，從docker中拉取MySQL映象 $ sudo docker pull mysql $ sudo docker images 第二步，建立並啟動一個MySQL容器 $ sudo doc

C++11併發學習之四：執行緒同步（續）

有時候，在第一個執行緒完成前，可能需要等待另一個執行緒執行完成。C++標準庫提供了一些工具可用於這種同步操作，形式上表現為條件變數(condition variable)和期望(future)。 一.條件變數(condition variable) C++標準庫對條件變數有兩套實現：std::c

USB開裝置開發學習之四：USB傳輸之控制傳輸

原文：https://blog.csdn.net/go_str/article/details/80782229  前言     USB控制傳輸分為以下四種： 批量傳輸：批量傳輸一般用於批量的和非實時的資料傳輸，通俗的來說就是用於資料量大但對時間

六天搞懂“深度學習”之四：基於神經網路的分類

分類用於確定資料所歸屬的類別，而回歸是根據已知的資料進行推斷或估計某個未知量，比如根據年齡和教育水平進行收入預測分析。分類的典型應用是垃圾郵件過濾和字元識別。 雖然神經網路適用於分類和迴歸，但卻很少用於迴歸。這不是因為它的效能不好，而是因為大多數迴歸問題可以用更簡單的模型來解決。（迴歸問

C++系統學習之九：順序容器

　　元素在順序容器中的順序與其加入容器時的位置相對應。關聯容器中元素的位置由元素相關聯的關鍵字值決定。所有容器類都共享公共的介面，不同容器按不同方式對其進行擴充套件。 　　一個容器就是一些特定型別物件的集合。順序容器為程式設計師提供了控制元素儲存和訪問順序的能力。 1. 順序容器概述 容器的兩種效能： 向容

VVC程式碼 BMS 幀內預測學習之四：xFillReferenceSamples()

xFillReferenceSamples()函式是參考畫素的獲取過程。 主要步驟： 1、分析臨近的畫素是否可獲取 2、進行參考樣本的填充：若臨近的畫素全部可獲取，則賦值；全部不可獲取，則賦預設值；若部分可獲取，則對可獲取的賦對應的值，不可獲取的用預設值填充。

MyBatis學習之四：MyBatis配置檔案

在定義sqlSessionFactory時需要指定MyBatis主配置檔案： Xml程式碼   <bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean"> 

NLTK學習之四：文字資訊抽取

1 資訊抽取 從資料庫中抽取資訊是容易的，但對於從自然文字中抽取資訊則不那麼直觀。通常資訊抽取的流程如下圖： 它開始於分句，分詞。接下來進行詞性標註，識別其中的命名實體，最後使用關係識別搜尋相近實體間的可能的關係。 2 分塊 分塊是實體識別(NE

WebRTC學習之四：最簡單的語音聊天

VoiceEngine中與最簡單語音聊天相關的標頭檔案有五個，如下表所示： 標頭檔案 包含的類 說明 voe_base.h VoiceEngineObserver VoiceEngine VoEBase 1.預設使用G.711通過RTP進行全雙工的VoI

深度學習之四：卷積神經網路基礎

計算機視覺在深度學習的幫助下取得了令人驚歎的進展，其中發揮重要作用的是卷積神經網路。本節總結了卷積神經的原理與實現方法。 1 卷積神經網路 1.1 計算機視覺與深度學習 計算機視覺要解決的問題是如何讓機器理解現實世界的現象。目前主要處理的問題如影象

tomcat學習之四：tomcat的類載入機制

        tomcat的類載入機制遵循了java類載入機制中經典的雙親委派模型。所以要了解tomcat的類載入機制需要先了解雙親委派模型。         在程式中用到的類需要由類載入器將類的class檔案載入到記憶體中，然後經由JVM驗證、解析、初始化後才能使用，

強化學習之四：基於策略的Agents (Policy-based Agents)

本文是對Arthur Juliani在Medium平臺釋出的強化學習系列教程的個人中文翻譯，該翻譯是基於個人分享知識的目的進行的，歡迎交流！（This article is my personal translation for the tutorial wri

Echarts學習之四：series-pie餅圖

mytextStyle={ color:"#333", //文字顏色 fontStyle:"normal", //italic斜體 oblique傾斜 fontWeight:"normal",

jackson學習之四：WRAP_ROOT_VALUE（root物件）

### 歡迎訪問我的GitHub [https://github.com/zq2599/blog_demos](https://github.com/zq2599/blog_demos) 內容：所有原創文章分類彙總及配套原始碼，涉及Java、Docker、Kubernetes、DevOPS等； ### 系