java 操作ElasticSearch進行搜尋 遇到的問題記錄
由於公司需求,要我通過es去帥選資料,公司也沒人會,只好自己慢慢摸索,期間碰到不少坑,現在記錄下來:
1:分組查詢(比如按欄位:orderId)時報:
java.lang.IllegalArgumentException: Fielddata is disabled on text fields by default. Set fielddata=true on [orderid] in order to load fielddata in memory by uninverting the inverted index.
這是因為5.x+版本預設關閉了文字欄位,防止佔用過多空間,需開啟fielddata:
{
"properties": {
"interests": {
"type": "text",
"fielddata": true //不建議
}
}
}
現在我們是用的Java操作,只需在你需要分組的欄位後面加上 .keyword 就可以了。如
.addAggregation(AggregationBuilders.terms("orderId_Term").field("orderid.keyword")
2:es預設只顯示10條記錄的問題
我們查找出的資料預設只顯示10條,要想顯示所有的呢?1·可以設定size,不靈活。2·通過scroll,即遊標。程式碼如下:
SearchResponse response = client.prepareSearch("responseqrcodes") .setScroll(new TimeValue(60000)).get();
也就是說在在1min內保持scroll開啟狀態,setScroll("1m")引數也可這樣寫,即1分鐘。最後獲取的時候就可以:
do {
for (SearchHit hit : scrollResp.getHits().getHits()) {
//Handle the hit...
}
response = client.prepareSearchScroll(scrollResp.getScrollId()).setScroll(new TimeValue(60000)).execute().actionGet();
} while(scrollResp.getHits().getHits().length != 0); //所有資料讀取完畢則退出迴圈。
---------------------------------------------------------------------------------------------------------------------------
下面貼一段完整程式碼:
private static void executeSearch(TransportClient client) { SearchResponse response = client.prepareSearch("index") .setScroll(new TimeValue(60000)) .setTypes("type") .setQuery(QueryBuilders.matchAllQuery()) .setPostFilter(QueryBuilders.rangeQuery("money").gt(1).lt(100)) //金額大於1小於100 // .setQuery(QueryBuilders.matchQuery("id", "1")) //id等於1 .addAggregation(AggregationBuilders.terms("c_term").field("country.keyword") //按國家分組 // .order(Terms.Order.aggregation("moneySum", false)) //按總金額降序排序 true:升序 .subAggregation(AggregationBuilders.sum("moneySum").field("money")) //對應的總金額 .subAggregation(AggregationBuilders.avg("moneyAvg").field("money")) //平均金額 ) // .setFrom(0).setSize(20) //分頁,20條 .addSort("id", SortOrder.ASC) //按id升序排序 .get(); Terms terms = response.getAggregations().get("c_term"); List<Terms.Bucket> buckets = terms.getBuckets(); for (Terms.Bucket bucket : buckets) { Sum salarySum = bucket.getAggregations().get("moneySum"); Avg moneyAvg = bucket.getAggregations().get("moneyAvg"); System.out.println(bucket.getKey() + "----" + salarySum.getValue() + "=======" + moneyAvg.getValue()); System.out.println(bucket.getKey() + "--->" + bucket.getDocCount()); } SearchHits hits = response.getHits(); long totalHits = hits.getTotalHits(); System.out.println("總數:" + totalHits); SearchHit[] searchHits = hits.hits(); do { for (SearchHit searchHit : searchHits) { System.out.println(searchHit.getSourceAsString()); } response = client.prepareSearchScroll(response.getScrollId()).setScroll(new TimeValue(60000)).execute().actionGet(); } while (response.getHits().getHits().length != 0); //零點選標記滾動結束和while迴圈。 }