1. 程式人生 > >通過http.client解析url返回的數據時為什麽中文變成了unicode碼

通過http.client解析url返回的數據時為什麽中文變成了unicode碼

eval 數據 pen class except port nts sun ring

今天在解析json數據的時候得到了一堆這樣的數據:{"errNum":0,"errMsg":"success","retData":[{"title":"\u6536\u5e9f\u54c1\u5927\u53d4\u521a\u4e0a\u53f0\uff0c\u5c31\u60e8\u906d\u8bc4\u59d4\u706d\u706f\uff0c\u4f46\u63a5\u4e0b\u6765\u5168\u573a\u90fd\u9707\u60ca\u4e86\uff01","url":"http:\/\/toutiao.com\/group\/6263036756505920002\/","abstract":"\u8ba2\u9605\u6211\u83b7\u53d6\u66f4\u591a\u7cbe\u5f69\u5185\u5bb9\uff01","image_url":"http:\/\/p1.pstatp.com\/list\/2f90009a31a7ee8bb15"}]}

這是因為,為了更好的傳輸中文,json進行了Unicode編碼。

這樣一來,我們在解析json之前,就得要先將json數據中的Unicode編碼轉換為我們使用的中文;

一:http請求數據返回json中string字段包含unicode的轉碼

[java] view plain copy
  1. public static String decodeUnicode(String theString) {
  2. char aChar;
  3. int len = theString.length();
  4. StringBuffer outBuffer = new StringBuffer(len);
  5. for (int x = 0; x < len;) {
  6. aChar = theString.charAt(x++);
  7. if (aChar == ‘\\‘) {
  8. aChar = theString.charAt(x++);
  9. if (aChar == ‘u‘) {
  10. // Read the xxxx
  11. int value = 0;
  12. for (int i = 0; i < 4; i++) {
  13. aChar = theString.charAt(x++);
  14. switch (aChar) {
  15. case ‘0‘:
  16. case ‘1‘:
  17. case ‘2‘:
  18. case ‘3‘:
  19. case ‘4‘:
  20. case ‘5‘:
  21. case ‘6‘:
  22. case ‘7‘:
  23. case ‘8‘:
  24. case ‘9‘:
  25. value = (value << 4) + aChar - ‘0‘;
  26. break;
  27. case ‘a‘:
  28. case ‘b‘:
  29. case ‘c‘:
  30. case ‘d‘:
  31. case ‘e‘:
  32. case ‘f‘:
  33. value = (value << 4) + 10 + aChar - ‘a‘;
  34. break;
  35. case ‘A‘:
  36. case ‘B‘:
  37. case ‘C‘:
  38. case ‘D‘:
  39. case ‘E‘:
  40. case ‘F‘:
  41. value = (value << 4) + 10 + aChar - ‘A‘;
  42. break;
  43. default:
  44. throw new IllegalArgumentException(
  45. "Malformed \\uxxxx encoding.");
  46. }
  47. }
  48. outBuffer.append((char) value);
  49. } else {
  50. if (aChar == ‘t‘)
  51. aChar = ‘\t‘;
  52. else if (aChar == ‘r‘)
  53. aChar = ‘\r‘;
  54. else if (aChar == ‘n‘)
  55. aChar = ‘\n‘;
  56. else if (aChar == ‘f‘)
  57. aChar = ‘\f‘;
  58. outBuffer.append(aChar);
  59. }
  60. } else
  61. outBuffer.append(aChar);
  62. }
  63. return outBuffer.toString();
  64. }

二、普通string含有unicode轉碼方法

[java] view plain copy
  1. public static String reEncoding(String text, String newEncoding) {
  2. String str = null;
  3. try {
  4. str = new String(text.getBytes(), newEncoding);
  5. } catch (UnsupportedEncodingException e) {
  6. log.error("不支持的字符編碼" + newEncoding);
  7. throw new RuntimeException(e);
  8. }
  9. return str;
  10. }

三、說一下比較奇怪的方案,測試中無意發現的,暫時沒弄明白原理(有明白原理的大神,請告知一聲,謝謝)

我用HttpClent的post方式獲取的json數據,得到的是帶Unicode碼的數據,需要轉換成中文才行,但是轉換的時間感覺有點長,就用HttpURLConnection的get方式又試了一下,在不轉碼的情況下,經過gson解析後,竟然神奇的自動轉換成了中文:

技術分享

簡直是太神奇了,而且需要的時間相對於HttpClient的post請求方式的請求和處理時間更短,所以,果斷換用HttpURLConnection的get方式了

技術分享


①現在先貼一下HttpURLConnection的get的方式:

[java] view plain copy
  1. @Test
  2. public void test() {
  3. try {
  4. long start = System.currentTimeMillis();
  5. URL url = new URL("http://apis.baidu.com/songshuxiansheng/news/news");
  6. HttpURLConnection connection = (HttpURLConnection) url.openConnection();
  7. connection.addRequestProperty("apikey","0fc807e45a37ce264f45d169646f4a9e" );
  8. String dataString = new String(GsonTools.IsToByte(connection.getInputStream()),"utf-8");
  9. HeadlineJson newsJson = GsonTools.getObjectData(dataString, HeadlineJson.class);
  10. List<Headline>list = newsJson.getRetData();
  11. System.out.println(list.toString());
  12. long end = System.currentTimeMillis();
  13. System.out.println("timeGap:"+(end-start));
  14. } catch (Exception e) {
  15. // TODO Auto-generated catch block
  16. e.printStackTrace();
  17. }
  18. }


調用的GsonTools的方法:(之前的博文中有寫到過)

[java] view plain copy
  1. public static <T> T getObjectData(String jsonString, Class<T> type) {
  2. T t = null;
  3. try {
  4. Gson gson = new Gson();
  5. t = gson.fromJson(jsonString, type);
  6. } catch (JsonSyntaxException e) {
  7. // TODO Auto-generated catch block
  8. e.printStackTrace();
  9. }
  10. return t;
  11. }


②然後貼一下HttpClient的post方式:

[java] view plain copy
  1. @Test
  2. public void TestHeadLine() {
  3. long start = System.currentTimeMillis();
  4. List<NameValuePair> params = new ArrayList<NameValuePair>();
  5. String url = "http://apis.baidu.com/songshuxiansheng/news/news";
  6. String jsonString = HttpUtils.getBaiDuString2(url, params);
  7. HeadlineJson lineJson = GsonTools.getObjectData(jsonString, HeadlineJson.class);
  8. System.out.println(lineJson.toString());
  9. long end = System.currentTimeMillis();
  10. System.out.println("timeGap:"+(end-start));
  11. }


調用的HttpUtils的方法:

[java] view plain copy
  1. public static String getBaiDuString(String url,List<NameValuePair> params) {
  2. String serverDataString = null;
  3. HttpPost post = new HttpPost(url);
  4. try {
  5. post.setEntity(new UrlEncodedFormEntity(params, HTTP.UTF_8));
  6. post.addHeader("apikey", UrlUtils.BAIDU_API_KEY);
  7. HttpClient client = new DefaultHttpClient();
  8. HttpResponse response = client.execute(post);
  9. int code = response.getStatusLine().getStatusCode();
  10. System.out.println("StatusCode:" + code);
  11. if (code == 200) {
  12. serverDataString = decodeUnicode(EntityUtils.toString(response.getEntity()));
  13. // serverDataString = EntityUtils.toString(response.getEntity());
  14. System.out.println("接收字符串數據成功\nServerData:"+serverDataString);
  15. }
  16. } catch (Exception e) {
  17. // TODO Auto-generated catch block
  18. e.printStackTrace();
  19. }
  20. return serverDataString;
  21. }


③調用的HttpClient的get方式

[java] view plain copy
  1. public static String getBaiDuString2(String url,List<NameValuePair> params) {
  2. String serverDataString = null;
  3. HttpGet get = new HttpGet(url);
  4. try {get.addHeader("apikey", UrlUtils.BAIDU_API_KEY);
  5. HttpClient client = new DefaultHttpClient();
  6. HttpResponse response = client.execute(get);
  7. int code = response.getStatusLine().getStatusCode();
  8. System.out.println("StatusCode:" + code);
  9. if (code == 200) {
  10. // serverDataString = decodeUnicode(EntityUtils.toString(response.getEntity()));
  11. serverDataString = EntityUtils.toString(response.getEntity());
  12. System.out.println("接收字符串數據成功\nServerData:"+serverDataString);
  13. }
  14. } catch (Exception e) {
  15. // TODO Auto-generated catch block
  16. e.printStackTrace();
  17. }
  18. return serverDataString;
  19. }


谷歌提供的HttpClient的通信和HttpURLConnection網絡通信的時間間隔我也做了比較,明顯,HttpURLConnection的請求時間更短,所以果斷使用HttpURLConnection的方式

四、java中本身就提供了對Unicode 的url進行解碼的方法了:

[java] view plain copy
    1. System.out.println(URLDecoder.decode("\u82f9\u679c", "utf-8"));

通過http.client解析url返回的數據時為什麽中文變成了unicode碼