技術標籤：Hive

假設我們想要用Flink對hive進行讀寫,

並且其中一段程式碼如下:

//      如果hive中已經存在了相應的表，則這段程式碼省略
		String hiveSql = "CREATE external TABLE fs_table (\n" +
		                 "  user_id STRING,\n" +
		                 "  order_amount DOUBLE" +
		                 ") partitioned by (dt string,h string,m string) " +
		                 "stored as ORC " +
		                 "TBLPROPERTIES (\n" +
		                 "  'partition.time-extractor.timestamp-pattern'='$dt $h:$m:00',\n" +
		                 "  'sink.partition-commit.delay'='0s',\n" +
		                 "  'sink.partition-commit.trigger'='partition-time',\n" +
		                 "  'sink.partition-commit.policy.kind'='metastore'" +
		                 ")";
		tEnv.executeSql(hiveSql);
 

顯然這個程式碼和原有的flink 程式碼耦合性太強,

我們希望抽離出來,在hive的客戶端來執行.

為了除錯列印上面的這個語句顯然用java太麻煩了.

用python程式碼printsql.py如下(上面的整個程式碼直接複製下來,然後+改成\即可):

a="CREATE external TABLE fs_table (\n" \
                         "  user_id STRING,\n" \
                         "  order_amount DOUBLE" \
                         ") partitioned by (dt string,h string,m string) " \
                         "stored as ORC " \
                         "TBLPROPERTIES (\n" \
                         "  'partition.time-extractor.timestamp-pattern'='$dt $h:$m:00',\n" \
                         "  'sink.partition-commit.delay'='0s',\n" \
                         "  'sink.partition-commit.trigger'='partition-time',\n" \
                         "  'sink.partition-commit.policy.kind'='metastore'" \
                         ")"
print(a)
 

python printsql.py後得到sql語句,把該sql語句在hive中執行如下:

0: jdbc:hive2://Desktop:10000> CREATE external TABLE fs_table (
. . . . . . . . . . . . . . .> user_id STRING,
. . . . . . . . . . . . . . .> order_amount DOUBLE) partitioned by (dt string,h string,m string) stored as ORC TBLPROPERTIES (
. . . . . . . . . . . . . . .> 'partition.time-extractor.timestamp-pattern'='$dt $h:$m:00',

. . . . . . . . . . . . . . .> 'sink.partition-commit.delay'='0s',
. . . . . . . . . . . . . . .> 'sink.partition-commit.trigger'='partition-time',
. . . . . . . . . . . . . . .> 'sink.partition-commit.policy.kind'='metastore')
. . . . . . . . . . . . . . .>
. . . . . . . . . . . . . . .> ;
No rows affected (4.864 seconds)

建表順利完成