Ambari 安裝 HDP(Centos7,HDP-2.6.3.0-235,HDP-UTILS-1.1.0.21,ambari-2.6.0.0)
阿新 • • 發佈：2018-12-14
centos1：192.168.1.4 2G 2核 60G
centos2：192.168.1.5 1G 1核 60G
centos3：192.168.1.6 1G 1核 60G
(1) SSH免密登陸
hadoop.001
[[email protected] etc]# ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Created directory '/root/.ssh'.
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:fquK7a5XfTlDvIv3DMsHITcM1KumYf6jTnk33vqJE84 (2) hosts配置（所有機器必須執行）
[[email protected] ~]# find -name hosts
[[email protected] /]# cd etc
[[email protected] etc]# vi hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.1.4 hadoop.001
192.168.1.5 hadoop.002
192.168.1.6 hadoop.003
//重啟所有機器
(3) 防火牆管理 （所有機器必須執行）
//關閉防火牆 需要在所有機器上進行操作
[[email protected] etc]# systemctl disable firewalld
[[email protected] etc]# systemctl stop firewalld
(4) 安裝JDK （所有機器必須執行）
//安裝jdk
[[email protected] ~]# tar -zxvf jdk-8u181-linux-x64.tar.gz -C /opt/java/
[[email protected] ~]# vi /etc/profile
export JAVA_HOME=/opt/java/jdk1.8.0_181
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
PATH=$PATH:$HOME/bin:$JAVA_HOME/bin
source /etc/profile
(5) 構建本地yum源(master 主機安裝)
//製作本地映象源
[[email protected] etc]# yum install httpd
[[email protected] etc]# firewall-cmd --add-service=http
[[email protected] etc]# firewall-cmd --permanent --add-service=http
新增 Apache 服務到系統層使其隨系統自動啟動
[[email protected] etc]# systemctl start httpd.service
[[email protected] etc]# systemctl enable httpd.service
安裝本地源製作相關工具
[[email protected] etc]# yum install yum-utils createrepo
在httpd網站根目錄,預設是即/var/www/html/，建立目錄ambari，
並且將下載的壓縮包解壓到/var/www/html/ambari目錄
HDP-UTILS需要先建立一個HDP-UTILS資料夾並將HDP-UTILS-1.1.0.21-centos7.tar.gz放到HDP-UTILS目錄下再解壓
[[email protected] html]# cd ambari/
[[email protected] ambari]# ls
ambari-2.6.0.0-centos7.tar.gz HDP-2.6.3.0-centos7-rpm.tar.gz HDP-UTILS-1.1.0.21-centos7.tar.gz
[[email protected] ambari]# tar -zxvf ambari-2.6.0.0-centos7.tar.gz
[[email protected] ambari]# mkdir HDP-UTILS
[[email protected] ambari]# ls
ambari ambari-2.6.0.0-centos7.tar.gz HDP-2.6.3.0-centos7-rpm.tar.gz HDP-UTILS HDP-UTILS-1.1.0.21-centos7.tar.gz
[[email protected] ambari]# mv HDP-UTILS-1.1.0.21-centos7.tar.gz ./HDP-UTILS
[[email protected] ambari]# tar -zxvf HDP-UTILS-1.1.0.21-centos7.tar.gz
首先下載上面資源列表中的相應repo檔案，修改其中的URL為本地的地址，相關配置如下
將上面的修改過的源放到/etc/yum.repos.d/下面
[[email protected] yum.repos.d]# yum clean all
[[email protected] yum.repos.d]# yum list update
[[email protected] yum.repos.d]# yum makecache
[[email protected] yum.repos.d]# yum repolist
(6) 安裝mysql (master 主機安裝)
//安裝mysql
Ambari安裝會將安裝等資訊寫入資料庫，建議使用MariaDB資料庫，也可以不安裝而使用預設資料庫PostgreSQL
[[email protected] ~]# yum install mariadb-server
[[email protected] ~]# systemctl start mariadb
[[email protected] ~]# systemctl enable mariadb
#資料庫初始化設定
[[email protected] ~]# mysql_secure_installation
#首先是設定密碼，會提示先輸入密碼
Enter current password for root (enter for none):<–初次執行直接回車
#設定密碼
Set root password? [Y/n] <– 是否設定root使用者密碼，輸入y並回車或直接回車
New password: <– 設定root使用者的密碼
Re-enter new password: <– 再輸入一次你設定的密碼
#其他配置
Remove anonymous users? [Y/n] <– 是否刪除匿名使用者，回車
Disallow root login remotely? [Y/n] <–是否禁止root遠端登入,回車,
Remove test database and access to it? [Y/n] <– 是否刪除test資料庫，回車
Reload privilege tables now? [Y/n] <– 是否重新載入許可權表，回車
#初始化MariaDB完成，接下來測試登入，輸入密碼能正常登陸就完成了
(7) 初始化Ambari和HDP庫 (master 主機)
[[email protected] ~]# mysql -uroot -p
create database ambari character set utf8 ;
CREATE USER 'ambari'@'%'IDENTIFIED BY 'Ambari-123';
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%';
FLUSH PRIVILEGES;
create database hive character set utf8 ;
CREATE USER 'hive'@'%'IDENTIFIED BY 'Hive-123';
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%';
FLUSH PRIVILEGES;
create database oozie character set utf8 ;
CREATE USER 'oozie'@'%'IDENTIFIED BY 'Oozie-123';
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%';
FLUSH PRIVILEGES;
[[email protected] ~]# yum install mysql-connector-java
(8) 安裝Ambari (master 主機)
//安裝Ambari
[[email protected] ~]# yum install ambari-server
(9) 配置Ambari 啟動項
[[email protected] ~]# ambari-server setup
下面是配置執行流程，按照提示操作
1.檢查SELinux是否關閉，如果關閉不用操作
Using python /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
2.提示是否自定義設定。輸入：y
Customize user account for ambari-server daemon [y/n] (n)? y
3.ambari-server 賬號。輸入：ambari
Enter user account for ambari-server daemon (root):ambari
Adjusting ambari-server permissions and ownership...
4.檢查防火牆，如果關閉則不用操作
Checking firewall status...
Redirecting to /bin/systemctl status iptables.service
5.設定JDK。輸入：3
Checking JDK...
Do you want to change Oracle JDK [y/n] (n)? y
[] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[] Oracle JDK 1.7 + Java Cryptography Extension (JCE) Policy Files 7
[] Custom JDK
==============================================================================
Enter choice (1): 3
6.如果上面選擇3自定義JDK,則需要設定JAVA_HOME。輸入：/opt/java/jdk1.8.0_152
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /opt/java/jdk1.8.0_152
Validating JDK on Ambari Server...done.
Completing setup...
7.資料庫配置。選擇：y
Configuring database...
Enter advanced database configuration [y/n] (n)? y
8.選擇資料庫型別。輸入：3
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
==============================================================================
Enter choice (3): 3
9.設定資料庫的具體配置資訊，根據實際情況輸入，如果和括號內相同，則可以直接回車。
Hostname (localhost):
Port (3306):
Database name (ambari):
Username (ambari):
Enter Database Password (Ambari-123):
10.提示必須安裝MySQL JDBC，回車結束ambari配置
WARNING: Before starting Ambari Server, you must copy the MySQL JDBC driver JAR file to /usr/share/java.
Press <enter> to continue.
11.將Ambari資料庫指令碼匯入到資料庫
如果使用自己定義的資料庫，必須在啟動Ambari服務之前匯入Ambari的sql指令碼
#用Ambari使用者（上面設定的使用者）登入mysql
mysql -u ambari -p
use ambari；
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
(10) 啟動Ambari
[[email protected] ~]# ambari-server start
(11) 安裝HDP 參考:https://blog.csdn.net/mawenwu1983/article/details/78983275 問題： 本文總結在使用ambari時產生的異常，以及如何解決的。 如果發生了異常，在介面上不能直觀的看出異常的原因，那麼我一般通過檢視日誌的方法解決。 通過下面的命令檢視
vim /var/log/ambari-server/ambari-server.log1
若該日誌檔案沒有異常資訊，可在其他日誌檔案裡查詢
1、異常一
異常發生在add host並confirm host的時候，會卡住不動，然後看日誌會發現異常資訊：
Error executing bootstrap Cannot create /var/run/ambari-server/bootstrap1
1.1 具體的異常
七月 2018 14:29:08,339 INFO [ambari-client-thread-22654] BootStrapImpl:108 - BootStrapping hosts ambari.slave1.com:
09 七月 2018 14:29:08,340 INFO [Thread-4609] BSRunner:189 - Kicking off the scheduler for polling on logs in /var/run/ambari-server/bootstrap/3
09 七月 2018 14:29:08,340 INFO [Thread-4609] BSRunner:372 - Error executing bootstrap Cannot create /var/run/ambari-server/bootstrap
09 七月 2018 14:29:08,343 ERROR [Thread-4609] BSRunner:441 - java.io.FileNotFoundException: /var/run/ambari-server/bootstrap/3/ambari.slave1.com.done (沒有那個檔案或目錄)
09 七月 2018 14:29:08,344 WARN [Thread-4609] BSRunner:401 - File does not exist: /var/run/ambari-server/bootstrap/3/sshKey
09 七月 2018 14:40:28,033 INFO [ambari-client-thread-22676] BootStrapImpl:108 - BootStrapping hosts amabri.slave1.com:
09 七月 2018 14:40:28,034 INFO [Thread-4635] BSRunner:189 - Kicking off the scheduler for polling on logs in /var/run/ambari-server/bootstrap/4
09 七月 2018 14:40:28,035 INFO [Thread-4635] BSRunner:372 - Error executing bootstrap Cannot create /var/run/ambari-server/bootstrap
09 七月 2018 14:40:28,036 ERROR [Thread-4635] BSRunner:441 - java.io.FileNotFoundException: /var/run/ambari-server/bootstrap/4/amabri.slave1.com.done (沒有那個檔案或目錄)
09 七月 2018 14:40:28,037 WARN [Thread-4635] BSRunner:401 - File does not exist: /var/run/ambari-server/bootstrap/4/sshKey
09 七月 2018 14:44:02,863 INFO [ambari-client-thread-22659] BootStrapImpl:108 - BootStrapping hosts ambari.slave1.com:
123456789101112
1.2 解決辦法
通過檢視，/var/run/ambari-server目錄下確實沒有bootstrap資料夾，手動新增即可
mkdir bootstrap1
但是，如果只建立該資料夾的話，ambari依然報錯，大概是沒有許可權相關的，就不貼出詳細的異常了
為ambari使用者新增許可權即可
chown -R ambari:ambari /var/run/ambari-server/bootstrap/
sudo chown -R ambari /var/run/ambari-server
問題2
ambari註冊主機的時候，ambari-agent出現如下錯誤：
NetUtil.py:96 - EOF occurred in violation of protocol (_ssl.c:579)
NetUtil.py:97 - SSLError: Failed to connect. Please check openssl library versions.
解決方案 在ambari-agent的配置檔案ambari-agent.ini 在 ［security］增加一項
[security]
force_https_protocol=PROTOCOL_TLSv1_2
所有的機器都必須執行