1. 程式人生 > >Hadoop偽分散式模式安裝

Hadoop偽分散式模式安裝

環境資訊

作業系統:Ubuntu 17.04 64bit

安裝步驟

1. 安裝配置JDK1.1 複製到/opt/bigdata/目錄下,解壓
cp ~/Downloads/jdk-8u161-linux-x64.tar.gz /opt/bigdata/
cd /opt/bigdata/
tar -xvf jdk-8u161-linux-x64.tar.gz
rm jdk-8u161-linux-x64.tar.gz
1.2 新增環境變數開啟bashrc檔案
vi ~/.bashrc
在末尾新增如下命令
export JAVA_HOME=/opt/bigdata/jdk1.8.0_161
export PATH=$PATH:$JAVA_HOME/bin
生效
source ~/.bashrc
2. 安裝配置ssh安裝
sudo apt-get install ssh
基於空口令生成一個新的金鑰,以實現無密碼登入
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
測試是否能夠連線
ssh localhost
3. 安裝配置Hadoop3.1 下載3.2 解壓
cp ~/Downloads/hadoop-3.0.0.tar.gz /opt/bigdata/
cd /opt/bigdata/
tar -xvf hadoop-3.0.0.tar.gz
3.3 配置需要修改hadoop-3.0.0/etc/hadoop/目錄下的4個配置檔案和一個shell檔案,內容如下:core-site.xml
<configuration>
        <property>
                <name>fs.defaultFS</name>
                <value>hdfs://localhost</value>
        </property>
</configuration>
hdfs-site.xml
<configuration>
        <property>
                <name>dfs.replication</name>
                <value>1</value>
        </property>
</configuration>
yarn-site.xml
<configuration>
        <property>
                <name>yarn.resourcemanager.hostname</name>
                <value>localhost</value>
        </property>
        <property>
                <name>yarn.noemanager.aux-services</name>
                <value>mapreduce_shuffle</value>
        </property>
</configuration>
mapred-site.xml
<configuration>
        <property>
                <name>mapreduce.framework.name</name>
                <value>yarn</value>
        </property>
</configuration>
hadoop-env.sh
新增下面的命令
export JAVA_HOME=/opt/bigdata/jdk1.8.0_161

3.4 格式化HDFS檔案系統
cd /opt/bigdata/hadoop-3.0.0/bin/
./hdfs namenode -format

3.5 啟動hadoop
cd /opt/bigdata/hadoop-3.0.0/sbin/
./start-all.sh
看到下面輸出
WARNING: Attempting to start all Apache Hadoop daemons as pxchen in 10 seconds.
WARNING: This is not a recommended production deployment configuration.
WARNING: Use CTRL-C to abort.
Starting namenodes on [localhost]
Starting datanodes
Starting secondary namenodes [pxchen-VirtualBox]
Starting resourcemanager
Starting nodemanagers
3.5 檢查Resource Manager開啟瀏覽器,輸入下面地址:http://localhost:8088/可以看到:
Name Node輸入地址:http://localhost:9870/dfshealth.html#tab-overview
參考:1. 《Hadoop權威指南》第4版