ubuntu server 12 上搭建 hadoop2.2 单机伪分布式,ubuntuhadoop2.2
注:我把解压后的hadoop文件重命名为hadoop,放在opt目录下$source /etc/profile
- export JAVA_HOME=/opt/jdk
- <configuration>
- <property>
- <name>fs.default.name</name>
- <value>hdfs://localhost:9000</value>
- </property>
- <property>
- <name>hadoop.tmp.dir</name>
- <value>/tmp/hadoop/hadoop-hadoop</value>
- </property>
- </configuration>
其中,/home/hadoop/dfs/name,/home/hadoop/dfs/data都是文件系统中的目录,需要先新建
- <configuration>
- <property>
- <name>dfs.namenode.name.dir</name>
- <value>file:/home/hadoop/dfs/name</value>
- <description>Determines where on the local filesystem the DFS name node
- should store the name table. If this is a comma-delimited list
- of directories then the name table is replicated in all of the
- directories, for redundancy. </description>
- <final>true</final>
- </property>
- <property>
- <name>dfs.datanode.data.dir</name>
- <value>file:/home/hadoop/dfs/data</value>
- <description>Determines where on the local filesystem an DFS data node
- should store its blocks. If this is a comma-delimited
- list of directories, then data will be stored in all named
- directories, typically on different devices.
- Directories that do not exist are ignored.
- </description>
- <final>true</final>
- </property>
- <property>
- <name>dfs.replication</name>
- <value>1</value>
- </property>
- <property>
- <name>dfs.permissions</name>
- <value>false</value>
- </property>
- </configuration>
- <configuration>
- <property>
- <name>mapreduce.framework.name</name>
- <value>yarn</value>
- </property>
- <property>
- <name>mapred.system.dir</name>
- <value>file:/home/hadoop/mapred/system</value>
- <final>true</final>
- </property>
- <property>
- <name>mapred.local.dir</name>
- <value>file:/home/hadoop/mapred/local</value>
- <final>true</final>
- </property>
- </configuration>
- <configuration>
- <!-- Site specific YARN configuration properties -->
- <property>
- <name>yarn.resourcemanager.resource-tracker.address</name>
- <value>localhost:8081</value>
- <description>host is the hostname of the resource manager and
- port is the port on which the NodeManagers contact the Resource Manager.
- </description>
- </property>
- <property>
- <name>yarn.resourcemanager.scheduler.address</name>
- <value>localhost:8082</value>
- <description>host is the hostname of the resourcemanager and port is the port
- on which the Applications in the cluster talk to the Resource Manager.
- </description>
- </property>
- <property>
- <name>yarn.resourcemanager.scheduler.class</name>
- <value>org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler</value>
- <description>In case you do not want to use the default scheduler</description>
- </property>
- <property>
- <name>yarn.resourcemanager.address</name>
- <value>localhost:8083</value>
- <description>the host is the hostname of the ResourceManager and the port is the port on
- which the clients can talk to the Resource Manager. </description>
- </property>
- <property>
- <name>yarn.nodemanager.local-dirs</name>
- <value></value>
- <description>the local directories used by the nodemanager</description>
- </property>
- <property>
- <name>yarn.nodemanager.address</name>
- <value>0.0.0.0:port</value>
- <description>the nodemanagers bind to this port</description>
- </property>
- <property>
- <name>yarn.nodemanager.resource.memory-mb</name>
- <value>10240</value>
- <description>the amount of memory on the NodeManager in GB</description>
- </property>
- <property>
- <name>yarn.nodemanager.remote-app-log-dir</name>
- <value>/app-logs</value>
- <description>directory on hdfs where the application logs are moved to </description>
- </property>
- <property>
- <name>yarn.nodemanager.log-dirs</name>
- <value></value>
- <description>the directories used by Nodemanagers as log directories</description>
- </property>
- <property>
- <name>yarn.nodemanager.aux-services</name>
- <value>mapreduce.shuffle</value>
- <description>shuffle service that needs to be set for Map Reduce to run </description>
- </property>
- </configuration>
启动hdfs以及yarn
完成以上配置后可以检测是否配置成首先格式化namenode
$ hdfs namenode -format
然后启动hdfs
$ start-dfs.sh
或者
$ hadoop-daemon.sh start namenode
$ hadoop-daemon.sh start datanode
接着启动yarn daemons
$ start-yarn.sh
或者
$ yarn-daemon.sh start resourcemanager
$ yarn-daemon.sh start nodemanager
启动完成后可以进入http://localhost:50070/dfshealth.jsp 查看dfs状态,
本站文章为和通数据库网友分享或者投稿,欢迎任何形式的转载,但请务必注明出处.
同时文章内容如有侵犯了您的权益,请联系QQ:970679559,我们会在尽快处理。