http://nolink.github.io/ Recent content on nolink boxer Hugo -- gohugo.io en-us http://nolink.github.io/posts/cloudera/ Mon, 01 Jan 0001 00:00:00 +0000 http://nolink.github.io/posts/cloudera/ title: 在Centos上搭建Cloudera Manager及Hadoop集群 date: 2016-02-26 17:18:05 tags: [cloudera,hadoop] 安装cloudera可以直接选择用repo安装，也可以直接下载RPM包进行安装，本文暂述通过repo安装的方法 准备好机器，并修改hostname，注意：hostname不能有下划线 使用以下两个命令来修改Hostname hostname hadoop-prod-slave1 vi /etc/sysconfig/network 准备好cloudera相关的repo wget http://archive.cloudera.com/cm5/redhat/6/x86_64/cm/cloudera-manager.repo cp cloudera-manager.repo /etc/yum.repos.d/ 安装cloudera server端，即cloudera manager，这里需要使用到数据库，本文使用cloudera内嵌postgresql yum install cloudera-manager-daemons yum install cloudera-manager-server-db-2 yum install cloudera-manager-server 修改cloudera数据库设置 vim /etc/cloudera-scm-server/db.properties 修改监控等功能数据库设置 vim /etc/cloudera-scm-server/db.mgmt.properties 启动cloudera service cloudera-scm-server-db start service cloudera-scm-server start 安装Hadoop集群——安装Cloudera agent，agent也需要使用daemons里面的包，因此各个机器也需要再安装一遍daemons yum install cloudera-manager-daemons yum install cloudera-manager-agent service cloudera-scm-agent start 安装Hadoop集群——部署Zookeeper, Kafka, HDFS, HBase 登录cloudera, http://cloudera:7180，用户名默认为admin, 密码默认为admin http://nolink.github.io/posts/elasticsearch/ Mon, 01 Jan 0001 00:00:00 +0000 http://nolink.github.io/posts/elasticsearch/ title: 在Centos上搭建Elastic Search集群 date: 2016-02-26 17:18:05 tags: [es,elastic-search] 下载安装包 地址 https://www.elastic.co/downloads/elasticsearch wget https://download.elasticsearch.org/elasticsearch/release/org/elasticsearch/distribution/rpm/elasticsearch/2.2.0/elasticsearch-2.2.0.rpm 安装es yum -y install elasticsearch-2.2.0.rpm 如果无法找到java命令，可以修改es的配置，/etc/sysconfig/elasticsearch 配置java路径，并将ES集群的堆内存调整为4GB，默认为256MB export JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera/ export PATH=${PATH}:${JAVA_HOME}/bin ES_HEAP_SIZE=4g 修改es集群的配置, /etc/elasticsearch/elasticsearch.yml cluster.name: clog path.data: /opt/es discovery.zen.ping.unicast.hosts: ["xxx:9300","xxx:9300","xxx:9300"] threadpool.bulk.size: 4 threadpool.bulk.queue_size: 500 安装Head插件 /usr/share/elasticsearch/bin/plugin --install mobz/elasticsearch-head 启动es服务 mkdir /opt/es chown elasticsearch:elasticsearch /opt/es service elasticsearch start 使用head插件直接查看数据 http://elasticsearch:9200/_plugin/head http://nolink.github.io/posts/etcd-cluster/ Mon, 01 Jan 0001 00:00:00 +0000 http://nolink.github.io/posts/etcd-cluster/ title: 在Centos上搭建ETCD集群 date: 2016-02-26 13:52:32 tags: etcd 首先，下载可执行包 wget https://github.com/coreos/etcd/releases/download/v2.2.2/etcd-v2.2.2-linux-amd64.tar.gz 解压 bash tar -xzf etcd-v2.2.2-linux-amd64.tar.gz 使etcd命令可直接执行 cp etcd-v2.2.2-linux-amd64/etcd /usr/bin 编辑启动命令，每个etcd节点都相同 vi /etc/init/etcd.conf 内容如下 description "etcd 2.2 distributed key-value store" start on (net-device-up and local-filesystems and runlevel [2345]) stop on runlevel [016] respawn respawn limit 10 5 script if [ -f "/etc/default/etcd" ]; then . /etc/default/etcd fi exec /usr/bin/etcd >>/opt/log/etcd.log 2>&1 end script 编辑启动参数，每个etcd节点都不同 第二个节点的内容如下，其他节点内容依次类推 http://nolink.github.io/posts/github-blog/ Mon, 01 Jan 0001 00:00:00 +0000 http://nolink.github.io/posts/github-blog/ title: 使用github搭建Blog 使用Hexo + Github搭建博客 http://acupple.github.io/2016/02/25/Hexo%20+%20Github%E6%90%AD%E5%BB%BA%E8%87%AA%E5%B7%B1%E7%9A%84%E5%8D%9A%E5%AE%A2/