#头条创作挑战赛#
ranger大数据领域的一个集中式安全管理框架，它可以对诸如hdfs、hive、kafka、storm等组件进行细粒度的权限控制。本文将介绍部署过程
1. 部署准备
ranger: 进入apach官网下载 http://ranger.apache.org/download.html， 本次使用的是ranger1.2.0 ，地址为http://mirror.bit.edu.cn/apache/ranger/1.2.0/apache-ranger-1.2.0.tar.gz
maven: 进入Apache的maven官网http://maven.apache.org/download.cgi下载, 本次用的是maven3.6 ，下载地址 http://mirrors.tuna.tsinghua.edu.cn/apache/maven/maven-3/3.6.1/binaries/apache-maven-3.6.1-bin.tar.gz
python2.7: 因编译及试用中需要Python2.7版本的Python，因此如果为Centos6系统，需要手动升级Python至Python2.7，升级过程可参考历史文章Python升级
MySQL： 需要mysql数据库，如无可用MySQL需要部署一套MySQL，部署方法请参考历史文章MySQL部署
mysql-connector-java: 进入MySQL官网下载 https://dev.mysql.com/downloads/connector/j/5.1.html
bc命令: 使用过程中需要使用bc命令
## 下载rangerwget http://mirror.bit.edu.cn/apache/ranger/1.2.0/apache-ranger-1.2.0.tar.gz## 下载mavenwget http://mirrors.tuna.tsinghua.edu.cn/apache/maven/maven-3/3.6.1/binaries/apache-maven-3.6.1-bin.tar.gz## 安装bc命令yum install bc -y2. 编译安装过程
2.1 解压
# 解压ranger及maven包tar -zxvf apache-ranger-1.2.0.tar.gztar -zxvf apache-maven-3.6.1-bin.tar.gz# 将maven相关命令路径配置到环境变量vim /etc/profileexport PATH=/usr/local/maven/bin:$PATH2.2 编译
cd apache-ranger-1.2.0mvn clean# 编译mvn clean compile package assembly:assembly install -DskipTests -Drat.skip=true编译过程中会有如下信息执行过程中会出现很多信息，且编译时间很长，如果出现异常，根据对应的报错信息进行处理，例如Python包不存在时需要手动添加；如果某个jar不存在，可以手动下载放置对应的路径或修改pom地址或文件。
经过很长一段时间，且多次解决错误后，编译完成，出现如下结果
[INFO] Reactor Summary for ranger 1.2.0:[INFO] [INFO] ranger ............................................. SUCCESS [ 0.244 s][INFO] Jdbc SQL Connector ................................. SUCCESS [ 0.573 s][INFO] Credential Support ................................. SUCCESS [ 0.391 s][INFO] Audit Component .................................... SUCCESS [ 1.065 s][INFO] Common library for Plugins ......................... SUCCESS [ 1.332 s][INFO] Installer Support Component ........................ SUCCESS [ 0.153 s][INFO] Credential Builder ................................. SUCCESS [ 0.293 s][INFO] Embedded Web Server Invoker ........................ SUCCESS [ 0.345 s][INFO] Key Management Service ............................. SUCCESS [ 0.984 s][INFO] ranger-plugin-classloader .......................... SUCCESS [ 0.209 s][INFO] HBase Security Plugin Shim ......................... SUCCESS [ 0.830 s][INFO] HBase Security Plugin .............................. SUCCESS [ 1.092 s][INFO] Hdfs Security Plugin ............................... SUCCESS [ 0.786 s][INFO] Hive Security Plugin ............................... SUCCESS [ 1.135 s][INFO] Knox Security Plugin Shim .......................... SUCCESS [ 0.524 s][INFO] Knox Security Plugin ............................... SUCCESS [ 0.712 s][INFO] Storm Security Plugin .............................. SUCCESS [ 0.533 s][INFO] YARN Security Plugin ............................... SUCCESS [ 0.398 s][INFO] Ranger Util ........................................ SUCCESS [ 1.099 s][INFO] Unix Authentication Client ......................... SUCCESS [ 0.259 s][INFO] Security Admin Web Application ..................... SUCCESS [ 12.847 s][INFO] KAFKA Security Plugin .............................. SUCCESS [ 0.458 s][INFO] SOLR Security Plugin ............................... SUCCESS [ 1.208 s][INFO] NiFi Security Plugin ............................... SUCCESS [ 0.386 s][INFO] NiFi Registry Security Plugin ...................... SUCCESS [ 0.381 s][INFO] Unix User Group Synchronizer ....................... SUCCESS [ 1.016 s][INFO] Ldap Config Check Tool ............................. SUCCESS [ 0.222 s][INFO] Unix Authentication Service ........................ SUCCESS [ 0.330 s][INFO] KMS Security Plugin ................................ SUCCESS [ 0.490 s][INFO] Tag Synchronizer ................................... SUCCESS [ 1.211 s][INFO] Hdfs Security Plugin Shim .......................... SUCCESS [ 0.311 s][INFO] Hive Security Plugin Shim .......................... SUCCESS [ 0.800 s][INFO] YARN Security Plugin Shim .......................... SUCCESS [ 0.320 s][INFO] Storm Security Plugin shim ......................... SUCCESS [ 0.365 s][INFO] KAFKA Security Plugin Shim ......................... SUCCESS [ 0.313 s][INFO] SOLR Security Plugin Shim .......................... SUCCESS [ 0.701 s][INFO] Atlas Security Plugin Shim ......................... SUCCESS [ 0.398 s][INFO] KMS Security Plugin Shim ........................... SUCCESS [ 0.342 s][INFO] ranger-examples .................................... SUCCESS [ 0.042 s][INFO] Ranger Examples - Conditions and ContextEnrichers .. SUCCESS [ 0.335 s][INFO] Ranger Examples - SampleApp ........................ SUCCESS [ 0.140 s][INFO] Ranger Examples - Ranger Plugin for SampleApp ...... SUCCESS [ 0.322 s][INFO] Ranger Tools ....................................... SUCCESS [ 0.457 s][INFO] Atlas Security Plugin .............................. SUCCESS [ 0.722 s][INFO] Sqoop Security Plugin .............................. SUCCESS [ 0.472 s][INFO] Sqoop Security Plugin Shim ......................... SUCCESS [ 0.293 s][INFO] Kylin Security Plugin .............................. SUCCESS [ 0.410 s][INFO] Kylin Security Plugin Shim ......................... SUCCESS [ 0.315 s][INFO] Unix Native Authenticator .......................... SUCCESS [ 0.491 s][INFO] ------------------------------------------------------------------------[INFO] BUILD SUCCESS[INFO] ------------------------------------------------------------------------[INFO] Total time: 01:24 h[INFO] Finished at: 2019-08-19T09:47:05+08:00[INFO] ------------------------------------------------------------------------在target文件夹下将有如下包
即表示编译成功完成。
2.3 安装并启动ranger-admin
2.3.1 修改配置文件
关于数据库安装，权限设置等，本文不再展开。
# 进入target目录cd /opt/apache-ranger-1.2.0/target/# 解压ranger-1.2.0-admin.tar.gztar -zxvf ranger-1.2.0-admin.tar.gz# 进入ranger-1.2.0-admin目录cd ranger-1.2.0-admin# 修改 install.propertiesvim install.propertiesSQL_CONNECTOR_JAR=/usr/share/java/mysql-connector-java-8.0.17.jar // 修改为准备工作中下载的jar包及路径db_root_user=rootdb_root_password=123456db_host=192.168.56.105db_name=rangerdb_user=rangeradmindb_password=rangeradmin# 可以注销如下内容#Source for Audit Store. Currently only solr is supported.# * audit_store is solr## audit_store=solr# * audit_solr_url URL to Solr. E.g. http://<solr_host>:6083/solr/ranger_audits## audit_solr_urls=## audit_solr_user=## audit_solr_password=## audit_solr_zookeepers=2.3.2 修改 setup.sh
// 注销如下内容## if [ "$?" != "0" ]## then## validateDefaultUsersPassword 'admin' "${rangerAdmin_password}"## validateDefaultUsersPassword 'rangertagsync' "${rangerTagsync_password}"## validateDefaultUsersPassword 'rangerusersync' "${rangerUsersync_password}"## validateDefaultUsersPassword 'keyadmin' "${keyadmin_password}"## fi2.3.3 安装
./ setup.sh注意，如果中途报如下错误
2019-08-20 08:54:22,460 [I] '/usr/local/java/bin/java' command foundsetup.sh:行325: bc: 未找到命令setup.sh: 第 325 行:[: -eq: 期待一元表达式即缺少bc命令，安装后即可
最终出现如下结果
2019-08-20 09:00:18,240 [I] --------- Verifying Ranger DB connection ---------2019-08-20 09:00:18,240 [I] Checking connection..geradmin' -p '********' -noheader -trim -c \; -query "SELECT version();"Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.2019-08-20 09:00:18,838 [I] Checking connection passed.2019-08-20 09:00:19,091 [I] DB FLAVOR :MYSQL2019-08-20 09:00:19,092 [I] --------- Verifying Ranger DB connection ---------2019-08-20 09:00:19,092 [I] Checking connection..geradmin' -p '********' -noheader -trim -c \; -query "SELECT version();"Loading class `com.mysql.jdbc.Driver'. This is deprecated. The new driver class is `com.mysql.cj.jdbc.Driver'. The driver is automatically registered via the SPI and manual loading of the driver class is generally unnecessary.2019-08-20 09:00:19,660 [I] Checking connection passed.Installation of Ranger PolicyManager Web Application is completed.表示已完成安装。
2.4 启动ranger-admin
# 进入ews目录cd ews # 启动服务./ranger-admin-services.sh start/** 正常情况下出现如下结果*/ Starting Apache Ranger Admin ServiceApache Ranger Admin Service with pid 236275 has started.# 查看是否启动ps -ef|grep ranger或netstat -lntp|grep 60802.5 web端验证
启动完成后，可以用web端登录验证，默认端口为6080,默认用户名密码均为admin
登录后，结果如下
3. 配置hdfs插件
3.1 在ranger-admim上配置一个服务
注意配置的服务名与底层配置文件里配置的服务名一致
3.2 修改配置文件
# 解压tar -zxvf ranger-1.2.0-hdfs-plugin.tar.gz# 进入目录cd /opt/apache-ranger-1.2.0/target/ranger-1.2.0-hdfs-plugin # 修改install.propertiesvim install.properties/** 主要修改如下内容*/# Example:# POLICY_MGR_URL=http://policymanager.xasecure.net:6080#POLICY_MGR_URL=http://192.168.1.17:6080## This is the repository name created within policy manager## Example:# REPOSITORY_NAME=hadoopdev#REPOSITORY_NAME=hadoopdev## Set hadoop home when hadoop program and Ranger HDFS Plugin are not in the# same path.#COMPONENT_INSTALL_DIR_NAME=/opt/software/hadoop/hadoop-3.2.03.3 启动插件
./enable-hdfs-plugin.sh /** 结果如下*/Custom user and group is available, using custom user and group.+ Tue Aug 20 18:04:14 CST 2019 : hadoop: lib folder=/opt/software/hadoop/hadoop-3.2.0/share/hadoop/hdfs/lib conf folder=/opt/software/hadoop/hadoop-3.2.0/etc/hadoop+ Tue Aug 20 18:04:14 CST 2019 : Saving current config file: /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/hdfs-site.xml to /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/.hdfs-site.xml.20190820-180414 ...+ Tue Aug 20 18:04:14 CST 2019 : Saving current config file: /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/ranger-hdfs-audit.xml to /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/.ranger-hdfs-audit.xml.20190820-180414 ...+ Tue Aug 20 18:04:15 CST 2019 : Saving current config file: /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/ranger-hdfs-security.xml to /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/.ranger-hdfs-security.xml.20190820-180414 ...+ Tue Aug 20 18:04:15 CST 2019 : Saving current config file: /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/ranger-policymgr-ssl.xml to /opt/software/hadoop/hadoop-3.2.0/etc/hadoop/.ranger-policymgr-ssl.xml.20190820-180414 ...+ Tue Aug 20 18:04:16 CST 2019 : Saving current JCE file: /etc/ranger/hadoopdev/cred.jceks to /etc/ranger/hadoopdev/.cred.jceks.20190820180416 ...Ranger Plugin for hadoop has been enabled. Please restart hadoop to ensure that changes are effective.网页上测试成功后即可
其他组件的测试和hdfs类似，可自行尝试。如果大家使用过程中编译或配置有问题，可以添加微信或和我沟通讨论。