Versions Compared

Old Version 52

changes.mady.by.user Chaoran Wang

Saved on

compared with

New Version Current

changes.mady.by.user Chaoran Wang

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Ranger Cluster Setup

...

# * audit_solr_url URL to Solr. E.g. http://<solr_host>:6083/solr/ranger_audits
audit_solr_urls=http://localhosthostname:6083/solr/ranger_audits

 

...

XAAUDIT.SOLR.ENABLE=true
XAAUDIT.SOLR.URL=http://localhosthostname:6083/solr/ranger_audits

 

...

 

# DB UserId used for the XASecure schema
db_name=ranger
db_user=rangeradmin
db_password=rangeradmin
 
# DB UserId for storing auditlog infromation
audit_db_name=ranger
audit_db_user=rangerlogger
audit_db_password=rangerlogger

 

 

 

How to Upgrade MySQL 5.1 to MySQL 5.5 on CentOS 6.7 (Optional, only if you install ranger latest versions after 0.7)

https://www.zerostopbits.com/how-to-upgrade-mysql-5-1-to-mysql-5-5-on-centos-6-7/

1. Confirm your version of MySQL is CentOS 6.7 default

 

Type in the following to confirm that you have CentOS 6.7 default of MySQL 5.1 installed:

 

rpm -qa | grep mysql

 

Outputshouldlooksimilar to the following:

 

mysql-server-5.1.73-5.el6_6.x86_64
mysql-libs-5.1.73-5.el6_6.x86_64
mysql-5.1.73-5.el6_6.x86_64

 

Everything looks good, moving on!

 

 

2. Install and activate the REMI and EPEL RPM Repositories

 

If you have not done so already, install and activate the REMI and EPEL repositories;

 

wget https://dl.fedoraproject.org/pub/epel/epel-release-latest-6.noarch.rpm && rpm -Uvh epel-release-latest-6.noarch.rpm

wget http://rpms.famillecollet.com/enterprise/remi-release-6.rpm && rpm -Uvh remi-release-6*.rpm

 

Now to enable the REMI repository globally:

 

nano /etc/yum.repos.d/remi.repo

 

Under the section that looks like [remi] make the following changes:

 

[remi]
name=Remi's RPM repository for Enterprise Linux 6 - $basearch
#baseurl=http://rpms.remirepo.net/enterprise/6/remi/$basearch/
mirrorlist=http://rpms.remirepo.net/enterprise/6/remi/mirror
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-remi

 

Type CTRL-O to save, and CTRL-X to exit

 

 

2. Update MySQL from 5.1 to 5.5

 

Simply type in the following:

 

yum -y update mysql*

 

Once that’s done, we can verify:

 

rpm -qa | grep mysql

 

And we should see something similar to the below:

 

mysql-5.5.45-1.el6.remi.x86_64
compat-mysql51-5.1.54-1.el6.remi.x86_64
mysql-libs-5.5.45-1.el6.remi.x86_64
mysql-server-5.5.45-1.el6.remi.x86_64

 

Now we need to make sure MySQL is working correctly.

 

If you’ve set a root password already, type the following;

 

mysql -u root -p

 

If you have not yet set a root password, you can simply type;

 

mysql

 

You should see something similar to the following

 

Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 1

 

mysql -u root -p

SET @@global.innodb_large_prefix = 1;

 

 

use 

 

mysql_upgrade -u root -p

 

to solve

 


 

SQLException : SQL state: HY000 java.sql.SQLException: Column count of mysql.user is wrong. Expected 42, found 39. Created with MySQL 50173, now running 50556. Please use mysql_upgrade to fix this error. ErrorCode: 1558

 

error

 

e. Once all the required properties are updated run the setup.sh script

 

./setup.sh

 

 

 

Will encountering the following error, if one user ranger >0.7.0 version and with a sql version <5.5:

then run


SET @@global.innodb_large_prefix = 1;

 

Error executing: CREATE TABLE `x_portal_user` ( `id` bigint(20) NOT NULL AUTO_INCREMENT, `create_time` datetime DEFAULT NULL, `update_time` datetime DEFAULT NULL, `added_by_id` bigint(20) DEFAULT NULL, `upd_by_id` bigint(20) DEFAULT NULL, `first_name` varchar(1022) DEFAULT NULL, `last_name` varchar(1022) DEFAULT NULL, `pub_scr_name` varchar(2048) DEFAULT NULL, `login_id` varchar(767) DEFAULT NULL, `password` varchar(512) NOT NULL, `email` varchar(512) DEFAULT NULL, `status` int(11) NOT NULL DEFAULT '0', `user_src` int(11) NOT NULL DEFAULT '0', `notes` varchar(4000) DEFAULT NULL, PRIMARY KEY (`id`), UNIQUE KEY `x_portal_user_UK_login_id` (`login_id`), UNIQUE KEY `x_portal_user_UK_email` (`email`), KEY `x_portal_user_FK_added_by_id` (`added_by_id`), KEY `x_portal_user_FK_upd_by_id` (`upd_by_id`), KEY `x_portal_user_cr_time` (`create_time`), KEY `x_portal_user_up_time` (`update_time`), KEY `x_portal_user_name` (`first_name`(767)), KEY `x_portal_user_email` (`email`), CONSTRAINT `x_portal_user_FK_added_by_id` FOREIGN KEY (`added_by_id`) REFERENCES `x_portal_user` (`id`), CONSTRAINT `x_portal_user_FK_upd_by_id` FOREIGN KEY (`upd_by_id`) REFERENCES `x_portal_user` (`id`) )AUTO_INCREMENT=2 ROW_FORMAT=DYNAMIC;
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 1000 bytes
SQLException : SQL state: 42000 com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 1000 bytes ErrorCode: 1071
2017-06-17 01:02:03,297 [E] xa_core_db.sql file import failed!
2017-06-17 01:02:03,297 [I] CORE_DB_SCHEMA might being imported by some other process
2017-06-17 01:04:03,382 [JISQL] /usr/lib/jvm/java/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/local/ranger-0.7.2-SNAPSHOT-admin/jisql/lib/* org.apache.util.sql.Jisql -driver mysqlconj -cstring jdbc:mysql://localhost/ranger -u 'rangeradmin' -p '********' -noheader -trim -c \; -query "show tables like 'xa_access_audit';"
2017-06-17 01:04:03,673 [I] Table xa_access_audit does not exist in database ranger
2017-06-17 01:04:03,673 [JISQL] /usr/lib/jvm/java/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/local/ranger-0.7.2-SNAPSHOT-admin/jisql/lib/* org.apache.util.sql.Jisql -driver mysqlconj -cstring jdbc:mysql://localhost/ranger -u 'rangeradmin' -p '********' -noheader -trim -c \; -query "delete from x_db_version_h where version='CORE_DB_SCHEMA' and active='N' and updated_by='ranger24355-1000.dev.continuuity.net';"
2017-06-17 01:04:03,955 [E] CORE_DB_SCHEMA import failed!

...

inside /usr/local cd ranger-usersync


POLICY_MGR_URL = http://localhosthostname:6080
SYNC_SOURCE = unix 
logdir = /var/log/ranger/usersync

...

./ranger-usersync-services.sh start

make sure this script run without error and there is no symlink loop.

 

Enabling Ranger HDFS Plugins (Optional, from HDFS, HIVE etc choose one plugin) (must choose one, suggest hive)

...

then cp lookup jars to, if not there use mkdir -p to make path:

/usr/local/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/cdap

grant 

cp /home/chaoran/lookup-0.1.0-SNAPSHOT-jar-with-dependencies.jar /usr/local/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins

chown ranger:ranger to this jar filels -l to look at user/group property to confirm it has 

chown ranger:ranger

...

restart ranger admin: ranger-admin restart

 cdap

cd cdap

mv /usr/local/cdap-ranger/lookup-0.1.0-SNAPSHOT.jar ./

chown ranger:ranger /usr/local/ranger-admin/ews/webapp/WEB-INF/classes/ranger-plugins/lookup-0.1.0-SNAPSHOT-jar-with-dependencies.jar

ls -l to look at user/group property to confirm it has ranger:ranger

restart ranger admin: ranger-admin restart

 

scp /Users/chaoranwang/Desktop/cdap-security-extn/cdap-ranger/binding/target/binding-0.1.0-SNAPSHOT.jar cdaphostname:~

...

cp /home/chaoran/binding-0.1.0-SNAPSHOT.jar /usr/local/cdap-ranger/

chown cdap:cdap to this jar filecdap  /usr/local/cdap-ranger/binding-0.1.0-SNAPSHOT.jar

and then restart server

restart cdap service:

...

tail -f /usr/local/ranger-admin/ews/logs/catalina.out


correctly setup should show:

Image Added


Ranger plugin and CDAP architecture

Image Added

 

Image Added

 


Make enforcement working

configure cdap property: first 3

...