This will be a quick and dirty setup guide, please if you see something stupid point it out. I started from the latest Centos 7.3 minimal and I applied this on 3 nodes. This helped me understand how NAS works, and it is very rough draft. I felt bad not writing it once I learned this cause the documentation now i have is 1 month old and I it may be lacking, thus I apologize. before hand. Please read this guide with the notion of understanding and not copy/paste implementing. I mean after you understand the goal below and understand it I reckon you can implement it better by enhancing it.
I recommend before you start to partition Centos in such a way:
2 GiB boot | 50 GiB root | 2 GiB swap
Cause GlusterFS needs to be away from root / partition.
And leave 1 empty unallocated partition/space for Gluster, we will create in latter steps.
Before we start, the imaginary goal I created is create 3 nodes that replicate the Filesystem, and if 1 node can go down, users can have the option to connect to node 2 or 3 (manually by entering different IP address), and users will be using WinSCP (cause it rocks in stability and security and performance also it tries to make it easy for dumb users). So let us start:
yum search centos-release-gluster # check LTS version (if you find latest + LTS install)
yum -y install centos-release-gluster310 -y
sed -i -e "s/enabled=1/enabled=0/g" /etc/yum.repos.d/CentOS-Gluster-3.10.repo
yum --enablerepo=centos-gluster310,epel -y install glusterfs-server
systemctl enable glusterd
systemctl start rpcbind
systemctl enable rpcbind
systemctl restart glusterd
Part 1 finished, we installed and started required packages and services.
Prep the servers/nodes by changing their hostnames and configured /etc/hosts to reflect that, for example ensure hostname is something like centos.gluster.1 | centos.gluster.2 | centos.gluster.3 (one on each node)
nano /etc/hostname
Then:
nano /etc/hosts
And put the hostnames to manually point to IP of their respective machines, in each of the nodes
192.168.1.20 centos.gluster.1
192.168.1.19 centos.gluster.2
192.168.1.18 centos.gluster.3
Now on all nodes do this:
mkdir -p /glusterfs
mkdir -p /replica_files
Then use cfdisk to create/write the new partition without a type. it has command line interface simply navigate using the arrow keys and create partition.
Then run the below on all 3 nodes
partprobe
mkfs.xfs /dev/sda4 (replace /dev/sda4 with the your partition)
partprobe
mount /dev/sda4 /glusterfs/
nano /etc/fstab -> /dev/sda4 /glusterfs xfs defaults 0 0
mkdir -p /glusterfs/replicafs
Part 2 done we created the folder structure and prepared the filesystem
Now let us connect the peers:
gluster peer probe centos.gluster.1 #(do this from node 1/2/3)
gluster peer probe centos.gluster.2 #(do this from node 1/2/3)
gluster peer probe centos.gluster.3 #(do this from node 1/2/3)
and check that all is good by running:
gluster peer status
If you have any issues, trying pinging the hostnames from the nodes and see if they translate to IP properly, if not check your hosts file.
Let us create the replication between 3 nodes, you need to run this on 1 node only
gluster volume create vol_replica replica 3 transport tcp \
centos.gluster.1:/glusterfs/replicafs \
centos.gluster.2:/glusterfs/replicafs \
centos.gluster.3:/glusterfs/replicafs
gluster volume start vol_replica
gluster volume info
You can then mount this replicafs by running the below on each node respectively :
mount -t glusterfs centos.gluster.1:/vol_replica /replica_files (run only on node 1)
mount -t glusterfs centos.gluster.2:/vol_replica /replica_files (run only on node 2)
mount -t glusterfs centos.gluster.3:/vol_replica /replica_files (run only on node 3)
I had trouble using /etc/fstab to mount the above 3 commands, so I just scheduled the above commands to run at startup.
Now the fun starts with WebDAV you can do this on seperate node or on one of the 3 nodes or on all 3 nodes, it is up to you.
yum -y install httpd
systemctl start httpd
systemctl enable httpd
Create SSL/TLS key + Certificate + Certificate Signing request (3 files server.key|server.cert|server.csr)
cd /etc/pki/tls/certs
make server.key
openssl rsa -in server.key -out server.key
make server.csr
openssl x509 -in server.csr -out server.crt -req -signkey server.key -days 365
yum -y install mod_ssl
nano /etc/httpd/conf.d/ssl.conf
# line 59: uncomment -> DocumentRoot "/var/www/html"
# line 60: uncomment and specify the server name -> ServerName 192.168.1.20:443
# line 100: change to the one created earlier -> SSLCertificateFile /etc/pki/tls/certs/server.crt
# line 107: change to the one created earlier -> SSLCertificateKeyFile /etc/pki/tls/certs/server.key
# line 110: comment out -> # SSLCertificateChainFile
To Change SSL/TLS 443 default port, edit 3 lines:
nano /etc/httpd/conf.d/ssl.conf
Listen 7777 https
<VirtualHost _default_:7777>
ServerName 192.168.1.20:7777
Using WebDAV + GlusterFS:
make sure you mounted the glusterfs on /replica_files first
mkdir /replica_files/public
chown apache. /replica_files/public
chmod 770 /replica_files/public
nano /etc/httpd/conf.d/webdav.conf
DavLockDB "/tmp/DavLock"
Alias /public /replica_files/public
<Location /public>
DAV On
SSLRequireSSL
Options None
AuthType Basic
AuthName "Public Share"
AuthUserFile /etc/httpd/conf/.htpasswd
<RequireAny>
Require method GET POST OPTIONS
Require valid-user
</RequireAny>
</Location>
Create user
htpasswd -B -C 10 /etc/httpd/conf/.htpasswd jane # Create Bcrypt strong pass for jane (Works with WinSCP)
What if you want to create a personal share, not public one, will you need to understand a public share needs to be accessed with username and pass, but you can share this credentials to multiple users.
However you can also separate database password files, which adds separation layer, for example even if Jane password is correct she wont access another folder (hr folder for example).
mkdir -p /replica_files/hr
chown apache. /replica_files/hr
chmod 770 /replica_files/hr
nano /etc/httpd/conf.d/webdav.conf
And add:
Alias /hr /replica_files/hr
<Directory /replica_files/hr>
DAV On
AuthType Basic
AuthName "HR Private Share"
AuthUserFile /etc/httpd/conf/hr.passwd
Require valid-user
The only noteable change is hr.passwd instead of .htpasswd
to create user there (hr.passwd):
htpasswd -c -B -C 10 /etc/httpd/conf/hr.passwd marc
add another user
htpasswd -B -C 10 /etc/httpd/conf/hr.passwd marie
dont use the -c twice it will re-create the database file. Other useful commands include:
htpasswd -v /etc/httpd/conf/.htpasswd jane # to verify user exist
htpasswd -D /etc/httpd/conf/.htpasswd jane # delete user jane
htpasswd password encryption info:
-m default is MD5 htpasswd
-B is Bcrypt which is very secure can be paired with -C valid values 4-31 (higher is more secure but slower)
-s SHA encryption (insecure)
If SELinux is enabled, change rules like follows.
chcon -R -t httpd_sys_rw_content_t /webdav_folder_location
semanage fcontext -a -t httpd_sys_rw_content_t /webdav_folder_location
In WinSCP you can connect using one liner:
https://marc:[email protected]:7777/hr/
copy and paste this in Hostname
In the end you can create structure, if you installed Apace+WebDav on all 3 nodes, that if one node goes down you can instruct users to use node 2 IP address, and it will have exactly the same files. but users need to know the 3 nodes Address before hand, I still didnt know how to group all 3 nodes location behind load balancing proxy or something similar. will learn that in the future.
Many of the article especially GlusterFs take from guides already found in ML, see scott guide:
https://mangolassi.it/topic/8619/installing-gluster-on-centos-7/1