| |
MogileFS Profile
MogileFS is a set of open source distributed storage components by LiveJournal Danga Interactive's research and development, mainly used to store vast amounts of small files.
MogileFS has three components:
1) Tracker Nodes: the file system is the core of the cluster nodes, with the node database preservation metadata, the main features include monitoring backend Storage nodes Storage nodes and instructions to finish adding data, delete, query, and data replication between nodes, etc. , it processes running on that node mogilefsd. Tracker nodes often have multiple, for load balancing and high availability;
2) Storage Nodes: Location data is actually stored, WebDAV-based mode, to complete the file create, delete, and rename files, processes running as mogstored;
3) database Nodes: Help Tracker node storing metadata and global information, including the host, domain, class and so on. Database high availability cluster is recommended to prevent a single point of failure.
Implementation Mechanism
In the cluster file system, the data blocks each will have multiple copies, stored on multiple Storage Node, a number of different systems copy files varies (generally recommended as 3 parts) in the Tracker Node will separately record storage location of each data block (in which storage and path). Storage If a node is not online, the client node requests to obtain data to Tracker, Tracker node returns the storage location for redundant copy to the client. This ensures high availability of the entire file system data. When data is stored to a storage node, each storage node through the inter-node communication will be sent to other nodes according to a copy of the instructions Tracker. If storage is down a copy on the node will be re-supplied by other nodes to ensure that the number of copies is not less than a specified value.
How it works: When a client needs to access data, the first interaction with the Tracker, Tracker to get the client to access the data required to access metadata by querying the database, and then returned to the client, the client using the results returned Tracker, and Storage nodes nodes interact to complete data access (once there is a possible link may also contact multiple).
Deployment MogileFS
Deployment Tracker Nodes
1) installation package
Use cpan online installation, first install the necessary perl library.
[Root @ node1 ~] # cpan Sys :: Syscall
[Root @ node1 ~] # cpan IO :: WrapTie
[Root @ node1 ~] # cpan Danga :: Socket
[Root @ node1 ~] # cpan IO :: AIO
[Root @ node1 ~] # cpan MogileFS :: Client
[Root @ node1 ~] # cpan Net :: Netmask
[Root @ node1 ~] # cpan Perlbal
Installation MogileFS Server:
[Root @ node1 ~] # cpan MogileFS :: Server
Or download the source package to compile and install:
[Root @ node1 ~] # wget http://mirrors.ustc.edu.cn/CPAN/authors/id/D/DO/DORMANDO/MogileFS-Server-2.72.tar.gz
[Root @ node1 ~] # tar xf MogileFS-Server-2.72.tar.gz
[Root @ node1 ~] # cd MogileFS-Server-2.72
[Root @ node1 MogileFS-Server-2.72] # perl Makefile.PL
[Root @ node1 MogileFS-Server-2.72] # make && make install
2) to complete the configuration
Create a profile directory and add a profile:
[Root @ www ~] # mkdir / etc / mogilefs
[Root @ node1 ~] # vim /etc/mogilefs/mogilefsd.conf
daemonize = 1
pidfile = /var/run/mogilefsd/mogilefsd.pid
db_dsn = DBI: mysql: mogilefs: host = 192.168.1.134; port = 3306; mysql_connect_timeout = 5 # database connection information
db_user = user name on the database mogilefs #
db_pass = mogilefs # corresponding user name password
listen = 0.0.0.0:7001 # listen address and port
conf_port = 7001
query_jobs = 10 # process for queries
delete_jobs = 1 # number of processes for the delete operation
replicate_jobs = 5 # number of processes for replication
reaper_jobs = 1 # number of processes for the recovery of resources
Create a database on the database node, and authorized users:
MariaDB [(none)]> create database mogilefs;
Query OK, 1 row affected (0.04 sec)
MariaDB [(none)]> grant all on mogilefs * to 'mogilefs'@'192.168.%.%' identified by 'mogilefs'.;
Query OK, 0 rows affected (0.00 sec)
MariaDB [(none)]> flush privileges;
Query OK, 0 rows affected (0.00 sec)
Initialize the database:
[Root @ node1 ~] # mogdbsetup --dbhost = 192.168.1.134 --dbname = mogilefs --dbuser = mogilefs --dbpass = mogilefs --dbrootpass = mogilefs
If there are mistakes, there is no connection to install perl mysql driver caused install perl-DBD-MySQL can.
Failed to load MogileFS :: Store :: MySQL: Can not locate DBD / mysql.pm in @INC (@INC contains: lib / usr / local / lib64 / perl5 / usr / local / share / perl5 / usr / lib64 / perl5 / vendor_perl / usr / share / perl5 / vendor_perl / usr / lib64 / perl5 / usr / share / perl5.) at /usr/local/share/perl5/MogileFS/Store/MySQL.pm line 5 ...... ...
........
###########################################
[Root @ node1 ~] # yum install perl-DBD-MySQL
3) Start the service
mogilefsd not run as root, you must run with other users, add mogilefs for this user, and then start the service:
[Root @ node1 ~] # useradd -r mogilefs
[Root @ node1 ~] # su - mogilefs -c "mogilefsd -c /etc/mogilefs/mogilefsd.conf --daemon"
A better way is to add the service script, and then start the service through the script.
There is a need to pay attention, version 0.25 Sys :: Syscall presence of bug, the file system is running, add data to the file system, it does not retain two copies in accordance with the default policy, the default policy is as follows:
Upload file information:
We can see, only one copy of the data stored on the 192.168.1.127 node. Sys :: Syscall version 0.25 is the latest version, use the default installation cpan is this version, you can downgrade to version 0.23, or directly download and install version 0.23.
wget http://mirrors.ustc.edu.cn/CPAN/authors/id/B/BR/BRADFITZ/Sys-Syscall-0.23.tar.gz
###############################
perl Makefile.PL
make && make install
Deployment Storage Nodes
1) installation package (above identified)
[Root @ node2 ~] # cpan Sys :: Syscall
[Root @ node2 ~] # cpan IO :: WrapTie
[Root @ node2 ~] # cpan Danga :: Socket
[Root @ node2 ~] # cpan IO :: AIO
[Root @ node2 ~] # cpan MogileFS :: Client
[Root @ node2 ~] # cpan Net :: Netmask
[Root @ node2 ~] # cpan Perlbal
[Root @ node2 ~] # cpan MogileFS :: Server
2) to complete the configuration
Create a directory and add the configuration file:
[Root @ node2 ~] # mkdir / etc / mogilefs
[Root @ node2 ~] # vim /etc/mogilefs/mogstored.conf
maxconns = 10000 # The maximum number of concurrent connections
httplisten = 0.0.0.0:7500 #http listening address
mgmtlisten = 0.0.0.0:7501 # Listen Address Management Interface
docroot = / mogdata / data # Directory for storage
Add, mount the partition, according to /etc/mogilefs/mogstored.conf the docroot directory configuration information to add, and modify the owner is a group:
[Root @ node2 ~] # fdisk / dev / sdb
[Root @ node2 ~] # mkfs -t ext4 -b 4096 / dev / sdb1
[Root @ node2 ~] # mkdir -pv / mogdata / data
mkdir: created directory `/ mogdata '
mkdir: created directory `/ mogdata / data '
[Root @ node2 ~] # mount / dev / sdb1 / mogdata / data /
[Root @ node2 ~] # chown -R mogilefs.mogilefs / mogdata /
3) Start the service
Add users, start the service (same as above, preferably using a script):
[Root @ node2 ~] # useradd -r mogilefs
[Root @ node2 ~] # su - mogilefs -c "mogstored -c /etc/mogilefs/mogstored.conf --daemon"
Installation MogileFS-Utils management MogileFS
MogileFS-Utils installed in the control node can be used to add host, device, domain, class and other management operations.
[Root @ node1 ~] # cpan MogileFS :: Utils
or
[Root @ node1 ~] # wget http://mirrors.ustc.edu.cn/CPAN/authors/id/D/DO/DORMANDO/MogileFS-Utils-2.29.tar.gz
[Root @ node1 ~] # tar xf MogileFS-Utils-2.29.tar.gz
[Root @ node1 ~] # cd MogileFS-Utils-2.29
[Root @ node1 MogileFS-Utils-2.29] # perl Makefile.PL
[Root @ node1 MogileFS-Utils-2.29] # make && make install
We recommend direct use cpan install automatically install dependent packages.
Nginx-based access MogileFS
Storge Node support when the stored data to the interface output above http mode, the client interacts with Storge Node usually required after API-based programming MogileFS provided by the program and storge communications. nginx_mogilefs nginx on module is one such program, using nginx as a reverse proxy MogileFS, resources, as long as the user needs to access the form url to nginx request, both to get the resources needed.
nginx_mogilefs works: mogielfs nginx as a third-party module that automatically work on nginx, decoding user request url, as part of the url domain, as a key part of the return to send to the Tracker, Tracker will be retrieved fid mogielfs module, mogielfs use this fid and Storge Nod interactive access to data, and the data obtained is returned to the client. nginx can accept user request to upload files to the corresponding url resolves domain and key, and to store files on a node according to the instructions stored in the Tracker.
Download nginx_mogilefs_module modules: http: //www.grid.net.ru/nginx/mogilefs.en.html
Configure multiple Tracker, use nginx load balancing, it is recommended in the actual environment in the database node configured as high-availability cluster configuration is only here for the convenience of a node.
Configuration
1) First, the four node deployment Tracker and Storage in accordance with the above-described manner, and connected to the database Tracker
Two Tracker connected to the same database.
2) Configuration MogileFS Environment
Add each device to Tracker:
[Root @ node1 ~] # mogadm --trackers = 192.168.1.126: 7001 host add node1 --ip = 192.168.1.106 --status = alive
[Root @ node1 ~] # mogadm --trackers = 192.168.1.126: 7001 host add node2 --ip = 192.168.1.126 --status = alive
[Root @ node1 ~] # mogadm --trackers = 192.168.1.106: 7001 host add node3 --ip = 192.168.1.127 --status = alive
[Root @ node1 ~] # mogadm --trackers = 192.168.1.106: 7001 host add node4 --ip = 192.168.1.131 --status = alive
[Root @ node1 ~] # mogadm --trackers = 192.168.1.106: 7001 host list # view the added host
Add Device (docroot directory parameter specifies the need to create the corresponding directory in the configuration file, such as adding a device number is 1, then the directory name dev1 added):
[Root @ node1 ~] # mogadm --trackers = 192.168.1.106: 7001 device add node1 1 --status = alive
[Root @ node1 ~] # mogadm --trackers = 192.168.1.106: 7001 device add node2 2 --status = alive
[Root @ node1 ~] # mogadm --trackers = 192.168.1.106: 7001 device add node3 3 --status = alive
[Root @ node1 ~] # mogadm --trackers = 192.168.1.106: 7001 device add node4 4 --status = alive
Create the corresponding directory on each Storage Node:
Storage Node1 on
[Root @ node1 ~] # mkdir / mogdata / data / dev1 /
[Root @ node1 ~] # chown -R mogilefs.mogilefs / mogdata /
##################
On Storage Node2
[Root @ node2 ~] # mkdir / mogdata / data / dev2 /
[Root @ node1 ~] # chown -R mogilefs.mogilefs / mogdata /
.........
.........
Check Status: mogadm --trackers = 192.168.1.106: 7001,192.168.1.126: 7001 check
Add namespace:
[Root @ node1 ~] # mogadm --trackers = 192.168.1.106: 7001 domain add file
Adding class:
[Root @ node1 ~] # mogadm --trackers = 192.168.1.106: 7001 class add file bigimg
3) Installation and Configuration nginx
[Root @ www ~] # tar xf nginx-1.8.0.tar.gz
[Root @ www ~] # tar xf nginx_mogilefs_module-1.0.4.tar.gz # using version 1.0.4
[Root @ www ~] # yum install pcre-devel
[Root @ www ~] # cd nginx-1.8.0
[Root @ node2 ~] # ./configure \
--prefix = / usr / local / nginx \
--conf-path = / etc / nginx / nginx.conf \
--error-log-path = / var / log / nginx / error.log \
--http-log-path = / var / log / nginx / access.log \
--pid-path = / var / run / nginx / nginx.pid \
--lock-path = / var / lock / nginx.lock \
--user = nginx \
--group = nginx \
--with-http_ssl_module \
--with-http_flv_module \
--with-http_stub_status_module \
--with-http_gzip_static_module \
--http-client-body-temp-path = / usr / local / nginx / client / \
--http-proxy-temp-path = / usr / local / nginx / proxy / \
--http-fastcgi-temp-path = / usr / local / nginx / fcgi / \
--http-uwsgi-temp-path = / usr / local / nginx / uwsgi \
--http-scgi-temp-path = / usr / local / nginx / scgi \
--with-pcre \
--with-debug \
--add-module = / root / nginx_mogilefs_module-1.0.4 \ # add third-party modules
###############################
[Root @ www nginx-1.8.0] # make && make install
[Root @ www nginx-1.8.0] # useradd -r nginx
Add service script:
.................
[Root @ www ~] # chmod + x /etc/init.d/nginx
[Root @ www ~] # chkconfig --add nginx
Configuring nginx.conf, Tracker no need to maintain user sessions in front of the nginx load balancing simple weighted round robin can do:
[Root @ node2 nginx] # vim nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
include mime.types;
default_type application / octet-stream;
sendfile on;
keepalive_timeout 65;
upstream trackers {
server 192.168.1.106:7001 weight = 1; # specify two Tracker
server 192.168.1.126:7001 weight = 1;
}
server {
listen 80;
server_name localhost;
location / file {
mogilefs_tracker trackers;
# Specify domain; mogilefs_domain file
mogilefs_pass {
proxy_pass $ mogilefs_path;
proxy_hide_header Content-Type;
proxy_buffering off;
}
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root html;
}
}
}
After configuration, start the service.
[Root @ www ~] # service nginx start
#########################
[Root @ www ~] # ansible tracker -m service -a 'name = mogilefsd state = started enabled = yes'
[Root @ www ~] # ansible storage -m service -a 'name = mogstored state = started enabled = yes'
carry out testing
[Root @ www ~] # mogupload --trackers = 192.168.1.106: 7001 --domain = file --class = bigimg --key = 'hello.jpg' --file = '/ root / Public / 24_P.jpg'
Stop mogilefsd and mogstored service 192.168.1.106 on:
[Root @ node1 ~] # service mogilefsd stop
Stopping mogilefsd [OK]
[Root @ node1 ~] # service mogstored stop
Stopping mogstored [OK] |
|
|
|