Home PC Games Linux Windows Database Network Programming Server Mobile  
           
  Home \ Server \ Amazon EC2 server using Rsync + Inotify real-time synchronization     - SQL Server 2008 database synchronization Notes (Database)

- Linux system started to learn: Teaches you install Ubuntu 15.04 on VirtualBox (Linux)

- Use innobackupex full realization of MySQL hot backup (Database)

- SpringMVC garbage processing (Programming)

- OpenSSL Introduction and compilation steps on Windows, Linux, Mac systems (Linux)

- LVM management reduces swap partition space to the root partition (Linux)

- sed and awk in shell usage and some examples (Linux)

- Standardized UNIX command-line tool (Linux)

- Transfer files and permissions from Windows to Linux system by Samba (Linux)

- Windows SmartGit installation (Linux)

- Linux System Getting Started Learning: From VirtualBox from the client host access NAT (Linux)

- Git / Github use notes (Linux)

- Github Remote Assistance (Linux)

- Windows 7 hard disk to install Ubuntu 14.10 (Linux)

- Memory leak analysis using Android studio (Programming)

- RHEL5 / 6 Installation Notes (Linux)

- Lua4.0 interpreter entrance (Programming)

- How Vim playing a mature IDE (Linux)

- Getting Started with Linux system to learn: how to get the process ID (PID) in the script (Linux)

- Security: set limits on password (Linux)

 
         
  Amazon EC2 server using Rsync + Inotify real-time synchronization
     
  Add Date : 2018-11-21      
         
         
         
  Background:

Amazon needs to synchronize data between two servers, just pass on one machine, you can automatically sync to the other machine.

Install Rsync:

CentOS 6.7 with automatic on Rsync, does not require installation.

##Case Analysis
It is assumed that there are two servers: A and B. Where A is the primary web server (155.28.81.0), B is the server from the server (155.28.82.0). We want A server / home / test / backed up to the server B / home / test / directory.

A server configuration ##
#### A compilation server installation
Compile rsync installation is very simple, just the following simple steps:

[Root @ www ~] # cd / usr / local / src /
[Root @ www src] # wget http://rsync.samba.org/ftp/rsync/src/rsync-3.0.9.tar.gz
[Root @ www src] # tar zxvf rsync-3.0.9.tar.gz
[Root @ www src] # cd rsync-3.0.9
[Root @ www rsync-3.0.9] # ./configure --prefix = / usr / local / rsync /
[Root @ www rsync-3.0.9] # make
[Root @ www rsync-3.0.9] # make install But note that rsync must be installed on servers A and B, based on server mode (passive) run rsync server on which the A, B and places the customer end mode (active) run rsync. Thus runs on web server A rsync daemon running on the client B timed program to back up the contents of the web server A to be backed up.

#### To create a user and password authentication file

[Root @ www rsync-3.0.9] # echo "backup: bk_passwd"> /usr/local/rsync/rsyncd.passwd Remember password file server end establishment, including user name and password, and the client side password file created only password, no user name.

#### Is set to read-only permissions

[Root @ www rsync-3.0.9] # cd / usr / local / rsync
[Root @ www rsync] # chmod 600 rsyncd.passwd might otherwise error:
@ERROR: Auth failed on module ***
rsync error: error starting client-server protocol (code 5) at main.c (1503)

#### Establishing rsync profile

[Root @ www rsync] # vi /usr/local/rsync/rsyncd.conf
uid = root
gid = root
use chroot = no
max connections = 4
strict modes = yes
hosts allow = 121.42.46.213 # spaces can allow multiple
port = 873
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
log file = /var/log/rsyncd.log

[Test]
path = / home / test
ignore errors
read only = true
list = false
auth users = backup
secrets file = /usr/local/rsync/rsyncd.passwd#### way to start rsync daemon server

[Root @ www rsync] # rsync --daemon --config = / usr / local / rsync / rsyncd.confrsync default service port 873, the port server receiving client or anonymous authentication mode backup request.

#### If you want service is set from the start, you can add rc.local

Edit /etc/rc.d/rc.local, add at the end:

/ Usr / local / rsync / bin / rsync --daemon --config = / etc / rsyncd.conf

## Client Configuration B

Compile and install Ibid., Generic error occurs in the server B

#### Establishing access server A password authentication file

[Root @ www rsync] # echo "bk_passwd"> /usr/local/rsync/rsync.passwd#### set permissions to read-only

[Root @ www rsync] # chmod 0600 rsync.passwd #### after rsync installed, run the following command to synchronize backup

[Root @ www rsync] # rsync -vzrtopg --delete --progress --password-file = / usr / local / rsync / rsync.passwd backup@115.28.81.0 :: test / home / test wherein address backup@115.28. 81.0 :: test, backup server a user, 115.28.81.0 server a IP address or domain name, test server a configuration module.

The above command line -vzrtopg where v is verbose, z is compressed, r is recursive, topg original file attributes are maintained in the case of the main parameters, time, - progress means showing details of the progress - -delete means that if the server to delete this file, then the client is also appropriate to delete the file, keep it real consensus. --password-file = / usr / local / rsync / rsync.passwd to specify a password file, so that you can interactively without having to enter a password to use in the script, to note here is this property to the password file permissions set to be readable only by root.

Here the contents of the backup stored in the backup machine / home / test / directory.

#### Install inotify

[Root @ www rsync] # cd / usr / local / src /
[Root @ www src] # wget http://cloud.github.com/downloads/rvoicilas/inotify-tools/inotify-tools-3.14.tar.gz
[Root @ www src] # tar zxvf inotify-tools-3.14.tar.gz
[Root @ www src] # cd inotify-tools-3.14
[Root @ www inotify-tools-3.14] # ./configure --prefix = / usr / local / inotify
[Root @ www inotify-tools-3.14] # make
[Root @ www inotify-tools-3.14] # make install

Rsync synchronization scripts:

#! / Bin / bash
#variables
current_date = $ (date +% Y% m% d_% H% M% S)
source_path = / home / www /
log_file = / usr / local / rsync / rsync_client.log
  
#rsync configuration
rsync_server = 115.28.81.0
rsync_module = www
rsync_user = backup
rsync_pwd = / usr / local / rsync / rsync.passwd
INOTIFY_EXCLUDE = '(.. * / * Log |.. * / * Swp) $'
INOTIFY_EXCLUDE_LIST = '/ usr / local / inotify / inotify_exclude.lst'
RSYNC_EXCLUDE = '/ etc / rsync_exclude.list'
  
#rsync client pwd check
if [! -e $ {rsync_pwd}]; then
    echo -e "rsync client passwod file $ {rsync_pwd} does not exist!"
    exit 0
fi
  
#inotify_function
#This Function is used to monitor folder (/ home / www) files, but exclude subfolders (storage, bootstrape / cache).
inotify_fun () {
    / Usr / local / inotify / bin / inotifywait -mrq --timefmt '% Y /% m /% d-% H:% M:% S' --format '% T% w% f'
          --exclude $ {INOTIFY_EXCLUDE} --fromfile $ {INOTIFY_EXCLUDE_LIST} -e modify, delete, create, move, attrib $ {source_path}
          | While read file
      do
          / Usr / bin / rsync -auvrtzopgP --exclude-from = $ {RSYNC_EXCLUDE} --progress --bwlimit = 500 --password-file = $ {rsync_pwd} $ {source_path} $ {rsync_user} @ $ {rsync_server}: : $ {rsync_module}
      done
}
#inotify log
inotify_fun >> $ {log_file} 2> & 1

Wherein INOTIFY_EXCLUDE_LIST = '/ usr / local / inotify / inotify_exclude.lst' as follows:
[Ec2-user @ ip-172-31-7-248 rsync] $ cat /usr/local/inotify/inotify_exclude.lst
/ Home / www
@ / Home / www / www.xxx.com / storage
@ / Home / www / www.xxx.com / bootstrap / cache
@ / Home / www / xxx.com / storage

RSYNC_EXCLUDE = '/ etc / rsync_exclude.list'

[Ec2-user @ ip-172-31-7-248 rsync] $ cat /etc/rsync_exclude.list
www.xxx.com/storage
www.xxx.com/bootstrap/cache
xxx.com/storage
     
         
         
         
  More:      
 
- Java Generics (Programming)
- Schema snapshot rollback (Database)
- Under CentOS Linux automatic backup MySQL database daily (Database)
- Switching Virtual Machine + Ubuntu graphical interface and terminal interface (Linux)
- Analysis JavaBean (Programming)
- Linux scheduling summary (Linux)
- UNIX how to restrict users by IP Telnet (Linux)
- Elaborate 10-point difference between the new and malloc (Programming)
- Use pg_basebackup build PostgreSQL streaming replication environment (Database)
- Oracle archive log full cause abnormal slow database performance (Database)
- Installation Enpass secure password manager on Ubuntu (Linux)
- Android in the event delivery and handling mechanism (Programming)
- Unable to solve the official version of Android Studio online update problem (Linux)
- SQL Beginner Guide (Database)
- How to add a new hard disk without restarting the CentOS 7 / RHEL 7 virtual machine (Linux)
- Ubuntu Series Installation Docker (Linux)
- Unix average load average load calculation method (Server)
- Oracle 11G using DG Broker create DataGuard (Database)
- Linux using DenyHosts prevents ssh cracks (Linux)
- tar command to extract a file error: stdin has more than one entry (Linux)
     
           
     
  CopyRight 2002-2022 newfreesoft.com, All Rights Reserved.