Home PC Games Linux Windows Database Network Programming Server Mobile  
           
  Home \ Database \ MongoDB 3.2 Cluster Setup     - HTTP Client Hints Introduction (Server)

- VMware virtual machine operating system log Error in the RPC receive loop resolve (Linux)

- Story timestamp and time zones: daily programmer (Programming)

- SteamOS installation under Ubuntu 14.04 (Linux)

- linux smartd [FAILED] appears at startup (Linux)

- Kubernetes Cluster Setup problems encountered and solutions (Server)

- Protect your files, modify the Linux value Umask (Linux)

- XenServer virtual machines installed in dual-card configuration (Server)

- Easy to install CentOS 6.6 desktop environment (Linux)

- Server Security Analysis attack on Linux (Linux)

- Create your own YUM repository (Linux)

- History and Statistics tuptime use tools to view Linux server system boot time (Server)

- How Glances monitoring system on Ubuntu (Linux)

- Linux basis: a comprehensive study pwd command (Linux)

- Install Git on CentOS (Linux)

- VirtualBox virtual machine to install Linux (Linux)

- Shell Programming Regular Expressions (Programming)

- Linux some lessons learned about network security (Linux)

- Ubuntu 14.04 and derivative versions of the user install Pantheon Photos 0.1 (Linux)

- How to install CentOS CentOS web page (Server)

 
         
  MongoDB 3.2 Cluster Setup
     
  Add Date : 2018-11-21      
         
         
         
  First, the theoretical knowledge clusters

1.1 cluster member

MongoDB clusters similar GreenPlum cluster, with the results of the statistical distribution of the node is responsible for a task entry, fragmentation nodules responsible for the implementation tasks. Different GP, one more config servers.

Cluster has three components:
A. shards: fragmentation, that is, data nodes, storing data and perform calculations. In order to ensure high availability and data consistency, production environment shards should be made
        replicasets (to prevent loss of data). Cluster has a primary shards, perform non-fragmented tasks.
B. mongos (query routers): query routing, responsible for client connections, and the task given to shards, and then collect the results. A cluster can have more than one query
         routers (replica sets), to share client requests (load balancing).
C. config server: configure the server. Save metadata cluster (for example, data on which shards), query router config server configuration by letter
      Interest decided to assign the task to which shards. 3.2 From the start, config servers can be made replica sets.

1.2 data fragmentation

MongoDB collection level for data fragmentation to shard key to fragmentation.
shard key should be an index field or a composite index field.
MongoDB create chunks according shard key, then chunks balanced distributed over shards.
Two kinds of partitioning: range and hash Results in Oracle range partitioning and hash partitioning understanding about.
 
1.3 Performance slice

range fragmentation: for range query, can use the batch I / O. However, if the partition key is not evenly distributed, you may only use less data node cluster, you can not play to the performance of the cluster.
hash slice: on each node data evenly distributed. But the scope of the query when read together each node, the efficiency is lower than the range fragmentation.
MongoDB supports data tags, users can specify the data on a particular node to a label distribution.
1.4 balance data

1.4.1 division (splite)

    When the maximum size of a large chunk of the more than chunk when the chunk will be split into two. This process only modify the metadata, not data migration. This process is a background process is complete, it does not need to be performed manually. chunk defaults 64M.

Split features:
    1) The default chunk is 64M. The advantage is that small chunk of data balance, the downside is that often do data balance. A large chunk of the opposite
    2) Only insert or update occurs only when split chunk. If artificially change the size of the chunk is split does not occur.
    3) a specific shard key values of the chunk size may default chunk size, and can not be split
 
1.4.2 equilibrium (balance)

    If a node in the cluster to save too much data, it will automatically assign data to other nodes. This process is a background process is complete, it does not need to be performed manually.
At equilibrium data, starting with the original node copy data to a new node, and then update the replicated data. Only after the completion of the balance, it will delete the data on the original node. If an error occurs, the data will not affect the original node.
    Add and delete nodes can cause balance data.
    Note: Not all data are automatically uneven balance, in general, at least one node can store hundreds chunk (a chunk other nodes can not) without triggering a balancing operation. So no need to use a small amount of data clusters.
    Balance data features:
    1) automation, and is enabled by default
    2) a time to move a chunk, but the former may not remove the source of the chunk start new mobile
    3) Only chunk very large gap between the number of nodes when it is triggered.
4) has been moved to the number of chunk on any two nodes differs by less than 2 so far.
    5) If you set the maximum disk shard small value can be used, if the disk using a node exceeds this value will not be balance to this node.
    6) After the balance is complete, modify confige server information
    7) _secondaryThrottle default is true, at least when set into a secondary.
1.5 config server

1.5.1 config serve configured to limit replicatsion sets of

    config servers configured to replicate sets has the following limitations:
        1) no arbitration node
        2) there is no delay node

        3) build indexes must be true
1.5.2 config server to read and write

    Write: Only cluster metadata changes are updated config server information. Such as adding, deleting nodes, split chunk. Use write majority way to write.
    Read: When mongos program restart or cluster metadata changes mongos will read metadata. Read majority use and read.
1.6 System Architecture

1.6.1 for production architecture

    For the production of architecture must ensure data redundancy, mongos, shards, config server must be made replication sets. Each cluster must have a separate config server (one config server can not use more).

Explanation:
        Do replications set of words, shard node is no data redundancy function, if the data is lost, then found again (this time the other node can work). Because of a production environment, it is necessary to do the copying. Similarly, confige server is the same, but also to do replication.
 
1.6.2. Architecture for testing

    Test architecture can do replication sets, in order to save the machine for the purpose could mongos and config server and shards put together.

1.7 shard key shards key

    Shard key is used to determine the distribution of data on which node. Slice key columns must be index key or a combination of index key, shard key can also be combined (eg key (c1, c2)). Shard key is inserted in the data can not be changed.
1.7.1 hash fragments

    hash key can be a very good uniform distribution of data on different nodes. Select as the hash key keys should have a great base, generally object_id or timestamp column as the hash key.
    For empty, hash slice automatically creates two chunk on each node by modifying the numInitialChunks shardCollection parameters to determine the number of empty chunk.
 
1.7.2 points Tablets key on performance

     Different shard key to performance have different effects.
     1) Use object_id uniqueness of this large base or as a key distribution key, all the data is discrete distributed to each node, this can significantly improve write performance clusters, suitable for storing large amounts of data applications.
     2) query the data in the cluster, the fastest way is to query contains a shard key, which can be positioned directly to the node where the data resides, or we need to perform a full table scan on all nodes. For read operations, consider the following points:
 A. The most common determination column being queried
 B. Consider these columns which the greatest impact on performance
If the last column of the selected base is relatively small, then added two key points in the wrinkled sheet, do a small collection of twelve key component to the base (in contrast oracle combination of the index).
 
Cluster 1.8 High Availability

 1.8.1 shards node

    If you do not shars node replication sets, if the node is down, this part of the data is inaccessible. Therefore, to business continuity, it is necessary to make the shards node replication sets.
1.8.2 config server

   1) and shards, if not done at this time and replication set down, the entire cluster can be used not. If you make replications sets, after which a machine goes down, it will automatically select the master node, the entire cluster can also be used. If too much downtime can not elect a node, the entire cluster can still use (read and write), but can not have a chunk operations (create chunk of movement).
   2) If you do not use the mirror replication set way to do configuration config server, after confige server goes down, you need to restart all cluster members to connect to the mirror confige server. Restart way to solve is to configure DNS
1.8.3 mongos

    mongos is the entry of the application, if mongos downtime, the application will not be able to use the cluster. So generally would also replication Set. With respect to the use of resources mongos shards and confige server is minimal, it can be on a single machine and application server release. When mongos repair downtime, automatically reads data from the config server.
 
Query 1.9 Cluster

1.9.1 Query Path

    mongos is the interface of the application. mongos by config server information query data in which node thus assigned tasks. If the results of the sort (), primary shard shards merged data is then returned to the sort mongos then returned to client.limit () operate directly on the shards. skip () will not be sent to the shards to perform.
1.9.2 mongos logo

    After connecting the cluster application, perform isMaster () command returns:
{
"Ismaster": true,
"Msg": "isdbgrid",
"MaxBsonObjectSize": 16777216,
"Ok": 1
}
Said is mongos. If msg is not isdbgrid not mongos.
 
Second, the cluster structures

Step 2.1 Cluster Bases

2.1.1 confige server configuration

    The following code is to build a three node replication sets the config server:
1) Create replications sets
mongod --configsvr --replSet configReplSet --port --dbpath
Or using a configuration file
sharding:
clusterRole: configsvr
replication:
replSetName: configReplSet
net:
port:
storage:
dbpath:
2) Initialize. Go to one of the mongo shell:
rs.initiate ({
_id: "configReplSet",
configsvr: true,
members: [
{_id: 0, host: ": "},
{_id: 1, host: ": "},
{_id: 2, host: ": "}
]
})
2.1.2 Create mongos instance (routing instance)

mongos --configdb configReplSet / , ,
2.1.3 added shards

1) connection mongos instance:
mongo --host --port
2) On one of the nodes mongos added:
sh.addShard ( "rs1 / mongodb0.example.net: 27017") (replications sets only need to add a node to rs)
sh.addShard ( "mongodb0.example.net:27017") (Single)
It may take some time to migrate data
2.1.4 Setting the slice

2.1.4.1 Setting database fragmentation

Before you set up a collection of fragments, it must be set to slice database. Connection mongos:
mongo --host --port
carried out:
sh.enableSharding ( "") or db.runCommand ({enableSharding: })
2.1.4.2 fragmented collection of settings

1) to determine the shard key collection. If you already have a collection of data, the index is created on the shard key. If there is no data, the cluster will automatically create an index for the shard key
2) will be added to the collection slice
sh.shardCollection ( ". ", shard-key-pattern)
Such as:
sh.shardCollection ( "records.people", { "zipcode": 1, "name": 1}) shard key is zipcode, if you have the same name zipcode then according to points
sh.shardCollection ( "people.addresses", { "state": 1, "_id": 1}) Ibid.
sh.shardCollection ( "assets.chairs", { "type": 1, "_id": 1}) Ibid.
sh.shardCollection ( "events.alerts", { "_id": "hashed"}) hash slice
2.1.5 Configuration config server mirroring

Note: not recommended mirroring, please use the replication sets.
Start mongod instances on each config server:
mongod --configsvr --dbpath / data / configdb --port 27019
Each routing node starts mongs, - configdb behind the connection string to be consistent
mongos --configdb cfg0.example.net:27019,cfg1.example.net:27019,cfg2.example.net:27019
2.2 Experimental

2.2.1 experimental environment

node1 192.168.75.10 config server1 (configRS 37017) mongos1 (27017) shard1 (47017)
node2 192.168.75.11 config server2 (configRS 37017) mongos2 (27017) shard2 (47017)
node3 192.168.75.12 config server3 (configRS 37017) shard3 (47017)
On the three hosts were safe mongoConfig, mongoShard, mongoRouter three examples
2.2.2. Configuration config server

1) Profile
In three configution host configuration file:
[Root @ node1 mongoConfig] # cat mongodb.config
dbpath = / usr / local / mongoConfig / data
logpath = / usr / local / mongoConfig / log / mongo.log
port = 37017
fork = true
# Master = true
replSet = configRS
configsvr = true
2) Start config server instances on three machines
[Root @ node1 bin] # ./mongod -f /usr/local/mongoConfig/mongodb.conf
about to fork child process, waiting until server is ready for connections.
forked process: 3032
child process started successfully, parent exiting
Note Open firewall port 37017
3) Initialization config server
It is connected to one of the configer server:
[Root @ node1 bin] # ./mongo --port 37017
Perform the following initialization:
> Rs.initiate ({
... _id: "ConfigRS",
... Configsvr: true,
... Members: [
... {_id: 0, host: "192.168.75.10:37017"},
... {_id: 1, host: "192.168.75.11:37017"},
... {_id: 2, host: "192.168.75.12:37017"}
...]
...});
{ "Ok": 1}
2.2.3. Configuring mongos

On each machine execute:
./mongos --configdb configRS / 192.168.75.10: 37017,192.168.75.11: 37017,192.168.75.12: 37017 --port 27017 --fork --logpath = / usr / local / mongoRouter / log / mongo.log
2.2.4. Start three shard examples

Modify the configuration file in three machines:
[Root @ node1 mongoShard] # vi mongodb.config
dbpath = / usr / local / mongoShard / data
logpath = / usr / local / mongoShard / log / mongo.log
port = 47017
fork = true
Start instances on three machines:
[Root @ node1 bin] # ./mongod -f /usr/local/mongoShard/mongodb.config
about to fork child process, waiting until server is ready for connections.
forked process: 17508
child process started successfully, parent exiting
2.2.5 would be added to the cluster shards

Connection mongos instances on a single machine
./mongo --port 27017
carried out:
sh.addShard ( "192.168.75.10:47017")
sh.addShard ( "192.168.75.11:47017")
sh.addShard ( "192.168.75.12:47017")
2.2.6 would be added to the database fragmentation

sh.enableSharding ( "testShards")
2.2.7. The collection is added slices

sh.shardCollection ( "testShards.test", { "_id": "hashed"});
2.2.8. Inserting data

On a mongos execution:
mongos> use testShards
switched to db testShards
mongos> show collections;
test
mongos> db.test.insert ({ "name": "testshrads", "msg": "ok"});
WriteResult ({ "nInserted": 1})
mongos> db.test.insert ({ "name": "testshrads2", "msg": "ok"});
WriteResult ({ "nInserted": 1})
mongos> db.test.insert ({ "name": "testshrads3", "msg": "ok"});
WriteResult ({ "nInserted": 1})
mongos> db.test.insert ({ "name": "testshrads4", "msg": "ok"});
WriteResult ({ "nInserted": 1})
mongos> db.test.insert ({ "name": "testshrads5", "msg": "ok"});
WriteResult ({ "nInserted": 1})
mongos> db.test.insert ({ "name": "testshrads6", "msg": "ok"});
WriteResult ({ "nInserted": 1})
mongos> db.test.insert ({ "name": "testshrads7", "msg": "ok"});
WriteResult ({ "nInserted": 1})
mongos> db.test.insert ({ "name": "testshrads8", "msg": "ok"});
WriteResult ({ "nInserted": 1})
mongos> db.test.insert ({ "name": "testshrads9", "msg": "ok"});
WriteResult ({ "nInserted": 1})
 
View data:
mongos> db.test.find (); { "_id": ObjectId ( "56815a0617de6d7dfc1051b5"), "name": "testshrads", "msg": "ok"} { "_id": ObjectId ( "56815a0e17de6d7dfc1051b6"), "name": "testshrads2", "msg": "ok"} { "_id": ObjectId ( "56815a1717de6d7dfc1051b8"), "name": "testshrads4", "msg": "ok"} { "_id": ObjectId ( "56815a1b17de6d7dfc1051b9"), "name": "testshrads5", "msg": "ok"} { "_id": ObjectId ( "56815a1e17de6d7dfc1051ba"), "name": "testshrads6", "msg": "ok"} { "_id": ObjectId ( "56815a2617de6d7dfc1051bc"), "name": "testshrads8", "msg": "ok"} { "_id": ObjectId ( "56815a1217de6d7dfc1051b7"), "name": "testshrads3", "msg ":" ok "} {" _id ": ObjectId (" 56815a2117de6d7dfc1051bb ")," name ":" testshrads7 "," msg ":" ok "} {" _id ": ObjectId (" 56815a2917de6d7dfc1051bd ")," name ": "testshrads9", "msg": "ok"}
 
2.2.9. Test Data

Connected to one of shards:
#. / Mongod --port 47017
carried out:
> Use testShards
switched to db testShards
> Db.test.find ();
{ "_id": ObjectId ( "56815a1217de6d7dfc1051b7"), "name": "testshrads3", "msg": "ok"}
{ "_id": ObjectId ( "56815a2117de6d7dfc1051bb"), "name": "testshrads7", "msg": "ok"}
{ "_id": ObjectId ( "56815a2917de6d7dfc1051bd"), "name": "testshrads9", "msg": "ok"}
Discovery Data fragmentation is normal.
mongoDB cluster set up is completed.
     
         
         
         
  More:      
 
- OpenSSL to generate public and private key (Linux)
- STL source code analysis - iterator each container classification (Programming)
- MySQL main and backup replication structures (using mysqld_multi) (Database)
- BackTrack (BT3, BT4) Linux installation tutorial (Linux)
- 10 really interesting Linux command (Linux)
- Linux process or thread is bound to a CPU (Programming)
- Ubuntu 14.10 / 14.04 / 12.04 How to install Kodi 14.0 RC3 (Linux)
- Lazarus IDE Start Basics Tutorial (Linux)
- 10 Nginx safety tips (Linux)
- CentOS of NFS (Server)
- Linux to achieve a simple cp command (Linux)
- HA-Federation-HDFS + Yarn cluster deployment (Server)
- Terminal multiplexing tool tmux use (Linux)
- Linux Getting Started tutorial: 3D effects and beautify your desktop (Linux)
- Docker installation under CentOS7 (Linux)
- Linux operating system security tools of the Logs (Linux)
- Linux smart small switch rfkill (Linux)
- Use matplotlib scientific drawing in Linux (Linux)
- The most concise explanation of JavaScript closures (Programming)
- Create and modify Oracle temporary table space (Database)
     
           
     
  CopyRight 2002-2020 newfreesoft.com, All Rights Reserved.