Home IT Linux Windows Database Network Programming Server Mobile  
           
  Home \ Database \ MongoDB3.0.6 wiredtiger MMAPv1     - A drop datafile Oracle bug (Database)

- Heartbeat (v1, v2, pacemaker) cluster components Overview (Server)

- Ubuntu users to install the system indicator SysPeek 0.3 (Linux)

- MySQL multi-master-slave synchronization (Database)

- Linux /var/spool/ insufficient clientmqueue space solutions (Linux)

- RHEL 6.6 install GNOME desktop environment (Linux)

- Examples of Python any parameters (Programming)

- ogg start being given libnnz11.so: can not open shared object file (Database)

- Linux Command Tutorial: du command to view disk space (Linux)

- To install PostgreSQL 9.4 (Database)

- Linux System Getting Started Tutorial: how to find information on Linux-embedded module (Linux)

- Linux open handle limit adjustment (Linux)

- Git delete files (Linux)

- Creating and extracting archives 11 tar command examples in Linux (Linux)

- How to manage and use Logical Volume Management LVM in Ubuntu (Linux)

- Linux kernel update error, update-initramfs: failed Solution (Linux)

- Linux process stack and process function stack frame (Linux)

- SVN hook code set to synchronize Web directory (Server)

- How to use systemd timer (Linux)

- Solaris 10 nagios monitoring system (Linux)

 
         
  MongoDB3.0.6 wiredtiger MMAPv1
     
  Add Date : 2017-08-31      
         
       
         
  One, the business needs:
    An APP application database companies have achieved a vertical split logs with the business, the original set of RAC, split into two, the current database yet relatively stable, server load is also within the normal range, but existing users the number of 4.5 million daily active users reached 100 million and daily logs produce 10 million records, the amount of data 100G, and the target number of users approaching 18 million, the estimated daily database will then reach 60 million / article, and need to meet single record query needs, plans to use MongoDB to replace Oracle RAC, now comparing the test write MongoDB WiredTiger engine and MMAPv1 engine.
 
Two, MongoDB3.0.x version properties (with respect MongoDB2.6 and below):
Increased wiredtiger engine:
Open source storage engine;
Support multi-core CPU, full use of memory / cache chip level (Note: October 14 just released the 3.0.7 version, to further improve the performance of memory operations);
Based on B-TREE and LSM algorithm;
Provide document-level locking (document-level concurrency control), similar to row-level locking relational databases;
Support file compression (compression mechanism wherein snappy 5% extra CPU consumption, 70% reduction in the use of space, the compression ratio can also be adjusted according to need), three compression type:
No compression;
Snappy Compression: The default compression, Snappy is Google internal production environment used by many projects compression library, including BigTable, MapReduce and RPC, etc., compression faster than Zlib faster, but the size of the compressed document processing than Zlib 20% larger -100%, Snappy for the text of the compression ratio from 1.5 to 1.7, for HTML is 2-4 for JPEG, PNG and other already compressed data compression ratio of 1.0. In I7 i7 5500u single core CPU test, the compression performance in 200M / s-500M / s;
Zlib Compression: Z1ib is a free, universal, cross-platform, without any legal obstacles, lossless data compression library development, with respect to the Snappy compression, high consumption of CPU performance, compression slow, but good compression effect.
MMAPv1 engine (MongoDB2.6 and below using MMAP engine):
Lock collection (collection-level concurrency control), in the MMAP version, only the database lock (only when a user of a collection operation, the other collection has also been suspended);
Seamless Migration (MMAP version of the data can be migrated to the online version MMAPV1 can also migrate to wiredtiger engine).
Pluggable Storage Engine API (similar to mysql multi-engine driven)
To meet the different needs of data through different data engine;
Future expansion more scenes.
Ops manager (MongoDB Enterprise Edition provides features)
Living web graphical management interface
Reduce the daily work routine and configuration
 
Fourth, the test environment:
Server hardware configuration: 172.16.16.169 CPU: 2 * E5620 @ 2.40GHz, Memory: 8G

Client Hardware Configuration: 172.16.40.92 CPU: 4 * I5-4300U @ 1.90GHz, memory: 4G

Database: mongodb V3.0.6

Development Environment: python3.4.3, pycharm4.5.4

Test Prerequisite: the mongodb, python has been installed, pycharm
 
 
Five test scenario: 1 million insertion time consumption data comparison;
      1. Start the MongoDB server side (default MMAPv1 engine, the default port is 27017, the start MongoDB MongoDB installation directory, define the data and the log path):
#. / Mongod --dbpath = / data / db --logpath = / data / log
      2. Log MongoDB:
#. / Mongo
      3. To view the current engine status:
> Db.serverStatus ()
"StorageEngine": {
                "Name": "mmapv1"
      4. On the server side, start the MongoDB (WiredTiger engine, port: 11111, start in MongoDB MongoDB installation directory, define the data and the log path defined WiredTiger engine):
#. / Mongod --dbpath = / data / wiredtiger --logpath = / data / wiredtiger / log
      5. Log on to start the engine WiredTiger Mongodb:
#. / Mongo 127.0.0.1:11111
      6. Review the current engine status:
> Db.serverStatus ()
"StorageEngine": {
                "Name": "wiredTiger"
 
      7. python connection MongoDB MMAPv1 write configuration:
import time
import datetime
import timeit
import math
ISOTIMEFORMAT = '% Y-% m-% d% X'

from pymongo import MongoClient
mc = MongoClient ( "172.16.16.169", 27017)
db = mc.users
def dateDiffInSeconds (date1, date2):
    timedelta = date2 - date1
    return timedelta.days * 24 * 3600 + timedelta.seconds
date1 = datetime.datetime.now ()
db.users.drop ()
for i in range (0,1000000): db.users.insert ({ "name": "ljai", "age": i, "addr": "fuzhou"})
c = db.users.find (). count ()
print ( "count is", c)
date2 = datetime.datetime.now ()
print (date1)
print (date2)
print (dateDiffInSeconds (date1, date2), "seconds")
mc.close ()
  8. Test results: 1 million data write takes 12 minutes 28 seconds:


Python connection MongoDB WiredTiger write configuration:

import time
import datetime
import timeit
import math
ISOTIMEFORMAT = '% Y-% m-% d% X'

from pymongo import MongoClient
mc = MongoClient ( "172.16.16.169", 27017)
db = mc.users
def dateDiffInSeconds (date1, date2):
    timedelta = date2 - date1
    return timedelta.days * 24 * 3600 + timedelta.seconds
date1 = datetime.datetime.now ()
db.users.drop ()
for i in range (0,1000000): db.users.insert ({ "name": "ljai", "age": i, "addr": "fuzhou"})
c = db.users.find (). count ()
print ( "count is", c)
date2 = datetime.datetime.now ()
print (date1)
print (date2)
print (dateDiffInSeconds (date1, date2), "seconds")
mc.close ()
 9. Test results: 1 million data write took 10 minutes and 3 seconds:

1 million data write, WireTiger engine faster than MMAPv1, follow-up and then continue with other aspects of the performance tests.
     
         
       
         
  More:      
 
- lack of SWAP space during installation of Oracle (Database)
- Dockerfile use to build a mirror-based CentOS 7 (Linux)
- Two classic macro definition under Linux (Linux)
- Why do you need close contact Rust 1.0 (Programming)
- Calling Qt libraries to implement functional processes of some summary (Programming)
- Composition and dynamic memory allocation C program (Programming)
- Oracle Linux 5.9 configuration Xmanager (Linux)
- Experience RHEL7 new features (Linux)
- apt-get install openstack pkg Troubleshooting (Linux)
- Linux, see picture not resolve the problem (Linux)
- Oracle 11g can not export a variety of empty table solution (Database)
- How to choose the correct HTTP status code (Server)
- Matters Oracle 11.2 single instance when connecting ASM need to pay attention and deal with the problem (Database)
- Linux OOM killer mechanism (Linux)
- Forwarding module with Apache reverse proxy server (Server)
- MySQL binary packages install for RedHat Linux Enterprise 6.4 (Database)
- Four IDS intrusion detection tool under Linux environment (Linux)
- How to use Aptik to backup and restore Apps/PPAs under ubuntu (Linux)
- Linux security configuration (Linux)
- Linux system security knowledge (Linux)
     
           
     
  CopyRight 2002-2016 newfreesoft.com, All Rights Reserved.