Home IT Linux Windows Database Network Programming Server Mobile  
           
  Home \ Database \ MongoDB3.0.6 wiredtiger MMAPv1     - Use JMS listener Oracle AQ, trigger the execution of Java programs in the database changes (Database)

- C / C ++ language variable scope: local variables, global variables, file-level variables (Programming)

- How a lot of blocking malicious IP address in Linux (Linux)

- Configuring xdebug debugging environment in Ubuntu 14.04 under the PhpStorm (Linux)

- Windows Desktop use VNC remote connect Linux (Linux)

- Try the command ip, ifconfig is obsolete under Linux (Linux)

- Linux kernel likely and unlikely to resolve macro definitions (Linux)

- To teach you a trick to find the real IP address (Linux)

- To install the Oracle 10.2.0.1.0 process notes on Oracle Linux 4u4 (Database)

- Git remote repository Explanation (Linux)

- Django 1.8 TEMPLATE_DIRS configuration and STATICFILES_DIRS (Server)

- CentOS install expect (Linux)

- Linux Network Programming - non-blocking program (Programming)

- Ubuntu 14.04 to install file editor KKEdit 0.1.5 version (Linux)

- Linux system security infrastructure Highlights (Linux)

- U disk to install Windows and Ubuntu 15.04 dual system (Programming)

- Ambari and Hadoop configuration, management and monitoring of the project Getting (Server)

- MySQL display operation control tips (Database)

- C ++ precision performance test function (Programming)

- Postmodern systems programming language (Programming)

 
         
  MongoDB3.0.6 wiredtiger MMAPv1
     
  Add Date : 2017-08-31      
         
       
         
  One, the business needs:
    An APP application database companies have achieved a vertical split logs with the business, the original set of RAC, split into two, the current database yet relatively stable, server load is also within the normal range, but existing users the number of 4.5 million daily active users reached 100 million and daily logs produce 10 million records, the amount of data 100G, and the target number of users approaching 18 million, the estimated daily database will then reach 60 million / article, and need to meet single record query needs, plans to use MongoDB to replace Oracle RAC, now comparing the test write MongoDB WiredTiger engine and MMAPv1 engine.
 
Two, MongoDB3.0.x version properties (with respect MongoDB2.6 and below):
Increased wiredtiger engine:
Open source storage engine;
Support multi-core CPU, full use of memory / cache chip level (Note: October 14 just released the 3.0.7 version, to further improve the performance of memory operations);
Based on B-TREE and LSM algorithm;
Provide document-level locking (document-level concurrency control), similar to row-level locking relational databases;
Support file compression (compression mechanism wherein snappy 5% extra CPU consumption, 70% reduction in the use of space, the compression ratio can also be adjusted according to need), three compression type:
No compression;
Snappy Compression: The default compression, Snappy is Google internal production environment used by many projects compression library, including BigTable, MapReduce and RPC, etc., compression faster than Zlib faster, but the size of the compressed document processing than Zlib 20% larger -100%, Snappy for the text of the compression ratio from 1.5 to 1.7, for HTML is 2-4 for JPEG, PNG and other already compressed data compression ratio of 1.0. In I7 i7 5500u single core CPU test, the compression performance in 200M / s-500M / s;
Zlib Compression: Z1ib is a free, universal, cross-platform, without any legal obstacles, lossless data compression library development, with respect to the Snappy compression, high consumption of CPU performance, compression slow, but good compression effect.
MMAPv1 engine (MongoDB2.6 and below using MMAP engine):
Lock collection (collection-level concurrency control), in the MMAP version, only the database lock (only when a user of a collection operation, the other collection has also been suspended);
Seamless Migration (MMAP version of the data can be migrated to the online version MMAPV1 can also migrate to wiredtiger engine).
Pluggable Storage Engine API (similar to mysql multi-engine driven)
To meet the different needs of data through different data engine;
Future expansion more scenes.
Ops manager (MongoDB Enterprise Edition provides features)
Living web graphical management interface
Reduce the daily work routine and configuration
 
Fourth, the test environment:
Server hardware configuration: 172.16.16.169 CPU: 2 * E5620 @ 2.40GHz, Memory: 8G

Client Hardware Configuration: 172.16.40.92 CPU: 4 * I5-4300U @ 1.90GHz, memory: 4G

Database: mongodb V3.0.6

Development Environment: python3.4.3, pycharm4.5.4

Test Prerequisite: the mongodb, python has been installed, pycharm
 
 
Five test scenario: 1 million insertion time consumption data comparison;
      1. Start the MongoDB server side (default MMAPv1 engine, the default port is 27017, the start MongoDB MongoDB installation directory, define the data and the log path):
#. / Mongod --dbpath = / data / db --logpath = / data / log
      2. Log MongoDB:
#. / Mongo
      3. To view the current engine status:
> Db.serverStatus ()
"StorageEngine": {
                "Name": "mmapv1"
      4. On the server side, start the MongoDB (WiredTiger engine, port: 11111, start in MongoDB MongoDB installation directory, define the data and the log path defined WiredTiger engine):
#. / Mongod --dbpath = / data / wiredtiger --logpath = / data / wiredtiger / log
      5. Log on to start the engine WiredTiger Mongodb:
#. / Mongo 127.0.0.1:11111
      6. Review the current engine status:
> Db.serverStatus ()
"StorageEngine": {
                "Name": "wiredTiger"
 
      7. python connection MongoDB MMAPv1 write configuration:
import time
import datetime
import timeit
import math
ISOTIMEFORMAT = '% Y-% m-% d% X'

from pymongo import MongoClient
mc = MongoClient ( "172.16.16.169", 27017)
db = mc.users
def dateDiffInSeconds (date1, date2):
    timedelta = date2 - date1
    return timedelta.days * 24 * 3600 + timedelta.seconds
date1 = datetime.datetime.now ()
db.users.drop ()
for i in range (0,1000000): db.users.insert ({ "name": "ljai", "age": i, "addr": "fuzhou"})
c = db.users.find (). count ()
print ( "count is", c)
date2 = datetime.datetime.now ()
print (date1)
print (date2)
print (dateDiffInSeconds (date1, date2), "seconds")
mc.close ()
  8. Test results: 1 million data write takes 12 minutes 28 seconds:


Python connection MongoDB WiredTiger write configuration:

import time
import datetime
import timeit
import math
ISOTIMEFORMAT = '% Y-% m-% d% X'

from pymongo import MongoClient
mc = MongoClient ( "172.16.16.169", 27017)
db = mc.users
def dateDiffInSeconds (date1, date2):
    timedelta = date2 - date1
    return timedelta.days * 24 * 3600 + timedelta.seconds
date1 = datetime.datetime.now ()
db.users.drop ()
for i in range (0,1000000): db.users.insert ({ "name": "ljai", "age": i, "addr": "fuzhou"})
c = db.users.find (). count ()
print ( "count is", c)
date2 = datetime.datetime.now ()
print (date1)
print (date2)
print (dateDiffInSeconds (date1, date2), "seconds")
mc.close ()
 9. Test results: 1 million data write took 10 minutes and 3 seconds:

1 million data write, WireTiger engine faster than MMAPv1, follow-up and then continue with other aspects of the performance tests.
     
         
       
         
  More:      
 
- Java, boolean operators & =, | = ^ = use (Programming)
- Memory leak analysis using Android studio (Programming)
- Ubuntu compiler installation R Full Record (Linux)
- How to monitor Linux system performance Nmon (Linux)
- MySQL optimization of the relevant Group By (Database)
- MySQL my.cnf sql_mode schema modifications (Database)
- Try to use Lets Encrypt (Linux)
- You may not know the Linux command-line network monitoring tool (Linux)
- Linux performance optimization tools perf top (Linux)
- How to use the TF / SD card making Exynos 4412 u-boot boot disk in Ubuntu (Linux)
- Use CutyCapt to convert HTML pages to png images on Linux (Linux)
- UNIX and class UNIX system security check notes (Linux)
- Upgrading Oracle 11.2.0.1.0 to 11.2.0.3.0 (Database)
- How to find on Linux and delete duplicate files: FSlint (Linux)
- Linux Command - ps: a snapshot of the current process (Linux)
- Run Windows in FreeBSD bhyve (Linux)
- Linux asynchronous read documents by AIO (Programming)
- High-performance JavaScript loops and flow control (Programming)
- Linux (SUSE) mount NTFS mobile hard practice (Linux)
- osprofiler use OpenStack Cinder Lane (Server)
     
           
     
  CopyRight 2002-2016 newfreesoft.com, All Rights Reserved.