Home PC Games Linux Windows Database Network Programming Server Mobile  
  Home \ Database \ Use HugePages optimize memory performance     - Ubuntu 14.10 / 14.04 how to install Quick Start tool Mutate 2.2 (Linux)

- Common Linux system performance monitoring command (Linux)

- Start Linux ISO image directly from the hard disk (Linux)

- Build Docker based MongoDB replication cluster environment (Database)

- Migrate Oracle database files to ASM (Database)

- MongoDB study notes - polymerization (Database)

- The new features of MySQL 5.7 Generated Column (index function) (Database)

- To install and deploy Java applications under CentOS 6.5 (Linux)

- Bootable ISO image using GRUB (Linux)

- MySQL in order by inaccurate results in problems and solutions (Database)

- Linux / Unix: chroot command examples to explain (Linux)

- ORA-00845: MEMORY_TARGET not supported on this system Problem (Database)

- Cryptography development environment to build under Ubuntu (Linux)

- C ++ Supplements - References (Lvalue Reference, Rvalue Reference) (Linux)

- Ubuntu 14.04 running ASP.NET Configuration Mono + Jexus (Server)

- The Linux-based security settings Ipchains Firewall (Linux)

- Zabbix monitoring disk IO status (Server)

- ORA-01839 error caused by incorrect system date setting (Database)

- Linux remote wake the computer original code [C] (Linux)

- Linux ls command (Linux)

  Use HugePages optimize memory performance
  Add Date : 2018-11-21      
  1. Introduction

System process is accessing memory through virtual addresses, but the CPU must convert it away physical memory address can actually access memory. In order to improve the conversion efficiency, CPU caches the latest mapping between virtual memory addresses and physical memory address by the CPU and stored in a mapping table maintained. In order to maximize access speed of memory, you need to save as many mappings in the mapping table.

In Linux, the memory pages are in the form of division, by default each page is 4K, which means that if a lot of physical memory, the entry of the mapping table will very much affect the retrieval efficiency of the CPU. Because the memory size is fixed, in order to reduce entry map can be taken only way to increase the size of the page.

2. HugePages Profile

2.1 Related Concepts

HugePages in Linux2.6 kernel is introduced, mainly to provide a 4k page and larger page selection.


Concept note

page table

page table is a data structure model virtual memory system for the operating system for storing correspondence between virtual addresses and physical addresses.

When we access memory page table is accessed first, then Linux through the page table mapping to access the real physical memory (ram + swap)


A Translation Lookaside Buffer (TLB)

TLB is allocated in the cpu in a fixed-size buffer (or cache), to save part of the page table, the CPU faster access and address translation.


hugetlb is TLB entry in the record and point Hugepages.


This is a new 2.6 kernel based on the memory file system like tmpfs.

In the TLB by hugetlb to point hugepage. These are allocated hugepage as a memory file system hugetlbfs (similar tmpfs) to the processes.


2.2 HugePages significance

HugePages is a feature of the Linux kernel, it can use hugepage with a larger memory pages to replace the traditional 4K pages. Use HugePage mainly the following benefits:

1. HugePages at system startup, direct distribution and to retain the size of the corresponding memory area.

2. HugePages after the startup, if there is no administrator intervention is not going to release and change.

3. There is no swap.

Notswappable: HugePages are not swappable Therefore thereis no page-in / page-outmechanism overhead.HugePages are universally regarded aspinned..

4. greatly improves the CPU cache stored in the page table covered memory size, thereby improving the TLB hit rate.

Virtual memory addresses to connect to the process page table and then connect to the physical memory. So when accessing memory page tables need to get access to the mapping between the virtual and physical memory, and then access to physical memory.

CPU cache TLB in part used to store part of the page table to improve the conversion speed. Because the page size bigger, so the same size of the TLB, covered memory size is also larger. Improve TLB hit rate, but also improve the speed of address translation.

5. To reduce the page table load.

When XXX system performance test, if no HugePages, pagetable size on the database server is about 5G (which should also lead to the database server memory performance test when the main reason for lack of):

node74: / home / Oracle # cat / proc / meminfo

MemTotal: 16323732 kB

PageTables: 5442384kB

After configuring HugePages, pagetable size is only 124M (Performance Test when memory usage stable at around 80%):

node74: / home / oracle # cat / proc / meminfo

MemTotal: 16323732 kB

PageTables: 127384 kB

Eliminated page tablelookup overhead: because hugepage is not swappable, and all will be no page table lookups.

Faster overall memory performance: Because the virtual memory requires two steps to correspond to the actual physical memory addresses, so fewer pages, reduce the heat to access the page table, the page table to avoid hot bottlenecks.

6. improve memory performance and reduce CPU load, the principle above


2.3 HugePages need attention

1. Hugepages allocation will be set aside after, which is the sum of all SGA size than some instances on the server to a large, almost not possible.

For example Hugepages set to 8G, oracle SGA to 9G, then the oracle at startup will not use this to 8G Hugepages. This 8G wasted. So when setting Hugepages To calculate the size of the SGA, we will give back a script to calculate.

2. The other processes can not use Hugepages memory, so do not set too large, slightly bigger than the SGA SGA can be used to ensure hugepages enough.

3. meminfo and Hugepage related, there are four:

HugePages_Total: 4611

HugePages_Free: 474

HugePages_Rsvd: 467

Hugepagesize: 2048 kB

The number of pages HugePages_Total to the assigned, and Hugepagesize is obtained by multiplying the allocated memory size. 4611 * 2/1024 about 9GB

HugePages_Free the number Hugepages I never have been used before. Even oraclesga this memory has been allocated, but if not actually written, then see or Free of. This is easily misunderstood.

HugePages_Rsvd reserve has been allocated for the page number but not yet used. When Oracle has just started, most of the memory are Reserved and should be Free, with the oracle SGA use, Reserved and Free will continue to decrease.

HugePages_Free-HugePages_Rsvd this part is not being used to memory, if no other oracle instance, this memory may never be used to, which is being wasted.

4. HugePages and oracle AMM (Automatic Memory Management) are mutually exclusive, so use HugePages must set the memory parameters MEMORY_TARGET / MEMORY_MAX_TARGET zero.


3. Configure HugePages


3.1 modify kernel parameters memlock

Modify kernel parameters memlock, the unit is KB, if the memory is 16G, slightly smaller than the size of memlock of physical memory. Program lock 12GB memory size. Parameter is set to greater than SGA is no harm.

Log in as root two database servers, file editing limits.conf:

node74: ~ # vi /etc/security/limits.conf

Add the following two lines:

* Soft memlock 12582912

* Hard memlock 12582912


3.2 verify memlock limit

Root and oracle users to log in again, check memlocklimit

node74: ~ # ulimit -l


oracle @ node74: ~> ulimit -l



3.3 Disable AMM

If you are using 11G and later, AMM has been enabled by default, but the AMM and Hugepages are not compatible, you must first disable AMM. Disable AMM, follow these steps:


3.3.1 Close database instance

Oracle user has two database servers, through sqlplus closed two database instances.

oracle @ node74: ~> sqlplus / as sysdba

SQL> shutdown immediate


3.3.2 Create pfile

Users log on to the oracle one of the hosts, execute the following command to create a pfile:

oracle @ node74: ~> sqlplus / as sysdba

SQL> create pfile = '/ home / oracle / pfile.ora' fromspfile = '+ DG_ORA / orcl / spfileorcl.ora';


3.3.3 Editing pfile

Edit pfile, and delete memory_max_target memory_target parameters:

oracle @ node74: ~> vi /home/oracle/pfile.ora

Delete the following lines:

orcl1.memory_max_target = 11114905600

orcl2.memory_max_target = 11114905600

* .memory_max_target = 0

orcl1.memory_target = 11114905600

orcl2.memory_target = 11114905600

* .memory_target = 0

Save the modified file.


3.3.4 Create spfile

Execute the following command to create the spfile:

oracle @ node74: ~> sqlplus / as sysdba

SQL> create spfile = '+ DG_ORA / orcl / spfileorcl.ora'from pfile =' / home / oracle / pfile.ora ';


3.3.5 modify the system parameters kernel.shmall

Kernel.shmall maximum shared memory size is a system that can be used. Units are page (4KB). After disabling AMM, you need to modify the system parameters kernel.shmall, this parameter is set too low, it may cause the database startup failure ORA-27102 (see Appendix 4.2).

ORACLE recommended that it is set to the sum of all SGA database instance system. For example SGA the sum of 9GB, you need to set kernel.shmall = 9 * 1024 * 1024/4 = 2359296.

Log in as root two database servers, edit sysctl.conf file.

node74: ~ # vi /etc/sysctl.conf

Kernel.shmall modify parameters:

kernel.shmall = 2359296

Execute sysctl -p to validate the configuration:

node74: ~ # sysctl -p


3.3.6 Start the database instance

Users log on to the oracle two database servers, start two instances of the database through sqlplus.

oracle @ node74: ~> sqlplus / as sysdba

SQL> startup


3.4 Calculation need to use hugepage page size

Ensure that all instances have been started (including ASM), then run hugepages_settings.sh (script content Appendix 4.1) to evaluate the need to set Hugepages size to root user.

node74: / home / oracle # ./hugepages_settings.sh

This script is provided by Doc ID 401749.1 from MyOracle Support

(Http://support.oracle.com) where it is intended tocompute values ​​for

the recommended HugePages / HugeTLB configuration forthe current shared

memory segments. Before proceeding with the executionplease make sure


 * OracleDatabase instance (s) are up and running

 * OracleDatabase 11g Automatic Memory Management (AMM) is not setup

  (See Doc ID749851.1)

 * The sharedmemory segments can be listed by command:

    # Ipcs -m

 Press Enter toproceed ...

---- Press the Enter key

Recommended setting: vm.nr_hugepages = 4611


You can also manually calculate:

nr_hugepages> = SGA_Target / Hugepagesize

= 9G * 1024M / 2M

4608 =

Take a slightly larger than the 4608 value.


3.5 modify the parameters vm.nr_hugepages

Log in as root two database servers, edit /etc/sysctl.conf:

node74: ~ # vi /etc/sysctl.conf

Modify vm.nr_hugepages parameter value calculated from the previous step:

vm.nr_hugepages = 4611

Execute sysctl -p to validate the configuration:

node74: ~ # sysctl -p


3.6 Stop the database instance, restart the OS

Stop all database instances, restart the operating system. (Theoretically without rebooting the operating system, it is recommended to restart)


3.7 Check the settings have taken effect

After rebooting the system, start the whole database, using the following command to check the configuration to take effect:

node74: ~ # grep HugePages / proc / meminfo

HugePages_Total: 4611

HugePages_Free: 2394

HugePages_Rsvd: 2387

HugePages_Surp: 0


4. Appendix

4.1 Script hugepages_settings.sh

#! / Bin / bash
# Hugepages_settings.sh
# Linux bash script to compute values ​​for the
# Recommended HugePages / HugeTLB configuration
# Note: This script does calculation for all shared memory
# Segments available when the script is run, no matter it
# Is an Oracle RDBMS shared memory segment or not.
# This script is provided by Doc ID 401749.1 from My Oracle Support
# Http://support.oracle.com
# Welcome text
echo "
This script is provided by Doc ID 401749.1 from My Oracle Support
(Http://support.oracle.com) where it is intended to compute values ​​for
the recommended HugePages / HugeTLB configuration for the current shared
memory segments. Before proceeding with the execution please make sure
* Oracle Database instance (s) are up and running
* Oracle Database 11g Automatic Memory Management (AMM) is not setup
(See Doc ID 749851.1)
* The shared memory segments can be listed by command:
# Ipcs -m
Press Enter to proceed ... "
# Check for the kernel version
. KERN = `uname -r | awk -F '{printf ("% d% d \ n. ", $ 1, $ 2);}'`
# Find out the HugePage size
HPG_SZ = `grep Hugepagesize / proc / meminfo | awk '{print $ 2}'`
# Initialize the counter
NUM_PG = 0
# Cumulative number of pages required to handle the running shared memory segments
for SEG_BYTES in `ipcs -m | awk '{print $ 5}' | grep" [0-9] [0-9] * "`
MIN_PG = `echo" $ SEG_BYTES / ($ HPG_SZ * 1024) "| bc -q`
if [$ MIN_PG -gt 0]; then
NUM_PG = `echo" $ NUM_PG + $ MIN_PG + 1 "| bc -q`
RES_BYTES = `echo" $ NUM_PG * $ HPG_SZ * 1024 "| bc -q`
# An SGA less than 100MB does not make sense
# Bail out if that is the case
if [$ RES_BYTES -lt 100000000]; then
echo "***********"
echo "** ERROR **"
echo "***********"
echo "Sorry! There are not enough total of shared memory segments allocated for
HugePages configuration. HugePages can only be used for shared memory segments
that you can list by command:
# Ipcs -m
. Of a size that can match an Oracle Database SGA Please make sure that:
* Oracle Database instance is up and running
* Oracle Database 11g Automatic Memory Management (AMM) is not configured "
exit 1
# Finish with results
case $ KERN in
'2.4') HUGETLB_POOL = `echo" $ NUM_PG * $ HPG_SZ / 1024 "| bc -q`;
echo "Recommended setting: vm.hugetlb_pool = $ HUGETLB_POOL" ;;
'2.6') echo "Recommended setting: vm.nr_hugepages = $ NUM_PG" ;;
*) Echo "Unrecognized kernel version $ KERN. Exiting." ;;
# End

4.2 database startup error ORA-27102

Upon startup of Linux database getORA-27102: out of memory Linux-X86_64 Error: 28: No space left on device


Upon startup of Linux database getORA-27102: out of memory Linux-X86_64 Error: 28: No space left on device

Doc ID:

Note: 301830.1



Last Revision Date:




In this Document






-------------------------------------------------- ------------------------------

Applies to:

OracleServer- Enterprise Edition - Version: to

UnitedLinux x86-64

Red Hat Enterprise Linux Advanced Serverx86-64 (AMD Opetron Architecture)

x86 64 bit


When trying to increase the SGA to approachhalf available RAM with an Oracle 64bit version on a Linux 64bit operatingsystem, even though shmmax is set to match half the amount of RAM, youget the following error when trying to start the instance:

SQL> startup nomount

ORA-27102: out of memory

Linux-x86_64 Error: 28: No space left ondevice


shmall is too small, most likely is set tothe default setting of 2097152

$ Cat / proc / sys / kernel / shmall



shmall is the total amount of sharedmemory, in pages, that the system can use at one time.


Set shmall equal to the sum of all the SGAson the system, divided by the page size.

The page size can be determined using thefollowing command:

$ Getconf PAGE_SIZE


For example, if the sum of all the SGAs onthe system is 16Gb and the result of '$ getconf PAGE_SIZE' is 4096 (4Kb) then set shmall to 4194304 (4Mb)

As the root user set the shmall to 4194304in the /etc/sysctl.conf file:

kernel.shmall = 4194304

then run the following command:

# Sysctl -p

# Cat / proc / sys / kernel / shmall



The above command loads the new value and areboot is not necessary

Switch back to being the oracle user andretry the startup command.


Note 169706.1 - Oracle Database onAIX, HP-UX, Linux, Mac OS X, Solaris, Tru64 Unix Operating SystemsInstallation and Configuration Requirements Quick Reference (8.0.5 to 11.1)


- The REVERSE function of DB2 (Database)
- The difference between statement and preparedStatement of the jdbc (Database)
- Definition Format Oracle basis of various statements (Database)
- Linux PXE unattended installation PXE-E32: TFTP OPen timeout the solution (Linux)
- Upgrading Oracle to (Database)
- MongoDB collection data migration to MySQL database (Database)
- C ++ casts New Standard Comments (Programming)
- Linux host dual LAN transceiver package ARP problem (Linux)
- Understanding Java classes (Programming)
- MySQL appears data too long for column 'name' at row 1 to solve (Database)
- Linux users should be aware that should the 7 ls command unique skills (Linux)
- Linux disk virtualization (Linux)
- Android timer use (Programming)
- 22 Port weak passwords and SSH connection program of the Linux server (Linux)
- Merge sort Java implementation (Programming)
- How do you turn on and off IPv6 address on Fedora (Linux)
- C # and JavaScript arrays to re-summary (Programming)
- Binary tree traversal: the first sequence in order preorder recursive and non-recursive and traversal sequence (Programming)
- ARM runtime environment built from scratch using QEMU emulator (Linux)
- Linux initialization init systems - Systemd (Linux)
  CopyRight 2002-2022 newfreesoft.com, All Rights Reserved.