Home PC Games Linux Windows Database Network Programming Server Mobile  
  Home \ Linux \ Linux Fundamentals of the memory management mechanism     - Java memory-mapped file MappedByteBuffer (Programming)

- SSH mutual trust configuration (Server)

- Intel Graphics Installer 1.0.3 released, support for Ubuntu (Linux)

- Linux top command to get started (Linux)

- Linux firewall rules example Extracts (Linux)

- Linux and Unix systems really do network more secure (Linux)

- VirtualBox 4.3 Can not open a new task for a virtual computer solution (Linux)

- GNU Linux system variables (sysctl configuration commands) integrated use (Linux)

- C ++ overloaded stream insertion operator and the stream extraction operator (Programming)

- Explore Android SQLite3 multithreading (Programming)

- Embedded Linux to solve the problem in the time zone (Linux)

- C language function pointer and a callback function (Programming)

- Git Installation and Configuration (Network Agent settings) (Linux)

- Using Java arrays implement sequential stack (Programming)

- How to use the Vault secure password storage and API key (Linux)

- HAProxy performance under high concurrency (Server)

- How to use the ps command to monitor progress in the implementation of Linux commands (Linux)

- VirtualBox snapshots (Linux)

- Recover accidentally deleted Nginx logs (Server)

- Linux Network Programming - raw socket instance: MAC Address Scanner (Programming)

  Linux Fundamentals of the memory management mechanism
  Add Date : 2017-08-31      
  1 Linux memory management of the main features

-------------------------------------------------- ------------------------------
Regardless of how much physical memory, Linux will make full use of some of the program called the hard disk data read into memory, the use of memory to read and write high-speed features to improve the Linux system data access performance. And Windows is only in need of memory, only to allocate memory for the application, and can not take full advantage of large-capacity memory space. This feature of Linux, is to use the free physical memory, a part of space, as cache, buffers, in order to improve data access performance. The page cache is a primary disk cache implemented by the Linux kernel. It is mainly used to reduce disk I / O operations. Specifically, access to the disk is made accessible to physical memory by caching data in the disk to physical memory.

2 physical memory, virtual memory

-------------------------------------------------- ------------------------------
Physical memory is the size of memory provided by the system hardware, is the real memory, as opposed to physical memory, in Linux there is a concept of virtual memory, virtual memory is to meet the physical memory deficiencies and strategies, it is the use of disk space Virtual out of a piece of logical memory, used as virtual memory disk space is called swap space (Swap Space).
Linux memory management is a paging access mechanism, in order to ensure that the physical memory can be fully utilized, the kernel will be appropriate when the physical memory is not often used to automatically switch to the virtual memory block, and will often use the Information is retained in physical memory.

Linux memory operating mechanism:
2.1 Linux system from time to time the page exchange operation, in order to maintain as much free physical memory
2.2 Linux for page exchange is conditional, not all pages are switched to virtual memory when not in use, Linux kernel under the "most recently used" algorithm, only some of the less frequently used page file to swap virtual memory
2.3 swap space in the use of the first page will be switched to physical memory, if there is not enough physical memory to accommodate these pages, they will be immediately swap out

3 Linux memory monitoring

-------------------------------------------------- ------------------------------
Free -m:
[Root @ rango backup_CentOS] # free -m
            Total used free shared buffers cached
Mem: 3805 3683 121 0 120 690
- / + buffers / cache: 2872 932
Swap: 2041 134 1907
Total: total size of physical memory used: the physical memory size has been used free: free physical memory size
Shared: shared memory size of multiple processes buffers / cached: the size of the disk cache
Mem: on behalf of the physical memory usage (- / + buffers / cached): on behalf of the disk cache usage
Swap: Indicates swap memory usage
(-buffers / cache) used Memory: 2872M (referring to the first part of the Mem line used-buffers - cached)
(+ Buffers / cache) free Memory: 932M (referring to the first part of the Mem line in the free + buffers + cached)
From the perspective of the kernel can use the memory: 121M
Available memory from the application: free + buffers / cache = 121 + 120 + 690 = 931M, the free value for the second line. For applications, buffers / cached is available because buffers / cached is designed to improve the performance of files, and buffers / cached is quickly reclaimed when the application needs memory. For use by the application.

Linux cache mechanism: buffers and cached are memory operations, used to save the system has opened the file and file attribute information, so that when the operating system needs to read some files, will be the first buffers and cached memory area to find, if found , Direct read out to the application, if not find the need for data, read from the disk.
Buffers are used to buffer the block device to do, it only records the file system metadata (metadata) and tracking in-flightpages, and cached is used to buffer the file. More popular point that: buffers are mainly used to store what the contents of the directory, the file attributes and permissions, and so on. The cached directly used to remember the files we have opened and procedures.

4 Linux memory release process

-------------------------------------------------- ------------------------------
4.1 free -m View the memory usage

4.2 sync: Use the sync command to ensure file system integrity. The sync command runs the sync subroutine, writes all unwritten system buffers to disk, containing the modified i-node, delayed block I / O And read and write mapping files. To ensure reliability, the sync command should be executed twice because the sync command does not guarantee that the information actually written to the disk.

4.3 Modifying / proc / sys / vm / drop_caches:
Echo 3> / proc / sys / vm / drop_caches
1) / proc is a virtual file system, we can read and write it as a means of communication with the kernel entities. That is, you can modify the file in / proc to make adjustments to the behavior of the current kernel. That is, we can adjust the / proc / sys / vm / drop_caches to release memory.
2) drop_caches:
Writing to this file causes the kernel to drop cleancaches, dentries and inodes from memory, causing that memory tobecomefree.
To free pagecache, use echo 1> / proc / sys / vm / drop_caches;
To free dentries and inodes, use echo 2> / proc / sys / vm / drop_caches;
To free pagecache, dentries and inodes, use echo 3> / proc / sys / vm / drop_caches.
Because this is a non-destructive operation and dirty objects arenot freeable, the user should run syncfirst.

5 Linux memory mapping

-------------------------------------------------- ------------------------------
When the executable file is ready to run, the contents of the executable file is only mapped to the corresponding process virtual address space, and not transferred to the physical memory. When the program began to run and use this part of the Linux through the page to interrupt them from the disk transferred to memory. This process of connecting files to the process virtual address space is called memory mapping.
In general, user space is not and should not directly access the device, but the device driver can be achieved mmap () function, this function allows users to directly access the physical address of the device space. In fact, mmap () to achieve such a mapping process, it will be a user memory space associated with the device memory, when the user access to user space in this address range, will actually translate into the device access.
When the user calls mmap (), the kernel will handle the following:
(1) in the process of virtual space to find a VMA.
(2) The VMA will be mapped.
(2) If the device driver or file system file_operations defined mmap () operation, then call it.
(4) Insert this VMA into the VMA list of processes.
Vma contains the information used to access the virtual address of the device, so a lot of work done by the kernel. To execute mmap, the driver simply needs to set up the appropriate page table for the address return and replace vma-> vm_ops with a series of new actions.


6 Linux shared memory

-------------------------------------------------- ------------------------------
6.1 Shared Memory

-------------------------------------------------- ------------------------------
Shared memory can be said to be the most useful method of inter-process communication, but also the fastest form of IPC. Two different processes A, B shared memory means that the same piece of physical memory is mapped to the process A, B, the process address space. Process A can immediately see the process B updates to the shared memory data, and vice versa. As multiple processes share the same block of memory, some kind of synchronization mechanism is required, and mutexes and semaphores can be used.

An obvious benefit of using shared memory communication is the high efficiency, since the process can read and write directly to memory without requiring any copies of the data. For communications such as pipes and message queues, four copies of data are required in the kernel and user space, while shared memory copies only twice: once from the input file to the shared memory area, and from the shared memory area to the shared memory area Output file. In fact, the shared memory between the process, not always read and write a small amount of data after the lifting of the mapping, a new communication, and then re-establish the shared memory area. But to keep the shared area, until the communication is completed so that the data content has been stored in the shared memory, and not write back the file. Content in shared memory is often written back to the file when it is unmapped. Therefore, the use of shared memory communication efficiency is very high.

6.2 release of shared memory

-------------------------------------------------- ------------------------------
In the use of shared memory program abnormal exit, because there is no release out of shared memory, an error in debugging.
To release shared memory:
1) If you always end by Crtl + C, you can do a signal processor, when receiving this signal, the first release of shared memory, and then exit the program.
2) through the linux command ipcrm shm shmid to release, you can use the command before the ipcs-m command to view the shared memory.
View shared memory: ipcs-m-m show only shared memory information, -q only show the message queue, -s only show the semaphore
[Root @ rango backup_CentOS] # ipcs -m

------ Shared Memory Segments --------
Key shmid owner perms bytes nattch status
0x00000000 98304 rango 600 393216 2 dest
0x00000000 131073 rango 600 393216 2 dest
0x00000000 163842 rango 600 393216 2 dest
0x00000000 196611 rango 600 393216 2 dest
0x00000000 229380 rango 600 393216 2 dest
0x00000000 262149 rango 600 393216 2 dest
0x00000000 294918 rango 600 393216 2 dest
0x00000000 327687 rango 600 393216 2 dest
0x00000000 360456 rango 600 393216 2 dest
The first column is the shared memory key; the second column is the shared memory number shmid;
The third column is to create the user owner; fourth column is the permissions perms;
The fifth column to create the size bytes; the sixth column is connected to the number of shared memory process nattach;
The seventh column is the status of the shared memory. "Dest" is displayed and the shared memory segment has been deleted. However, there are some users who are using it, and "dest" is displayed when the mode field of the segment memory is set to SHM_DEST. When the user calls shmctl IPC_RMID, the first memory to see how many processes associated with this memory, if the number of associations is 0, it will destroy the shared memory, or set the memory of the mod mode bit is SHM_DEST, if all Process is not used to delete this shared memory.

Release shared memory: ipcrm < shmid> # release a single shared memory
Doce sudo ipcrm -m $ s; done # release all shared memory. I have read the following statement: ipcs -m | awk $ 2 ~ / [0-9] + / {print $ 2}
Ps: release the shared memory python script

#! / Usr / bin / env python
# Description: Remove the share memory
# Author: RangoChen
# Date: 2013.12.19
# - * - coding: utf-8 - * -
# Remove the share memory
Import os
Import sys
Import getopt


Def usage ():
  Print "usage: python rmsharemem.py -h -o < owner> -ssize < shmid list>"
  Print "-h show help information"
  Print "-o < owner> the owner create share memory needto delete"
  Print "-s < size> the share memory size"
  Print "< shmid list> the shmid list need to delete"


Def getsharemem ():
  Sharemap = {}
  Fp = os.popen ( 'ipcs -m')
  Lines = fp.readlines ()
  For l in lines:
      If not l.startswith ( '0x'):
  S = l.split ()
  If sharemap.has_key (s [2]):
      Sharemap [s [2]]. Append (s)
        Sharemap [s [2]] = [s]
  #print 'Share memory map: \ n', sharemap
Return sharemap


If __name__ == "__main__":
  Opts, args = getopt.getopt (sys.argv [1:], "o: hs:")
    # Opts is the parameter with options
  # Args is the parameter no ptions
    Owner = None
    Size = 0
  For o, p in opts:
      If o == '-h':
      Usage ()
      Sys.exit (0)
  Elif o == '-o':
      Owner = p
  Elif o == '-s':
      Size = p


If not owner:
  Val = raw_input ( "Are you sure to remove all Sharememory? (Yes / no)")
  If (val == "yes"):
      Usage ()
      Sys.exit (0)


Count = 0
Total = 0
If len (args)> 0:
    For shmid in args:
        Cmd = 'ipcrm -m% s'% shmid
      Print 'execute command:% s'% cmd
        Ret = os.system (cmd)
        Total + = 1
      If ret == 0:
            Count + = 1
            Print 'remove% s shared memory success'% shmid
          Print 'remove% s shared memory failed'% shmid
      Shmmap = getsharemem ()
      For o, l in shmmap.items ():
          If owner and o == owner:
      For p in l:
            Total + = 1
            If size and size == p [4]:
          Cmd = 'ipcrm -m% s'% p [1]
          Print 'execute command:% s'% cmd
          Ret = os.system (cmd)
          If ret == 0:
                Count + = 1
              Print 'remove% s shared memory success'% p [1]
              Print 'remove% s shared memory failed'% p [1]
Print 'total share memory number =% s'% total
Print 'remove success number =% s'% count
Sys.exit (0)


6.3 Modification of shared memory size

-------------------------------------------------- ------------------------------
Check the size of the shared memory: cat / proc / sys / kernel / shmmax
Temporary changes: Echo 268435456> / proc / sys / kernel / shmmax # The shared memory size is set to 256MB;
Permanent Modifications: vim /etc/rc.d/rc.local: echo 268435456> / proc / sys / kernel / shmmax
You can modify the shared memory to 256MB each time you start it.

7 Summary

-------------------------------------------------- ------------------------------
This paper describes the Linux memory management mechanisms, including memory mapping, shared memory implementation mechanism.
- hexdump Linux command (Linux)
- Ubuntu 14.04 LTS NTFS partition can not access solution (Linux)
- Linux system security audit tools scan nessus installation tutorial (Linux)
- Programmers Do not neglect debugging techniques (Programming)
- Proper use Core Data multithreaded 3 ways (Programming)
- How to configure Ceph stored on CentOS 7.0 (Server)
- PostgreSQL Select source code analysis (Database)
- Terminal Linux command prints - echo (Linux)
- Oracle EBS R12 for Linux installation (Database)
- How a lot of blocking malicious IP address in Linux (Linux)
- MySQL Online DDL tools of pt-online-schema-change (Database)
- Ansible module Know (Linux)
- Using LLVM Clang and Blocks under Linux (Programming)
- The simple multi-threaded Python (Programming)
- Using Python to find a particular file extension directory (Programming)
- To share Internet access through NAT mode under Virtual Linux VMware Workstation (Linux)
- Through Oracle to help restore MySQL data problems (Database)
- The ActiveMQ JMS installation and application examples for Linux (Linux)
- Windows Ubuntu dual system a key Ghost, grub rescue prompt solution (Linux)
- U disk to install Windows and Ubuntu 15.04 dual system (Programming)
  CopyRight 2002-2022 newfreesoft.com, All Rights Reserved.