This article explores the benefits of using containers in shared memory, and several methods of C ++ template containers in shared memory.
1 Why use template containers in shared memory?
Why to avoid the common memory and select shared memory, it is certainly the advantage of using shared memory:
Shared memory can be shared between multiple processes, to the way the process of communication.
Shared memory can exist outside the lifecycle of the process. This ensures that the service process can continue to use the shared memory data after a brief stop (service process coredump, update changes).
If these advantages in addition to the C + + container template easy to use, the advantages of rapid development, is undoubtedly the two-pronged approach to become a blade server development.
2 in the use of shared memory template container biggest difficulty is?
But if you want to do to allow the container to use in the template, the biggest trouble is? Is a pointer. (Synchronization of course is a problem, but I emphasize here is the container transplant)
Of course, in general, the address of the pointer or general point to the internal memory of the shared memory. Why not point to their own private data situation? , If the two processes A, shared data inside a pointer in the process A said that the process of private address A, inside the process B identifies the private address of B, which is obviously your logical design problems.
The internal indicators for the above-mentioned two advantages, which are natural enemies. Also note that if the data need to share multiple processes, your data must also be POD data, if there is a virtual table pointer, then it is impossible to achieve sharing.
Multi-process sharing between the shared memory, the address is likely not the same. Of course, shared memory on the API can generally suggest a fixed starting address, but since it is recommended, it may not comply, and this requires you to be familiar with the address space distribution process, but also for developers and operators, once used to share More memory, the use of fixed address is absolutely a nightmare.
For the server, the last mapping of the address, and the mapping may be inconsistent after the restart.
/ / Mmap function of the first parameter is the proposed address
Void * mmap (void * addr,
Size_t off = 0);
// The last argument to the Window's API, lpBaseAddress, is also the suggested address.
LPVOID WINAPI MapViewOfFileEx (
_In_ DWORD dwDesiredAccess,
_In_ DWORD dwFileOffsetHigh,
_In_ DWORD dwFileOffsetLow,
_In_ SIZE_T dwNumberOfBytesToMap,
_In_opt_ LPVOID lpBaseAddress
The best way to solve the problem with pointers (or the only way) is to not write pointers, but to record the relative offset addresses. All calculations are handled according to the offset address.
At present to explore the use of shared memory template, I have seen the idea and the realization of roughly three kinds, one is the custom STL container memory allocator, one is provided by the ACE address-independent distribution method, one is BOOST the interprocess implementation, we talk about the advantages and disadvantages of these methods.
3 custom STL distributor
If the early years (04 years ago) in the online forum for the answer, most of the answer is this, the surface of this is a better and simple answer, the maximum use of STL containers existing code.
Template < class_Tp, class _Alloc> class list;
But in fact the answer is not necessarily Kaopu. Write a shared memory allocator is certainly not difficult. Difficult to achieve If we want to put containers into shared memory for several purposes, use STL container implementations. Why, or because the pointer.
First of all, many STL containers to achieve which is a large number of indicators, such as the list of the circular queue prev and next pointer pointer, map the underlying red and black tree to achieve the three pointers, these containers are used in the real memory address that.
So the answer depends entirely on whether your STL's internal implementation of a pointer, and if so, that is basically not feasible (of course you put the data into shared memory is possible, but you can not share and reuse). Such as SGI implementation and STLport implementation.
4-position-independent assignment of the ACE
The second (around 2005) approach I saw was ACE, which is described in the ACE Programmer's Guide, The Practical Programming Patterns for Network and Systems Programming book entitled ACE Programmer's Guide, The ACE method is to provide an address-independent memory allocator (exactly, it should be the control block ACE_PI_Control_Block), while providing an ACE_Based_Pointer_Basic template to record the relative address. While the ACE_Based_Pointer_Basic template achieves almost the same behavior as a pointer by overloading the operator T * () function (actually calling addr () to get the real address). ACE to achieve a bit mean, we also have some effort to analyze.
Process A, B share a shared memory, respectively, mapped to a different address, there is a shared memory structure S, S to record a shared memory in this address of another address char *, structure S can use ACE ACE_Based_Pointer_Basic < Char> record the address, ACE_Based_Pointer_Basic < char> Record the length of the shared memory start address and address to the beginning of the shared memory, using two lengths, respectively. And then two processes can be through this pointer, and two offset length, calculated to be recorded address.
ACE implementation, its own memory allocator records the starting address and length of the address space, ACE_Based_Pointer_Basic < T> in the construction time will be based on their own address to determine what they need to calculate the starting address. But also in the package to consider more comfortable. But need to be reminded that you need to record the address must still be shared on this piece of memory, otherwise ... ... (not explained).
Also note that ACE's own container, although it also supports the use of shared memory allocator, but ACE containers also have a large number of internal indicators, rather than recording the relative address, the ACE container is not in fact shared memory. So ACE academic atmosphere is more intense, the practicality is not high (ACE container itself is not easy to use). So it can be said, ACE kicked open the door, but did not enter this hall.
5 BOOST interprocess container memory allocator
In recent years BOOST began to pop, BOOST interprocess library in a shared memory container memory allocator implementation, but pay attention to its use of container vector, list, BOOST own container container. This allocator can not be used with most existing STL implementations.
Can be said that in fact the realization of BOOST and ACE is similar to the idea is to allocate a block of shared memory for this piece of shared memory to generate a container memory allocator, the distributor for the container services, the use of shared memory container dispenser, All addresses within the container record relative addresses, not absolute addresses.
Template < class PointedType, class DifferenceType, class OffsetType, std :: size_t OffsetAlignment>
Comparison of ACE, BOOST to achieve the different, on the one hand BOOST shared memory management and container memory allocator idea is very clear, the overall design ideas or in the STL system under the ACE birth of the premature age, the overall system and STL container completely Compatible, on the other hand, BOOST in the relative address of the deal is also simple. He only records offset_ptr object this address to the need to record the length between the addresses.
In addition, although the BOOST code and the STL container consistent, but the difficulty of reading BOOST code at least double in STLPort, the traditional method of tracking the code to track the single-step though effective, but many variables can not query the actual value of the macro flying. Looking forward to the analysis of the great god of the code BOOST appear.
6 Shared memory capacity
And another small problem is that we apply for the size of the shared memory is a size limit, and STL containers are often on-demand growth characteristics, and this feature and there are some shared memory is not reconciled.
ACE problem in this issue gives some academic solutions, rely on signals, exceptions, etc., give you the opportunity to expand their own memory, but estimates are limited to the scope of academic research.
BOOST more clear on this issue, when the memory allocation is not enough bad_alloc exception thrown. But a little more clear.
7 Another solution to fix the maximum length of the container
BOOST implementation is good, but there are a few are not so perfect place, but at that time can refer to the idea has not so much, (I own containers when 2005, BOOST interprocess library in 2008 only )
First, put N T-type data containers in the end how much shared memory? Because the container itself is consumed, and this point BOOST and no interface to tell me. For containers that use shared memory, we all know what the maximum number we need to use.
Second, if the maximum size we already know. Then in fact, we can all of the beginning of the allocation of good space, rather than in each push_back when the transfer alloctor to allocate the address, in fact alloctor still use the red-black tree to manage all the distribution of addresses, frankly trouble. And because the maximum size of fixed, all of our internal positional relationship between the data can be used array subscript positioning. This also eliminates the need for a pointer to the trouble.
The above considerations. Our design idea is generally based on your incoming parameters to determine the size of the memory required to inform the caller, the caller to allocate their own memory (which can be shared memory), according to the allocation of memory addresses construct a container, container operation and Templates are basically the same, but also provide methods such as iterator methods such as access, the internal structure of the container
STL containers and the syntax of this method is basically compatible with the performance of the speed that BOOST should be faster (not every node in a node alloc). Inch has a director, a short foot, which is also considered an idea to.
Specific code will be estimated after the open source.
The key issue with using template containers in shared memory is the problem with pointers, which is a relatively good solution to this problem. A more general solution is to change all pointers into a relative address record. There is also a way for the container to be handled by allocating all the data in the container to the maximum number, using subscripts.