| |
RAC system for data blocks between nodes and node consistency is guaranteed through mechanisms to ensure that the message, that is, we often say gcs and ges of these messages to ensure the. These messages respectively LMD and LMS process between instances of transmission;
LMD handles message information, such as the state of the block, lock level and other information, and LMS will be responsible for transferring data block, we do not discuss this mechanism consistency, focusing on the message transfer and flow control.
Whether or LMD LMS messaging, the information transmitted through the UDP protocol for passing messages through the private network, and transfer time between instances of message transmission, the message is interactive with each other, can not It has been sent by one node while the other node is only responsible for sustained receives;
So time between instances of transmission must balance the chance of transmission of the message, the reasonable control of the flow between each other, RAC by introducing the concept of ticket in the transmission between each other and the probability of traffic control;
For ticket conceptual understanding and best practice, Oracle has two related documents can refer to:
Resolving ORA-481 and "terminating the instance due to error 481" (Doc ID 1950963.1)
Best Practices and Recommendations for RAC databases using very large SGA (e.g. 100 GB) (Doc ID 1619155.1)
Here we can see that the number of Oracle is held by a node ticket to send and receive messages flow control, a node sends a message queue at the same time will take a ticket to the peer, peer ticket will increased local ticket will be reduced, the local node will be calculated locally available tickets based on the available information and the buffer has been received and sent a request ticket information (null-req),
When there is no ticket available locally, the local LMS / LMD will enter the message waiting queue and constant checking ticket latch inside information, to determine whether there is information available ticket available until the end of the sent back message, and bring back available after the tickets, locally to continue to send messages to the peer;
We can query the dynamic view: GV $ GES_TRAFFIC_CONTROLLER to get the number of tickets avalible on each node, and you can TCKT_WAIT judge whether LMD LMS or are waiting ticket, if we continue to see each other here waiting, UDP buffer described in package information is not timely treatment, or during the transmission of the lost;
Of course, we can empirically see lms / lmd message transmission in the case before the crash of the queue to determine whether the problem is caused by lack of ticket, but usually, it does require a guess, not as good as direct access to the GV $ GES_TRAFFIC_CONTROLLER intuitive.
Our most common problem is described in 1619155.1 RAC hang / LMD crashes instance with ORA-600 [kjmscndscq: timeout]
and
ORA-00481 After "The instance eviction reason is 0x2" due to Lack of Ticket (Doc ID 1644015.1)
These two issues are relatively clear:
Alarm log: ORA-600 [kjmscndscq: timeout] before or ora-00481 error of "The instance eviction reason is 0x2"
Because instructions are between instances tickets shortages led to fault message transmission LMS / LMD between timeouts occur.
This problem is particularly evident on the HP platform, because LMS HP platforms are not generally true process of RR (real time) mode, leading LMS did not get the timely dispatch of the CPU priority questions about the process, please refer to the following documents:
HP-UX: HPUX_SCHED_NOAGE and Scheduling Priority-Policy for LMS in RAC (Doc ID 759082.1)
So for users of HP platforms, if the SGA area is relatively large (usually more than 100G), business frequently update the same data block table live on multiple nodes, it is very easy to run into such problems.
Solution:
The following document gives most of the solution:
ORA-00481 After "The instance eviction reason is 0x2" due to Lack of Ticket (Doc ID 1644015.1)
Best Practices and Recommendations for RAC databases using very large SGA (e.g. 100 GB) (Doc ID 1619155.1)
If you run your system on the 12.1.0.2 version should be noted that LMD process between each node should be consistent (and the default number is the number of CPU-related) number, if the number of different LMD need patch 17,821,214; |
|
|
|