Home PC Games Linux Windows Database Network Programming Server Mobile  
           
  Home \ Linux \ Linux kernel netpoll framework netconsole     - About MongoDB query method according to fuzzy field (Database)

- Advanced Search Oracle study notes (Database)

- Linux file permissions chmod chown (Linux)

- Detailed steps - GAMIT solver (Linux)

- Inherent limitations of Linux systems network security (Linux)

- C ++ containers (Programming)

- Deploy Apache Spark cluster environment in Ubuntu (Server)

- CentOS7 virtual machine settings, and bridging problems (Linux)

- Encrypted with GnuPG signature to verify the authenticity and integrity of downloaded file (Linux)

- Use smartmontools view the health status of hard disk (Linux)

- 10 useful Linux command Interview Questions and Answers (Linux)

- Java Class file format parsing (Programming)

- Vmstat command Linux Performance Monitoring (Linux)

- CentOS 5.11 Open VNC access (Linux)

- How to Install Redis server on CentOS 7 (Server)

- Production environment Linux bonding method standby mode slave card switch (Server)

- Zabbix configuration of SNMP (Server)

- Setting Lns firewall against ARP attacks (Linux)

- Ubuntu the ARP (arptables) (Linux)

- When Linux virtual machine to another copy of the operating system, a static IP NAT mode Invalid (Linux)

 
         
  Linux kernel netpoll framework netconsole
     
  Add Date : 2018-11-21      
         
         
         
  Although network-related, but it is not part of the network protocol stack, which is netpoll.

It's just a doorway processing framework. The so-called network, its terminal node is the host, the host data from the card is issued, through an exit process, the card receives a data packet, through an entrance process, this one into the treatment process is divided into two ways :

1. Interrupt way

Export process - data into the transmit queue, read a specific register value, to be sent for NIC status type, send, waiting to be sent after the interruption notification, continue.

Entrance processes - data received by the NIC, NIC interrupts CPU, CPU and then processes the received data.

This entrance is interrupted treatment received under the network protocol stack.

2.poll way

Export process - data into the transmit queue, read a specific register value, to be sent for NIC status type, send, without waiting for an interrupt notification after reading the register and continue to weigh whether according to the queue for sending.

Entrance processes - data received by the card, wait for the poll logic at the appropriate time to take the initiative poll card, if the data will be read out from a specific NIC cache.

This approach is the use of Linux netpoll. This approach does not rely entirely interrupted.

Facts are always more trouble than speaking.

Talk clearly this theme a bit confusing, the entrance had to introduce a third approach, which is a combination of interrupts and poll, this is the NAPI mode.

a. interrupt

b.poll way

c.NAPI way

Understand this, the rest is all understood. As to why there NAPI, in this paper we can simply say: In the high-speed high-bandwidth network, data packets continue into each packet, then interrupts the CPU, CPU bit much, but delayed the CPU processes the data packets, if before packet has not been processed, the best way is the packet into a queue, and then silence, do not bother CPU, CPU and so empty down when he went to poll the queue inside the packet, which is the NAPI.

Pure interrupt the way we are familiar with, it is the most direct way. But why have a pure poll method? Pure poll occasions in case of interruption completely ineffective. In one example, after a system panic. The interrupt controller may be disable out, however, that at this time the machine has lost contact with the outside world, but if you must now need a way to advertise outside their own death, then, this way netpoll comes in handy because it is handmade, and completely independent of the system interrupt mechanism. Another case is better than some of the panic that stack fault, if interrupted or NAPI way, because it is on the next stack, netif_receive_skb he also nothing HOOK point, you can change this time using netpoll packet processing path through a remote agent can debug.

Do not interrupt imagine too mysterious. It is nothing more than a notification mechanism, telling CPU, Please check my status, the Gansha just do it. In fact, when the CPU receives an interrupt when the card, do not know doing it, it will only call the interrupt handler, it will be to read the internal state of a write register before you can know what to do, for example, can send a packets, such as a received data packets. That being the case, even in the case disable interrupts, if not rely on interrupts, CPU choose the active call about the NIC interrupt handler, read-write register state, it is not able to know what to do it? Of course, the answer is yes! This is netpoll logic, which uses a two-step task:

1. Take the initiative to call the NIC interrupt handler to get the transmission packet or a data packet is received;

2. Direct hard_xmit packets or use NAPI interface card to the poll data.

Comes netconsole principle, I did not think he was so simple. I remember I once wrote a module that will send information back to a remote panic, this is inspired by a xtables-addons modules, initially failed, but in the end I carefully debug kernel code, a success. The success of the process, he found many previously unknown things. But now look at netconsole it, what it does not need complicated things, just two steps:

1. Register a console, after which the kernel buffer data will be sent to the console;

Then netpoll of netpoll_send_skb, after which the process logic netpoll 2. Under the console.

Even after the panic, interrupt has been closed, even when the interrupt controller are closed, as long as the card is no water, you can still send and receive data, it is completely independent of the protocol stack and the interrupt. This is fantastic! About netconsole, how much I do not like to write kernel Document to the good: $ kernel / Documentation / networking / netconsole.txt.

netpoll Linux kernel is one of the protocol stack is not available or when an interrupt mechanism means abnormal communication with the outside world, of course, it is also a method of bypassing the protocol stack. Sufficient underlying this position, write something and certainly more fun than Netfilter based. Netfilter is a special point in the protocol stack packet capture, and netpoll it can capture data packets on the network card, which even the bottom of the stack are not. Later, if you want to direct contracting in kernel mode, users no longer have PACKET sockets from state began to construct a data packet, the interface directly issued by netpoll. The problem is that it uses a manually-triggered interrupt handler manner, the efficiency of how to be tested. Therefore, this mechanism is best limited to sending a small amount of kernel debugging and auditing information bar. Use it to do VPN, I think hanging ...

The price index

Generally speaking, if MySQL can find ways to use the index to handle inquiries more quickly, it will do so. This means that for the most part, if you do not have an index on the table, it will cause damage to property. This is what I describe the advantages of index views. But it has weaknesses? Yes, it has a cost in terms of time and space. In practice, the value of the index will generally exceed the advantages of these shortcomings, but you should know that in the end what are some drawbacks.

First, the index to speed up the retrieval speed, but slows down the speed of insertion and deletion, but also slows down the update indexed data value in the column speed. That is, the index slowed down most operations involved in writing speed. When this behavior occurs because when a record is written not only need to write data line, also need to change all the indexes. Index data tables with the more, the more we need to make modifications, reducing the average level of performance will be. "Data loading high efficiency" in the section of this article, we will be more detailed understanding of these phenomena and find treatment.

Secondly, the index takes disk space, multiple indexes accordingly takes more disk space. This may lead to faster reach the data table size limits:

For MyISAM tables, the index may frequently cause the index file data files faster than the maximum limit.

- For BDB table, it is the data and index values are stored together in the same file, add an index to cause such a table quickly reaches the maximum file limit.

- All tables in the InnoDB shared tablespace compete are allocated using the same pool of public space, so adding the index table space will run out of memory faster. However, files with MyISAM and BDB tables use different, InnoDB table space is not shared by the operating system file size limit, because we can configure it to use multiple files. As long as there is additional disk space, you can extend the table space by adding new components.

Use a separate table space and BDB InnoDB table by table constraint is the same, because its data and index values are stored in a single file.

The actual meaning of these elements are: If you do not use special index to help the query run faster, do not be indexed.

Select Index

Assuming you already know the syntax for indexing, but the syntax does not tell you how to index data table. This requires us to consider how the data table. This section guides you how to identify alternative data columns for the index, and how best to build the index:

For indexing data column to search, sort, and group and not just for the output display. In other words, the best alternative for the index data columns are those columns appear in the WHERE clause, join clause, ORDERBY or GROUPBY clause of. Only appear in the output data column SELECT keyword list later in the data column is not a good candidate columns:

SELECT
col_a <- not an alternative column
FROM
tbl1LEFT JOIN tbl2
ON tbl1.col_b = tbl2.col_c <- Alternate column
WHERE
col_d = expr; < - Alternate column
 


Of course, the data columns in the WHERE clause and columns used in the display may be the same. Our view is a good candidate column data columns in the list on the nature of the output is not used for the index.

Join clause or WHERE clause similar col1 = col2 form of expression data columns are particularly good options for index columns. Shown earlier query col_b and col_c is one such example. If MySQL can use to optimize the query join column, it will be by reducing the full table scan to significantly reduce the potential for a table - row combination.

Consider the data column cardinality (cardinality). Cardinality is the number of distinct values of the data contained in the column. For example, a data column contains values 1,3,7,4,7,3, then its base is 4. (That is the value of the column contains many different values, repeat rarely) cardinality of the index relative to the higher data table rows, when it works best. If a data column contains many different age groups, the index will quickly distinguish rows. If a data column for recording sex (only "M" and "F" two kinds of value), then the index's usefulness is not big. If the probability value appears almost equal, then no matter what the value of the search are likely to get half the rows. In these cases, it is best not to simply use the index because the query optimizer finds that the value occurs in the data row of the table in a high percentage, it will generally ignore the index, a full table scan. The percentage is the usual boundaries of "30%." Now the query optimizer is more complicated, some other factors are also taken into account, so this percentage is not the only factor that MySQL decided to use scan or index.

Shorter index values. Whenever possible, use smaller data types. For example, if you save enough MEDIUMINT value you need to store, do not use BIGINT data columns. If your value is not longer than 25 characters, do not use CHAR (100). Smaller values of the index improves processing speed by several aspects:

- Shorter value can be relatively quickly, so the faster the search index.

- Smaller values result in smaller indexes, require less disk I / O.

Use shorter key when the key index of the cache block (block) can hold more keys. MySQL can be maintained once more keys in memory without the need for additional indexes read from disk blocks, increase the likelihood of key-value targeting.

For InnoDB and BDB, etc. Using clustered indexes (clusteredindex) storage engine, maintaining the primary key (primarykey) short advantage is more prominent. Clustered index data in rows and the primary key value are stored together (clustered together). Other indexes are secondary indexes; they store the primary key value and a secondary index. The index yield secondary primary key values, which are used to locate the data row. This implies that primary key values are copied to each sub-index, so if the primary key is long, each secondary index will require more extra space.

Prefix index string value (prefixe). If you need to index a column of string data, then the best should specify a prefix length in any appropriate case. For example, if there is CHAR (200) column of data, if the previous 10 or 20 characters are different, do not index the entire data columns. Index preceding 10 or 20 characters can save a lot of space, and may make your queries faster. Through shorter index values, you can get the benefits of those savings compared with the speed and disk I / O related. Of course, you also need to use common sense. Only a data column index of the first string may be less useful, because if you do this, you will not have too many unique values in the index.

You can index CHAR, prefix VARCHAR, BINARY, VARBINARY, BLOB and TEXT data columns.

Use the leftmost (leftmost) prefix. A multi-column composite index, you actually build multiple indexes MySQL can use. A composite index can be used as multiple indexes used because the leftmost column of the index set can be used to match the data rows. This set of columns is called "the most left-prefix" (which is an index of a column prefix different kind of indexing the first few characters of a column as the index value).

Suppose you create a composite index on the state of the table, city and zip data columns. Index data rows in accordance with state / city / zip arranged in order, so they are automatically arranged according to the state / city and state order. This means that even if you only specify the state values in the query, or specify the state and city values, MySQL can also use this index. Thus, the index data can be used to search a combination of columns as follows:

state, city, zip
state, city
state


MySQL can not use the index to search for content not included in the most left-prefix. For example, if you follow the city or zip to search, it will not use the index. If you search for a given state and specific ZIP Code (index 1 and 3), the index is not available for this combination of values, although MySQL can use an index to find the matching state to narrow your search.

Do not overload the index. Do not think that "the more the index, the higher the performance," are not indexed for each data column. We mentioned earlier, each additional index will spend more disk space, and write performance. When you modify the contents of the table when the index must be updated, and may even rearrange. If you rarely use or never use an index, you do not need to modify the operation to reduce the speed of the table. In addition, as the search operation when the execution plan is generated, MySQL will consider the index. Create additional indexes will increase query optimizer more work. If too many indexes, it is possible (not necessarily) appear MySQL choose the best index in case of failure. We must maintain an index to help the query optimizer to avoid such mistakes.

If you consider to have indexed tables add an index, then you must consider whether you will increase the index has been the most left-prefix Multi-column index. If so, do not increase the index, because it already has (for example, if you create an index on the state, city and zip, so no need to increase the state's index).

Let index type and the type of comparison that you performed to match. When you build an index, the index will choose the most storage engines that will be used to achieve them. For example, InnoDB typically use B-tree indexes. MySQL also uses B-tree index, it uses R-tree indexes on the three-dimensional data types. However, MEMORY storage engine supports hash index and B-tree indexes, and allows you to choose which index. To select the index type, we need to consider the type of operation in comparison index data columns will be performed:

- For the hash (hash) index, a hash function is applied at each data column values. The results generated by the hash value stored in the index, and is used to execute the query. Hash function algorithm achieve similar produce different hash values for different input values. The benefits of using the hash value of the hash value is higher than the comparative efficiency of the original value. When the hash index is used to perform an exact match or = <=> operation very quickly. However, a range of values for the query effect is very poor:

id <30
weight BETWEEN 100 AND 150


- B-tree index can be used to perform accurate or based on a range (operation using <, <=, =,> =,>, <>,! =, And BETWEEN) of relatively high efficiency. B-tree index can also be used in LIKE pattern matching, provided that the mode character string rather than the beginning wildcards.

If you use the data table MEMORY only the exact value of the query, the hash index is a good choice. This is the default index type MEMORY table to use, so you do not need to specially designated. If you want to perform relatively range-based, you should use B-tree indexes on MEMORY tables. To specify this type of index, index definitions needed to add USINGBTREE. E.g:

CREATETABLE lookup
(
id INT NOT NULL,
name CHAR (20),
PRIMARYKEY USING BTREE (id)
) ENGINE = MEMORY;

If you want to allow the type of statement is executed, a single MEMORY tables can have a hash index and the B-tree index, even on the same data column.

Some types of comparison can not use the index. If you just pass by value to the function (for example STRCMP ()) to perform the comparison operation, the index has no value. The server must calculate the function values for each data line, it would rule out the use of indexes on columns of data.

Use slow query (slow-query) logs to identify poor performance of queries. This log can help you find benefit from the index query. You can directly view the log (which is a text file), or use statistical tools to mysqldumpslow its contents. If a given query several times in the "slow query" log, which is a clue, a query optimization may not be prepared. You can rewrite it to make it run faster. You have to remember, in the assessment of "slow queries" log time, "slow" is based on the actual time of measurement, the load on the larger servers "slow query" query appears in the log will be more.
     
         
         
         
  More:      
 
- CentOS set up FTP server (Server)
- Android to determine whether the device to open WIFI, GPRS data connection (Programming)
- Use XtraBackup to MySQL database online incremental backup and recovery (Database)
- Linux Platform Oracle 11g Single Instance Installation Deployment Configuration Quick Reference (Database)
- Linux suffered SYN flood attack setting (Linux)
- Upgrading to MySQL 5.7.9 MySQL 5.6.23 (Database)
- Introduction Linux namespace (Linux)
- How to install Git on CentOS 7 (Linux)
- The AWK use Cygwin tools mysql binlog log viewer (Database)
- Error: Unable to establish connection with FTP SFTP server (Server)
- How to compile and install wxWidgets in Ubuntu / Debian / Linux Mint (Linux)
- Oracle delete table space prompted ORA-00604 and ORA-38301 (Database)
- findmnt- Looking mounted file system (Linux)
- Redis Design and Implementation study notes (Database)
- Git commands (Linux)
- The several technical presentation Raid under Linux (Linux)
- Deploy the project to the server from GitHub (Server)
- Based kubernetes Construction Docker Cluster Management Comments (Server)
- Necessity in Java packages (Programming)
- Installation and Configuration OpenVPN server and client on Ubuntu 15.04 (Server)
     
           
     
  CopyRight 2002-2022 newfreesoft.com, All Rights Reserved.