Home PC Games Linux Windows Database Network Programming Server Mobile  
           
  Home \ Database \ Redis application of Sina Weibo     - 14.04.3 Ubuntu configuration and successfully compiled source code Android 6.0 r1 (Programming)

- Linux file and directory management - ls, cp, mv (Linux)

- Nginx installation configuration Zabbix (Server)

- MySQL DATE_FORMAT () function (Database)

- DNF Command Tutorial (Linux)

- Oracle Linux 6.4 (BOND) dual NIC teaming combat - Annotated (Linux)

- Linux command binlog by filtering out the complete SQL statement (Database)

- Bash mathematical extension (Programming)

- Node.js simple interface server (Server)

- OpenSUSE GPG Comments (Linux)

- Ubuntu deployment Flask + UWSGI + Nginx Comments (Server)

- HTTP Client Hints Introduction (Server)

- GitLab installation configuration notes (Linux)

- Linux Troubleshooting: How to save the status of the SSH session is closed (Linux)

- RHEL5 / 6 Installation Notes (Linux)

- Git Getting Started tutorial (Linux)

- Father of Python: Why Python zero-based index (Programming)

- How to modify the Sublime in Tab four spaces (Linux)

- Install Ubuntu open source drawing program MyPaint 1.2.0 (Linux)

- 20 Advanced Java interview questions summary (Programming)

 
         
  Redis application of Sina Weibo
     
  Add Date : 2018-11-21      
         
         
         
  Redis application of Sina Weibo

Redis Introduction

1. Support 5 kinds of data structures

Support strings, hashes, lists, sets, sorted sets
string is a good way to store, used to store count. sets great library for indexing;

2. K-V vs K-V cache memory

98% were persistent application Weibo currently using 2% of the cache, the server uses 600 +
Redis application and persistence in non-persistent manner not very different:
Non-persistent for 80,000-90,000 tps, so persistent at around 70,000 to 80,000 tps;
When a persistent need to take into account the persistence and write performance ratio, which is to consider the proportion of redis used memory size and the hard disk write speed of calculation;

3. Active community

Redis There are currently more than 30,000 lines of code, write code to streamline, there are many clever implementations, author technical cleanliness
Redis high community activity, which is an important indicator of the quality of open source software, open source software is generally the initial no commercial technical support services, if there is no active support community to do, nowhere to help in the event of problems;

Redis Fundamentals

redis persistence (aof) append online file:
Write log (aof), then to a certain extent, and memory consolidation. Append then an additional, sequential write disk, the performance impact is very small

1. Single Instance Single-Process

Redis using a single process, so when configuring an instance will only use a CPU;
When configuring, if you need to maximize CPU utilization, you can configure Redis instances corresponding to the number of CPU, Redis instances corresponding to the number (8-core Cpu, 8 instances, 8 ports) ports, to improve concurrency:
When a stand-alone test, a single 200-byte data, test results from 8 to 90,000 tps;

2. Replication

Process: Data written to the master -> master to slave stored in the rdb -> slave rdb loaded into memory.
Storage point (save point): When the network outage, and even after, continue to pass.
Master-slave synchronization is the first time under a full pass, followed by incremental synchronization;,

3. Data Consistency

The possibility of inconsistencies between multiple nodes after the long run;
The development of two utilities:
1. For the large amount of data, the total amount will periodically check;
2. Check the real-time incremental data, whether the consistency;

For the main library from the library are not synchronized in time lead to inconsistencies, called latency issues;
For consistency requirement is not so stringent scenario, we only need to ensure that the eventual consistency can;
The delay problem, the need to analyze the characteristics of business scenarios, from policy to increase the application level to solve this problem;
E.g:
1. The newly registered user, you must first check the main library;
2. After successful registration, we need to wait for 3s after the jump, and at that point in doing background data synchronization.

Sina use Redis Course

In 2009, use memcache (for non-durable content), memcacheDB (for persistence + count),
memcacheDB Sina on the basis of memcache, using BerkeleyDB as persistent data storage implementation;

1. Problems facing

Demand data structure (Data Structure) more and more, but there is no memcache affect development efficiency

Performance requirements, with increase in the amount of read operations need to be solved through the process are:
Database separate read and write (M / S) -> Database Using multiple Slave -> increase Cache (memcache) -> Go Redis

Write to solve problems:
Split level, split the table, some users will be on the table, some users on another table;

Reliability requirements
Cache of the "avalanche" issue tangled
Cache is facing the challenges of rapid recovery

Development cost requirements
DB Cache consistency and increasing maintenance costs (to clean your DB, and then clean the cache, ah, too slow!)
Developers need to keep up with the influx of demand for products
The most expensive is the cost of the hardware aspects of the database machine, substantially more expensive than the front end of the machine several times, mostly IO-intensive, a waste of hardware;

Maintenance complexity
Consistency maintenance costs higher and higher;
BerkeleyDB use B-tree, will always write a new internal organization will not have to re-file; this will lead to more and more documents; big time required for filing, archiving operations to be regular;
Thus, we need to have some down time;

Based on the above considerations, chose Redis

2. Find ways open source software and criteria

For the open-source software, the first to see what it can do, but more need to focus on what it can not do, it will be a problem?

Rises to a certain size, there may be a problem, whether to accept?

On google code, foreign forum to find material (domestic than foreign technology lag 5 years)

Observation of the individual code level

Redis application scenarios

1. Business use

hash sets: Watchlist, fan list, two-way Watchlist (key-value (field), sorting)

string (counter): the number of microblogging, number of fans, ... (to avoid select count (*) from ...)

sort sets (automatic sorting): TopN, popular microblogging, automatic sorting

lists (queue): push / sub reminder, ...

The four, from fine control, hash sets and string (counter) is recommended, sort sets and lists (queue) is not recommended
But also by secondary development, streamline. For example: storage character to store plastic, 1.6 billion data only 16G memory
Stored within the storage type three kinds, recommended not more than three kinds;

The memcache + myaql replaced by Redis:
Redis as a store and provide access to background is no longer used mysql, solve the problem of data consistency between multiple copies;

2. The storage of large data tables

(Eg: 140 words microblogging storage)
A library to keep the uniqueness of id and 140 words;
Another stock id and user name, release date, clicks, and other information used to calculate, sort, etc., in order to calculate the final required data and then to show the first library to extract the micro-blog content;

Improved three steps:
1) found problems with the existing system;
2) the discovery of new things, how to see how well the new comprehensive turn things;
3) rational regression to determine which fit the new stuff, which is not suitable, not appropriate to move back to the old system

3. Some tips

Many applications can withstand a database connection failure, but can not afford a slow process

Copy of the data, multiple indexes (for different query scenes)

The only way to solve the IO bottleneck: memory used

Changes in the amount of data that is not the case, preferred Redis

Problems encountered and solutions

(Note: are particularly large amount of time there will be, how small the amount of hard to say)

1.Problem: After Replication interrupt retransmission -> network burst traffic

Solution: Replication rewrite the code, rdb + aof (rolling)

2.Problem: capacity issues

Solution: Capacity planning and M / S features of sharding (share nothing, abstracted from the associated data between data objects is very small)
Add some configuration, split, for example: 1,2,3,4, the machine 1 processing, machine processing 2% 2% 2 = 0 = 1.
Less than half the amount of memory, otherwise the expansion (proposed data Redis instance, the maximum should not exceed 80% of the memory)
Our line 96G / 128G memory server does not recommend a single-instance capacity of more than 20 / 30G.
Microblogging application data in a single table top has a 2T data, but it has some application powerless;
Each port should not exceed 20G; test time save disk do need, how long can all be written; the larger the memory, the longer the time to write;
Single Instance large amounts of memory, the immediate problem is caused by the recovery time from a library or Rebuild a long time, for ordinary hard disk loading speed, our experience is generally redis load 1G 1 minute; (loaded speed depends on the complexity of the data volume size and data)
When Redis rewrite aof and save rdb, it will bring a very large and long system pressure, and memory is used, it may lead to insufficient system memory line failure and other serious impact on performance.

reblance: redistribute existing data in accordance with the above configuration.
Rear intermediate layer, routing of HA;
Note: The current official are doing this thing, Redis Cluster, HA solve the problem;

3. Problem: bgsave or bgwriteaof ice crystals problem

Solution: Disk Performance Planning and write speed limitations, such as: provision disk speed of 200M / s write, steady, even though the arrival of large amounts of data but be aware that the write speed to meet the two objective limits:
Compliance with disk speed
Comply with the time limit (guaranteed before the arrival of the peak, you have to finish)

4.Problem: operation and maintenance problems

1) Inner Crontab: Redis move to the Crontab inside, reducing migration time pressure
The multi-port at the same time to avoid doing - can do
The same business multiport (distributed multi-machine), while avoiding doing - do
2) Live Upgrade: .so file loaded first, and then manage the configuration, switch to the new code (Config set command)
The improvement of redis things are packaged into lib.so file, so to support dynamic upgrade
To change their own time to consider upgrading the community. When the community has a new version, new features good use of time, to be able to easily merge well with our improved version;
Upgrade Prerequisites: Modular, module units upgrade
Loading time depends on two aspects: the data size, the complexity of the data structures in general, 40G data takes 40 minutes
Two core issues of distributed systems: A. B.HA routing problem

3) handling of dangerous command: For example: fresh all deletes all the data have to be controlled
Operation and maintenance can not talk about data backup, data recovery have to consider the time required;
Increasing the certification authority (administrator privileges only) eg: flashall certification authority, and must have a password to do;
Of course, high-speed data exchange are not normally carried in every certification authority, a common treatment strategy for the first time certification, recertification late do not have;
Control hash policy (no key, it can not find the value; do not know the hash policy, you can not get key)

4) Config Dump:
Dynamic memory configuration item modified according to certain policies written to disk (Redis already supported)
5) bgsave bring aof write very slowly:
fdatasync When do bgsave, not sync aof (have data access)
6) cost: (22T memory, there 10T for counting)
Redisscounter (16 Yi data occupancy 16G memory) - all become integral storage remainder (string, etc.) do not full
Redis + SSD (counterService counter service)
Order increment, table in the order written, filled with 10 table automatically fall (to SSD)
Storage tiering: memory allocation problem, 10K and 100K wrote a piece of debris will only waste Sina has been optimized to less than 5% (already good!)

5.Problem: Distributed Problem

1.Config Server: namespace, particularly access to tell, are not suitable for the agent because the agent to reduce speed, but, Sina with a (single multi-port, Redis Cluster, sentinel)
Zookeeper put on Config Server
Foremost is naming service, followed by a stateless twmemproxy (twitter improved, written in C), it is redis behind;
2.twmemproxy
Applications need not be concerned about the connection fails, the agent responsible for reconnection
The Hash algorithm into agents
Upgrade agents behind the front end does not care to solve the problem of HA
No state, more than one agent does not matter
3.AS -> Proxy -> Redis
4.Sina of Redis are stand-alone version, and Redis-Cluster exchanged too complicated, do not use
HA do so, we must do with the monitor, then if hung up, how to do the follow-up;
Not a single pursuit of performance, but the throughput of the cluster, which can support wireless extensions;

Experience

Planning well in advance the amount of data to reduce sharding (Internet companies are generally in years)

Only keep meticulous data (memory will be a premium!)

Storing user data Dimensions
Object Dimension Data have a life cycle
Especially in particularly large amount of data when it is necessary to be divided up;

Common process exposed and services: IP -> Load Balancing -> domain name -> Naming Service (a table: Name + resource (IP + port))

For hardware consumption, IO, network and CPU compared, Redis is the most consumed CPU, complex data types will certainly lead to CPU consumption;

Weibo timeout response time currently set to 5s; (very slow return to record key, need to record analysis, slow log);

Data backup to run on a regular basis about the production data; to check the validity of the backup data;

slave master will certainly be linked to more than compare the effects caused; Weibo currently using M / S is a drag, mainly used for disaster recovery;
Synchronization is a separate process to fork and slave synchronize; the process will not take up the query;

After the upgrade to 2.6.30 linux kernel;
Well at least to 2.6.30 deal with the problem of soft interrupt, performance improvement effect is obvious, almost 15% to 30% of the gap;

redis not have separate read and write, each request is a single thread, why should separate read and write.Redis application of Sina Weibo

Redis Introduction

1. Support 5 kinds of data structures

Support strings, hashes, lists, sets, sorted sets
string is a good way to store, used to store count. sets great library for indexing;

2. K-V vs K-V cache memory

98% were persistent application Weibo currently using 2% of the cache, the server uses 600 +
Redis application and persistence in non-persistent manner not very different:
Non-persistent for 80,000-90,000 tps, so persistent at around 70,000 to 80,000 tps;
When a persistent need to take into account the persistence and write performance ratio, which is to consider the proportion of redis used memory size and the hard disk write speed of calculation;

3. Active community

Redis There are currently more than 30,000 lines of code, write code to streamline, there are many clever implementations, author technical cleanliness
Redis high community activity, which is an important indicator of the quality of open source software, open source software is generally the initial no commercial technical support services, if there is no active support community to do, nowhere to help in the event of problems;

Redis Fundamentals

redis persistence (aof) append online file:
Write log (aof), then to a certain extent, and memory consolidation. Append then an additional, sequential write disk, the performance impact is very small

1. Single Instance Single-Process

Redis using a single process, so when configuring an instance will only use a CPU;
When configuring, if you need to maximize CPU utilization, you can configure Redis instances corresponding to the number of CPU, Redis instances corresponding to the number (8-core Cpu, 8 instances, 8 ports) ports, to improve concurrency:
When a stand-alone test, a single 200-byte data, test results from 8 to 90,000 tps;

2. Replication

Process: Data written to the master -> master to slave stored in the rdb -> slave rdb loaded into memory.
Storage point (save point): When the network outage, and even after, continue to pass.
Master-slave synchronization is the first time under a full pass, followed by incremental synchronization;,

3. Data Consistency

The possibility of inconsistencies between multiple nodes after the long run;
The development of two utilities:
1. For the large amount of data, the total amount will periodically check;
2. Check the real-time incremental data, whether the consistency;

For the main library from the library are not synchronized in time lead to inconsistencies, called latency issues;
For consistency requirement is not so stringent scenario, we only need to ensure that the eventual consistency can;
The delay problem, the need to analyze the characteristics of business scenarios, from policy to increase the application level to solve this problem;
E.g:
1. The newly registered user, you must first check the main library;
2. After successful registration, we need to wait for 3s after the jump, and at that point in doing background data synchronization.

Sina use Redis Course

In 2009, use memcache (for non-durable content), memcacheDB (for persistence + count),
memcacheDB Sina on the basis of memcache, using BerkeleyDB as persistent data storage implementation;

1. Problems facing

Demand data structure (Data Structure) more and more, but there is no memcache affect development efficiency

Performance requirements, with increase in the amount of read operations need to be solved through the process are:
Database separate read and write (M / S) -> Database Using multiple Slave -> increase Cache (memcache) -> Go Redis

Write to solve problems:
Split level, split the table, some users will be on the table, some users on another table;

Reliability requirements
Cache of the "avalanche" issue tangled
Cache is facing the challenges of rapid recovery

Development cost requirements
DB Cache consistency and increasing maintenance costs (to clean your DB, and then clean the cache, ah, too slow!)
Developers need to keep up with the influx of demand for products
The most expensive is the cost of the hardware aspects of the database machine, substantially more expensive than the front end of the machine several times, mostly IO-intensive, a waste of hardware;

Maintenance complexity
Consistency maintenance costs higher and higher;
BerkeleyDB use B-tree, will always write a new internal organization will not have to re-file; this will lead to more and more documents; big time required for filing, archiving operations to be regular;
Thus, we need to have some down time;

Based on the above considerations, chose Redis

2. Find ways open source software and criteria

For the open-source software, the first to see what it can do, but more need to focus on what it can not do, it will be a problem?

Rises to a certain size, there may be a problem, whether to accept?

On google code, foreign forum to find material (domestic than foreign technology lag 5 years)

Observation of the individual code level

Redis application scenarios

1. Business use

hash sets: Watchlist, fan list, two-way Watchlist (key-value (field), sorting)

string (counter): the number of microblogging, number of fans, ... (to avoid select count (*) from ...)

sort sets (automatic sorting): TopN, popular microblogging, automatic sorting

lists (queue): push / sub reminder, ...

The four, from fine control, hash sets and string (counter) is recommended, sort sets and lists (queue) is not recommended
But also by secondary development, streamline. For example: storage character to store plastic, 1.6 billion data only 16G memory
Stored within the storage type three kinds, recommended not more than three kinds;

The memcache + myaql replaced by Redis:
Redis as a store and provide access to background is no longer used mysql, solve the problem of data consistency between multiple copies;

2. The storage of large data tables

(Eg: 140 words microblogging storage)
A library to keep the uniqueness of id and 140 words;
Another stock id and user name, release date, clicks, and other information used to calculate, sort, etc., in order to calculate the final required data and then to show the first library to extract the micro-blog content;

Improved three steps:
1) found problems with the existing system;
2) the discovery of new things, how to see how well the new comprehensive turn things;
3) rational regression to determine which fit the new stuff, which is not suitable, not appropriate to move back to the old system

3. Some tips

Many applications can withstand a database connection failure, but can not afford a slow process

Copy of the data, multiple indexes (for different query scenes)

The only way to solve the IO bottleneck: memory used

Changes in the amount of data that is not the case, preferred Redis

Problems encountered and solutions

(Note: are particularly large amount of time there will be, how small the amount of hard to say)

1.Problem: After Replication interrupt retransmission -> network burst traffic

Solution: Replication rewrite the code, rdb + aof (rolling)

2.Problem: capacity issues

Solution: Capacity planning and M / S features of sharding (share nothing, abstracted from the associated data between data objects is very small)
Add some configuration, split, for example: 1,2,3,4, the machine 1 processing, machine processing 2% 2% 2 = 0 = 1.
Less than half the amount of memory, otherwise the expansion (proposed data Redis instance, the maximum should not exceed 80% of the memory)
Our line 96G / 128G memory server does not recommend a single-instance capacity of more than 20 / 30G.
Microblogging application data in a single table top has a 2T data, but it has some application powerless;
Each port should not exceed 20G; test time save disk do need, how long can all be written; the larger the memory, the longer the time to write;
Single Instance large amounts of memory, the immediate problem is caused by the recovery time from a library or Rebuild a long time, for ordinary hard disk loading speed, our experience is generally redis load 1G 1 minute; (loaded speed depends on the complexity of the data volume size and data)
When Redis rewrite aof and save rdb, it will bring a very large and long system pressure, and memory is used, it may lead to insufficient system memory line failure and other serious impact on performance.

reblance: redistribute existing data in accordance with the above configuration.
Rear intermediate layer, routing of HA;
Note: The current official are doing this thing, Redis Cluster, HA solve the problem;

3. Problem: bgsave or bgwriteaof ice crystals problem

Solution: Disk Performance Planning and write speed limitations, such as: provision disk speed of 200M / s write, steady, even though the arrival of large amounts of data but be aware that the write speed to meet the two objective limits:
Compliance with disk speed
Comply with the time limit (guaranteed before the arrival of the peak, you have to finish)

4.Problem: operation and maintenance problems

1) Inner Crontab: Redis move to the Crontab inside, reducing migration time pressure
The multi-port at the same time to avoid doing - can do
The same business multiport (distributed multi-machine), while avoiding doing - do
2) Live Upgrade: .so file loaded first, and then manage the configuration, switch to the new code (Config set command)
The improvement of redis things are packaged into lib.so file, so to support dynamic upgrade
To change their own time to consider upgrading the community. When the community has a new version, new features good use of time, to be able to easily merge well with our improved version;
Upgrade Prerequisites: Modular, module units upgrade
Loading time depends on two aspects: the data size, the complexity of the data structures in general, 40G data takes 40 minutes
Two core issues of distributed systems: A. B.HA routing problem

3) handling of dangerous command: For example: fresh all deletes all the data have to be controlled
Operation and maintenance can not talk about data backup, data recovery have to consider the time required;
Increasing the certification authority (administrator privileges only) eg: flashall certification authority, and must have a password to do;
Of course, high-speed data exchange are not normally carried in every certification authority, a common treatment strategy for the first time certification, recertification late do not have;
Control hash policy (no key, it can not find the value; do not know the hash policy, you can not get key)

4) Config Dump:
Dynamic memory configuration item modified according to certain policies written to disk (Redis already supported)
5) bgsave bring aof write very slowly:
fdatasync When do bgsave, not sync aof (have data access)
6) cost: (22T memory, there 10T for counting)
Redisscounter (16 Yi data occupancy 16G memory) - all become integral storage remainder (string, etc.) do not full
Redis + SSD (counterService counter service)
Order increment, table in the order written, filled with 10 table automatically fall (to SSD)
Storage tiering: memory allocation problem, 10K and 100K wrote a piece of debris will only waste Sina has been optimized to less than 5% (already good!)

5.Problem: Distributed Problem

1.Config Server: namespace, particularly access to tell, are not suitable for the agent because the agent to reduce speed, but, Sina with a (single multi-port, Redis Cluster, sentinel)
Zookeeper put on Config Server
Foremost is naming service, followed by a stateless twmemproxy (twitter improved, written in C), it is redis behind;
2.twmemproxy
Applications need not be concerned about the connection fails, the agent responsible for reconnection
The Hash algorithm into agents
Upgrade agents behind the front end does not care to solve the problem of HA
No state, more than one agent does not matter
3.AS -> Proxy -> Redis
4.Sina of Redis are stand-alone version, and Redis-Cluster exchanged too complicated, do not use
HA do so, we must do with the monitor, then if hung up, how to do the follow-up;
Not a single pursuit of performance, but the throughput of the cluster, which can support wireless extensions;

Experience

Planning well in advance the amount of data to reduce sharding (Internet companies are generally in years)

Only keep meticulous data (memory will be a premium!)

Storing user data Dimensions
Object Dimension Data have a life cycle
Especially in particularly large amount of data when it is necessary to be divided up;

Common process exposed and services: IP -> Load Balancing -> domain name -> Naming Service (a table: Name + resource (IP + port))

For hardware consumption, IO, network and CPU compared, Redis is the most consumed CPU, complex data types will certainly lead to CPU consumption;

Weibo timeout response time currently set to 5s; (very slow return to record key, need to record analysis, slow log);

Data backup to run on a regular basis about the production data; to check the validity of the backup data;

slave master will certainly be linked to more than compare the effects caused; Weibo currently using M / S is a drag, mainly used for disaster recovery;
Synchronization is a separate process to fork and slave synchronize; the process will not take up the query;

After the upgrade to 2.6.30 linux kernel;
Well at least to 2.6.30 deal with the problem of soft interrupt, performance improvement effect is obvious, almost 15% to 30% of the gap;

redis not have separate read and write, each request is a single thread, why should separate read and write.
     
         
         
         
  More:      
 
- Ubuntu 14.04 Enable root and disable the guest (Linux)
- How Bluetooth turned off by default in Ubuntu 14.04 (Linux)
- C # DateTime structure common method (Programming)
- Ubuntu installed Komodo editor by PPA (Linux)
- CentOS 6.4 under PXE + Kickstart unattended operating system installation (Programming)
- OpenSUSE / Linux network configuration (Linux)
- Linux security configuration (Linux)
- Circular list of Java programming (Programming)
- Source encountered problems and solutions when installing SaltStack (Server)
- MySQL tmpdir parameter modification (Database)
- Nginx Load Balancing (standby) + Keepalived (Server)
- Install multiple Linux distributions and Fedora 21 first experience on the same hard disk (Linux)
- Setup Qt development environment under Linux (Linux)
- Python 3.5 will support Async / Await Asynchronous Programming (Programming)
- Singleton (Linux)
- Linux Kernel 4.2 Installation Instructions (Linux)
- How to install the client sqlplus under linux (Database)
- Ubuntu install Lighttpd + Mono support .net (Linux)
- Linux simple commands (Linux)
- Django1.8 return json json string and the string contents of the received post (Programming)
     
           
     
  CopyRight 2002-2022 newfreesoft.com, All Rights Reserved.