The large dynamic application system platform is mainly aimed at the bottom system architecture of high traffic and high concurrency websites. The operation of large websites needs a reliable, safe, scalable and easy to maintain application system platform as the support to ensure the smooth operation of website applications.
Large dynamic application system can be divided into several subsystems:
(1) Web front-end system
(2) Load balancing system
(3) Database cluster system
(4) Cache system
(5) Distributed storage system
(6) Distributed server management system
(7) Code distribution system
1、 Web front-end system
In order to share servers of different applications, avoid single point of failure, centralized management, unified configuration and other purposes, instead of dividing servers by applications, all servers are used in a unified way. Each server can provide services for multiple applications. When the access of some applications increases, the performance of the entire server cluster can be improved by adding server nodes, At the same time, his application will also benefit.
The web front-end system is based on Apache/Lighttpd/Eginx and other virtual host platforms, providing a java program running environment. The server is transparent to developers and does not require developers to intervene in server management
2、 Load balancing system
The load balancing system is divided into hardware and software.
Hardware load balancing is efficient but expensive, such as F5. The software load balancing system is cheaper or free, and its efficiency is lower than that of the hardware load balancing system. However, it is also sufficient for websites with general or slightly larger traffic, such as lvs and nginx. Most websites use both hardware and software load balancing systems.
3、 Database cluster system
Because the Web front-end adopts a load balanced cluster structure to improve the effectiveness and scalability of the service, the database must also be highly reliable to ensure the high reliability of the entire service system. How to build a highly reliable database system that can provide large-scale concurrent processing?
We can adopt the scheme shown in the figure above:
(1) With MySQL database, considering the characteristics of Web applications with more read and less write databases, we mainly optimized the read database, provided special read database and write database, and implemented read and write operations in applications to access different databases respectively.
(2) The MySQL Replication mechanism is used to quickly replicate the database of the primary database (write database) to the secondary database (read database). One master database corresponds to multiple slave databases, and the master database data is synchronized to the slave database in real time.
(3) There are multiple write databases, each of which can be used by multiple applications. This can solve the performance bottleneck and single point of failure of the write database.
(4) There are multiple read databases, and load balancing devices are used to achieve load balancing, so as to achieve high performance, high reliability and high scalability of read databases.
(5) The database server and application server are separated.
(6) Use BigIP for load balancing from the database.
4、 Cache system
Cache includes file cache, memory cache and database cache. Memory cache is the most used and most efficient in large Web applications. The most commonly used memory cache tool is Memcached. Using the correct cache system can achieve the following goals:
(1) Using cache system can improve access efficiency, server throughput and user experience.
(2) Reduce access pressure to database and save set server.
(3) There are multiple Memcached servers to avoid a single point of failure, provide high reliability and scalability, and improve performance.
5、 Distributed storage system
The storage requirements in the Web system platform have the following two characteristics:
(1) The storage capacity is very large, often reaching a scale that cannot be provided by a single server, such as photo albums, videos and other applications. Therefore, professional large-scale storage systems are needed.
(2) Each node in the load balancing cluster can access any data object, and the processing of data by each node can also be shared by other nodes. Therefore, logically, the data to be operated by these nodes can only be a whole, not independent data resources.
Therefore, high-performance distributed storage system is a very important part for large-scale website applications. (A brief introduction to a distributed storage system needs to be added here.)
6、 Distributed server management system
With the continuous increase of website access traffic, most network services are provided in the form of load balancing clusters. With the expansion of the cluster scale, the original stand-alone server management model can no longer meet our needs. The new needs must be able to manage servers in a centralized, grouped, batch and automated manner, Able to execute planned tasks in batches.
There are some excellent software in the distributed server management system software, among which Cfengine is the ideal one. It can group servers, and different groups can customize system configuration files, scheduled tasks and other configurations. It is based on the C/S structure. All server configuration and management scripts are saved on the Cfengine Server, and the managed server runs the Cfengine Client program. The Cfengine Client regularly sends requests to the server through an SSL encrypted connection to obtain the latest configuration files and management commands, scripts, patch installation and other tasks.
With Cfengine, a centralized server management tool, we can efficiently implement large-scale server cluster management. Managed servers and Cfengine servers can be distributed anywhere. As long as the network can be connected, we can achieve rapid and automated management.
7、 Code release system
With the continuous increase of website access traffic, most network services are provided in the form of load balancing clusters. With the expansion of the cluster scale, in order to meet the batch distribution and update of program code in the cluster environment, we also need a program code publishing system.
This publishing system can help us achieve the following goals:
(1) The server of the production environment provides services in the form of virtual host, without the involvement of developers in maintenance and direct operation, and the delivery system can realize the distribution of programs to the target server without logging on to the server.
(2) We need to manage the four development stages of internal development, internal testing, production environment testing, and production environment release. The release system can be involved in the code release of each stage.
(3) We need to implement source code management and version control, and SVN can implement this requirement.
The common tool Rsync can be used to develop corresponding script tools to achieve synchronous code distribution between server clusters.
If you want to know more, you can trust me personally!
1 SpringBoot+high concurrency message processing EDM? Project practice
2 SpringBoot ELK? Distributed data analysis
3 Netty? High concurrency UTS? Project practice
4 SpringCloud? Design of microservice+NoSQL+load balancing platform
