Detailed Description
In order to make those skilled in the art better understand the technical solutions in the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art without any inventive work based on the embodiments in the present application shall fall within the scope of protection of the present application.
Fig. 1 shows a schematic diagram of a distributed data structure. In the distributed database architecture, a control node and a plurality of data nodes are included, and the control node is connected with each data node in a communication mode. And the control node receives the operation instruction of the database, analyzes the operation instruction and distributes calculation tasks to the data nodes according to the analysis result. And after the data node executes the calculation task, the calculation result is fed back to the control node. And the control node collects the calculation results fed back by the data nodes, and feeds back the collected calculation results to the user as the execution results of the operation instruction. Each database node comprises a control module, a calculation module and a storage module. The storage module is used for storing data in a database table, the calculation module is responsible for calculation tasks of the data in the storage module, and the control module is used for performing data interaction (such as receiving control distributed tasks, sending calculation results to the control nodes and the like) with the control nodes, controlling internal processing flows of the nodes and the like.
The embodiment of the specification provides a database building method, and the method can automatically complete the building work of a database by adopting a management and control server. The control server is a server different from the control node and the data node. Before executing the database building method, a database building person only needs to configure the management and control server, upload a required file to the management and control server, and open a physical connection channel between each server for building data and the management and control server.
As shown in fig. 2, the database building method provided by the embodiment of the present specification includes the following steps, which may be performed by the administrative server.
S110: and receiving a database building request, wherein the database building request carries the performance requirement of the database.
The performance requirement may be, for example, a computing power requirement, a storage power requirement, a disk capacity, the number of control nodes and data nodes, the maximum allowable number of connections of the database, the maximum allowable number of accesses of the database, a limit on the number of computations per unit time, a limit on the number of queries per unit time, and the like. Demands such as "8 cores + 64G" and "4 cores + 32G" are typically carried, where 8 cores and 4 cores refer to computing power, and 64G and 32G refer to data storage power.
In some embodiments, the database building request may be entered manually, such as through a command line window.
In some embodiments, the database building request is given by other electronic devices in communication connection with the management and control server, and is used for instructing the management and control server to build an automatic database.
S120: and selecting a template matched with the performance requirement from a preset template library as a target template.
The template library is preset in the management and control server, each template in the template library comprises a combination of a control node and a plurality of data nodes, each combination can realize a database architecture, and each node in the combinations is recorded in the template through a database node name. The combination also comprises the number of data nodes, the amount of computing resources and the amount of storage resources of the control node, the amount of computing resources and the amount of storage resources of each data node, and the maximum computing capacity, the maximum storage capacity, the maximum allowable connection number, the maximum allowable access times, the limit of computing times in unit time, the limit of query times in unit time and the like of the database formed by the combination.
The template matched with the performance requirement may be any one of the above combinations that meets all the requirements in the performance requirement, or may be a designated combination of the above combinations that meets all the requirements in the performance requirement, where the designated combination may be a combination with the smallest number of data nodes, a combination with the smallest amount of computing resources or storage resources of the control node, or a combination with the smallest amount of computing resources or storage resources of the data nodes.
A template that matches performance requirements may be exactly matched to the performance requirements on one hand, but much larger on the other hand. For example, the computing power requirement is 8G, the storage power requirement is 64G, and the matched templates in the template library only have the combination of "" 8G +128G "".
S130: the number of data nodes in the target template, the calculation resource quantity and the storage resource quantity of the control node, and the calculation resource quantity and the storage resource quantity of each data node are obtained.
S140: and selecting one or more target servers matched with the number of the data nodes in the target template, the calculation resource quantity and the storage resource quantity of the control node and the calculation resource quantity and the storage resource quantity of each data node from a plurality of target servers managed by the management and control server to create the database node.
Nanotube, i.e., inclusion management. Means that the target server can be operated by the management and control server through the network. The management and control server manages a plurality of servers, and the servers can be used as target servers and selected to be used for creating database nodes according to the performance requirements of the database.
S150: and configuring a database component for each selected target server according to the node type matched with each selected target server to form a database.
And the node type is a control node and a data node in the database architecture. Generally, there is one control node and a plurality of data nodes.
The target template comprises two types of control nodes and data nodes, and when the matched target server is selected, the target server to be selected needs to be matched with information such as the calculation resource amount and the storage resource amount of the control nodes and the data nodes in the target template. The node type matched with each selected target server is the node type corresponding to the finally selected target server when the finally selected target server is matched with information such as the computing resource amount and the storage resource amount of the control node and the data node in the target template.
According to the database building method, the database is automatically built through the execution steps of the control server, compared with the prior art of building the database manually, the time consumption is short, the manual execution operation is less, and the database building efficiency can be improved.
In some embodiments, the management server may select a specified installer from a pre-stored installer library, the specified installer being an installer corresponding to a node type that matches each of the selected target servers, then send the specified installer to the selected target server, and then control the selected target server to execute the installer.
In some embodiments, as shown in fig. 3, step S150 may include the following steps:
s151: selecting a specified mirror image file from a pre-stored mirror image file library; the specified image file is an image file corresponding to the node type matched with each selected target server.
The mirror image file library only stores corresponding mirror image files for different types of nodes. That is, the mirror image file library does not store the mirror image files of all different nodes in the database architecture.
S152: sending the specified mirror image file to the selected target server, wherein the mirror image file carries: and forming a file of the database component, wherein the database component comprises a database node name.
The database node name is a server identifier for distinguishing each server constituting the database. For example, CN denotes a control node server, and DN1, DN2, and DN3 denote three data node servers, respectively.
S153: and mounting the mirror image file to the file system directory of the selected target server.
S154: and replacing the database node name in the mirror image file with the database node name in the target template.
Because the combination of the control nodes and the data nodes in the target template forms a database architecture, and the nodes in the combination are recorded in the target template through the database node names, the database node names in the mirror image files mounted by the selected target server are replaced by the database node names in the target template, so that the selected target server can be connected to realize the database architecture.
In some embodiments, the target server hosted by the governing server has not yet installed an operating system. In this case, the image files in steps S151 to S154 may further include files constituting an operating system, and accordingly, after the image file is mounted to the file system directory of the selected target server, the selected target server has the operating system.
The database assembly and the operating system of the target server are formed in the form of mounting the mirror image file, the process is simple, the consumed time is short, and the process of building the database can be accelerated.
In some embodiments, before step S110, account information of the target server is further obtained, the target server is logged in through the account information, and the network configuration and the directory structure of the file system in the target server are copied to the management and control server, so that the management and control server manages the target server.
In some embodiments, before step S110, account information of the target server is further obtained, the target server is logged in through the account information, and a monitoring software client is installed in the target server to supervise the target server, so that the management and control server manages the target server.
Two ways for the management server to manage the target server are given above.
In some embodiments, as shown in fig. 4, after step S150, the following steps are further included:
s160: and controlling each target server in the formed database to start the database process.
S170: it is checked whether a database process exists in each target server.
In some embodiments, the management and control server may directly log in each node server through an account and a password of each node server, start a database process by controlling a database component on each node server to run, and directly check whether a database process exists in the process list.
In some embodiments, the management and control server may also start the database process by controlling the operation of the database component on each node server through the monitoring software client pre-installed on each node server, and check whether the database process exists in the process list through the monitoring software client.
S180: testing the formed database in the case that the database process is determined to exist, wherein the testing comprises at least one of the following: adding operation, deleting operation, modifying operation and inquiring operation.
Therefore, step S170 is to test whether the database process can be normally started, and step S180 is to test whether the database formed by the joint test of the servers can realize the most basic operations of adding, deleting, and modifying the database.
In some embodiments, the "testing the formed database" in step S180 includes:
s181: and acquiring an address link and a user account of the formed database.
S182: and connecting to the formed database through an address link.
S183: after linking to the formed database, logging in the formed database through the user account.
S184: and sending a test instruction to the database under the condition that the login is successful, wherein the test instruction comprises at least one of the following: adding instructions, deleting instructions, modifying instructions and inquiring instructions.
In some embodiments, the "testing the formed database" in step S180 includes:
s185: and sending the pre-stored test script to a control node server in the formed database.
S186: controlling the test script to execute through a monitoring software client pre-installed on a control node server so as to send a test instruction to a database, wherein the test instruction comprises at least one of the following: adding instructions, deleting instructions, modifying instructions and inquiring instructions.
The embodiment of the specification provides a database building device, which can be used for realizing the database building method shown in fig. 2. As shown in fig. 5, the apparatus includes a receiving module 10, a selecting module 20, a first obtaining module 30, a selecting module 40, and a configuring module 50.
The receiving module 10 is configured to receive a database construction request, where the database construction request carries a performance requirement of a database.
The selection module 20 is configured to select a template matching the performance requirement from a preset template library as a target template.
The first obtaining module 30 is configured to obtain the number of data nodes in the target template, the amount of computing resources and the amount of storage resources of the control node, and the amount of computing resources and the amount of storage resources of each data node.
The selecting module 40 is configured to select, from a plurality of target servers managed by the management and control server, one or more target servers matched with the number of data nodes in the target template, the amount of computing resources and the amount of storage resources of the control node, and the amount of computing resources and the amount of storage resources of each data node to create a database node.
The configuration module 50 is configured to configure a database component for each selected target server according to the node type matched with each selected target server, so as to form a database.
In some embodiments, the target template further comprises a database node name; as shown in fig. 6, the configuration module 50 includes a selection sub-module 51, a sending sub-module 52, a mounting sub-module 53, and a replacement sub-module 54.
The selection sub-module 51 is used for selecting a specified image file from a pre-stored image file library; and the specified mirror image file is a mirror image file corresponding to the node type matched with each selected target server.
The sending submodule 52 is configured to send the specified image file to the selected target server, where the image file carries: and forming a file of a database component, wherein the database component comprises a database node name.
The mounting submodule 53 is configured to mount the image file to a file system directory of the selected target server.
The replacing sub-module 54 is configured to replace the database node name in the image file with the database node name in the target template.
In some embodiments, the database building apparatus further comprises a second obtaining module, a logging module, and a copying module.
The second acquisition module is used for acquiring the account information of the target server. And the login module is used for logging in the target server through the account information. The copying module is used for copying the network configuration and the directory structure of the file system in the target server to the management and control server.
In some embodiments, the database building apparatus further includes a third obtaining module, a login module, and an installation module.
The third acquisition module is used for acquiring the account information of the target server. And the login module is used for logging in the target server through the account information. The installation module is used for installing a monitoring software client in the target server so as to supervise the target server.
In some embodiments, the database building apparatus further comprises a start control module, an inspection module and a test module.
The starting control module is used for controlling each target server in the formed database to start the database process. The checking module is used for checking whether a database process exists in each target server. The testing module is used for testing the formed database under the condition that the database process is determined to exist, wherein the testing comprises at least one of the following steps: adding operation, deleting operation, modifying operation and inquiring operation.
In some embodiments, the test module includes an acquisition submodule, a connection submodule, a login submodule, and an instruction sending submodule.
And the acquisition submodule is used for acquiring the address link and the user account of the formed database. And the connecting submodule is used for connecting to the formed database through the address link. And the login submodule is used for logging in the formed database through the user account after the formed database is linked. The instruction sending submodule is used for sending a test instruction to the database under the condition that the login is successful, wherein the test instruction comprises at least one of the following: adding instructions, deleting instructions, modifying instructions and inquiring instructions.
In some embodiments, the test module includes a script sending sub-module and a control execution sub-module.
And the script sending submodule is used for sending the pre-stored test script to the control node server in the formed database. The control execution submodule is used for controlling the test script to execute through a monitoring software client pre-installed on the control node server so as to send a test instruction to a database, wherein the test instruction comprises at least one of the following: adding instructions, deleting instructions, modifying instructions and inquiring instructions.
An embodiment of the present invention further provides an electronic device, as shown in fig. 7, the electronic device may include a processor 71, a memory 72, and a communication module 73, where the processor 71, the memory 72, and the communication module 73 may be connected by a bus or in another manner, and fig. 7 takes the connection by the bus as an example.
The processor 71 may be a Central Processing Unit (CPU). The Processor 71 may also be other general purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field Programmable Gate Arrays (FPGAs) or other Programmable logic devices, discrete Gate or transistor logic devices, discrete hardware components, or combinations thereof.
The memory 72, as a non-transitory computer-readable storage medium, may be used to store a non-transitory software program, a non-transitory computer-executable program, and modules, such as program instructions/modules corresponding to the database building method in the embodiment of the present invention (for example, the receiving module 10, the selecting module 20, the first obtaining module 30, the selecting module 40, and the configuring module 50 shown in fig. 5). The processor 71 executes various functional applications and data processing of the processor by running the non-transitory software programs, instructions and modules stored in the memory 72, that is, implements the database building method in the above method embodiment.
The memory 72 may include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created by the processor 71, and the like. Further, the memory 72 may include high speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid state storage device. In some embodiments, the memory 72 may optionally include memory located remotely from the processor 71, and such remote memory may be connected to the processor 71 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The one or more modules are stored in the memory 72 and when executed by the processor 71 perform the database building method as in the embodiment shown in fig. 2.
The details of the electronic device may be understood with reference to the corresponding related description and effects in the embodiment of fig. 2, and are not described herein again.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a Flash Memory (Flash Memory), a Hard Disk (Hard Disk Drive, abbreviated as HDD), a Solid State Drive (SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.
In the 50 s of the 20 th century, improvements in a technology could clearly be distinguished between improvements in hardware (e.g., improvements in circuit structures such as diodes, transistors, switches, etc.) and improvements in software (improvements in process flow). However, as technology advances, many of today's process flow improvements have been seen as direct improvements in hardware circuit architecture. Designers almost always obtain the corresponding hardware circuit structure by programming an improved method flow into the hardware circuit. Thus, it cannot be said that an improvement in the process flow cannot be realized by hardware physical modules. For example, a Programmable Logic Device (PLD), such as a Field Programmable Gate Array (FPGA), is an integrated circuit whose Logic functions are determined by programming the Device by a user. A digital system is "integrated" on a PLD by the designer's own programming without requiring the chip manufacturer to design and fabricate a dedicated integrated circuit chip 2. Furthermore, nowadays, instead of manually making an Integrated Circuit chip, such Programming is often implemented by "logic compiler" software, which is similar to a software compiler used in program development and writing, but the original code before compiling is also written by a specific Programming Language, which is called Hardware Description Language (HDL), and HDL is not only one but many, such as abel (advanced Boolean Expression Language), ahdl (alternate Language Description Language), traffic, pl (core unified Programming Language), HDCal, JHDL (Java Hardware Description Language), langue, Lola, HDL, laspam, hardbyscript Description Language (vhr Description Language), and the like, which are currently used by Hardware compiler-software (Hardware Description Language-software). It will also be apparent to those skilled in the art that hardware circuitry that implements the logical method flows can be readily obtained by merely slightly programming the method flows into an integrated circuit using the hardware description languages described above.
The embodiments in the present specification are described in a progressive manner, and the same and similar parts among the embodiments can be referred to each other, and each embodiment focuses on the differences from the other embodiments.
The systems, devices, modules or units described in the above embodiments may be implemented by a computer chip or an entity, or by a product with certain functions.
For convenience of description, the above devices are described as being divided into various units by function, and are described separately. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.
From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present application may be essentially or partially implemented in the form of software products, which may be stored in a storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and include instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods of some parts of the embodiments of the present application.
The application is operational with numerous general purpose or special purpose computing system environments or configurations. For example: personal computers, server computers, hand-held or portable devices, tablet-type devices, multiprocessor systems, microprocessor-based systems, set top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments that include any of the above systems or devices, and the like.
The application may be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The application may also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote computer storage media including memory storage devices.
Although the present application has been described in terms of embodiments, those of ordinary skill in the art will recognize that there are numerous variations and permutations of the present application without departing from the spirit of the application, and it is intended that the appended claims encompass such variations and permutations without departing from the spirit of the application.