KR20190081306A - Method for allocating resource for bigdata analyzing software and apparatus for allocating virtual resource using the method - Google Patents

Abstract

Disclosed is a method for dynamically allocating resources to big data analysis software using a cloud resource. The method comprises the steps of: executing analysis software performing analysis based on big data; monitoring an analysis shape of the analysis software; grasping resource state information according to the analysis shape; and determining a cloud resource to be allocated to the analysis software according to the resource state information.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to a resource allocation method for a large data analysis software, a virtual resource allocation method for a large data analysis software,

The present invention relates to a resource allocation method for big data analysis software, a virtual resource allocation apparatus using the method, and more particularly, to a method for dynamically allocating resources to big data analysis software, a virtual resource allocation Device and a cloud system.

Recently, data types are becoming diverse, and as the amount and size of data becomes larger, there is an increasing tendency to provide application services through analysis and insight of data. The analysis environment plays an important role in providing these application services, and it has a great influence on the speed and accuracy of service provision, and furthermore, it is the key to determining the success or failure of service provision.

However, this analytical environment is currently in the form of a developer who is a cloud user requesting resources required for development in advance, and a cloud provider providing the requested resources. It is also used to identify user ratings, provide cloud hardware resources by rating, or re-scale to provide cloud resources. In addition, in the current analysis environment, cloud resources are allocated according to resource usage by monitoring instance resource usage. That is, in the existing analysis environment, all resource allocations are statically determined before program development.

However, a developer who is a user of a real cloud resource can use different resources such as CPU / GPU and storage according to the configuration of analysis of the currently developed analysis software program, and needs an alternative for this. Therefore, there is a need for methods to dynamically allocate resources to the software reflecting the resource shape of the analysis software currently being performed.

In order to solve the above problems, an object of the present invention is to provide a method of dynamically allocating resources to big data analysis software.

Another object of the present invention is to provide an apparatus for allocating cloud virtualization resources to analysis software for analyzing big data using the above method.

Another object of the present invention is to provide a cloud system including the cloud virtualization resource allocation apparatus.

According to another aspect of the present invention, there is provided a method for allocating resources to a large data analysis software, the method comprising: allocating resources to dynamic analysis software for analyzing big data using cloud resources; Executing analytical software to perform an analysis with the analysis software; Monitoring an analysis shape of the analysis software; Analyzing the resource status information according to the analysis shape; And determining a cloud resource to be allocated to the analysis software according to the resource status information.

The step of executing the analysis software for performing the analysis based on the big data may include collecting at least one of the information on the location of the analysis object and the analysis log in the cloud.

The resource status information may be provided in the form of an API (Application Programming Interface).

The cloud resource is a virtualization resource and may include at least one of a central processing unit (CPU), at least one graphics processing unit (GPU), and at least one memory residing in the cloud.

The resource status information may include at least one of an amount of data to be analyzed and a repetition frequency.

The resource allocation method for the big data analysis software may further include allocating the resource to the analysis software according to the determined cloud resource.

The resource allocation method may operate on a platform as a service (PaaS) and an infrastructure as a service (IaaS) of a cloud system.

The step of executing analysis software for performing analysis based on the big data may be executed in response to an analysis request from SaaS (Software as a Service) of the cloud system.

According to another aspect of the present invention, there is provided an apparatus for dynamically allocating resources to analysis software for analyzing big data using a cloud virtualization resource, the apparatus comprising: a processor; And a memory for storing at least one instruction executed via the processor.

The at least one instruction causing the computer to execute analysis software that performs analysis based on the Big Data; Instructions for monitoring an analysis feature of the analysis software; An instruction to grasp resource status information according to the analysis shape; And an instruction to determine a cloud resource to be allocated to the analysis software according to the resource status information.

The at least one command may further include instructions for collecting at least one of a location of a data to be analyzed and information on an analysis log in the cloud.

The at least one instruction may further include instructions for causing the analysis software to allocate the resource according to the determined cloud resource.

The virtualization resource allocation apparatus may operate based on at least one of a platform as a service (PaaS) and an infrastructure as a service (IaaS) of a cloud system.

According to another aspect of the present invention, there is provided a cloud system, comprising: an analysis software for performing analysis based on big data, A platform as a service (PaaS) layer for determining a cloud resource to be allocated to the analysis software according to the resource status information; And an IaaS (Infrastructure as a Service) layer that allocates the resources to the analysis software according to the cloud resource determined from the PaaS layer.

The cloud system may also include a Software as a Service (SaaS) layer requesting to perform an analysis on the analysis software.

The cloud resource is a virtualization resource and may include at least one of a central processing unit (CPU), at least one graphics processing unit (GPU), and at least one memory residing in the cloud.

According to the embodiments of the present invention as described above, the developer can determine the resources required by himself / herself while executing the analysis program without requesting the cloud resource in advance to perform the developed analysis software.

Also, since the necessary resources are allocated by the program itself, there is no need for a cloud user or resource provider pre-protocol.

In addition, appropriate resources are allocated to the analysis program to be performed, so that resources can be efficiently used.

1 is a configuration diagram of a cloud system according to an embodiment of the present invention.
2 is a block diagram of PaaS in a cloud system according to an embodiment of the present invention.
3 is a block diagram of IaaS in a cloud system according to an embodiment of the present invention.
4 is a flowchart illustrating an operation of a resource allocation method for a big data analysis software according to an embodiment of the present invention.
5 is a block diagram of a virtualization resource allocation apparatus according to an embodiment of the present invention.

While the invention is susceptible to various modifications and alternative forms, specific embodiments thereof are shown by way of example in the drawings and will herein be described in detail. It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. Like reference numerals are used for like elements in describing each drawing.

The terms first, second, A, B, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component. The term "and / or" includes any combination of a plurality of related listed items or any of a plurality of related listed items.

It is to be understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, . On the other hand, when an element is referred to as being "directly connected" or "directly connected" to another element, it should be understood that there are no other elements in between.

The terminology used in this application is used only to describe a specific embodiment and is not intended to limit the invention. The singular expressions include plural expressions unless the context clearly dictates otherwise. In the present application, the terms "comprises" or "having" and the like are used to specify that there is a feature, a number, a step, an operation, an element, a component or a combination thereof described in the specification, But do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. Terms such as those defined in commonly used dictionaries are to be interpreted as having a meaning consistent with the contextual meaning of the related art and are to be interpreted as either ideal or overly formal in the sense of the present application Do not.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a configuration diagram of a cloud system according to an embodiment of the present invention.

A cloud system is a system that provides virtualized computing resources as services through a network. Users can leverage the hardware resources and software resources required by the service as much as necessary without holding them directly. The cloud system can be classified into an infrastructure as a service (IaaS), a platform as a service (PaaS), and a software as a service (SaaS) according to the providing model.

Referring to FIG. 1, a cloud system according to an embodiment of the present invention includes a software as a service (SaaS) 100 for providing application programs operating in a cloud environment in a service form, A platform as a service (PaaS) 200, an infrastructure as a service (IaaS) 200 that enables easy and convenient utilization in a virtual environment such as server resources, IP, network, storage, (300).

SaaS 100 is a service that provides software, that is, an application as a service. That is, the SaaS 100 provides a service environment 110 for SaaS application management so that a user can access an application service and provide various application services. In the service environment 110, SaaS management for at least one SaaS application is performed.

PaaS (200) provides a development platform as a service so that users can develop and distribute software. For example, a service that acts as a platform, such as a desktop window or iOS or Android on a mobile device. The PaaS 200 is not an operating system, but plays a role similar to an operating system in terms of providing an application programming interface (API) and a framework.

An API is a language or message format used when an application communicates with a system program, such as an operating system or database management system, and an API is implemented by calling a function that provides a connection to a particular subroutine for execution within the program . Therefore, an API has several program modules or routines that already exist or must be connected to perform the task requested by the function call.

An API can be called a collection of functions, and programs can use it to make things worse for the operating system to handle. For example, using Windows APIs, a program can open a window, open a file, create a message box, or handle more complex things as a single command.

PaaS also provides a developer platform environment for analytical software developers, where the developer platform is tailored to individual characteristics depending on which framework (eg, analysis tools, analytics environment, etc.) is being analyzed.

The development platform environment of the PaaS 200 can be driven and controlled by the platform engine. Further, each development platform may include an analysis framework according to the present invention.

The IaaS 300 provides the entire infrastructure, including clients, servers, software, data centers, and network equipment, as services. That is, IaaS provides a virtual machine for realizing a development platform and provides a physical virtualization environment 310 such as a repository for storing data for actual analysis execution. In the virtualization environment 310, IaaS management can be performed on at least one virtual machine (VM) instance, storage, and the like. The virtualization environment 310 may be driven or controlled by the virtualization engine.

In order to achieve the object of the present invention based on the cloud system as shown in FIG. 1, an API (Application) which can process the situation of the analysis configuration information of the analysis software program developed by the developer who is an actual cloud resource user Protocol interface and software components should be provided and software components that dynamically allocate cloud resources such as CPU (Central Processing Unit) / GPU (storage) and storage should be provided according to the provided shape information.

More specifically, two main functions are required for realization of the present invention.

First, the development platform 210, which is a platform service of the cloud for developing a user application service, needs to perform a framework for analysis among various platform providing functions.

The analysis framework executes the analysis software that performs the analysis based on the big data developed by the developer and monitors the shape of the software to be analyzed by the analysis shape information processing function when the analysis using the actual cloud resource is executed. In addition, analytical data processing functions are also performed to identify the data to be analyzed. Such situation information can be utilized by the resource decision processing function or the storage data acquisition functions for controlling the actual cloud resource obtained as the API type information so that the situation information can be grasped by any analysis software.

Second, the resource allocation function is required to allocate the actual cloud resource based on the resource situation information mentioned in the first. These resource allocation functions determine whether to allocate a GPU according to a resource situation, whether it is a grid type, an independent GPU, a CPU should be allocated, and a memory capacity do. In addition, searching for stored data among the context information is performed to determine how much storage is to be allocated to the cloud storage, so that actual resource allocation is performed.

2 is a block diagram of PaaS in a cloud system according to an embodiment of the present invention.

FIG. 2 is a conceptual diagram for processing the situation of analysis information when performing an analysis on a development platform. SaaS 100 requests analysis of the analysis software (e.g., application 1 of FIG. 2) to the PaaS development platform via the API to obtain analysis results for the application service.

In the development platform environment of PasS (200) receiving the analysis request, analysis framework among the various platform providing functions on the development platform, which is a platform service of the cloud for developing user application service, is executed.

The analysis framework 210 may include a big data analysis module 211, a resource determination module 212, and a storage data acquisition module 213.

The big data analysis module 211 analyzes the associated resources and acquires the analysis shape information while the analysis software is executed. Specifically, the big data analysis module 211 may include an analysis shape information processing module and an analysis data information processing module.

In the analysis framework, the analysis software that performs analysis based on the big data developed by the developer is executed using the actual cloud resource. At this time, the analysis shape information processing module 211 monitors the shape of the analysis software do. Such resource status information can be provided in an API form (206) so that the resource status information can be grasped regardless of any analysis SW. Here, the resource status information may include information on the amount of data to be analyzed or the number of analyzes to be repeated.

In addition, the analysis data information processing module 207 collects and processes the location of the data to be analyzed or log information to analyze the data to be analyzed. At this time, the analysis data information processing module 207 requests the corresponding information to the big data engine or the DB repository existing in the cloud in API form. Here, the analysis data information processing module 207 can request the corresponding information rule to the IaaS through the stored data acquisition module 213, and receive the information retrieved from the IaaS.

The resource status information acquired through the big data analysis module 211 is provided to the resource determination module 212 for cloud resource control and processed. The resource determination module 212 substantially determines resources to be allocated to the analysis software based on the resource status information.

3 is a block diagram of IaaS in a cloud system according to an embodiment of the present invention.

More specifically, FIG. 3 illustrates the operation of the IaaS blocks and corresponding blocks that allocate analysis resources and process the stored data to the analysis software in a cloud infrastructure virtualization environment in accordance with an embodiment of the present invention.

In order to allocate resources for the Big Data Analysis software according to the present invention, the IaaS 300 layer may include a resource allocation module 310 and a stored data retrieval module 320.

The resource allocation module 310 maps the virtualization resources 330 of the virtualization engine to the analysis software according to the resource conditions by mapping the virtualization resources to the actual cloud resources based on the resource status information obtained through the layer of the PaaS 200. [ The virtualization resource 330 may include at least one VM instance, and each VM instance may include a GPU, a CPU, and storage.

More specifically, the resource allocation module 310 determines whether to allocate a GPU to the analysis software, allocate a GPU, be in a grid form, be an independent GPU, or allocate a CPU, Whether or not it is possible to perform the assignment, and so on.

Meanwhile, the stored data search module 320 performs a search for stored resources among the context information. The stored data retrieval module 320 may perform a search for the stored resources through a search for the virtualized big data or DB repository 340. [ To this end, the Big Data or DB repository 340 may comprise a collection module and a storage module serving as a collection repository for the collected data.

The resource allocation module 310 can determine the extent to which cloud storage is allocated to the analysis software based on the search result of the storage resource, and ultimately, actual resource allocation is performed based on the determination.

4 is a flowchart illustrating an operation of a resource allocation method for a big data analysis software according to an embodiment of the present invention.

The resource allocation method for the big data analysis software shown in FIG. 4 may be performed in part or in whole by the virtualization resource allocation apparatus according to the embodiment of the present invention, the analysis framework provided in the development platform in PaaS, . However, the operation subject of the resource allocation method for the analysis software is not limited to these.

The resource allocation method for the big data analysis software according to an embodiment of the present invention is a method of dynamically allocating resources to analysis software for analyzing big data using cloud resources.

For example, if the virtualization resource allocation device operating in the PaaS layer in the cloud system receives an analysis request for the analysis software from the SaaS layer in the cloud system (Yes in S410), the analysis requested software is executed (S420) .

When the execution of the analysis software is started, the analysis form of the analysis software is monitored (S430). Here, the step of monitoring the analysis form of the analysis software may include collecting at least one of the information on the location of the analysis target data and the analysis log in the cloud. Then, the resource status information can be grasped according to the analyzed analysis shape (S440). Once the resource status information is obtained, the cloud resource to be allocated to the analysis software is determined according to the identified resource status information (S450). Here, the cloud resource may include virtualization resources.

In addition, in the IaaS layer in the cloud system, the resource can be allocated to the analysis software according to the determined cloud resource (S460).

5 is a block diagram of a virtualization resource allocation apparatus according to an embodiment of the present invention.

The virtualization resource allocation apparatus 220 according to an embodiment of the present invention allocates virtual resources to the big data analysis software, and includes a processor 221, a memory 222 storing at least one instruction executed through the processor, A storage 223, and an input / output interface 224.

The virtualization resource allocation device may be one of at least one server that provides a development platform on the PaaS and may perform operations partially or wholly overlapping with the operation of the development platform or analysis framework described in the embodiment of FIG. have.

The virtualization resource allocation device may also perform operations partially or entirely overlapping with those of the resource allocation module or the storage data retrieval module on the IaaS described above with reference to the embodiment of FIG.

The memory (222) stores at least one instruction executed via a processor, the at least one instruction including instructions for executing analysis software that performs analysis based on the Big Data, instructions for monitoring an analysis shape of the analysis software An instruction to grasp resource status information according to an analysis shape; And an instruction to determine a cloud resource to be allocated to the analysis software according to the resource situation information.

The repository 223 stores information on the location and analysis log of the data to be analyzed in the cloud collected by the processor 221. The repository 223 may also store resource status information captured by the processor 221.

The input / output interface 224 may process inputs from other components in the system and output to other components.

The operation of the method according to an embodiment of the present invention can be implemented as a computer-readable program or code on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. The computer-readable recording medium may also be distributed and distributed in a networked computer system so that a computer-readable program or code can be stored and executed in a distributed manner.

Also, the computer-readable recording medium may include a hardware device specially configured to store and execute program instructions, such as a ROM, a RAM, a flash memory, and the like. Program instructions may include machine language code such as those produced by a compiler, as well as high-level language code that may be executed by a computer using an interpreter or the like.

While some aspects of the invention have been described in the context of an apparatus, it may also represent a description according to a corresponding method, wherein the block or apparatus corresponds to a feature of the method step or method step. Similarly, aspects described in the context of a method may also be represented by features of the corresponding block or item or corresponding device. Some or all of the method steps may be performed (e.g., by a microprocessor, a programmable computer or a hardware device such as an electronic circuit). In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined in the appended claims. It can be understood that it is possible.

100: SaaS 110: Service environment
200: PaaS 210: Development platform environment
211: Big Data Analysis Module 212: Resource Decision Module
213: Stored data acquisition module 220: Virtual resource allocation device
221: processor 222: memory
223: storage 224: input / output interface
300: IaaS 310: Virtualization Environment

Claims (18)

CLAIMS 1. A method of dynamically allocating resources to analysis software for analyzing large data using cloud resources,
Executing analytical software that performs analysis based on the Big Data;
Monitoring an analysis shape of the analysis software;
Analyzing the resource status information according to the analysis shape; And
And determining a cloud resource to be allocated to the analysis software according to the resource status information. How to allocate resources to big data analysis software. The method according to claim 1,
Wherein the executing the analysis software for performing analysis based on the big data comprises:
And collecting at least one of information about the location of the analysis target data and the analysis log in the cloud. The method according to claim 1,
Wherein the resource status information is provided in the form of an API (Application Programming Interface). The method according to claim 1,
Wherein the cloud resource is a virtualization resource and comprises at least one of a central processing unit (CPU), at least one graphics processing unit (GPU), and at least one memory residing in the cloud, Resource allocation method for. The method according to claim 1,
Wherein the resource status information comprises:
The amount of data to be analyzed, and the number of iterations to be performed repeatedly. The method according to claim 1,
And allocating the resource to the analysis software in accordance with the determined cloud resource. The method according to claim 1,
Wherein the resource allocation method operates on a platform as a service (PaaS) and an infrastructure as a service (IaaS) of a cloud system. The method of claim 7,
Wherein the step of executing analysis software that performs analysis based on the Big Data is executed in response to an analysis request from SaaS (Software as a Service) of the cloud system. An apparatus for dynamically allocating resources to analysis software for analyzing big data using cloud virtualization resources,
A processor; And
And a memory for storing at least one instruction executed via the processor,
Wherein the at least one instruction comprises:
Instructions to execute analysis software that performs analysis based on the Big Data;
Instructions for monitoring an analysis feature of the analysis software;
An instruction to grasp resource status information according to the analysis shape; And
And to determine a cloud resource to be allocated to the analysis software according to the resource status information. The method of claim 9,
Wherein the at least one instruction comprises:
Further comprising instructions for collecting at least one of information about the location of the data to be analyzed and the analysis log in the cloud. The method of claim 9,
Wherein the resource status information is provided in the form of an API (Application Programming Interface). The method of claim 9,
Wherein the cloud resource is a virtualization resource and comprises at least one of a central processing unit (CPU), at least one graphics processing unit (GPU), and at least one memory residing within the cloud. The method of claim 9,
Wherein the resource status information comprises:
The amount of data to be analyzed, and the number of iterations to be performed repeatedly. The method of claim 9,
Wherein the at least one instruction comprises:
And to cause the analysis software to allocate the resource according to the determined cloud resource. The method of claim 9,
Wherein the virtualization resource allocation device comprises:
And operating based on at least one of Platform as a Service (PaaS) and Infrastructure as a Service (IaaS) of the cloud system. The analyzing software for analyzing based on the big data is executed to monitor the analysis shape of the analysis software, grasp the resource situation information according to the analysis shape, and calculate the cloud resource to be allocated to the analysis software according to the resource situation information A Platform as a Service (PaaS) layer for determining; And
And an IaaS (Infrastructure as a Service) layer for allocating resources to the analysis software according to the cloud resources determined from the PaaS layer. 18. The method of claim 16,
Further comprising a Software as a Service (SaaS) layer requesting execution of an analysis on the analysis software. 18. The method of claim 16,
Wherein the cloud resource is a virtualization resource and comprises at least one of a central processing unit (CPU), at least one graphics processing unit (GPU), and at least one memory residing within the cloud.

Images