WO2018188539A1 - Data processing method, terminal, device, and storage medium - Google Patents
Data processing method, terminal, device, and storage medium Download PDFInfo
- Publication number
- WO2018188539A1 WO2018188539A1 PCT/CN2018/082229 CN2018082229W WO2018188539A1 WO 2018188539 A1 WO2018188539 A1 WO 2018188539A1 CN 2018082229 W CN2018082229 W CN 2018082229W WO 2018188539 A1 WO2018188539 A1 WO 2018188539A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- rule
- data
- configuration
- configuration rule
- processed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/44—Arrangements for executing specific programs
- G06F9/445—Program loading or initiating
- G06F9/44505—Configuring for program initiating, e.g. using registry, configuration files
- G06F9/4451—User profiles; Roaming
Definitions
- the present application relates to the field of computer technologies, and in particular, to a data processing method, a terminal, a device, and a storage medium.
- the existing processing methods have the following disadvantages: if the data administrator needs to write the data processing program code every time the data is processed, the processing method of the data increases the data management in a certain program. The workload of the staff and affects the efficiency of the data collection process.
- the embodiment of the present application provides a data processing method, a terminal, a device, and a storage medium, which can reduce the workload of the data administrator and improve the efficiency of the data collection link.
- the embodiment of the present application provides a data processing method, where the method includes:
- Defining a configuration rule for data processing in a database the configuration rule including one or more rules
- the data to be processed collected by the user is received
- the to-be-processed data is processed according to the configuration rule.
- an embodiment of the present application provides a terminal, where the terminal includes:
- a definition unit configured to pre-define a configuration rule of data processing in a database, where the configuration rule includes one or more rules;
- a writing unit configured to write the configuration rule into a description file
- a receiving unit configured to: if the user collects data collected in the database, receive data to be processed collected by the user;
- a reading unit configured to read the description file and parse a configuration rule in the description file
- the first processing unit is configured to process the to-be-processed data according to the configuration rule.
- the defining unit includes:
- a selection unit for selecting a program source for which the rule definition is performed
- a creating unit configured to create the configuration rule according to the program source.
- the embodiment of the present application further provides a data processing device, including: a memory, configured to store a program that implements a data processing method; and a processor, configured to run the data processing method stored in the memory a program to perform the following operations: pre-defining a configuration rule of data processing in the database, the configuration rule includes one or more rules; writing the configuration rule to the description file; and detecting the user collecting data in the database Receiving the to-be-processed data collected by the user; reading the description file and parsing the configuration rule in the description file; and processing the to-be-processed data according to the configuration rule.
- a data processing device including: a memory, configured to store a program that implements a data processing method; and a processor, configured to run the data processing method stored in the memory a program to perform the following operations: pre-defining a configuration rule of data processing in the database, the configuration rule includes one or more rules; writing the configuration rule to the description file; and detecting the user collecting data in the database Receiving the to-be-process
- the embodiment of the present application further provides a computer readable storage medium, where the one or more computer programs are stored, and the one or more computer programs may be one or more
- the processor executes to implement the following steps: pre-defining a configuration rule of data processing in the database, the configuration rule includes one or more rules; writing the configuration rule to the description file; if the user detects the data in the database Receiving, receiving data to be processed collected by the user; reading the description file and parsing a configuration rule in the description file; and processing the to-be-processed data according to the configuration rule.
- the embodiment of the present application can reduce the workload of the data administrator and improve the efficiency of the data collection link.
- FIG. 1 is a schematic flowchart of a data processing method according to an embodiment of the present application.
- FIG. 2 is another schematic flowchart of a data processing method according to an embodiment of the present application.
- FIG. 3 is another schematic flowchart of a data processing method according to an embodiment of the present application.
- FIG. 4 is a schematic block diagram of a terminal provided by an embodiment of the present application.
- FIG. 5 is another schematic block diagram of a terminal provided by an embodiment of the present application.
- FIG. 6 is another schematic block diagram of a terminal provided by an embodiment of the present application.
- FIG. 7 is a schematic structural diagram of a data processing device according to an embodiment of the present application.
- FIG. 1 is a schematic flowchart of a data processing method according to Embodiment 1 of the present application.
- the method can be run on terminals such as smart phones (such as Android phones, IOS phones, etc.), tablets, laptops, and smart devices.
- the method mainly deals with the data in the database, which can reduce the workload of the database administrator and improve the efficiency of collecting data in the database.
- the steps of the method include S101 to S105.
- S101 Pre-define a configuration rule of data processing in a database, where the configuration rule includes one or more rules.
- the configuration rule is defined by the user, and the user formulates a configuration rule according to the needs of the development.
- the configuration rule includes a field value rule and a separator splicing rule, where the field value rule is selected and required to be set.
- the character replaces the selected character with other characters; the separator splicing rule is to select a character that needs to set a splicing rule, and splicing the selected character with a separator.
- step S101 includes steps S201 to S202.
- the user defines a rule by selecting a program source corresponding to the configuration rule, where the type of the field value rule includes a custom field default value, a field format adjustment, a field desensitization, and a specific character filtering. For example, for desensitizing a defined field, it is necessary to retain the first character and replace all other characters with "*”.
- the separator splicing rule is to splicing a row of data with a selected separator and forming a row-level string, for example, the value of the field A is 2016-10-27, the value of the field B is 69998.5, and the value of the field C is 0. , using the selected separator "/" for splicing, you can call the StringBuilder class splicing method, the A, B, C three field values are spliced to 2016-10-27/69998.5/0.
- the defined configuration rules are different according to different user selection program sources, and the program source may be defined by the user, or may be an existing method in which the user uses the open source program code. This is not limited.
- the program source corresponding to the rule 109 is StringUtils.rightPad(StringUtils.left(str,1), StringUtils.length(str), "*"
- the description file includes a file in an XML format, and the configuration rule is written into the description file, and the program source forming the configuration rule is written into the description file according to a logical execution order of the rule.
- the location of the different configuration rules in the description file is different, and the location of the configuration rule in the description file is defined by the user.
- the data collection by the user refers to connecting the database and collecting the data in the database, and the user can connect to the database through the JDBC method, obtain the data table in the database, and read the fields in the table and Field properties and so on, to get the data to be processed.
- the embodiment of the present application collects data through the JDBC connection mysql database.
- the code execution logic of the JDBC connection mysql database is as follows:
- the code execution logic for obtaining the information of the mysql database table through JDBC is as follows:
- m_DBMetaData m_Connection.getMetaData()
- ResultSet tableRet m_DBMetaData.getTables(null, "%”, m_TableName, new S tring[] ⁇ "TABLE” ⁇ )
- % means the meaning of *, that is, any meaning
- m_TableName is the name of the data table to be obtained, if you want to get the names of all the tables, you can use "%" as a parameter.
- all configuration rules in the description file are read, and the read configuration rule is parsed to obtain the type of the configuration rule, and the execution corresponding to the configuration rule is determined according to the type of the configured configuration rule.
- the configuration rule will process the data according to the rules specified in the program source.
- the configuration rule is determined by the user according to the development requirements.
- the user may select all the configuration rules in the description file to process the data, or select the The data is processed by one or more configuration rules in the description file. If the user selects one or more configuration rules in the description file to process the data, the user may find that the description file is not selected.
- the program source corresponding to the configuration rule, the unselected configuration rule is annotated by adding a comment code, the annotated configuration rule will not be executed or skipped, and the uncommented configuration rule will be based on the corresponding
- the program source execution logic executes and processes the data.
- step S105 includes steps S301 to S302.
- S301 Process the to-be-processed data according to execution logic of each rule and obtain processed data.
- the processed data is automatically saved to a preset file in a preset path, and the preset file may be a word document, which may be An excel document can also be other documents with a recording function, which is not limited herein.
- the embodiment of the present application pre-defines a configuration rule of data processing in a database, where the configuration rule includes one or more rules, and the configuration rule is written into the description file, and if the user detects the data in the database Collecting, receiving the data to be processed collected by the user, reading the description file and parsing the configuration rule in the description file, and processing the data to be processed according to the configuration rule, thereby reducing the work of the data administrator Volume and improve the efficiency of the data collection process.
- the embodiment of the present application further provides a terminal, where the terminal 100 includes: a defining unit 101, a writing unit 102, a receiving unit 103, a reading unit 104, and a first processing unit. 105.
- the definition unit 101 is configured to pre-define a configuration rule for data processing in a database, where the configuration rule includes one or more rules.
- the configuration rule is defined by the user, and the user formulates a configuration rule according to the needs of the development.
- the configuration rule includes a field value rule and a separator splicing rule, where the field value rule is selected and required to be set.
- the character replaces the selected character with other characters; the separator splicing rule is to select a character that needs to set a splicing rule, and splicing the selected character with a separator.
- the writing unit 102 is configured to write the configuration rule into the description file.
- the description file includes a file in an XML format, and the configuration rule is written into the description file, and the program source forming the configuration rule is written into the description file according to a logical execution order of the rule.
- the location of the different configuration rules in the description file is different, and the location of the configuration rule in the description file is defined by the user.
- the receiving unit 103 is configured to receive data to be processed collected by the user if the user collects data in the database.
- the data collection by the user refers to connecting the database and collecting the data in the database, and the user can connect to the database through the JDBC method, obtain the data table in the database, and read the fields in the table and Field properties and so on, to get the data to be processed.
- the embodiment of the present application collects data through the JDBC connection mysql database.
- the code execution logic of the JDBC connection mysql database is as follows:
- the code execution logic for obtaining the information of the mysql database table through JDBC is as follows:
- m_DBMetaData m_Connection.getMetaData()
- ResultSet tableRet m_DBMetaData.getTables(null, "%”, m_TableName, new S tring[] ⁇ "TABLE” ⁇ )
- % means the meaning of *, that is, any meaning
- m_TableName is the name of the data table to be obtained, if you want to get the names of all the tables, you can use "%" as a parameter.
- the reading unit 104 is configured to read the description file and parse the configuration rule in the description file.
- all configuration rules in the description file are read, and the read configuration rule is parsed to obtain the type of the configuration rule, and the execution corresponding to the configuration rule is determined according to the type of the configured configuration rule.
- Logic, and executing the rule according to the corresponding program source, the configuration rule will process the data according to the rules specified in the program source.
- the configuration rule is determined by the user according to the development requirements.
- the user may select all the configuration rules in the description file to process the data, or select the The data is processed by one or more configuration rules in the description file. If the user selects one or more configuration rules in the description file to process the data, the user may find that the description file is not selected.
- the program source corresponding to the configuration rule, the unselected configuration rule is annotated by adding a comment code, the annotated configuration rule will not be executed or skipped, and the uncommented configuration rule will be based on the corresponding
- the program source execution logic executes and processes the data.
- the first processing unit 105 is configured to process the to-be-processed data according to the configuration rule.
- the embodiment of the present application pre-defines a configuration rule of data processing in a database, where the configuration rule includes one or more rules, and the configuration rule is written into the description file, and if the user detects the data in the database Collecting, receiving the data to be processed collected by the user, reading the description file and parsing the configuration rule in the description file, and processing the data to be processed according to the configuration rule, thereby reducing the work of the data administrator Volume and improve the efficiency of the data collection process.
- the defining unit 101 includes:
- the selecting unit 1011 is configured to select a program source for performing rule definition.
- the user defines a rule by selecting a program source corresponding to the configuration rule, where the type of the field value rule includes a custom field default value, a field format adjustment, a field desensitization, and a specific character filtering. For example, for desensitizing a defined field, it is necessary to retain the first character and replace all other characters with "*”.
- the separator splicing rule is to splicing a row of data with a selected separator and forming a row-level string, for example, the value of the field A is 2016-10-27, the value of the field B is 69998.5, and the value of the field C is 0. , using the selected separator "/" for splicing, you can call the StringBuilder class splicing method, the A, B, C three field values are spliced to 2016-10-27/69998.5/0.
- the creating unit 1012 is configured to create the configuration rule according to the program source.
- the defined configuration rules are different according to different user selection program sources, and the program source may be defined by the user, or may be an existing method in which the user uses the open source program code. This is not limited.
- the program source definition field value rule proRule "109”
- the program source corresponding to the rule 109 is StringUtils.rightPad (StringUtils.left (str, 1), StringUtils. length (str), "*”);
- the first processing unit 105 includes:
- the second processing unit 1051 is configured to process the processed data according to execution logic of each rule and obtain processed data.
- execution logic of each rule for example, for a field with a field value of “yyyy-mm-dd”, it is necessary to retain the first character and replace all other characters with “*”.
- the code logic of the program source at the time of execution is as follows :StringUtils.rightPad(StringUtils.left(str,1), StringUtils.length(str), "*”), the field value after data processing is output as "y***-**-**".
- the value of the field A is 2016-10-27
- the value of the field B is 69998.5
- the value of the field C is 0.
- the saving unit 1052 is configured to save the processed data into a preset document.
- the processed data is automatically saved to a preset file in a preset path, and the preset file may be a word document, which may be An excel document can also be other documents with a recording function, which is not limited herein.
- the embodiment of the present application defines a configuration rule for data processing, where the configuration rule includes one or more rules, and the configuration rule is written into the description file, and if the user collects data, the receiving is received.
- the above terminal can be implemented in the form of a computer program that can be run on a data processing device as shown in FIG.
- FIG. 7 is a schematic structural diagram of a data processing device according to the present application.
- the device 300 can include an input device 301, an output device 302, a transceiver 303, a memory 304, and a processor 305, where:
- the input device 301 is configured to receive input data of an external access control device.
- the input device 301 in the embodiment of the present application may include a keyboard, a mouse, a photoelectric input device, a sound input device, a touch input device, a scanner, and the like.
- the output device 302 is configured to output output data of the access control device to the outside.
- the output device 302 described in this embodiment of the present application may include a display, a speaker, a printer, and the like.
- the transceiver device 303 is configured to send data to or receive data from other devices through a communication link.
- the transceiver 303 of the embodiment of the present application may include a transceiver device such as a radio frequency antenna.
- the memory 304 may include a non-volatile storage medium and an internal memory.
- the non-volatile storage medium can store an operating system and store program instructions for implementing a data processing method.
- the internal memory provides an environment for the operation of program instructions in a non-volatile storage medium.
- the processor 305 is used to provide computing and control capabilities to support the operation of the entire data processing device. When the above program instructions are executed by the processor 305, the processor 305 can be caused to execute the above data processing method.
- the processor 305 is configured to run a program that implements the data processing method stored in the memory 304 to perform the following operations:
- Defining a configuration rule for data processing in a database the configuration rule including one or more rules
- the data to be processed collected by the user is received
- the to-be-processed data is processed according to the configuration rule.
- the configuration rule of the predefined data processing specifically includes:
- the configuration rule includes a field value rule and a separator splicing rule, wherein the field value rule is to select a character to be set, and replace the selected character with another character; the separator splicing The rule is to select the characters that need to set the splicing rules, and splicing the selected characters with separators.
- the processing the data to be processed according to the configuration rule specifically includes:
- the description file includes a file in XML format.
- the embodiment of the data processing device shown in FIG. 7 does not constitute a limitation on the specific configuration of the data processing device.
- the data processing device may include more or less than the illustration. Parts, or combine some parts, or different parts.
- the data processing device may include only the memory and the processor. In such an embodiment, the structure and function of the memory and the processor are the same as those of the embodiment shown in FIG. 7, and details are not described herein.
- the application provides a computer readable storage medium storing one or more computer programs, the one or more computer programs being executable by one or more processors to implement the data processing described above method.
- the foregoing storage medium of the present application includes: a magnetic disk, an optical disk, a read-only memory (ROM), and the like, which can store various program codes.
- the units in all the embodiments of the present application may be implemented by a general-purpose integrated circuit, such as a CPU (Central Processing Unit), or by an ASIC (Application Specific Integrated Circuit).
- a general-purpose integrated circuit such as a CPU (Central Processing Unit), or by an ASIC (Application Specific Integrated Circuit).
- the steps in the data processing method of the embodiment of the present application may be sequentially adjusted, merged, and deleted according to actual needs.
- the units in the terminal in the embodiment of the present application may be combined, divided, and deleted according to actual needs.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Databases & Information Systems (AREA)
- Data Mining & Analysis (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Stored Programmes (AREA)
Abstract
Description
本申请要求于2017年4月10日提交中国专利局、申请号为CN 201710228803.6、申请名称为“一种数据处理方法及终端”的中国专利申请的优先权,其全部内容通过引用结合在本申请中。This application claims the priority of the Chinese Patent Application, filed on Apr. 10, 2017, the number of which is hereby incorporated by reference. in.
本申请涉及计算机技术领域,尤其涉及一种数据处理方法、终端、设备以及存储介质。The present application relates to the field of computer technologies, and in particular, to a data processing method, a terminal, a device, and a storage medium.
目前,信息化技术正在以前所未有的速度向前迈进,各种各样的数据信息急剧膨胀。随着计算机网络技术的不断发展,人与人之间的信息共享和数据交换也在不断扩大着,使得计算机网络的数据量日渐庞大。At present, information technology is moving forward at an unprecedented speed, and various kinds of data information are expanding rapidly. With the continuous development of computer network technology, information sharing and data exchange between people are also expanding, making the data volume of computer networks increasingly large.
对于数据库中数据的处理,现有的处理方式中存在着以下缺点:若数据管理员每次对数据进行处理都需要编写数据处理程序代码,这种数据的处理方式在一定程序上增加了数据管理员的工作量,并且影响了数据采集环节中的效率。For the processing of data in the database, the existing processing methods have the following disadvantages: if the data administrator needs to write the data processing program code every time the data is processed, the processing method of the data increases the data management in a certain program. The workload of the staff and affects the efficiency of the data collection process.
发明内容Summary of the invention
本申请实施例提供一种数据处理方法、终端、设备以及存储介质,可以减轻数据管理员的工作量、并提高了数据采集环节的效率。The embodiment of the present application provides a data processing method, a terminal, a device, and a storage medium, which can reduce the workload of the data administrator and improve the efficiency of the data collection link.
一方面,本申请实施例提供了一种数据处理方法,该方法包括:In one aspect, the embodiment of the present application provides a data processing method, where the method includes:
预先定义数据库中数据处理的配置规则,所述配置规则包括一个或者多个规则;Defining a configuration rule for data processing in a database, the configuration rule including one or more rules;
将所述配置规则写入至描述文件中;Writing the configuration rule into the description file;
若检测到用户对数据库中数据的采集,接收用户所采集的待处理数据;If the user collects data in the database, the data to be processed collected by the user is received;
读取所述描述文件并解析所述描述文件中的配置规则;Reading the description file and parsing configuration rules in the description file;
根据所述配置规则对所述待处理数据进行处理。The to-be-processed data is processed according to the configuration rule.
另一方面,本申请实施例提供了一种终端,该终端包括:On the other hand, an embodiment of the present application provides a terminal, where the terminal includes:
定义单元,用于预先定义数据库中数据处理的配置规则,所述配置规则包括一个或者多个规则;a definition unit, configured to pre-define a configuration rule of data processing in a database, where the configuration rule includes one or more rules;
写入单元,用于将所述配置规则写入至描述文件中;a writing unit, configured to write the configuration rule into a description file;
接收单元,用于若检测到用户对数据库中数据的采集,接收用户所采集的待处理的数据;a receiving unit, configured to: if the user collects data collected in the database, receive data to be processed collected by the user;
读取单元,用于读取所述描述文件并解析所述描述文件中的配置规则;a reading unit, configured to read the description file and parse a configuration rule in the description file;
第一处理单元,用于根据所述配置规则对所述待处理数据进行处理。The first processing unit is configured to process the to-be-processed data according to the configuration rule.
进一步地,所述定义单元,包括:Further, the defining unit includes:
选择单元,用于选择进行规则定义的程序源;a selection unit for selecting a program source for which the rule definition is performed;
创建单元,用于根据所述程序源创建所述配置规则。a creating unit, configured to create the configuration rule according to the program source.
又一方面,本申请实施例还提供了一种数据处理设备,其包括:存储器,用于存储实现数据处理方法的程序;以及处理器,用于运行所述存储器中存储的实现数据处理方法的程序,以执行如下操作:预先定义数据库中数据处理的配置规则,所述配置规则包括一个或者多个规则;将所述配置规则写入至描述文件中;若检测到用户对数据库中数据的采集,接收用户所采集的待处理数据;读取所述描述文件并解析所述描述文件中的配置规则;根据所述配置规则对所述待处理数据进行处理。In another aspect, the embodiment of the present application further provides a data processing device, including: a memory, configured to store a program that implements a data processing method; and a processor, configured to run the data processing method stored in the memory a program to perform the following operations: pre-defining a configuration rule of data processing in the database, the configuration rule includes one or more rules; writing the configuration rule to the description file; and detecting the user collecting data in the database Receiving the to-be-processed data collected by the user; reading the description file and parsing the configuration rule in the description file; and processing the to-be-processed data according to the configuration rule.
再一方面,本申请实施例还提供了一种计算机可读存储介质,所述计算机可读存储介质存储有一个或者一个以上计算机程序,所述一个或者一个以上计算机程序可被一个或者一个以上的处理器执行,以实现如下步骤:预先定义数据库中数据处理的配置规则,所述配置规则包括一个或者多个规则;将所述配置规则写入至描述文件中;若检测到用户对数据库中数据的采集,接收用户所采集的待处理数据;读取所述描述文件并解析所述描述文件中的配置规则;根据所述配置规则对所述待处理数据进行处理。In still another aspect, the embodiment of the present application further provides a computer readable storage medium, where the one or more computer programs are stored, and the one or more computer programs may be one or more The processor executes to implement the following steps: pre-defining a configuration rule of data processing in the database, the configuration rule includes one or more rules; writing the configuration rule to the description file; if the user detects the data in the database Receiving, receiving data to be processed collected by the user; reading the description file and parsing a configuration rule in the description file; and processing the to-be-processed data according to the configuration rule.
本申请实施例可以减轻数据管理员的工作量、并提高了数据采集环节的效率。The embodiment of the present application can reduce the workload of the data administrator and improve the efficiency of the data collection link.
为了更清楚地说明本申请实施例技术方案,下面将对实施例描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图是本申请的一些实 施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings used in the description of the embodiments will be briefly described below. Obviously, the drawings in the following description are some embodiments of the present application, For the ordinary technicians, other drawings can be obtained based on these drawings without any creative work.
图1是本申请实施例提供的一种数据处理方法的示意流程图。FIG. 1 is a schematic flowchart of a data processing method according to an embodiment of the present application.
图2是本申请实施例提供的一种数据处理方法的另一示意流程图。FIG. 2 is another schematic flowchart of a data processing method according to an embodiment of the present application.
图3是本申请实施例提供的一种数据处理方法的另一示意流程图。FIG. 3 is another schematic flowchart of a data processing method according to an embodiment of the present application.
图4是本申请实施例提供的一种终端的示意性框图。FIG. 4 is a schematic block diagram of a terminal provided by an embodiment of the present application.
图5是本申请实施例提供的一种终端的另一示意性框图。FIG. 5 is another schematic block diagram of a terminal provided by an embodiment of the present application.
图6是本申请实施例提供的一种终端的另一示意性框图。FIG. 6 is another schematic block diagram of a terminal provided by an embodiment of the present application.
图7是本申请实施例提供的一种数据处理设备的结构组成示意图。FIG. 7 is a schematic structural diagram of a data processing device according to an embodiment of the present application.
下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例,都属于本申请保护的范围。The technical solutions in the embodiments of the present application are clearly and completely described in the following with reference to the drawings in the embodiments of the present application. It is obvious that the described embodiments are a part of the embodiments of the present application, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present application without departing from the inventive scope are the scope of the present application.
应当理解,当在本说明书和所附权利要求书中使用时,术语“包括”和“包含”指示所描述特征、整体、步骤、操作、元素和/或组件的存在,但并不排除一个或多个其它特征、整体、步骤、操作、元素、组件和/或其集合的存在或添加。The use of the terms "comprising", "comprising", "","," The presence or addition of a plurality of other features, integers, steps, operations, elements, components, and/or collections thereof.
还应当理解,在此本申请说明书中所使用的术语仅仅是出于描述特定实施例的目的而并不意在限制本申请。如在本申请说明书和所附权利要求书中所使用的那样,除非上下文清楚地指明其它情况,否则单数形式的“一”、“一个”及“该”意在包括复数形式。The terminology used herein is for the purpose of describing particular embodiments and is not intended to be limiting. As used in the specification and the appended claims, the claims
请参阅图1,图1是本申请实施例一提供的一种数据处理方法的示意流程图。该方法可以运行在智能手机(如Android手机、IOS手机等)、平板电脑、笔记本电脑以及智能设备等终端中。该方法主要是对数据库中的数据进行处理,可以减轻数据库管理员的工作量、并提高针对数据库中的数据进行采集的效率。如图1所示,该方法的步骤包括S101~S105。Please refer to FIG. 1. FIG. 1 is a schematic flowchart of a data processing method according to Embodiment 1 of the present application. The method can be run on terminals such as smart phones (such as Android phones, IOS phones, etc.), tablets, laptops, and smart devices. The method mainly deals with the data in the database, which can reduce the workload of the database administrator and improve the efficiency of collecting data in the database. As shown in FIG. 1, the steps of the method include S101 to S105.
S101,预先定义数据库中数据处理的配置规则,所述配置规则包括一个或者多个规则。S101. Pre-define a configuration rule of data processing in a database, where the configuration rule includes one or more rules.
在本申请实施例中,所述配置规则由用户自行定义,用户根据开发的需要制定配置规则,所述配置规则包括字段值规则和分隔符拼接规则,其中,所述字段值规则是选择需要设置的字符,将所选择的字符用其它字符进行替换;所述分隔符拼接规则是选择需要设置拼接规则的字符,将所选择的字符用分隔符进行拼接。In the embodiment of the present application, the configuration rule is defined by the user, and the user formulates a configuration rule according to the needs of the development. The configuration rule includes a field value rule and a separator splicing rule, where the field value rule is selected and required to be set. The character replaces the selected character with other characters; the separator splicing rule is to select a character that needs to set a splicing rule, and splicing the selected character with a separator.
进一步地,如图2所示,步骤S101包括步骤S201~S202。Further, as shown in FIG. 2, step S101 includes steps S201 to S202.
S201,选择进行规则定义的程序源。S201, selecting a program source for performing rule definition.
在本申请实施例中,用户通过选择与配置规则相对应的程序源来定义规则,其中,所述字段值规则的类型包括自定义字段默认值、字段格式调整、字段脱敏和特定字符过滤,例如,对于定义字段脱敏,需要保留首字符,并将其它字符全部替换为“*”,用户可以定义proRule=″109″,在执行时,获取规则109,该规则在执行时,首先读取字段值,并找到相对应的方法处理字段中的字符,与109相对应的程序源的代码逻辑如下:StringUtils.rightPad(StringUtils.left(str,1),StringUtils.length(str),″*″)。所述分隔符拼接规则是将一行数据以选定的分隔符进行拼接,并形成行级字符串,例如,将字段A的值2016-10-27,字段B的值69998.5,字段C的值0,使用选定分隔符“/”进行拼接,可以调用StringBuilder类的拼接方法,将A、B、C三个字段值拼接后为2016-10-27/69998.5/0。In the embodiment of the present application, the user defines a rule by selecting a program source corresponding to the configuration rule, where the type of the field value rule includes a custom field default value, a field format adjustment, a field desensitization, and a specific character filtering. For example, for desensitizing a defined field, it is necessary to retain the first character and replace all other characters with "*". The user can define proRule="109". When executed, the rule 109 is obtained. When executed, the rule is read first. The field value, and find the corresponding method to handle the characters in the field, the code logic of the program source corresponding to 109 is as follows: StringUtils.rightPad(StringUtils.left(str,1),StringUtils.length(str),"*" ). The separator splicing rule is to splicing a row of data with a selected separator and forming a row-level string, for example, the value of the field A is 2016-10-27, the value of the field B is 69998.5, and the value of the field C is 0. , using the selected separator "/" for splicing, you can call the StringBuilder class splicing method, the A, B, C three field values are spliced to 2016-10-27/69998.5/0.
S202,根据所述程序源创建所述配置规则。S202. Create the configuration rule according to the program source.
在本申请实施例中,根据用户选择程序源的不同,所定义的配置规则便不相同,所述程序源可以是用户自行定义的,也可以是用户使用开源程序代码中的现有方法,在此不做限定。例如,使用程序源定义字段值规则proRule=″109″,与规则109相对应的程序源为StringUtils.rightPad(StringUtils.left(str,1),StringUtils.length(str),″*″);使用程序源定义分隔符拼接规则,与所述分隔符拼接规则相对应的程序源为调用StringBuilder类中的方法splitSign=″/″等等。In the embodiment of the present application, the defined configuration rules are different according to different user selection program sources, and the program source may be defined by the user, or may be an existing method in which the user uses the open source program code. This is not limited. For example, using the program source to define a field value rule proRule="109", the program source corresponding to the rule 109 is StringUtils.rightPad(StringUtils.left(str,1), StringUtils.length(str), "*"); The program source defines a separator splicing rule, and the program source corresponding to the separator splicing rule is to call the method in the StringBuilder class splitSign="/" and so on.
S102,将所述配置规则写入至描述文件中。S102. Write the configuration rule into the description file.
在本申请实施例中,所述描述文件包括XML格式的文件,将所述配置规则写入至描述文件中,根据规则的逻辑执行顺序将形成所述配置规则的程序源写入至描述文件中,不同的配置规则在所述描述文件中的位置不相同,所述配置规则在所述描述文件中的位置由用户自行定义。例如,不同的配置规则在所述描述文件中的位置可以为:<templets encoding=″utf-8″splitSign=″/″ proRule=″108″>。In the embodiment of the present application, the description file includes a file in an XML format, and the configuration rule is written into the description file, and the program source forming the configuration rule is written into the description file according to a logical execution order of the rule. The location of the different configuration rules in the description file is different, and the location of the configuration rule in the description file is defined by the user. For example, the location of the different configuration rules in the description file may be: <templets encoding = "utf-8" splitSign = "/" proRule = "108">.
S103,若检测到用户对数据库中数据的采集,接收用户所采集的待处理数据。S103. If the user collects data collected in the database, the data to be processed collected by the user is received.
在本申请实施例中,用户进行数据的采集指的是连接数据库并对数据库中的数据进行采集,用户可以通过JDBC方式连接数据库,并获取数据库中的数据表,以及读取表中的字段和字段属性等等,从而得到待处理数据。In the embodiment of the present application, the data collection by the user refers to connecting the database and collecting the data in the database, and the user can connect to the database through the JDBC method, obtain the data table in the database, and read the fields in the table and Field properties and so on, to get the data to be processed.
需要说明的是,本申请实施例通过JDBC连接mysql数据库采集数据,例如,JDBC连接mysql数据库的代码执行逻辑如下:It should be noted that the embodiment of the present application collects data through the JDBC connection mysql database. For example, the code execution logic of the JDBC connection mysql database is as follows:
class.forName(″com.mysql.jdbc.Driver″).newInstance();class.forName("com.mysql.jdbc.Driver").newInstance();
Connection conn=DriverManager.getConnection(″jdbc:mysql://localhost/test?user=root&password=123456″)Connection conn=DriverManager.getConnection("jdbc:mysql://localhost/test?user=root&password=123456")
通过JDBC获取mysql数据库表的信息的代码执行逻辑如下:The code execution logic for obtaining the information of the mysql database table through JDBC is as follows:
m_DBMetaData=m_Connection.getMetaData();m_DBMetaData=m_Connection.getMetaData();
ResultSet tableRet=m_DBMetaData.getTables(null,″%″,m_TableName,new S tring[]{″TABLE″})ResultSet tableRet=m_DBMetaData.getTables(null, "%", m_TableName, new S tring[]{"TABLE"})
其中,″%″就是表示*的意思,也就是任意所有的意思;m_TableName就是要获取的数据表的名字,如果想获取所有的表的名字,就可以使用″%″作为参数了。Among them, "%" means the meaning of *, that is, any meaning; m_TableName is the name of the data table to be obtained, if you want to get the names of all the tables, you can use "%" as a parameter.
S104,读取所述描述文件并解析所述描述文件中的配置规则。S104. Read the description file and parse the configuration rule in the description file.
在本申请实施例中,读取所述描述文件中所有的配置规则,并解析所读取的配置规则得到所述配置规则的类型,根据所解析的配置规则的类型确定配置规则所对应的执行逻辑,并根据相对应的程序源执行该规则,所述配置规则便会按照程序源中所规定的规则对数据进行处理。In the embodiment of the present application, all configuration rules in the description file are read, and the read configuration rule is parsed to obtain the type of the configuration rule, and the execution corresponding to the configuration rule is determined according to the type of the configured configuration rule. Logic, and executing the rule according to the corresponding program source, the configuration rule will process the data according to the rules specified in the program source.
需要说明的是,由于所述配置规则由用户根据开发的需要进行制定,在执行所述配置规则时,用户可以选择所述描述文件中所有的配置规则对数据进行处理,也可以根据需要选择所述描述文件中的一项或者多项配置规则对数据进行处理,若用户选择所述描述文件中的一项或者多项配置规则对数据进行处理,用户可以找到所述描述文件中未被选择的配置规则所对应的程序源,通过增加注释代码的方式将未被选择的配置规则进行注释,被注释的配置规则将不会执行或者被跳过执行,未被注释的配置规则将根据相对应的程序源执行逻辑进行 执行,并对数据进行处理。It should be noted that, the configuration rule is determined by the user according to the development requirements. When the configuration rule is executed, the user may select all the configuration rules in the description file to process the data, or select the The data is processed by one or more configuration rules in the description file. If the user selects one or more configuration rules in the description file to process the data, the user may find that the description file is not selected. The program source corresponding to the configuration rule, the unselected configuration rule is annotated by adding a comment code, the annotated configuration rule will not be executed or skipped, and the uncommented configuration rule will be based on the corresponding The program source execution logic executes and processes the data.
S105,根据所述配置规则对所述待处理数据进行处理。S105. Process the to-be-processed data according to the configuration rule.
具体地,如图3所示,步骤S105包括步骤S301~S302。Specifically, as shown in FIG. 3, step S105 includes steps S301 to S302.
S301,按照各个规则的执行逻辑对所述待处理数据进行处理并得到已处理数据。S301. Process the to-be-processed data according to execution logic of each rule and obtain processed data.
在本申请实施例中,例如对于字段值为“yyyy-mm-dd”的字段脱敏,需要保留首字符,并将其它字符全部替换为“*”,在执行时的程序源的代码逻辑如下:StringUtils.rightPad(StringUtils.left(str,1),StringUtils.length(str),″*″),经过数据处理后的字段值输出为“y***-**-**”。再例如,使用分隔符拼接规则将一行数据以选定的分隔符进行拼接,并形成行级字符串,将字段A的值2016-10-27,字段B的值69998.5,字段C的值0,使用选定分隔符“/”进行拼接,可以调用StringBuilder类的拼接方法,将A、B、C三个字段值拼接后为2016-10-27/69998.5/0。In the embodiment of the present application, for example, for a field with a field value of “yyyy-mm-dd”, it is necessary to retain the first character and replace all other characters with “*”. The code logic of the program source at the time of execution is as follows :StringUtils.rightPad(StringUtils.left(str,1), StringUtils.length(str), "*"), the field value after data processing is output as "y***-**-**". For another example, using a separator stitching rule to splicing a row of data with a selected separator, and forming a row-level string, the value of the field A is 2016-10-27, the value of the field B is 69998.5, and the value of the field C is 0. Using the selected separator "/" for splicing, you can call the splicing method of the StringBuilder class, and splicing the three field values A, B, and C to 2016-10-27/69998.5/0.
S302,将所述已处理数据保存至预设的文档中。S302. Save the processed data to a preset document.
在本申请实施例中,当对待处理数据进行处理并得出已处理数据后,自动保存该已处理数据至预设路径下的预设文件中,该预设文件可以为一word文档,可以为一excel文档,也可以为具有记录功能的其它文档,在此不做限定。In the embodiment of the present application, after the data to be processed is processed and the processed data is obtained, the processed data is automatically saved to a preset file in a preset path, and the preset file may be a word document, which may be An excel document can also be other documents with a recording function, which is not limited herein.
由以上可见,本申请实施例通过预先定义数据库中数据处理的配置规则,所述配置规则包括一个或者多个规则,将所述配置规则写入至描述文件中,若检测到用户对数据库中数据的采集,接收用户所采集的待处理数据,读取所述描述文件并解析所述描述文件中的配置规则,根据所述配置规则对所述待处理数据进行处理,可以减轻数据管理员的工作量、并提高了数据采集环节的效率。It can be seen from the above that the embodiment of the present application pre-defines a configuration rule of data processing in a database, where the configuration rule includes one or more rules, and the configuration rule is written into the description file, and if the user detects the data in the database Collecting, receiving the data to be processed collected by the user, reading the description file and parsing the configuration rule in the description file, and processing the data to be processed according to the configuration rule, thereby reducing the work of the data administrator Volume and improve the efficiency of the data collection process.
请参阅图4,对应上述一种数据处理方法,本申请实施例还提出一种终端,该终端100包括:定义单元101、写入单元102、接收单元103、读取单元104、第一处理单元105。Referring to FIG. 4 , corresponding to the foregoing data processing method, the embodiment of the present application further provides a terminal, where the terminal 100 includes: a defining
其中,所述定义单元101,用于预先定义数据库中数据处理的配置规则,所述配置规则包括一个或者多个规则。在本申请实施例中,所述配置规则由用户自行定义,用户根据开发的需要制定配置规则,所述配置规则包括字段值规则和分隔符拼接规则,其中,所述字段值规则是选择需要设置的字符,将所选择的字符用其它字符进行替换;所述分隔符拼接规则是选择需要设置拼接规则的 字符,将所选择的字符用分隔符进行拼接。The
写入单元102,用于将所述配置规则写入至描述文件中。在本申请实施例中,所述描述文件包括XML格式的文件,将所述配置规则写入至描述文件中,根据规则的逻辑执行顺序将形成所述配置规则的程序源写入至描述文件中,不同的配置规则在所述描述文件中的位置不相同,所述配置规则在所述描述文件中的位置由用户自行定义。例如,不同的配置规则在所述描述文件中的位置可以为:<templets encoding=″utf-8″splitSign=″/″proRule=″108″>。The
接收单元103,用于若检测到用户对数据库中数据的采集,接收用户所采集的待处理的数据。在本申请实施例中,用户进行数据的采集指的是连接数据库并对数据库中的数据进行采集,用户可以通过JDBC方式连接数据库,并获取数据库中的数据表,以及读取表中的字段和字段属性等等,从而得到待处理数据。The receiving
需要说明的是,本申请实施例通过JDBC连接mysql数据库采集数据,例如,JDBC连接mysql数据库的代码执行逻辑如下:It should be noted that the embodiment of the present application collects data through the JDBC connection mysql database. For example, the code execution logic of the JDBC connection mysql database is as follows:
class.forName(″com.mysql.jdbc.Driver″).newInstance();class.forName("com.mysql.jdbc.Driver").newInstance();
Connection conn=DriverManager.getConnection(″jdbc:mysql://localhost/test?user=root&password=123456″)Connection conn=DriverManager.getConnection("jdbc:mysql://localhost/test?user=root&password=123456")
通过JDBC获取mysql数据库表的信息的代码执行逻辑如下:The code execution logic for obtaining the information of the mysql database table through JDBC is as follows:
m_DBMetaData=m_Connection.getMetaData();m_DBMetaData=m_Connection.getMetaData();
ResultSet tableRet=m_DBMetaData.getTables(null,″%″,m_TableName,new S tring[]{″TABLE″})ResultSet tableRet=m_DBMetaData.getTables(null, "%", m_TableName, new S tring[]{"TABLE"})
其中,″%″就是表示*的意思,也就是任意所有的意思;m_TableName就是要获取的数据表的名字,如果想获取所有的表的名字,就可以使用″%″来作为参数了。Among them, "%" means the meaning of *, that is, any meaning; m_TableName is the name of the data table to be obtained, if you want to get the names of all the tables, you can use "%" as a parameter.
读取单元104,用于读取所述描述文件并解析所述描述文件中的配置规则。在本申请实施例中,读取所述描述文件中所有的配置规则,并解析所读取的配置规则得到所述配置规则的类型,根据所解析的配置规则的类型确定配置规则所对应的执行逻辑,并根据相对应的程序源执行该规则,所述配置规则便会按照程序源中所规定的规则对数据进行处理。The
需要说明的是,由于所述配置规则由用户根据开发的需要进行制定,在执 行所述配置规则时,用户可以选择所述描述文件中所有的配置规则对数据进行处理,也可以根据需要选择所述描述文件中的一项或者多项配置规则对数据进行处理,若用户选择所述描述文件中的一项或者多项配置规则对数据进行处理,用户可以找到所述描述文件中未被选择的配置规则所对应的程序源,通过增加注释代码的方式将未被选择的配置规则进行注释,被注释的配置规则将不会执行或者被跳过执行,未被注释的配置规则将根据相对应的程序源执行逻辑进行执行,并对数据进行处理。It should be noted that, the configuration rule is determined by the user according to the development requirements. When the configuration rule is executed, the user may select all the configuration rules in the description file to process the data, or select the The data is processed by one or more configuration rules in the description file. If the user selects one or more configuration rules in the description file to process the data, the user may find that the description file is not selected. The program source corresponding to the configuration rule, the unselected configuration rule is annotated by adding a comment code, the annotated configuration rule will not be executed or skipped, and the uncommented configuration rule will be based on the corresponding The program source execution logic executes and processes the data.
第一处理单元105,用于根据所述配置规则对所述待处理数据进行处理。The
由以上可见,本申请实施例通过预先定义数据库中数据处理的配置规则,所述配置规则包括一个或者多个规则,将所述配置规则写入至描述文件中,若检测到用户对数据库中数据的采集,接收用户所采集的待处理数据,读取所述描述文件并解析所述描述文件中的配置规则,根据所述配置规则对所述待处理数据进行处理,可以减轻数据管理员的工作量、并提高了数据采集环节的效率。It can be seen from the above that the embodiment of the present application pre-defines a configuration rule of data processing in a database, where the configuration rule includes one or more rules, and the configuration rule is written into the description file, and if the user detects the data in the database Collecting, receiving the data to be processed collected by the user, reading the description file and parsing the configuration rule in the description file, and processing the data to be processed according to the configuration rule, thereby reducing the work of the data administrator Volume and improve the efficiency of the data collection process.
如图5所示,所述定义单元101,包括:As shown in FIG. 5, the defining
选择单元1011,用于选择进行规则定义的程序源。在本申请实施例中,用户通过选择与配置规则相对应的程序源来定义规则,其中,所述字段值规则的类型包括自定义字段默认值、字段格式调整、字段脱敏和特定字符过滤,例如,对于定义字段脱敏,需要保留首字符,并将其它字符全部替换为“*”,用户可以定义proRule=″109″,在执行时,获取规则109,该规则在执行时,首先读取字段值,并找到相对应的方法处理字段中的字符,与109相对应的程序源的代码逻辑如下:StringUtils.rightPad(StringUtils.left(str,1),StringUtils.length(str),″*″)。所述分隔符拼接规则是将一行数据以选定的分隔符进行拼接,并形成行级字符串,例如,将字段A的值2016-10-27,字段B的值69998.5,字段C的值0,使用选定分隔符“/”进行拼接,可以调用StringBuilder类的拼接方法,将A、B、C三个字段值拼接后为2016-10-27/69998.5/0。The selecting
创建单元1012,用于根据所述程序源创建所述配置规则。在本申请实施例中,根据用户选择程序源的不同,所定义的配置规则便不相同,所述程序源可以是用户自行定义的,也可以是用户使用开源程序代码中的现有方法,在此不做限定。例如,使用程序源定义字段值规则proRule=″109″,与规则109相对应的程序源为StringUtils.rightPad(StringUtils.left(str,1),StringUtils.length(str),″*″); 使用程序源定义分隔符拼接规则,与所述分隔符拼接规则相对应的程序源为调用StringBuilder类中的方法splitSign=″/″等等。The creating unit 1012 is configured to create the configuration rule according to the program source. In the embodiment of the present application, the defined configuration rules are different according to different user selection program sources, and the program source may be defined by the user, or may be an existing method in which the user uses the open source program code. This is not limited. For example, using the program source definition field value rule proRule = "109", the program source corresponding to the rule 109 is StringUtils.rightPad (StringUtils.left (str, 1), StringUtils. length (str), "*"); The program source defines a separator splicing rule, and the program source corresponding to the separator splicing rule is to call the method in the StringBuilder class splitSign="/" and so on.
如图6所示,所述第一处理单元105,包括:As shown in FIG. 6, the
第二处理单元1051,用于按照各个规则的执行逻辑对所述处理数据进行处理并得到已处理数据。在本申请实施例中,例如对于字段值为“yyyy-mm-dd”的字段脱敏,需要保留首字符,并将其它字符全部替换为“*”,在执行时的程序源的代码逻辑如下:StringUtils.rightPad(StringUtils.left(str,1),StringUtils.length(str),″*″),经过数据处理后的字段值输出为“y***-**-**”。再例如,使用分隔符拼接规则将一行数据以选定的分隔符进行拼接,并形成行级字符串,将字段A的值2016-10-27,字段B的值69998.5,字段C的值0,使用选定分隔符“/”进行拼接,可以调用StringBuilder类的拼接方法,将A、B、C三个字段值拼接后为2016-10-27/69998.5/0。The
保存单元1052,用于将已处理数据保存至预设的文档中。在本申请实施例中,当对待处理数据进行处理并得出已处理数据后,自动保存该已处理数据至预设路径下的预设文件中,该预设文件可以为一word文档,可以为一excel文档,也可以为具有记录功能的其它文档,在此不做限定。The saving unit 1052 is configured to save the processed data into a preset document. In the embodiment of the present application, after the data to be processed is processed and the processed data is obtained, the processed data is automatically saved to a preset file in a preset path, and the preset file may be a word document, which may be An excel document can also be other documents with a recording function, which is not limited herein.
由以上可见,本申请实施例通过预先定义数据处理的配置规则,所述配置规则包括一个或者多个规则,将所述配置规则写入至描述文件中,若检测到用户进行数据的采集,接收用户所采集的待处理数据,读取所述描述文件并解析所述描述文件中的配置规则,根据所述配置规则对所述待处理数据进行处理,可以减轻数据管理员的工作量、并提高了数据采集环节的效率。It can be seen from the above that the embodiment of the present application defines a configuration rule for data processing, where the configuration rule includes one or more rules, and the configuration rule is written into the description file, and if the user collects data, the receiving is received. The data to be processed collected by the user, the description file is read and the configuration rule in the description file is parsed, and the to-be-processed data is processed according to the configuration rule, thereby reducing the workload of the data administrator and improving The efficiency of the data collection process.
上述终端可以实现为一种计算机程序的形式,计算机程序可以在如图7所示的数据处理设备上运行。The above terminal can be implemented in the form of a computer program that can be run on a data processing device as shown in FIG.
图7为本申请一种数据处理设备的结构组成示意图。如图7所示,该设备300可包括:输入装置301、输出装置302、收发装置303、存储器304以及处理器305,其中:FIG. 7 is a schematic structural diagram of a data processing device according to the present application. As shown in FIG. 7, the
所述输入装置301,用于接收外部访问控制设备的输入数据。具体实现中,本申请实施例所述的输入装置301可包括键盘、鼠标、光电输入装置、声音输入装置、触摸式输入装置、扫描仪等。The
所述输出装置302,用于对外输出访问控制设备的输出数据。具体实现中, 本申请实施例所述的输出装置302可包括显示器、扬声器、打印机等。The
所述收发装置303,用于通过通信链路向其他设备发送数据或者从其他设备接收数据。具体实现中,本申请实施例的收发装置303可包括射频天线等收发器件。The
所述存储器304,所述存储器304,可以包括非易失性存储介质和内存储器。非易失性存储介质可存储操作系统和存储用于实现数据处理方法的程序指令。内存储器为非易失性存储介质中的程序指令的运行提供环境。处理器305用于提供计算和控制能力,支撑整个数据处理设备的运行。上述程序指令被处理器305执行时,可使得处理器305执行上述数据处理方法。The
具体地,所述处理器305,用于运行所述存储器304中存储的实现数据处理方法的程序,以执行如下操作:Specifically, the
预先定义数据库中数据处理的配置规则,所述配置规则包括一个或者多个规则;Defining a configuration rule for data processing in a database, the configuration rule including one or more rules;
将所述配置规则写入至描述文件中;Writing the configuration rule into the description file;
若检测到用户对数据库中数据的采集,接收用户所采集的待处理数据;If the user collects data in the database, the data to be processed collected by the user is received;
读取所述描述文件并解析所述描述文件中的配置规则;Reading the description file and parsing configuration rules in the description file;
根据所述配置规则对所述待处理数据进行处理。The to-be-processed data is processed according to the configuration rule.
在一个实施例中,所述预先定义数据处理的配置规则具体包括:In an embodiment, the configuration rule of the predefined data processing specifically includes:
选择进行规则定义的程序源;根据所述程序源创建所述配置规则。Selecting a program source for the rule definition; creating the configuration rule based on the program source.
在一个实施例中,所述配置规则包括字段值规则和分隔符拼接规则,其中,所述字段值规则是选择需要设置的字符,将所选择的字符用其它字符进行替换;所述分隔符拼接规则是选择需要设置拼接规则的字符,将所选择的字符用分隔符进行拼接。In one embodiment, the configuration rule includes a field value rule and a separator splicing rule, wherein the field value rule is to select a character to be set, and replace the selected character with another character; the separator splicing The rule is to select the characters that need to set the splicing rules, and splicing the selected characters with separators.
在一个实施例中,所述根据所述配置规则对所述待处理数据进行处理具体包括:In an embodiment, the processing the data to be processed according to the configuration rule specifically includes:
按照各个规则的执行逻辑对所述待处理数据进行处理并得到已处理数据;将所述已处理数据保存至预设的文档中。Processing the to-be-processed data according to execution logic of each rule and obtaining processed data; saving the processed data to a preset document.
在一个实施例中,所述描述文件包括XML格式的文件。In one embodiment, the description file includes a file in XML format.
本领域技术人员可以理解,图7中示出的数据处理设备的实施例并不构成对数据处理设备具体构成的限定,在其他实施例中,数据处理设备可以包括比 图示更多或更少的部件,或者组合某些部件,或者不同的部件布置。例如,在一些实施例中,数据处理设备可以仅包括存储器及处理器,在这样的实施例中,存储器及处理器的结构及功能与图7所示实施例一致,在此不再赘述。It will be understood by those skilled in the art that the embodiment of the data processing device shown in FIG. 7 does not constitute a limitation on the specific configuration of the data processing device. In other embodiments, the data processing device may include more or less than the illustration. Parts, or combine some parts, or different parts. For example, in some embodiments, the data processing device may include only the memory and the processor. In such an embodiment, the structure and function of the memory and the processor are the same as those of the embodiment shown in FIG. 7, and details are not described herein.
本申请提供了一种计算机可读存储介质,计算机可读存储介质存储有一个或者一个以上计算机程序,所述一个或者一个以上计算机程序可被一个或者一个以上的处理器执行,以实现上述数据处理方法。The application provides a computer readable storage medium storing one or more computer programs, the one or more computer programs being executable by one or more processors to implement the data processing described above method.
本申请前述的存储介质包括:磁碟、光盘、只读存储记忆体(Read-Only Memory,ROM)等各种可以存储程序代码的介质。The foregoing storage medium of the present application includes: a magnetic disk, an optical disk, a read-only memory (ROM), and the like, which can store various program codes.
本申请所有实施例中的单元可以通过通用集成电路,例如CPU(Central Processing Unit,中央处理器),或通过ASIC(Application Specific Integrated Circuit,专用集成电路)来实现。The units in all the embodiments of the present application may be implemented by a general-purpose integrated circuit, such as a CPU (Central Processing Unit), or by an ASIC (Application Specific Integrated Circuit).
本申请实施例数据处理方法中的步骤可以根据实际需要进行顺序调整、合并和删减。The steps in the data processing method of the embodiment of the present application may be sequentially adjusted, merged, and deleted according to actual needs.
本申请实施例终端中的单元可以根据实际需要进行合并、划分和删减。The units in the terminal in the embodiment of the present application may be combined, divided, and deleted according to actual needs.
以上所述,仅为本申请的具体实施方式,但本申请的保护范围并不局限于此,任何熟悉本技术领域的技术人员在本申请揭露的技术范围内,可轻易想到各种等效的修改或替换,这些修改或替换都应涵盖在本申请的保护范围之内。因此,本申请的保护范围应以权利要求的保护范围为准。The foregoing is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto, and any equivalents can be easily conceived by those skilled in the art within the technical scope disclosed in the present application. Modifications or substitutions are intended to be included within the scope of the present application. Therefore, the scope of protection of this application should be determined by the scope of protection of the claims.
Claims (20)
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710228803.6 | 2017-04-10 | ||
| CN201710228803.6A CN107798035A (en) | 2017-04-10 | 2017-04-10 | A kind of data processing method and terminal |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2018188539A1 true WO2018188539A1 (en) | 2018-10-18 |
Family
ID=61530255
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2018/082229 Ceased WO2018188539A1 (en) | 2017-04-10 | 2018-04-09 | Data processing method, terminal, device, and storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN107798035A (en) |
| WO (1) | WO2018188539A1 (en) |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107798035A (en) * | 2017-04-10 | 2018-03-13 | 平安科技(深圳)有限公司 | A kind of data processing method and terminal |
| CN111125589B (en) * | 2018-10-31 | 2023-09-05 | 新方正控股发展有限责任公司 | Data acquisition method and device and computer readable storage medium |
| CN109814923B (en) * | 2018-12-28 | 2024-05-10 | 平安科技(深圳)有限公司 | Data processing method, device, computer equipment and storage medium |
| CN109783543B (en) * | 2019-01-14 | 2021-07-02 | 广州虎牙信息科技有限公司 | Data query method, device, equipment and storage medium |
| CN109947429B (en) * | 2019-03-13 | 2022-07-26 | 咪咕文化科技有限公司 | Data processing method and device |
| CN111124548B (en) * | 2019-12-31 | 2023-10-27 | 科大国创软件股份有限公司 | Rule analysis method and system based on YAML file |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102708142A (en) * | 2012-03-14 | 2012-10-03 | 北京神州数码思特奇信息技术股份有限公司 | Method and system for acquiring background data through structured query language (SQL) custom-made template |
| CN103488634A (en) * | 2012-06-11 | 2014-01-01 | 腾讯科技(深圳)有限公司 | Method and device for checking data relation of data sheet |
| US20150026114A1 (en) * | 2013-07-18 | 2015-01-22 | Dania M. Triff | System and method of automatically extracting data from plurality of data sources and loading the same to plurality of target databases |
| CN104537012A (en) * | 2014-12-17 | 2015-04-22 | 广州酷狗计算机科技有限公司 | Data processing method and device |
| CN106201537A (en) * | 2016-07-18 | 2016-12-07 | 浪潮通用软件有限公司 | A kind of data processing method and device |
| CN107798035A (en) * | 2017-04-10 | 2018-03-13 | 平安科技(深圳)有限公司 | A kind of data processing method and terminal |
-
2017
- 2017-04-10 CN CN201710228803.6A patent/CN107798035A/en active Pending
-
2018
- 2018-04-09 WO PCT/CN2018/082229 patent/WO2018188539A1/en not_active Ceased
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102708142A (en) * | 2012-03-14 | 2012-10-03 | 北京神州数码思特奇信息技术股份有限公司 | Method and system for acquiring background data through structured query language (SQL) custom-made template |
| CN103488634A (en) * | 2012-06-11 | 2014-01-01 | 腾讯科技(深圳)有限公司 | Method and device for checking data relation of data sheet |
| US20150026114A1 (en) * | 2013-07-18 | 2015-01-22 | Dania M. Triff | System and method of automatically extracting data from plurality of data sources and loading the same to plurality of target databases |
| CN104537012A (en) * | 2014-12-17 | 2015-04-22 | 广州酷狗计算机科技有限公司 | Data processing method and device |
| CN106201537A (en) * | 2016-07-18 | 2016-12-07 | 浪潮通用软件有限公司 | A kind of data processing method and device |
| CN107798035A (en) * | 2017-04-10 | 2018-03-13 | 平安科技(深圳)有限公司 | A kind of data processing method and terminal |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107798035A (en) | 2018-03-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2018188539A1 (en) | Data processing method, terminal, device, and storage medium | |
| US8661023B1 (en) | Optimizing search query logic to speed retrieval | |
| US9798774B1 (en) | Graph data search method and apparatus | |
| US9189506B2 (en) | Database index management | |
| CN110909015B (en) | Splitting method, device and equipment of microservice and storage medium | |
| CN111897891B (en) | Data processing method and device | |
| TW202029079A (en) | Method and device for identifying irregular group | |
| JP6070936B2 (en) | Information processing apparatus, information processing method, and program | |
| CN112632038A (en) | Database migration method, device, terminal and storage medium | |
| WO2020215689A1 (en) | Query method and apparatus for column-oriented files | |
| CN104516921B (en) | Automatic reply method and device | |
| WO2019161645A1 (en) | Shell-based data table extraction method, terminal, device, and storage medium | |
| CN103777934B (en) | A kind of method and system generating controller CAN alternation of bed based on MATLAB | |
| CN111309593A (en) | JSON interface verification method, device and equipment and computer readable storage medium | |
| WO2021051624A1 (en) | Data acquisition method and apparatus, and electronic device and storage medium | |
| CN115617338A (en) | A method, device, and readable storage medium for quickly generating business pages | |
| CN107102995A (en) | A kind of determination method and device of SQL executive plans | |
| CN104537012B (en) | Data processing method and device | |
| CN110222046B (en) | List data processing method, device, server and storage medium | |
| CN110209885B (en) | Graph query method and system | |
| CN112685557B (en) | Visual information resource management method and device | |
| WO2025222808A1 (en) | Code editing method and apparatus, device, and storage medium | |
| CN118820396A (en) | MOM business code retrieval and recommendation method based on abstract syntax tree, computer program product and terminal | |
| CN117435611A (en) | Query statement generation method, device, equipment and storage medium | |
| CN114528313A (en) | Data processing method and device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 18784553 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 17.01.2020) |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 18784553 Country of ref document: EP Kind code of ref document: A1 |