[go: up one dir, main page]

WO2019006634A1 - Big data storage system for computer - Google Patents

Big data storage system for computer Download PDF

Info

Publication number
WO2019006634A1
WO2019006634A1 PCT/CN2017/091579 CN2017091579W WO2019006634A1 WO 2019006634 A1 WO2019006634 A1 WO 2019006634A1 CN 2017091579 W CN2017091579 W CN 2017091579W WO 2019006634 A1 WO2019006634 A1 WO 2019006634A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
unit
storage system
collection
password
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/091579
Other languages
French (fr)
Chinese (zh)
Inventor
陈钦鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Comix Group Co Ltd
Original Assignee
Shenzhen Comix Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Comix Group Co Ltd filed Critical Shenzhen Comix Group Co Ltd
Priority to PCT/CN2017/091579 priority Critical patent/WO2019006634A1/en
Publication of WO2019006634A1 publication Critical patent/WO2019006634A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/215Improving data quality; Data cleansing, e.g. de-duplication, removing invalid entries or correcting typographical errors
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/26Visual data mining; Browsing structured data
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/284Relational databases
    • G06F16/285Clustering or classification
    • G06F16/287Visualization; Browsing

Definitions

  • the invention belongs to the technical field of computers, and in particular relates to a computer big data storage system.
  • an embodiment of the present invention provides a computer big data storage system, which can perform effective deduplication screening and integration of distributed, different types and different standards of data.
  • a computer big data storage system comprising: a data collection unit, a data integration unit, a data exchange unit, a data classification unit, a data processing unit, a storage unit, and a display unit; wherein the data The collection unit is connected to the network server, and collects corresponding data from the network server; the data integration unit is configured to acquire various types of data collected by the data collection unit, and separately perform data cleaning on various types of data to exclude Abnormal data; the data exchange unit for exchanging and supplementing various types of data missing values, and publishing the exchanged and supplemental complete data to a data classification unit; the data classification unit for classifying the data Processing, and distributing the classified processed data to the data processing unit; the number of the data processing units is one or more, and is used for performing sample data extraction on various types of data distributed by the data classification unit, and removing redundant data to extract the reduced data.
  • a set the number of storage units and the data processing unit The same amount, and the storage unit and the data processing unit is connected to one
  • the storage unit is a mechanical hard disk, a hybrid hard disk, and/or a solid state hard disk.
  • a password setting unit and a password collection and identification unit connected to the storage unit are further included.
  • the password setting unit is a symbol password setting unit
  • the password collection and identification unit is a symbol collection and identification unit
  • the password setting unit is a fingerprint password setting unit
  • the password collection and identification unit is a fingerprint collection and identification unit
  • the display unit is a smart device with a touch display screen.
  • the solid state hard disk includes a first data interface, a first bridge controller, at least one multi-level unit MLC flash drive, and at least one single-layer unit SLC flash drive.
  • the solid state hard disk further includes:
  • a first flash controller coupled between the first bridge controller and the multi-level cell MLC flash drive
  • a second flash controller is coupled between the first bridge controller and the single layer unit SLC flash drive.
  • the solid state hard disk includes a second data interface, a second bridge controller, at least one multi-layer unit TLC flash disk, and at least one single-layer SLC flash disk.
  • the solid state hard disk further includes:
  • a third flash controller coupled between the second bridge controller and the multi-layer cell TLC flash drive
  • a fourth flash controller is coupled between the second bridge controller and the single layer unit SLC flash drive.
  • the computer big data storage system performs data cleaning on various types of data collected by the data collection unit from the network server through the data integration unit, and excludes abnormal data; and the various types of data are deleted through the data exchange unit.
  • the values are exchanged and supplemented, the exchanged and supplemented complete data is distributed to the data classification unit for classification processing, and the classified processed data is distributed to the data processing unit for sample data extraction, and the redundant data extraction and reduced data set is stored to the corresponding
  • the storage unit can input and operate the operation information and view the data information through the display unit, and can effectively filter and integrate the decentralized, different types and different standards of data, and ensure data quality and transmission stability.
  • FIG. 1 is a schematic structural diagram of a computer storage system according to an embodiment of the present invention.
  • FIG. 2 is a schematic structural diagram of a storage unit according to an embodiment of the present invention.
  • FIG. 3 is a schematic structural diagram of another storage unit according to an embodiment of the present invention.
  • the computer big data storage system performs data cleaning on various types of data collected by the data collection unit from the network server through the data integration unit, and excludes abnormal data; and the various types of data are deleted through the data exchange unit.
  • the values are exchanged and supplemented, the exchanged and supplemented complete data is distributed to the data classification unit for classification processing, and the classified processed data is distributed to the data processing unit for sample data extraction, and the redundant data extraction and reduced data set is stored to the corresponding
  • the storage unit can input and operate the operation information and view the data information through the display unit, and can effectively filter and integrate the decentralized, different types and different standards of data, and ensure data quality and transmission stability.
  • a computer big data storage system includes: a data collection unit 100, a data integration unit 200, a data exchange unit 300, a data classification unit 400, a data processing unit 500, and a storage unit. And the display unit 700; wherein the data collection unit 100 is connected to the network server, and collects corresponding data from the network server; the data integration unit 200 is configured to acquire various types of data collected by the data collection unit 100. And separately performing data cleaning on various types of data to exclude abnormal data; the data exchange unit 300 is configured to exchange and supplement various types of data missing values, and publish and replenish the exchanged data to the data.
  • a classification unit 400 configured to perform classification processing on data, and distribute the classified processing data to the data processing unit 500; the number of the data processing units 500 is one or more for pairing data
  • the data of various types distributed by the classification unit 400 is subjected to sample data extraction, and the redundant data is extracted to extract the reduced data set;
  • the number of the storage units 600 is the same as the number of the data processing units 500, and the storage unit 600 is connected to the data processing unit 500 one by one;
  • the display unit 700 is configured for the user to input operation instructions and view data information. . It can effectively filter and integrate scattered, different types and different standards of data, and ensure data quality and transmission stability.
  • the storage unit 600 is a mechanical hard disk, a hybrid hard disk, and/or a solid state hard disk, and may be selectively installed according to actual needs or personal wishes.
  • a preferred one may be a solid state hard disk.
  • the system further includes a password setting unit 800 and a password collection and identification unit 900 connected to the storage unit 600.
  • the password setting unit 800 may be a symbol password setting unit
  • the password collection and identification unit 900 may be a symbol collection and identification unit.
  • the user may pre-store the first symbol password set by the password setting unit, and when the user inputs the second symbol password in the password input area on the password setting unit, the password collection and recognition unit collects the second symbol password input by the user, and The collected second symbol password is matched with the pre-stored first symbol password.
  • the second symbol password matches the first symbol password
  • the user can view the data information stored in the storage unit through the display unit. Conversely, the user cannot view the data information stored in the storage unit.
  • the password setting unit 800 can be a fingerprint password setting unit
  • the password collection and identification unit 900 can be a fingerprint collection and identification unit.
  • the user may pre-store the set first fingerprint password through the password setting unit.
  • the password collection and recognition unit collects the second fingerprint password input by the user, and The collected second fingerprint password is matched with the pre-stored first fingerprint password.
  • the second fingerprint password matches the first symbol password
  • the user can view the data information stored in the storage unit through the display unit. Conversely, the user cannot view the data information stored in the storage unit.
  • the password setting unit 800 can also be a portrait password setting unit, and the password collection and identification unit 900 can also be a portrait collection and recognition unit; the password setting unit 800 can also be an audio password setting unit, the password.
  • the collection and identification unit 900 can also be an audio collection and identification unit; specifically, it can be set according to actual needs or personal wishes, and is not limited.
  • the display unit 700 is a smart device with a touch display screen; the touch display screen can be a capacitive touch display, a resistive touch display, an infrared touch display, and the like.
  • the type of touch display is not limited.
  • the solid state hard disk includes a first data interface 610, a first bridge controller 620, at least one multi-level cell MLC flash disk 630, and at least one single layer unit SLC flash memory. Disk 640.
  • the solid state hard disk further includes: a first flash controller 650 connected between the first bridge controller 620 and the multi-level cell MLC flash disk 630; and a second flash controller 660, connected Between the first bridge controller 620 and the single layer unit SLC flash disk 640.
  • the solid state hard disk includes a second data interface 601, a second bridge controller 602, at least one multi-layer unit TLC flash disk 603, and at least one single-layer SLC flash memory. Disk 604.
  • the solid state hard disk further includes: a third flash controller 605 connected between the second bridge controller 602 and the multi-layer cell TLC flash disk 603; and a fourth flash controller 606, connected Between the second bridge controller 602 and the single layer unit SLC flash disk 604.
  • the computer big data storage system performs data cleaning on various types of data collected by the data collection unit from the network server through the data integration unit, and excludes abnormal data; and uses various types of data to be deleted through the data exchange unit.
  • the values are exchanged and supplemented, the exchanged and supplemented complete data is distributed to the data classification unit for classification processing, and the classified processed data is distributed to the data processing unit for sample data extraction, and the redundant data extraction and reduced data set is stored to the corresponding
  • the storage unit can input and operate the operation information and view the data information through the display unit, and can effectively filter and integrate the decentralized, different types and different standards of data, and ensure data quality and transmission stability.

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Quality & Reliability (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

A big data storage system for a computer comprises a data collection unit (100), a data integration unit (200), a data exchange unit (300), a data classification unit (400), data processing units (500), storage units (600), and a display unit (700). The data integration unit (200) performs data cleaning on all types of data collected by the data collection unit (100) from a network server, and abnormal data is removed. The data exchange unit (300) exchanges and complements all types of data missing values, the exchanged and complemented data is sent to the data classification unit (400) for classification processing, the data on which classification processing has been performed is sent to the data processing units (500) for sample data extraction, and redundant data is removed, simplified data sets are extracted and stored into the corresponding storage units (600). A user can input an operation instruction and view data information by means of the display unit (700), repeated data in dispersive data of different types and different standards can be effectively removed, and the dispersive data is effectively screened and integrated, and the quality and the transmission stability of the data are ensured.

Description

一种计算机大数据存储系统  Computer big data storage system 技术领域Technical field

本发明属于计算机技术领域,尤其涉及一种计算机大数据存储系统。The invention belongs to the technical field of computers, and in particular relates to a computer big data storage system.

背景技术Background technique

随着科技技术的进步,全球的数据量呈几何级数逐年递增,其数据资源的庞大,导致了数据存储系统的复杂化,如何对获取的数据进行自动分类整理,筛选掉无光紧要的数据,成为大数据存储中亟待解决的技术问题。With the advancement of technology, the amount of data in the world has increased geometrically year by year, and its huge data resources have led to the complication of data storage systems. How to automatically sort the acquired data and filter out the data that is not important. It has become a technical problem to be solved in big data storage.

技术问题technical problem

为了解决上述技术问题,本发明实施例提供一种计算机大数据存储系统,能够将分散的、不同类型的,不同标准的数据进行有效的除重筛选和整合。In order to solve the above technical problem, an embodiment of the present invention provides a computer big data storage system, which can perform effective deduplication screening and integration of distributed, different types and different standards of data.

技术解决方案Technical solution

本发明实施例是这样实现的,一种计算机大数据存储系统,包括:数据采集单元、数据整合单元、数据交换单元、数据分类单元、数据处理单元、存储单元以及显示单元;其中,所述数据采集单元与网络服务器连接,从网络服务器中采集相应的数据;所述数据整合单元,用于获取数据采集单元所采集的各种类型的数据,并分别对各种类型的数据进行数据清理,排除异常数据;所述数据交换单元,用于对各种类型的数据缺失值进行交换和补充,并将交换和补充完整的数据发布至数据分类单元;所述数据分类单元,用于将数据进行分类处理,并将分类处理的数据分发至数据处理单元;所述数据处理单元的数量为一个或多个,用于对数据分类单元分发的各类数据进行样本数据抽取,去除冗余数据提取精简数据集;所述存储单元的数量与所述数据处理单元数量相同,且所述存储单元与所述数据处理单元一一对应连接;所述显示单元,用于用户输入操作指令和查看数据信息。The embodiment of the present invention is implemented as follows: a computer big data storage system, comprising: a data collection unit, a data integration unit, a data exchange unit, a data classification unit, a data processing unit, a storage unit, and a display unit; wherein the data The collection unit is connected to the network server, and collects corresponding data from the network server; the data integration unit is configured to acquire various types of data collected by the data collection unit, and separately perform data cleaning on various types of data to exclude Abnormal data; the data exchange unit for exchanging and supplementing various types of data missing values, and publishing the exchanged and supplemental complete data to a data classification unit; the data classification unit for classifying the data Processing, and distributing the classified processed data to the data processing unit; the number of the data processing units is one or more, and is used for performing sample data extraction on various types of data distributed by the data classification unit, and removing redundant data to extract the reduced data. a set; the number of storage units and the data processing unit The same amount, and the storage unit and the data processing unit is connected to one correspondence; and the display unit for a user to input instructions and data view.

作为本发明的一个优选方案,所述存储单元为机械硬盘、混合硬盘和/或固态硬盘。As a preferred embodiment of the present invention, the storage unit is a mechanical hard disk, a hybrid hard disk, and/or a solid state hard disk.

作为本发明的一个优选方案,还包括连接存储单元的密码设置单元及密码采集识别单元。As a preferred solution of the present invention, a password setting unit and a password collection and identification unit connected to the storage unit are further included.

作为本发明的一个优选方案,所述密码设置单元为符号密码设置单元,所述密码采集识别单元为符号采集识别单元。As a preferred solution of the present invention, the password setting unit is a symbol password setting unit, and the password collection and identification unit is a symbol collection and identification unit.

作为本发明的一个优选方案,所述密码设置单元为指纹密码设置单元,所述密码采集识别单元为指纹采集识别单元。As a preferred solution of the present invention, the password setting unit is a fingerprint password setting unit, and the password collection and identification unit is a fingerprint collection and identification unit.

作为本发明的一个优选方案,所述显示单元为具有触控显示屏的智能设备。As a preferred solution of the present invention, the display unit is a smart device with a touch display screen.

作为本发明的一个优选方案,所述固态硬盘包括第一数据接口、第一桥接控制器、至少一多层单元MLC闪存盘以及至少一单层单元SLC闪存盘。As a preferred embodiment of the present invention, the solid state hard disk includes a first data interface, a first bridge controller, at least one multi-level unit MLC flash drive, and at least one single-layer unit SLC flash drive.

作为本发明的一个优选方案,所述固态硬盘还包括:As a preferred solution of the present invention, the solid state hard disk further includes:

第一闪存控制器,连接在所述第一桥接控制器及多层单元MLC闪存盘之间;以及a first flash controller coupled between the first bridge controller and the multi-level cell MLC flash drive;

第二闪存控制器,连接在所述第一桥接控制器及单层单元SLC闪存盘之间。A second flash controller is coupled between the first bridge controller and the single layer unit SLC flash drive.

作为本发明的一个优选方案,所述固态硬盘包括第二数据接口、第二桥接控制器、至少一个多层单元TLC闪存盘以及至少一个单层SLC闪存盘。As a preferred embodiment of the present invention, the solid state hard disk includes a second data interface, a second bridge controller, at least one multi-layer unit TLC flash disk, and at least one single-layer SLC flash disk.

作为本发明的一个优选方案,所述固态硬盘还包括:As a preferred solution of the present invention, the solid state hard disk further includes:

第三闪存控制器,连接在所述第二桥接控制器及多层单元TLC闪存盘之间;以及a third flash controller coupled between the second bridge controller and the multi-layer cell TLC flash drive;

第四闪存控制器,连接在所述第二桥接控制器及单层单元SLC闪存盘之间。A fourth flash controller is coupled between the second bridge controller and the single layer unit SLC flash drive.

有益效果Beneficial effect

本发明实施例提供的计算机大数据存储系统,通过数据整合单元对数据采集单元从网络服务器采集的各种类型的数据进行数据清理,排除异常数据;并通过数据交换单元对各种类型的数据缺失值进行交换和补充,将交换和补充完整的数据发布至数据分类单元进行分类处理,并将分类处理的数据分发至数据处理单元进行样本数据抽取,去除冗余数据提取精简数据集存储至对应的存储单元;用户可通过显示单元输入操作指令和查看数据信息,能够将分散的、不同类型的,不同标准的数据进行有效的除重筛选和整合,且保证了数据质量和传输稳定性。The computer big data storage system provided by the embodiment of the present invention performs data cleaning on various types of data collected by the data collection unit from the network server through the data integration unit, and excludes abnormal data; and the various types of data are deleted through the data exchange unit. The values are exchanged and supplemented, the exchanged and supplemented complete data is distributed to the data classification unit for classification processing, and the classified processed data is distributed to the data processing unit for sample data extraction, and the redundant data extraction and reduced data set is stored to the corresponding The storage unit can input and operate the operation information and view the data information through the display unit, and can effectively filter and integrate the decentralized, different types and different standards of data, and ensure data quality and transmission stability.

附图说明DRAWINGS

为了更清楚地说明本发明实施例或现有技术中的技术方案,下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍,显而易见地,下面描述中的附图是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are Some embodiments of the present invention may also be used to obtain other drawings based on these drawings without departing from the art.

以下附图仅旨在于对本发明做示意性说明和解释,并不限定本发明的范围。The following drawings are only intended to illustrate and explain the present invention, and do not limit the scope of the invention.

图1是本发明实施例提供的一种计算机存储系统的结构示意图;1 is a schematic structural diagram of a computer storage system according to an embodiment of the present invention;

图2是本发明实施例提供的一种存储单元的结构示意图;2 is a schematic structural diagram of a storage unit according to an embodiment of the present invention;

图3是本发明实施例提供的另一种存储单元的结构示意图。FIG. 3 is a schematic structural diagram of another storage unit according to an embodiment of the present invention.

本发明的实施方式Embodiments of the invention

为了使本发明的目的、技术方案及优点更加清楚明白,以下结合附图及实施例,对本发明进行进一步详细说明。应当理解,此处所描述的具体实施例仅仅用以解释本发明,并不用于限定本发明。The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

本发明实施例提供的计算机大数据存储系统,通过数据整合单元对数据采集单元从网络服务器采集的各种类型的数据进行数据清理,排除异常数据;并通过数据交换单元对各种类型的数据缺失值进行交换和补充,将交换和补充完整的数据发布至数据分类单元进行分类处理,并将分类处理的数据分发至数据处理单元进行样本数据抽取,去除冗余数据提取精简数据集存储至对应的存储单元;用户可通过显示单元输入操作指令和查看数据信息,能够将分散的、不同类型的,不同标准的数据进行有效的除重筛选和整合,且保证了数据质量和传输稳定性。The computer big data storage system provided by the embodiment of the present invention performs data cleaning on various types of data collected by the data collection unit from the network server through the data integration unit, and excludes abnormal data; and the various types of data are deleted through the data exchange unit. The values are exchanged and supplemented, the exchanged and supplemented complete data is distributed to the data classification unit for classification processing, and the classified processed data is distributed to the data processing unit for sample data extraction, and the redundant data extraction and reduced data set is stored to the corresponding The storage unit can input and operate the operation information and view the data information through the display unit, and can effectively filter and integrate the decentralized, different types and different standards of data, and ensure data quality and transmission stability.

以下结合具体实施例对本发明的具体实现进行详细描述。The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

如图1所示,在本发明实施例中,一种计算机大数据存储系统,包括:数据采集单元100、数据整合单元200、数据交换单元300、数据分类单元400、数据处理单元500、存储单元600以及显示单元700;其中,所述数据采集单元100与网络服务器连接,从网络服务器中采集相应的数据;所述数据整合单元200,用于获取数据采集单元100所采集的各种类型的数据,并分别对各种类型的数据进行数据清理,排除异常数据;所述数据交换单元300,用于对各种类型的数据缺失值进行交换和补充,并将交换和补充完整的数据发布至数据分类单元400;所述数据分类单元400,用于将数据进行分类处理,并将分类处理的数据分发至数据处理单元500;所述数据处理单元500的数量为一个或多个,用于对数据分类单元400分发的各类数据进行样本数据抽取,去除冗余数据提取精简数据集;所述存储单元600的数量与所述数据处理单元500数量相同,且所述存储单元600与所述数据处理单元500一一对应连接;所述显示单元700,用于用户输入操作指令和查看数据信息。能够将分散的、不同类型的,不同标准的数据进行有效的除重筛选和整合,且保证了数据质量和传输稳定性As shown in FIG. 1 , in the embodiment of the present invention, a computer big data storage system includes: a data collection unit 100, a data integration unit 200, a data exchange unit 300, a data classification unit 400, a data processing unit 500, and a storage unit. And the display unit 700; wherein the data collection unit 100 is connected to the network server, and collects corresponding data from the network server; the data integration unit 200 is configured to acquire various types of data collected by the data collection unit 100. And separately performing data cleaning on various types of data to exclude abnormal data; the data exchange unit 300 is configured to exchange and supplement various types of data missing values, and publish and replenish the exchanged data to the data. a classification unit 400; the data classification unit 400 is configured to perform classification processing on data, and distribute the classified processing data to the data processing unit 500; the number of the data processing units 500 is one or more for pairing data The data of various types distributed by the classification unit 400 is subjected to sample data extraction, and the redundant data is extracted to extract the reduced data set; The number of the storage units 600 is the same as the number of the data processing units 500, and the storage unit 600 is connected to the data processing unit 500 one by one; the display unit 700 is configured for the user to input operation instructions and view data information. . It can effectively filter and integrate scattered, different types and different standards of data, and ensure data quality and transmission stability.

在本实施例中,所述存储单元600为机械硬盘、混合硬盘和/或固态硬盘,具体可根据实际需求或个人意愿进行选择安装,在本实施例中优选的可为固态硬盘。In this embodiment, the storage unit 600 is a mechanical hard disk, a hybrid hard disk, and/or a solid state hard disk, and may be selectively installed according to actual needs or personal wishes. In this embodiment, a preferred one may be a solid state hard disk.

在本实施例中,所述系统还包括连接存储单元600的密码设置单元800及密码采集识别单元900。In this embodiment, the system further includes a password setting unit 800 and a password collection and identification unit 900 connected to the storage unit 600.

作为本发明的一个优选实施例,所述密码设置单元800可为符号密码设置单元,所述密码采集识别单元900可为符号采集识别单元。例如,用户可通过密码设置单元预存设置的第一符号密码,当用户在密码设置单元上的密码输入区域内输入第二符号密码时,密码采集识别单元采集用户输入的第二符号密码,并将所采集的第二符号密码与预存设置的第一符号密码进行匹配,当所述第二符号密码与所述第一符号密码相匹配时,用户可通过显示单元查看存储单元内存储的数据信息,反之,则用户无法查看存储单元内存储的数据信息。As a preferred embodiment of the present invention, the password setting unit 800 may be a symbol password setting unit, and the password collection and identification unit 900 may be a symbol collection and identification unit. For example, the user may pre-store the first symbol password set by the password setting unit, and when the user inputs the second symbol password in the password input area on the password setting unit, the password collection and recognition unit collects the second symbol password input by the user, and The collected second symbol password is matched with the pre-stored first symbol password. When the second symbol password matches the first symbol password, the user can view the data information stored in the storage unit through the display unit. Conversely, the user cannot view the data information stored in the storage unit.

作为本发明的又一个优选实施例,所述密码设置单元800可为指纹密码设置单元,所述密码采集识别单元900可为指纹采集识别单元。例如,用户可通过密码设置单元预存设置的第一指纹密码,当用户在密码设置单元上的密码输入区域内输入第二指纹密码时,密码采集识别单元采集用户输入的第二指纹密码,并将所采集的第二指纹密码与预存设置的第一指纹密码进行匹配,当所述第二指纹密码与所述第一符号密码相匹配时,用户可通过显示单元查看存储单元内存储的数据信息,反之,则用户无法查看存储单元内存储的数据信息。As still another preferred embodiment of the present invention, the password setting unit 800 can be a fingerprint password setting unit, and the password collection and identification unit 900 can be a fingerprint collection and identification unit. For example, the user may pre-store the set first fingerprint password through the password setting unit. When the user inputs the second fingerprint password in the password input area on the password setting unit, the password collection and recognition unit collects the second fingerprint password input by the user, and The collected second fingerprint password is matched with the pre-stored first fingerprint password. When the second fingerprint password matches the first symbol password, the user can view the data information stored in the storage unit through the display unit. Conversely, the user cannot view the data information stored in the storage unit.

可以理解的,所述密码设置单元800还可为人像密码设置单元,所述密码采集识别单元900还可为人像采集识别单元;所述密码设置单元800还可为音频密码设置单元,所述密码采集识别单元900还可为音频采集识别单元;具体可根据实际需求或个人意愿进行设置,具体不做限制。It can be understood that the password setting unit 800 can also be a portrait password setting unit, and the password collection and identification unit 900 can also be a portrait collection and recognition unit; the password setting unit 800 can also be an audio password setting unit, the password. The collection and identification unit 900 can also be an audio collection and identification unit; specifically, it can be set according to actual needs or personal wishes, and is not limited.

在本实施例中,所述显示单元700为具有触控显示屏的智能设备;所述触控显示屏可为电容式触控显示屏、电阻式触控显示屏、红外触控显示屏等其他类型的触控显示屏,具体不做限制。In this embodiment, the display unit 700 is a smart device with a touch display screen; the touch display screen can be a capacitive touch display, a resistive touch display, an infrared touch display, and the like. The type of touch display is not limited.

作为本发明的一个优选实施例,如图2所示,所述固态硬盘包括第一数据接口610、第一桥接控制器620、至少一多层单元MLC闪存盘630以及至少一单层单元SLC闪存盘640。As a preferred embodiment of the present invention, as shown in FIG. 2, the solid state hard disk includes a first data interface 610, a first bridge controller 620, at least one multi-level cell MLC flash disk 630, and at least one single layer unit SLC flash memory. Disk 640.

在本实施例中,所述固态硬盘还包括:第一闪存控制器650,连接在所述第一桥接控制器620及多层单元MLC闪存盘630之间;以及第二闪存控制器660,连接在所述第一桥接控制器620及单层单元SLC闪存盘640之间。In this embodiment, the solid state hard disk further includes: a first flash controller 650 connected between the first bridge controller 620 and the multi-level cell MLC flash disk 630; and a second flash controller 660, connected Between the first bridge controller 620 and the single layer unit SLC flash disk 640.

作为本发明的又一个优选实施例,如图3所示,所述固态硬盘包括第二数据接口601、第二桥接控制器602、至少一个多层单元TLC闪存盘603以及至少一个单层SLC闪存盘604。As still another preferred embodiment of the present invention, as shown in FIG. 3, the solid state hard disk includes a second data interface 601, a second bridge controller 602, at least one multi-layer unit TLC flash disk 603, and at least one single-layer SLC flash memory. Disk 604.

在本实施例中,所述固态硬盘还包括:第三闪存控制器605,连接在所述第二桥接控制器602及多层单元TLC闪存盘603之间;以及第四闪存控制器606,连接在所述第二桥接控制器602及单层单元SLC闪存盘604之间。In this embodiment, the solid state hard disk further includes: a third flash controller 605 connected between the second bridge controller 602 and the multi-layer cell TLC flash disk 603; and a fourth flash controller 606, connected Between the second bridge controller 602 and the single layer unit SLC flash disk 604.

上述发明实施例提供的计算机大数据存储系统,通过数据整合单元对数据采集单元从网络服务器采集的各种类型的数据进行数据清理,排除异常数据;并通过数据交换单元对各种类型的数据缺失值进行交换和补充,将交换和补充完整的数据发布至数据分类单元进行分类处理,并将分类处理的数据分发至数据处理单元进行样本数据抽取,去除冗余数据提取精简数据集存储至对应的存储单元;用户可通过显示单元输入操作指令和查看数据信息,能够将分散的、不同类型的,不同标准的数据进行有效的除重筛选和整合,且保证了数据质量和传输稳定性The computer big data storage system provided by the above embodiment of the invention performs data cleaning on various types of data collected by the data collection unit from the network server through the data integration unit, and excludes abnormal data; and uses various types of data to be deleted through the data exchange unit. The values are exchanged and supplemented, the exchanged and supplemented complete data is distributed to the data classification unit for classification processing, and the classified processed data is distributed to the data processing unit for sample data extraction, and the redundant data extraction and reduced data set is stored to the corresponding The storage unit can input and operate the operation information and view the data information through the display unit, and can effectively filter and integrate the decentralized, different types and different standards of data, and ensure data quality and transmission stability.

以上所述仅为本发明的较佳实施例而已,并不用以限制本发明,凡在本发明的精神和原则之内所作的任何修改、等同替换和改进等,均应包含在本发明的保护范围之内。The above is only the preferred embodiment of the present invention, and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection of the present invention. Within the scope.

Claims (10)

一种计算机大数据存储系统,其特征在于,包括:数据采集单元、数据整合单元、数据交换单元、数据分类单元、数据处理单元、存储单元以及显示单元;其中,所述数据采集单元与网络服务器连接,从网络服务器中采集相应的数据;所述数据整合单元,用于获取数据采集单元所采集的各种类型的数据,并分别对各种类型的数据进行数据清理,排除异常数据;所述数据交换单元,用于对各种类型的数据缺失值进行交换和补充,并将交换和补充完整的数据发布至数据分类单元;所述数据分类单元,用于将数据进行分类处理,并将分类处理的数据分发至数据处理单元;所述数据处理单元的数量为一个或多个,用于对数据分类单元分发的各类数据进行样本数据抽取,去除冗余数据提取精简数据集;所述存储单元的数量与所述数据处理单元数量相同,且所述存储单元与所述数据处理单元一一对应连接;所述显示单元,用于用户输入操作指令和查看数据信息。 A computer big data storage system, comprising: a data collection unit, a data integration unit, a data exchange unit, a data classification unit, a data processing unit, a storage unit, and a display unit; wherein the data collection unit and the network server Connecting, collecting corresponding data from the network server; the data integration unit is configured to acquire various types of data collected by the data collection unit, and separately perform data cleaning on various types of data to exclude abnormal data; a data exchange unit for exchanging and supplementing various types of data missing values, and publishing the exchanged and supplemental complete data to a data classification unit; the data classification unit for classifying the data and classifying the data The processed data is distributed to the data processing unit; the number of the data processing units is one or more, for performing sample data extraction on various types of data distributed by the data classification unit, and removing the redundant data extraction reduced data set; the storing The number of units is the same as the number of data processing units, and The storage unit is connected to the data processing unit in one-to-one correspondence; the display unit is configured to input an operation instruction and view data information by a user. 如权利要求1所述的计算机大数据存储系统,其特征在于,所述存储单元为机械硬盘、混合硬盘和/或固态硬盘。The computer big data storage system of claim 1 wherein said storage unit is a mechanical hard disk, a hybrid hard disk, and/or a solid state hard disk. 如权利要求1或2所述的计算机大数据存储系统,其特征在于,还包括连接存储单元的密码设置单元及密码采集识别单元。The computer big data storage system according to claim 1 or 2, further comprising a password setting unit and a password collection and identification unit connected to the storage unit. 如权利要求3所述的计算机大数据存储系统,其特征在于,所述密码设置单元为符号密码设置单元,所述密码采集识别单元为符号采集识别单元。The computer big data storage system according to claim 3, wherein the password setting unit is a symbol password setting unit, and the password collection and recognition unit is a symbol collection and recognition unit. 如权利要求3所述的计算机大数据存储系统,其特征在于,所述密码设置单元为指纹密码设置单元,所述密码采集识别单元为指纹采集识别单元。The computer big data storage system according to claim 3, wherein the password setting unit is a fingerprint password setting unit, and the password collection and recognition unit is a fingerprint collection and identification unit. 如权利要求1所述的计算机大数据存储系统,其特征在于,所述显示单元为具有触控显示屏的智能设备。The computer big data storage system according to claim 1, wherein the display unit is a smart device having a touch display screen. 如权利要求2所述的计算机大数据存储系统,其特征在于,所述固态硬盘包括第一数据接口、第一桥接控制器、至少一多层单元MLC闪存盘以及至少一单层单元SLC闪存盘。The computer big data storage system according to claim 2, wherein said solid state drive comprises a first data interface, a first bridge controller, at least one multi-level unit MLC flash drive, and at least one single-layer unit SLC flash drive . 如权利要求7所述的计算机大数据存储系统,其特征在于,所述固态硬盘还包括:The computer big data storage system according to claim 7, wherein the solid state hard disk further comprises: 第一闪存控制器,连接在所述第一桥接控制器及多层单元MLC闪存盘之间;以及a first flash controller coupled between the first bridge controller and the multi-level cell MLC flash drive; 第二闪存控制器,连接在所述第一桥接控制器及单层单元SLC闪存盘之间。A second flash controller is coupled between the first bridge controller and the single layer unit SLC flash drive. 如权利要求2所述的计算机大数据存储系统,其特征在于,所述固态硬盘包括第二数据接口、第二桥接控制器、至少一个多层单元TLC闪存盘以及至少一个单层SLC闪存盘。The computer big data storage system of claim 2 wherein said solid state drive comprises a second data interface, a second bridge controller, at least one multi-level cell TLC flash drive, and at least one single layer SLC flash drive. 如权利要求9所述计算机大数据存储系统,其特征在于,所述固态硬盘还包括:The computer big data storage system according to claim 9, wherein the solid state hard disk further comprises: 第三闪存控制器,连接在所述第二桥接控制器及多层单元TLC闪存盘之间;以及a third flash controller coupled between the second bridge controller and the multi-layer cell TLC flash drive; 第四闪存控制器,连接在所述第二桥接控制器及单层单元SLC闪存盘之间。 A fourth flash controller is coupled between the second bridge controller and the single layer unit SLC flash drive.
PCT/CN2017/091579 2017-07-04 2017-07-04 Big data storage system for computer Ceased WO2019006634A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/091579 WO2019006634A1 (en) 2017-07-04 2017-07-04 Big data storage system for computer

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/091579 WO2019006634A1 (en) 2017-07-04 2017-07-04 Big data storage system for computer

Publications (1)

Publication Number Publication Date
WO2019006634A1 true WO2019006634A1 (en) 2019-01-10

Family

ID=64949584

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/091579 Ceased WO2019006634A1 (en) 2017-07-04 2017-07-04 Big data storage system for computer

Country Status (1)

Country Link
WO (1) WO2019006634A1 (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104361022A (en) * 2014-10-22 2015-02-18 浪潮软件集团有限公司 Method based on collected data statistics and foreground display
CN104461551A (en) * 2014-12-16 2015-03-25 芜湖乐锐思信息咨询有限公司 Parallel data processing based big data processing system
CN104933124A (en) * 2015-06-10 2015-09-23 柳州市智融科技有限公司 Internet big data processing system
US20160314134A1 (en) * 2015-04-24 2016-10-27 Google Inc. Apparatus and Methods for Optimizing Dirty Memory Pages in Embedded Devices

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104361022A (en) * 2014-10-22 2015-02-18 浪潮软件集团有限公司 Method based on collected data statistics and foreground display
CN104461551A (en) * 2014-12-16 2015-03-25 芜湖乐锐思信息咨询有限公司 Parallel data processing based big data processing system
US20160314134A1 (en) * 2015-04-24 2016-10-27 Google Inc. Apparatus and Methods for Optimizing Dirty Memory Pages in Embedded Devices
CN104933124A (en) * 2015-06-10 2015-09-23 柳州市智融科技有限公司 Internet big data processing system

Similar Documents

Publication Publication Date Title
CN107391390A (en) A kind of computer big data storage system
US10521396B2 (en) Placement policy
CN103235817B (en) A kind of extensive infection control data storage processing method
CN108269056A (en) Government information resources manage system
CN113220235B (en) Read and write request processing methods, devices, equipment and storage media
US10474620B2 (en) System and method for improving peripheral component interface express bus performance in an information handling system
WO2023096027A1 (en) Data backup system having enhanced security
WO2017008598A1 (en) Big data exchange method and device
CN104391760B (en) A kind of method of data backup, server and system
CN109981697A (en) A file dumping method, system, server and storage medium
WO2019006634A1 (en) Big data storage system for computer
CN110692045A (en) Big data-based stationery information distributed planning system
CN205453758U (en) KVM exchange board based on CPCI interface
CN206021155U (en) A kind of fusion architecture server
CN111797175A (en) Data storage method and device, storage medium and electronic equipment
CN109033444A (en) The method and device across organizational boundary's data sharing is realized based on object storage technology
WO2013159491A1 (en) Method for implementing software tool for use in usb flash disk privacy protection
CN105469487A (en) Face, fingerprint, palm print and weight composite identification access control system based on FPGA (field-programmable gate array)
WO2018157330A1 (en) Big data partitioning method and system
WO2023033199A1 (en) Methods for automatically identifying a match between a product image and a reference drawing based on artificial intelligence
WO2018157391A1 (en) Big-data enterprise evaluation method and system
CN207677880U (en) A kind of video investigation cooperative engagement systems
CN205405508U (en) A data integration dispensing device for database
WO2018157333A1 (en) Method and system for processing big data
CN115934807A (en) A computer data mining exploration method based on big data

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17917026

Country of ref document: EP

Kind code of ref document: A1