CN108062320A - A kind of method for the data acquisition of multipad, conversion and loading - Google Patents
A kind of method for the data acquisition of multipad, conversion and loading Download PDFInfo
- Publication number
- CN108062320A CN108062320A CN201610980683.0A CN201610980683A CN108062320A CN 108062320 A CN108062320 A CN 108062320A CN 201610980683 A CN201610980683 A CN 201610980683A CN 108062320 A CN108062320 A CN 108062320A
- Authority
- CN
- China
- Prior art keywords
- data
- conversion
- multipad
- loading
- processing module
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/254—Extract, transform and load [ETL] procedures, e.g. ETL data flows in data warehouses
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/28—Databases characterised by their database models, e.g. relational or object models
- G06F16/283—Multi-dimensional databases or data warehouses, e.g. MOLAP or ROLAP
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of method for the data acquisition of multipad, conversion and loading, the described method includes：Include the following steps：a）Data pick-up, b）Data transmission and transfer, c）Data receiver and parsing, d）Data conversion, e）Data load.The method of Data collecting conversion and loading proposed by the present invention for multipad, conveniently and efficiently can provide data supporting for other operation systems, to reduce intersystem interface exploitation expense.
Description
Technical field
The invention belongs to distributed medical technical field, it is related to that the source data amount being applicable to is smaller and other side is desktop
The occasion of application program more particularly to a kind of method for the data acquisition of multipad, conversion and loading.
Background technology
At present, hospital information plays a good role in modern hospital construction, in Hospital Informatization process
In, due to the intercommunity between the hardware, software, database of information system be obstructed cannot effectively realize information sharing with hand over
It changes, has triggered a series of problem：Redundancy including system data communicates object in the missing of data, even different system
Description data grave fault；Also there are problems that secondary data typing, related system data maintenance person may be to multiple systems
The system relevant data of typing, have objectively aggravated the workload of system data maintenance person.In order to be better achieved, " disease is
The target of doctor " gives numerous service objects to provide quality services, reduce medical treatment cost, it is necessary under eliminate that " information is lonely energetically
Island ".
At present during informatization, particularly towards analysis and the IN service process excavated, frequently refer to pair
Largely, the scattered, data of isomery are handled, and ETL is essential part during this.ETL：Extract-
The abbreviation of Transform-Load, i.e. data pick-up, conversion, the process loaded.It is responsible in by distribution, heterogeneous data source
Data are such as cleaned after relation data, flat data file are drawn into interim interlayer, convert, integrate, and are finally loaded into
In data warehouse or Data Mart, become the basis of on-line analytical processing, data mining.However using ELT modes, there are many
Defect：
1st, without reusability, portability, typically for specific external data and loaded targets, there is specific data processing
Logic, these logics have been solidificated in an ETL program；So such realization method is only used for specific ETL scenes,
In other scenes, achievement before can not be got up up to new scene or recycling, it can only be again for new specific field
Scape completes a new realization；
2nd, maintainability, ease for use are poor, some technical solutions be by way of " script " or " code ", describe from external data to
The ETL processes of loaded targets.ETL processes as maintenance are related to the management to substantial amounts of " script " or " code ", very mixed
Disorderly, and this proposes quite high requirement to the technical merit for implementing personnel, is otherwise difficult to realize.
The content of the invention
The technical problems to be solved by the invention are：The data that a set of acquisition desktop application is shown are provided, are coordinated in data
Turn, transmission, while configurable data conversion and data loading are directed into a whole set of solution of data warehouse.
In order to solve the above technical problems, the present invention adopts the following technical scheme that：
A kind of method for the data acquisition of multipad, conversion and loading, which is characterized in that for by interface data
The data in source are loaded to data warehouse, included the following steps：
a）Data pick-up,
b）Data transmission and transfer,
c）Data receiver and parsing,
d）Data conversion,
e）Data load.
The step a data pick-ups, including standard control data capture, GDI data captures, image data capture, event
Analysis, condition judgment.
The step d data conversions, including form amendment, field fractionation, field merging, calculated value, dictionary lookup, unit
Conversion, date-time are converted, summarized, leading outer bond reconstruction.
The step e data load, and increment reprinting, destructive merging, constructive merging, condition merge.
The step a data pick-ups for the data that desktop application is shown, inject desktop application journey by developing client
Sequence is sent to data processing server according to being locally configured to obtain the data for needing to capture and pass through transmission channel；
The client supports that process name or other applications feature is specified to carry out the monitoring of multipad.
The client injects multipad, using the function that the message of Windows hooks by specified dynamic base
Destination application is injected, loads the configuration specified, for determining to obtain mode and the time of destination application data.
The time for obtaining destination application data is realized by the way of windows messaging interception.
The data for obtaining destination application are realized by the way of WindowsAPI functions or APIHook.
The step b）Data transmission and transfer, it is characterised in that：
The data transmission, it is contemplated that the fire wall of most of hospitals and port restriction strategy are realized using WebService.
The data relay, it is contemplated that most of hospitals run on Intranet, using in the server portion of the double network segments of intranet and extranet
It affixes one's name to transfer method of service to realize, including WebService transfers service and FTP transfer services.
The WebService transfers service carries out data forwarding using the ServiceHost separate configurations service of WCF.
The FTP transfers service carries out file forwarding using ftp server plus FileWatcher services.
The step c）Data receiver and parsing, it is characterised in that：WebService uses the ServiceHost master of WCF
It services to receive and parse through data；FTP carries out data receiver using ftp server, and FileWatcher is main to be serviced to parse
Data.
The step d）Data conversion, step e）Data are loaded to be realized by developing data processing module.
The data processing module is embedded in main service, to support the conversion of data and loading.
The data processing module uses the preconfigured processing of Windows Workflow Foundation engine loads
Flow.
The data processing module is supported to load different flows using the process identification in metadata information for handling
Different types of data.
The data processing module support to access data in itself, metadata, predefined database, file etc..
The data processing module supports branch process, delay disposal, abnormality processing, issued transaction.
The data processing module can extend the function of its module by self-defined Activity.
The beneficial effects of the present invention are：Data collecting conversion and loading proposed by the present invention for multipad
Method, other side can be obtained in the case where other side's system data structure need not be understood and apply the data shown on interface.
The data shown in most cases are that most intuitive, the data of most availability, this programme focus on to pay close attention to what is on interface showed
Data, and data grabber, data transmission, data forwarding, data conversion, data are provided and load a series of total solutions.Together
When by the configuration of flow chart of data processing, using self-defined Activity and support the mode of script that can not lose flexible journey
In the case of degree, flow chart of data processing is gone out with graphical interfaces rapid configuration, is reached with lightweight solutions similar to ETL system
Effect.
Description of the drawings
Fig. 1 is flow chart of the present invention.
Specific embodiment
The preferred embodiment that the invention will now be described in detail with reference to the accompanying drawings.
Data pick-up part mode operation is resided using client.Previously according to the interfacial structure feature configuration of intended application
It needs to obtain the forms of data, obtain the trigger event of data and the extracting mode of data.Client monitors intended application in real time,
According to preconfigured mode intercepted data and it is sent to the server of configuration.
Server end configuration data receives service, and one layer of forwarding service of intermediate configurations, is turned with adapting to intranet data if necessary
Hair.Data reception service contains data conversion and loading module, carries out data parsing after receiving the data, and submits data conversion
And loading module processing.Server-side uses ServiceHost when being carried out data transmission using WebService, need to configure service
Url, this type are influenced smaller by fire wall.Ftp server and permission need to be configured when transmitting data file using FTP, is needed simultaneously
Configure FileWatcher service monitorings path.
Data conversion and loading module by Windows Workflow Foundation engines, system Activity and from
Activity is defined to form.Advance process for using designer test data can be used to survey according to data rule design cycle
Flow implementing result is tried, flow can be preserved to server-side when meeting expected results.When server-side receives data, according to member
Process identification in data message loads predefined flow, completes the operation of data conversion and loading.
Here description of the invention and application are illustrative, are not wishing to limit the scope of the invention to above-described embodiment
In.The deformation and change of embodiments disclosed herein are possible, real for those skilled in the art
The replacement and equivalent various parts for applying example are well known.It should be appreciated by the person skilled in the art that the present invention is not being departed from
Spirit or essential characteristics in the case of, the present invention can in other forms, structure, arrangement, ratio and with other components,
Material and component are realized.In the case where not departing from scope and spirit of the present invention, can to embodiments disclosed herein into
The other deformations of row and change.
Claims (7)
- A kind of 1. method for the data acquisition of multipad, conversion and loading, which is characterized in that for by interface number According to the data in source, load to data warehouse, include the following steps：a）Data pick-up,b）Data transmission and transfer,c）Data receiver and parsing,d）Data conversion,e）Data load.
- 2. as described in claim 1 for the method for the data acquisition of multipad, conversion and loading, feature exists In：The step a data pick-ups, including standard control data capture, GDI data captures, image data capture, event analysis, Condition judgment；The step d data conversions, including form amendment, field fractionation, field merging, calculated value, dictionary lookup, list Position conversion, date-time are converted, summarized, leading outer bond reconstruction；The step e data load, and increment is reprinted, destructiveness merges, builds If property merges, condition merges.
- 3. as described in claim 1 for the method for the data acquisition of multipad, conversion and loading, feature exists In：The step a data pick-ups for the data that desktop application is shown, inject multipad by developing client, The data captured are needed according to acquisition is locally configured；The client, support to specify process name or other applications feature into The monitoring of row multipad；The client injects multipad, and the function of being hooked using the message of Windows will The dynamic base injection destination application specified, loads the configuration specified, for determining to obtain the side of destination application data Formula and time；The time for obtaining destination application data is realized by the way of windows messaging interception；It is described The data for obtaining destination application are realized by the way of WindowsAPI functions or APIHook.
- 4. as described in claim 1 for the method for the data acquisition of multipad, conversion and loading, feature exists In：The step b）Data transmission and transfer, it is characterised in that：The data transmission, it is contemplated that the fire wall of most of hospitals and port restriction strategy are realized using WebService；The data relay, it is contemplated that most of hospitals run on Intranet, using in the server disposition of the double network segments of intranet and extranet Turn method of service realization, including WebService transfers service and FTP transfer services；The WebService transfers service carries out data forwarding using the ServiceHost separate configurations service of WCF；The FTP transfers service carries out file forwarding using ftp server plus FileWatcher services.
- 5. as described in claim 1 for the method for the data acquisition of multipad, conversion and loading, feature exists In：The step c）Data receiver and parsing, it is characterised in that：WebService uses the main services of ServiceHost of WCF For receiving and parsing through data；FTP carries out data receiver using ftp server, and FileWatcher is main to be serviced to parse number According to.
- 6. as described in claim 1 for the method for the data acquisition of multipad, conversion and loading, feature exists In：The step d）Data conversion, step e）Data are loaded to be realized by developing data processing module.
- 7. as claimed in claim 6 for the method for the data acquisition of multipad, conversion and loading, feature exists In：The data processing module is embedded in main service, to support the conversion of data and loading；The data processing module uses the preconfigured processing stream of Windows Workflow Foundation engine loads Journey；The data processing module is supported to load different flows using the process identification in metadata information for handling difference The data of type；The data processing module support to access data in itself, metadata, predefined database, file etc.；The data processing module supports branch process, delay disposal, abnormality processing, issued transaction；The data processing module can extend the function of its module by self-defined Activity.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610980683.0A CN108062320A (en) | 2016-11-08 | 2016-11-08 | A kind of method for the data acquisition of multipad, conversion and loading |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610980683.0A CN108062320A (en) | 2016-11-08 | 2016-11-08 | A kind of method for the data acquisition of multipad, conversion and loading |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108062320A true CN108062320A (en) | 2018-05-22 |
Family
ID=62137646
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610980683.0A Pending CN108062320A (en) | 2016-11-08 | 2016-11-08 | A kind of method for the data acquisition of multipad, conversion and loading |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108062320A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110659323A (en) * | 2019-09-26 | 2020-01-07 | 卓尔购信息科技(武汉)有限公司 | Real-time and off-line big data processing system, method, storage medium and terminal |
| CN112953988A (en) * | 2019-12-10 | 2021-06-11 | 浙江宇视科技有限公司 | Data docking method and device based on FTP |
| CN113220757A (en) * | 2021-04-24 | 2021-08-06 | 上海钢银科技发展有限公司 | Real-time warehouse automatic ETL method, system, equipment and computer storage medium |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101075304A (en) * | 2006-05-18 | 2007-11-21 | 河北全通通信有限公司 | Method for constructing decision supporting system of telecommunication industry based on database |
| US20080140692A1 (en) * | 2006-12-11 | 2008-06-12 | Scott Gehring | System and method of data movement between a data source and a destination |
| CN101710280A (en) * | 2009-11-05 | 2010-05-19 | 金蝶软件（中国）有限公司 | Packaging method and devices of BI product members |
| CN101969475A (en) * | 2010-11-15 | 2011-02-09 | 张军 | Business data controllable distribution and fusion application system based on cloud computing |
| CN102752372A (en) * | 2012-06-18 | 2012-10-24 | 天津神舟通用数据技术有限公司 | File based database synchronization method |
| CN103745424A (en) * | 2014-02-14 | 2014-04-23 | 上海市东方医院（同济大学附属东方医院） | Hospital comprehensive performance information processing system and method |
| CN104735104A (en) * | 2013-12-18 | 2015-06-24 | 顺软科技发展（大连）有限公司 | WeChat intelligent public platform system |
| CN105005618A (en) * | 2015-07-21 | 2015-10-28 | 杭州合众数据技术有限公司 | Data synchronization method and system among heterogeneous databases |
| CN105389402A (en) * | 2015-12-29 | 2016-03-09 | 曙光信息产业(北京)有限公司 | Big-data-oriented ETL (Extraction-Transformation-Loading) method and device |
| CN105760174A (en) * | 2016-03-09 | 2016-07-13 | 浪潮软件集团有限公司 | Suggested data extraction method based on Windows timed task |
| CN105955765A (en) * | 2016-04-22 | 2016-09-21 | 北京小米移动软件有限公司 | Method and device for application preloading |
-
2016
- 2016-11-08 CN CN201610980683.0A patent/CN108062320A/en active Pending
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101075304A (en) * | 2006-05-18 | 2007-11-21 | 河北全通通信有限公司 | Method for constructing decision supporting system of telecommunication industry based on database |
| US20080140692A1 (en) * | 2006-12-11 | 2008-06-12 | Scott Gehring | System and method of data movement between a data source and a destination |
| CN101710280A (en) * | 2009-11-05 | 2010-05-19 | 金蝶软件（中国）有限公司 | Packaging method and devices of BI product members |
| CN101969475A (en) * | 2010-11-15 | 2011-02-09 | 张军 | Business data controllable distribution and fusion application system based on cloud computing |
| CN102752372A (en) * | 2012-06-18 | 2012-10-24 | 天津神舟通用数据技术有限公司 | File based database synchronization method |
| CN104735104A (en) * | 2013-12-18 | 2015-06-24 | 顺软科技发展（大连）有限公司 | WeChat intelligent public platform system |
| CN103745424A (en) * | 2014-02-14 | 2014-04-23 | 上海市东方医院（同济大学附属东方医院） | Hospital comprehensive performance information processing system and method |
| CN105005618A (en) * | 2015-07-21 | 2015-10-28 | 杭州合众数据技术有限公司 | Data synchronization method and system among heterogeneous databases |
| CN105389402A (en) * | 2015-12-29 | 2016-03-09 | 曙光信息产业(北京)有限公司 | Big-data-oriented ETL (Extraction-Transformation-Loading) method and device |
| CN105760174A (en) * | 2016-03-09 | 2016-07-13 | 浪潮软件集团有限公司 | Suggested data extraction method based on Windows timed task |
| CN105955765A (en) * | 2016-04-22 | 2016-09-21 | 北京小米移动软件有限公司 | Method and device for application preloading |
Non-Patent Citations (1)
| Title |
|---|
| 吴亚杰: "《数字化医院》", 31 August 2015, 河南科学技术出版社 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110659323A (en) * | 2019-09-26 | 2020-01-07 | 卓尔购信息科技(武汉)有限公司 | Real-time and off-line big data processing system, method, storage medium and terminal |
| CN112953988A (en) * | 2019-12-10 | 2021-06-11 | 浙江宇视科技有限公司 | Data docking method and device based on FTP |
| CN112953988B (en) * | 2019-12-10 | 2023-03-24 | 浙江宇视科技有限公司 | Data docking method and device based on FTP |
| CN113220757A (en) * | 2021-04-24 | 2021-08-06 | 上海钢银科技发展有限公司 | Real-time warehouse automatic ETL method, system, equipment and computer storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Liu et al. | Caesar: cross-camera complex activity recognition | |
| US12368822B2 (en) | Scalable visual computing system | |
| CN108062320A (en) | A kind of method for the data acquisition of multipad, conversion and loading | |
| CN105631026A (en) | Security data analysis system | |
| US8656398B2 (en) | Synchronization of workflows in a video file workflow system | |
| US20210144238A1 (en) | Harmonized data for engineering simulation | |
| CN103096029B (en) | Multi-camera selective video method for supervising | |
| US12340493B2 (en) | Centralized analytics of multiple visual inspection appliances | |
| CN107193668B (en) | Calling method and system of interface server | |
| US12069318B2 (en) | Unified routing and caching logic for applications utilizing a content delivery network | |
| CN104065631B (en) | Local area network PACS is serviced to WADO service system and to its access method | |
| CN101615270A (en) | A kind of based on the workflow analysis of B/S pattern and the system and method that presents | |
| US10185307B2 (en) | Method of and system for controlling manufacturing processes in discrete production lines | |
| CN111611523B (en) | Resource management system, resource management method, device and storage medium | |
| JP2004511859A (en) | Hospital information management method and system | |
| CN112800064B (en) | Real-time big data application development method and system based on Confluent community open source version | |
| CN105915377A (en) | Flume and Spark streaming integration method and system | |
| CN101645072A (en) | Changed data extracting method realized by being based on Oracle CDC technique | |
| CN102970330A (en) | Browser interaction achievement method and device | |
| Detmold et al. | Middleware for distributed video surveillance | |
| CN104346678A (en) | High-efficiency legal service method capable of integrating service receiving with service management and high-efficiency legal service system | |
| Tran et al. | Event actors based approach for supporting analysis and verification of event-driven architectures | |
| CN114338339B (en) | Digital power plant integration system and integration method | |
| CN117640134A (en) | System and method for expanding three-party authentication protocol based on industrial Internet platform | |
| Barth et al. | A Modernized Architecture for the Post Mortem System at CERN |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20180522 |
|
| WD01 | Invention patent application deemed withdrawn after publication |