CN100396036C - Computer system with graphical user interface and virtual remote control system and method - Google Patents

Abstract

The present invention uses a pure software manner to execute a UPnP (Universal Plug and Play) 'virtual remote controller device' on a home personal computer, so that a Tablet PC (Tablet PC) and a Personal Digital Assistant (PDA) connected together through a wireless or wired network can control the functions provided by the home personal computer by operating the 'virtual remote controller device'.

Description

Computer system with graphic user interface and virtual remote control system and method

technical field

The present invention relates to a virtual remote controller device, in particular to a virtual remote controller device using universal plug and play technology under wireless area network.

Background technique

UPnP applies many existing standard communication protocols. These standardized communication protocols can implement interoperability between manufacturers' implementations. And we found that everywhere in the Internet and local area networks, these communication protocols used to implement UPnP are used, so the knowledge of how to implement and deploy solutions based on these communication protocols is also quite common. Since they all use the same communication protocol, the UPnP device can operate in the existing network environment without taking other actions.

UPnP supports communication between devices and control points, and the basic network connection and necessary addressing are provided by network media, TCP/IP communication protocol suite and HTTP. On top of these open standard Internet communication protocols, UPnP defines a set of HTTP servers that specifically handle exploration, description, control, event, and presentation operations.

Referring to FIG. 1 , FIG. 1 is a schematic diagram of a known UPnP communication protocol stack. UPnP manufacturers, "UPnP Discussion Working Committee" and "UPnP Device Architecture" documents jointly define the highest-level communication protocol used to implement UPnP. The working committee defines device-specific specifications for VCRs, HVAC systems, dishwashers, and other home appliances based on the device architecture, allowing "UPnP device manufacturers" to add their device-specific data, such as model names and URLs.

The TCP/IP network communication protocol stack is the foundation upon which other UPnP communication protocols are based. With this popular standard TCP/IP communication protocol suite, UPnP applies the functions of the communication protocol, spans different physical media, and implements the interoperability between multiple vendors. UPnP devices can use many communication protocols in the TCP/IP stack, including TCP, UDP, IGMP, ARP, and IP, as well as TCP/IP services such as DHCP and DNS. TCP/IP provides network connection between UPnP devices through the basic communication protocol stack. And HTTP, which is critical to the success or failure of the Internet, is also an important part of UPnP. Every level of UPnP is based on HTTP or its variants. In U.S. Patents US2002/0027569A1, US2001/0053274A1 and US2002/0180579A1, there are user operation interfaces developed according to this structure, but the main points are still limited to using the screen connected to the host computer as the operation interface, and by this The generated control interface also does not conform to the usage habits of ordinary people who use the remote controller alone.

In addition, the most direct way for general manufacturers is to specially produce an infrared transmitter with the required buttons (such as a remote control for general household appliances) according to the functions to be provided, and then install an infrared receiver on the personal computer. , through a certain hardware interface, such as USB (Universal Serial Bus), PS/2 (Personal System/2), ISA (Industry Standard Architecture Bus), PCI (Peripheral Component Interconnect Peripheral Component Connection Interface), etc. , to import the signal into the personal computer, and then use the software to connect the actions of these buttons with the operation of the application program. However, although this method allows the user to have one more choice in the use of the original remote controller, it does not have much benefit in terms of cost considerations or convenience of use.

Contents of the invention

Therefore, the present invention aims at the above-mentioned subject, and implements a UPnP "virtual remote control device" on a home personal computer by means of pure software through the operation of a personal digital assistant, so that devices connected together through a wireless or wired network can all By operating and using this "virtual remote control device", the functions provided by the home personal computer can be controlled.

Refer to Figure 2. FIG. 2 is a schematic diagram of the UPnP communication protocol stack of the present invention.

HTTPU (and HTTPMU) is a variant of HTTP (Hyper Text Transfer Protocol, hypertext transfer mode), whose purpose is to transmit messages through UDP/IP (instead of TCP/IP). These communication protocols are all used by SSDP, which will be described later. The basic message format used by these communication protocols is closely related to the format of HTTP. This format must be used when performing multicast communication and when message delivery does not require additional costs to enhance reliability.

SSDP (Simple Service Discovery Protocol, Simplified Service Discovery Protocol), as the name suggests, mainly defines how to discover network services on the network. SSDP defines methods based on HTTPU and HTTPMU, allowing control points to find resources of interest on the network, and allowing devices to announce their availability on the network. SSDP is a way to define search requirements and presence declarations, saving the cost of using only one of the mechanisms. Therefore, each control point on the network can not only reduce the amount of network data, but also ensure complete network status information. Both control points and devices use SSDP. When a UPnP control point starts up, it sends an SSDP discovery request (via HTTPMU) to discover devices and services available on the network. The control point can narrow the search to only a certain type of device (such as a VCR), a certain type of service (such as a device that performs a clock service), or even a certain type of device. The UPnP device will listen to the multicast port. Whenever a search request is received, the device checks the search criteria to see if they match. If a match is found, a unicast SSDP (over HTTPU) response is sent to the control point. Similarly, after the device is plugged into the network, it will also send multiple SSDP existence announcements to inform everyone which services it supports. Both the Announcement of Presence and Unicast Device Response messages contain a pointer to the location of the device description file, which contains information about the set of content and services supported by the device.

GENA (Generic Event Notification Architecture, General Event Notification Architecture) mainly uses HTTP over TCP/IP and multicast UDP to send and receive notifications. GENA also defines the notion of subscribers and publishers for notifications that initiate events. UPnP also uses the GENA format to create a presence announcement, transmit it with the Simple Service Discovery Protocol (SSDP), and notify the service status change for UPnP event operations. A control point that intends to receive event notifications subscribes to the event source by sending a request, which includes the interested service, the destination for sending the event, and the subscription time for the event notification. Subscriptions must be renewed regularly in order to continue to receive notifications and can also be canceled using GENA.

XML (Extensible Markup Language, Extensible Markup Language) is a common format for network structured data. XML, on the other hand, can put almost any kind of structured data in a text document. XML looks a lot like HTML because it also uses tags and attributes. In fact, the two are quite different, because these tags and attributes are not defined in terms of their meaning as a whole, but are interpreted according to their usage in context. These XML capabilities enable the development of schemas for various document types. The use of XML as a schema language is defined by the W3C. The main part of UPnP used by XML device and service descriptions, control messages and event operations.

According to the described stack, a framework or template is first defined by the UPnP Device Architecture to create device and service descriptions for any device or service type. The various devices and service types are then standardized by individual working committees, creating a template for each device or service type individually. Finally, the manufacturer fills in the template with specific information about the device or service, such as device name, model, manufacturer name, and URL leading to the service description. These data will be encapsulated in the UPnP-specific communication protocol and defined in the "UPnP Device Framework" document (such as the XML device description template). Necessary UPnP-specific information is inserted in all messages, formatted using SSDP, GENA, and SOAP, and delivered via HTTP, HTTPU, or HTTPMU.

The present invention provides a computer system with a graphic user interface, which is a system used under the UPnP framework in a wireless network, including: a host, a built-in media center that supports UPnP control terminal functions, and a virtual remote control device host The software module, the media center is used to emulate a current window operating system and an input mode acceptable to the media center, and is connected to the UPnP virtual remote control device host software module with a signal to simulate the input mode of the user Operating system; an operating device, at least including a user interface, the system of the operating device has a built-in virtual remote driver, and the virtual remote driver passes the clicked signal received on the user interface through The wireless network transmits a control message corresponding to the signal to the host software module of the virtual remote control device; an audio and video device at least includes a receiver and an audio and video output device, and the receiver receives information from the virtual remote control in the wireless network. The input signal of the device host software module, the audio and video output device outputs the input signal, wherein the virtual remote control device host software module is constructed on the built-in UPnP protocol stack provided by the window operating system, and the protocol includes IP , TCP, UDP, HTTP, HTTPU, HTTPMU, SSDP, and GENA.

The present invention also provides a method for controlling a home audio and video device with a virtual remote control under the UPnP framework, comprising the steps of: using an operating device to connect to a UPnP virtual remote control device on a wireless network, and obtaining services through the virtual remote control device; The virtual remote control driver program built in the system of the operating device displays the screen of the virtual remote control on the user operation interface of the operating device, and the screen has a plurality of virtual keys; The virtual button receives a clicked signal, and the virtual remote control driver sends a control message corresponding to the signal to the UPnP virtual remote control device host software module built in the personal computer through the wireless network; the UPnP virtual remote control device host The software module receives the control message, so that the virtual remote driver simulates an input program corresponding to the control message; the UPnP virtual remote control device host software module transmits the input program to the operating system and application software; the operation The system and application software execute the input program; the UPnP virtual remote control device host software module outputs the result of executing the input program to an audio and video device through a wireless network, wherein the virtual remote control device host software module is constructed on a window operating system The provided built-in UPnP protocol stack includes IP, TCP, UDP, HTTP, HTTPU, HTTPMU, SSDP, and GENA.

The present invention also provides a virtual remote control system under the UPnP architecture, including: a host, a built-in media center supporting the UPnP control terminal function, and a virtual remote control device host software module, and the media center is used for simulation generation A current window operating system and the input modes acceptable to the media center are connected to the host software module of the UPnP virtual remote control device with signals to simulate the operating system that generates the user's input modes; Contains a user operation interface, the virtual remote control driver is built in the system of the operating device, and the virtual remote control driver transmits the clicked signal received on the user operation interface through the wireless network and sends the signal corresponding to the signal Control messages to the host software module of the virtual remote control device; an audio-visual adapter (Set Top Box), used to receive input signals from the host software module of the virtual remote control device in the wireless network; an audio and video device, through a transmission line It is connected to the audio-video adapter, receives the signal from the audio-video adapter, and outputs the signal.

The present invention also provides a method for controlling a home audio and video device with a virtual remote control under the UPnP framework, comprising the steps of: using an operating device to connect to a UPnP virtual remote control device on a wireless network, and obtaining services through the virtual remote control device; The built-in virtual remote control driver program of the system of the operating device displays the screen of the virtual remote control on the user operation interface of the operation device, and the screen has a plurality of virtual keys; The virtual button receives a clicked signal, and the virtual remote control driver sends a control message corresponding to the signal to the UPnP virtual remote control device host software module built in the personal computer through the wireless network; the UPnP virtual remote control device The host software module receives the control message, and makes the virtual remote driver simulate an input program corresponding to the control message; the host software module of the UPnP virtual remote control device transmits the input program to the operating system and application software; The operating system and application software execute the input program; the UPnP virtual remote control device host software module outputs the result of executing the input program to an audio-visual adapter (Set Top Box) through the wireless network; the result of the input program is passed through The transmission line is transmitted from the audio-video adapter to a TV for output.

Description of drawings

FIG. 1 is a schematic diagram of a known UPnP communication protocol stack.

FIG. 2 is a schematic diagram of the UPnP communication protocol stack of the present invention.

Fig. 3 is a diagram of the usage framework of the present invention.

FIG. 4 is a diagram showing a processing architecture under the UPnP architecture.

FIG. 5 is a diagram showing an operation screen displayed on the screen of the personal digital assistant.

Explanation of reference numbers

100 Wi-Fi

101 Media Center Host

102 UPnP client

103 Audio-visual installations

110 Media Center

120 Virtual remote control device under UPnP architecture

130 UPnP stack

140 hid.dll

210 HIDCLASS.SYS

220 virtual remote control device host software module

230 network drives

240 virtual remote control HID driver module

310 IR Receiver

320 wireless network receiver

400 personal digital assistant hosts

401 Personal Digital Assistant Screen

402 virtual remote

410 start button

420 play key

Detailed ways

Referring to FIG. 3 , FIG. 3 is a diagram of the usage architecture of the present invention. Under this architecture, a media center that supports UPnP console functions is built in the computer host, which is used to simulate a current window operating system and the input modes that the media center can accept, and is connected to the UPnP virtual remote control device host with a signal The software module emulates the operating system in the user's input mode. In this way, we can use an operating device, such as a personal digital assistant, or a notebook computer, or a tablet computer, and build a virtual remote driver in the system using the operating device. When the user clicks on the screen in the operating device, then The driving mode of the virtual remote control will transmit the received clicked signal to the host software module of the virtual remote control device through the wireless network to send a control message corresponding to the signal. When the host software module of the virtual remote control device receives the control message, it makes the virtual remote control driver simulate an input program corresponding to the control message and transmits the input program to the operating system and application software, and the operating system and application software to execute the input program;

The host software module of the virtual remote control device outputs the result of executing the input program to an audio and video device through the wireless network.

An audio and video device at least includes a receiver and an audio and video output device, the receiver receives the input signal from the host module of the virtual remote control device in the wireless network, and the audio and video output device outputs the input signal. The receiver can also be independent from the audio-video device in the form of an audio-video adapter (Set-TopBox), and output the received audio-video signal to the audio-video device through a transmission line.

Referring to FIG. 4, FIG. 4 shows a processing architecture diagram of the present invention under the UPnP architecture.

In the present invention, two main software modules are added in the Windows OS operating system to complete the overall operating program.

One is the host software module (220) of the virtual remote control device.

In this part, the present invention generates a virtual remote control device (a software-based remote control device) according to the UPnP standard. This system is constructed in the built-in UPnP protocol stack (built-in protocols stacks) (130) provided by the window operating system, and the window operating system includes IP, TCP, UDP, HTTP, HTTPU, HTTPMU shown in Figure 2 , SSDP, and GENA. And it will follow the basic framework of UPnP to address (Addressing), search (Discovery), control (Control), and event (Eventing) and so on. In the software-controlled virtual remote control provided by the present invention, the button services are used to allow the user to control the object to be remotely controlled at the UPnP control terminal through the network. After the host module (220) of the virtual host remote control device receives the user's command through the network at the remote terminal, it will carry out signal transmission with the following modules, and an input event will be generated under the window operating system through the application software .

Second, it is a virtual remote control HID (Human Interface Device) driver module (240).

In the window operating system, all input devices (such as keyboards, mice, IR receivers, etc.) enter the kernel program mode (KernelMode) after passing through the HIDCLASS.SYS (210) system module. Generally, there are standard network drivers on the market. (230) Connecting the module and the wireless network adapter (320) by signal. If the standard driver interface is used, the application software on the window operating system can generate a device-independent screen corresponding to the user's input, and the corresponding screen can present different patterns according to different hardware devices. Therefore, by writing a specific HID driver module (just the virtual remote control HID driver module) (240), it is possible to emulate what an existing window operating system and existing application software (including Media Center PC Shell) can Accepted input events (input events) to accommodate the requirements of the standard interface HIDCLASS.SYS (210). On the other hand, it can also be signal-connected to the host software module (220) of the UPnP virtual remote control device to determine which operating system to emulate in the user's input mode.

As for the extended use mode and extended concept of the remote control keyboard, if the user wants to replace the traditional remote control with a virtual remote control keyboard, and does not want actual hardware buttons to appear on the virtual remote control keyboard, then the present invention provides The fully defined 101/102 virtual keyboard under the UPnP framework can achieve the same purpose of use. The UPnP control terminal finds a full-featured virtual keyboard device to use through the discovery of the UPnP device and UPnP service on the control side. As long as the user presses any key displayed in the virtual keyboard device, the same functional effect as that of pressing the physical keyboard can be produced. Once the command from the virtual keyboard device is received, the host software module of the virtual remote control device can communicate with the virtual remote control HID driver (240) through the network.

With regard to the virtual remote control HID driver module (240), a full-featured keyboard will be virtualized for HIDCLASS.SYS (210), just as the Standard LocalUSB Keyboard Driver (Standard LocalUSB Keyboard Driver) is carried out. The difference is that the input signal through HIDCLASS.SYS (210) will be received by the UPnP virtual remote control device host software module (220) instead of the physical keyboard. From the perspective of window operation interface and application software, this is a device-independent screen corresponding to user input, regardless of where the user's instruction comes from, whether it is from the local area or not. It doesn't matter whether it's a physical keyboard, or a software module from either of those over the web.

UPnP is an industry standard. After executing this software on our home personal computer, use the Advertisement (notification) function defined in UPnP to display this "virtual remote control device" at any time in the WLAN (Wireless LocalArea Network wireless area network) "The presence. Any device that supports UPnP Control Point (UPnP Control Point), such as PDA (Personal Digital Assistant), Tablet PC (tablet computer), or even other desktop computers, can automatically Discover this "virtual remote control device" clearly, and can further use it, that is, operate the functions that can be remotely controlled on the home personal computer.

In the embodiment of the present invention, the virtual remote control device under the UPnP framework is completed with a personal computer and a personal digital assistant having a media center (Media Center).

In the present invention, the two modules of the virtual remote control device host software module (220) and the virtual remote control HID driver module (240) are used on the personal computer with the media center (110), and the UPnP control point (UPnP Control Point) software is built on the virtual remote control device (120) under the UPnP architecture, that is, the personal digital assistant, and then the personal computer with the media center (110) and the personal digital assistant (120) are connected in the home wireless network ( such as 802.11a/b/g). When starting the personal computer and executing its corresponding virtual remote control device host software module (220) and virtual remote control HID driver module (240), then by the Advertisement (notification) function defined in UPnP, UPnP can be detected A virtual remote control device under the framework.

Then, start the personal digital assistant to make it into the range covered by the home wireless network, then the personal digital assistant will start searching for available UPnP devices in the network under the UPnP protocol, so as to find the virtual remote control device through the network , and then further display the service options obtained through the virtual remote control device on the screen of the personal digital assistant and display the user interface (sometimes in HTML format), so as to transmit the user's instructions using the device into the IR receiver (310) in the home wireless network environment.

Referring to FIG. 5, FIG. 5 shows an operation screen diagram displayed on the screen (401) of the PDA body (400).

In an embodiment, the screen of the virtual remote controller (402) graphically displayed is presented on the screen (401) of the personal digital assistant in the form of a plurality of virtual keys. With the virtual remote control (402) with full-function buttons presented on the screen of the personal digital assistant (401), the user can hold the main body of the personal digital assistant (400) in one hand, and directly click on the personal digital assistant with a stylus in the other hand Buttons of the virtual remote controller (402) on the screen (401). When the user clicks the button on the personal digital assistant screen (401), the built-in control point software of the personal digital assistant will send a "control" message to the UPnP virtual remote control built in the personal computer through the wireless network In the device host software module (220). When the host software module of the UPnP virtual remote control device receives an instruction from the user.

For example, when the user clicks a start button (410), the UPnP virtual remote control device host software module (220) will send a message to cause the virtual remote control HID driver (240) to emulate an input program for a keyboard click action, and send out To the operating system and application software to execute accordingly. Thereby, after the application software receives the input instruction of the simulated start key (410) keyboard click action, it simultaneously executes the action of turning on the power supply.

Another example is when the user clicks a PLAY (play) button (420), the UPnP virtual remote control device host software module will send a message to cause the virtual remote control HID driver (240) to emulate an input program for a keyboard click action, and transmit to the operating system and application software for execution. In this way, when the application software receives the input instruction of the simulated PLAY (play) key (420) keyboard click action, the operation of playing is performed simultaneously.

In this control mode, it is not necessary to confirm where the user's command comes from. In other words, the user's command can come from the infrared remote control of the home entity, or it can be as in this embodiment, click on the screen of the personal digital assistant (401) on the virtual remote control to perform functions such as playback. In this way, the user can make the personal digital assistant (400) realize the purpose of a virtual wireless remote control through the functions of the wireless network and the personal computer.

This idea can be further expanded into a "virtual wireless keyboard", or other general PC input devices, to operate all other functions on a home PC.

Claims (8)

1. one kind has the computer system that figure uses the interface, is in wireless network, and the system that the UPnP framework uses down includes:

One main frame, built-in one supports the media center of UPnP control end function, an and virtual remote control device host software module, described media center be used for emulation produce an existing Windows and this media center the input pattern that can accept, and being connected in described UPnP virtual remote control device host software module with signal, emulation produces the operating system of user's input pattern;

One uses operating means, at least comprise user's operation-interface, this uses the built-in virtual remote control driver of system of operating means, and this virtual remote control driver spreads out of control message corresponding to this signal to described virtual remote control device host software module with received clicked signal on described user's operation-interface by wireless network;

One audio-video apparatus comprises a receiver and an audio frequency and video output device at least, and from the input signal of described virtual remote control device host software module, described audio frequency and video output device is exported described input signal in this receiver reception wireless network,

Wherein, described virtual remote control device host software modular construction is in the built-in UPnP protocol stack that Windows provided, and described protocol package contains IP, TCP, UDP, HTTP, HTTPU, HTTPMU, SSDP and GENA.

2. computer system as claimed in claim 1, wherein, described use operating means is a personal digital assistant or a mobile computer or a flat computer.

One kind under the UPnP framework with the method for virtual remote control device remote control home audio video-unit, comprise step:

Use operating means on wireless network, to connect UPnP virtual remote control device, and obtain service by this virtual remote control device;

The built-in virtual remote control driver of the system of described use operating means shows the picture of virtual remote control device on user's operation-interface of described use operating means, this picture has a plurality of virtual keys;

Arbitrary described virtual key on described user's operation-interface receives clicked signal, described virtual remote control driver by wireless network spread out of corresponding to the control message of this signal in be built in the UPnP virtual remote control device host software module of personal computer;

Described UPnP virtual remote control device host software module receives described control message, makes described virtual remote control driver simulate the input program of a corresponding described control message;

Described UPnP virtual remote control device host software module reaches operating system and application software with this input program;

Described operating system and application software are carried out this input program;

The result that described UPnP virtual remote control device host software module will be carried out this input program exports an audio-video apparatus to by wireless network,

Wherein, described virtual remote control device host software modular construction is in the built-in UPnP protocol stack that Windows provided, and described protocol package contains IP, TCP, UDP, HTTP, HTTPU, HTTPMU, SSDP and GENA.

4. remote control thereof as claimed in claim 3, wherein, described use operating means is a personal digital assistant or a mobile computer or a flat computer.

5. virtual remote control system under the UPnP framework includes:

One main frame, built-in one supports the media center of UPnP control end function, an and virtual remote control device host software module, described media center be used for emulation produce an existing Windows and this media center the input pattern that can accept, and be connected in described UPnP virtual remote control device host software module with signal, be used for emulation to produce the operating system of user's input pattern;

One uses operating means, at least comprise user's operation-interface, this uses the built-in virtual remote control driver of system of operating means, and this virtual remote control driver spreads out of control message corresponding to this signal to described virtual remote control device host software module with received clicked signal on described user's operation-interface by wireless network;

One audio-visual adapter (Set Top Box) is used for receiving in the wireless network input signal from described virtual remote control device host software module;

One audio-video apparatus is connected in described audio-visual adapter by transmission line, receives the signal from described audio-visual adapter, and exports this signal.

6. virtual remote control as claimed in claim 5 system, wherein, described use operating means is a personal digital assistant or a mobile computer or a flat computer.

One kind under the UPnP framework with the method for virtual remote control device remote control home audio video-unit, comprise step:

Use operating means on wireless network, to connect UPnP virtual remote control device, and obtain service by this virtual remote control device;

The built-in virtual remote control driver of the system of described use operating means shows the picture of virtual remote control device on user's operation-interface of described use operating means, this picture has most virtual keys;

Arbitrary described virtual key on described user's operation-interface receives clicked signal, described virtual remote control driver spreads out of corresponding this signal of son by wireless network control message in be built in the UPnP virtual remote control device host software module of personal computer;

Described UPnP virtual remote control device host software module receives described control message, makes described virtual remote control driver simulate the input program of a corresponding described control message;

Described UPnP virtual remote control device host software module reaches operating system and application software with this input program;

Described operating system and application software are carried out this input program;

The result that described UPnP virtual remote control device host software module will be carried out this input program exports an audio-visual adapter (Set Top Box) to by wireless network;

The result of described input program reaches a television set by transmission line by described audio-visual adapter and exports.

8. the method with virtual remote control device remote control home audio video-unit as claimed in claim 7, wherein, described use operating means is a personal digital assistant or a mobile computer or a flat computer.

Images