JP5471667B2 - Client and image display system - Google Patents

Description

  The present invention relates to a client and an image display system, and more particularly to a technique that appropriately copes with a decrease in operability associated with a change in display resolution in a client.

  Among documents known before the application of the present invention, a document describing a technique related to the present invention is, for example, Patent Document 1.

  Patent Document 1 downloads a screen transmission application and a mobile phone terminal application program from a management server to each of a personal computer and a mobile phone terminal, and displays the full screen of the personal computer on the display device of the mobile phone terminal. (For example, paragraph 0033). Here, since the display screen of the personal computer and the display screen of the mobile phone terminal are often different in size, in the technology disclosed in Patent Document 1, the screen of the mobile phone terminal is operated by a center server that relays screen information. The aspect ratio and size of the transmitted image are compressed according to the size.

JP 2007-251630 A

  Recently, a remote control technique of a computer called “remote desktop” is known. This is because, for example, a personal computer (hereinafter referred to as “PC”) at home and a server and a client that are communicably connected via a network, such as a mobile PC, can output an image output from the server, particularly a desktop image. It is sent to the client and displayed on the display device of the client. At the same time, an operation performed on the displayed desktop image (for example, pointing or character input by a pointing device) is uploaded to the server, and the operation is reflected on the server.

  As a client in a remote desktop, in addition to a mobile PC, it is becoming realistic to use a mobile phone terminal against the background of performance improvement and miniaturization of a processing device. However, the size of the display screen is usually different between the home PC and the mobile phone terminal, and the aspect ratio may be different. Then, on the home PC on the desktop screen sending side, the desktop screen is resized (mainly, the size to be reduced) or partly cut out, and the cut-out desktop image is sent to the mobile phone terminal. It will be necessary.

  On the other hand, some mobile phone terminals have a gravity sensor, an acceleration sensor, etc., and detect the tilt state of the mobile phone terminal, and change the screen orientation displayed on the display device according to the detected tilt state. There is something to prepare. In other words, it is a function of switching the orientation of the screen by detecting that the user is viewing the screen vertically or horizontally. With this function, the user can switch to a vertically long screen for Web browsing and a landscape screen when browsing a large amount of horizontal content such as video and photo browsing.

  Consider such a cellular phone terminal as a client of the above-mentioned remote desktop and use a function of switching the screen orientation when displaying the above-described desktop screen. Then, the resolution is changed by switching the screen orientation. This resolution change is performed by changing the screen resolution displayed on the server and sending the changed screen image to the client. The problem is that it takes time to complete the process. After the resolution is changed, the operability is lowered due to the time required for display.

  In addition, here, it takes time to display on the client when the screen resolution is changed in consideration of a mobile phone terminal having a function of detecting the tilt state of the terminal and switching the screen orientation, This pointed out the problems of the prior art that the operability was lowered by that. However, the opportunity for changing the resolution of the screen is not necessarily limited to the switching of the screen orientation (rotation of the screen). For example, a manual resolution change may be considered. For example, the above-mentioned problem exists even when a notebook PC, PDA, or the like is employed as a client in addition to a mobile phone terminal.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a client and an image display system in which a decrease in operability associated with a change in display resolution is improved.

  To achieve the above object, according to a first aspect of the present invention, there is provided a client connected to a server via a network, a display unit for displaying a desktop screen sent from the server, and a display on the display unit Remote operation control means for controlling the display on the display means to be displayed at the resolution before the change until the screen with the changed resolution is sent when the resolution of the image to be changed is changed. The present invention provides a client characterized by comprising:

  According to a second aspect of the present invention, there is provided an image display system in which a server and a client are communicably connected via a network, wherein the client displays a desktop screen sent from the server; When the resolution of the image displayed on the display means is changed, a resolution change notification including the changed resolution information is sent to the server, and until the screen with the changed resolution is sent, Remote operation control means for controlling the display on the display means to be displayed at the resolution before the change, and when the server receives the resolution change notification, the size and the size adapted to the resolution information after the change A desktop screen generating means for generating a desktop screen for orientation and sending the desktop screen to the client; There is provided a display system.

  ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to provide the client and image display system with which the fall of operativity accompanying the change of display resolution was improved.

It is a figure which shows the whole structure of one Embodiment which implemented this invention suitably. It is a conceptual diagram for demonstrating the change of the resolution in this embodiment. It is a figure which shows the example of a transition of the screen displayed on the display part of FIG.

FIG. 1 shows an overall configuration of an embodiment in which the present invention is suitably implemented.
In the present embodiment, a server and a client that are communicably connected via a network are used to send an image output from the server, particularly a desktop image, to the client for display on the client display device. An embodiment of a so-called “remote desktop” technology in which operations performed on the displayed desktop image (for example, pointing by a pointing device, character input, etc.) are uploaded to the server and the operations are reflected on the server It is.
Although not limited, in this embodiment, the home PC 2 is used as the server device, and the mobile phone terminal 1 is used as the client device. In addition, an electronic device such as a mobile PC or a PDA may be used as the client device.

First, the configuration of the home PC 2 will be described.
The home PC 2 is installed with an operation system 201 that is basic software that implements predetermined information processing such as computation using the hardware resources of the home PC 2. Similarly, the home PC 2 cooperates with hardware and software. A functional block called a desktop screen generation unit 202 is configured by the operation.

  The operation system 201 includes a GUI (graphical user interface), and provides an interface for the user to control the operation system 201 or the home PC 2 via the GUI.

If a display such as a liquid crystal display device is connected to the home PC 2 for example, this GUI generates a screen that will be displayed in the entire display area of the display as shown by a dotted line in FIG. Yes.
FIG. 2 is a conceptual diagram for explaining the resolution change in the present embodiment.

On the other hand, the desktop screen generation unit 202 displays on the display unit 102 of the mobile phone terminal 1 from the screen generated by the GUI based on the resolution information included in the resolution change notification received from the mobile phone terminal 1. A screen (this is called a “desktop screen”) is generated. This desktop screen is illustrated in a portion surrounded by a solid line in a portion surrounded by a dotted line in FIG.
The desktop screen generation unit 202 transmits the generated desktop screen to the mobile phone terminal 1. The transmission timing may be determined by the frame rate in the display unit 102 (for example, 30 fps is transmitted 30 times per second).

Next, the configuration of the mobile phone terminal 1 will be described.
The mobile phone terminal 1 includes a remote operation control unit 101, a display unit 102, a tilt state detection unit 103, an operation unit 104, and an operation recording unit 105.

  The remote operation control unit 101 is a functional block that executes various processes in order to realize remote operation and resolution change according to the present embodiment. The remote operation control unit 101 receives the desktop screen transmitted from the home PC 2 and displays it on the display unit 102. At this time, in the case where an area that cannot be displayed outside the display unit 102 and / or an area that cannot be displayed due to lack of screen data is generated, the former is ignored and is not displayed. Image processing such as painting with black or image processing such as reducing the displayed desktop screen and displaying it is performed.

  Further, the remote operation control unit 101 recognizes that the resolution has been changed when the tilt state detection unit 103 detects the tilt of the mobile phone terminal 1 and transmits a resolution change notification to the home PC 2. However, it is performed when an inclination (change in inclination such as (a) → (b) in FIG. 2) greater than the degree of switching of the screen provided by the display unit 102 is detected. The resolution change notification includes the resolution information after the change. The resolution change notification is input to the desktop screen generation unit 202.

  Note that the resolution is changed automatically by the inclination detection by the inclination state detection unit 103, in addition to manual change by the user using the operation unit 104, automatic by an application (not shown) installed in the mobile phone terminal 1. Changes are possible. It is not limited to only the detection by tilt detection.

Further, the remote operation control unit 101 receives the operation by the user accepted by the operation unit 104 as operation information from the operation unit 104, and transmits the operation information to the home PC 2. The operation information is input to the operation system 201.
The operation system 201 that has received the operation information performs a predetermined process based on the operation information (file operation, character input, etc.).

  Note that the remote operation control unit 101 records the operation information received within a certain time due to the change in resolution described above in the operation recording unit 105 for a while without transmitting it to the home PC 2 as it is. Process. This is because the desktop screen generation unit 202 does not generate a desktop screen immediately after the resolution is changed, and the processing based on the transmitted operation information cannot be executed and becomes invalid.

  The display unit 102 is a functional block that displays the desktop screen received by the remote operation control unit 101, and can be sufficiently implemented by a conventionally known technique such as a liquid crystal display device or an organic EL display device.

  The tilt state detection unit 103 detects the tilt state of the mobile phone terminal 1. Since the mobile phone terminal 1 is usually held by a user's hand, the tilted state is a concept including the state of the terminal orientation depending on how it is held. The tilt state detection unit 103 can be sufficiently implemented by a conventionally known technique such as a gravity sensor or an acceleration sensor.

  The operation unit 104 is a functional block that accepts user operations. You may carry out with a numeric keypad. Although the present invention is not limited, the touch panel may be implemented as a display unit 102 and an operation unit 104. The operation unit 104 generates operation information whose contents include character input, movement of a pointing cursor, and the like by an operation by a user, and passes them to the remote operation control unit 101.

  The operation recording unit 105 can be implemented, for example, in a memory or the like, and temporarily records the operation information received from the operation unit 104 by the remote operation control unit 101 within a certain time due to a change in resolution. Function block. The operation information recorded in the operation recording unit 105 is appropriately read out by the remote operation control unit 101.

With reference to FIG. 2 again, the resolution change in the present embodiment will be described.
In the present embodiment, the tilted state (in other words, the holding mode) of the mobile phone terminal 1 is detected and matched to the respective resolutions in the vertical state (FIG. 2 (a)) and the horizontal state (FIG. 2 (b)). In this system, a desktop screen of the home PC 2 is generated, transmitted to the mobile phone terminal 1, and remotely operated. The tilt state detection unit 103 detects tilt states such as a vertical state and a horizontal state.

Furthermore, with reference to FIG. 3, the change of the resolution in this embodiment is demonstrated.
FIG. 3 is a diagram illustrating a transition example of screens displayed on the display unit 102 in FIG. 1.
As illustrated, when a change in resolution is detected during display at the first resolution, the remote operation control unit 101 transmits a resolution change notification to the home PC 2. However, the home PC 2 side receives the resolution change notification, generates a new desktop screen, sends the generated desktop screen, and receives it by the remote operation control unit 101. A time lag occurs because reconnection is required.

  Until the newly generated desktop image is displayed with the new resolution, that is, the second resolution, in the present embodiment, first, the resolution before the resolution change (first resolution) is used. Continue to display. Next, operation information input from the operation unit 104 is recorded in the operation recording unit 105. When the desktop screen generated with the new resolution is displayed, the operation information recorded by the operation recording unit 105 is reflected on the desktop screen.

  That is, in this embodiment, even after the screen orientation is switched, there is no deterioration in operability such as the display being stopped until the desktop screen generated with the new resolution is displayed, and the previous resolution is maintained. The desktop screen continues to be displayed. Then, in the display device of the mobile phone terminal, an area that protrudes and cannot be displayed and / or an area that cannot be displayed without screen data is generated. In contrast, in the present embodiment, image processing is performed in which the former is ignored and not displayed, and the latter is painted black. However, in such a correspondence, the entire screen area of the display device cannot be fully utilized, and there is room for further improvement in operability degradation. Therefore, in the present embodiment, the operation information input to the desktop screen that is continuously displayed even after the resolution is changed is recorded and held, and the desktop screen at the new resolution is reflected after reception. Therefore, the user can continue the operation without interruption, and the operability is further improved.

  According to the present embodiment described above, in the “remote desktop”, even when the client having the display unit changes the resolution, the display unit continues to send the desktop screen with the changed resolution. Since the display continues to be displayed at the resolution before the change, the desktop screen will not disappear. Therefore, the deterioration of operability accompanying the change in resolution is improved.

  Further, according to the above embodiment, the operation performed on the desktop screen that is continuously displayed at the resolution before the change is temporarily stored as an operation record and reflected on the desktop screen after the resolution change. The user can continue the operation without interruption.

1 Mobile phone terminal (client)
DESCRIPTION OF SYMBOLS 101 Remote operation control part 102 Display part 103 Inclination state detection part 104 Operation part 105 Operation recording part 2 Home PC (server)
201 Operation System 202 Desktop Screen Generation Unit

Claims (6)

A client connected to a server via a network,
Display means for displaying a screen sent from the server;
When the change in the resolution of the screen is detected in the client, and the change is detected, when the resolution change notification including the changed resolution information is received, the server that sends the screen with the changed resolution is sent to the server The resolution change notification including the resolution information after the detected change is transmitted, and in addition, the screen until the server having received the resolution change notification transmits the screen with the changed resolution is transmitted. a remote operation control means for during control so that the screen and the display means displays continues to be displayed on the pre-change resolution not,
Operation means for accepting an operation by a user and passing the operation information to the remote operation control means;
An operation recording means for temporarily recording the operation information;
The remote control means is
The operation information passed after the resolution change notification is transmitted is recorded in the operation recording means,
After receiving the screen with the changed resolution, when the operation information is received, the operation information recorded in the operation recording unit is transmitted to the server that executes processing based on the operation information.
A client characterized by that. The client according to claim 1 , wherein the change in resolution is a change in aspect ratio between vertical and horizontal. The client according to claim 1 , wherein the change in resolution includes a change in size. The remote control means is
When the operation information includes the movement of the pointing cursor,
In the movement distance of the pointing cursor in the recorded operation information,
The movement distance obtained by multiplying the size before and after the change by the size change,
The client according to claim 3 , wherein the client is a movement distance reflected on the desktop screen generated based on the resolution information after the change. An image display system in which a server and a client are communicably connected via a network,
The server
A screen is generated and sent to the client, and when a resolution change notification including the changed resolution information is received, the screen having a size and orientation suitable for the changed resolution information is generated and sent to the client. Desktop screen generation means
Have
The client
Display means for displaying the screen sent from the server;
When the change in the resolution of the screen is detected in the client and the change is detected, the resolution change notification including the resolution information after the detected change is transmitted to the server, in addition, The remote control that controls the display of the screen displayed by the display means to continue to be displayed at the resolution before the change, while the server that has received the resolution change notification does not send the screen until the server transmits the screen with the changed resolution. Operation control means;
Operation means for accepting an operation by a user and passing the operation information to the remote operation control means;
An operation recording means for temporarily recording the operation information ;
The remote operation control means further includes:
The operation information passed after the resolution change notification is transmitted is recorded in the operation recording means,
After receiving the screen with the changed resolution, the operation information recorded in the operation recording means is transmitted to the server,
The server further includes:
When the operation information is received, a process based on the operation information is executed . In the client connected to the server via the network,
Display the screen sent from the server,
When the change in the resolution of the screen is detected in the client, and the change is detected, when the resolution change notification including the changed resolution information is received, the server that sends the screen with the changed resolution is sent to the server The resolution change notification including the resolution information after the detected change is transmitted, and in addition, the screen until the server having received the resolution change notification transmits the screen with the changed resolution is transmitted. While not doing so, the screen displayed by the display means is controlled to continue to be displayed at the resolution before the change,
Accept user operations as operation information,
Record the operation information passed after transmission of the resolution change notification in the operation recording means,
After receiving the screen with the changed resolution, when the operation information is received, the operation information recorded in the operation recording unit is transmitted to the server that executes processing based on the operation information.
A client control method.

Images