HK1172471A - Image playback apparatus capable of playing back panoramic image - Google Patents

Abstract

A digital camera 1 includes a storing unit 18, an orientation sensor 23, and an image playback unit 53. The storing unit 18 stores a panoramic image in association with orientation data indicative of shooting direction when capturing an image. The orientation sensor 23 acquires orientation data indicative of the direction of the image playback apparatus. The image playback unit 53 plays back an partial area of the panoramic image corresponding to the orientation data of the image playback apparatus, based on the orientation data in association with the panoramic image recorded by the storing unit 18, and the orientation data acquired by the orientation acquiring unit.

Description

Image reproducing apparatus capable of reproducing panoramic image

Technical Field

The present invention relates to an image reproducing apparatus, an image reproducing method, and a recording medium, and more particularly, to an image reproducing apparatus and an image reproducing method for reproducing a panoramic image.

Background

According to the related art, some digital cameras have an image capturing function with a horizontally long wide screen (hereinafter, referred to as a "panoramic image").

Japanese patent application laid-open No. h 06-303562 discloses a technique in which a user rotates and moves a digital camera in a horizontal direction while keeping a shutter switch pressed, while keeping the digital camera substantially fixed in a vertical direction around his body, and performs imaging processing a plurality of times during this period. Further, the above patent document also discloses a technique of generating image data of a panoramic image by synthesizing a plurality of image data obtained as a result of the plurality of times of imaging processing in a horizontal direction (horizontal direction).

Disclosure of Invention

The present invention is directed to reproducing a panoramic image that enables reproduction of a landscape captured during reproduction.

In order to achieve the above object, according to a 1 st aspect of the present invention, there is provided an image reproducing apparatus comprising: a recording unit that records data of a panoramic image recorded in association with azimuth data indicating a shooting direction at the time of shooting; an azimuth acquisition unit that acquires azimuth data indicating an azimuth of the image reproduction apparatus itself; and a reproduction unit that reproduces and displays an image area of the panoramic image corresponding to the orientation data of the image reproduction apparatus itself, based on the orientation data recorded in association with the panoramic image recorded by the recording unit and the orientation data acquired by the orientation acquisition unit.

In order to achieve the above object, according to a 2 nd aspect of the present invention, there is provided an image reproducing apparatus comprising: a recording unit that records a panoramic image recorded in association with angle data indicating an inclination angle of an image; an angle acquisition unit that acquires angle data indicating a tilt angle of the image reproduction device itself; and a reproduction unit that reproduces an image area of the panoramic image corresponding to the angle data of the image reproduction apparatus itself, based on the angle data recorded in association with the image constituting the panoramic image recorded by the recording unit and the angle data acquired by the angle acquisition unit.

Drawings

Fig. 1 is a block diagram showing a hardware configuration of a digital camera as an embodiment of a playback apparatus according to the present invention.

Fig. 2 is a functional block diagram showing a functional configuration of the digital camera of fig. 1 for executing an image pickup process and a reproduction process.

Fig. 3 is a diagram for explaining an image pickup operation in a case where a normal image pickup mode and a panoramic image pickup mode are selected as operation modes of the digital camera of fig. 2, respectively.

Fig. 4 is a diagram showing an example of a panoramic image generated in the panoramic imaging mode shown in fig. 3.

Fig. 5 is a top view showing how a user performs a playback operation when the panoramic playback mode is selected as the playback mode of the digital camera 1 shown in fig. 2.

Fig. 6 is a flowchart showing an example of the flow of the panoramic imaging process executed by the digital camera of fig. 2.

Fig. 7 is a flowchart showing a detailed flow of the panoramic image generation and recording process in the panoramic imaging process of fig. 6.

Fig. 8 is a flowchart showing a detailed flow of the panoramic image generation and recording process in the panoramic imaging process of fig. 6.

Fig. 9 is a flowchart showing an example of the flow of the panorama reproduction processing executed by the digital camera of fig. 2.

Detailed Description

Embodiments according to the present invention will be described below with reference to the drawings.

Fig. 1 is a block diagram showing a hardware configuration of a digital camera 1 as an embodiment of a recording apparatus and a reproducing apparatus according to the present invention.

The digital camera 1 includes: a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random access Memory) 13, a bus 14, an optical system 15, an image pickup Unit 16, an image Processing Unit 17, a storage Unit 18, a display Unit 19, an operation Unit 20, a communication Unit 21, an angular velocity sensor 22, an orientation sensor 23, and a driver 24.

The CPU11 executes various processes in accordance with a program stored in the ROM12 or a program loaded from the storage section 18 into the RAM 13.

The ROM12 also suitably stores data and the like necessary for the CPU11 to execute various processes.

For example, in the present embodiment, programs for realizing the functions of the main control unit 51 to the image reproduction unit 53 of fig. 2 described later are stored in the ROM12 or the storage unit 18. Therefore, the CPU11 executes the processing according to these programs, thereby realizing the functions of the main control unit 51 to the image reproduction unit 53 in fig. 2, which will be described later.

Further, at least some of the functions of the main control unit 51 to the image reproduction unit 53 shown in fig. 2, which will be described later, can be handed over to the image processing unit 17.

The CPU11, ROM12, and RAM13 are connected to each other via the bus 14. The optical system 15, the imaging unit 16, the image processing unit 17, the storage unit 18, the display unit 19, the operation unit 20, the communication unit 21, the angular velocity sensor 22, the orientation sensor 23, and the driver 24 are also connected to the bus 14.

The optical system 15 is constituted by a lens, such as a focus lens or a zoom lens, for converging light to photograph an object. The focus lens is a lens that forms a subject image on a light receiving surface of an image pickup device of the image pickup unit 16. The zoom lens is a lens in which a focal length is freely changed within a certain range. The optical system 15 is provided with peripheral devices for adjusting focus, exposure, and the like, as necessary.

The imaging unit 16 is composed of a photoelectric conversion element, an AFE (Analog Front End), and the like. The photoelectric conversion element is formed of, for example, a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) type photoelectric conversion element. The photoelectric conversion element photoelectrically converts (captures) an optical signal of the subject image that has been incident and accumulated during this period at regular intervals, and sequentially supplies the resultant analog electrical signal to the AFE.

The AFE performs various signal processes such as an a/D (Analog/Digital) conversion process on the Analog electric signal, and outputs a resultant Digital signal as an output signal of the image pickup unit 16.

Hereinafter, the output signal of the imaging unit 16 is referred to as "image data of the captured image". Therefore, the image data of the captured image is output from the imaging unit 16 and is appropriately supplied to the image processing unit 17 and the like.

The image processing unit 17 is constituted by a DSP (Digital Signal Processor), a VRAM (Video Random Access Memory), or the like.

The image processing unit 17 performs image processing such as noise reduction, white balance, and hand shake correction on the image data of the captured image input from the imaging unit 16 in cooperation with the CPU 11.

The image data of the captured image input from the imaging unit 16 at regular intervals is hereinafter referred to as "image data". In the present embodiment, the image data of the frame is used as a processing unit.

That is, the image processing unit 17 performs various image processing and outputs the image data of the frame supplied from the imaging unit 16.

The storage unit 18 is configured by a DRAM (Dynamic Random Access Memory) or the like, and temporarily stores image data of a frame or the like output from the image processing unit 17. The storage unit 18 also stores various data necessary for various image processing.

The display unit 19 is configured as a flat panel display panel including, for example, an LCD (Liquid Crystal Device) and an LCD driver. The display unit 19 displays an image expressed by the image data of the frame supplied from the storage unit 18 or the like, for example, a live view image described later, on a frame-by-frame basis.

The operation unit 20 includes a plurality of switches, such as a power switch and an imaging mode switch, not shown, in addition to the shutter switch 41 and the playback switch 42. The operation unit 20, when a predetermined switch among the plurality of switches is pressed, supplies a command assigned to the predetermined switch to the CPU 11.

The communication unit 21 controls communication with other devices not shown via a network including the internet.

The angular velocity sensor 22 is configured by a gyroscope or the like, detects an amount of angular displacement of the digital camera 1, and supplies a digital signal indicating the detection result (hereinafter, simply referred to as "amount of angular displacement") to the CPU 11.

The orientation sensor 23 includes, for example, an MI element (Magneto-Impedance element) whose Impedance changes in accordance with a change in an external magnetic field. The azimuth sensor 23 detects a 2-axis (X, Y) component of geomagnetism using the MI element, and outputs data indicating the detection result. Hereinafter, data indicating the detection result of the orientation sensor 23 will be referred to as "orientation data".

The drive 24 is preferably provided with a removable medium 31 including a magnetic disk, an optical disk, an opto-magnetic disk, a semiconductor memory, or the like. Then, the program read from the removable medium 31 is installed in the storage section 18 as necessary. The removable medium 31 can also store various data such as image data stored in the storage unit 18, similarly to the storage unit 18.

Fig. 2 is a functional block diagram showing a functional configuration for executing an image capturing process and a reproducing process among the processes executed by the digital camera 1 of fig. 1.

The "image pickup processing" is a series of processing for picking up an image of a subject until image data of a resultant picked-up image is recorded on the removable medium 31.

The "playback processing" is a series of processing for reading frame image data from the removable medium 31 and displaying an image represented by the frame image data on the display unit 19.

As shown in fig. 2, the CPU11 is provided with a main control unit 51, an image combining unit 52, and an image reproducing unit 53.

As described above, the functions of the main control unit 51 to the image reproduction unit 53 do not particularly need to be mounted on the CPU11 as in the present embodiment, and at least a part of these functions can be handed over to the image processing unit 17.

The main control unit 51 controls the whole of the execution of the image pickup processing and the reproduction processing. For example, when controlling the execution of the image capturing process, the main control unit 51 selectively switches between a normal image capturing mode and a panoramic image capturing mode as the operation mode of the digital camera 1. Then, the main control section 51 can execute processing in accordance with the operation mode after the switching.

When the panoramic imaging mode is set, the image combining unit 52 operates under the control of the main control unit 51.

Here, in order to facilitate understanding of the main control unit 51 and the image combining unit 52, the panoramic imaging mode will be described in detail with reference to fig. 3 and 4 before explaining the configurations of these functions.

Fig. 3 is a diagram for explaining an image pickup operation in a case where the normal image pickup mode and the panoramic image pickup mode are selected as the operation modes of the digital camera 1 in fig. 1, respectively.

In detail, fig. 3A is a diagram explaining an image pickup operation in the normal image pickup mode. Fig. 3B is a diagram explaining an image capturing operation in the panoramic image capturing mode.

In each of fig. 3A and 3B, a drawing located at the back of the digital camera 1 shows the appearance of the real world including the subject of the digital camera 1. In addition, broken lines in the vertical direction shown in fig. 3B show the respective positions a, B, and c in the moving direction of the digital camera 1. The moving direction of the digital camera 1 is a direction in which the optical axis of the digital camera 1 moves when the user changes the imaging direction (angle) of the digital camera 1 about his body.

The normal imaging mode is an operation mode in the case of capturing an image of a size (resolution) corresponding to the angle of view of the digital camera 1.

In the normal imaging mode, as shown in fig. 3A, the user presses the shutter switch 41 of the operation unit 20 to the lower limit in a state where the digital camera 1 is fixed. In addition, the operation of pressing the shutter switch 41 to the lower limit in this manner is hereinafter referred to as "full-press operation".

The main control section 51 controls execution of a series of processes until image data of a frame output from the image processing section 17 immediately after the full-press operation is performed is recorded as a recording target on the removable medium 31.

Hereinafter, a series of processes executed by the control of the main control section 51 in the normal imaging mode as described above will be referred to as "normal imaging processes".

On the other hand, the panoramic image capturing mode is an operation mode in the case of capturing a panoramic image.

In the panoramic imaging mode, as shown in fig. 3B, the user moves the digital camera 1 in the direction of the black arrow in the figure while maintaining the fully-pressed operation of the shutter switch 41.

While the full-press operation is maintained, the main control unit 51 controls the image combining unit 52 and the like. While the full-press operation is maintained, the main control unit 51 temporarily stores the image data of the frame immediately after the accumulation of the angular displacement amount supplied from the angular velocity sensor 22 in the storage unit 18 every time the angular displacement amount reaches a certain value, and repeats the process.

Thereafter, the user performs an operation of releasing the full-press operation, that is, an operation of separating the finger or the like from the shutter switch 41 (hereinafter, such an operation is referred to as a "release operation"), thereby instructing the end of the panoramic imaging.

The main control unit 51 controls the image combining unit 52 and the like, and after instructing the end of panoramic imaging, combines the image data of the plurality of frames stored in the storage unit 18 in the horizontal direction in the order of storage, thereby generating image data of a panoramic image.

Then, the main control unit 51 controls the image synthesizing unit 52 and the like to store the image data of the panoramic image as a recording target in the removable medium 31.

In this manner, the main control unit 51 controls the image synthesizing unit 52 and the like in the panoramic imaging mode, and controls a series of processes for generating image data of a panoramic image and recording the image data on the removable medium 31 as a recording target.

Hereinafter, a series of processes executed by the control of the main control unit 51 in the panoramic imaging mode will be referred to as "panoramic imaging processes".

In other words, the image combining unit 52 executes the following processing under the control of the main control unit 51.

That is, the image combining unit 52 receives an acquisition command from the main control unit 51 every time the digital camera 1 moves by a predetermined amount (every time the cumulative value of the angular displacement amounts becomes a constant value). Then, the image combining unit 52 acquires the image data of 1 frame from the image processing unit 17 in accordance with the acquisition command, and stores the image data in the storage unit 18.

At this time, each time the image combining unit 52 acquires image data of 1 frame, the azimuth data indicating the azimuth of the current shooting optical axis is acquired from the azimuth sensor 23.

The image combining unit 52 combines the image data of the plurality of frames thus obtained to generate image data of a panoramic image, and records the generated image data of the panoramic image on the removable medium 31. At this time, the image combining unit 52 records at least a part of the orientation data acquired so far in association with the image data of the panoramic image on the removable medium 31.

More specifically, in the present embodiment, the Image data of the panoramic Image is included in a predetermined File in the form of an Exif (Exchangeable Image File Format). Hereinafter, a file containing image data of such a panoramic image will be referred to as a "panoramic file". The panoramic file may contain various meta information in addition to the pixel data of the panoramic image.

Therefore, in the present embodiment, at least a part of the azimuth data obtained in the panoramic imaging is included in the panoramic file as one of the meta information. And records such a panorama file on the removable medium 31.

Here, in the present embodiment, image data of a frame to be synthesized is acquired every time the accumulated value of the angular displacement amount of the digital camera 1 becomes a constant value at the time of panoramic imaging. Therefore, when 1 piece of azimuth data is acquired at the same time when acquiring 1 piece of predetermined image data, for example, the 1 st image data, it is possible to estimate the azimuth at each position in the horizontal direction of the panoramic image to some extent by using the azimuth data.

On the other hand, as a matter of course, the greater the number of orientation data, the more the accuracy of determining the orientation of each position in the horizontal direction of the panoramic image is improved.

Therefore, the number of azimuth data included as meta information in the panorama file may be any number.

Fig. 4 is image data showing a panoramic image generated by the image combining unit 52 in the panoramic imaging mode shown in fig. 3.

That is, in the panoramic imaging mode, when the imaging operation as shown in fig. 3B is performed, the image data of the panoramic image P3 as shown in fig. 4 is generated by the image combining unit 52 under the control of the main control unit 51 and recorded in the removable medium 31.

The data of the panoramic image P3 is recorded on the removable medium 31 in association with the orientation data for each area corresponding to the image of 1 frame before the synthesis.

Thus, as shown in FIG. 4, it is possible to easily recognize which position in the horizontal direction of the panoramic image P3 each of the directions of N (North: North) E (East: East) S (South: South) W (West: West) corresponds to.

The above description has been made of a functional configuration for executing the image pickup processing among the functional configurations of the digital camera 1 of fig. 2.

Next, a functional configuration of the digital camera 1 for executing the reproduction processing will be described.

The reproduction processing is executed by the image reproduction section 53 under the control of the main control section 51.

That is, when controlling the execution of the playback process, the main control unit 51 selectively switches between the normal playback mode and the panoramic playback mode as the operation mode of the digital camera 1. Then, the main control section 51 can execute processing in accordance with the operation mode after the switching.

The normal reproduction mode is a mode for reproducing data of a captured image obtained in the normal imaging mode or data of an image having the same size (resolution) as the captured image.

The panoramic reproduction mode is a mode for reproducing data of a panoramic image obtained in the panoramic imaging mode or data of a panoramic image acquired by another device or a recording medium.

When the panorama playback mode is selected, the image playback unit 53 operates under the control of the main control unit 51.

Fig. 5 is a top view of the appearance of the reproduction operation by the user in the case where the panoramic reproduction mode is selected as the operation mode of the digital camera 1 in fig. 2.

In the panoramic reproduction mode of the present embodiment, an image of a region in the panoramic image to be reproduced, which corresponds to the orientation in which the digital camera 1 is facing (the orientation in the shooting optical axis direction from the inside of the imaging unit 16 of the digital camera 1 to the outside), is displayed.

Here, when the digital camera 1 is configured such that the direction of the shooting optical axis of the imaging unit 16 and the direction of the user's line of sight with respect to the display screen of the display unit 19 are different from each other, the direction of the user's current line of sight with respect to the display screen of the display unit 19 can be adopted as the direction in which the digital camera 1 is facing in the panoramic reproduction mode.

Therefore, as shown in fig. 5, when the user rotates the digital camera 1 (moves in the direction of the black arrow in the figure) in the same manner as in the panoramic imaging, the display contents are sequentially updated so as to be an image of an area corresponding to the orientation of the digital camera 1 after the movement.

Here, the image of the area corresponding to the azimuth of the digital camera 1 is an image around a position corresponding to the azimuth among the horizontal positions of the panoramic image, that is, an image captured by reflecting the image within the angle of view of the digital camera 1 facing the azimuth at the time of panoramic imaging.

For example, in a case where the reproduction object is the panoramic image P3 of fig. 4, and the digital camera 1 faces the direction D1 in fig. 5, that is, the direction of N (north), an image of an area in the panoramic image P3 at the position of N (north) in the horizontal direction is displayed.

Then, the digital camera 1 rotates and moves, and when facing the direction D2 in fig. 5, that is, the northeast direction, an image of an area in the horizontal direction between N (north) and E (east) in the panoramic image P3 is displayed.

The same applies to the reverse rotational movement of the digital camera 1. That is, when the digital camera 1 rotates and moves and faces the direction Di in fig. 5, that is, the direction W (west), an image of an area whose horizontal position is W (west) in the panoramic image P3 is displayed.

In order to enable such display of the panoramic image, the image reproduction unit 53 acquires orientation data (in the present embodiment, orientation data included in one of the meta information in the panoramic file) stored in association with the panoramic image to be reproduced under the control of the main control unit 51.

As described above, the azimuth data serves as an index for specifying the azimuth corresponding to the horizontal position of the panoramic image. For this reason, the azimuth data is hereinafter referred to as "index azimuth data".

Here, the azimuth data acquired as the index azimuth data may be all or a part of the azimuth data stored in association with the image data of the panoramic image.

When image data of a panoramic image to be reproduced is stored in the removable medium 31 in a panoramic file, at least a part of azimuth data included as meta information in the panoramic file is acquired as index azimuth data.

Under the control of the main control unit 51, the image reproduction unit 53 acquires orientation data indicating a direction of the shooting optical axis of the image pickup unit 16 or a direction of the display screen of the display unit 19 from the orientation sensor 23 as orientation data (hereinafter referred to as "current orientation data") indicating a current orientation (direction) of the digital camera 1.

Then, under the control of the main control unit 51, the image reproduction unit 53 extracts image data of an area corresponding to the current azimuth from the image data of the panoramic image to be reproduced, using the index azimuth data and the current azimuth data. Thereafter, the image reproduction unit 53 causes the display unit 19 to display an image represented by the extracted image data.

In this manner, the image reproduction unit 53 controls a series of processes until the panoramic image stored as image data in the removable medium 31 or the like is displayed on the display unit 19 in the panoramic reproduction mode under the control of the main control unit 51.

Hereinafter, the series of processing performed by the image reproduction unit 53 in the panoramic reproduction mode is referred to as "panoramic reproduction processing".

In this way, when the user rotates and moves the digital camera 1 to any orientation, the area in the panoramic image to be reproduced that corresponds to the current orientation of the digital camera 1 can be displayed on the display unit 19. Therefore, the user can view the panoramic image with a feeling that the landscape of the imaging place of the panoramic imaging is wide.

The functional configuration of the digital camera 1 according to the present embodiment is described above with reference to fig. 2 to 5.

Next, a flow of the panoramic imaging process in the imaging process performed by the digital camera 1 having such a functional configuration will be described with reference to fig. 6.

Fig. 6 is a flowchart showing an example of the flow of the panoramic imaging process.

In the present embodiment, the panoramic imaging process starts with setting of the panoramic imaging mode.

In step S1, the main control unit 51 of fig. 2 executes an initial setting process.

As one of the initial setting processes of the present embodiment, a process of setting a constant value of the angular displacement amount and a threshold value (for example, 360 degrees) that is a maximum limit of the angular displacement amount is adopted. Specifically, a certain value of the angular displacement amount and a threshold value (for example, 360 degrees) as a maximum limit of the angular displacement amount are stored in advance in the ROM12 of fig. 1, and are set by reading from the ROM12 and writing into the RAM 13. Note that the fixed value of the angular displacement amount is used in the determination processing of step S35 in fig. 7, which will be described later. On the other hand, a threshold value (for example, 360 degrees) that is a maximum limit of the angular displacement amount is used in the determination processing of step S43 in fig. 7.

In the present embodiment, as shown in steps S34, S39, and the like of fig. 7 described later, the angular displacement amounts detected by the angular velocity sensor 22 are accumulated, and the accumulated angular displacement amount or the integrated angular displacement amount (the difference between the two will be described later) as the accumulated value is stored in the RAM 13. Therefore, as one of the initial setting processes of the present embodiment, a process of resetting the accumulated angular displacement amount or the integrated angular displacement amount to 0 is employed. The cumulative angular displacement amount is compared with the above-described fixed value in the determination processing of step S35 in fig. 7, which will be described later. On the other hand, the integrated angle displacement amount is compared with the above-described threshold value in the determination processing of step S43 in fig. 7 described later.

Further, as one of the initial setting processes of the present embodiment, a process of resetting the error flag to 0 is also employed. The error flag is a flag that is set to 1 when an error occurs in the panoramic image generation/recording process (see step S45 in fig. 8 described later).

In step S2, the main control portion 51 starts the live view image capturing process and the live view display process.

That is, the main control unit 51 controls the imaging unit 16 and the image processing unit 17 to continue the imaging operation performed by the imaging unit 16. While the imaging unit 16 continues the imaging operation, the main control unit 51 temporarily stores in a memory (in the present embodiment, the memory 18) image data sequentially output from the image processing unit 17 via the imaging unit 16. Here, a series of control processing performed by the main control section 51 is referred to as "live view imaging processing" here.

The main control unit 51 controls the display control unit 54 to sequentially read each image data temporarily stored in the memory (the storage unit 18 in the present embodiment) at the time of live view imaging. Then, the main control unit 51 sequentially displays images corresponding to the read image data on the display unit 19. The series of control processing performed by the main control unit 51 is referred to as "live view display processing" herein. Hereinafter, an image displayed on the display unit 19 by the live view display processing will be referred to as a "live view image".

In step S3, the main control unit 51 determines whether or not the shutter switch 41 is half-pressed.

Here, the half-press operation refers to an operation of pressing the shutter switch 41 of the operation unit 20 halfway (to a predetermined position less than the lower limit), and is hereinafter also referred to as a "half-press operation" as appropriate.

If the shutter switch 41 is not half-pressed, the determination at step S3 is no, and the process proceeds to step S10.

In step S10, the main control unit 51 determines whether or not an instruction to end the process has been given.

Although the instruction to end the processing is not particularly limited, in the present embodiment, a notification indicating that the power supply of the digital camera 1, not shown, is turned off is used.

Therefore, in the present embodiment, when the power supply is turned off and the main control unit 51 is notified of the power supply off state, the determination at step S10 is yes, and the entire panoramic imaging processing is ended.

On the other hand, when the power supply is in the off state, no notification is given to the effect that the power supply is in the off state, and therefore, no determination is made in step S10. Then, the process returns to step S2, and the process thereafter is repeated. That is, in the present embodiment, as long as the power supply remains in the on state, step S3 is repeatedly executed until the shutter switch 41 is half-pressed: no and step S10: otherwise, the loop processing is performed, and the image pickup processing is in a standby state.

In the live view display processing, if the shutter switch 41 is half-pressed, the determination at step S3 is yes, and the processing proceeds to step S4.

In step S4, the main control unit 51 controls the image pickup unit 16 to execute a so-called AF (Auto Focus) process.

In step S5, the main control portion 51 determines whether or not the shutter switch 41 is fully pressed.

If the shutter switch 41 is not fully pressed, the determination at step S5 is no. In this case, the process returns to step S4, and the process thereafter is repeated. That is, in the present embodiment, until the shutter switch 41 is fully pressed, steps S4 and S5 are repeatedly executed: no loop processing, and at this time AF processing is performed.

Thereafter, when the shutter switch 41 is fully pressed, the determination at step S5 is yes, and the process proceeds to step S6.

In step S6, the main control unit 51 basically performs a series of processing (hereinafter referred to as "panoramic image generation recording processing") until image data of a panoramic image is generated and recorded on the removable medium 31.

Details of the panoramic image generation recording process will be described later with reference to fig. 7 and 8.

When the panoramic image generation and recording process of step S6 is completed, the process proceeds to step S7.

In step S7, the main control portion 51 determines whether or not the error flag is 1.

Although details will be described later with reference to fig. 7 and 8, if the image data of the panoramic image is recorded as a recording target on the removable medium 31 and the panoramic image generation recording process of step S6 is normally ended, the error flag will be 0. In this case, the determination at step S7 is no, and the process proceeds to step S10. Further, the processing after step S10 is described above, so the description thereof is omitted here.

In contrast, if any error occurs in the panoramic image generation/recording process of step S6, the panoramic image generation/recording process is ended. In this case, since the error flag is 1, the determination at step S7 is yes, and the process proceeds to step S8.

In step S8, the main control unit 51 displays the contents of the error on the display unit 19. Specific examples of the contents of the displayed error will be described later.

In step S9, the main control unit 51 releases the panoramic imaging mode and resets the error flag to 0.

This completes the entire panoramic imaging process.

The flow of the panoramic imaging process is described above with reference to fig. 6.

Next, with reference to fig. 7 and 8, a detailed flow of the panoramic image generation and recording process of step S6 in the panoramic imaging process of fig. 6 will be described.

Fig. 7 and 8 are flowcharts illustrating a detailed flow of the panoramic image generation and recording process.

As described above, when the shutter switch 41 is fully pressed in the panoramic imaging mode, the process proceeds to step S6 when it is determined as yes in step S5 of fig. 6, and the following process is executed as the panoramic image generation recording process.

That is, in step S31 of fig. 7, the main control unit 51 acquires the angular displacement amount from the angular velocity sensor 22.

In step S32, the main control unit 51 determines whether or not the angular displacement amount obtained in the process of step S31 is greater than 0.

Since the angular displacement amount is 0 in a state where the user has not moved the digital camera 1, the determination at step S32 is no, and the process proceeds to step S33.

In step S33, the main control portion 51 determines whether or not a predetermined time has elapsed since the continuation of the angular displacement amount 0. As the predetermined time, for example, an appropriate time longer than the time required from the time when the user fully presses the shutter switch 41 to the time when the user starts moving the digital camera 1 can be used.

If the predetermined time has not elapsed, the determination at step S33 is no. Then, the process returns to step S31, and the subsequent processes are repeated. That is, when the duration of the state in which the user has not moved the digital camera 1 is shorter than the predetermined time, the main control section 51 repeatedly executes steps S31 to S33: otherwise, the loop processing is performed to set the panoramic image generation recording processing to the standby state.

In this standby state, when the user moves the digital camera 1, the amount of angular displacement from the angular velocity sensor 22 becomes a value greater than 0. In this case, the determination at step S32 is yes, and the process proceeds to step S34.

In step S34, the main control unit 51 updates the accumulated angular displacement amount by adding the angular displacement amount obtained in the process of step S32 to the accumulated angular displacement amount thus far (the accumulated angular displacement amount is equal to the accumulated angular displacement amount + the angular displacement amount thus far). That is, the value stored in the RAM13 as the accumulated angle displacement amount is updated.

The cumulative angular displacement amount is a value obtained by adding up the angular displacement amounts in this manner, and indicates the amount of movement of the digital camera 1.

In the present embodiment, every time the user moves the digital camera 1 by a certain amount, 1 frame of image data (object to be synthesized) for generating image data of a panoramic image is supplied from the image processing unit 17 to the image synthesizing unit 52.

To achieve this, in the initial setting processing of step S1 in fig. 6, a "constant value" is given in advance to the cumulative angular displacement amount corresponding to the "constant amount" which is the movement amount of the digital camera 1.

That is, in the present embodiment, every time the accumulated angular displacement amount reaches a certain value, not only the image data (object to be synthesized) of 1 frame is supplied from the image processing unit 17 to the image synthesizing unit 52, but also the accumulated angular displacement amount is reset to 0.

Such a series of processes is executed as the next process from step S35 onward.

That is, in step S35, the main control portion 51 determines whether or not the accumulated angle displacement amount has reached a certain value.

If the cumulative angular displacement amount does not reach the fixed value, the determination at step S35 is no. Thereafter, the process returns to step S31, and the subsequent processes are repeated. That is, the user moves the digital camera 1 by a certain amount, and the main control section 51 repeatedly executes the loop processing of step S31 to step S35 as long as the accumulated angular displacement amount does not reach a certain value.

Thereafter, when the accumulated angular displacement amount reaches a certain value by the user moving the digital camera 1 by a certain amount, the determination of yes is made in step S35, and the process proceeds to step S36.

In step S36, the image combining unit 52 acquires image data of 1 frame from the image processing unit 17 under the control of the main control unit 51.

That is, when the main control unit 51 determines that the accumulated angular displacement amount has reached the fixed value and the process proceeds to step S36, it issues an acquisition command to the image combining unit 52.

The image combining unit 52 having received the acquisition instruction acquires image data of 1 frame from the image processing unit 17 as the process of step S36.

In step S37, the main control unit 51 acquires azimuth data indicating the current azimuth from the azimuth sensor 23.

In step S38, the main controller 51 temporarily stores the image data of 1 frame acquired in the process of step S36 in the storage unit 18 or the like in association with the orientation data acquired in the process of step S37.

In step S39, the main control unit 51 updates the integrated angle displacement amount by adding the current angle displacement amount (approximately a constant value) to the integrated angle displacement amount thus far (the integrated angle displacement amount is the integrated angle displacement amount + the accumulated angle displacement amount thus far). That is, the value stored as the integrated angle displacement amount in the RAM13 is updated.

In step S40, the main control portion 51 resets the accumulated angular displacement amount to 0. That is, the value stored as the accumulated angle displacement amount in the RAM13 is updated to 0.

In this way, the accumulated angular displacement amount is used to control the timing (timing) at which the image data (object to be synthesized) of 1 frame is supplied from the image processing unit 17 to the image synthesizing unit 52, that is, the issuing timing of the acquisition instruction. Therefore, the cumulative angle displacement amount is reset to 0 every time the acquisition instruction is issued when it reaches a certain amount.

Therefore, even if the accumulated angular displacement amount is used, the main control unit 51 cannot recognize how far the digital camera has moved since the start of the panoramic image generation and recording process until now.

To enable such recognition, in the present embodiment, the integrated angular displacement amount is used separately from the accumulated angular displacement amount.

That is, although the integrated angular displacement amount is a value obtained by adding up the angular displacement amounts, the integrated angular displacement amount is not reset to 0 even if it reaches a certain amount, and is continuously accumulated until the panoramic image generation and recording process is completed (specifically, until the process of step S46 described later is executed).

In this manner, not only is the integrated angular displacement amount updated in the processing of step S39, but also the accumulated angular displacement amount is reset to 0 in the processing of step S40, and then the processing proceeds to step S41.

In step S41, the main control portion 51 determines whether or not the removal operation is performed.

If the release operation is not performed, that is, if the full-press of the shutter switch 41 by the user is continued, the determination at step S41 is no, and the process proceeds to step S42.

In step S42, the main control unit 51 determines whether or not an error in image acquisition has occurred.

Although the error in image acquisition is not particularly limited, in the present embodiment, for example, a method is adopted in which the digital camera 1 is moved by a predetermined amount or more in the tilt direction, the up-down direction, or the reverse direction as an error.

If no error has occurred in the acquisition of another image, the determination at step S42 is no, and the process proceeds to step S43.

In step S43, the main control portion 51 determines whether or not the integrated angle displacement amount exceeds a threshold value.

As described above, the integrated angular displacement amount is an accumulated value of angular displacement amounts from the start of the panoramic image generation recording process (from the execution of the full-press operation) to the point of time when the process of step S39 is executed.

Here, in the present embodiment, the maximum amount of movement by which the user can move the digital camera 1 during panoramic imaging is determined in advance. The total angular displacement amount corresponding to the "maximum movement amount" as the movement amount of the digital camera 1 can be given as a "threshold value" in advance by the initial setting processing of step S1 in fig. 6.

As described above, in the present embodiment, the fact that the integrated angular displacement amount reaches the threshold value means that the digital camera 1 has moved by the maximum amount of movement.

Therefore, the user can continue moving the digital camera 1 even when the integrated angular displacement amount has not reached the threshold value, that is, when the moving amount of the digital camera 1 has not reached the maximum moving amount, and therefore it is determined as no in step S43. Thereafter, the process returns to step S31, and the subsequent processes are repeated.

That is, if the continuation of the angular displacement amount 0 for the predetermined time (the digital camera 1 is not moved for the predetermined time) is one of the errors, the loop processing of steps S31 to S43 is repeatedly executed as long as the full-press operation is continued in a state in which no error occurs.

Thereafter, when a release operation is performed in a state where no error occurs (yes in the process of step S41) or the digital camera 1 has moved by the maximum movement amount (yes in the process of step S43), the process proceeds to step S44.

In step S44, the main control unit 51 generates image data of the panoramic image by the image synthesis unit 52, associates the image data with at least a part of the orientation data acquired so far, and records the image data in the removable medium 31.

Specifically, the main control unit 51 records the coordinate information of each image area constituting the panoramic image and the orientation data corresponding to each image area in the removable medium 31 so as to be included in the panoramic file as meta information.

That is, a panoramic file containing image data of a panoramic image and some orientation data as one of meta information is recorded on the removable medium 31.

Then, in step S46 of fig. 8, the main control portion 51 resets the integrated angular displacement amount to 0.

This completes the panoramic image generation and recording process. That is, the process of step S6 in fig. 6 normally ends, and it is determined as no in the next process of step S7. Note that the processing after the determination as no in the processing of step S7 has been described above, and therefore, the description thereof is omitted here.

In addition, in the case where some error has occurred in the series of processing described above, that is, in the case where it is determined as yes in the processing of step S33 of fig. 7 or in the case where it is determined as yes in the processing of step S42, the processing proceeds to step S45 of fig. 8.

In step S45, the main control unit 51 sets the error flag to 1.

In this case, the process of step S44 is not executed, that is, the panoramic image generation recording process is ended without recording the image data of the panoramic image.

That is, the process of step S6 in fig. 6 is ended, and the process of the next step S7 is determined as yes. Thereafter, the content of the error is displayed in the process of step S8.

Although the display of the error content in this case is not particularly limited as described above, for example, a message display such as "acquisition failure of direction data" or "timeout" may be used.

The detailed flow of the panoramic imaging process is described above with reference to fig. 6 to 8.

When such panoramic imaging processing is executed 1 or more times, image data of a panoramic image obtained by panoramic imaging of each of the images is included. At this time, 1 or more panoramic files containing orientation data obtained by panoramic imaging of each of the meta information are generated and recorded in the removable medium 31.

In this case, the image reproduction unit 53 can reproduce panoramic image data included in a panoramic file to be reproduced (hereinafter referred to as "panoramic image to be reproduced") with a predetermined 1 file out of 1 or more panoramic files being a reproduction target under the control of the main control unit 51.

Therefore, a flow of the panorama reproduction processing capable of such reproduction will be described with reference to fig. 9.

Fig. 9 is a flowchart showing an example of the flow of the panorama reproduction processing.

In the present embodiment, the panorama playback processing starts with setting of the panorama playback mode. In the present embodiment, for the sake of convenience of explanation, the panoramic image to be reproduced is selected in advance by the selection operation of the operation unit 20 by the user.

In step S51, the image reproduction unit 53 in fig. 2 acquires, from the portable medium 31 or the like, azimuth data stored in association with the coordinate information of each image area constituting the panoramic image to be reproduced as the meta information of the panoramic file, as index azimuth data.

In step S52, the image reproduction unit 53 acquires azimuth data indicating the current azimuth from the azimuth sensor 23 as current azimuth data.

That is, the image reproduction unit 53 acquires the orientation (direction) in which the digital camera 1 faces as the orientation of the current line of sight of the user holding the digital camera 1.

In step S53, the image reproduction unit 53 compares the index azimuth data acquired in the process of step S51 with the current azimuth data acquired in the process of step S52. Then, the image reproduction unit 53 extracts data of an image area having index azimuth data closest to the current azimuth data from the panoramic image based on the comparison result.

That is, the image reproduction unit 53 extracts data for an image area corresponding to the same direction as the current direction of the user's line of sight from the panoramic image data.

In step S54, the image reproduction unit 53 displays an image represented by the data of the extracted image area on the display unit 19.

At this time, the image displayed on the display unit 19 is an image of a landscape that is extended in the same direction as the current direction of the user's line of sight at the shooting location of the panoramic image. In this way, at the current standing position of the user, the scenery at the place where the panoramic image was captured can be virtually represented.

In step S55, the image reproduction unit 53 determines whether or not an instruction to end the processing has been given.

Although the instruction to end the processing is not particularly limited, in the present embodiment, a method of notifying that the panorama reproduction mode is to be cancelled is employed.

Therefore, in the present embodiment, when the panorama playback mode is released and the main control unit 51 is notified of the release, the determination at step S55 is yes and the entire panorama playback process is ended.

On the other hand, if the panorama playback mode is not canceled, the determination at step S55 is no, and the process returns to step S52 and repeats the subsequent processes. That is, in the present embodiment, as long as the panorama reproduction mode is maintained, steps S52 to S55 are repeatedly performed: no loop processing.

Thus, the user rotates the digital camera 1, and every time the direction of the current line of sight of the user is changed, the display content of the display unit 19 is updated, and an image of a landscape that is present in the same direction as the direction of the current line of sight of the user is displayed at the shooting point of panoramic shooting.

In this way, when the panorama reproduction processing is executed, the panoramic image of the scene at the time of the reproduction shooting is reproduced.

As described above, the digital camera 1 of the present embodiment includes: an image processing unit 17, a main control unit 51, an image combining unit 52, and an image reproducing unit 53.

The main control unit 51 acquires current azimuth data indicating the current azimuth of the digital camera 1.

The image reproduction unit 53 extracts image data of an area corresponding to the current azimuth from the image data of the panoramic image, based on the index azimuth data associated with the image data of the panoramic image and the current azimuth data acquired by the main control unit 51.

The image reproducing unit 53 performs control to display an image represented by the image data of the extracted region on the display unit 19.

In this way, a region in the panoramic image that matches the current orientation of the digital camera 1 (the orientation in the direction in which the user's line of sight is facing) is displayed.

That is, among the stereoscopically expanded scenery, the scenery existing in the same direction as the current direction of the digital camera 1 (the direction in which the user's line of sight faces) can be virtually displayed at the shooting point of the panoramic image. Thus, the user can view the panoramic image with the feeling that the scenery at the shooting location of the panoramic image appears to be present on the spot.

The present invention is not limited to the above-described embodiments, and includes modifications, improvements, and the like within a range in which the object of the present invention can be achieved.

For example, in the above-described embodiment, although the reproduction of the image data of the panoramic image recorded in association with the orientation data indicating the orientation at the time of imaging is performed, the reproduction target is not particularly limited.

For example, in the above-described embodiment, although reproduction of image data of a panoramic image generated by synthesizing image data captured by swinging the imaging device in the horizontal direction is performed, the reproduction target is not limited to this.

This is applicable even when image data of a panoramic image generated by synthesizing image data captured by swinging the imaging device in an oblique direction (vertical direction) is reproduced.

In this case, the digital camera 1 includes a sensor capable of detecting the tilt angle at the time of shooting, and records tilt angle data indicating the detected tilt angle at the time of shooting in association with each image data constituting the panoramic image. In addition, each image area of the panoramic image corresponding to the current tilt angle of the digital camera 1 can be displayed at the time of reproduction.

In the above-described embodiment, for example, the playback device to which the present invention is applied is described as an example of the digital camera 1.

However, the present invention is not particularly limited to this, and can be applied to an electronic device having an imaging function capable of imaging a panoramic image in general, and for example, the present invention can be widely applied to a portable personal computer, a portable navigation device, a portable game machine, and the like.

The series of processes described above can be executed by hardware, and can also be executed by software.

In the case where a series of processes is executed by software, a program constituting the software is installed from a network or a recording medium into a playback apparatus, a computer controlling the playback apparatus, or the like. Here, the computer may be a computer embedded in a dedicated hardware group. Alternatively, in addition, the computer may be a computer that performs various functions by installing various programs, such as a general-purpose personal computer.

The recording medium containing such a program may be configured not only by the removable medium 31 distributed independently from the apparatus main body in order to provide the program to the user, but also by a recording medium or the like provided to the user in a state of being embedded in the apparatus main body in advance. The removable medium 31 is constituted by, for example, a magnetic disk (including a flexible disk), an optical disk, an optomagnetic disk, or the like. The recording medium provided to the user in a state of being embedded in the apparatus main body in advance is constituted by, for example, the ROM12 in which the program is recorded, or a hard disk included in the storage unit 18.

Further, in the present specification, the steps describing the program recorded on the recording medium include, of course, processes performed in time series along this order, and even if not necessarily, processes in time series can include processes performed in parallel or individually.

Claims (10)

1. An image reproduction device is provided with:

a recording unit that records data of a panoramic image recorded in association with orientation data indicating a shooting direction at the time of image capturing;

an azimuth acquisition unit that acquires azimuth data indicating an azimuth of the image reproduction apparatus itself; and

and a reproduction unit that reproduces and displays an image area of the panoramic image corresponding to the orientation data of the image reproduction apparatus itself, based on the orientation data recorded in association with the panoramic image recorded by the recording unit and the orientation data acquired by the orientation acquisition unit.

2. The image reproducing apparatus according to claim 1,

the playback device further includes a comparison unit that compares azimuth data recorded in association with the panoramic image with the azimuth data acquired by the azimuth acquisition unit,

the reproduction means reproduces and displays an image area of the panoramic image having orientation data closest to orientation data of the image reproduction apparatus itself, based on a comparison result of the comparison means.

3. The image reproducing apparatus according to claim 1,

the panoramic image is an image generated from a plurality of images including at least 1 image recorded in association with azimuth data indicating a shooting direction at the time of image capturing.

4. The image reproducing device according to any one of claims 1 to 3,

the reproduction means reproduces and displays an image area corresponding to the orientation data acquired from the panoramic image by the orientation acquisition means, based on the orientation data of any one of the plurality of images constituting the panoramic image.

5. An image reproduction device is provided with:

a recording unit that records a panoramic image recorded in association with angle data indicating an inclination angle at the time of image capturing;

an angle acquisition unit that acquires angle data indicating a tilt angle of the image reproduction device itself; and

and a reproduction unit that reproduces an image area of the panoramic image corresponding to the angle data of the image reproduction apparatus itself, based on the angle data recorded in association with the image constituting the panoramic image recorded by the recording unit and the angle data acquired by the angle acquisition unit.

6. The image reproducing apparatus according to claim 5,

the playback device further includes a comparison unit that compares angle data recorded in association with the panoramic image with the angle data acquired by the angle acquisition unit,

the reproduction means reproduces and displays an image area of the panoramic image having angle data closest to angle data of an image reproduction apparatus itself, based on a comparison result of the comparison means.

7. The image reproducing apparatus according to claim 5 or 6,

the panoramic image is an image generated from a plurality of images including at least 1 image recorded in association with angle data indicating a tilt angle at the time of image capturing.

8. The image reproducing apparatus according to any one of claims 5 to 7,

the reproduction means reproduces an image area corresponding to the angle data acquired from the panoramic image by the angle acquisition means, based on angle data of any one of the plurality of images constituting the panoramic image.

9. An image reproducing method for a reproducing apparatus having a recording means for recording data of a panoramic image recorded in association with azimuth data indicating a shooting direction at the time of shooting,

the image reproducing method includes:

an orientation acquisition step of acquiring orientation data indicating an orientation of the image reproduction apparatus itself; and

and a reproduction step of reproducing and displaying an image area of the panoramic image corresponding to the orientation data of the image reproduction apparatus itself, based on the orientation data stored in association with the panoramic image and the orientation data acquired by the processing of the orientation acquisition step.

10. An image reproducing method for a reproducing apparatus having a recording means for recording data of a panoramic image recorded in association with angle data indicating a tilt angle at the time of image pickup,

the image reproducing method includes:

an angle acquisition step of acquiring angle data indicating a tilt angle of the image reproduction apparatus itself; and

and a reproduction step of reproducing an image area of the panoramic image corresponding to the angle data of the image reproduction apparatus itself, based on the angle data recorded in association with the image constituting the panoramic image recorded by the recording unit and the angle data acquired by the angle acquisition step.