WO2025150216A1 - Camera control program, camera control method, and information processing device - Google Patents

Abstract

This camera control program causes a computer to execute a process for acquiring a captured image from a camera and, when controlling panning of the camera in accordance with the position of an imaged object in the captured image, permitting panning control only in a predetermined direction.

Description

Camera control program, camera control method, and information processing device

The present invention relates to a camera control program, a camera control method, and an information processing device.

A type of camera known as a PTZ camera has pan, tilt and zoom functions. For example, when the subject of a photograph is a moving object, pan control can be performed based on the position of the subject in the captured image by linking the PTZ camera with object detection using AI (Artificial Intelligence).

JP 2013-162502 A

However, when AI-based object detection is used, as the moving speed of the subject increases, the relative position between the position of the subject on the captured image and the target position for photographing the subject on the captured image changes more frequently to either side of the subject's travel direction. As a result, the pan movement direction changes frequently, making it difficult to perform smooth pan control.

In one aspect, the present invention aims to provide a camera control program, a camera control method, and an information processing device that can achieve smooth pan control.

In one aspect, the camera control program causes a computer to execute a process that acquires an image captured by a camera, and when controlling the panning of the camera according to the position of the subject in the captured image, allows panning control only in a predefined direction.

According to one embodiment, smooth pan control can be achieved.

FIG. 1 is a diagram showing an example of the configuration of a camera control system. FIG. 2 is a diagram (1) showing one aspect of the problem-solving approach. FIG. 3 is a diagram (2) showing one aspect of the problem-solving approach. FIG. 4 is a block diagram illustrating an example of a functional configuration of the information processing device. FIG. 5 is a schematic diagram showing an example of division of sections. FIG. 6 is a diagram showing an example of the section setting data. FIG. 7 is a schematic diagram showing an example of object detection. FIG. 8 is a schematic diagram showing an example of calculation of the control parameters. FIG. 9 is a flowchart showing the procedure of the camera control process. FIG. 10 is a schematic diagram showing an application example of section division. FIG. 11 is a diagram illustrating an example of a hardware configuration.

Below, examples of a camera control program, a camera control method, and an information processing device according to the present disclosure will be described with reference to the attached drawings. Note that this example merely illustrates one example or aspect, and such an example does not limit the structure, action, function, nature, characteristics, method, or use of the present disclosure.

Example 1
<System Configuration>
Fig. 1 is a diagram showing an example of the configuration of a camera control system. Fig. 1 shows, as an example, a camera control system 1 that provides a camera control function for causing a camera 30 to capture an image of a moving object moving on a course C.

As an example of a usage scenario for such a camera control system 1, a horse race broadcast will be given, but this is just one example, and usage scenarios to which the camera control system 1 can be applied are not limited to specific races, specific courses, or specific subjects.

For example, the camera control system 1 may be applied to other sports besides horse racing, such as track and field, bicycle racing, boat racing, car racing, and triathlons. Furthermore, the camera control system 1 does not necessarily have to be applied to a course determined for a sport. For example, the "course" may be a course determined for training, such as hill training or training tests, or may be a course formed in a facility or area to be monitored.

In this way, the camera control system 1 can capture any subject, such as a race horse, a person, a vehicle, an aircraft, a robot, or other moving object, depending on the usage scene.

As shown in FIG. 1, the camera control system 1 may include an information processing device 10 and a camera 30. The information processing device 10 and the camera 30 may be connected to each other so that they can communicate with each other, whether wired or wirelessly.

The information processing device 10 is an example of a computer that provides the above-mentioned camera control function. For example, the information processing device 10 can be realized by a computer on which a camera control program that realizes the above-mentioned camera control function is installed. Note that the above-mentioned camera control function can be realized as a Software as a Service (SaaS) type application, thereby providing it as a cloud service, or as a Web server that provides the above-mentioned camera control function on-premise.

Camera 30 is an example of an imaging device that captures images. For example, camera 30 may be realized by a PTZ camera that allows remote control of pan (horizontal rotation around the Y axis), tilt (vertical rotation around the X axis), and optical zoom. Such a camera 30 is installed so that it can capture all or part of course C. This makes it possible to capture images of a moving object such as a racehorse moving along course C. Note that while a PTZ camera that combines three elements for controlling the camera's field of view has been given as an example of camera 30 here, it may also be a camera that allows control of at least one or more elements.

As one aspect, images captured by the camera 30 on a frame-by-frame basis may be transmitted between the information processing device 10 and the camera 30. For example, communication from the information processing device 10 to the camera 30 may be realized by any interface standard, such as HDMI (registered trademark) (High-Definition Multimedia Interface). However, this is not limited to this, and does not prevent it from being realized by any communication technology, such as an intranet, the Internet, or a low-power wireless communication standard for the IoT (Internet of Things).

As another aspect, commands for remotely operating the camera 30 may be transmitted between the information processing device 10 and the camera 30. For example, communication from the information processing device 10 to the camera 30 may be realized by a protocol for transmitting commands for controlling the camera, such as VISCA (Video System Control Architecture).

<One aspect of the issue>
As explained in the Background Art section above, the following problem occurs when AI-based object detection is used in conjunction. That is, as the moving speed of the object to be photographed increases, the relative position between the object to be photographed on the photographed image and the target position for photographing the object on the photographed image changes more frequently in front of or behind the moving direction of the object to be photographed. As a result, the pan movement direction changes frequently, making it difficult to perform smooth pan control.

<One aspect of the problem-solving approach (1)>
Therefore, when controlling the pan of the camera 30 in accordance with the position of the subject in the image captured by the camera 30, the camera control function according to this embodiment permits pan control only in a predefined direction.

FIG. 2 is a diagram (1) showing one aspect of the problem-solving approach. For example, FIG. 2 shows the results of performing object detection on an image 20 captured by a camera 30. Furthermore, FIG. 2 shows an example in which a control parameter is determined to control the pan direction of the camera 30 with the position of the leading racehorse on the captured image 20 as the target.

As shown in Figure 2, as a result of executing object detection by AI, three bounding boxes BB1 to BB3 are detected from the captured image 20. In this case, of the three bounding boxes BB1 to BB3, the bounding box BB1 that corresponds to the leading edge of the "left direction" that corresponds to the traveling direction of the captured object is targeted.

For example, let us take the case where the target position for photographing the subject is the center of the screen. In this case, a specific position of the bounding box BB1, for example the two-dot chain line corresponding to the left edge, is set as the aim, and control parameters are set to align with the target position, for example the dashed line corresponding to the center of the screen, such as the relative pan movement amount and absolute pan position.

Here, when the pan control parameters are set based solely on the results of object detection by AI, as in the above-mentioned conventional technology, the pan movement direction is set to the "right direction" which is the opposite direction to the traveling direction of the subject in the example shown in FIG. 2. That is, in the example shown in FIG. 2, when the aiming position is located ahead of the target position in the traveling direction of the subject, the pan movement direction of the camera 30 is set to the "right direction" which is the opposite direction to the traveling direction of the subject. Therefore, the pan movement direction switches between the traveling direction and the opposite direction.

On the other hand, in the camera control function of this embodiment, pan control is permitted only in a predefined direction, for example, the "left" direction, which is the traveling direction of the subject. In other words, if the aiming position is located ahead of the target position in the traveling direction of the subject, it is prohibited to set the pan movement direction of the camera 30 to the "right" direction, which is the opposite direction to the traveling direction of the subject. This makes it possible to prevent the pan movement direction from switching between the traveling direction and the opposite direction.

Therefore, the camera control function according to this embodiment makes it possible to realize smooth pan control of the camera 30. Furthermore, the camera control function according to this embodiment realizes automatic shooting that allows smooth pan control of the camera 30, which makes it possible to eliminate the need for personnel for shooting. Therefore, the location where the camera 30 is installed is not restricted by the ease of access for the photographer or the space available for the photographer to shoot. As a result, it is possible to install the camera 30 in a location where the camera angle is favorable or where occlusion due to competition or spectator facilities is unlikely to occur.

<One aspect of the problem-solving approach (2)>
In addition, the camera control function of this embodiment allows pan control to move at a predefined speed when controlling the pan of camera 30 in accordance with the position of the object being photographed in the image captured by camera 30 and the pan coordinates of camera 30 are within a specified angle range.

FIG. 3 is a diagram (2) showing one aspect of the problem-solving approach. In FIG. 3, a clockwise course C is shown as an example of a course. As shown in FIG. 3, when the pan angle of the camera 30 is within the angle ranges A1, A2, and A3, pan control is permitted to move in the pan direction at a predefined speed V1, V2, or V3.

For example, in pan angle range A1, the racehorse, which is the subject of the photography, is closer to the camera 30 than in other angle ranges. For this reason, the speed at which the racehorse moves in the captured image is also relatively faster. As the moving speed of the subject of the photography increases in this way, the frequency with which the relative position between the position of the subject of the photography in the captured image and the target position for photographing the subject in the captured image switches between before and after the subject's traveling direction increases. The moving speed of a racehorse is largely fixed, and by setting a fixed pan speed V1 in such pan angle range A1 as a control parameter, it is possible to prevent the pan movement direction from switching between the traveling direction and the opposite direction.

Furthermore, in pan angle range A2 and pan angle range A3, the racehorse, which is the subject of the shot, is farther away from camera 30 than in other angle ranges. As the subject of the shot moves farther away from camera 30 in this way, the accuracy of object detection by the AI decreases or becomes unstable. As a result, the bounding box corresponding to the leading horse is detected or lost, and the aim that determines the pan control parameters changes frequently. By setting a fixed pan speed V2 or speed V3 as the control parameter in pan angle range A2 and pan angle range A3, it is possible to prevent the pan movement direction from switching between the forward direction and the opposite direction.

Therefore, the camera control function according to this embodiment makes it possible to realize smooth pan control of the camera 30. Furthermore, the camera control function according to this embodiment realizes automatic shooting that allows smooth pan control of the camera 30, which makes it possible to eliminate the need for personnel for shooting. Therefore, the location where the camera 30 is installed is not restricted by the ease of access for the photographer or the space available for the photographer to shoot. As a result, it is possible to install the camera 30 in a location where the camera angle is favorable or where occlusion due to competition or spectator facilities is unlikely to occur.

<Configuration of information processing device>
Next, a functional configuration of the information processing device 10 according to the present embodiment will be described. Fig. 4 is a block diagram showing an example of the functional configuration of the information processing device 10. Fig. 4 is a block diagram showing an example of the functional configuration of the information processing device 10. Fig. 4 shows a schematic diagram of blocks related to a camera control function of the information processing device 10.

As shown in FIG. 4, the information processing device 10 has a communication control unit 11, a storage unit 13, and a control unit 15. Note that FIG. 4 merely shows an excerpt of the functional units related to the above-mentioned camera control function, and the information processing device 10 may also be provided with functional units other than those shown, such as an input unit and a display unit.

The communication control unit 11 is a functional unit that controls communication with other devices such as the camera 30. For example, the communication control unit 11 can be realized by a network interface card. In one aspect, the communication control unit 11 can accept captured images from the camera 30. In another aspect, the communication control unit 11 can output commands for controlling the camera 30 and control parameters specified by the commands to the camera 30.

The storage unit 13 is a functional unit that stores various types of data. For example, the storage unit 13 is realized by internal, external, or auxiliary storage of the information processing device 10. In one embodiment, the storage unit 13 stores information such as section setting data 13A. Note that the storage unit 13 may also store information other than the section setting data 13A, such as photographed images, programs in which races are held, and information on the jockeys and racehorses participating in each race.

The section setting data 13A is data in which definitions regarding camera control are set for each section. The term "section" here refers to the segments into which course C is divided. Figure 5 is a schematic diagram showing an example of section division. Figure 5 shows a clockwise course C as an example of a course. As shown in Figure 5, course C is divided into eight sections, sections 1 to 8, which allow the movement direction of the subject to be photographed within a section to be narrowed down to only one direction. For example, in each of sections 1 to 8, the movement direction of a racehorse can be limited to left, left, left, right, right, down, down, and left.

For example, the section setting data 13A may be data that associates settings such as a definition of the conditions for transitioning to the next section for each section and a definition of the direction in which operations are permitted or prohibited in that section.

FIG. 6 is a diagram showing an example of section setting data 13A. For example, the PTZ coordinates are expressed in the PTZ coordinate system described below. For example, the pan coordinate system is expressed in the range from the coordinate "-8704" corresponding to 170 degrees left to the coordinate "8704" corresponding to 170 degrees right. The tilt coordinate system is expressed in the range from the coordinate "-1024" corresponding to 20 degrees down to the coordinate "4608" corresponding to 90 degrees up. Furthermore, the zoom coordinate system is expressed in the range from the coordinate "0" corresponding to the minimum zoom out to the coordinate "16384" corresponding to the maximum zoom in.

Furthermore, the PTZ control parameters can be specified within the range of PTZ speeds, which are explained below. For example, the pan speed can be specified within the range of -24 to 24, with a positive sign representing "left" and a negative sign representing "right". The tilt speed can be specified within the range of -24 to 24, with a positive sign representing "up" and a negative sign representing "down". The zoom speed can be specified within the range of -7 to 7, with a positive sign representing "zoom in" and a negative sign representing "zoom out".

Here, Figure 6 shows an excerpt of the settings for Section 1 and Section 2 out of the eight sections shown in Figure 5.

For example, in the case of section 1, the pan coordinate condition for transitioning to section 2, the next section after section 1, is that the pan coordinate moves up to 4100. Additionally, the tilt coordinate condition is that the tilt coordinate moves up to -19. Additionally, the zoom coordinate condition is that the zoom coordinate moves up to 16384.

Furthermore, in section 1, it is defined that the aim used to calculate the pan control parameters is set based on the left direction, while the aim used to calculate the tilt and zoom control parameters is not set. Furthermore, in section 1, it is defined that the traveling direction of the object to be photographed is the left direction.

Furthermore, in section 1, it is defined that operations within the speed range described below are permitted. For example, pan speed is permitted in the range from 0 to 3. This has the aspect of being equivalent to defining the left direction as the permitted direction of pan rotation. Tilt speed is permitted in the range from -3 to 0. This has the aspect of being equivalent to defining the downward direction as the permitted direction of tilt rotation. Zoom speed is permitted in the range from -2 to 0. This has the aspect of being equivalent to defining zoom out as the permitted direction of zoom. Furthermore, for zooming, it is defined that reduction up to 14921 is permitted and enlargement up to 16384 is permitted. Note that while FIG. 6 shows an example in which a speed range for permitted operations is defined, a speed range for prohibited operations may also be defined.

Section 1 further defines the center of the screen as the target used to calculate the pan control parameters. Section 1 further defines that bounding boxes obtained by object detection using AI with a confidence level of 0.2 or higher are used.

Next, in the example of section 2, the pan coordinate condition for transitioning to section 3, the next section after section 2, is that the pan coordinate moves to 3462. Also, the tilt coordinate condition is that the tilt coordinate moves to -87. And the zoom coordinate condition is that the zoom coordinate moves to 14921.

Furthermore, section 2 defines that the aim used to calculate the pan control parameter is set based on the left direction.Furthermore, it defines that the aim used to calculate the tilt control parameter is set based on the downward direction.Furthermore, it defines that the aim used to calculate the zoom control parameter is set based on zoom out.Furthermore, section 2 defines that the traveling direction of the subject to be photographed is the left direction.

Furthermore, section 2 defines that operations in the speed ranges described below are permitted. For example, pan speeds are permitted in the range of 1 to 7. This has the aspect of being equivalent to defining the left direction as the permitted direction of pan rotation. Also, tilt speeds are permitted in the range of -3 to -1. This has the aspect of being equivalent to defining the downward direction as the permitted direction of tilt rotation. Furthermore, zoom speeds are permitted in the range of -3 to -1. This has the aspect of being equivalent to defining zoom out as the permitted direction of zoom. Furthermore, for zooming, it is defined that shrinking up to 8931 is permitted and zooming in up to 14921 is permitted.

Furthermore, Section 2 defines that the target used in calculating the pan control parameters is the center of the screen in the height direction and 50 pixels to the left of the center of the screen in the width direction. Furthermore, Section 2 defines that bounding boxes obtained by object detection using AI should be those with a confidence level of 0.3 or higher.

Note that in Figure 6, definitions relating to Sections 1 and 2 have been excerpted and explained, but it goes without saying that definitions are also set for the other sections. For example, Sections 3, 5, and 6 correspond to the angle range A1 to A3 shown in Figure 3, so the PTZ speed range can be set narrower than the speed range defined for the other sections, or a single value, for example a value corresponding to V1 to V3, can be set. This allows the PTZ speed to be fixed at a substantially constant value. In addition, for Section 8, which corresponds to the goal of Course C, the movement of the PTZ can be stopped by defining the end point of coordinates for each function of changing the field of view of the PTZ instead of the transition condition to the next section.

Returning to the explanation of FIG. 4, the control unit 15 is a functional unit that performs overall control of the information processing device 10. For example, the control unit 15 can be realized by a hardware processor. As shown in FIG. 4, the control unit 15 has an image acquisition unit 15A, an object detection unit 15B, a control amount calculation unit 15C, a camera control unit 15E, and an output control unit 15F. Note that the control unit 15 may also be realized by hardwired logic, etc.

The image acquisition unit 15A is a processing unit that acquires captured images. In one embodiment, the image acquisition unit 15A can acquire captured images from the camera 30 on a frame-by-frame basis. Note that, although an example of acquiring captured images transmitted from the camera 30 has been given here, the captured images may also be acquired via an external device, such as a file server or a storage medium, such as removable media.

The object detection unit 15B is a processing unit that detects objects from a captured image. Such object detection may be realized by a machine learning model that executes an object detection task that takes an image as input and outputs the position and category of an object in the image. For example, the machine learning model may be realized by YOLO (You Only Look Once), Faster-RCNN (Regions with Convolutional Neural Network), DETR (End-to-End Object Detection with Transformers), etc.

To train such machine learning models, a training dataset can be used, which is training data that matches images with bounding boxes indicating the location of objects in the images and ground truth labels that include classes indicating the object categories.

On one hand, in the training phase, the image is used as the explanatory variable, the correct label is used as the objective variable, and a machine learning model can be trained according to a machine learning algorithm, such as deep learning. This results in a trained machine learning model.

On the other hand, in the prediction phase, the captured image is input to a trained machine learning model. Given the input scene, the machine learning model outputs a bounding box, a class and a confidence score.

In the following, the term "object detection model" may refer to a trained machine learning model that performs object detection tasks.

In one embodiment, when a difference image of a new frame is acquired by the image acquisition unit 15A, the object detection unit 15B inputs the acquired image to the object detection model, thereby acquiring the object detection result (prediction result) output by the object detection model.

FIG. 7 is a schematic diagram showing an example of object detection. As shown in FIG. 7, the object detection unit 15B inputs the captured image 20 to the object detection model. The object detection model to which the captured image 20 is input in this manner outputs an object detection result including a bounding box, class, and confidence level for each object detected from the captured image 20. For example, in the example shown in FIG. 7, a "class name" such as horse and a "confidence level" of 0 to 1 are predicted for each of the bounding boxes BB1 to BB3 detected from the captured image 20.

The control amount calculation unit 15C is a processing unit that calculates the control parameters that control the PTZ of the camera 30. Below, an example of a control parameter in which the speed of the PTZ is specified is given, but it goes without saying that it is not limited to specifying a relative movement from the current position, and the coordinates of the PTZ can be specified as absolute values.

In one embodiment, the control amount calculation unit 15C executes the following process for each function of changing the field of view of the camera 30, for example, for each PTZ. That is, the control amount calculation unit 15C calculates control parameters according to the settings of the aim and target defined for the section corresponding to the progress of the race by the subject of the image in the section setting data 13A stored in the memory unit 13. For example, the progress of the race can be managed by updating the loop counter of the section based on the monitoring result of whether or not the coordinates of the PTZ observed in the new frame satisfy the transition condition to the next section each time a photographed image of a new frame is acquired.

FIG. 8 is a schematic diagram showing an example of calculation of control parameters. For example, FIG. 8 shows an example of calculation of PTZ control parameters from object detection results for the captured image 20 shown in FIG. 7. Furthermore, FIG. 8 shows an example of calculation of PTZ control parameters according to the aim setting defined in section 2 of the section included in the section setting data 13A. Furthermore, FIG. 8 shows an example of calculation of the pan speed, which is one of the control parameters.

As shown in FIG. 8, as a result of executing the object detection model, three bounding boxes BB1 to BB3 are detected from the captured image 20. In this case, of the three bounding boxes BB1 to BB3, the bounding box BB1 corresponding to the beginning of the "left direction" corresponding to the traveling direction of the captured object is set as the target. At this time, the bounding box to be set as the target may be determined after narrowing down to bounding boxes with a certainty level equal to or greater than the threshold value of "0.3" according to the definition of certainty level in Section 2.

Here, in section 2, as explained using Figure 6, position T1 is defined as the center of the screen in the height direction and 50 pixels to the left of the center of the screen in the width direction, as the target used to calculate the pan control parameters.

In this case, a specific position of the bounding box BB1, for example the two-dot chain line corresponding to the left side, is used as the aim, and a pan speed is calculated to align with a targeted position, for example the position T1 where the dashed line and the one-dot chain line intersect. For example, a higher pan speed is calculated as the distance D1 from the aim to the target increases, while a lower pan speed is calculated as the distance D1 from the aim to the target decreases. Furthermore, if the aim position is located before the target position in the traveling direction of the subject, a negative sign, i.e., a rightward direction, is set for the pan speed, whereas if the aim position is located after the target position in the traveling direction of the subject, a positive sign, i.e., a leftward direction, is set. Note that although an example of calculating the pan speed has been described here, the tilt speed and zoom speed can also be calculated in a similar manner.

The camera control unit 15E is a processing unit that controls the PTZ of the camera 30. As an embodiment, the camera control unit 15E executes the following process for each function of changing the field of view of the camera 30, for example, for each PTZ. That is, the camera control unit 15E judges whether the control parameter calculated by the control amount calculation unit 15C is an allowed operation defined in the section setting data 13A corresponding to the progress state of the race by the subject to be photographed. At this time, if the control parameter is not an allowed operation, the camera control unit 15E specifies the boundary value corresponding to the sign of the control parameter calculated by the control amount calculation unit 15C among the boundary values of the range of the control parameter defined as the allowed operation of the relevant section as the control parameter of the remote control command. On the other hand, if the control parameter is an allowed operation, the camera control unit 15E specifies the control parameter calculated by the control amount calculation unit 15C as the control parameter of the remote control command of the camera 30. After that, the camera control unit 15E transmits to the camera 30 a remote control command of the PTZ in which the control parameter is specified for each function of changing the field of view of the camera 30.

The output control unit 15F is a processing unit that executes output control of the captured image acquired by the image acquisition unit 15A. In one embodiment, the output control unit 15F can output the captured image acquired by the image acquisition unit 15A on a frame-by-frame basis. For example, the captured images captured in chronological order may be output as a video. Such output may be implemented in any form, such as television broadcasting, internet distribution, live broadcasting using SNS (Social Networking Service), etc.

<Processing flow>
Next, a process flow of the information processing device 10 according to the embodiment will be described. Fig. 9 is a flowchart showing the procedure of the camera control process. This process can be started when the above-mentioned camera control function is activated.

As shown in FIG. 9, the camera control unit 15E moves the camera 30 to an initial position (step S101), and the camera control unit 15E sets an initial value, for example "1", to the loop counter m for the section (step S102).

Then, the image acquisition unit 15A acquires a captured image of a new frame (step S103). Then, the object detection unit 15B inputs the captured image acquired in step S103 into the object detection model, thereby acquiring the object detection result (prediction result) output by the object detection model (step S104).

Then, the control amount calculation unit 15C identifies the bounding box of the target defined in the section corresponding to the value of the loop counter m from among the bounding boxes obtained as the object detection result in step S104 (step S105).

Then, loop process 1 is executed in which the processes from step S106 to step S108 below are repeated a number of times corresponding to the number N of field of view changing functions of camera 30. Note that, although an example is given here in which the processes from step S106 to step S108 below are repeated, the processes from step S106 to step S108 below may be executed in parallel.

In other words, the control amount calculation unit 15C calculates the control parameters related to the nth field of view change function based on the position of the bounding box identified as the aim in step S105 and the position of the target defined in the section corresponding to the value of the loop counter m in the section setting data 13A (step S106).

Then, the camera control unit 15E determines whether the control parameters calculated in step S106 are permitted operations defined in the section corresponding to the value of the loop counter m in the section setting data 13A (step S107).

Here, if the control parameter related to the nth field of view change function is not a permitted operation (No in step S107), the camera control unit 15E specifies, among the boundary values of the range of control parameters defined as permitted operations for the section corresponding to the value of the loop counter m, the boundary value on the side corresponding to the sign of the control parameter calculated in step S106 as the control parameter of the remote control command (step S108).

On the other hand, if the control parameters are not a permitted operation (Yes in step S107), the process of step S108 is skipped. In this case, the control parameters calculated in step S106 are specified as the control parameters of the command for remotely controlling the camera 30.

By repeating this loop process 1, control parameters are specified for each field of view change function of the camera 30.

Then, the camera control unit 15E transmits a remote control command to the camera 30, in which control parameters are specified for each function of changing the field of view of the camera 30 (step S109).

The camera control unit 15E then determines whether the coordinates of the PTZ satisfy the transition condition to the next section defined in the section corresponding to the value of the loop counter m in the section setting data 13A (step S110).

At this time, if the PTZ coordinates satisfy the transition condition to the next section (step S110: Yes), the camera control unit 15E increments the loop counter m (step S111). After that, the camera control unit 15E obtains the section setting corresponding to the value of the loop counter m from the section setting data 13A (step S112), and proceeds to the processing of step S103.

If the coordinates of the PTZ do not satisfy the transition condition to the next section (No in step S110), the processing proceeds to step S103 without executing steps S111 and S112.

<One aspect of the effect>
As one aspect, the information processing device 10 according to the present embodiment allows pan control only in a predefined direction when controlling the pan of the camera 30 according to the position of the object in the image captured by the camera 30. This makes it possible to prevent the pan movement direction from switching between the forward direction and the opposite direction. Therefore, the camera control function according to the present embodiment makes it possible to realize smooth pan control of the camera 30.

As another aspect, when the information processing device 10 according to this embodiment controls the pan of the camera 30 according to the position of the subject in the image captured by the camera 30, if the pan coordinates of the camera 30 are within a predetermined angle range, pan control that moves at a predefined speed is permitted. This makes it possible to prevent the pan movement direction from switching between the forward direction and the opposite direction. Therefore, the camera control function according to this embodiment makes it possible to realize smooth pan control of the camera 30.

Furthermore, the camera control function according to this embodiment realizes automatic shooting with smooth pan control of the camera 30, eliminating the need for personnel for shooting. Therefore, the location where the camera 30 is installed is not restricted by the ease of access for the photographer or the space available for the photographer to take pictures. As a result, it is possible to install the camera 30 in a location where the camera angle is favorable or where occlusion due to the facilities for the competition or spectators is unlikely to occur.

Example 2
Although the embodiments of the present disclosure have been described above, various applications are possible, and further, the present disclosure may be embodied in various different forms other than the above-described first embodiment.

<Numeric values, etc.>
The matters described in the first embodiment above, such as the types of control parameters and the number of cameras 30, are merely examples and may be changed. In addition, the order of processes in the flowcharts described in the embodiments may be changed within a range that does not cause inconsistencies.

<Application Examples>
For example, in the above embodiment 1, one camera 30 is used to shoot the course C, but two or more cameras 30 can be used to shoot the horse race broadcast. FIG. 10 is a schematic diagram showing an application example of section division. FIG. 10 shows an example in which two cameras 30, camera 30A and camera 30B, are used to shoot the horse race broadcast. In this case, as shown in FIG. 10, camera 30A can be assigned to shoot sections 1 to 4 and sections 7 to 8 out of sections 1 to 8, and camera 30B can be assigned to shoot sections 5 and 6. By allocating sections in this way, close-up photography of the racehorses can be achieved over the entire course C, and occlusion caused by facilities for the race or spectator, such as large display boards, can be suppressed.

<System>
The information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed arbitrarily unless otherwise specified. For example, any one or more of the functional units among the image acquisition unit 15A, the object detection unit 15B, the control amount calculation unit 15C, the camera control unit 15E and the output control unit 15F may be configured as separate devices.

Furthermore, each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. In other words, the specific form of distribution and integration of each device is not limited to that shown in the figure. In other words, all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc. Each configuration may also be a physical configuration.

Furthermore, each processing function performed by each device can be realized, in whole or in part, by a CPU (Central Processing Unit) and a program that is analyzed and executed by the CPU, or can be realized as hardware using wired logic.

<Hardware>
Next, a hardware configuration example of the computer described in the above embodiment will be described. Fig. 11 is a diagram showing the hardware configuration example. As shown in Fig. 11, the information processing device 10 has a communication device 10a, a storage device 10b, a memory 10c, and a processor 10d. Note that each unit shown in Fig. 11 may be connected to each other by a bus or the like.

The communication device 10a is a network interface card or the like. The storage device 10b is a storage device such as a hard disk drive (HDD) or a solid state drive (SSD). For example, the storage device 10b stores programs and databases that realize the functions shown in FIG. 4.

The processor 10d reads a program that executes the same processing as the processing unit shown in FIG. 4 from the storage device 10b, etc., and loads it into the memory 10c, thereby operating a process that executes the functions described in FIG. 4.

Such a process realizes the same functions as the processing units of the information processing device 10. For example, the processor 10d reads out a program having the same functions as the image acquisition unit 15A, the object detection unit 15B, the control amount calculation unit 15C, the camera control unit 15E, the output control unit 15F, etc. from the storage device 10b, etc. Then, the processor 10d executes a process that executes the same processing as the image acquisition unit 15A, the object detection unit 15B, the control amount calculation unit 15C, the camera control unit 15E, the output control unit 15F, etc.

In this way, the information processing device 10 operates as an information processing device that executes a calculation method by reading and executing a program. The information processing device 10 can also realize functions similar to those of the above-mentioned embodiment by reading the program from a recording medium using a media reading device and executing the read program. Note that the program in these other embodiments is not limited to being executed by the information processing device 10. For example, the present invention can be similarly applied to cases where another computer or server executes a program, or where these cooperate to execute a program.

The above program can be distributed via a network such as the Internet. The above program can also be recorded on any recording medium and executed by a computer by reading it from the recording medium. For example, the recording medium can be a hard disk, a flexible disk (FD), a CD-ROM, an MO (Magneto-Optical disk), a DVD (Digital Versatile Disc), etc.

REFERENCE SIGNS LIST 1 camera control system 10 information processing device 11 communication control unit 13 storage unit 13A section setting data 15 control unit 15A image acquisition unit 15B object detection unit 15C control amount calculation unit 15E camera control unit 15F output control unit 30 camera

Claims (7)

Acquire the image captured by the camera,
When controlling the pan of the camera in accordance with the position of the photographing object in the photographed image, pan control is permitted only in a predefined direction.
A camera control program that causes a computer to execute a process. The camera control program according to claim 1, characterized in that the permission process includes a process of permitting pan control only in a direction associated with a section corresponding to the pan coordinates of the camera, among setting data that specifies the pan angle range corresponding to each section into which the course along which the subject moves and the directions in which the pan movement of the camera is permitted. The camera control program according to claim 1, further comprising causing the computer to execute a process of stopping the pan movement of the camera when the pan coordinates of the camera reach a goal position of the course along which the subject moves. The camera control program according to claim 1, characterized in that the permission process includes a process for permitting tilt control only in a predefined direction when controlling the tilt of the camera according to the position of the subject in the captured image. The camera control program according to claim 1, characterized in that the permission process includes a process for permitting zoom control only in a predefined direction when controlling the zoom of the camera according to the position of the subject in the captured image. Acquire the image captured by the camera,
When controlling the pan of the camera in accordance with the position of the photographing object in the photographed image, pan control is permitted only in a predefined direction.
A camera control method, the processing of which is executed by a computer. Acquire the image captured by the camera,
When controlling the pan of the camera in accordance with the position of the photographing object in the photographed image, pan control is permitted only in a predefined direction.
1. An information processing device comprising: a control unit that executes processing.