WO2019044345A1 - Railway equipment monitoring device and railway equipment monitoring system - Google Patents

Abstract

Provided are a railway equipment monitoring device and railway equipment monitoring system capable of suppressing influence from changes in external environmental conditions and displaying the state of a pantograph so as to be easy for observer monitoring personnel to see. This railway equipment monitoring device 1 comprises: a stereo camera 11 that comprises two cameras (11a, 11b) and is for capturing images of a pantograph 9 that is a subject, a camera rotation and holding unit 12 for rotatably holding the stereo camera 11, and a control and calculation unit 10 for making the stereo camera 11 shift to each prescribed rotation angle, generating a distance image of the pantograph 9 from the captured images obtained at each rotation angle, and determining the angle by which the camera rotation and holding unit 12 is to rotate the stereo camera 11 on the basis of the distance image of the pantograph.

Description

Railway equipment monitoring device and railway equipment monitoring system

The present invention relates to a railway facility monitoring apparatus for monitoring a pantograph, which is a railway vehicle facility (hereinafter referred to as a railway facility), and in particular, a railway that can improve the acquisition degree of distance information between a stereo camera and a pantograph. The present invention relates to a facility monitoring device and a railway facility monitoring system.

The railway operators deal with maintenance of structures such as tunnels and facilities such as overhead lines and tracks mainly by work by supervisors, etc., and require a certain cost. Efforts are being made to reduce costs using various sensors, IoT (Internet of Things), etc. as a measure for cost reduction. As an example of the approach, there is monitoring of a pantograph installed on a train (Non-Patent Document 1). In Non-Patent Document 1, with regard to pantograph monitoring, the optical axes of two cameras capable of imaging pantographs are installed in the upper part of a railway vehicle in a non-parallel state, and after obtaining an image including pantographs, pantograph using pattern matching To detect However, in addition to pantographs, railway facilities (overhead lines) are also targets to be monitored, and "an image extracted from only pantographs" that is easy for the observer to view is not provided. Also, the execution of pattern matching generally increases the computation time.
With regard to detection of railway equipment without pattern matching, the optical axes of two cameras with the same installation height mounted on the top of the leading vehicle of a railway inspection vehicle are made parallel, and then the railway equipment etc. A method of ranging is proposed (Non-Patent Document 2). In Non-Patent Document 2, there is a possibility that an incomplete three-dimensional map may be generated because the distance measurement of a railway installation parallel to a line (baseline) connecting the centers of two cameras is insufficient.
In order to solve such a problem, for example, a technology described in Patent Document 1 has been proposed. In Patent Document 1, a camera with an optical axis parallel to the binocular part of the robot is installed, distance measurement is performed by a stereo method using two cameras, and the stereo camera is horizontal to the ground and parallel to the camera. If sufficient distance measurement can not be performed for the object or area, the robot's neck is tilted, that is, the object or area parallel to the stereo camera horizontal to the ground is measured by rotating the stereo camera in the roll angle direction. It is possible.

Japanese Patent Application Publication No. 2003-200377

Niwakawa et al., "Measurement of pantographs and overhead wires of Kyushu Shinkansen using stereo analysis method," Journal of the Institute of Electrical Engineers of Japan, Vol.27, No.2, pp.118-123, 2007 Fukai et al., “A Method of Automatic Measurement of Building Limits Using Image Processing,” Railway and Electrical Technology, Vol. 27, No. 2, pp. 44-48, February 2016

In Patent Document 1, when rotating a stereo camera, the camera is rotated to the same angle (single angle). At this time, it is described that it is suitable to rotate at once to the stopper position attached beforehand. The reason for this is to facilitate calculation and to improve ranging accuracy. It can be easily imagined that the robot's head tilt angle can be stably maintained by rotating it to the angle of the stopper position, but it is impossible to measure the distance in the camera non-rotation state with a single rotation angle. Not all feature points in the image captured by the camera can be associated, ie, it may lead to poor ranging results.
In Patent Document 1, a captured image is obtained by a stereo camera at a single rotation angle. However, the external environmental conditions (weather, time, amount of solar radiation, day / night, etc.) change, and there is a possibility that the ranging accuracy may be reduced at a single rotation angle due to the change of the external environmental conditions. In other words, Patent Document 1 does not consider any change in external environmental conditions.

Therefore, the present invention provides a railway equipment monitoring apparatus and a railway equipment monitoring system capable of suppressing the influence of changes in external environmental conditions and displaying the state of the pantograph in a manner easily visible to a supervisor.

In order to solve the above problems, a railway facility monitoring apparatus according to the present invention includes a stereo camera configured to capture a pantograph as a target object, and a camera rotation holding unit configured to rotatably hold the stereo camera. The stereo camera is changed for each predetermined rotation angle, and a distance image of a pantograph is generated from captured images obtained at each rotation angle, and the rotation angle of the stereo camera by the camera rotation holding unit based on the distance image of the pantograph And a control operation unit that determines
Further, a railway facility monitoring system according to the present invention comprises a railway facility monitoring device, an electronic terminal installed on a ground facility including an operation control center, and a communication network for communicably connecting these to each other, the railway facility The monitoring device is composed of two cameras, and changes a stereo camera that captures a pantograph that is a target object, a camera rotation holding unit that rotatably holds the stereo camera, and changes the stereo camera for each predetermined rotation angle. And generating a distance image of a pantograph from captured images obtained at respective rotation angles, and determining a rotation angle of the stereo camera by the camera rotation holding unit based on the distance image of the pantograph. I assume.

ADVANTAGE OF THE INVENTION According to this invention, it becomes possible to suppress the influence by the change of external environmental conditions, and to provide the railway installation monitoring apparatus and the railway installation monitoring system which can display the state of a pantograph in a legible manner to a supervisor.
Problems, configurations, and effects other than those described above will be apparent from the description of the embodiments below.

BRIEF DESCRIPTION OF THE DRAWINGS It is a whole schematic block diagram of the railway installation monitoring system of Example 1 which concerns on one Example of this invention. It is the schematic which shows the positional relationship of the stereo camera and pantograph shown in FIG. 1, and a pantograph existing range. It is a figure which shows an example of the original image of the pantograph imaged with the stereo camera. It is a side view of the camera rotation holding part which mounts a stereo camera shown in FIG. 1, and a stereo camera. It is a front view of the camera rotation holding part which mounts a stereo camera shown in FIG. 1, and a stereo camera. It is a figure which shows an example of the distance image which represented the distance data of a pantograph which was produced | generated by the stereo process part shown in FIG. 1 by the gray scale. It is a figure which shows an example of the pantograph distance image which carried out the mask process of the outside of the existence range of a pantograph with a distance filter produced | generated by the target object extraction part shown in FIG. It is a figure which shows an example of the silhouette image of a pantograph obtained by performing a binarization process to the pantograph distance image which carried out the mask process by the distance filter by the object extraction part shown in FIG. It is a figure which shows an example of the data structure stored in the memory | storage part shown in FIG. 1 which matches a camera rotation angle and the number of distance measurement possible pixels. It is a conceptual diagram of the process which obtains the image from which only the pantograph was extracted by the target object extraction part shown in FIG. It is a flowchart which shows the processing flow performed by the railway installation monitoring apparatus shown in FIG. 1, Comprising: It is a flowchart which sets camera rotation angle (theta) to an optimal value immediately after train starting. It is a flowchart which shows the processing flow performed by the railway installation monitoring apparatus shown in FIG. 1, Comprising: It is a flowchart which changes camera rotation angle (theta) to an optimal value, when illumination intensity changes during driving | running | working. The example of the data structure which matches the camera rotation angle and the similarity with respect to the pantograph of a distance measurement possible pixel stored in the memory | storage part which comprises the railway installation monitoring apparatus of Example 2 which concerns on the other Example of this invention is shown. FIG.

In this specification, a railway facility monitoring device that extracts a pantograph-only image from an image captured by the stereo camera using a stereo camera whose “optical axis is parallel” and “the positions of two cameras are horizontal” Although the railway installation monitoring system is described as an example, the configuration of the stereo camera itself is not limited to this.
Further, in the present specification, “granularity of camera rotation angle” refers to the predetermined rotation angle in the case of imaging by rotating a stereo camera by a predetermined rotation angle, in other words, one unit of rotation angle division as the particle size It is called.
Further, in the present specification, a vehicle on which the railway facility monitoring device is installed may be referred to as a "train vehicle" or a "train".
Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is an overall schematic configuration diagram of a railway facility monitoring system according to a first embodiment of the present invention. As shown in FIG. 1, the railway facility monitoring system 100 includes a railway facility monitoring device 1, an electronic terminal 2 or server installed on a ground facility such as an operation control center, and a communication network 3 communicably connecting them. It consists of. Although FIG. 1 shows an example in which one railway equipment monitoring device 1 is communicably connected to the electronic terminal 2 via the communication network 3 for convenience of explanation, the railway equipment monitoring device 1 described later in detail will be described as one organization. The ground facilities such as the operation control center where the electronic terminal 2 or the server is installed are installed in each of a plurality of train cars to be configured and communicate with a plurality of organizations via the communication network 3 It is possible to be connected. The communication network 3 may be wired or wireless.

<Configuration of Railway Equipment Monitoring Device 1>
As shown in FIG. 1, the railway facility monitoring device 1 mounts and holds the stereo camera 11 and the stereo camera 11 whose “optical axes of two cameras are parallel” and “the two cameras are horizontal”. A camera rotation holding unit 12, a control calculation unit 10, an image display unit 16, and an input unit 19 are provided. In addition, an illuminance meter 20 for measuring external environmental conditions (weather, solar radiation amount, day / night, etc.), and a position information notification module 30 for measuring position information of a train car are connected to the railway equipment monitoring device 1 It is done. Here, as the position information notification module 30, for example, a GPS (Global Positioning System) is used.
The control operation unit 10 performs distance measurement according to a stereo method based on triangulation using an original image captured by the stereo camera 11 as a stereo method, and generates a distance image by the stereo processing unit 13. The distance image and stereo by the stereo processing unit 13 An object extraction unit 14 that generates an image in which only a pantograph that is an object is extracted based on an original image captured by the camera 11, and a rotation correction unit that rotates and corrects an image in which only the pantograph is extracted by the object extraction unit 14 A communication control unit 18 for transmitting an image in which only the pantograph after rotation correction is extracted to the electronic terminal 2 via the communication network 3 and the storage unit 17 is provided. The stereo processing unit 13, the object extraction unit 14, the rotation correction unit 15, and the communication control unit 18 are, for example, a processor such as a CPU, a ROM that stores a program, and a process in which the processor executes a program read from the ROM. This is realized by a storage device such as a RAM that temporarily stores data and the like.

FIG. 2 is a schematic view showing the positional relationship between the stereo camera 11 and the pantograph shown in FIG. 1 and the pantograph existence range, and FIG. 3 is a view showing an example of the original image of the pantograph captured by the stereo camera. . As shown in FIG. 2, the railway equipment monitoring device 1 has a stereo camera 11 and a camera rotation holding unit 12 (not shown in FIG. 2) that holds the stereo camera 11 and rotatably holds the upper portion of the train car. Is installed on the roof structure, and the pantograph 9 of the train car is installed facing the pantograph 9 so as to be capable of imaging a state in contact with the trolley wire (not shown in FIG. 2) and separated by a predetermined distance. . In a state where the pantograph 9 of the train car is in contact with the trolley wire, the stereo camera 11 acquires an original image of the pantograph 9 (in FIG. 3 also, the trolley wire is not shown) as shown in FIG.
The control calculation unit 10, the image display unit 16, and the input unit 19 that constitute the railway equipment monitoring device 1 are installed in the train car.

The detailed configuration of each part will be described below.

[Camera rotation holding unit 12]
FIG. 4 is a side view of the stereo camera 11 shown in FIG. 1 and the camera rotation holding unit 12 on which the stereo camera 11 is mounted. As shown in FIG. 4, the camera rotation holding unit 12 has a mount member 121 on which the stereo camera 11 can be mounted, and the stereo camera 11 with the center of a baseline described later installed on the mount member 121 as a rotation axis. And a rotation information recording unit 123 for outputting a signal that is a predetermined rotation angle command to the rotation control mechanism 122 via the signal line L31. The mount member 121 is made of, for example, stainless steel, and is fixed, for example, by bolt fastening to a housing (not shown) that accommodates two cameras constituting the stereo camera 11 so that the optical axis is parallel to the horizontal. The rotation control mechanism 122 rotates the mount member 121 at a predetermined rotation angle by, for example, a servomotor (not shown) that configures the rotation control mechanism 122 in the roll angle direction specified by the signal from the rotation information recording unit 123 Hold that attitude. In addition, the rotation information recording unit 123 is stored in a storage device such as a processor such as a CPU, a ROM storing a program, a RAM temporarily storing data etc. of the process of executing the program read from the ROM. Is realized.
FIG. 5 is a front view of the camera rotation holding unit 12 mounting the stereo camera 11 and the stereo camera 11 shown in FIG. The upper view of FIG. 5 shows a state in which the rotation angle θ is 0 °. As shown in the upper drawing of FIG. 5, the centers of the right camera 11a and the left camera 11b are positioned on the baseline (line connecting the centers of two cameras), that is, the right camera 11a and the left camera 11b are positioned horizontally. And their respective optical axes are parallel to one another. When the pantograph 9 is in contact with the trolley wire, the camera rotation angle θ = 0 ° as shown in the upper part of FIG. 5 immediately after the start of the railway equipment monitoring device 1.
The lower part of FIG. 5 shows a state in which the mount member 121 is rotated clockwise by the roll rotation angle θ with the center of the baseline as the rotation axis and held. As described above, the mount member 121 can be rotated by the rotation control mechanism 122 about the center of the baseline as an axis of rotation in an arbitrary roll rotation angle direction, and is held with the stereo camera 11 rotated by a predetermined rotation angle θ. Ru. When the pantograph 9 is in contact with the trolley wire, the camera rotation angle θ ≠ 0 ° as shown in the lower diagram of FIG. It becomes.
In addition to outputting a signal that is a predetermined rotation angle command to the rotation control mechanism 122 via the signal line L31, the rotation information recording unit 123 can perform distance measurement, which will be described in detail later, at least for each rotation angle of the stereo camera 11. Has a function of recording the number of pixels, and the predetermined number of storage areas of the storage unit 17 which forms the control calculation unit 10 via the signal line L10 (FIG. 1). Store in The storage unit 17 is configured of at least one or more storage media.
Further, the stereo camera 11 mounted on the mount member 121 constituting the camera rotation holding unit 12 captures two original images captured and captured by the right camera 11a and the left camera 11b at a predetermined cycle, as shown in FIG. As shown in FIG. 4, the rotation information recording that configures the camera rotation holding unit 12 to the stereo processing unit 13 via the signal line L1, to the object extraction unit 14 via the signal line L2, and via the signal line L8. Output to the part 123.

[Stereo processing unit 13]
As shown in FIG. 1, the stereo processing unit 13 constituting the control calculation unit 10 is an original image captured by the stereo camera 11 held by the camera rotation holding unit 12 in a posture rotated at a predetermined rotation angle. (Stereo original image) is input through the signal line L1. Then, the stereo processing unit 13 performs ranging to the input original image (stereo original image) by the stereo method based on triangulation, generates a distance image, and generates a distance image generated via the signal line L3 as an object It is output to the extraction unit 14. Here, an example of the distance image will be described. FIG. 6 is a view showing an example of a distance image in which distance data of a pantograph is represented in grayscale, which is generated by the stereo processing unit 13 shown in FIG. As shown in FIG. 6, the distance image (the pantograph distance image) is generated as a grayscale image that is displayed brighter (higher luminance) near the stereo camera 11 and darker (lower luminance) as it goes away. As can be seen by comparison with the original image (original pantograph image) shown in FIG. 3, the distance image (pantograph distance image) generated as a gray scale image is the image of the stereo camera 11 and the pantograph 9 shown in FIG. It is an image that reflects the distance between them.

The distance image generated by the stereo processing unit 13 will be described in more detail. Disparity occurs in the respective images (stereo original images) acquired by the stereo camera 11 (right camera 11 a and left camera 11 b) in which two horizontal and parallel optical axes are parallel. The distance between the stereo camera 11 and the pantograph 9 which is the target object is measured based on the amount of displacement (the number of pixels in the image data) due to the parallax. Therefore, the distance image (pantograph distance image) is an image including information of the measured distance. As shown in FIG. 1, the stereo processing unit 13 receives characteristic points from the original image captured by the right camera 11 a and the original image captured by the left camera 11 b, which are input from the stereo camera 11 via the signal line L 1. Extract and associate feature points called disparity calculation. Since the stereo camera 11 has "the positions of the two cameras (right camera 11a and left camera 11b) are horizontal" and "the optical axes of the two cameras (right camera 11a and left camera 11b) are parallel", the correspondence of feature points In addition, a parallax image is obtained by calculating the difference (parallax) of horizontal coordinates at which each feature point pixel exists in the original image captured by the right camera 11a and the original image captured by the left camera 11b. Based on the principle of triangulation using each pixel value (parallax value) in this parallax image and the specifications (baseline length, focal length) of the stereo camera 11, the distance to the target object present in front of the stereo camera 11 is shown. A distance image can be acquired.

The above-described distance image (pantograph distance image) is not limited to the gray scale image. For example, the distance image (pantograph distance image) may be generated as a color gradation image represented as a color gradation such as “red” as the portion closer to the stereo camera 11 and “blue” or “black” as the distance increases. good.

[Object extraction unit 14]
As shown in FIG. 1, the object extraction unit 14 constituting the control calculation unit 10 generates a distance filter for masking a value other than the existence range of the target object which can be set by the stereo processing unit 13. It is applied to the image (pantograph distance image) to extract the coordinates of the pixel where the distance information of the target object exists, and the binary object existing area image (white represents the existing area) composed of the target object existing area The multiplication with the original image input from the stereo camera 11 via the signal line L2 is performed to generate an image in which only the target object is extracted.

More specifically, as shown in FIG. 1, the object extraction unit 14 generates an image other than the existing region of the pantograph 9 shown in FIG. 2 with respect to the distance image input from the stereo processing unit 13 via the signal line L3. Apply the distance filter (only the distance information between near and far is valid) to mask. That is, the “distance filter” is a filter that extracts only image data within the depth range from [near] to [far] shown in FIG. The positions of [near] and [far] in the distance filter can be appropriately set to desired values through the input unit 19 shown in FIG. 1, and the positions of [near] and [far] in the set distance filter Information on the position is stored in a predetermined storage area of the storage unit 17 via the signal line L11. Therefore, when applying the distance filter, the object extraction unit 14 reads the information on the positions of [near] and [far] stored in the storage unit 17 through a signal line (not shown). FIG. 7 is a view showing an example of a pantograph distance image which is generated by the object extraction unit 14 shown in FIG. 1 and which is mask-processed by using a distance filter outside the pantograph existence range. The object extraction unit 14 applies a distance filter to the distance image input from the stereo processing unit 13 described above by performing mask processing on areas other than the area where the pantograph 9 shown in FIG. 2 is present, as shown in FIG. A distance image to which a distance filter has been applied, which includes pixels indicating the distance from the stereo camera 11 to the pantograph 9, is obtained.

Thereafter, the object extraction unit 14 generates a silhouette image of the object in the pantograph existing region as shown in FIG. 8 by performing binarization processing on the distance image to which the distance filter has been applied (FIG. 7). . Here, in the binarization processing, in the distance image to which the distance filter has been applied (distance image data), “1” (white) is present for pixels for which an effective distance value is present, and “0 for Assign "(black). Note that a threshold may be provided for the distance value.
The object extraction unit 14 holds the camera rotation image of the object in the pantograph present region obtained by performing the binarization processing shown in FIG. 8 through the signal line L4 as shown in FIGS. 1 and 4 The information is output to the rotation information recording unit 123 constituting the unit 12. In the rotation information recording unit 123, the effective pixel ("1" (white) is the effective pixel and "0" (black) is present in the silhouette image of the object in the pantograph existing region input through the signal line L4). The invalid pixels are counted, and the information of the camera rotation angle corresponding to the counted value (the number of distance measurement possible pixels) is stored in a predetermined area of the storage unit 17 of the control calculation unit 10 via the signal line L10. FIG. 9 is a diagram showing an example of a data structure stored in the storage unit 17 shown in FIG. 1 and associating the camera rotation angle with the number of distance measurement enabled pixels. As illustrated in FIG. 9, the storage unit 17 associates (associates with) the camera rotation angle θ [°] with the number of pixels capable of distance measurement. For example, when “camera rotation angle θ [°]” is “0”, “distance measurement possible pixel count” is “625”, and “camera rotation angle θ [°]” is “15” “distance measurement possible The "number of pixels" is "777", and the "number of pixels capable of distance measurement" is "900" when the "camera rotation angle θ [°]" is "30", and the "camera rotation angle θ [°]" is " In the case of 45 "," the number of pixels capable of distance measurement "is" 625 ". Also, when the "camera rotation angle θ [°]" is "60", the "distance measurement possible pixel number" is "888", and when the "camera rotation angle θ [°]" is "75""distance measurement is possible The “number of pixels” is “800”, and the “number of pixels capable of distance measurement” is “700” when the “camera rotation angle θ [°]” is “90”. As described above, the number of pixels capable of distance measurement when the rotation angle of the stereo camera 11 is sequentially changed every predetermined rotation angle of 15 ° (grain size) is stored in association. In addition, in the case where there is no change in the external environmental conditions, generally, 45 ° is the optimum camera rotation angle of the stereo camera 11. However, in reality, the optimum camera rotation angle of the stereo camera 11 deviates from 45 ° according to the change of the external environmental conditions. Therefore, with the particle size set to 15 °, the camera rotation angle of the stereo camera 11 is changed to count the number of pixels capable of distance measurement. The particle size is not limited to 15 °, and for example, the particle size may be set to 5 °.

FIG. 10 is a conceptual view of processing for obtaining an image in which only the pantograph 9 is extracted by the object extraction unit 14 shown in FIG. After generating a silhouette image of an object in the pantograph existing region, the object extraction unit 14 receives the original image (FIG. 3) and the silhouette image of the pantograph 9 captured by the stereo camera 11 input through the signal line L2. By multiplying with, as shown in FIG. 10, an image in which only the pantograph 9 is extracted is generated. As described above, in the silhouette image, "1" (white) is assigned to the pixels corresponding to the area of the pantograph 9, and "0" (black) is assigned to the pixels corresponding to the areas other than the pantograph 9. Since the pixels corresponding to the area other than the pantograph 9 become “0” by multiplying the silhouette image by the original image, the image in which only the pantograph 9 is extracted is obtained. The object extraction unit 14 outputs an image in which only the pantograph 9 shown in FIG. 10 is extracted to the rotation correction unit 15 via the signal line L5.

[Rotation correction unit 15]
As shown in FIG. 1, when the rotation correction unit 15 constituting the control calculation unit 10 captures an image of the target object in a posture in which the stereo camera 11 is rotated by a predetermined rotation angle, the image in which only the target object is extracted Correction processing of the rotation angle of the stereo camera 11 at the time of imaging is applied to generate an image after the reverse rotation correction in which only the target object having the same posture as in the case where the stereo camera 11 was photographed without rotation Do.
Specifically, the rotation correction unit 15 inputs an image (a pantograph extraction image) from which only the pantograph 9 generated by the object extraction unit 14 is extracted via the signal line L5, and also via the signal line L6. The camera rotation angle θ of the stereo camera 11 is input from the rotation information recording unit 123 constituting the camera rotation holding unit 12. If the input camera rotation angle θ is not zero, the rotation correction unit 15 performs reverse rotation correction processing on the pantograph-extracted image by the camera rotation angle θ to generate an image that is easy for the monitoring personnel to see. Here, for example, processing such as affine transformation is used as the reverse rotation correction processing. Note that, if the camera rotation angle θ = 0, the rotation correction unit 15 does not execute the reverse rotation correction process.
The rotation correction unit 15 stores the image after reverse rotation correction processing (image after reverse rotation correction) in a predetermined storage area of the storage unit 17 via the signal line L9, and the image display unit via the signal line L7. 16 and the communication control unit 18 respectively.

[Image display unit 16]
The image display unit 16 displays the reverse rotation corrected image input from the rotation correction unit 15 via the signal line L7 on the screen. As a result, the train crew can easily grasp the state of the pantograph 9.

[Communication control unit 18]
As shown in FIG. 1, the communication control unit 18 constituting the control calculation unit 10 can obtain the reverse rotation corrected image input from the rotation correction unit 15 via the signal line L7 from the position information notification module 30 The position information of the train car is transmitted to the electronic terminal 2 or server installed in the ground facility such as the operation control center via the communication network 3. As a result, the monitoring personnel in the ground facility such as the operation control center can easily grasp the state of the pantograph 9.
<Operation of Railway Equipment Monitoring Device 1>
The operation of the railway facility monitoring device 1 will be described below. In particular, the whole process and operation by the railway facility monitoring device 1 will be described including the search process of the optimum value of the camera rotation angle of the stereo camera 11 which can suppress the influence by the change of the environmental condition. FIG. 11 is a flowchart showing the processing flow executed by the railway facility monitoring device 1 shown in FIG. 1, and is a flowchart for setting the camera rotation angle θ to an optimum value immediately after the start of the train.
As shown in FIG. 11, in step S11, a train equipped with the railway equipment monitoring device 1 is activated in a garage.
In step S12, the camera rotation angle θ of the stereo camera 11 is 0 ° when the railway equipment monitoring device 1 is activated. In this state, the rotation information recording unit 123 constituting the camera rotation holding unit 12 counts the effective pixels present in the above-mentioned silhouette image, and records the total value as the initial value of the optimum value of the number of distance measurement enable pixels. .

In step S13, the rotation information recording unit 123 of the camera rotation holding unit 12 outputs a signal for rotating the mount member 121 in the roll angle direction by a predetermined camera rotation angle (here, + 15 °) as the signal line L31. And the rotation control mechanism 122 that constitutes the camera rotation holding unit 12. Then, the rotation information recording unit 123 counts the effective pixels present in the silhouette image described above, records the total value as the number of distance measurement possible pixels in association with the camera rotation angle θ, and via the signal line L10. It is stored in a predetermined storage area of the storage unit 17 of the control calculation unit 10. Further, the rotation information recording unit 123 outputs the camera rotation angle θ to the rotation correction unit 15 via the signal line L6.

In step S14, it is determined whether or not the camera rotation angle θ has reached 90 °. If the camera rotation angle θ has not reached 90 °, the process returns to step S13 and a predetermined camera rotation angle (15 °) is further added to perform similar processing. repeat. On the other hand, if the camera rotation angle θ has reached 90 °, the process proceeds to step S15.
In step S15, the rotation information recording unit 123 determines the camera rotation angle θ at which the number of distance measurement possible pixels, which is a total value, is maximized. Then, after the rotation information recording unit 123 determines the camera rotation angle θ of the stereo camera 11 based on the optimum value (the maximum number of distance measurement possible pixels), the camera rotation at which the total value (the number of distance measurement possible pixels) becomes the maximum. A signal for holding the mount member 121 at the angle θ is output to the rotation control mechanism 122 via the signal line L31.

In step S16, the rotation control mechanism 122 rotates the mount member 121 in the roll angle direction so as to be the camera rotation angle θ determined in step S15, which is input from the rotation information recording unit 123 via the signal line L31. The attitude of the camera 11 is maintained. At this time, the measured value by the illuminance meter 20 shown in FIG. 1 is stored in the storage unit 17 as a traveling illuminance value.

In step S18, the train leaving the garage moves to the first train station and proceeds to step S21 described later.
FIG. 12 is a flowchart showing a process flow executed by the railway facility monitoring device 1 shown in FIG. 1, and is a flowchart for changing the camera rotation angle θ to an optimal value when the illuminance changes during traveling. Hereinafter, the change of the camera rotation angle θ of the train that has left the station will be described according to the process flow of FIG.
As shown in FIG. 12, in step S21, the train which has left the garage and arrived at the first train station starts traveling.

In step S22, it is determined whether the traveling illumination value, which is a measurement value by the illumination meter 20 shown in FIG. 1, deviates by a predetermined value or more for a predetermined time or more. If the result of the determination is that there is no divergence, the process proceeds to step S25 to continue traveling. On the other hand, if it is determined that the traveling illuminance value deviates by a predetermined value or more as a result of the determination, the process proceeds to step S23.

In step S23, it is determined based on the position information indicated by the position information notification module 30 shown in FIG. 1 whether or not a tunnel, an underground, or an overpass is being traveled. If it is determined that the vehicle is traveling in a tunnel, underground, or overpass, the process proceeds to step S24, continues to step S25 without changing the camera rotation angle θ of the stereo camera 11 of the train, and continues traveling. Here, the reason for not changing the camera rotation angle θ of the stereo camera 11 is that when the pantograph 9 is separated from the trolley wire while the train is traveling in a tunnel or underground, an arc (a spark is generated) can be captured and the arc This is because it can be determined by recording that it is a section in which the trolley wire should be maintained. On the other hand, when the tunnel, the underground, and the overpass are not traveled, the process proceeds to step S26.

In step S26, the above-described steps S12 to S16 shown in FIG. 11 are executed while continuing the train traveling.
In step S27, the traveling illuminance value stored in the storage unit 17 constituting the control calculation unit 10 is updated, and the process returns to step S22, and the same processing is repeatedly executed.
The railway equipment monitoring apparatus 1 executes the process flow of FIG. 11 and FIG. 12 described above, and thereby, the optimal stereo camera 11 according to the change of the external environmental conditions (weather, time, solar radiation amount, day / night etc.). The camera rotation angle θ can be determined, and the influence of a change in external environmental conditions can be suppressed, and the state of the pantograph 9 can be displayed so as to be easily seen by a supervisor.

As described above, according to the present embodiment, it is possible to provide a railway equipment monitoring apparatus and a railway equipment monitoring system capable of suppressing the influence of changes in external environmental conditions and displaying the state of the pantograph in a manner easily visible to the observer. .

FIG. 13 is a data structure that is stored in the storage unit of the railway facility monitoring apparatus of the second embodiment according to the second embodiment of the present invention, and associates the camera rotation angle with the similarity to the distance measurable pixel with the pantograph. Is a diagram illustrating an example of The configuration itself of the railway equipment monitoring system 100 and the railway equipment monitoring device 1 is the same as that of the above-described first embodiment, but the data structure stored in the storage unit 17 constituting the control calculation unit 10 and the camera rotation holding unit 12 The processing content of the rotation information recording unit 123 to be configured is different from that of the first embodiment. The differences from the first embodiment will be described below.

The rotation information recording unit 123 (FIG. 4) constituting the camera rotation holding unit 12 in the present embodiment is a pantograph present area generated by the object extraction unit 14 constituting the control calculation unit 10 inputted through the mark L4. The following processing is executed instead of totaling the effective pixels present in the silhouette image of the object inside.
The rotation information recording unit 123 according to the present embodiment is effective in existing in the silhouette image of the object in the pantograph existing area generated by the object extraction unit 14 which is input through the signal line L4 and which constitutes the control operation unit 10. The original image of the pantograph 9 shown in FIG. 3 (camera rotation angle θ shown in FIG. 3 input from the stereo camera 11 via the signal line L8 shown in FIGS. 1 and 4, and the shape of the pantograph 9 in the silhouette image. Compare the shape of the original image (captured with = 0). As a result of comparison, the camera rotation angle θ at which the similarity of the shape of the effective pixel present in the silhouette image to the original image of the pantograph 9 is maximum is determined as the optimal camera rotation angle θ of the stereo camera 11. In other words, the rotation information recording unit 123 uses the original image of the pantograph 9 captured by the stereo camera 11 when the camera rotation angle θ is 0 ° as a template, and is within the silhouette image obtained at each camera rotation angle. Pattern matching processing is performed on the shape of the pantograph 9, and the similarity to the pantograph is determined for each camera rotation angle. The calculated similarity to the pantograph for each camera rotation angle is stored in a predetermined storage area of the storage unit 17 via the signal line L10 shown in FIG.

As illustrated in FIG. 13, the storage unit 17 stores (associates) the camera rotation angle θ [°] and the similarity to the pantograph in association with each other. For example, when “camera rotation angle θ [°]” is “0”, “similarity to pantograph” is “0.625”, and when “camera rotation angle θ [°]” is “15” “for pantograph "Similarity" is "0.777", and when "camera rotation angle θ [°]" is "30", "similarity to pantograph" is "0.9", "camera rotation angle θ [°]" When "" is "45", the "similarity to pantograph" is "0.92." Also, when “camera rotation angle θ [°]” is “60”, “similarity to pantograph” is “0.85”, and when “camera rotation angle θ [°]” is “75” “for pantograph The similarity is "0.8", and the "similarity to the pantograph" is "0.7" when the "camera rotation angle θ [°]" is "90". In this manner, the similarity to the pantograph when the rotation angle of the stereo camera 11 is sequentially changed every predetermined rotation angle of 15 ° (grain size) is stored in association.

The operation of the railway facility monitoring apparatus 1 according to the present embodiment is the same as in the processing step in FIG. 11 described in the above-described first embodiment except that “number of pixels capable of distance measurement” is replaced with “similarity to pantograph”. To be executed. The same applies to FIG. 12 described in the first embodiment.

If the threshold of similarity is set in advance and the maximum "similarity to the pantograph" obtained as a result of the pattern matching process by the rotation information recording unit 123 described above is less than the threshold, the pantograph 9 which is the target object The communication control unit 18 in the control operation unit 10 of the railway facility monitoring apparatus 1 determines that there is a damage or a defect in the signal from the reverse rotation correction input from the rotation correction unit 15 via the signal line L7. The image is transmitted to an electronic terminal 2 or a server installed on a ground facility such as an operation control center via the communication network 3 together with the position information of the train car obtained from the position information notification module 30, and to the pantograph 9. Ground equipment such as a ground operation control center with a warning (message or alert) that there may be damage or loss Installed is transmitted to the electronic terminal 2. The surveillance staff visually checks the image after reverse rotation correction of the pantograph 9 displayed on the screen of the electronic terminal 12 according to the warning to judge the necessity of parts replacement or the preparation and maintenance of replacement parts. It becomes possible to respond promptly, such as arrangement of the worker who works.
In the present embodiment, although the original image of the pantograph 9 captured by the stereo camera 11 when the camera rotation angle θ is 0 ° is used as the template, the present invention is not limited to this. For example, the camera rotation angle of the stereo camera 11 is sequentially changed at predetermined camera rotation angle θ (for example, 15 °) intervals by a new camera pantograph 9 previously installed on the roof structure of a train car, and captured by the stereo camera 11 The original image of the pantograph 9 obtained may be used as a template for each predetermined camera rotation angle θ. In this case, a template corresponding to each predetermined camera rotation angle θ is stored in advance in the storage unit 17 so that the template corresponding to the camera rotation angle θ of the stereo camera 11 during operation of the vehicle is stored from the storage unit 17 The pattern matching process is executed by reading out.

As described above, according to the present embodiment, in addition to the effects of the first embodiment, it becomes possible to quickly grasp the presence or absence of breakage or loss of the target object pantograph 9 with the ground facility such as the operation control center. Arrangement of parts replacement or maintenance schedule (such as assignment of workers) can be made in advance. In addition, maintenance work (part replacement) is performed only when damage or loss of the pantograph 9 that is the target object occurs, although what has been maintenance work (part replacement etc.) every predetermined months until now. It becomes possible to set it as a form to carry out and the cost reduction in maintenance becomes possible.
The present invention is not limited to the embodiments described above, but includes various modifications. For example, the embodiments described above are described in detail in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those having all the configurations described. Also, part of the configuration of one embodiment can be replaced with the configuration of another embodiment, and the configuration of another embodiment can be added to the configuration of one embodiment.

DESCRIPTION OF SYMBOLS 1 ... Railway equipment monitoring apparatus, 2 ... Electronic terminal, 3 ... Communication network, 9 ... Pantograph, 10 ... Control arithmetic part, 11 ... Stereo camera, 11a ... Right camera, 11b ... Left camera, 12 ... Camera rotation holding part, 13 ... Stereo processing unit, 14 ... Object extraction unit, 15 ... Rotation correction unit, 16 ... Image display unit, 17 ... Storage unit, 18 ... Communication control unit, 19 ... Input unit, 20 ... Illuminance, 30 ... Position information notification Module, 100 ... railway equipment monitoring system, 121 ... mounting member, 122 ... rotation control mechanism, 123 ... rotation information recording unit

Claims (17)

A stereo camera composed of two cameras and imaging a pantograph as a target object;
A camera rotation holder for rotatably holding the stereo camera;
The stereo camera is changed for each predetermined rotation angle, and a distance image of a pantograph is generated from captured images obtained at each rotation angle, and the rotation angle of the stereo camera by the camera rotation holding unit is generated based on the distance image of the pantograph A railway facility monitoring device comprising: In the railway equipment monitoring device according to claim 1,
The two cameras have optical axes parallel and are arranged horizontally to each other,
The railway camera according to claim 1, wherein the camera rotation holding unit rotates the stereo camera in a roll rotation angle direction at each of the predetermined rotation angles, with the rotation axis as an approximate center of a baseline connecting the centers of the two cameras. Equipment monitoring equipment. In the railway equipment monitoring device according to claim 1,
Equipped with an image display unit,
The control calculation unit
A stereo processing unit that generates a distance image of a pantograph from captured images obtained at each of the rotation angles;
An object extraction unit which extracts the pantograph as the object and generates an image distinguishable from others based on the distance image of the pantograph generated by the stereo processing unit and the original image of the pantograph taken by the stereo camera When,
And a rotation correction unit that performs reverse rotation correction processing on an image that can be extracted from the pantograph generated by the object extraction unit and can be discriminated from others according to the determined rotation angle of the stereo camera. A railway facility monitoring device characterized by extracting a pantograph generated by the object extraction unit after reverse rotation correction processing by a rotation correction unit and displaying an image distinguishable from others on the image display unit. In the railway equipment monitoring device according to claim 3,
The railway facility monitoring device, wherein the object extraction unit has a distance filter that performs mask processing on the distance image of the pantograph generated by the stereo processing unit except for the region where the pantograph is present. In the railway equipment monitoring device according to claim 4,
It has an input unit and a storage unit,
Storing the application range of the distance filter set via the input unit in the storage unit;
The railway facility monitoring device, wherein the object extraction unit performs mask processing based on an application range of the distance filter stored in the storage unit. In the railway equipment monitoring device according to claim 5,
Storing the predetermined rotation angle set via the input unit in the storage unit;
The railway equipment monitoring device, wherein the camera rotation holding unit rotates the stereo camera based on a predetermined rotation angle stored in the storage unit. In the railway equipment monitoring device according to claim 6,
The storage unit extracts the pantograph generated by the object extraction unit, counts the number of pixels that can be obtained by aggregating the number of pixels corresponding to the pantograph in an image distinguishable from others, and the predetermined rotation angle Store in association with
The camera rotation holding unit determines the predetermined rotation angle corresponding to the maximum value among the number of distance measurement possible pixels stored in the storage unit as a rotation angle of a stereo camera by the camera rotation holding unit. Railway equipment monitoring equipment. In the railway equipment monitoring device according to claim 6,
The camera rotation holding unit extracts the pantograph generated by the object extraction unit and compares the shape of pixels corresponding to the pantograph in the image distinguishable from others with the image of the pantograph captured by the stereo camera, The railway equipment monitoring device, wherein the predetermined rotation angle at which the similarity of the shape of the pixel corresponding to the pantograph to the image of the pantograph is maximum is determined as the rotation angle of the stereo camera by the camera rotation holding unit. The railway equipment monitoring device, an electronic terminal installed on the ground equipment including the operation control center, and a communication network for communicably connecting these to each other,
The railway equipment monitoring device
A stereo camera composed of two cameras and imaging a pantograph as a target object;
A camera rotation holder for rotatably holding the stereo camera;
The stereo camera is changed for each predetermined rotation angle, and a distance image of a pantograph is generated from captured images obtained at each rotation angle, and the rotation angle of the stereo camera by the camera rotation holding unit is generated based on the distance image of the pantograph A railway facility monitoring system comprising: In the railway equipment monitoring system according to claim 9,
The two cameras have optical axes parallel and are arranged horizontally to each other,
The railway camera according to claim 1, wherein the camera rotation holding unit rotates the stereo camera in a roll rotation angle direction at each of the predetermined rotation angles, with the rotation axis as an approximate center of a baseline connecting the centers of the two cameras. Equipment monitoring system. In the railway equipment monitoring system according to claim 9,
The railway facility monitoring device includes an image display unit.
The control calculation unit
A stereo processing unit that generates a distance image of a pantograph from captured images obtained at each of the rotation angles;
An object extraction unit which extracts the pantograph as the object and generates an image distinguishable from others based on the distance image of the pantograph generated by the stereo processing unit and the original image of the pantograph taken by the stereo camera When,
And a rotation correction unit that performs reverse rotation correction processing on an image that can be extracted from the pantograph generated by the object extraction unit and can be discriminated from others according to the determined rotation angle of the stereo camera. A railway facility monitoring system, which extracts a pantograph generated by the object extraction unit after reverse rotation correction processing by a rotation correction unit and displays an image distinguishable from others on the image display unit. In the railway equipment monitoring system according to claim 11,
The railway facility monitoring system, wherein the object extraction unit has a distance filter that performs mask processing on the distance image of the pantograph generated by the stereo processing unit except for the region where the pantograph is present. In the railway equipment monitoring system according to claim 12,
The railway facility monitoring device includes an input unit and a storage unit.
Storing the application range of the distance filter set via the input unit in the storage unit;
The railway facility monitoring system, wherein the object extraction unit performs mask processing based on the application range of the distance filter stored in the storage unit. In the railway equipment monitoring system according to claim 13,
Storing the predetermined rotation angle set via the input unit in the storage unit;
The railway equipment monitoring system, wherein the camera rotation holding unit rotates the stereo camera based on a predetermined rotation angle stored in the storage unit. In the railway equipment monitoring system according to claim 14,
The storage unit extracts the pantograph generated by the object extraction unit, counts the number of pixels that can be obtained by aggregating the number of pixels corresponding to the pantograph in an image distinguishable from others, and the predetermined rotation angle Store in association with
The camera rotation holding unit determines the predetermined rotation angle corresponding to the maximum value among the number of distance measurement possible pixels stored in the storage unit as a rotation angle of a stereo camera by the camera rotation holding unit. Railway equipment monitoring system. In the railway equipment monitoring system according to claim 14,
The camera rotation holding unit extracts the pantograph generated by the object extraction unit and compares the shape of pixels corresponding to the pantograph in the image distinguishable from others with the image of the pantograph captured by the stereo camera, The railway facility monitoring system, wherein the predetermined rotation angle at which the similarity of the shape of the pixel corresponding to the pantograph to the image of the pantograph is maximum is determined as the rotation angle of the stereo camera by the camera rotation holding unit. In the railway facility monitoring system according to claim 16,
The railway facility monitoring apparatus determines that the pantograph is damaged or missing if the similarity of the shape of the pixel corresponding to the pantograph to the image of the pantograph to be maximum is less than a threshold of the similarity set in advance. A railway facility monitoring system, which determines and transmits a warning indicating to the electronic terminal via the communication network that the pantograph may be damaged or lost.

Images