WO2024162454A1 - Information processing device, information processing method, and program - Google Patents

Abstract

An information processing device (1) comprises a movement direction acquisition unit (12) and a sound source localization calculation unit (14). The movement direction acquisition unit (12) acquires the direction of movement of a user along a path. The sound source localization calculation unit (14) calculates a position, which is located at a predetermined distance away from the user in the movement direction, as sound source localization for a notification sound for guiding movement in the movement direction.

Description

Information processing device, information processing method, and program

The present invention relates to an information processing device, an information processing method, and a program.

Voice navigation systems are widely used in car navigation systems and smartphones. In this type of system, the current location is detected using GPS or other devices, and the route to the destination is notified by voice.

JP 2020-188494 A

The sense of direction and distance varies from person to person. This makes it difficult to communicate one's course accurately in words. One option would be to set up sound sources at intermediate points or guide facilities along the way to the destination and have the sound emit guidance sounds, but if there is an error in the self-position, this could result in incorrect guidance.

In view of this, this disclosure proposes an information processing device, an information processing method, and a program that can guide a user accurately by voice.

According to the present disclosure, there is provided an information processing device having a direction of travel acquisition unit that acquires the direction of travel of a user's path, and a sound source localization calculation unit that calculates a position a predetermined distance away from the user in the direction of travel as the sound source localization of a notification sound that guides movement in the direction of travel. In addition, according to the present disclosure, there is provided an information processing method in which the information processing of the information processing device is executed by a computer, and a program for causing a computer to realize the information processing of the information processing device.

FIG. 1 is a diagram illustrating a conventional navigation system. FIG. 1 is a diagram illustrating a conventional navigation system. FIG. 1 is a diagram illustrating a conventional navigation system. FIG. 1 is a diagram illustrating a conventional navigation system. FIG. 1 is a diagram illustrating a conventional navigation system. FIG. 1 is a diagram illustrating a conventional navigation system. FIG. 1 is a diagram illustrating a conventional navigation system. FIG. 1 illustrates an example of a configuration of an information processing device. 11A and 11B are diagrams illustrating an example of a notification sound generation process. 11A and 11B are diagrams illustrating an example of a notification sound generation process. FIG. 13 is a diagram showing an example of mixing of notification sounds. FIG. 13 is a diagram showing an example of mixing of notification sounds. FIG. 13 is a diagram showing an example of mixing of notification sounds. FIG. 13 is a diagram showing an example of mixing of notification sounds. FIG. 13 is an explanatory diagram of a sound effect. FIG. FIG. 1 is a diagram illustrating an example of a system configuration of a navigation system. FIG. 11 is a diagram showing another example of the system configuration of a navigation system. FIG. 13 is a diagram showing an example of a UI related to notification of a traveling direction. FIG. 13 is a diagram illustrating an example of a UI related to a danger area approach notification. FIG. 2 illustrates an example of a hardware configuration of an information processing device. 11A and 11B are diagrams illustrating an example of control of a notification sound based on a head direction. FIG. 13 illustrates an example of a configuration of an information processing device according to a modified example. FIG. 13 is a diagram illustrating an example of a notification method. FIG. 13 is a diagram illustrating an example of a notification method. FIG. 11 is a diagram illustrating an example of a flow of generating a notification sound. FIG. 11 is a diagram illustrating another example of the flow of generating a notification sound. FIG. 13 is a diagram illustrating an example of settings regarding notifications within a notification target range. FIG. 13 is a diagram showing an example of a notification of surrounding facilities. FIG. 13 is a diagram illustrating an example of a configuration of a main part of an information processing device according to a modified example. FIG. 11 is a diagram showing an example of a process flow relating to surrounding facility guidance. FIG. 13 is a diagram showing an example of a notification of surrounding facilities. FIG. 13 is a diagram showing an example of a notification of surrounding facilities. FIG. 13 is a diagram showing an example of a notification of surrounding facilities. FIG. 13 is a diagram showing an example of a notification of surrounding facilities. FIG. 35 is a diagram showing an example of a processing flow relating to setting of the sound source localization in FIG. 34 .

Below, embodiments of the present disclosure will be described in detail with reference to the drawings. In each of the following embodiments, the same parts will be designated by the same reference numerals, and duplicated descriptions will be omitted.

The explanation will be given in the following order.
[1. Overview]
2. Configuration of information processing device
3. Information Processing Method
[4. Notification sound mixing example]
5. Sound Effects
6. Warning Sound
7. System configuration example
[8. Notification sound selection UI]
[9. Hardware Configuration Example]
[10. Effects]
11. Modifications
[12. Modifications regarding surrounding facility guidance]

[1. Overview]
Navigation systems that allow users to set a destination on a map and guide them to the destination by map display or voice are widely used. However, when using a map display, there is a possibility that users may not pay enough attention to objects ahead while driving while looking at the map. Voice route guidance is also widely used, but voice guidance generally presents guide information at intervals of tens to hundreds of meters because frequent voice announcements can be irritating, and it can only provide limited information. For these reasons, voice route guidance has the following problems:

Figures 1 to 7 are diagrams explaining conventional navigation systems.

First, as shown in Fig. 1, there is a problem that the distance presented by the voice guidance may differ from the actual distance traveled based on the sense of the traveler (user US). Fig. 1 shows an example in which the user is unable to accurately grasp the distance traveled after hearing the voice "Turn right 100 m ahead" and ends up on a different road (incorrect route CS _E ) near the road he or she actually wants to turn onto (correct route CS _R ). This problem occurs frequently, especially for visually impaired people, who have difficulty distinguishing between roads and non-roads, for example, between the entrance to a parking lot and not a road.

As shown in Figure 2, there is also a problem that the detailed angle of the turn cannot be presented. A voice such as "Turn right" cannot indicate the degree of curvature of the turn. For this reason, for example, if "Turn right" is given on a turn road with little curvature, and the actual road (actual course CS _A ) is not as curved as the traveler imagines (or vice versa), this can cause confusion for the traveler. In particular, visually impaired people cannot see the flow of traffic, so they cannot know how far they should turn if they are only told "turn right."

As shown in Figure 3, there is also the issue of losing track of one's own position and traveling direction TD before the next voice guidance. When concentrating on something, such as walking while looking at a smartphone, or in an open area or place with few landmarks, there are cases where one loses track of how far and in which direction one has traveled in response to the previous voice guidance. In particular, it is known that visually impaired people can experience a phenomenon called "deviation of track" when the road is wide, causing them to turn even when they think they are going straight.

To solve these problems, a method has been proposed to notify the direction and distance to the destination using localized stereophonic sound. For example, JP 2020-188494 A proposes a method of notifying the direction and distance to the destination using a sound source with a localized position. However, there are often obstacles such as buildings between the current location and the destination, and it is common for the direction to the destination to differ from the direction of travel on the actual route. For this reason, even if the direction to the destination is known, the user must find a route that bypasses the obstacles and move around it. Furthermore, if the destination is far away, it is difficult to perceive the distance to the destination from sound alone.

In response to this, as shown in Figure 4, there is a method in which a sound source SC is placed at a specific position (navigation point) on a travel route RT (see Figure 6) set on a map, the position is determined from the positional relationship between the moving object's own position on the map and the sound source SC, and the direction of travel on the travel route RT is notified using stereophonic sound. However, this method has the following problems.

First, if the accuracy of the user's own position or map information is poor, the notification sound may be heard from a direction different from the actual direction of travel (see Figure 4). It is also difficult to set an appropriate notification sound volume that is not too loud when approaching and not too quiet when far away, as well as the spacing between navigation points (see Figure 5). Furthermore, if there are multiple navigation points within the moving person's hearing range, the notification sounds may interfere with each other, which is an issue (see Figure 6).

With regard to the issue in Figure 6, JP 2013-47653 A proposes a method of using stereophonic audio to provide notifications similar to the so-called "cross-talk" audio signals, which alternate between audio of the starting point and audio of the destination point.

In addition, Japanese Patent Application Laid-Open No. 2002-5675 proposes a mechanism in which a notification sound is emitted from the navigation point where the traveler's own position is closest, and when that navigation point is reached, the sound source location of the notification sound is moved to the next navigation point, thereby providing guidance along the travel route RT with a single notification sound. However, this method does not solve the problem in Figure 4.

7, for example, when the navigation point is near a corner, it is possible to place a sound source SC _C on the current route CS _C , and when the navigation point reaches the vicinity of the next route CS _N , place a sound source SC _N on the next route CS _N for guiding the user to the next route CS _N. In this case, if the sound source localization moves in the direction beyond the corner before the user's own position reaches the corner due to the influence of a position error, problems may occur such as the user being unable to determine the direction of travel up to the corner, or the direction of the sound source SC being misaligned with the direction of travel on the travel route RT.

The present disclosure has been made in consideration of the above-mentioned problems. The present disclosure aims to provide a navigation system that can guide a user with high accuracy by voice. For example, one of the features of the information processing device according to the present disclosure is that it solves some or all of the above-mentioned problems by using the following method.

I. The notification sound using stereophonic sound conveys the direction of travel with higher resolution than voice guidance.
II. By emitting a notification sound from a fixed position that always maintains a constant distance from the moving person and conveying only the direction of travel, the correct direction of travel can be conveyed even if the accuracy of the self-location and map information is poor.
III. The next direction, the current direction, and the position of the corner are conveyed by changes in the volume and melody of the sound source conveying the current direction and the sound source conveying the next direction.

2. Configuration of information processing device
FIG. 8 is a diagram showing an example of the configuration of the information processing device 1.

The information processing device 1 generates navigation information to a destination based on sensor information acquired from various sensors such as a GPS (Global Positioning System), a GNSS (Global Navigation Satellite System), a beacon, a camera, a gyro sensor, and a geomagnetic sensor. These sensors function as a sensor unit for detecting the position and orientation of the head of the user US. The navigation information includes acoustic information such as a notification sound for guiding the user US and a warning sound for alerting the user US.

Details will be described later, but the information processing device 1 divides the travel route into a number of routes, and sets a direction of travel, a notification sound, and a sound source for each route. In the following explanation, the route, direction of travel, notification sound, and sound source are represented by the symbols "CS", "TD", "SD", and "SC", respectively. When distinguishing the direction of travel TD, notification sound SD, and sound source SC for each route CS, a number or letter is added to each symbol to make the distinction.

The information processing device 1 has a route planning unit 11, a traveling direction acquisition unit 12, a head direction acquisition unit 13, a sound source localization calculation unit 14, a sound generation unit 15, a volume adjustment unit 16, and a notification sound output unit 17.

The route planning unit 11 acquires the position and orientation of the head of the user US as user position information. The user position information is detected from sensor information using, for example, the above-mentioned GPS, GNSS, or a camera-based VPS (Visual Positioning System) to measure the absolute position, and, if necessary, a relative position estimation method such as PDR (Pedestrian Dead Reckoning). The route planning unit 11 plans a travel route RT to the destination based on the user position information. The head direction acquisition unit 13 acquires the orientation of the head of the user US as the head direction of the user US.

The traveling direction acquisition unit 12 acquires the traveling direction TD of the route CS of the user US. The traveling direction TD of the route CS means the direction of the movement line that passes through the center of the route CS. For example, the traveling direction acquisition unit 12 divides the movement route RT into multiple routes CS based on information such as the corners or curvature of the movement route RT. The route CS is a roughly straight route whose curvature satisfies an acceptable standard. The acceptable standard is set arbitrarily by the system developer. The traveling direction acquisition unit 12 acquires multiple routes CS divided along the movement route RT. For each divided route CS, the traveling direction acquisition unit 12 acquires the direction along the route CS as the traveling direction TD of the route CS.

The traveling direction acquisition unit 12 can acquire a traveling direction TD for both the current course CS _C and the next course CS (next course CS _N ) following the current course CS _C from the travel route RT. For example, the traveling direction acquisition unit 12 has a first traveling direction acquisition unit 12A and a second traveling direction acquisition unit 12B. The first traveling direction acquisition unit 12A acquires a traveling direction TD _C of the current course CS _C. The second traveling direction acquisition unit 12B acquires a traveling direction TD _N of the next course CS _N.

The sound source localization calculation unit 14 calculates a position a predetermined distance away from the user US in the traveling direction TD _C as the sound source localization of a notification sound SD _C (see FIG. 11 ) that guides the user US to move in the traveling direction TD _C. The sound source localization of the notification sound SD _C can be calculated based on the position and orientation of the head of the user US.

"Based on the position and orientation of the head" means that the position is specified based on a relative positional relationship to the position and orientation of the head. The user US can move accurately on the course CS _C by moving in the direction in which the notification sound SD _C is sounded. The distance to the sound source localization is set arbitrarily by the system developer. For example, the sound source localization of the notification sound SD _C can be set to a position on the course CS _C through which the user US must pass.

The sound source localization calculation unit 14 can generate a notification sound SD for each of the current course CS _C and the next course CS _N. For example, the sound source localization calculation unit 14 has a first sound source localization calculation unit 14A and a second sound source localization calculation unit 14B. The first sound source localization calculation unit 14A calculates a position a predetermined distance away from the user US in the traveling direction TD _C of the current course CS _C as the sound source localization of the notification sound SD _C that guides the user to move in the traveling direction TD _C of the current course CS _C. The second sound source localization calculation unit 14B calculates a position a predetermined distance away from the user US in the traveling direction TD _N of the next course CS _N as the sound source localization of the notification sound SD _N (see FIG. 11 ) that guides the user to move to the next course CS _N.

The sound generating unit 15 generates various notification sounds SD and warning sounds WS (see FIG. 16) for navigation. The notification sounds SD and warning sounds WS can be generated, for example, as sounds (loop sounds) that are output with a certain rhythm or at playback time intervals. The notification sounds SD can include pulse sounds and sound effects for notifying the user US of the traveling direction TD. The warning sounds WS can include pulse sounds and sound effects for alerting the user US to the presence of a warning area WA (see FIG. 16) and a danger area DA (see FIG. 16), etc.

The sound generating unit 15 can generate two notification sounds SD related to the current route CS _C and the next route CS _N. For example, the sound generating unit 15 has a first sound generating unit 15A and a second sound generating unit 15B. The first sound generating unit 15A generates a notification sound SD _C related to the traveling direction TD _C of the current route CS _C. The second sound generating unit 15B generates a notification sound SD _N related to the traveling direction TD _N of the next route CS _N.

The volume adjustment unit 16 adjusts the volume of the two notification sounds SD to generate a notification sound SD for output. The notification sound output unit 17 outputs the notification sound SD generated by the sound generation unit 15 or the volume adjustment unit 16 via a speaker or the like. For example, the position that is the boundary between the current course CS _C and the next course CS _N is set as the midpoint BP (see FIG. 11). The volume adjustment unit 16 adjusts the volume of the two notification sounds SD related to the current course CS _C and the next course CS _N based on the distance from the midpoint BP to the user US. The volume adjustment unit 16 generates the notification sound SD obtained by mixing the two notification sounds SD with the adjusted volume as a notification sound that induces switching from the current course CS _C to the next course CS _N.

"Generating a notification sound SD obtained by mixing two notification sounds SD" means "generating a notification sound SD that is perceived by the user US as a sound that is a mix of two notification sounds SD." The mixing may be performed at the signal processing stage, or may be performed in the brain of the user US when viewing. An example of the latter is a method in which two notification sounds SD are played alternately, like a "different species calling method." For example, one possible method is to set two notification sounds SD to be played alternately, with 5 seconds of play and 5 seconds of stop, and play the other while one is stopped.

[3. Information Processing Method]
9 and 10 are diagrams illustrating an example of a process for generating a notification sound SD.

The traveling direction acquisition unit 12 acquires the traveling direction TD _C of the current course CS _C and the traveling direction TD _N of the next course CS _N (step S1). The traveling direction TD can be acquired based on self-location information and map information detected using, for example, GPS, GNSS, a beacon, PDR, etc. The traveling direction TD _C of the current course CS _C may be designated by the user US by pointing the device equipped with the sensor unit in the direction in which the user wants to travel, without using map information.

The traveling direction acquisition unit 12 divides the travel route RT to the destination into a plurality of routes CS based on the positions of corners. The traveling direction acquisition unit 12 sets the position of the corner toward the next route CS _N as the midpoint BP. The traveling direction acquisition unit 12 acquires the distance on the current route CS _C from the current location to the midpoint BP based on the distance information of the travel route RT (step S2).

In order to notify the traveling direction TD using stereophonic sound, it is necessary to obtain the traveling direction TD on the head coordinate system CO with the face orientation set to 0° (front orientation). For example, a geomagnetic sensor is built into earphones or the like and attached to the head of the user US, and the geomagnetic sensor detects the direction in which the face is facing. The head direction obtaining unit 13 obtains the detected face (head) orientation based on the sensor information of the geomagnetic sensor as the head direction (step S3).

The sound source localization calculation unit 14 sets a head coordinate system CO based on the position and orientation of the head. The head coordinate system CO is, for example, a three-dimensional coordinate system in which the position of the head of the user US is the origin, and any one of the x-axis, y-axis, and z-axis coincides with the head direction. The sound source localization calculation unit 14 calculates the direction along the course CS from the position of the head as the traveling direction TD of the course CS on the head coordinate system CO. The sound source localization calculation unit 14 calculates a position a predetermined distance away from the position of the head in the traveling direction TD _C of the current course CS _C on the head coordinate system CO as the sound source localization of the notification sound SD _C. The sound generation unit 15 generates a notification sound SD _C that is localized to the calculated sound source localization (step S4).

The traveling direction acquisition unit 12 determines whether the distance to the next corner (midpoint BP) is equal to or less than a threshold (step S5). If the distance is greater than the threshold (step S5: No), the notification sound output unit 17 outputs the notification sound SD _C to the speaker. If the distance is equal to or less than the threshold (step S5: Yes), the sound source localization calculation unit 14 calculates a position that is a predetermined distance away from the head position in the traveling direction TD _N of the next course CS _N on the head coordinate system CO as the sound source localization of the notification sound SD _N. The sound generation unit 15 generates the notification sound SD _N that is localized to the calculated sound source localization (step S6). The volume adjustment unit 16 outputs the notification sound SD, the volume of which is adjusted for the notification sound SD _C and the notification sound SD _N according to the distance, to the speaker (step S7).

In the example of Fig. 10, the user US is walking on the sidewalk in front of him/her toward the crosswalk. The sidewalk in front of him/her is the current route CS _C , and the crosswalk is the next route CS _N. If it is still a long way from the current route CS _C to the next route CS _N , the sound source SC _N notifying the switch to the next route CS _N is silent, and only the sound source SC _C notifying the traveling direction TD _C of the current route CS _C outputs the notification sound SD _C (see the diagram on the left side of Fig. 10).

As the user US approaches the next route CS _N , the notification sound _{SDN that} guides the user to the next route CS _N gradually becomes louder (see the diagram in the center of FIG. 10). When the user US passes the waypoint BP and enters the next route CS _N , the notification sound _SDN that notifies the user of the traveling direction _TDN of the previous route CS _C gradually becomes quieter and eventually becomes silent. Then, only the notification sound _SDN that notifies the user of the traveling direction _TDN of the next route CS _N is output (see the diagram on the right side of FIG. 11).

[4. Notification sound mixing example]
11 and 12 are diagrams showing examples of mixing of notification sounds SD.

The user US moves along two L-shaped intersecting routes CS (route CS _C , next route CS _N ) along a planned travel route. Fig. 12 shows notification sounds SD of five waypoints PT set on the route CS. Waypoint PT ₃ coincides with the boundary (midpoint BP) between route CS _C and next route CS _N.

When the user US travels along the route CS _C , the user US moves according to the notification sound _SDC output from the sound source _SCC ahead. When the user US reaches the waypoint _PT2 near the midpoint BP, the notification sound _SDC , which guides the user US to move to the next route CS _N , starts to be output from the sound source SCC set on the travel direction TD _N side of the next route CS _N. The system developer arbitrarily sets how close the user US is to the midpoint BP when the notification sound _SDC starts to be output. The distance from the user US to the sound source localization of the notification sound _{SDC and} the distance from the user US to the sound source localization of the notification sound _SDC are constant, for example, regardless of the position of the user US.

The volume adjustment unit 16 increases the volume of the notification sound SD _N that guides the user US to move to the next route CS _N as the user US approaches the midpoint BP on the current route CS _C. Conversely, the volume adjustment unit 16 decreases the volume of the notification sound SD _C that guides the user US to move in the traveling direction of the current route CS _C as the user US approaches the midpoint BP on the current route CS _C. For example, when the user US reaches the midpoint BP, the volume adjustment unit 16 adjusts the mixing amount of the two notification sounds SD so that the volume of the notification sound SD _C and the volume of the notification sound SD _N become equal.

When the user US passes the waypoint BP and enters the next route CS _N , the route CS to be guided is switched from the route CS _C to the next route CS _N. The volume adjustment unit 16 increases the volume of the notification sound SD _N that guides the user US to move in the traveling direction TD _N of the next route CS _N as the user US moves away from the waypoint BP on the next route CS _N. Conversely, the volume adjustment unit 16 decreases the volume of the notification sound SD _C that guides the user US to move in the traveling direction TD _C of the previous route CS _C as the user US moves away from the waypoint BP on the next route CS N. For example, the volume adjustment unit 16 adjusts the volume of the notification sound _{SD C to} zero when the user US reaches the waypoint PT ₄ .

FIGS. 13 and 14 show other mixing examples of notification sounds SD.

The user US heads from the current route CS _C to the next route CS _N along the planned movement route, but mistakenly continues straight ahead without turning left at the midpoint BP. When the user US moves to the wrong route CS _E , the route planning unit 11 resets the movement route RT. The route planning unit 11 plans a movement route RT that moves in the opposite direction along the wrong route CS _E toward the midpoint BP.

The traveling direction acquisition unit 12 acquires the direction returning to the midpoint BP as the traveling direction TD of the course CS _E. The sound source localization calculation unit 14 sets the sound source SC of the notification sound SD _E , which guides the user US to move to the midpoint BP, behind the user US. For example, when the user US passes the midpoint BP and moves in a direction different from the next course CS _N , the sound source localization calculation unit 14 calculates a position a predetermined distance away in the direction of the midpoint BP as the sound source localization of the notification sound SD _E. The volume of the notification sound SD _E becomes louder the farther it is from the midpoint BP and becomes quieter the closer it is to the midpoint BP. The volume of the notification sound SD _N becomes quieter the farther it is from the midpoint BP and becomes louder the closer it is to the midpoint BP.

5. Sound Effects
FIG. 15 is an explanatory diagram of sound effects.

When notifying the user through stereo sound, there is little difference between left and right in stereo sound, especially in the forward direction, and it can be difficult to determine the position. For this reason, if you want to convey more accurately that the user is walking straight in the direction of the sound source SC (travel direction TD), you can make the user US aware of how appropriately they are moving by adding sound effects that correspond to the deviation from the direction of the sound source SC.

For example, the sound generation unit 15 acquires the angle range within which movement is permitted as the guidance range RG. The guidance range RG is set as a predetermined angle range centered on the traveling direction TD. The guidance range RG is set arbitrarily by the system developer. When the movement direction of the user US is included in the guidance range RG, the sound generation unit 15 can impart a sound effect to the notification sound SD that guides movement in the traveling direction TD.

In the example of FIG. 15, the direction of the head of the moving user US (head direction HD) is detected as the direction of movement of the user US. When the head direction HD is within an angular range of ±n° centered on the traveling direction TD of the course CS, sound effects such as increasing the volume of the notification sound SD, strengthening the reverberation, and increasing the sound spread are applied. This makes it possible to more accurately convey that the direction of movement is within ±n° of the traveling direction.

6. Warning Sound
FIG. 16 is an explanatory diagram of the warning sound.

In the example of FIG. 15, the appropriateness of movement is constantly notified through sound effects. However, instead of continuously notifying the accurate direction of travel TD, a warning sound WS may be output if the direction of travel deviates from TD.

For example, the sound source localization calculation unit 14 sets the traffic line at the center of the course CS. The sound source localization calculation unit 14 sets an area slightly off the traffic line as a warning area WA, and sets an area outside the warning area into which it is not recommended to enter as a danger area DA. The sound source localization calculation unit 14 sets a safe area near the traffic line that is neither a warning area WA nor a danger area DA as a safety area SA.

The sound source localization calculation unit 14 can identify dangerous areas based on sensor information and set warning areas WA and danger areas DA. The sensor information can include image information captured by a camera, distance information measured by a distance sensor, and location information acquired from GPS/GNSS or a Visual Positioning System (VPS). The widths of the safety area SA, warning area WA, and danger area DA are set arbitrarily by the system developer. For example, the warning area WA is an area that is a certain distance or more away from the traffic flow, and the danger area DA is an area that is a certain distance or more away from the warning area WA toward the outside.

The sound source localization calculation unit 14 calculates a position on the boundary BD between the warning area WA and the danger area DA as the sound source localization of the warning sound WS. The sound generation unit 15 generates a warning sound WS having a sound image at the calculated sound source localization, and outputs it to the notification sound output unit 17 when the user UD approaches the boundary BD. The system developer can arbitrarily set how close the user US must be to the boundary BD before the warning sound WS is output.

The warning area WA may be an area that is parallel to the traffic flow and a certain distance away from the traffic flow, and the danger area DA may be an area that is parallel to the boundary between the safety area SA and the warning area WA and is a certain distance or more away from the boundary between the safety area SA and the warning area WA. Furthermore, for example, the edge of the platform may be detected based on information obtained from a camera or distance sensor, and an area that is a certain distance or more away from the edge of the platform may be set as the danger area DA or warning area WA. The danger area DA and warning area WA may also be set in other ways.

In the example of FIG. 16, when the user US deviates from the line of movement and approaches a danger area DA (e.g., the edge of a station platform, the edge of a sidewalk, etc.) that is parallel to the line of movement and has a certain width, a warning sound WS is output from a sound source SC _W on a boundary BD of the danger area DA. The sound source localization calculation unit 14 can calculate a point on the boundary BD or a planar area along the boundary BD as the sound source localization of the warning sound WS. When a planar area along the boundary BD is set as the sound source SC _W of the warning sound WS, for example, the position where a straight line connecting the normal line of the line of movement and the head intersects with the boundary BD can be set as the center of the sound source SC _W. Note that the boundary BD of the line of movement and the danger area DA may be a curve.

7. System configuration example
FIG. 17 is a diagram showing an example of the system configuration of the navigation system NV.

The navigation system NV has a server SV, a client terminal TM, and headphones HP. The client terminal TM is a portable information terminal such as a smartphone. The headphones HP have a sensor unit SE for detecting the position and orientation of the head of the user US. Note that earphones, glasses-type devices, head-mounted displays, etc. may be used instead of the headphones HP.

The functions of the information processing device 1 described above are shared between the server SV and the client terminal TM. The information processing device 1 calculates a position a predetermined distance away from the user US in the traveling direction TD of the user US's path CS, based on the position and orientation of the user US's head, as the sound source localization of the notification sound SD that guides movement in the traveling direction TD. Which functions are assigned to the client terminal TM and which functions are assigned to the server SV is set arbitrarily by the system developer. The headphones HP have speakers that reproduce the sound image of the notification sound SD at the calculated sound source localization.

For example, the client terminal TM detects the position and orientation of the head of the user US based on sensor information acquired from the sensor unit SE. The client terminal TM generates a notification sound SD and a warning sound WS and outputs them to the headphones HP. The client terminal TM sends its own position information to the server SV, and acquires map information and information related to the traveling direction TD from the server SV. If the headphones HP have a high-speed calculation device, the notification sound SD and warning sound WS may be generated by the headphones HP.

FIG. 18 shows another example of the system configuration of the navigation system NV.

In the example of FIG. 18, the head orientation is acquired based on sensor information from an external camera MD or a depth sensor. In this example, the external camera MD or depth sensor functions as a sensor unit SE. The result of head pose estimation performed using face tracking technology is transmitted from the server SV to the client terminal TM. After receiving the pose estimation result, the client terminal TM generates a notification sound SD and a warning sound WS using a method similar to that of the example of FIG. 17.

[8. Notification sound selection UI]
An example of a UI related to notification will be described below. Fig. 19 is a diagram showing an example of a UI related to notification of the traveling direction TD. Fig. 20 is a diagram showing an example of a UI related to a danger area approach notification.

Pressing the "Select notification sound" button allows you to select the type of melody and the type of notification sound (whether to be notified by increasing or decreasing the volume or by changing the tempo). Pressing the "Notification ON/OFF" button switches between enabled and disabled notification.

In the "distance to sound source" field, the distance to the sound source SC that notifies the traveling direction TD can be specified. In the "emphasis accuracy range" field, the guidance range RG shown in FIG. 16 can be specified. In the "turn notification start distance" field, the distance from the midpoint BP to the output start point of the notification sound _SDN (waypoint _PT2 in the example of FIG. 11) can be specified. In the "danger zone warning start distance" field, the distance to the boundary BD of the danger zone DA where the output of the warning sound WS starts can be specified. In addition, the position for localizing the sound image of the notification sound SD and the change in sound pressure may be arbitrarily set on the UI.

[9. Hardware Configuration Example]
FIG. 21 is a diagram illustrating an example of a hardware configuration of the information processing device 1.

The information processing of the information processing device 1 is realized, for example, by a computer 1000. The computer 1000 has a CPU (Central Processing Unit) 1100, a RAM (Random Access Memory) 1200, a ROM (Read Only Memory) 1300, a HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input/output interface 1600. Each part of the computer 1000 is connected by a bus 1050.

The CPU 1100 operates based on a program (program data 1450) stored in the ROM 1300 or HDD 1400, and controls each component. For example, the CPU 1100 loads a program stored in the ROM 1300 or HDD 1400 into the RAM 1200, and executes processing corresponding to the various programs.

The ROM 1300 stores boot programs such as the BIOS (Basic Input Output System) that is executed by the CPU 1100 when the computer 1000 starts up, as well as programs that depend on the hardware of the computer 1000.

HDD 1400 is a non-transitory computer-readable recording medium that non-temporarily records programs executed by CPU 1100 and data used by such programs. Specifically, HDD 1400 is a recording medium that records an information processing program according to an embodiment, which is an example of program data 1450.

The communication interface 1500 is an interface for connecting the computer 1000 to an external network 1550 (e.g., the Internet). For example, the CPU 1100 receives data from other devices and transmits data generated by the CPU 1100 to other devices via the communication interface 1500.

The input/output interface 1600 is an interface for connecting the input/output device 1650 and the computer 1000. For example, the CPU 1100 receives data from an input device such as a keyboard or a mouse via the input/output interface 1600. The CPU 1100 also transmits data to an output device such as a display device, a speaker, or a printer via the input/output interface 1600. The input/output interface 1600 may also function as a media interface that reads programs and the like recorded on a specific recording medium. Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase change rewritable Discs), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, and semiconductor memories.

For example, when the computer 1000 functions as the information processing device 1 according to the embodiment, the CPU 1100 of the computer 1000 executes an information processing program loaded onto the RAM 1200 to realize the functions of each of the above-mentioned units. The information processing program, various models, and various data according to the present disclosure are stored in the HDD 1400. The CPU 1100 reads and executes the program data 1450 from the HDD 1400, but as another example, the CPU 1100 may obtain these programs from other devices via the external network 1550.

[10. Effects]
The information processing device 1 has a traveling direction acquisition unit 12 and a sound source localization calculation unit 14. The traveling direction acquisition unit 12 acquires a traveling direction TD of a path CS of the user US. The sound source localization calculation unit 14 calculates a position a predetermined distance away from the user US in the traveling direction TD as the sound source localization of a notification sound SD that guides movement in the traveling direction TD. In the information processing method disclosed herein, the processing of the information processing device 1 is executed by a computer 1000. The program disclosed herein causes the computer 1000 to realize the processing of the information processing device 1.

With this configuration, the user US is guided to move in the traveling direction TD by the notification sound SD heard from ahead in the traveling direction TD. Therefore, even if the accuracy of the user's own position or map information is poor, the user US can be guided in the traveling direction TD with high accuracy.

The sound source localization calculation unit 14 calculates the sound source localization of the notification sound SD to be a position a predetermined distance away from the user US in the traveling direction TD, based on the position and orientation of the head of the user US.

With this configuration, the traveling direction TD is calculated based on the direction of the head. Therefore, the traveling direction TD is accurately recognized.

The information processing device 1 has a route planning unit 11. The route planning unit 11 plans a travel route RT to a destination. The travel direction acquisition unit 12 acquires a plurality of routes CS partitioned along the travel route RT. For each partitioned route CS, the travel direction acquisition unit 12 acquires the direction along the route CS as the travel direction TD of the route CS.

With this configuration, guidance is provided for each divided route CS. Therefore, even if the travel route RT is complicated and intricate, proper guidance is provided to the destination.

The distance from the user US to the sound source location of the notification sound SD is constant regardless of the position of the user US.

This configuration makes it difficult for the sound source SC of the notification sound SD to be lost. This allows the direction of the sound source SC (travel direction TD) to be reliably recognized.

The traveling direction acquisition unit 12 acquires a traveling direction TD _N of a next route CS _N following the route CS _C. The sound source localization calculation unit 14 calculates a position a predetermined distance away from the user US in the traveling direction TD _N of the next route CS _N as the sound source localization of a notification sound SD _N that guides the user US to move to the next route CS _N.

According to this configuration, while traveling on the route CS _C , it is possible to grasp the switch to the next route CS _N and the traveling direction of the next route CS _N.

The distance from the user US to the sound source location of the notification sound _SDN that guides the user US to move to the next route _CSN is constant regardless of the position of the user US.

According to this configuration, it is difficult to lose sight of the sound source _SCN of the notification sound _SDN , so that the direction of the sound source _SCN (the direction in which the course CS changes) can be reliably recognized.

The information processing device 1 has a volume adjustment unit 16. The volume adjustment unit 16 adjusts the volumes of two notification sounds SD related to the current route CS _C and the next route CS _N based on the distance from the midpoint BP to the user US. The midpoint BP is a point that is a boundary between the current route CS _C and the next route CS _N. The volume adjustment unit 16 generates a notification sound SD obtained by mixing the two notification sounds SD whose volumes have been adjusted as a notification sound SD that induces switching from the current route CS _C to the next route CS _N.

With this configuration, the change in the route CS can be intuitively understood by the change in volume.

The volume adjustment unit 16 increases the volume of the notification sound _SDN that guides the user US to move to the next route CS _N as the user US approaches the midpoint BP on the current route CS _C.

With this configuration, the approach of the midpoint BP can be intuitively grasped by the change in volume.

The volume adjustment unit 16 increases the volume of the notification sound _SDN , which guides the user US to move in the traveling direction _TDN of the next route _CSN , as the user US moves away from the midpoint BP on the next route _CSN .

According to this configuration, the entrance of the CS _N into the next route can be intuitively grasped by the change in volume.

The volume adjustment unit 16 reduces the volume of the notification sound SD _C that guides the user US to move in the traveling direction of the current route CS _C as the user US approaches the midpoint BP on the current route CS _C.

With this configuration, the approach of the midpoint BP can be intuitively grasped by the change in volume.

The volume adjustment unit reduces the volume of the notification sound SD _C that guides the user US to move in the traveling direction of the previous route CS _C as the user US moves away from the midpoint BP on the next route CS _N.

According to this configuration, the entrance of the CS _N into the next route can be intuitively grasped by the change in volume.

When the user US passes the midpoint BP and moves in a direction different from the next route CS _N , the sound source localization calculation unit 14 calculates a position a predetermined distance away in the direction of the midpoint BP as the sound source localization of the notification sound _SDE .

With this configuration, the change in the position of the sound image allows the listener to intuitively understand that they have passed the midpoint BP.

The information processing device 1 has a sound generating unit 15. When the moving direction of the user US is included in a predetermined angle range centered on the traveling direction TD of the course CS, the sound generating unit 15 imparts a sound effect to the notification sound SD that guides the user US to move in the traveling direction TD.

With this configuration, the sound effects allow the player to intuitively understand whether the movement is being performed appropriately.

The sound source localization calculation unit 14 calculates the position on the boundary BD between the warning area WA and the danger area DA as the sound source localization of the warning sound WS.

This configuration allows the approach of the danger zone DA to be intuitively recognized.

The sound source localization calculation unit 14 calculates the planar area along the boundary BD between the warning area WA and the danger area DA as the sound source localization of the warning sound WS.

This configuration allows the boundary BD of the danger area DA to be intuitively grasped.

The warning area WA is an area that is a certain distance or more away from the traffic flow line that is the center of the route CS, and the danger area DA is an area that is a certain distance or more away toward the outside from the warning area WA.

With this configuration, safety levels are managed in stages according to the distance from the traffic flow.

The sound source localization calculation unit 14 identifies dangerous areas based on sensor information and sets warning areas WA and danger areas DA.

With this configuration, the warning area WA and danger area DA are set appropriately.

Note that the effects described in this specification are merely examples and are not limiting, and other effects may also be present.

11. Modifications
Another example of notification control will be described below. Fig. 22 is a diagram showing an example of control of the notification sound SD based on the head direction. The following description will focus on the differences from the above embodiment.

In the above embodiment, notification of the traveling direction TD was performed using stereophonic sound. However, stereophonic sound alone can make it difficult to sense the positioning in front. In this modified example, in order to accurately guide the user US in the forward direction, the pulse interval of the notification sound SD (pulse sound) is changed according to the direction of the head of the user US. The difference between the traveling direction TD and the actual direction of movement can be intuitively grasped from the pulse interval, making it easier to guide the user in the traveling direction TD.

FIG. 23 shows an example of the configuration of the information processing device 2 of this modified example.

In this modified example, the sound generation unit 15 has an angle difference calculation unit 151, a movement detection unit 152, and a notification sound output determination unit 153. The angle difference calculation unit 151 calculates the angle difference between the orientation of the head of the user US and the traveling direction TD. The movement detection unit 152 detects the movement of the head of the user US. The notification sound output determination unit 153 determines whether or not to output the notification sound SD based on the movement of the head of the user US. The notification sound output unit 17 outputs the notification sound SD based on the determination result of the notification sound output determination unit 153.

FIGS. 24 and 25 show examples of notification methods.

The head direction is acquired by the head direction acquisition unit 13. The head direction acquisition unit 13 acquires the head direction of the user US as the head direction HD using a geomagnetic sensor or the like. For example, the notification sound output determination unit 153 determines that it is appropriate to perform a notification when the angular difference between the head direction HD and the traveling direction TD is within the notification target range.

The notification sound output determination unit 153 can also determine whether or not to notify based on the direction of head movement. For example, the notification sound output determination unit 153 can determine to notify when the user US is heading in the traveling direction TD, and not to notify when the user US is heading in a direction unrelated to the traveling direction TD.

The notification range refers to the angular range that is the subject of notification. The notification sound output unit 17 outputs a notification sound SD that guides movement in the traveling direction TD when the angular difference between the head direction HD and the traveling direction TD is within the notification range. In the example of FIG. 24, the maximum angular difference at which notification begins is shown as the maximum notification angle Θ. The notification range is the angular range in which the angular difference between the traveling direction TD and the head direction HD is greater than or equal to -Θ and less than or equal to Θ. However, the method for setting the notification range is not limited to this. The notification range is set arbitrarily by the system developer.

The notification sound output unit 17 outputs the notification sound SD as a pulse sound. The smaller the angular difference between the traveling direction TD and the head direction HD, the higher the notification frequency of the pulse sound. For example, the notification sound output unit 17 sets the notification interval of the pulse sound to a magnitude that is inversely proportional to the angular difference between the head direction HD and the traveling direction TD.

Figure 26 is a diagram explaining an example of the generation flow of the notification sound SD.

The travel direction acquisition unit 12 acquires the travel direction TD _C of the current course CS _C (step S11). The head direction acquisition unit 13 acquires the head direction HD of the user US (step S12). The angle difference calculation unit 151 calculates the angle difference between the travel direction TD _C and the head direction HD (step S13). The movement detection unit 152 detects the movement of the head of the user US. The notification sound output determination unit 153 detects the direction in which the head moves and determines whether or not to notify based on whether the head is moving in a direction approaching the travel direction TD (step S14).

When the head is moving toward the traveling direction TD (step S14: Yes), the notification sound output determination unit 153 determines whether the angular difference between the traveling direction TD _C and the head direction HD is within the notification target range (step S15). When the angular difference is within the notification target range (step S15: Yes), the notification sound output determination unit 153 determines that it is appropriate to perform notification.

The notification sound output unit 17 outputs the notification sound SD _C based on the results of a series of determinations made by the notification sound output determination unit 153. That is, when the angular difference is within the notification target range and the head of the user US is moving in a direction approaching the traveling direction TD _C , the notification sound output unit 17 outputs the notification sound SD _C that guides the user US to move in the traveling direction TD _C.

If the head is moving away from the traveling direction TD _C (step S14: No) and if the angular difference is outside the notification range (step S15: No), the notification sound output determiner 153 determines that it is not appropriate to provide a notification. In this case, the process returns to step S11, and the above-described process is repeated until it is determined that it is appropriate to provide a notification.

In the above flow, the head movement is determined (step S14) and then the angle difference is determined (step S15). However, either of these determinations can be performed first. The notification sound output determination unit 153 comprehensively considers both determination results and determines the final validity of the notification.

FIG. 27 is a diagram illustrating another example of the generation flow of the notification sound SD.

This example differs from the example in FIG. 26 in that step S26 has been added as a condition for determining whether or not to notify. Steps S21 to S25 are generally similar to steps S11 to S15. The only difference is that the order of steps S24 to S25 is different from that of steps S14 to S15.

In the example of Fig. 27, the time elapsed since the head starts to move n in the appropriate direction is added as a judgment condition (step S26). The notification sound output unit 17 monitors the time elapsed since the judgment results of steps S24 to S25 are issued. The notification sound output unit 17 starts outputting the notification sound SD _C after a preset time interval has elapsed since the head of the user US starts to move in a direction approaching the traveling direction TD _C. The time to start outputting the notification sound is set arbitrarily by the system developer.

Figure 28 shows an example of settings for notifications within the notification range.

The notification frequency, notification volume, and maximum notification angle Θ within the notification target range can be set using the UI. In the "Notification frequency" field, the time interval of the pulse sound output as the notification sound SD can be specified as the notification frequency. In the "Notification volume" field, the volume of the notification sound SD can be specified. In the "Maximum notification angle" field, the maximum angle difference at which notifications begin can be specified as the maximum notification angle Θ. The notification frequency and notification volume are specified as numerical values with standardized levels ranging from 0 to 1, for example.

[12. Modifications regarding surrounding facility guidance]
The following describes a modified example of the surrounding facility guidance. Fig. 29 is a diagram showing an example of notification of the surrounding facilities SF. The following describes mainly the differences from the above embodiment.

In places like busy shopping streets lined with shops, large commercial facilities such as shopping malls, and large retail stores with partitioned sales areas, it is often difficult to intuitively know where the desired store or sales area is located. In such cases, by providing audio notification of what is in which direction around you, it is possible to reduce the inconvenience of having to stop to look at a map and the risk of accidents such as bumping into or falling when walking while looking at a map.

In addition, if the user is visually impaired, they need to receive information about all surrounding locations beyond the reach of their eyesight through audio, etc., not just the locations mentioned above, so the need for non-visual information to be presented through audio, etc. is greater than for able-bodied users.

Patent No. 4315211 describes a method in which the user selects an object to be guided from a list of guide candidates by operating the UI of a mobile information terminal, and is notified of the location of the object or its location along the route via stereophonic audio data through headphones.

However, if you receive a voice notification of the names of stores or facilities within a few hundred meters of you and want to obtain information related to that location, such as special offers at the store, restaurant menus, or upcoming events, you will need to take your mobile information device out of your pocket or bag, open a browser to search for the name of the facility, or open a dedicated app and select it from a list to find out more information. If the facility is far away and you do not know the route, you will need to open a navigation app to find the route and look at a map to get to the facility.

With such complex operating procedures, voice notifications have few benefits for able-bodied people, as the amount of information they provide is less than when information is obtained visually from the screen of a mobile information terminal used to operate the device. In addition, for visually impaired people, the operating procedures are complex, making it difficult to operate the device without relying on vision.

This modified example has been devised to solve the above-mentioned problems. In this modified example, the location and direction of the surrounding facilities SF are notified using stereophonic sound. When the user US performs a specific trigger action, detailed information about the surrounding facilities SF and a travel route RT are presented. This makes it possible to execute processes that previously required complex operations from an app on a mobile information terminal with fewer operations. In particular, if trigger actions can be performed using voice or gesture operations, hands-free operation can be realized without the need for a separate mobile information terminal or the like.

The detailed information may include various information related to the surrounding facility SF other than the facility name. If the surrounding facility SF is a shop, examples of detailed information include business hours, sales information, and the distance to the surrounding facility SF. If the surrounding facility SF is a station, examples of detailed information include line information, operation information, and the distance to the surrounding facility SF.

In the example of Fig. 29, a notification sound SD (facility notification sound SD _F ) indicating the position of the peripheral facility SF is presented to the user US in stereophonic sound. For example, the facility notification sound SD _F is notified as a stereophonic sound having a sound source localization at the position of the peripheral facility SF (e.g., the center of the peripheral facility SF, or a position registered in a map database of the server SV) or on the passage PA facing the peripheral facility SF. The user US can recognize the existence and position of the peripheral facility SF based on the facility notification sound SD _F.

In the example of FIG. 29, outdoor facilities such as banks and train stations are shown as the surrounding facilities SF, but the surrounding facilities SF are not limited to outdoor facilities. The surrounding facilities SF that are the subject of notifications may also be indoor facilities such as stores in a shopping mall or partitioned sales corners in a mass retailer.

The user US can select a surrounding facility SF of interest by a preset trigger operation. Examples of the trigger operation include turning the head toward the surrounding facility SF (sound source location of the facility notification sound _SDF ), or performing a voice operation, a button operation, or a gesture operation using a UI (User Interface) 25 while the head is facing the surrounding facility SF. For example, when a trigger operation is performed during notification of the facility notification sound _SDF (for example, within a certain period of time from the start of notification), detailed information of the surrounding facility SF and a travel route RT to the surrounding facility SF are presented.

FIG. 30 is a diagram showing an example of the main configuration of information processing device 3 of this modified example. FIG. 30 illustrates only the configuration necessary for providing information about nearby facilities, and does not exclude the configurations of the above-mentioned embodiments (the configuration of information processing device 1 shown in FIG. 8 and the configuration of information processing device 2 shown in FIG. 23). The following description focuses on the differences from the above-mentioned embodiments.

In addition to the configuration of each of the embodiments described above, the information processing device 3 has a surrounding information acquisition unit 21, a self-location acquisition unit 22, a route information/facility information acquisition unit 23, an operation determination unit 24, and a UI 25.

The self-location acquisition unit 22 acquires the location of the user US, such as longitude and latitude, using a GPS/GNSS or the like installed in headphones or a mobile information terminal (client terminal TM) owned by the user US. The surrounding information acquisition unit 21 acquires map information around the self-location from the server and creates a list of surrounding facilities SF (notification target facilities) to be notified. When setting sound source localization on an aisle PA, the surrounding information acquisition unit 21 acquires the self-location and information on the aisle around the facility.

The surrounding information acquisition unit 21 can customize the surrounding facilities SF to be notified based on the facility usage history of the user US. Customization means selecting and prioritizing the surrounding facilities SF to be notified. For example, the facility usage history specifies the date and time of use by the user US for each surrounding facility SF. The facility usage history is linked to the user US and stored on the server SV. The surrounding information acquisition unit 21 can also customize the surrounding facilities SF to be notified based on the detection frequency of the trigger action for each surrounding facility SF.

Customization can be done using machine learning models. The machine learning model inputs user attributes, usage history, shopping history, etc., and outputs notification settings optimized for the user's habits. For example, for a user who goes to a specific hospital on Monday morning and then to a pharmacy every week, it can be set to prioritize notifications for the hospital and pharmacy on that day. One possible scenario would be to use the information that the user has visited the hospital as a trigger to prioritize notifications for the pharmacy.

Customization can also be performed by referring to the settings of other users. For example, the server SV holds facility usage histories of various users who have used the navigation system NV in the past. The machine learning model can use the facility usage histories of users with similar attributes to learn the selection and prioritization of surrounding facilities SF to be notified of.

In addition, information on the surrounding facilities SF and surrounding passageways can also be obtained by image recognition processing using a camera. For example, information on the surrounding facilities SF and surrounding passageways can be obtained from pattern matching of facility shapes, character recognition on signs, road surface recognition, etc. Distance information on the surrounding facilities SF may also be obtained from a depth sensor, etc. In this case, the surrounding information acquisition unit 21 is provided on the client terminal TM side, not on the server SV side. Also, by setting the position of the camera as the self-position, the self-position acquisition unit 22 can be omitted.

When there are multiple facilities to be notified around the user US, the facility notification sound _SDF is notified for each of the facilities to be notified. The order of the peripheral facilities SF to be notified can be set arbitrarily. For example, the notification sound output unit 17 can output the facility notification sounds _SDF of multiple peripheral facilities SF in the order of the furthest or closest distance between the peripheral facility SF and the user US. The user US selects and determines the desired peripheral facility SF using voice, gestures, etc. as a selection and determination operation. The operation determination unit 24 detects a selection and determination operation that specifies the direction of the sound source position of the facility notification sound _SDF as a trigger operation.

The selection/decision operation may include a selection operation and a decision operation. The selection operation is an operation for selecting the direction of the sound source localization of the facility notification sound _SDF . The decision operation is an operation for deciding a peripheral facility SF having a sound source localization in the selected direction as a desired peripheral facility SF. The decision operation may include a trigger operation for causing the information processing device 3 to carry out a necessary process (such as reading out detailed information or route guidance) for the decided peripheral facility SF.

For example, the selection operation may include an action of turning the head of the user US in the direction of the sound source localization of the facility notification sound SD _F within a predetermined time from the emission of the facility notification sound SD _F. The operation determination unit 24 can determine that the selection operation has been performed when the sound source localization is directed in front of the face or ear of the user US within a certain time from the start of notification by the facility notification sound SD _F.

The decision operation can include a predetermined button operation, a voice operation, and a gesture operation. The operation determination unit 24 can determine whether or not a decision operation has been performed based on sensor information (such as a touch panel or microphone signal). The decision operation can also be performed as an action of maintaining the position of the head facing the direction of the sound source localization for a predetermined period of time during the selection operation. This allows the selection and decision of a surrounding facility SF to be performed with a single action of turning the face in the direction of the surrounding facility SF. In order to distinguish this from the face just happening to be facing in the direction of the surrounding facility SF, a requirement that the up and down posture of the face be close to horizontal can also be added to the requirements for the decision operation.

The decision operation may include an operation by the user US in response to a notification of detailed information about the surrounding facility SF made in response to a selection operation. For example, in response to a decision operation made during notification of detailed information (e.g., within a certain period of time from the start of notification), the route planning unit 11 may plan a travel route RT with the notified surrounding facility SF as the destination. In this case, the decision operation may include an action of starting to move toward the surrounding facility SF. This allows for intuitive operation. The navigation system NV may also be equipped with a mechanism for earning advertising revenue when the user US selects or visits a surrounding facility SF.

The route information and facility information acquisition unit 23 reads information related to the facility to be notified from the storage device on the server SV or the client terminal TM. The route information and facility information acquisition unit 23 reads route information for route guidance from the storage device on the server SV or the client terminal TM.

Figure 31 shows an example of a processing flow for providing information about nearby facilities.

The surrounding information acquisition unit 21 acquires the position of the surrounding facility SF (notification target facility) that is the subject of notification from the map information stored in the server SV (step S31). The self-position acquisition unit 22 acquires the position of the head of the user US as the self-position using a GPS or the like (step S32). The head direction acquisition unit 13 acquires the direction of the head of the user US using a posture sensor (geomagnetic sensor, etc.) mounted on the head-worn device (step S33).

The head direction of the user US can also be obtained from the image of a surveillance camera installed around the user US. In this case, the server SV detects the head direction from the image of the user US captured by the surveillance camera and outputs it to the head direction acquisition unit 13. The head direction of the user US can also be obtained from the image of a camera mounted on the neckband.

The sound source localization calculation unit 14 acquires the position and orientation of the head of the user US as user position information. The sound source localization calculation unit 14 calculates the direction and distance to the notification target facility from the user position information and the position information of the notification target facility (step S34). The sound source localization calculation unit 14 sets a position away from the user US by the calculated direction and distance as the sound source localization of the facility notification sound _SDF .

The notification sound output unit 17 notifies the user US of the facility names of the surrounding facilities SF, etc., using the facility notification sounds _SDF generated as stereophonic sounds (step S35). If music or other notification sounds are being played when making the notification, the volume adjustment unit 16 may mix the facility notification sounds _SDF with sounds other than the facility notification sounds _SDF so that sounds other than the facility notification sounds _SDF are reduced.

The operation determination unit 24 determines whether or not a trigger operation related to the selection and determination of the surrounding facility SF has occurred based on the sensor information. For example, the operation determination unit 24 determines whether or not the user US has turned his/her face or the like in the direction of the surrounding facility _SF being notified as a selection operation during notification of the facility notification sound SDF (step S36). If no selection operation is detected (step S36: No), the information processing device 3 does not provide surrounding facility guidance.

When the selection operation is detected (step S36: Yes), the information processing device 3 notifies (feeds back) that the user US is facing the direction of the notified surrounding facility SF by sound or vibration (step S37). As the feedback means, the notification sound output unit 17 or a vibrator mounted on the head-mounted device can be used. For example, the feedback means provides feedback to the user US when the direction of the head of the user US is within a predetermined angle range centered on the direction of the sound source localization of the facility notification sound _SDF .

The operation determination unit 24 determines whether or not a trigger operation that serves as a decision operation has been performed during the notification (step S38). Decision operations include voice operation, button operation, and gesture operation. If a trigger operation is not detected (step S38: No), the information processing device 3 does not provide guidance to nearby facilities.

When the trigger action is detected (step S38: Yes), the information processing device 3 performs processing according to the trigger action. Examples of the processing include reading out detailed information about the surrounding facility SF and providing route guidance to the surrounding facility SF (step S39). When the trigger action related to route guidance is detected, the route planning unit 11 responds to the trigger action of the user US in response to the facility notification sound _SDF indicating the location of the surrounding facility SF, and plans a travel route RT having the surrounding facility SF as a destination.

Figures 32 to 35 show examples of notifications for surrounding facilities SF.

In the example of Fig. 32, the sound source location of the facility notification sound _SDF is set on the passage PA facing the surrounding facility SF. When the surrounding facility SF faces the user US's own passage PA _-C , the sound source location of the facility notification sound _SDF can be set on the user US's own passage PA _-C . The user US's own passage PA _-C means the passage PA through which the user US is moving. When there are multiple branches on the passage PA, the passage PA in the straight direction becomes the user US's own passage PA- _C .

The notification sound output unit 17 presents the facility notification sound _SDF of the surrounding facility SF facing the user US's own passage _PAC to the user US on the user's own passage _PAC . By setting the sound source localization on the user's own passage _PAC , the user US can easily grasp the position of the surrounding facility SF. The user US can move toward the surrounding facility SF by relying on the facility notification sound _SDF .

In the example of Fig. 33, the sound source localization of the facility notification sound _SDF is set on the passage PA intersecting with the user's own passage PA _C. The notification sound output unit 17 presents the facility notification sound _SDF when the user US reaches the intersection between the passage PA where the sound source localization is set and the current user's own passage PA _C.

In the example of Fig. 34, the sound source location of the facility notification sound _SDF is set on a search route SP searched as a route from the surrounding facility SF to the user US. The sound source location of the facility notification sound _SDF is relocated from the position of the surrounding facility SF registered in a map database or the like to a position on the search route SP that is a branch point from the own passage _PAC when accessing the surrounding facility SF.

For example, the facility notification sound _SDF is notified as a stereophonic sound having a sound source localization at a position on an overlapping passage OV where the search route SP and the user's own passage _PAC of the user US overlap. In the example of Fig. 34, the overlapping portion between the user's own passage _PAC and the search route SP to the university is shown as an overlapping passage OV1. The overlapping portion between the user's own passage _PAC and the search route SP to the bank or station is shown as an overlapping passage OV2.

The sound source localization of facility notification sounds _SDF for multiple peripheral facilities SF (e.g., a bank and a train station) is set on the overlapping passage OV2. In this case, the order of notification of each facility notification sound _SDF can be set according to the distance to the peripheral facility SF. For example, the notification sound output unit 17 can output the facility notification sounds _SDF for multiple peripheral facilities SF whose sound source localization is set on the overlapping passage OV2 in order of the furthest or closest distance between the peripheral facility SF and the user US.

By setting the sound source localization of the facility notification sound _SDF on the overlapping passage OV, the position of the surrounding facility SF can be easily grasped. The user US can move toward the surrounding facility SF by relying on the facility notification sound _SDF . In this case, if the sound source localization is set at the position of the corner (branch portion) toward the surrounding facility SF, the user US can smoothly change the course toward the surrounding facility SF. Therefore, it is preferable that the sound source localization calculation unit 14 sets the sound source localization of the facility notification sound _SDF to the branch portion branching from the own passage _PAC along the search route SP.

33 and 34, the sound source localization of the facility notification sound _SDF is set to the periphery of the surrounding facility SF. The sound source localization may be set anywhere in the periphery, but if it is set near the entrance of the surrounding facility SF, it is easier to enter the surrounding facility SF. Therefore, it is preferable that the sound source localization calculation unit 14 sets the sound source localization of the facility notification sound _SDF on the passage PA facing the entrance of the surrounding facility SF.

The position of the entrance of the surrounding facility SF can be obtained from the map information stored in the server SV. If the position of the entrance is not registered in the map information, the server SV can estimate the position of the entrance based on information on the flow of visitors to the surrounding facility SF and register it in the map information. A visitor means a past user US who has visited the surrounding facility SF using the navigation system NV. The flow of the user US is calculated based on the user position information transmitted to the server SV. The sound source localization calculation unit 14 can set the sound source localization of the facility notification sound _SDF based on the position of the entrance detected based on the flow of visitors to the surrounding facility SF.

In the example of Fig. 35, the own passage _PAC is curved. In this case, the sound source localization calculation unit 14 can set the sound source localization of the facility notification sound _SDF to a position where a straight line connecting the user US and the sound source localization of the facility notification sound _SDF does not go beyond the own passage _PAC . This prevents the user US from going off the path even if he or she moves straight toward the sound source localization.

FIG. 36 shows an example of a processing flow for setting the sound source localization in FIG. 34.

The surrounding information acquisition unit 21 acquires the location of the surrounding facility SF and information on the passage around the surrounding facility SF from a map database of the server SV or the like (step S41). The self-location acquisition unit 22 acquires the self-location from sensor information or the like. The self-location acquisition unit 22 also acquires information on the self-path _PAC from the self-location and the passage information (step S42). The head direction acquisition unit 13 acquires the head direction of the user US from the sensor information or the like (step S43).

The sound source localization calculation unit 14 determines the sound source localization of the facility notification sound _SDF on the passage PA facing the surrounding facility SF (step S44). The sound source localization calculation unit 14 determines whether the sound source localization of the facility notification sound _SDF exists on the own passage PA _-C (step S45). If the sound source localization of the facility notification sound _SDF exists on the own passage PA _-C (step S45: Yes), the notification sound output unit 17 outputs the facility notification sound _SDF .

If the sound source location of the facility notification sound _SDF is not on the own passage _PAC (step S45: No), the sound source location calculation unit 14 relocates the sound source location of the facility notification sound _SDF to a position on the search route SP that is close to the own passage _PAC and serves as a branch point when accessing the surrounding facility SF (step S47).

[Additional Notes]
The present technology can also be configured as follows.
(1)
A direction acquisition unit that acquires a direction of travel of a user's route;
a sound source localization calculation unit that calculates a position a predetermined distance away from the user in the traveling direction as a sound source localization of a notification sound that guides the user to move in the traveling direction;
An information processing device having the above configuration.
(2)
The sound source localization calculation unit calculates a position away from the user by the predetermined distance in the traveling direction as a sound source localization of the notification sound based on a position and orientation of the user's head.
The information processing device according to (1) above.
(3)
A route planning unit that plans a route to a destination,
The traveling direction acquisition unit acquires a plurality of routes partitioned along the travel route, and acquires, for each of the partitioned routes, a direction along the route as the traveling direction of the route.
The information processing device according to (1) or (2) above.
(4)
The distance from the user to the sound source location of the notification sound is constant regardless of the position of the user.
An information processing device according to any one of (1) to (3) above.
(5)
The traveling direction acquisition unit acquires a traveling direction of a next route following the route,
The sound source localization calculation unit calculates a position a predetermined distance away from the user in a traveling direction of the next route as a sound source localization of a notification sound that guides the user to move to the next route.
An information processing device according to any one of (1) to (4) above.
(6)
The distance from the user to the sound source location of the notification sound that guides the user to move to the next route is constant regardless of the user's position.
The information processing device according to (5) above.
(7)
a volume adjustment unit that adjusts volumes of the two notification sounds related to the current course and the next course based on a distance from a midpoint that is a boundary between the current course and the next course to the user, and generates a notification sound obtained by mixing the two notification sounds whose volumes have been adjusted as a notification sound that induces switching from the current course to the next course;
The information processing device according to (5) or (6) above.
(8)
The volume adjustment unit increases the volume of the notification sound that guides the user to move to the next route as the user approaches the waypoint on the current route.
The information processing device according to (7) above.
(9)
the volume adjustment unit increases the volume of the notification sound that guides the user to move in the traveling direction of the next route as the user moves away from the waypoint on the next route.
The information processing device according to (8) above.
(10)
The volume adjustment unit reduces the volume of the notification sound that guides the user to move in a traveling direction of the current course as the user approaches the waypoint on the current course.
The information processing device according to (8) or (9) above.
(11)
the volume adjustment unit reduces the volume of the notification sound that guides the user to move in the traveling direction of the previous route as the user moves away from the waypoint on the next route.
The information processing device according to (10) above.
(12)
The sound source localization calculation unit calculates a position at the predetermined distance in the direction of the midpoint as the sound source localization of the notification sound when the user passes the midpoint and moves in a direction different from the next route.
An information processing device according to any one of (7) to (11) above.
(13)
a sound generating unit that, when the moving direction of the user is within a predetermined angle range centered on the moving direction, adds a sound effect to the notification sound that guides the user to move in the moving direction;
13. The information processing device according to any one of (1) to (12) above.
(14)
The sound source localization calculation unit calculates a position on the boundary between the warning area and the danger area as the sound source localization of the warning sound.
14. The information processing device according to any one of (1) to (13) above.
(15)
The sound source localization calculation unit calculates a planar area along a boundary between the warning area and the danger area as a sound source localization of the warning sound.
The information processing device according to (14) above.
(16)
The warning area is an area that is a certain distance or more away from the flow line that is the center of the path, and the danger area is an area that is a certain distance or more away from the warning area toward the outside.
The information processing device according to (14) or (15) above.
(17)
The sound source localization calculation unit identifies a dangerous area based on sensor information and sets the warning area and the danger area.
17. The information processing device according to any one of (14) to (16) above.
(18)
an angle difference calculation unit that calculates an angle difference between a direction of the user's head and the traveling direction;
a notification sound output unit that outputs the notification sound for guiding movement in the traveling direction when the angular difference is within a notification range;
18. The information processing device according to any one of (1) to (17) above,
(19)
the notification sound output unit outputs the notification sound as a pulse sound,
The notification sound output unit increases the frequency of the pulse sound notification as the angle difference becomes smaller.
The information processing device according to (18) above.
(20)
A motion detection unit that detects a motion of the user's head,
the notification sound output unit outputs the notification sound for guiding the user to move in the traveling direction when the angular difference is within a notification target range and the user's head is moving in a direction approaching the traveling direction.
The information processing device according to (18) or (19) above.
(21)
the notification sound output unit starts outputting the notification sound after a preset time interval has elapsed since the user's head starts to move in a direction approaching the traveling direction,
The information processing device described in (20) above.
(22)
the route planning unit, in response to a trigger operation by the user in response to a facility notification sound indicating a location of a peripheral facility, plans the travel route with the peripheral facility as the destination;
The information processing device according to (3) above.
(23)
The facility notification sound is notified as a stereophonic sound having a sound source localization on the location of the peripheral facility or on the passageway facing the peripheral facility.
The information processing device described in (22) above.
(24)
The sound source localization calculation unit sets the sound source localization of the facility notification sound on the passageway facing the entrance of the surrounding facility.
The information processing device described in (23) above.
(25)
The sound source localization calculation unit sets the sound source localization of the facility notification sound based on a position of the entrance detected based on a flow line of visitors to the peripheral facility.
The information processing device according to (24) above.
(26)
an operation determination unit that detects a selection/determination operation that specifies a direction of the sound source localization of the facility notification sound as the trigger operation;
26. The information processing device according to claim 23, wherein the information processing device is a
(27)
The selection/confirmation operation includes a selection operation for selecting a direction of the sound source localization of the facility notification sound, and a confirmation operation by the user in response to a notification of detailed information of the surrounding facility performed in response to the selection operation.
The information processing device described in (26) above.
(28)
the selection operation includes an action of directing the user's head in the direction of the sound source localization of the facility notification sound within a predetermined time after the facility notification sound is emitted,
The information processing device described in (27) above.
(29)
a feedback means for providing feedback to the user when the direction of the user's head is within a predetermined angle range centered on the direction of the sound source localization of the facility notification sound;
The information processing device described in (28) above.
(30)
The decision operation includes an action of starting to move toward the surrounding facility.
30. The information processing device according to any one of (27) to (29) above.
(31)
a surrounding information acquisition unit that customizes the surrounding facilities to be notified based on the facility usage history of the user;
31. The information processing device according to claim 29, wherein the information processing device is a
(32)
The surrounding information acquisition unit customizes the surrounding facilities to be notified based on a detection frequency of the trigger operation for each surrounding facility.
The information processing device described in (31) above.
(33)
The facility notification sound is notified as a stereophonic sound having a sound source localization at a position on an overlapping path where a search route searched as a route from the peripheral facility to the user and a self-path on which the user is moving overlap.
The information processing device described in (22) above.
(34)
The sound source localization calculation unit sets the sound source localization of the facility notification sound to a branching portion that branches off from the own passage along the search route.
The information processing device described in (33) above.
(35)
When the user's path is curved, the sound source localization calculation unit sets the sound source localization of the facility notification sound to a position such that a straight line connecting the user and the sound source localization of the facility notification sound does not extend beyond the user's path.
The information processing device described in (33) above.
(36)
a notification sound output unit that outputs facility notification sounds of a plurality of peripheral facilities whose sound source localization is set on the overlapping passage in order of distance between the peripheral facilities and the user, in ascending or descending order;
36. The information processing device according to claim 33, wherein the information processing device is a
(37)
Obtaining the user's direction of travel;
A position that is a predetermined distance away from the user in the traveling direction is calculated as a sound source localization of a notification sound that guides the user to move in the traveling direction.
23. A computer-implemented information processing method comprising:
(38)
Obtaining the user's direction of travel;
A position that is a predetermined distance away from the user in the traveling direction is calculated as a sound source localization of a notification sound that guides the user to move in the traveling direction.
A program that makes a computer do this.
(39)
A sensor unit for detecting a position and a direction of a user's head;
The information processing device according to any one of (1) to (36) above, which calculates a position a predetermined distance away from the user in a traveling direction of the user's path based on a position and direction of the user's head as a sound source localization of a notification sound that guides the user to move in the traveling direction;
a speaker that reproduces a sound image of the notification sound at the calculated sound source localization;
A navigation system having:

1, 2, 3 Information processing device 11 Route planning unit 12 Travel direction acquisition unit 14 Sound source localization calculation unit 15 Sound generation unit 16 Volume adjustment unit 17 Notification sound output unit 151 Angle difference calculation unit 152 Movement detection unit BD Boundary BP Midpoints CS, CS _C route CS _Nth route DA Danger area HD Head direction (head orientation)
OV overlapping passage PA passage PA _C own passage RT moving route SD, SD _C , SD _N notification sound SD _F facility notification sound SF surrounding facility SP search route TD, TD _C , TD _N direction of travel US user WA warning area WS warning sound

Claims (20)

A direction acquisition unit that acquires a direction of travel of a user's route;
a sound source localization calculation unit that calculates a position a predetermined distance away from the user in the traveling direction as a sound source localization of a notification sound that guides the user to move in the traveling direction;
An information processing device having the above configuration. The sound source localization calculation unit calculates a position away from the user by the predetermined distance in the traveling direction as a sound source localization of the notification sound based on a position and a direction of the user's head.
The information processing device according to claim 1 . A route planning unit that plans a route to a destination,
The traveling direction acquisition unit acquires a plurality of routes partitioned along the travel route, and acquires, for each of the partitioned routes, a direction along the route as the traveling direction of the route.
The information processing device according to claim 1 . The distance from the user to the sound source location of the notification sound is constant regardless of the position of the user.
The information processing device according to claim 1 . The traveling direction acquisition unit acquires a traveling direction of a next route following the route,
The sound source localization calculation unit calculates a position a predetermined distance away from the user in a traveling direction of the next route as a sound source localization of a notification sound that guides the user to move to the next route.
The information processing device according to claim 1 . The distance from the user to the sound source location of the notification sound that guides the user to move to the next route is constant regardless of the user's position.
The information processing device according to claim 5 . a volume adjustment unit that adjusts volumes of the two notification sounds related to the current course and the next course based on a distance from a midpoint that is a boundary between the current course and the next course to the user, and generates a notification sound obtained by mixing the two notification sounds whose volumes have been adjusted as a notification sound that induces switching from the current course to the next course;
The information processing device according to claim 5 . The volume adjustment unit increases the volume of the notification sound that guides the user to move to the next route as the user approaches the waypoint on the current route.
The information processing device according to claim 7. the volume adjustment unit increases the volume of the notification sound that guides the user to move in the traveling direction of the next route as the user moves away from the waypoint on the next route.
The information processing device according to claim 8. The volume adjustment unit reduces the volume of the notification sound that guides the user to move in a traveling direction of the current course as the user approaches the waypoint on the current course.
The information processing device according to claim 8. the volume adjustment unit reduces the volume of the notification sound that guides the user to move in the traveling direction of the previous route as the user moves away from the waypoint on the next route.
The information processing device according to claim 10. The sound source localization calculation unit calculates a position at the predetermined distance in the direction of the midpoint as the sound source localization of the notification sound when the user passes the midpoint and moves in a direction different from the next route.
The information processing device according to claim 7. a sound generating unit that, when the moving direction of the user is within a predetermined angle range centered on the moving direction, adds a sound effect to the notification sound that guides the user to move in the moving direction;
The information processing device according to claim 1 . The sound source localization calculation unit calculates a position on the boundary between the warning area and the danger area as the sound source localization of the warning sound.
The information processing device according to claim 1 . The sound source localization calculation unit calculates a planar area along a boundary between the warning area and the danger area as a sound source localization of the warning sound.
The information processing device according to claim 14. The sound source localization calculation unit identifies a dangerous area based on sensor information and sets the warning area and the danger area.
The information processing device according to claim 14. an angle difference calculation unit that calculates an angle difference between a direction of the user's head and the traveling direction;
a notification sound output unit that outputs the notification sound for guiding movement in the traveling direction when the angular difference is within a notification range;
The information processing apparatus according to claim 1 , the notification sound output unit outputs the notification sound as a pulse sound,
The notification sound output unit increases the frequency of the pulse sound notification as the angle difference becomes smaller.
The information processing device according to claim 17. Obtaining the user's direction of travel;
A position that is a predetermined distance away from the user in the traveling direction is calculated as a sound source localization of a notification sound that guides the user to move in the traveling direction.
23. A computer-implemented information processing method comprising: Obtaining the user's direction of travel;
A position that is a predetermined distance away from the user in the traveling direction is calculated as a sound source localization of a notification sound that guides the user to move in the traveling direction.
A program that makes a computer do this.

Images