KR102023573B1 - System and method for providing intelligent voice imformation - Google Patents

Abstract

Embodiments may include a spatial context converter configured to generate a first spatial context obtained by converting an object of interest included in a target space into a node; A spatial processor configured to calculate a location of a user located in the target space and to generate a second spatial context by combining the location of the user and the first spatial context; And a voice processing unit configured to generate an answer corresponding to the voice query based on the second spatial context when the voice query associated with the target space is received, and the intelligent voice performed by the system. It relates to a method of providing information.

Description

Intelligent voice information providing system and method {SYSTEM AND METHOD FOR PROVIDING INTELLIGENT VOICE IMFORMATION}

Embodiments of the present invention relate to providing information, and more particularly, to analyze a space in which a user is located in a space context, and to input a voice answer based on the space context when a voice query related to the space is input. It is about providing.

Recently, due to the development of voice recognition technology and AI (Artificial Intelligence) technology, voice information providing technology for understanding the meaning included in the human voice and providing an answer corresponding thereto is also being developed. The related voice information service market (e.g., intelligent virtual assistant market) includes various voice information service providers of overseas global companies (e.g., Apple's Siri, Amazon's Echo, and Google's Google Assistant). Assistant), Microsoft's Cortana, and Samsung's Bixby) are growing rapidly.

Currently, voice information providing services are being advanced to provide answers to voices recognized in various aspects such as voice, visual and / or tactile based on advanced information processing technology, beyond simply voice recognition and voice information providing technology. Services are particularly helpful for users who are visually uncomfortable or disabled.

However, the current voice information providing service recognizes the space where the user is located based on the GPS-based macroscopic location technology and provides information on the space. Therefore, there is a problem in that the specific space is not recognized. For this reason, there is a limit in that specific voice information cannot be provided through the current voice information providing service for the micro space where the above users are currently located.

Patent Publication No. 10-2014-0068855 Patent Publication No. 10-2016-0069329

According to embodiments of the present invention, there is provided a system and method for analyzing a space in which a user is located in a space context, and providing a voice answer based on the space context when a voice query associated with the space is input. do.

An intelligent voice information providing system according to an aspect of the present invention comprises: a spatial context converter configured to generate a first spatial context obtained by converting an object of interest included in a target space into a node; A spatial processor configured to calculate a location of a user located in the target space and to generate a second spatial context by combining the location of the user and the first spatial context; And a voice processor configured to generate an answer corresponding to the voice query based on the second spatial context when the voice query associated with the target space is received.

In one embodiment, the spatial context converter obtains the raw image information of the target space, analyzes the raw image information, and shapes the object of interest included in the target space into an object of interest node, the object of interest node and the interest. It may be further configured to create a first spatial context that includes a connecting line connecting the object nodes.

In one embodiment, the spatial context transform unit reconstructs the object space into a three-dimensional basic space based on the raw image information, recognizes and indexes the object of interest located in the three-dimensional basic space, and the indexed object. May be further configured to shape the object of interest.

In one embodiment, the spatial context converter divides the three-dimensional basic space into segments including a single object of interest, and may be further configured to apply a spatial shape corresponding to the object of interest to each segment.

In one embodiment, the spatial processor shapes the user into a user node based on the user location, includes the user node in the first spatial context, sets the user node as a starting point, and alternates between each node. It may be further configured to create a second spatial context that includes connecting lines that do not pass through the node.

The spatial processor may be further configured to acquire a first-person image corresponding to the first-person view of the user and calculate a location of the user based on a shape of the object of interest included in the first-person view image. Can be.

In an embodiment, the spatial processor is further configured to acquire a third person image corresponding to the third person view of the user and calculate a location of the user based on the shape of the user included in the third person view image. Can be.

In an embodiment, when only the first object of interest is included in the content of the voice query, the answer may be expressed as a relative positional relationship with the second object of interest different from the first object of interest.

In one embodiment, the distance may be determined based on a distance from the first object of interest.

In an embodiment, the second object of interest may be determined based on attributes of the object of interest located in the target space.

In an embodiment, when a sub space of the target space is associated with the content of the voice query, the answer may be generated using the location of the sub space. Here, the location of the subspace may be searched for an object of interest associated with the subspace and determined based on the searched object of interest.

In one embodiment, the location of the subspace may be a center coordinate with respect to the location of the searched object of interest.

An intelligent voice information providing method performed by an intelligent voice information providing system according to another aspect of the present invention includes the steps of: obtaining a first spatial context in which an object of interest included in a target space is shaped into a node; Calculating a location of a user located in the target space; Combining a location of the user with the first spatial context to create a second spatial context; If a voice query associated with the target space is received, generating an answer corresponding to the voice query based on the second spatial context.

In an embodiment, the acquiring of the first spatial context may include acquiring raw image information of the object space; Analyzing the raw image information and shaping an object of interest located in the target space into an object of interest node; And generating a first spatial context including a connection line connecting the object of interest node and the object of interest node.

In an embodiment, analyzing the raw image information and shaping the object of interest located in the target space into the object of interest node may include reconfiguring the target space into a 3D basic space based on the raw image information; Recognizing an object of interest included in the three-dimensional basic space; Indexing the recognized object of interest; And shaping the indexed object into an object of interest node.

In an embodiment, analyzing the raw image information and shaping the object of interest located in the target space into an object of interest node may include: dividing the three-dimensional basic space into segments including a single object of interest; And applying a spatial shape corresponding to the single object of interest to the segment.

In one embodiment, creating the second spatial context comprises shaping the user into a user node based on the user location; And including the user node in the first spatial context.

In one embodiment, creating the second spatial context comprises setting the user node as a starting point; And generating a connection line connecting each node so as not to pass through another node.

In an embodiment, generating an answer corresponding to the voice query using the second spatial context may include analyzing the contents of the voice query and requesting necessary information; Searching for a node included in the second spatial context in response to the request; Generating and returning necessary information based on the search result; And generating an answer using the returned necessary information.

 When the instructions stored in the computer-readable recording medium are executed, the processor may perform the intelligent voice information providing method according to the embodiments.

According to the embodiments of the present invention, the user's location is reinterpreted in the context of a space composed of nodes and connecting lines, and the user's location of the object or the user's location within the space is transmitted to the user as the synesthesia information about the space and the time through voice. Can provide. Here, synesthesia information represents information representing a query object in a positional relationship with other objects.

As a result, the user is provided with a relative positional expression of the position of the object (user or object) to be known, so that the position of the object can be known more intuitively. In particular, the position of the object can be known by the relative positional relationship with the user can maximize the user convenience.

In addition, it can provide convenience for a user who is visually inconvenient to live in an indoor space where they spend most of their time. Since this process does not use macroscopic location-based technology such as GPS, it can smoothly provide services to the user even in indoor spaces where GPS signals are difficult to function.

In addition, the use of a position relative to the space allows a relatively simplified spatial context to be used to provide a faster processing speed.

The effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned will be clearly understood by those skilled in the art from the description of the claims.

1 is a conceptual diagram of an intelligent voice information providing system according to an embodiment of the present invention.
2 is a flowchart of an intelligent voice information providing method performed by an intelligent voice information providing system according to an embodiment of the present invention.
3 is a flowchart of a spatial preprocessing process according to an embodiment of the present invention.
4 is a diagram illustrating an image of a spatial preprocessing process according to an embodiment of the present invention.
5 is a flowchart of a process of generating an answer corresponding to a voice query according to an embodiment of the present invention.
The drawings depict various embodiments of the invention for illustration purposes only. Those skilled in the art will readily appreciate from the following description that alternative embodiments of the structures and methods described herein may be used without departing from the principles of the invention described herein.

Embodiments will be described with reference to the accompanying drawings, however, the principles disclosed herein may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In the detailed description of the invention, detailed descriptions of well-known features and techniques may be omitted to avoid unnecessarily obscuring the features of the embodiments.

In the present specification, a space where a user can be located generally refers to a living space where the user lives (eg, an entire house or a specific room), but a space where the user can spend a considerable time (eg, a workplace, a family home, Relatives' homes, etc.) may also be referred to.

In addition, in the present specification, the interior space is generally at least temporarily and / or partially blocked from the outside, and various spaces that can be conceptually divided with a certain area (eg, a space that is open for a considerable time while opening and closing a door, a vehicle Internal, rectal partitions, etc.).

In this specification, the object space refers to an object that is transformed into a space context in which a user's positionable space is more simply redefined.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a conceptual diagram of an intelligent voice information providing system 1000 according to an embodiment of the present invention. The system 1000 includes a spatial context converter 100, a spatial processor 300, and a voice processor 500.

The intelligent voice information providing system 1000 according to the embodiments may be entirely hardware, or partly hardware and partly software. For example, each component of the system 1000 may collectively refer to hardware and software related thereto for processing data in a specific format and content, and / or for electronic communication. That is, as used herein, "unit", "module", "device" or "system" and the like refer to hardware, a combination of hardware and software, or a computer-related entity such as software. For example, parts, modules, devices, or systems herein refer to running processes, processors, objects, executables, threads of execution, programs, and / or computers. computer, but is not limited thereto. For example, both an application running on a computer and a computer may correspond to a part, module, device, system, or the like herein.

Each portion 100, 300, 500 constituting the intelligent voice information providing system 1000 according to the embodiments is not intended to refer to a separate component that is physically separated. That is, in FIG. 1, the spatial context converter 100, the spatial processor 300, and the voice processor 500 are illustrated as separate blocks that are separated from each other, but according to an exemplary embodiment, the spatial context converter 100, the space Some or all of the processing unit 300 and the voice processing unit 500 may be integrated in the same device. For example, the space processor 300 and the voice processor 500 may be integrated into one smartphone. In addition, each unit (100, 300, 500) is only functionally divided devices according to the operations performed by the computing device they are implemented, and does not necessarily mean separate elements separated from each other. However, as an example, in another embodiment, one or more of the spatial context converter 100, the spatial processor 300, and the voice processor 500 may be implemented as separate devices that are physically separated from the other units. For example, the spatial processor 300 may be a component communicatively connected to the spatial context converter 100.

The spatial context converting unit 100 recognizes objects included in the space in which the user can locate and re-defines the space based on the recognized objects. In an embodiment, the raw image information of the target space is obtained, the raw image information is analyzed to form an object of interest included in the target space as an object of interest, and the object of interest and the object of interest node are connected. Create a first spatial context that includes a connecting line.

The spatial context converter 100 may obtain a first two-dimensional imager capable of obtaining a general two-dimensional image, for example, a 360 degree camera, a? K camera, a deep camera, or a Microsoft It may be connected via wired / wireless with an RGB-D camera such as Kinect and a combination thereof. The image information obtained here is image information directly representing a target space, and is referred to as raw image information. In some embodiments, the image information about the target space may be acquired in advance by the above-described image information obtaining means. In yet another embodiment, the first imager may be a 3D scanner that scans the shape of the object.

The spatial context converter 100 re-defines the target space including one or more objects of interest into a spatial context by obtaining raw image information. The process of the spatial context converting unit 100 to recognize the object of interest in the target space and generate the spatial context is described in more detail in step S130 below. The spatial context converter 100 may provide the generated spatial context information to the spatial processor 300.

The space processor 300 analyzes the current state inside the target space. The space processor 300 obtains information related to the target space, that is, a spatial context in which the target space is converted, calculates a current position of the user, and generates a response using the spatial context when a voice query related to the target space is received. Do it.

In one embodiment, the spatial processor 300 may receive the spatial context generated by the spatial context converter 100 to recognize the target space where the user is located. In this case, since a separate processing process such as a process of recognizing objects located in the target space in which the user is currently located is not required, the user may recognize the user's space state more quickly and conveniently.

In another exemplary embodiment, the space processor 300 may analyze the current target space in which the user is located in real time and convert the space into a spatial context considering the user and the object of interest at the same time. That is, a graph including a user node and an object node may be generated. In this case, the spatial context converter 100 and the spatial processor 300 are integrated into one component.

In some other embodiments, the spatial processor 300 may further perform a process of correcting the information on the objects currently located in the target space in the spatial context information generated by the spatial context converter 100. In this case, the position of the moved objects after the preprocessing can be modified, thereby providing more accurate spatial information to the user.

The space processor 300 may calculate a location of a user located in the target space. In an embodiment, the spatial processor 300 may obtain a first-person image corresponding to the first-person view of the user and calculate a location of the user based on the shape of the object of interest included in the first-person view image. In this case, the space processor 300 may be connected to the second imager 331 by wire or wirelessly. The second imager 331 captures an image corresponding to the first person view of the user, and may be, for example, a smart phone. The image information acquired by the second imager 331 is referred to as first person image information. In the above embodiment, the spatial processor 300 may recognize objects included in the first-person image information and calculate a current position of the user by calculating a distance between the objects and the user.

In another embodiment, the spatial processor 300 may acquire a third person image corresponding to the third person view of the user and calculate location information of the user based on the shape of the user included in the third person view image. In this case, the space processor 300 may be connected to the third imager 333 by wire or wirelessly. The third imager 330 photographs an image corresponding to the third person view of the user, for example, may be CCTV. The image information acquired by the third imager 333 is referred to as third person image information. In the above embodiment, the spatial processor 300 may calculate the current position of the user based on the shape of the user included in the third-person image information.

The space processor 300 may combine the calculated user location and the information (ie, the spatial context) related to the target space, and provide various information related to the target space to the user based on this. In one embodiment, when the space processor 300 is requested from the voice processor 500 for information necessary to generate an answer corresponding to a user's voice query, the space processor 300 generates return information using a spatial context in which the user's location is reflected. And provide it. The process of generating the return information by the space processor 300 is described in more detail in step S550.

The voice processor 500 is a means for analyzing a voice query of an input user and providing a voice answer corresponding to the voice query. The voice processing unit 500 is connected to the voice information input / output device for voice inquiry and response by wire / wireless, and the voice processing unit 500 analyzes the input voice query of the user and provides a corresponding response. These are well known to those skilled in the art through Apple's Siri, Amazon's Echo and / or Published Patent Publication No. 10-2016-0069329, and the like, for the purpose of clarity of the invention. Detailed description will be omitted.

In one embodiment, the speech processing unit 500 analyzes whether the speech query includes a representation related to the target space. The expression related to the target space includes the location of the user or the objects located in the target space.

When the received voice query is analyzed as a voice query related to the target space (for example, a query for the location of an object located in the target space), the voice processor 500 analyzes the spatial information necessary for the response. Ask. Processes performed in the voice processing unit 500 are described in more detail in step S500.

The system 1000 may execute or manufacture various software based on an operating system (OS), that is, a system. The operating system is a system program for enabling the software to use the hardware of the device, and the mobile computer operating system such as Android OS, iOS, Windows Mobile OS, Sea OS, Symbian OS, Blackberry OS, Windows, Linux, Unix, It can include any computer operating system, such as MAC, AIX, or HP-UX.

It will be apparent to those skilled in the art that the system 1000 may include other components not described herein. For example, it may further include other hardware elements necessary for the operations described herein, such as a network, a network interface, and a protocol connecting the spatial context converter 100 and the spatial processor 300 and the voice processor 500. have.

2 is a flowchart of an intelligent voice information providing method performed by the intelligent voice information providing system 1000 according to an embodiment of the present invention. The intelligent voice information providing method may include converting a target space including one or more objects of interest into a first spatial context (S100); Recognizing a user located in the target space, generating a second spatial context reflecting the user's location (S300) and performing a voice query and response based on the second spatial context (S500).

3 is a flowchart of a spatial context transformation process S100 performed by the spatial context transformation unit 100 according to an embodiment of the present invention, and FIG. 4 is a spatial context according to an embodiment of the present invention. A diagram illustrating a corresponding image according to each step of the conversion process (S100).

3-4, step S100 is a step of converting a target space including one or more objects of interest into a spatial context. First, the spatial context converter 100 obtains raw image information through a first imager. (S110). As shown in FIG. 4A, the raw image is image information that directly expresses a state of a target space. Thereafter, the spatial context converter 100 analyzes the raw image information and shapes each object of interest included in the target space into nodes (S130).

Once the spatial context converter 100 acquires the raw image information (S110), the spatial context converter 100 reconstructs the target space into a 3D basic space based on the raw image information (S131). Here, the three-dimensional basic space is a basic space before the state of the object space (for example, what objects are located, etc.) is analyzed and is generated from a two-dimensional raw image as shown in FIG. 4 (a). As shown in (b), it refers to a three-dimensional virtual space that represents the interior structure of the target space without being distinguished between the background and the internal objects.

Thereafter, the spatial context converter 100 recognizes an object of interest located in the 3D basic space (S133). Here, the object of interest refers to the main objects that can be used to determine a relative position among a plurality of objects located in the target space. In one embodiment, the object of interest includes a large, non-moving object object, such as a TV, bookcase, sink, bed, sofa, or the like. In another embodiment, the object of interest may include a large, non-moving object object, as well as a small but important object object or a frequently used object object, such as a smart phone, a remote controller, and the like. .

For example, referring to FIGS. 4 (a) and 4 (b), the cup A and the book B placed on the dining table are used in comparison with an object object located in a target space, but relatively small in size and used in a smartphone. It is not an object of interest because it is common and is not commonly used to indicate relative positional relationships with other objects.

These objects of interest may be located in the target space, in terms of their position, size, position change, relative position to other objects (for example, when other objects are distributed evenly around an object, etc.) It can be determined based on at least one. In some embodiments, the object of interest may be determined by user specification.

The process of recognizing the object of interest by the spatial context converter 100 may be performed mechanically. In one embodiment, the spatial context converter 100 learns an object of interest based on an attribute associated with a pre-stored object of interest, analyzes the interior of the three-dimensional basic space based on the learning result, and includes the interest included in the three-dimensional basic space. Determine the type and location of the object.

In the above embodiment, the algorithm for learning the object of interest based on the attributes associated with the object of interest previously stored may include a support vector machine (SVM), a decision tree, a K-nearest neighbor, and a neural network algorithm. And various supervised learning algorithms.

In another embodiment, the spatial context converting unit 100 searches inside the 3D basic space for a shape that is similar to the shape of the pre-stored 3D object model, and if the shape is similar, the 3D object model is searched. Decide on Here, whether or not the shape is similar is determined by whether at least partially coincides with an attribute of each three-dimensional object model of interest.

In the above other embodiment, the spatial context converter 100 may further include a separate database (not shown). The database includes various three-dimensional object models and attribute information associated with each three-dimensional object model in order to recognize objects of interest that can be located in the indoor space. When the spatial context converter 100 further includes a database as in the above embodiment, the stored database information may be obtained from an external database or may be obtained from a user input. It will be apparent to those skilled in the art that the database may also include other data stores not explicitly mentioned herein.

When the object of interest included in the 3D basic space is recognized (S133), the spatial context converter 100 indexes and recognizes the recognized objects of interest (S137). In one embodiment, the spatial context transformation unit 100 indexes the recognized objects of interest by type. Each index is an index for identifying an object located in a real space and represents a TV for the first index, a table for the second index, a bed for the third index, and a window for the fourth index.

In one embodiment, step S130 is a step of dividing the three-dimensional basic space into segments including a single object of interest indexed in step S137 (S134) and applying a spatial shape corresponding to the object of interest to each segment. Step S135 may be further included. Here, the spatial shape is a shape capable of decomposing an object of interest into a minimum shape as shown in FIG. 4C, and is a shape that can simply and intuitively express an object such as a circle or a rectangle. In this case, step S137 indexes the spatial shape as shown in FIG.

When the 3D space is divided into segments including the object of interest, the target space may be partially analyzed based on the object of interest.

The spatial context converter 100 shapes the indexed objects of interest into objects of interest (S137). Here, the object of interest node is a node associated with the object of interest, where each object node indicates information about attributes and the location of the object of interest, such as the type, size of the object of interest, the subspace in which the object of interest is generally located, etc. Include. Here, the position of the object of interest represents three-dimensional coordinates.

In this way, by individually indexing the types of objects, the type of each object can be efficiently identified. In addition, since the user generally recognizes the shape of the indoor space based on the type of the object, it is the basis for providing the user with familiar and intuitive spatial information.

Then, the spatial context converting unit 100 may generate a first spatial context including a connection line connecting the object of interest node and the object of interest (S150). In other words, by re-defining a target space containing information that is actually complex and unnecessary in a simpler spatial context in terms of spatial analysis, it traverses the nodes contained in the graph without having to analyze all the actual internal information. Analyzes are possible and thus have an advantage in terms of processing speed.

The space processor 300 recognizes a user located in an actual target space and performs a step (S300) of combining a user's location with a spatial context including an object node of interest. In one embodiment, the spatial processor 300 obtains information analyzing the actual target space through the spatial context (that is, graph) generated by the spatial context converter 100, and reflects the user's location therein (S300). ).

In the above embodiment, since the spatial processing unit 300 obtains information (first spatial context) that primarily analyzes the target space by the spatial context processing unit 100, the spatial processing unit 300 additionally reflects the user's location.

In an embodiment, the space processor 300 calculates a location of a user located in the target space (S310).

In an embodiment, the space processor 300 obtains first-person image information corresponding to the first-person view of the user and calculates a location of the user in the target space based on the obtained first-person view image information (S310). . The spatial processor 300 may analyze the shapes of the objects included in the first-person view image information to determine where the user is currently located in the target space. In some embodiments, the spatial processor 300 may determine the location of the second imager 331 as the location of the user.

In another embodiment, the spatial processor 300 obtains third-person image information corresponding to the third-person view of the user, and calculates a location of the user in the target space based on the obtained third-person view image information (S310). . The spatial processor 300 captures the shape of the user included in the third-person view image and calculates the position of the user by calculating a relationship between objects other than the user and the background. The space processor 300 may determine the location of the user by using coordinates of a part or all of the user's body. In the above embodiment, the space processor 300 and the third camera 333 are configured to enable wired / wireless communication with each other.

The spatial processor 30 combines the determined user location with the first spatial context to generate a second spatial context (S330). The spatial processor 300 forms a user located in the target space as a user node based on the determined user position, and generates a final spatial context for generating a response to a voice query by including the user node in the first spatial context. (S330).

In one embodiment, the space processor 300 sets a user node as a starting point, and generates a connection line connecting the nodes so as not to pass through other nodes. Referring to FIG. 4 (d), the spatial context is composed of a user node, an object of interest node, and a connection line connecting them, so whether an object or a user is located in a target space may be searched for the existence of the node. The positional relationship can be easily identified through the connecting line. As a result, in step S500 below, an answer to the voice query may be generated more quickly and conveniently.

In some other embodiments, the starting point can be set to a node other than the user node. In another embodiment, the starting point can be set based on the orientation. For example, the starting point can be set to the object node of interest located at the easternmost point. In this case, referring to FIG. 5 below, when the voice processing unit 500 receives a question “what is the object located in the far east”, there is an advantage of searching for necessary information for generating an answer more quickly.

5 is a flowchart of a process of generating an answer corresponding to a voice query according to an embodiment of the present invention.

When the voice processor 500 receives a voice query related to the target space, the voice processor 500 generates and provides an answer corresponding to the voice query to the user (S500). Here, the generated answer is based on a second spatial context. .

The voice processing unit 500 receives a voice query and analyzes the content. The speech processor 500 determines that the speech query is related to the target space when the speech query includes a space related expression (for example, the location of the object). Here, it is irrelevant whether the object object is located in the actual target space.

As a result of the analysis, when it is determined that the voice query is a query related to the target space, the voice processing unit 500 requests the space processing unit 300 for information (hereinafter required information) necessary for generating a corresponding answer (S510).

In response to the request, the space processor 300 searches for whether there is necessary information through the second spatial context (S530). The space processor 300 may search for nodes of the graph along a connection line and analyze the positional relationship between objects and users located in the space.

In one example, when the user wants to search the object closest to the user, the object node included in the one-ring layer is first searched, and among these, the node located closest to the user node is determined.

Since the connection is not via other nodes, the relative position between the object of interest and the object of interest can also be easily analyzed by looking along the connection line.

The spatial processing unit 300 searches for necessary information to extract information for generating the necessary information from the second spatial context, generates return information based on the extracted information, and returns to the voice processing unit 500 (S550). The voice processor 500 generates an answer using the return information and provides the generated answer to the user (S570).

The return information generated by the spatial processor 300 may vary according to the contents of the voice query. .

In one embodiment, if the content of the voice query is associated with two or more objects of interest, the return information includes information related to the relative positional relationship between the objects of interest to be queried.

For example, when the voice query "How far is the distance from the TV to the sofa" is input, the voice processing unit 500 analyzes the content of the voice query and the position of "TV", "sofa" and / or the distance thereof. It is determined that the information is necessary, and requests the necessary information to the space processing unit 300.

The space processor 300 first searches whether the "TV" "sofa" exists in the target space, and if present, calculates the distance between two objects from the positions of "TV" and "sofa". The spatial processor 300 calculates a distance between two objects of interest based on location information included in the object node. The calculation result is returned to the voice processing unit 500, and the voice processing unit 500 provides a voice answer to the user using the returned information. For example, the answer may be, “The distance between the TV and the sofa is about 1 meter.”

In another embodiment, when the content of the voice query is associated with only one object of interest, the return information is calculated as information related to a relative positional relationship with any one other than the object of interest to be queried.

For example, when a voice query “Where did I put the remote control?” Is input, the voice processing unit 500 analyzes the contents of the voice query and analyzes that only one “remote” is associated with the query. The voice processing unit 500 determines relative positional relationship information to express the location of the "remote control" as necessary information for the answer, and requests the spatial processing unit 300 to do so.

The space processor 300 first searches whether the "remote control" exists in the target space. Also, since only one object of interest is associated with the voice query, it determines the other object of interest needed to generate the answer. Here, the other object of interest refers to a node that is a reference of a relative positional relationship, excluding a query object of interest. In this case, the other node that is a reference of the relative position may be a node associated with at least one of the user, the nearest object, and the largest object.

If the object closest to the remote controller is determined as the dining table, the position of the remote controller is represented by a relative positional relationship with the dining table, and the expression is returned to the voice processing unit 500. As a result, the voice processing unit 500 may generate and provide an answer, "The remote control is placed on the table."

In another embodiment, when the subspace of the target space is associated with the content of the voice query, an answer is generated using the location of the subspace. Here, the location of the subspace may be searched for an object of interest associated with the subspace and determined based on the found object of interest. Here, the sub-space is a sub-space corresponding to the sub-concept of the rough space in terms of space. For example, when the target space is the entire living space, the sub-space includes a toilet, a kitchen, a living room, a private room, and the like.

For example, when the voice query “Guide to the toilet” is input, the voice processing unit 500 analyzes the contents of the voice query (S510) and determines that the location of the “toilet” and the user's location are necessary, and the space processing unit Request this to the 300 (S530).

The space processor 300 searches for attribute information of each node included in the spatial context and extracts all nodes associated with the “toilet”. Then, the approximate location of the "toilet" is determined based on the extracted location information of the nodes. In the above example, when "wash basin", "toilet" and "bath" are searched, the position of "toilet" is determined based on the positions of "sink", "toilet" and "bath". In one embodiment, the location of the "toilet" is determined by the coordinates centered on the "sink", "toilet" and "bathtub."

The space processing unit 300 calculates the distance between the “toilet” and the user and returns it to the voice processing unit 500 (S550). The voice processing unit 500 turns to the right using the returned information, and walks about 3 meters. Please ”you can create an answer (S570).

As such, the intelligent voice information providing system 1000 and the method abstract the target space where the user is actually located in a spatial context (that is, a graph) including nodes and connection lines to perform faster analysis and information processing in terms of space. can do. Furthermore, by providing a response provided to the user to be expressed in a relative positional relationship, the user may provide a more intuitive and familiar voice answer.

The steps shown in FIGS. 2, 3, and 5 are merely exemplary, and the order of the steps may be different, or may further include, or may be omitted.

Operations by the intelligent voice information providing system and method according to the embodiments described above may be implemented at least in part by a computer program.

The computer may be a computing device, such as a desktop computer, laptop computer, notebook, smart phone, or the like, or any device that may be integrated. A computer is a device having one or more alternative, special purpose processors, memory, storage, and networking components (either wireless or wired). The computer can run, for example, an operating system that is compatible with Microsoft's Windows, Apple OS X or iOS, a Linux distribution, or an operating system such as Google's Android OS.

In addition, the computer program for implementing the present embodiment is configured in the form of functional programs, instructions, and codes that can be executed in the computer described above. The operations described above and their associated units, modules, and the like may be implemented within software, firmware, hardware, or any combination thereof. For example, a software module is implemented with a computer program product, which is comprised of a computer-readable medium containing computer program code, which is executed by a computer processor to perform any or all of the steps, operations, or processes described. Can be executed. Functional programs, instructions, and codes for implementing the present embodiment will be readily understood by those skilled in the art.

In addition, operations by the intelligent voice information providing system and method may be recorded on a computer-readable recording medium. A computer-readable recording medium having recorded thereon a program for implementing an operation by an intelligent voice information providing system and method according to the embodiments includes all kinds of recording devices storing data that can be read by a computer. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage, and the like. The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion.

Although the present invention described above has been described with reference to the embodiments shown in the drawings, this is merely exemplary, and it will be understood by those skilled in the art that various modifications and variations may be made therefrom. However, such modifications should be considered to be within the technical protection scope of the present invention. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

Claims (20)

A spatial context converter configured to generate a first spatial context in which objects of interest included in the target space are converted into nodes;
A spatial processor configured to calculate a location of a user located in the target space and to generate a second spatial context by combining the location of the user and the first spatial context; And
If a voice query associated with the target space is received, a voice processor for generating an answer corresponding to the voice query based on the second spatial context,
The space processing unit,
And configure the user to be a user node based on the user location, and to include the user node in the first spatial context.
The method of claim 1,
The spatial context transform unit,
Acquiring the raw image information of the target space, analyzing the raw image information to shape an object of interest included in the target space into an object of interest, and a first line including a connection line connecting the object of interest and the object of interest Intelligent voice information providing system further configured to create a spatial context.
The method of claim 2,
The spatial context transform unit,
Wherein based on the raw image information, and reconstruct the target area in a three-dimensional basic space coming, and identify and index the object of interest located within the three-dimensional basic space, configured further to shape the said index object as objects of interest node Intelligent voice information providing system.
The method of claim 3,
The spatial context transform unit,
And dividing the three-dimensional basic space into segments including a single object of interest, and applying the spatial shape corresponding to the object of interest to each segment.
The method of claim 1,
The space processing unit,
And configured to create a second spatial context including a connection line that sets the user node as a starting point and connects each node so as not to pass through another node.
The method of claim 1,
The space processing unit,
And acquire a first-person image corresponding to the first-person view of the user, and calculate a location of the user based on a shape of the object of interest included in the first-person view image.
The method of claim 1,
The space processing unit,
And acquire a third person image corresponding to a third person view of the user, and calculate a location of the user based on a shape of the user included in the third person view image.
The method of claim 1,
And when only a first object of interest is included in the content of the voice query, the answer is expressed by a relative positional relationship with the second object of interest that is different from the first object of interest.
The method of claim 8,
The second object of interest is,
Intelligent voice information providing system is determined based on the distance to the first object of interest.
The method of claim 9,
The second object of interest is,
The intelligent voice information providing system is further determined based on the attributes of the object of interest located in the target space.
The method of claim 1,
When the subspace of the target space is related to the content of the voice query, the answer is generated using the location of the subspace,
The location of the subspace searches for the object of interest associated with the subspace, and is determined based on the found object of interest.
The method of claim 11,
And the location of the subspace is a center coordinate with respect to the found location of the object of interest.
An intelligent voice information providing method performed by an intelligent voice information providing system,
Obtaining a first spatial context in which the object of interest included in the target space is shaped as a node;
Calculating a location of a user located in the target space;
Combining a location of the user with the first spatial context to create a second spatial context; And
When receiving a voice query associated with the target space, generating an answer corresponding to the voice query based on the second spatial context,
Creating the second spatial context,
Shaping the user into a user node based on the user location; And
And including the user node in the first spatial context.
The method of claim 13,
Acquiring the first spatial context,
Obtaining raw image information of the target space;
Analyzing the raw image information and shaping an object of interest located in the target space into an object of interest node; And
And generating a first spatial context including a connection line connecting the object of interest node and the object of interest node.
The method of claim 14,
Analyzing the raw image information and shaping the object of interest located in the target space into an object of interest node,
Reconstructing a three-dimensional space, the basic target area on the basis of the source image information;
Recognizing an object of interest included in the three-dimensional basic space;
Indexing the recognized object of interest; And
Shaping the indexed object into an object of interest node.
The method of claim 15,
Analyzing the raw image information and shaping the object of interest located in the target space into an object of interest node,
Dividing the three-dimensional basic space into segments containing a single object of interest; And
And applying a spatial shape corresponding to the single object of interest to the segment.
delete The method of claim 13,
Creating the second spatial context,
Setting the user node as a starting point; And
And generating a connection line connecting each node so as not to pass through another node.
The method of claim 13,
Generating an answer corresponding to the voice query using the second spatial context,
Analyzing the contents of the voice query and requesting necessary information;
Searching for a node included in the second spatial context in response to the request;
Generating and returning necessary information based on the search result; And
Intelligent voice information providing method comprising the step of generating an answer using the returned necessary information.
A computer readable recording medium storing instructions, wherein when the instructions are executed, the processor performs the intelligent voice information providing method according to any one of claims 13 to 16, 18 and 19. Computer-readable recording medium.

Images