KR20160031366A - Apparatus and method of controlling an electric instrument and the electric instrument system - Google Patents

Abstract

An embodiment of the present invention includes a method of acquiring a fluid, the method comprising: acquiring a signal related to the receiving portion including an image of a receiving portion that receives fluid; Acquiring positional information of a predetermined object in contact with the fluid from the signal, and generating a negative parameter corresponding to the positional information; And outputting an audio signal based on the parameter.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an electronic musical instrument control method and apparatus,

Embodiments of the present invention relate to an electronic musical instrument control method and apparatus, and an electronic musical instrument system.

An electronic musical instrument represented by a synthesizer is divided into a musical instrument in which a sound is generated through a physical medium and a musical instrument in which a sound is generated in an electronic manner. In particular, the electronic musical instrument standard signal standard, which is represented by MIDI (Musical Instrument Digital Interface), transforms the parameters of the sound control to a code having 7 bits in most cases, thereby technically separating the sound source unit and the control unit of the musical instrument Contributed. Instruments such as electronic wind instruments and electronic drums have been created through the technology of implementing the control part of the musical instrument using various sensor hardware technologies and transmitting the information of the control part to the sound source part through the electronic musical instrument standard signal standard, have.

Embodiments of the present invention provide an electronic musical instrument control method and apparatus capable of imparting auditory, tactile, and visual effects, a computer program for executing the same on a computer, and an electronic musical instrument system.

An embodiment of the present invention includes a method of acquiring a fluid, the method comprising: acquiring a signal related to the receiving portion including an image of a receiving portion that receives fluid; Acquiring positional information of a predetermined object in contact with the fluid from the signal, and generating a negative parameter corresponding to the positional information; And outputting an audio signal based on the parameter.

Another embodiment of the present invention is directed to an image processing apparatus comprising: a signal acquiring unit acquiring a signal related to the accommodating unit including an image of an accommodating portion accommodating a fluid; A parameter generation unit for acquiring positional information on a predetermined object to be contacted with the fluid from a signal related to the accommodation unit and generating a negative parameter corresponding to the positional information; And an audio signal output unit for outputting an audio signal based on the parameter.

Another embodiment of the present invention is a fluidized-bed forming apparatus comprising: a receiving portion capable of receiving a fluid; A sensing unit including a camera for photographing at least one end face of the accommodating portion, the sensing unit acquiring a signal related to the accommodating portion including the image photographed by the camera; Acquiring position information of a predetermined object in contact with the fluid based on the signal obtained by the sensing unit, generating a negative parameter corresponding to the position information, and generating and outputting an audio signal based on the parameter ; And an output device for receiving the audio signal and outputting sound based on the audio signal.

Other aspects, features, and advantages will become apparent from the following drawings, claims, and detailed description of the invention.

An electronic musical instrument control method and apparatus and an electronic musical instrument system according to embodiments of the present invention provide a variety of sensory satisfaction to a user by expressing a tactile effect felt by a user with an auditory effect.

The electronic musical instrument control method and apparatus and electronic musical instrument system according to the embodiments of the present invention meet the user's needs for an electronic musical instrument that is easier and more intuitive.

The electronic musical instrument system according to the embodiments of the present invention provides a degree of freedom of performance in which a user can simultaneously control a plurality of sound heights and sizes simultaneously in real time by generating a plurality of sounds using a continuous physical medium .

1 is a schematic diagram showing an electronic musical instrument system 1 according to an embodiment of the present invention.
FIG. 2 shows an example of the receiving unit 10 and the sensing unit 20 in detail according to an embodiment of the present invention.
FIG. 3 shows a modification of the sensing unit 20 shown in FIG.
4 is a block diagram showing the configuration of the control unit 30 according to an embodiment of the present invention.
FIG. 5 shows an example of an image photographed by the sensing unit 20. As shown in FIG.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. The effects and features of the present invention and methods of achieving them will be apparent with reference to the embodiments described in detail below with reference to the drawings. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings, wherein like reference numerals refer to like or corresponding components throughout the drawings, and a duplicate description thereof will be omitted .

In the following embodiments, the terms first, second, and the like are used for the purpose of distinguishing one element from another element, not the limitative meaning. In the following examples, the singular forms "a", "an" and "the" include plural referents unless the context clearly dictates otherwise. In the following embodiments, terms such as inclusive or possessive are intended to mean that a feature, or element, described in the specification is present, and does not preclude the possibility that one or more other features or elements may be added.

1 is a schematic diagram showing an electronic musical instrument system 1 according to an embodiment of the present invention.

The electronic musical instrument system 1 according to an embodiment generates and outputs sound based on the position of the object in contact with the fluid. According to the electronic musical instrument system 1 according to the embodiment, the user can intuitively generate and output sound by contacting a part of the body or a predetermined object with the fluid.

An electronic musical instrument system 1 provided according to an embodiment includes a receiving portion 10 for receiving a fluid, a sensing portion 20 for sensing various signals including a camera for photographing the receiving portion, A control unit 30 for generating an output sound of the electronic musical instrument system by generating a signal, and an output device 40 for outputting sound based on the audio signal.

The receptacle 10 according to one embodiment receives fluid. The user can bring a predetermined object into contact with the fluid accommodated in the accommodating portion 10 to generate an audio signal. The object may be a part of the user's body, for example, a finger, but is not limited thereto, and may be an object such as a stick.

The sensing unit 20 according to an exemplary embodiment may sense various signals from the accommodation unit 10. [ The sensing unit 20 according to an exemplary embodiment may include a camera for photographing the accommodation portion 10. [ The sensing unit 20 according to an exemplary embodiment may include a plurality of cameras so as to capture the accommodation unit 10 at various angles. The sensing unit 20 according to an embodiment may include a vibration sensor that senses vibration of a fluid surface accommodated in the accommodating portion 10. The sensing unit 20 may include one or more ultrasonic sensors provided in the fluid to measure the distance to the object, or an infrared sensor. The sensing unit 20 may include a pressure sensor installed at an appropriate position to recognize that the object is in contact with the fluid. The sensing portion 20 may include a current field sensor that recognizes the current field in the fluid. The sensing unit 20 outputs various sensed signals to the control unit 30. [

On the other hand, the receiving portion 10 may be formed of a material having transparency of a predetermined threshold value or more so that the object can be identified when the receiving portion 10 is photographed with the camera. Also, the fluid received in the accommodating portion 10 may be formed of a material having transparency of a predetermined threshold value or more. For example, the fluid may be water.

The control unit 30 according to an embodiment acquires a signal recognized by the sensing unit 20, analyzes the signal to acquire positional information of the object, and generates an audio signal based on the positional information. For example, the control unit 30 analyzes the signal to acquire the position information, sets a parameter for the sound based on the position information, and generates an audio signal based on the parameter.

The location information may include plane coordinate information and depth information relative to the fluid surface. The position information may be obtained by processing the image photographed by the camera, obtained from the sensing value of the ultrasonic sensor or the infrared sensor, or may be obtained from the sensing value of the pressure sensor. Or when a plurality of pressure sensors are provided, the positional information of the object can be obtained from the position of the pressure sensor at which the water pressure above the threshold value is sensed. Alternatively, if distortion of the current field is detected from the current field sensor, the positional information of the object can be obtained from the current long-distortion position or the distortion pattern.

The output device 40 according to one embodiment obtains an audio signal and outputs a sound. As the output device 40, a general output device can be used.

According to an embodiment, the control unit 30 may be designed to output an audio signal and the output device 40 that has obtained the audio signal output audio, but may be modified. For example, the control unit 30 may output only the parameter set, and the output device 40 that has acquired the parameter set may be designed to output sound based on the parameter set. When the sound source is provided in the control unit 30, the control unit 30 can generate the audio signal using the direct sound source. However, when the sound source is provided in the output unit 40, In the case of a synthesizer, the control unit 30 may provide only the parameter set for the sound source, and the output device may output the sound corresponding to the parameter set. Such a level of design change is applicable to the embodiments of the present invention.

Although not shown in FIG. 1, a separate guide member may be provided in the accommodating portion 10. The guide member guides the user to the position reference on the receptacle 10. For example, the guide members may be arranged along one axis of the receiving portion 10, and the guide members may be installed at every point of reference of the parameter to provide parameter predictability. For example, when the parameters are scales, it is possible for the user to predict which scales will be produced by touching the object at any position.

Although not shown in Fig. 1, the electronic musical instrument system 1 may further include a user operation portion capable of directly operating the parameters of the sound as in the prior art. The user operation unit may be provided independently of the accommodating portion 10. [

FIG. 2 shows an example of the receiving unit 10 and the sensing unit 20 in detail according to an embodiment of the present invention.

Referring to FIG. 2, fluid is contained in the accommodation portion 10, and the sensing portion (20 in FIG. 1) senses various signals from the accommodation portion 10. 2, it can be seen that the sensing unit 20 includes the camera 21 and the vibration sensor 22, and the sensing unit 20 may include more sensors.

First, the camera 21 takes a photograph of the accommodation portion 10 containing the fluid. Referring to FIG. 2, the camera 21 can photograph the accommodation portion 10 from the front. By analyzing the image photographed from the front as described above, the depth information of the object (the finger in Fig. 2) and the one-dimensional position information of the object can be obtained on the basis of the fluid surface.

Although only one camera 21 is shown in FIG. 2, the electronic musical instrument system according to an embodiment may further include a camera for photographing the receptacle 10 from below and / or a camera for photographing from the side. By photographing the accommodating portion 10 in various directions as described above, the positional information of the object can be obtained on various criteria.

The positional information may include at least one coordinate information among depth information (z) based on the surface of the fluid, and two-dimensional coordinates (x, y) with the surface of the fluid as a plane. For example, from the image photographed using the camera 21 shown in Fig. 2, it is possible to obtain coordinate information (x) of one of two-dimensional coordinates in which the depth information z and the surface of the fluid are plane.

In order to obtain effective position information, the type of the camera 21 can be variously set. For example, a general camera can be used as well as an infrared camera can be used. When the camera is equipped with a depth camera using infrared sensing, the depth direction (z) of the captured image and the horizontal direction (x) of the image taken by the camera 21 among the two- Coordinates can be obtained, and y-coordinate information corresponding to the depth axis can be obtained from the depth information obtained from the infrared sensing based on the camera 21. [

The vibration sensor 22 senses the vibration of the fluid surface. The vibration sensor 22 may be replaced with a gyro sensor, an acceleration sensor, an angular velocity sensor, or the like. The vibration sensor 22 may be waterproofed or hermetically sealed to prevent the fluid from getting wet, and may have a separate structure for floating the vibration sensor 22, for example, an air bag, styrofoam, or the like.

Although not shown in FIG. 2, the sensing unit 20 may further include at least one ultrasonic sensor or an infrared sensor, which is provided in the fluid and measures the distance to the object. The sensing unit 20 may further include a pressure sensor installed at an appropriate position to recognize that the object is in contact with the fluid. For example, the pressure sensors may be installed at regular intervals on the bottom of the receiving portion 10. [ The positional information of the object can be obtained from the position of the pressure sensor at which the water pressure higher than the threshold value is sensed.

Although not shown in FIG. 2, an illuminating panel for irradiating light toward the fluid may be provided on a wall surface of the accommodating portion 10. When the fluid is irradiated with light as described above, there is an effect that it becomes easier to identify the object from the image acquired through the camera 21. [ For example, the area where the object exists is darker and the rest area becomes brighter, which improves the accuracy of edge recognition.

An image obtained by the camera 21 and a method of analyzing the image will be described later with reference to FIG.

FIG. 3 shows a modification of the sensing unit 20 shown in FIG. 2, and a description overlapping with the description of FIG. 2 will be omitted. Thus, although not described below with respect to FIG. 3, the description described above with respect to FIG. 2 may also be applied to FIG.

Referring to FIG. 3, a fluid is contained in the accommodation portion 10, and a sensing portion (20 in FIG. 1) senses various signals from the accommodation portion 10. Referring to FIG. 3, it can be seen that the camera 21 and the vibration sensor 22 are provided as the sensing unit 20, and the sensing unit 20 may include more sensors.

3, in order to change the position of the camera 21 for photographing the front face of the accommodation portion 10, there is further provided a reflection member 50 for reflecting the front face of the accommodation portion 10 downward . The front surface of the accommodating portion 10 can be photographed even from below because of the reflective member 50, thereby making it possible to constitute a more effective electronic musical instrument system.

For example, if the camera 21 is placed below the receiving portion 10 and a frame for fixing the receiving portion 10 and the camera 21 is provided, the height of the electronic musical instrument is increased, Is greatly reduced. In addition, it is easier to fix the position of the camera 21, and the possibility that the position is changed by the user is further reduced.

A camera 21 for photographing a front view of the receiving portion 10 reflected through the reflecting member 50 at a position lower than the receiving portion 10; A separate camera (not shown) for photographing may be provided side by side. According to this, space occupied by the electronic musical instrument system of the present invention can be reduced, and various spaces can be utilized.

4 is a block diagram showing the configuration of the control unit 30 according to an embodiment of the present invention.

4, the control unit 30 includes a signal acquisition unit 310 that acquires a signal from the sensing unit 20, a parameter generation unit 320 that generates a sound parameter based on the acquired signal, And an audio signal output unit 330 for generating and outputting a final audio signal based on the parameters.

The signal acquisition unit 310 may acquire an image from the sensing unit 20 according to an exemplary embodiment. The signal acquisition unit 310 may acquire a vibration sensing value of the fluid surface from the sensing unit 20. [

The parameter generating unit 320 analyzes the image obtained by the signal obtaining unit 310, generates position information of the target object, and generates a sound parameter based on the position information.

The parameter generating unit 320 can recognize a target object separated from the fluid surface as a separate target. For example, the parameter generation unit 320 can recognize each finger as a separate object when each finger is immersed in fluid, when a person immerses several fingers in the fluid, respectively. Thus, the position information of each object can be generated.

The parameter generator 320 matches the sound for each recognized object. For example, when three objects are recognized, a total of three notes are defined by matching the notes to each object, and parameters for three notes are generated, respectively. The parameter generation unit 320 generates negative parameters matched to each object based on the positional information of each object. For example, the first tone parameter set may be generated based on the position information of the first object, and the second tone parameter set may be generated based on the position information of the second object. Each parameter set may include at least one of parameters such as, for example, a pitch, a size, a vibration, a tone (an effect such as distortion), and the like.

Various algorithms such as an edge recognition algorithm in an image, a pattern recognition algorithm of a target object, and a color recognition algorithm of a target object can be used as a method for the parameter generator 320 to recognize an object in an image.

The parameter generating unit 320 may further generate the parameter using the vibration sensing value of the fluid surface. For example, parameters such as sustain, vibration, and tone of each sound can be further generated using the vibration sensing value of the fluid surface.

The audio signal output unit 330 outputs an audio signal of a format that can be output from the output device 40 of Fig. 1, based on the parameter set for one or more notes generated by the parameter generation unit 320. [ The audio signal output by the audio signal output unit 330 is transmitted to the output apparatus 40, and the output apparatus 40 outputs sound from the audio signal.

FIG. 5 shows an example of generating a parameter using the image photographed by the sensing unit 20. As shown in FIG.

The image 40 shown in FIG. 5 may be captured by the sensing unit 20 and transmitted to the control unit 30. Referring to the image 40, it can be seen that three of the fingers of the user are in contact with the fluid. Hereinafter, each of the three fingers will be referred to as a first object 421, a second object 422, and a third object 423.

Referring to FIG. 5, the parameter generation unit 320 of FIG. 4 analyzes the image 40 to acquire position information of the objects 421, 422, and 423. The depth information z1 and the coordinate information x1 are used as the position information of the first object 421 and the depth information z2 and the coordinate information x2 are used as the position information of the second object 422, 423, the depth information z3 and the coordinate information x3 are obtained. The point at which the depth information z1, z2, z3 is referred to may be set by directly recognizing the surface 41 of the fluid, but it goes without saying that it may be set to a fixed position from the beginning for convenience.

According to the example illustrated in FIG. 5, the parameter generator 320 will generate a set of parameters for three notes. For example, for the first object 421, a parameter set for a first sound having a sound height corresponding to x1 and a sound intensity corresponding to z1 is set to x2 for the second object 422 For the third target 423, a parameter set for the third note having the pitch intensity corresponding to z3 and a parameter set for the second note having the pitch intensity corresponding to z2 for the third target object 423, Parameter set. For example, the first note may correspond to a figure, the second note may correspond to a lie, and the third note may correspond to a bee.

The relationship between the x-axis coordinate value and the sound pitch, and the relationship between the z-axis coordinate value and the sound intensity can be defined according to pre-stored calculation methods or can be defined by a lookup table (LUT). For example, when the calculation method is stored, the tone height is obtained by assigning the x-axis coordinate value to the variable, and the z-axis coordinate value is assigned to the variable to obtain the tone intensity. When the axis-up table is predefined, the tone height matched to the range corresponding to the x-axis coordinate value is obtained and the tone intensity matched to the range corresponding to the z-axis coordinate value is obtained.

Different calculation methods can be used for each parameter. For example, a calculation method using a lookup table is used for the tone height, and a calculation method using the previously stored mathematical expression can be used for the tone strength.

If there is a vibration sensing value obtained from the vibration sensor, the parameter generator 320 may generate a new parameter for all notes (first note, second note, third note). For example, it is possible to generate a vibration parameter corresponding to the vibration sensing value for all notes.

Although not shown in FIG. 5, a y-axis coordinate value orthogonal to the x-axis and parallel to the surface of the fluid can be obtained from an image captured by a separate camera, and the y-axis coordinate value Can be generated. For example, the y-axis coordinate value may correspond to a negative pitch adjustment degree. The user can adjust the pitch of the sound by moving the position of the object along the y-axis. The parameter matched to the y value is only an example, and various parameters may be matched.

According to the above-described embodiments of the present invention, the user can intuitively play the electronic musical instrument in such a manner that the user freely changes the position of the finger while immersing it in the fluid. According to the embodiments of the present invention described above, the user can perform playing in the glyphometric method by continuously moving the fingers in the fluid from one end of the receptacle 10 to the other end. According to the embodiments of the present invention described above, a user can play a chord by immersing several fingers in fluid. According to the embodiments of the present invention described above, the user can arbitrarily adjust the parameters of each sound while simultaneously generating a plurality of sounds. According to the embodiments of the present invention described above, the user can experience the feedback of the tactile sensation touching the fluid to the auditory sensation, which can stimulate the user's desire, It matches.

10:
20: sensing unit
21: Camera
22: Vibration sensor
30:
40: Output device

Claims (7)

Acquiring a signal related to the receiving portion including an image of a receiving portion for receiving the fluid;
Acquiring positional information of a predetermined object in contact with the fluid from the signal, and generating a negative parameter corresponding to the positional information; And
And outputting an audio signal based on the parameter.
The method according to claim 1,
The location information
Wherein the information includes at least one of depth information based on a height of the surface of the fluid or a preset height and two-dimensional coordinates in which the surface of the fluid is a plane,
Wherein the generating step generates a parameter set including a first parameter corresponding to the depth information and a second parameter corresponding to the coordinate information.
The method according to claim 1,
Wherein the acquiring step further acquires the distance from the at least one ultrasonic sensor provided in the fluid to the object or the distance from the camera to the object obtained from the infrared sensor provided in the camera for photographing the image,
Wherein the generating step acquires the position information by further referring to the distance information.
The method according to claim 1,
Wherein the generating step acquires positional information on a plurality of objects from the image, respectively, generates negative parameters corresponding to the plurality of positional information,
Wherein the outputting step outputs an audio signal for simultaneously outputting the plurality of sounds based on the parameter.
A signal acquiring unit acquiring a signal related to the accommodating part including an image of a housing part accommodating the fluid;
A parameter generation unit for acquiring positional information on a predetermined object to be contacted with the fluid from a signal related to the accommodation unit and generating a negative parameter corresponding to the positional information; And
And an audio signal output unit for outputting an audio signal based on the parameter.
A receiving portion capable of receiving fluid;
A sensing unit including a camera for photographing at least one end face of the accommodating portion, the sensing unit acquiring a signal related to the accommodating portion including the image photographed by the camera;
Acquiring position information of a predetermined object in contact with the fluid based on the signal obtained by the sensing unit, generating a negative parameter corresponding to the position information, and generating and outputting an audio signal based on the parameter ; And
And an output device for receiving the audio signal and outputting sound based on the audio signal.
The electronic musical instrument system according to claim 6, wherein the electronic musical instrument system
A reflecting member which reflects a state of one surface of the receiving portion and makes the incident light incident on the camera,
An illumination panel for irradiating light toward the fluid at a wall surface of the accommodation portion,
And a guide member provided to guide a position reference on the receiving portion to a user,
Wherein the camera captures an image of one surface of the accommodating portion by photographing the reflective member.

Images