JP2009500730A - Computer graphic shader system and method - Google Patents

Abstract

A method and system for integrating various shading applications under a single language is described, which allows for easy reuse and re-use of shaders, and shader design without the need for computer programming And building is facilitated, allowing shader graphical debugging.
[Selection] Figure 16

Description

  The present invention relates generally to the fields of computer graphics, computer-aided design, and more particularly to generating a shader system and using the generated shader system for rendering an image of a scene. The system and method. The present invention provides a new type of component that is particularly useful in computer graphics systems (identified here as “phenomenon”) and is packaged and encapsulated shader DAG (“Directed Acyclic” A system that includes a set of "directed acyclic graphs") or cooperating shader DAGs, each of which can include one or more shaders, where the shaders collaborate accurately during rendering Generated and encapsulated to help define at least a portion of the scene in a manner that ensures that it can.

  In computer graphics, computer-aided geometric design, etc., an artist, drafter, or other user (commonly referred to herein as an “operator”) is an object in a scene that is maintained by the computer. And then each 2D image of the object in the scene is to be rendered from one or more directions. In the first representation generation phase, conventionally, a computer graphics system generates a three-dimensional representation from various two-dimensional line drawings including, for example, the contours and / or cross-sections of objects in the scene, and for such line drawings. Numerous operations are performed to generate a two-dimensional surface in three-dimensional space, and then correct or correct the shape of the representation of the object by modifying the parameters and control points of such surface.

  During this process, the operator also defines various characteristics of the object surface, the structure and characteristics of the light source that illuminates the scene, and the structure and characteristics of one or more simulated cameras that generate the image. After the structure and characteristics of the scene, light source, and camera are defined, in a second phase, the operator allows the computer to render an image of the scene from a special viewpoint direction.

  Objects, light sources, and cameras in the scene are defined in the first scene definition phase by individual multidimensional mathematical representations, including at least three spatial dimensions and possibly one time dimension. The mathematical representation is typically stored in a tree structured data structure. The surface qualities of the object are consequently defined by “shade trees”, each of which includes one or more shaders, which during the second scene rendering phase Enables the computer to render individual surfaces and in practice provides color values that represent the color of the individual surfaces. Shade tree shaders can be generated by an operator, or deduced by a computer graphics system, both in a high level language such as C or C ++, that allows the computer to render images of individual surfaces in the second scene rendering phase. Provided to.

  Numerous problems arise from the generation and use of shaders and shade trees, as typically provided in computer graphics mechanisms. First, shaders generally cannot cooperate with each other unless programmed to do so. Typically, the input value provided to the shader is a constant value, thereby limiting the shader's flexibility and ability to render features in an interesting live-like manner. Furthermore, it is generally difficult to set up a collaborative shader system that can obtain its input values from a common source.

  In order to provide a solution to such a problem, the above-cited US Pat. No. 6,496,190 can create and instantiate a new type of entity called “phenomenon” A computer graphics system is described that can be used in the rendering of an image. Phenomenon is an encapsulated shader DAG (“Directed Acyclic Graph”) with one or more nodes, where each node is shader or interconnected and instantiated to work together, With an encapsulated set of such DAGs attached to entities in the scene. Here, entities are the various types of scene characteristics, including the surface color and texture characteristics of objects in the scene, the volume and geometric characteristics of the scene, the characteristics of the light sources that illuminate the scene, and the simulation during rendering. Created during the scene definition process to define the characteristics of the simulated camera being simulated and many other features useful in rendering.

  The phenomenon selected by the operator for use in the context of the scene can be pre-defined, or the operator can be constructed from the basic shader node using the Phenomenon creator. The Phenomenon Creator ensures that the Phenomena is constructed so that shaders in the DAG or collaborative DAG can work correctly during the rendering of the scene image.

Prior to being added to the scene, the Phenomenon is instantiated by providing a value, or function, that is used to define a value for each of the Phenomenon parameters using the Phenomenon Editor.
After the scene representation is defined and the phenomina is added, the scene image generator can generate an image of the scene. In operation, the scene image generator operates in a series of phases, including a preprocessing phase, a rendering phase, and a postprocessing phase. During the preprocessing phase, the scene image generator can perform preprocessing such as shadow and photon mapping, multiple inheritance resolution. The scene image generator can perform pre-processing operations, for example, if the phenomenon added to the scene includes a geometric shader that generates a geometric shape defined for the scene. During the rendering phase, the scene image generator renders the image. During the post-processing phase, the scene image generator relies on the velocity and depth information that the Phenomenon added to the scene is stored in relation to, for example, depth of field or each pixel value of the rendered image. If it includes a shader that defines a post-processing operation such as motion blur calculation, the post-processing operation can be performed.

  The Phenomina system described in US Pat. No. 6,496,190 is very effective. However, in recent years, many shading platforms and languages have been developed, so existing shader languages, whether hardware for video game shading or software for shading video visual effects, A narrower focus on special platforms and applications. This platform dependency typical of traditional shader systems and languages can be a major limitation.

It would be desirable to provide a shader method and system that is platform independent and that allows various shading tools and applications to be integrated under a single language or system construct.
It also enables efficient and simple shader reuse and re-purpose, and is effective in integrating video games and feature films that occur frequently and frequently (eg, Lara Croft-Tomb Raider) It is also desirable to provide a system.

It would also be desirable to provide a method and system that facilitates the design and construction of shaders without the need for computer programming that is useful to artists.
In addition, it would be desirable to provide a method and system that enables graphical debugging of a shader and enables a shader writer to find and resolve defects in the shader.

  The present invention, whose various forms are collectively referred to herein as "Mental Mill", addresses the above limitations of the prior art and allows various shading applications to be described in a single language (here. Can be integrated under the “MetaSL shading language”, allowing easy re-use and re-purpose of shading to design and build shaders without the need for computer programming Provide a platform-independent method and system that facilitates, enables graphical debugging of shaders, and achieves many other functions.

One form of the invention is the creation of a simple and compact componentized shader (referred to herein as a Metanode) that can be combined to build more complex and visually interesting shaders in a shader network. Related to methods and systems that facilitate.
A further form of the invention is a mental mill shading language, referred to herein as MetaSL (Meta Shading Language). MetaSL is a simple but particularly expressive language designed to implement shaders. MetaSL also has existing shading applications that previously focused on special platforms and situations (eg, hardware shading for games, software for visual effects in feature films). It is also designed to integrate under a single language and management structure.

Mental mills thus allow the creation of metanodes (ie shader blocks) written in MetaSL, which can be added and combined to form sophisticated shader graphs and phenomina.
The shader graph provides an intuitive graphical user interface for creating shaders that is accessible to users who do not have technical expertise to write shader software code.

Another embodiment of the present invention relates to an API library for managing shader creation.
In a further form of the invention, the mental mill GUI library provides a complete GUI for building shader graphs and phenomina using the shader graph paradigm.
In another aspect of the present invention, the MetaSL shading language can be effectively constructed as a superset of all existing and future shading languages for a special hardware platform, and thus special purpose graphics hardware. Since it does not depend on such instantiations, a single, reusable Phenomenon MetaSL description (consisting of resulting Metanodes in MetaSL), and special in special purpose shader languages (Cg, HLSL, etc.) It is possible to use a dedicated compiler in a mental mill system that generates optimized software code for a specific target platform.

  For special purpose platforms / languages, platform / language optimized code can be translated into machine language for special hardware (integrated circuit chip) instantiation by a compiler specific to the shaded language in question. . (The native compiler does not have to be part of the mental mill, and in general it is not). This is particularly useful when, for example, a special chip is available at a given time but is quickly replaced by the next generation version of the chip.

  In this regard, note that previous attempts to represent complex visual effects, or phenomina, in shaded languages such as Cg resulted in code that was not optimal for compiling into machine language. . And in such a case, the compilation process is extremely slow, thereby ruining the desired real-time performance, or the hardware-level shader code is not as efficient as it can be, so the code is very slow Was executed. The foregoing aspects of the invention address and solve this problem with such prior attempts.

  A further aspect of the invention relates to a new interactive, visual and real-time debugger for shader programmers / writers (ie programmers in the MetaSL shading language) in the Phenomenon creation environment. The debugger is described in detail below, but the "viewport" where the test scene with the shader, metanode, or Phenomenon in development is always rendered, even with the effect of changes in just one line of code. , Immediately clarified by visual feedback.

Additional aspects, examples, details, embodiments, and practices of the invention are described below in the Detailed Description of the Invention.
The invention is specified by the particularity of the appended claims. The above further advantages of the present invention will be further understood by reference to the following description in conjunction with the accompanying drawings.

Cross-reference to related applications This patent application is commonly owned and issued on December 17, 2002, US Pat. No. 6,496,190 (“Used in Computer Graphic Systems”). Systems and methods for the generation and use of collaborative and encapsulated shader and shader DAG systems "), which is incorporated herein by reference. In addition, the US Provisional Application No. 60 / 696,120, filed July 1, 2005 (“Computer Graphic Shader System and Method” (patent attorney trial schedule number MNTL-103- PR)) and US Provisional Application No. 60 / 707,424 ("Improved Computer Graphic Shader System and Method") filed on August 11, 2005 (patent attorney schedule number MNTL- 105-PR)), which is incorporated herein by reference.

  The present invention provides an improvement to the computer graphic entity called “phenomenon” described in commonly owned US Pat. No. 6,496,190, incorporated herein by reference. Thus, in Section I below, various forms of the computer graphic “phenomenon” described in US Pat. No. 6,496,190 are discussed, and then divided into four sections. In section we consider this improvement to the Phenomenon entity.

Section I Computer Graphic “Phenomena”
U.S. Pat. No. 6,496,190 includes one or more shaders each generated in such a way as to ensure that the shaders in the shader DAG can work together correctly during rendering. A new computer graphics system and method is described that provides enhanced collaboration between shaders by facilitating the generation of packaged and encapsulated shader DAGs.

  In summary, a new type of entity called “phenomenon” can be created, instantiated, and used to render an image of a scene. Phenomenon can be an encapsulated shader DAG with one or more nodes, each with a shader, or interconnected and instantiated to cooperate during the scene definition process Created, surface color and texture features of objects in the scene, characteristics of volume and geometric shapes in the scene, features of light sources that illuminate the scene, simulated camera features that are simulated during rendering, And an encapsulated set of such DAGs attached to entities that define various types of features of the scene, including numerous other features useful for rendering.

  The phenomenon associated with the scene selected for use by the operator may be defined in advance, or the operator may be constructed from the basic shader node using the Phenomenon creator. The Phenomenon Creator ensures that the Phenomena is constructed so that shaders in the DAG or cooperating DAGs can collaborate accurately during the rendering of the scene image.

Prior to being added to the scene, the Phenomenon is instantiated using the Phenomenon Editor by providing a value, or function, used to define a value for each of the Phenomenon parameters.
After the scene representation is defined and the phenomina is added, the scene image generator can generate an image of the scene. In that operation, the scene image generator operates in a series of phases including a pre-processing phase, a rendering phase, and a post-processing phase. In the preprocessing phase, the scene image generator can perform preprocessing operations such as shadow and photon mapping, multiple inheritance resolution, and the like. The scene image generator can perform a pre-processing operation, for example, if the phenomenon added to the scene includes a geometric shader that generates a geometric shape defined thereby for the scene. During the rendering phase, the scene image generator renders the image. During the post-processing phase, the scene image generator, for example, has a depth of field or motion where the phenomenon added to the scene depends on the speed and depth information stored in association with each pixel value in the rendered image. A post-processing operation can be performed if it includes a shader that defines a post-processing operation, such as the blur calculation by.

  FIG. 1 illustrates components comprising a computer graphics system 10 constructed in accordance with the present invention. The computer graphics system 10 is used to define the features of the scene used in the rendering, creating a new computer graphics component, referred to herein as “Phenomenon” or “Phenomena” By providing enhanced collaboration between shaders. Phenomenon is a packaged and encapsulated system with one or more shaders, which are each DAG containing one or more shaders, and one or more directed Organized and interconnected in the form of an acyclic graph (“DAG”). Phenomina generated by computer graphics system 10 ensures that shaders (or multiple shaders) in each shader DAG can work together correctly during rendering to facilitate the rendering of realistic or complex visual effects. It is generated in such a way. Furthermore, for Phenomena with multiple cooperating shader DAGs, the computer graphics system 10 is rendering in order to facilitate the rendering of realistic or complex visual effects that the shaders in all shader DAGs will progress. The phenomina is generated so that it can cooperate correctly.

  Referring to FIG. 1, a computer graphics system 10 in one embodiment includes a processor module 11 and an operator input configuration such as a keyboard 12A and / or mouse 12B (generally identified as an operator input element 12). An operator interface element comprising elements and a computer including an operator output element such as a video display device 13. The illustrated computer system 10 is a type of conventional program storage computer architecture. The processor module 11 is, for example, a processor, memory, and disk and / or tape storage element (not shown separately) that performs processing and storage operations in conjunction with digital data provided thereto. Includes mass storage. The operator input element 12 is provided for an operator to input information for processing. The video display device 13 is provided for displaying output information generated by the processor module 11 on the screen 14 for the operator, and the operator inputs data for processing, and the operator controls the processing. The information to be entered is included along with the information generated during processing. The processor module 11 uses a so-called “graphical user interface (“ GUI ”)” to generate information for display by the video display device 13 and information for various application programs uses various “windows”. Is displayed. The computer system 10 is shown to include special components such as a keyboard 12A and mouse 12B for receiving input information from an operator, and a video display device 13 for displaying output information to the operator. The computer system 10 may include various components in addition to or instead of the components shown in FIG.

  In addition, the processor module 11 may include one or more network ports, identified generally by the reference numeral 14, and connected to a communication link that connects the computer system 10 within a computer network. The network port allows the computer system 10 to send information to and receive information from other computer systems and other devices in the network. For example, in a typical network configured with a client-server paradigm, one computer system in the network is designated as a server and data and programs (generally “information”) for processing by other client computer systems. So that client computer systems can conveniently share information. Client computer systems that need access to information maintained by a special server allow the server to download information there over the network. After processing the data, the client computer system also returns the processed data to the server for storage. In addition to computer systems (including the servers and clients described above), the network is also shared among various computer systems connected within the network, such as printers, facsimile machines, digital audio or video storage and distribution devices, for example. A possible device may be included. Communication links interconnecting computer systems within a network may comprise any convenient information carrying medium, including wires, optical fibers, or other media for carrying signals between computer systems, as is conventional. Good. Computer systems transfer information over a network by means of messages transferred over a communication link, each message including information and an identifier that identifies the device that receives the message.

  As described above, the computer graphics system 10 provides enhanced cooperation between shaders by facilitating the generation of phenomines with packaged and encapsulated shader DAGs or collaborative shaders DAG, with each shader The DAG comprises at least one shader that defines the characteristics of the 3D scene. Phenomina can be used to define various aspects of the scene, including the surface color of objects in the scene, texture characteristics, volume and geometric characteristics of the scene, the characteristics of the light source that illuminates the scene, rendering Included are features of a simulated camera or other image recording device that are simulated in, and numerous other features useful in rendering that will become apparent from the description below. Phenomina is built to ensure that shaders in a DAG or collaborative DAG can collaborate correctly while rendering an image of the scene.

  FIG. 2 shows a functional block diagram of a computer graphics system 10 used in one embodiment of the present invention. As shown in FIG. 2, the computer graphic system 10 includes two general parts, that is, a scene structure generation unit 20 and a scene image generation unit 21. The scene structure generator 20 is used during the scene entity generation phase by artists, drafters, etc. (generally “operators”) to generate representations of various elements, and this representation is used for scene image generation in scene rendering. Used by the unit 21, for example, the characteristics of the object and its surface in the scene, the structure and characteristics of the light source or source that illuminates the scene, and the camera that is simulated in generating the image during image rendering Including special equipment structure and characteristics. The expression generated by the scene structure generation unit 20 is a mathematical expression and is stored in the scene object database 22. The mathematical representation is evaluated by the image rendering unit 21 for display to the operator. The scene structure generation unit 20 and the scene image generation unit 21 can be resident on the same computer and can form a part thereof. In this case, the scene object database 22 can also be resident on the same computer. Or the computer 20 may reside on a server that is a client. Alternatively, the scene structure generation unit 20 and the scene image generation unit 21 may reside on different computers and may form a part thereof. In this case, the scene object database 22 is stored in a computer or a server of both computers. May be resident in either.

  More specifically, the scene structure generator 20 is used by the operator to simulate the object's geometric structure in the scene, the geometric properties of the light source that illuminates the scene, and the generation of the rendered image. Generate a mathematical representation that defines the position, geometric and optical properties of the camera to be played. The mathematical representation preferably defines three spatial dimensions, thereby specifying the position of the object in the scene and the characteristics of the object. An object can be a straight line or curve embedded in three-dimensional space, a two-dimensional surface embedded in three-dimensional space, one or more delimited and / or closed three-dimensional surfaces, or any combination thereof May be defined with primary, secondary, or three-dimensional features. In addition, the mathematical representation also defines temporal dimensions that are particularly useful in connection with computer animation where objects and their individual features are considered to move as a function of time.

  In addition to the mathematical representation of the object's geometric structure in the scene being rendered, the mathematical representation further defines one or more light sources that illuminate the scene and the camera. The mathematical representation of the light source defines, among other things, the structural features of the light source, including the location and / or orientation of the light source with respect to the scene, whether the light source is point light, straight or curved, flat or curved. The mathematical representation of the camera defines conventional camera parameters including, among others, one or more lenses, focal length, orientation of the imaging surface, and the like.

  The scene structure generator 20 also facilitates the generation of phenomena described in detail below and the association of the phenomenon with respect to individual elements of the scene. Phenomina generally defines other information necessary to complete the definition of the scene used for rendering. This information includes, but is not limited to, characteristics such as color and texture, and characteristics of the surface of the geometric entity defined by the scene structure generation unit 20. Phenomenon may include mathematical representations or other objects that, when evaluated during a rendering operation, allow the computer that produces the rendered image to display individual surfaces in the desired manner. enable. Under the control of the operator, the scene structure generator 20 associates the phenomina with a mathematical representation for the individual element (ie, object, surface, volume, etc.) used with it, and efficiently converts the phenomina to the individual element. "Add" well.

After the mathematical representation is generated by the scene structure generator 20 and stored in the scene representation database 22, the scene image generator 21 is used by the operator during the rendering phase, for example, the video display device 13 (FIG. 1). ) Generate an image of the above scene.
The scene structure generation unit 20 includes several elements, and includes an entity geometric expression generator 23, a phenomenon creator 24, a phenomenon database 25, a phenomenon editor 26, a basic shader node database 32, a phenomenon instance database 33, and a scene assembler 34. All of which operate under the control of operator input information entered via the operator interface 27. The operator interface 27 may generally include the operator input device 12 and the video display device 13 of the computer graphics system 10 as described above in connection with FIG. The entity geometric representation generator 23 facilitates the generation of mathematical representations of objects, light sources and cameras in the scene as described above under the control of operator input from the operator interface 27. Phenomenon Creator 24 allows the operator to use Phenomena in conjunction with the scene or otherwise (as described below) using the operator interface 27 and basic shader nodes from the basic shader node database 32. Provides a mechanism that can generate After the Phenomenon is generated by Phenomenon Creator 24, it is stored in the Phenomenon database 25 (ie, Phenomenon). After the phenomenon is stored in the phenomenon database 25, an instance of phenomenon can be created by the phenomenon editor 26. In that operation, the operator uses the Phenomenon Editor 26 to provide values for various Phenomenon parameters (if any). For example, based on input from an operator, a phenomenon that is established, adjusted, or modified when added or later is created to provide features such as color balance, texture roughness, and gloss. The Phenomenon Editor 26 allows the operator to establish, adjust or modify special features via the operator interface 27. The value for the parameter may be fixed or change according to a function of a variable (eg, time). An operator can use the scene assembler 34 to add a phenomenon instance generated using the phenomenon editor 26 to elements of the scene as generated by the entity geometric representation generator 23.

  Although the phenomenon editor 26 has been described to retrieve the phenomenon generated by the phenomenon creator 24 of the scene structure generation unit 20 of the computer graphics system 10 from the phenomenon database 25, one of the phenomena provided in the computer graphics system 10 is described. Alternatively, two or more, and possibly all, may be pre-defined and created by other devices (not shown) and stored in the Phenomenon database 25 for use by the Phenomenon Editor 26. In such a case, the operator can control the Phenomenon editor via the operator interface 27 to select a pre-defined phenomenon suitable for addition to the scene.

  The scene image generation unit 21 includes several components including an image generator 30 and an operator interface 31. When the scene image generation unit 21 forms part of the same computer as the scene structure generation unit 20, the operator interface 31 may include the same components as the operator interface 27 although it is not necessary. On the other hand, when the scene image generation unit 21 forms part of a different computer, the components of the two operator interfaces 31 and 27 are similar, but the operator interface 31 is a different component as the operator interface 27. Is generally provided. The image generator 30 retrieves a scene representation to be rendered from the scene representation database 22 under the control of the operator interface 31 and generates a rendered image for display on the video display device of the operator interface 31. To do.

  Before proceeding, it would be beneficial to further describe the “phenomenon” used in the present invention. Phenomenon is not limited to the mathematical representation generated by the entity geometric representation generator 23, but is limited thereto, but the surface color, texture, Provides information used to complete the definition of the scene used for rendering, including features such as closed volume. Phenomenon comprises one or more nodes interconnected in the form of a directed acyclic graph (DAG) or multiple collaborative DAGs. One of the nodes is the main root node and is used to add a phenomenon to an entity in the scene, and more specifically to the mathematical representation of the entity. Other types of nodes that can be used in Phenomenon consist of optional root nodes and shader nodes. Shader nodes can include any of several traditional shaders, including traditional simple shaders, and at the same time texture shaders, material shaders, volume shaders, environment shaders, shadow shaders, displacement shaders, and rendering A material shader can be provided that is used in connection with generating the rendered representation. In addition, Phenomenon can use many other types of shader nodes, including: (I) A geometric shader that can be used to add geometric objects to the scene. A geometric shader essentially comprises a mathematical representation of a predefined static or procedural entity in three-dimensional space, which is associated with the entity in the scene. Similar to the representation generated by the generator 23, except that they can be provided during preprocessing, for example to define individual regions where other shader limits used in individual Phenomenon are defined. The geometric shader essentially has access to the entity's scene building elements of the entity geometric representation generator 23 so that it can, for example, in a static or procedural manner, The scene representation can be changed when stored in the scene object database to modify or create new geometric elements. Note that Phenomenon, composed entirely of a geometric shader DAG or a set of collaborative geometric shaders DAG, can be used to represent objects in the scene in a procedural manner. This is in contrast to typical modeling, which is achieved in modeling by performing a series of modeling operations to obtain a desired representation of an object in a computer by a human operator. Thus, in essence, geometric Phenomenon represents an abstract modeling operation that is encapsulated, automated, and parameterized. An instance of a geometric Phenomenon (ie, a geometric Phenomenon that is related to a set of parameter values that are fixed or change in a predetermined manner over time, etc.) is the scene image at startup during the preprocessing phase When evaluated by the generator 30, this results in a special geometric scene extension. (Ii) A photon shader that can be used to control the characteristics of photon interactions with the surface of an object in the scene, such as absorption and reflection, of photon paths in the scene. The photon shader facilitates physical correction simulation of the overall lighting and fire rays associated with rendering. In one embodiment, the photon shader is used during rendering by the scene image generator 30 during the pre-processing operation. (Iii) A photon volume shader that is similar to a photon shader but differs in that it operates in relation to the three-dimensional volume of the space of the scene instead of on the surface of the object. This extends to volume and allows simulation of fire and global illumination with closed intervening media such as scattering of photons by dust or fog particles in the air, or water vapor in clouds or the like. (Iv) Photon emitter shaders that are similar to photon shaders, but differ in that they are related to the light source and thus related to the emission of photons. Simulated photons whose radiation is simulated in relation to photon emitter shaders are photon shaders that can be used to simulate simulated photon paths and surface interaction characteristics, and in particular for individual paths. Along with the photon volume shader that can be used to simulate paths and other features in the three-dimensional volume. (V) A contour shader used in conjunction with contour generation during rendering. In one embodiment, there are three subtypes of contour shaders: contour store shaders, contour contrast shaders, and contour generation shaders. A contour storage shader is used, for example, to collect contour sampling information for a surface. The contour contrast shader is used to compare two sets of sampling information collected through the use of a contour store shader. Finally, the contour generation shader is used for generated contour dot information for storage in the buffer, which is used by the output shader (described below) in generating the contour line. (Vi) An output shader used to process information in the buffer generated by the scene image generator 30 during rendering. The output shader can access the pixel information generated during rendering, and in one embodiment, the contour drawing from the configuration operations, complex variations, and the contour dot information generated by the contour generation shader, as described above. To do. (Vii) A three-dimensional volume shader used to control how light or other visible light passes through part or all of the empty three-dimensional space in the scene. Three-dimensional volume shaders can be used for any of a number of procedural effects such as, for example, volume effects including fog and smoke, flame, hair, and particulate clouds. Furthermore, since 3D volume shaders are used in the context of light, they are also effective in the context of shadows due to procedural effects. (Viii) light sources, including, for example, color, orientation, and attenuation characteristics due to characteristics such as individual light source shapes, texture protrusions, shadowing and other light characteristics Used to control the radiation characteristics of.

Other types of shaders that are valid in connection with the definition of the scene can also be used in Phenomenon.
Phenomenon is defined by: (I) a description of Phenomenon's externally controllable parameters; (ii) one primary root node and optionally one or more optional root nodes; (iii) a shader used as a node; A description of the internal structure of the Phenomenon, including specifying how it is interconnected to form a DAG or multiple cooperating DAGs; (iv) optionally for use by the Phenomenon Editor 26, A description, such as a dialog box, defined by Phenomenon that allows an operator to provide values for parameters or attributes that are used to evaluate individual Phenomenon. In addition, Phenomenon can contain code that can be linked and executed from external declaration statements and libraries, which is standard in programming.

  As noted above, Phenomenon can include multiple cooperating DAGs. In such a Phenomenon, during rendering, information generated by processing of one or more nodes of the first DAG in the Phenomenon is associated with one or more nodes of the second DAG in the Phenomenon. Can be used for The two DAGs are nevertheless processed independently and can be processed at different stages in the rendering process. The information generated by the individual nodes in the first DAG that can "cooperate" with the nodes in the second DAG (that is, can be used by the nodes in the second DAG in its processing) from the individual nodes in the first DAG Can be transferred over any convenient communication channel, such as a buffer that can be allocated for it. By providing all DAGs that need to work together in the above manner in a single Phenomenon, all conditions for cooperation are met. This condition is not met if the DAG is not encapsulated in a separate Phenomenon or other entity and provided without separation.

  As an example of a phenomenon that includes several cooperating DAGs, the phenomenon includes a number of DAGs including instructions for generating a material shader DAG, an output shader DAG, and a label frame buffer. The material shader DAG includes at least one material shader to generate color values for the material and is encountered during the processing of the material shader DAG in the label frame buffer established in connection with the processing of the label frame buffer generation command. It also stores label information about the object. The output shader DAG consequently includes at least one output shader that retrieves label information from the label frame buffer to facilitate the execution of object specific composition operations. In addition to the label frame buffer generation command, Phenomenon can also have a command to control the operation mode of the scene image generator 30 so that both DAGs can function and cooperate. For example, such a directive can control the minimum sample density required for the two DAGs being evaluated.

  As a second example of a Phenomenon containing multiple cooperating shaders DAG, the Material Phenomenon is a material simulated by both a Photon shader DAG containing at least one photon shader and a Material shader DAG containing at least one material shader. Can be expressed. During rendering, the photon shader DAG is evaluated during fire and global illumination pre-processing, and the material shader DAG is evaluated during subsequent image rendering. During photon shader DAG processing, the information representing the simulated photon can be used for subsequent material shader DAG region representations to add lighting contributions from the fireline or global lighting pre-processing stage. Stored in In one embodiment, the photon shader DAG stores simulated photon information in a photon map, and the photon shader DAG uses it to send simulated photon information to the material shader DAG.

  As a third example of a Phenomenon that includes a plurality of cooperating shaders DAG, the Phenomenon may include a contour shader DAG that includes at least one contour shader type shader and an output shader DAG that includes at least one output shader. The contour shader DAG is used to determine how the contour is drawn by storing the “dots”, transparency, width, and other attributes of the selected color. The output shader DAG is used to collect all the cells created during rendering and combines them into a contour line when rendering is complete. The contour shader DAG includes a contour storage shader, a contour contrast shader, and a contour generation shader. The contour store shader is used to collect sampling information for later use by the contour contrast shader. The contour contrast shader is then used to determine if the sampling information collected by the contour storage shader is such that contour dots are placed in the image, and if so, the contour generation shader is Actually put contour dots. The example Phenomenon is (1) a first stage where sampling information is collected (by the contour storage shader), (2) a second stage where the decision whether to place a contour cell is made (by the contour contrast shader), (3) A four-stage collaboration that includes a third stage in which contour dots are created (by the contour generation shader) and (4) a fourth stage in which the created contour dots are created (by the output shader DAG). Illustrates the work.

In either stage, shaders do not utilize another shader in another stage, but are processed and evaluated individually at different times, but work together to enable the production of the final result.
As a fourth example of a phenomenon that includes multiple cooperating shaders DAG, the phenomenon may include a volume shader DAG and a geometric shader DAG. The volume shader DAG includes at least one volume shader that defines the characteristics of the delimited volume, such as, for example, a hair shader that simulates hair in the delimited volume. The geometric shader DAG includes at least one geometric shader that is used to include the outer boundary surface in the scene before rendering begins as a new geometric shape, with the appropriate material and volume shader DAG Defines the calculations performed on the hair associated with the original volume shader DAG attached to the surface. In this illustrated Phenomenon, the collaboration is between a geometric shader DAG and a volume shader DAG, which is a geometric figure that follows the procedure that the geometric shader DAG supports the volume shader DAG. Is introduced. The volume shader DAG takes advantage of this geometric shape, but the geometric shape is generated using the geometric shader DAG during pre-processing operations prior to rendering, while the volume shader DAG is used during rendering. So the volume shader DAG cannot create the geometric shape itself. The cooperation illustrated in connection with this fourth exemplary example provides that the shader or shaders with geometric shaders provide the elements used by the volume shader DAG to follow the procedure, It does not store data like the collaboration in the context of the first to third example examples, so it differs from that illustrated in connection with the first to third example examples.

All these examples illustrate the computer graphic effect that an image of a scene can be rendered using a number of cooperating but independent shader DAGs bundled in a single Phenomenon. ing.
With this background, the operations performed in connection with Phenomenon Creator 24 and Phenomenon Editor 26 are described in connection with FIGS. 3 and 5, respectively. Further, an exemplary phenomenon created in connection with the phenomenon creator 24 is described in connection with FIG. 4 and details of operations performed by the phenomenon editor 26 associated with the phenomenon shown in FIG. 6A and 6B will be described. FIG. 3 shows a Phenomenon Creator window 40 that allows the Phenomenon Creator 24 to display this window to the operator on the operator interface 27 so that the operator can define new Phenomenon and modify existing Phenomenon definitions. The Phenomenon Creator window 40 includes a plurality of frames: a shelf frame 41, a supported graph node frame 42, a control frame 43, and a Phenomenon graph canvas frame 44. The shelf frame 41 may include one or more Phenomenon icons, which are collectively identified by reference numeral 45, each at least partially for use in the scene structure generator 20. Expresses the defined phenomenon. The supported graph node frame 42 includes one or more icons, which are collectively identified by the reference number 46 and allow the operator in the Phenomenon, such as various types of shaders that can be used in the interface, Phenomenon. Represents an entity that can be selected for use. As described below, the icons shown in the supported graph node frame 42 can be used by an operator to form a node in a directed acyclic graph that defines the phenomenon to be created or modified. . In one embodiment, there are multiple types of nodes, including: (I) The main root node that forms the root of the directed acyclic graph, forms the connection to the scene, and typically provides color values during rendering. (Ii) Several types of optional root nodes that can be used as anchor points in the Phenomenon DAG and support the main root node (section (i) above). Exemplary types of optional root nodes include the following: (A) A lens root node that can be used to insert a lens shader or lens shader DAG into the camera for use during rendering. (B) A volume root node that can be used to insert a global volume (or atmosphere) shader or shader DAG into the camera for use during rendering. (C) An environment root node that can be used to insert a global environment shader or shader DAG into the camera for use during rendering. (D) A geometric root node that can be preprocessed during rendering and can be used to specify a geometric shader or shader DAG that can be added to the scene database to support geometry or other elements that follow the scene's procedures. (E) A contour storage root node that can be used to insert a contour storage shader into the scene option data structure. (F) An output root node that can be used in conjunction with a post process after the rendering phase. And (g) a contour contrast route that can be used to insert a contour contrast shader into the scene option data structure. (Iii) A shader node that represents a shader, that is, a function written in a high-level language such as C or C ++. (Iv) Used in conjunction with a light source. An optical node that provides a light source with a light shader, color, intensity, origin and / or direction, and optionally a photon emitter shader. (V) Used in conjunction with the surface. The material node provides color values to the surface and has inputs to the opaque display indicating whether the surface is opaque, material, volume, environment, shadow, displacement, photon, photon volume, and contour shader. (Vi) A Phenomenon node that is an instance of Phenomenon. (Vii) A constant node that provides a constant value that is an input to any other node. Constant values can be most data types in programming languages used for entities such as shaders, and can be of other nodes such as scalars, vectors, logic (Boolean), colors, transformations, etc. Any of them can be expressed. (Viii) A dialog node that represents a dialog box that can be displayed to the operator by the Phenomenon Editor 26 and can be used by the operator to provide input information to control the Phenomenon before or during rendering. Dialog nodes allow the Phenomenon Editor 26 to display push buttons, sliders, wheels, etc. so that the operator can use colors and other values used in connection with the surface to which the Phenomenon containing the dialog node is connected, for example. Can be specified. As shown in FIG. 3, the shelf frame 41 and the supported graph node frame 42 both include left and right arrows, collectively identified as reference numeral 47, whereby the icons shown in the individual frames. Can be shifted to the left or right (as shown in FIG. 3), and if there are more entities than can be displayed at once, the icons can be shifted to be displayed in the Phenomenon Creator window 40.

The control frame 43 allows the operator to perform control operations such as deleting or duplicating nodes in the shelf frame 41 or the supported graph node frame 42, starting construction of a new phenomenon, starting an online help system, ending the phenomenon creator 24, etc. It includes an icon (not shown) representing a button that can be used to perform
The phenomenon graph canvas 44 provides an area where the phenomenon can be created or modified by the operator. When the operator desires to modify the existing phenomenon, the operator selects the icon 45 by a “drag and drop” method using a pointing device such as a mouse, and the shelf frame 41 expressing the phenomenon. To the Phenomenon graph canvas 44. After the selected icon 45 associated with the modified Phenomenon has been dragged onto the Phenomenon graph canvas 44, the operator is connected to the icon 45 by arrows to represent a graph defining the Phenomenon. Or it can be expanded to show more than one node. A graph 50 representing an exemplary phenomenon is shown in FIG. As shown in FIG. 3, the graph 50 includes a plurality of graph nodes, comprising circles and blocks, each associated with an entity that can be used in the Phenomenon, the nodes interconnected by arrows and associated with the Phenomenon. Define the graph to be used.

  After the graph associated with the icon 45 dragged to the phenomenon graph canvas 44 has been expanded to show the graph defining the phenomenon associated with the icon 45, the operator can modify the graph defining the phenomenon. In that operation, the operator uses the corresponding “drag and drop” method to select an icon 46 from the supported graph node frame 42 representing the entity to be added to the graph, from the phenomenon graph canvas 44. To establish a new node in the graph. After the new node is established, the operator can click on both nodes in an appropriate way, and an arrow will appear between them to interconnect it to the nodes in the existing graph. Nodes in the graph can also be disconnected from other nodes by deleting the arrows that extend between the individual nodes, or removed from the graph by appropriate activation of the delete push button in the control frame 43. .

  Similarly, when an operator wants to create a new phenomenon, the operator uses the corresponding “drag and drop” method to select an icon 46 to represent a supported graph that represents the entity being added to the graph. By dragging from the node frame 42 to the Phenomenon graph canvas 44, a new node can be established for the graph to be created. Once a new node has been established on the Phenomenon graph canvas 44, the operator clicks on both nodes in an appropriate manner so that an arrow appears between them, and cross-connects it to the nodes in the existing graph. Can connect. Nodes in the graph can also be disconnected from other nodes by deleting the arrows that extend between the individual nodes, or removed from the graph by appropriate activation of the delete push button in the control frame 43. .

  After the operator specifies a DAG or a set of collaborative DAGs for Phenomenon, before the Phenomenon represented by the graph is stored in the Phenomenon Database 25 for either a new Phenomenon or a modified Phenomenon, The Phenomenon Creator 24 examines the Phenomenon graph to verify that it is consistent and can be processed during rendering. In that operation, the Phenomenon Creator 24 ensures that the interconnection between graph nodes does not form a cycle, thereby ensuring that the graph or graphs associated with the Phenomenon form a directed acyclic graph. And ensure that the interconnections between graph nodes represent consistent input and output data types. If the Phenomenon Creator 24 determines that the graph node forms a cycle, the Phenomenon essentially forms an endless loop that cannot generally be handled properly. These operations ensure that the phenomenon created or modified as described above can be processed by the scene image generator when an image of the scene to which the phenomenon is added is rendered.

After the operator creates or modifies the phenomenon, it is stored in the phenomenon database 25.
FIG. 4 shows an example of a phenomenon created in conjunction with the phenomenon creator 24 that can be generated using the phenomenon creator window described in connection with FIG. The example of Phenomenon shown in FIG. 4 is one identified by reference number 60 and can be used for the surface features of wood material. Referring to FIG. 4, Phenomenon 60 includes one root node identified by reference numeral 61, which is used to add Phenomenon 60 to scene elements. Other nodes in the graph include a material shader node 62, a texture shader node 63, a coherent noise shader node 64, which represent the material shader, texture shader, and coherent noise shader, respectively, and include a dialog node 65. . Dialog node 65 represents a dialog box that is displayed by Phenomenon Editor 26 and that allows an operator to provide input information that is used with Phenomenon when the image is rendered.

  Details of material shaders, texture shaders, and coherent noise shaders are known to those skilled in the art and will not be described further here. In general, a material shader has one or more outputs represented by “results”, which are provided to the root node 61. The material shader thus has several inputs including and represented by "glossiness" input, "ambient" color input, "diffuse" color input, "transparency" input, and "lighting" input Material shader node 62 receives input from them from dialog node 65 (for gloss input), texture shader node 63 (for ambient and diffuse color inputs), hardwired constant (for transparency inputs), and lighting Shown as received from the list (in the case of lighting input). The hardwired constant value displayed as “0.0” and provided for the transparency input indicates that the material is opaque. The "Glossiness" input is connected to the "Glossiness" output provided by dialog node 65, and when the material shader represented by node 62 is processed during rendering, the glossiness input value for it is This is obtained from the dialog box represented by the node, and this state is described below with reference to FIGS. 6A and 6B.

  The perimeter and diffuse input of the material shader represented by node 62 is provided by the output of the texture shader, as shown by the connection of the “result” output of node 63 to the individual inputs of node 62. When the wood material Phenomenon 60 is processed during a rendering operation, and particularly when the material shader represented by node 62 is processed, the texture shader represented by the processed node 63 is the ambient and diffuse color. Allows you to provide input values. As a result, the texture shader will have three inputs, including the ambient and diffuse color inputs represented by the “Color 1” and “Color 2” inputs shown on node 63, and the “Mixed” input. The values for the ambient and diffuse color inputs are represented by the dialog node 65 by the operator as represented by connections from the respective diffuse and ambient color outputs from the dialog node 65 to the texture shader node 63 of FIG. Provided by using a dialog box.

  Further, the input value for the input of the texture shader represented by node 63 is provided by the coherent noise shader represented by node 64. In this way, when the texture shader represented by node 63 is processed during a rendering operation, the coherent noise shader represented by the processed node 64 can provide a mixed input value. The coherent noise shader has two inputs including a “turbulence” input and a “cylindrical” input. The value for the turbulent input is provided by the operator using the dialog box represented by dialog node 65, as represented by the connection from the turbulent output from dialog node 65 to the coherent noise shader node 64. Is done. The input value for the cylindrical input, indicated by the logic value “true”, is hardwired to the Phenomenon 60.

  The operations executed by the phenomenon editor 26 will be described with reference to FIG. FIG. 5 shows a Phenomenon Editor window 70 that allows the Phenomenon Editor 26 to be displayed by an operator interface 27, which in one embodiment of the present invention is used by an operator to add a Phenomenon added to a scene. Establish and adjust input values for. In particular, the operator uses the Phenomenon Editor window to associate dialog nodes, such as dialog node 65 (FIG. 4), established for each Phenomenon during creation or modification, as described above in FIG. A value for the phenomenon provided by the dialog box can be established. The Phenomenon editor window 70 includes a plurality of frames including a shelf frame 71 and a control frame 72, and also includes a Phenomenon dialog window 73 and a Phenomenon preview window 74. The shelf frame 71 shows icons 80 representing various phenomena that can be used to add to the scene. Similar to the Phenomenon Creator window 40 (FIG. 3), the shelf frame includes left and right arrow icons, collectively identified as reference numeral 81, so that the icons shown in the individual frames are left or right. If there are more icons that can be shifted (as shown in FIG. 3) and can be displayed at once, the icons can be shifted and displayed in the Phenomenon Editor window 70.

The control frame 73 has icons (not shown) representing buttons that can be used by the operator to perform control operations such as deleting or duplicating nodes in the shelf frame 71, starting an online help system, ending the phenomenon editor 26, and the like. Including.
An operator can select a phenomenon whose parameter value is established by appropriately operating a pointing device, such as a mouse, to create an instance of phenomenon. (A Phenomenon instance corresponds to a Phenomenon whose parameter value is fixed.) After the operator selects a Phenomenon, the Phenomenon Editor 26 causes the operator interface 27 to display a dialog box associated with that dialog node in the Phenomenon dialog window. Enable display. An example dialog box used in connection with one embodiment of the wood material Phenomenon 60 described above with respect to FIG. 4 is described below with reference to FIGS. 6A and 6B. When the operator provides and adjusts input values that can be provided via a dialog box, the phenomenon editor 26 efficiently processes the phenomenon and displays the resulting output in the phenomenon preview window 74. In this way, the operator can use the Phenomenon Editor window 70 to view the results of the values established by the operator using the inputs available through the dialog box displayed in the Phenomenon dialog window.

  6A and 6B show details of the dialog node (in the case of FIG. 6A) and an example associated dialog box (in the case of FIG. 6B) used in connection with the wood material Phenomenon 60 shown in FIG. Show. The dialog node identified by reference numeral 65 in FIG. 4 is defined and created by the operator using the Phenomenon Creator 24 during the process of creating or modifying the special Phenomenon associated therewith. Referring to FIG. 6A, the dialog box 65 includes a plurality of tiles: an ambient color tile 90, a diffuse color tile 91, a turbulent tile 92, and a gloss tile 93. It will be appreciated that the individual tiles 90-93 are associated with the ambient, diffuse, turbulent, and gloss output values provided by the dialog node 65, respectively, as described above in FIG. Ambient and diffuse color tiles are associated with color values and can be specified using traditional red / green / blue / alpha, or “RGBA” color / transparency specifications, thus each color tile Are actually associated with multiple input values, each for red, green and blue in the color representation and transparency (alpha). On the other hand, each of the turbulent and glossy tiles 92 and 93 is associated with a scalar value.

  FIG. 6B shows an example dialog box 100 associated with the dialog node 65 (FIG. 6A) as displayed by the operator interface 27 under the control of the Phenomenon editor 26. In the dialog box 100, each of the perimeter of the dialog node 65 and each of the diffuse color tiles 90 and 91 is displayed by the operator interface 27 as an individual slider set, generally identified by reference numerals 101 and 102, respectively. Associated with one of the color representation colors used during the processing of the associated Phenomenon during rendering. Further, the turbulence and glossiness tiles 92 and 93 of the dialog node 65 are displayed as individual sliders 103 and 104 by the operator interface, respectively. The sliders in the individual slider sets 101 and 102 are manipulated by an operator using a pointing device such as a mouse in a conventional manner so that the Phenomenon editor 26 is connected to a Phenomenon 60 (FIG. 4) by a dialog node 65. It is possible to adjust individual combinations of colors for individual ambient and diffuse color values provided to shaders associated with other nodes. In addition, the sliders 103 and 104 associated with turbulence and gloss input can be manipulated by the operator so that the Phenomenon Editor 26 can be moved by dialog nodes 65 to shaders associated with other nodes of the wood Phenomenon 60. It will be possible to adjust the individual turbulence and gloss values provided.

  Returning to FIG. 2, after the operator uses the Phenomenon Editor 26 to establish values for the various Phenomenon and Phenomenon instances associated with the scene, these values are stored with the scene in the scene object database 22. Thereafter, a scene image can be rendered by the scene image generation unit 21, particularly the scene image generator 30, and is displayed by the operator interface 31. The operations executed by the scene image generator 30 are outlined by the flowchart shown in FIG. Referring to FIG. 7, the scene image generator 30 operates in a series of phases including a pre-processing phase, a rendering phase, and a post-processing phase. In the preprocessing phase, the scene image generator 30 examines the phenomenon that has been added to the scene to determine whether preprocessing and / or postprocessing operations need to be performed in connection therewith (step 100). . Thereafter, the scene image generator 30 determines whether the operation in step 100 is necessary in relation to at least one phenomenon added to the scene (step 101) and, if necessary, preprocessing. The operation is executed (step 102). Example processing operations include, for example, generating a geometric figure to generate a geometric figure defined for the scene, if the Phenomenon added to the scene includes a geometric shader. . Other example pre-processing operations include, for example, shadow and photon mapping, multiple inheritance analysis, and the like. If the scene image generator 30 makes a negative determination at that step following step 102 or step 101, the scene image generator 30 is requested in connection with the scene representation prior to rendering and is added to the scene. Further pre-processing operations unrelated to can be executed (step 103).

  After step 103, the scene image generator 30 performs a rendering phase in which rendering operations associated with the preprocessed scene representation are performed to generate a rendered image (step 104). In that operation, the scene image generator 30 identifies the phenomenon stored in the scene object database 22 and added to the various components of the scene as having been generated by the entity geometric representation generator 23, and each of the phenomena. Add all major and optional root nodes to the appropriate scene component for the type of root node. Thereafter, the scene image generator 30 performs image rendering. In addition, the scene image generator 30 generates information that can be used in post-processing operations during the post-processing phase, if necessary.

  After the rendering phase (step 104), the scene image generator 30 performs a post-processing phase. In that operation, the scene image generator 30 determines whether the operation performed in step 100 has indicated that a post-processing operation is required in connection with the phenomenon added to the scene (step 105). If the scene image generator 30 makes a positive determination in step 105, the post-processing operation required in connection with the phenomenon added to the scene is executed (step 106). Furthermore, the scene image generator 30 can perform post-processing operations that are not related to Phenomenon at step 106. The scene image generator 30 can perform post-processing operations in conjunction with the manipulated pixel values for color correction and filter to provide various optical effects. In addition, the scene image generator 30 may, for example, store the velocity and depth information stored in the Phenomenon added to the scene, in one embodiment, in association with the rendered image, for example, in association with each pixel value. Depending on, a post-processing operation can be performed if it includes an output shader that defines post-processing operations such as depth of field or motion blur calculations that can be done entirely in the output shader.

  The present invention provides numerous advantages. In particular, the present invention provides a computer graphic system that provides a mechanism for creating (see Phenomenon Creator 24) and manipulating (see Phenomenon Editor 26). Phenomenon so created is processed by the Phenomenon creator 24 to ensure that they can be processed during rendering consistently. Since Phenomenon is created before being added to the scene, Phenomenon can be created by programmers or other computer program development specialists, thereby reducing the burden on artists, drafters, etc. who need to deploy the program. That will be understood. Phenomenon also frees artists from separating the complexity of constructing many different interrelated shaders in a scene into independent tasks that can be performed in advance by a Phenomenon creation expert user. If Phenomenon is used, the configuration can be considerably automated. Once a Phenomenon or Phenomenon instance is created, it is independent of the scene and can be reused for multiple scenes, avoiding repetitive work.

  It will be understood that numerous changes and modifications may be made to the invention. As mentioned above, Phenomenon can be created separately from scene related usage, so the Phenomenon Creator 24 used to create and modify Phenomenon and the Phenomenon Editor used to create Phenomenon instances. 26 can be provided in another computer graphics system. For example, a computer graphics system 10 that includes a Phenomenon Editor 26 may need to include a Phenomenon Creator 24, for example, if the Phenomenon database 25 includes a suitable pre-created Phenomenon and the operator does not need to create or modify the Phenomenon. Absent.

  Further, as described above, the value of the Phenomenon parameter may be fixed or may vary based on a function of one or more variables. For example, a Phenomenon instance can be time-dependent or “animated” when one or more values of individual parameters change with time as a variable. This is usually separated into time intervals that are categorized by the frame number of a series of frames comprising a video, but the time dependence is nevertheless a function of time with an arbitrary Phenomenon parameter value. Can take the form, each of which is tagged with an absolute time value, so that the shader is not bound to a separation interval, even if the image is rendered with consecutive frame numbers.

  In this context, the Phenomenon Editor is used to select time dependent values for one or more parameters of the Phenomenon, creating a time dependent “Phenomenon instance”. In one particular embodiment, the selection of time-dependent values for the parameters of the Phenomenon is made by making a graphical and interactive addition to the Phenomenon what is referred to herein as a “Phenomenon quality control tree”. Achieved. The Phenomenon attribute control tree may be in the form of a tree or DAG, but is added to the Phenomenon parameters, particularly effectively added outside the Phenomenon and stored with the Phenomenon in the Phenomenon instance database. A Phenomenon property control tree is composed of one or more nodes, each of which is a shader in the sense that it provides, for example, a motion curve, a data lookup function, and the like. The Phenomenon quality control tree is preferably kept shallow and usually has very few branch levels. The Phenomenon attribute control tree can consist of only one shader and defines a function that, when activated, calculates values for the associated parameters. The Phenomenon attribute control tree can be kept shallow, which allows Phenomenon to enable and facilitate the encapsulation of complex shader trees or DAGs and store data for reuse, for example, in the rendering step This is because the evaluation by the optimum method is promoted. Operators can add such a Phenomenon attribute control tree to allow control of Phenomenon parameters, greatly increasing the flexibility for users to achieve custom effects based on the use of predefined and packaged Phenomenon. To improve. The number of different Phenomenon instances that can be created in this way is therefore greatly increased, while ease of use is not reduced by all the encapsulation of complexity in Phenomenon.

Further, the appearance and structure of the windows used in connection with the phenomenon creator 24 and phenomenon editor 26 described in FIGS. 3 and 5 may be different from those described herein.
It will be appreciated that the system according to the present invention can be constructed in whole or in part from special purpose hardware or general purpose computer systems, any combination of which can be controlled by any suitable program. Any program may, in whole or in part, contain part of the system in a conventional manner or be stored in the system, or the whole or part of it may be networked or information in a conventional manner. Can be provided to the system through other mechanisms for transferring In addition, the system uses an operator input element (not shown) that can be directly connected to the system or that can transfer information to the system via a network or other mechanism for transferring information in a conventional manner, It can be operated and / or controlled by information provided by the operator.

  Although the Phenomenon system described in US Pat. No. 6,496,190 and discussed above has proven very effective, a number of shading platforms and languages have recently been developed, so Whether it is hardware shading for video games or software shading for visual effects in motion pictures, existing shader languages have been narrowly focused on special platforms and applications. This platform dependency typical of traditional shader systems and languages can be a significant limitation.

  Accordingly, in the following section and its sections, (1) a shader method and system that can integrate various shading tools and applications under a single language or system build, and (2) efficiency. Methods and systems that enable efficient and simple shader reuse and repurpose, and are effective in the integration of video games and feature films that occur frequently (eg, Lara Croft-Tomb Raider), (3) Eliminates the need for computer programming, facilitates the design and construction of shaders, and provides an effective method and system for artists. A method and system that can detect and resolve a problem is described.

Section II Mental Mill FIG. 8 shows a flowchart of an overall method 150 according to one aspect of the present invention. The described method enables the generation of an image of a scene from a representation with at least one instantiated Phenomenon comprising an encapsulated shader DAG comprising at least one shader node in a computer graphics system To do.

In step 151, a metanode environment is constructed that is operable for creation of metanodes with component shaders that can be combined to build more complex shaders in the network.
In step 152, a graphical user interface (GUI) is constructed that is in contact with the metanode environment, manages the metanode environment, and allows the user to build shader graphs and phenomenons using the metanode environment.

  In step 153, a software language is provided as an interface that can be used by human operators to manage the metanode environment, implement shaders, and integrate separate shading applications. The software language can be built as a superset of multiple selected shader languages for the selected hardware platform, and the compiler function selects from a single, reusable description of Phenomenon expressed in the software language It is possible to generate optimized software code for a selected hardware platform in a selected shader language.

In step 154, at least one GUI library is provided, which can be used in conjunction with the metanode environment to generate a GUI that can build shader graphs and phenomenons.
In step 155, an interactive, visual, real-time debugging environment is established that is in contact with the GUI, (1) allowing the user to detect and correct potential defects in the shader, and (2) Test scenes with shaders, metanodes, or phenomenons under test are operable to provide a viewing window that is always rendered.

  In step 156, the facility is built, which is in communication with the compiler function, and the selected hardware platform and software code optimized for the selected shader language are displayed for the selected shader language. It is operable to convert to machine code for the selected integrated circuit instantiation using a native compiler function.

The following discussion is summarized in four main sections:
A. Mental mill function overview Mental mill GUI specifications C.I. MetaSL design specifications MetaSL shader debugger

A. Mental Mill Functional Overview Mental Mill ™ technology provides an improved approach to creating shaders for visual effects. Mental mills solve many of the problems faced by shader writers today and make them resistant to future shader platform changes and deployments.

  In addition to providing a user interface for stand-alone operation, the mental mill further includes a library that provides an API for managing shader creation. This library can be integrated into third-party applications in a componentized manner, allowing the application to use only the mental mill components it requires.

  The basis for mental mill shading is the mental mill shading language MetaSL ™. MetaSL is a simple but expressive language specially designed to practice shaders. Mental mills facilitate the creation of simple, compact componentized shaders (referred to as Metanodes ™) that can be combined with shader networks to build more complex and visually interesting shaders.

  The goal of MetaSL is not to introduce another shading language, but to leverage the power of existing languages through a single metalanguage, MetaSL. Existing shader languages focus on relatively special platforms or situations, such as gaming hardware shading, or software shading for special film visual effects. MetaSL integrates these shading applications into a single language.

  Mental mills allow the creation of shader blocks called “metanodes” that are written in MetaSL and added and combined to form sophisticated shader graphs and Phenomenon ™. The shader graph provides an intuitive graphical user interface for creating shaders that are accessible to users who do not have the technical expertise to write shader code. Mental Mill's graphical user interface library utilizes the shader graph paradigm to provide users with a complete graphical user interface for building shader graphs and phenomena. As discussed in detail below, the present invention provides a “metanode environment”, an environment that is operable for creation and manipulation of metanodes. Further, as discussed below, the described metanode environment can be implemented as software, or a combination of software and hardware.

  Stand-alone applications are included as part of the mental mill, but the mental mill provides a cross-platform, componentized library so that it can be integrated into third-party applications. Designed. A stand-alone mental mill application simply uses these libraries just like any other application. The mental mill library can be subdivided as follows. (1) Phenomenon Creator Graphical User Interface (GUI), (2) Phenomenon Phenomenon Shader Graph Compiler, and (3) MetaSL Shading Language Compiler.

The Mental Mill Phenomenon Creator GUI library provides a group of GUI components that allow users with extensive technical expertise to create complex shaders and phenomenons.
The main GUI component is the shader graph view. This view allows a user to build a phenomenon by creating shader nodes (metanodes or other phenomena) and attaching them together to the described graph. The shader graph provides a clear visual representation of the shader program that cannot be found when looking at the shader code. This allows shaders to be created by users who do not have the technical expertise to write shader code. The GUI library also provides other user interface components, summarized below.

Shader parameter editor—provides sliders, color pickers, and other control tools that facilitate editing of shader parameter values.
Render preview window—provides user interactive feedback on the progress of the user's shader.
Phenomenon Library Explorer-Allows users to browse and maintain a pre-built Phenomenon library.

Metanode Library Explorer-allows users to view and maintain Metanode's organized toolbox, Phenomenon infrastructure building blocks.
Code editor and integrated development environment (IDE) —Provides the tools necessary for more advanced users to develop new Metanodes with MetaSL. The IDE is integrated with the mental mill GUI to provide interactive visual feedback. In addition, the IDE provides a high level interactive visual debugger for finding and correcting defect locations in the shader.

The mental mill GUI library is a cross-platform componentized. The library was developed without relying on any special operating system or platform user interface libraries.
In addition, the mental mill GUI library is designed for integration into third party applications. The GUI library components have a default look and feel, but a plug-in interface is provided to allow the appearance and feel of the Phenomenon Creator GUI to be customized to match the look and feel of the host application Is done.

The MetaSL shading language integrates the many shading languages available today and can be extended to support new languages and platforms as they emerge in the future. This allows MetaSL to break away from platform dependencies.
MetaSL is a simple but powerful language that targets the needs of shader writers. This allows shaders to be written in a compact and very readable syntax that can be approached by users who would otherwise not feel comfortable programming.

  Since MetaSL shaders can be written without any dependency on a special platform, MetaSL shaders can be used in a variety of different ways. A single shader can be used for off-line rendering in software or real-time rendering of hardware. The same shader can be used across different platforms, such as those used by next generation video game consoles.

By writing shaders in MetaSL, the time spent developing shaders is protected from being left behind in any language and is exploited by the possibility of reuse on many different platforms.
The MetaSL compiler itself that is part of the mental mill library is extensible. Since the compiler front end is a plug-in, a parser (operand parser) for other languages or syntax can be replaced with the MetaSL front end. Similarly, because the compiler backend is also a plug-in, the new target platform will be easily supported in the future. Extensibility of both ends of the mental mill compiler library allows it to become a shader generation hub.

Shader lighters typically face some difficulty on the front. The following sections outline these issues and the rationale behind the creation of mental mill technology designed to provide a complete solution set.
Shaders developed by mental mills are platform independent. This is an important feature of mental mills, and the effort spent developing shaders will not be wasted as the target platform evolves. This platform independence is provided for both shaders written in MetaSL and shader graphs for metanodes.

  The mental mill library provides an application programming interface (API) for dynamically generating shaders for special platforms on demand from Phenomenon shader graphs or monolithic MetaSL shaders. Alternatively, the mental mill allows the shader to be exported to a static file in the format required by the target platform. This allows the shader to be used without the need for a mental mill library.

  FIG. 9 shows a diagram of an overall system 200 according to one form of the invention. As shown in FIG. 9, system 200 includes a mental mill processing module 202 that includes a number of sub-modules and other components described below. The mental mill processing module 202 receives input in the form of Phenomenon 204 and MetaSL code 206. The mental mill processing module 202 provides the source code of the selected shader language, including Cg 208, HLSL 210, GLSL 212, Cell SPU 214, C ++ 216, etc. as output. Furthermore, the mental mill 202 can be adapted to provide as source the source code of future languages 218 that have not yet been developed.

  Platform independent components are isolated from special rendering algorithms. For example, hardware rendering often employs a different rendering algorithm compared to software rendering. Hardware rendering renders complex geometric shapes very fast, but does not directly support advanced lighting algorithms such as global lighting.

MetaSL is considered to be divided into three subsets or levels, each level differing both in its amount of expressiveness and its suitability for different rendering algorithms. FIG. 10 shows a diagram showing the levels of MetaSL 220 as a subset. Dotted oval region 224 shows C ++ as a subset of the reference.
Level 1 (221) —This is the most common subset of MetaSL. Shaders written within this subset can easily be the goal of a wide variety of platforms. Many types of shaders can write the whole in this subset.

  Level 2 (222)-a superset of Level 1 (221), Level 2 (222) adds features that are typically available only with software rendering algorithms such as ray tracing and global illumination. Like Level 1 (221), Level 2 (222) is still a relatively simplified language, and shaders written within Level 2 (222) are partially rendered on the hardware platform. Is possible. This allows a portion of the rendering to achieve a mix of rendering algorithms that are performed in hardware and software.

  Level 3 (223) —This is a superset of both Level 1 (221) and Level 2 (222). In addition, level 3 (223) is also a general C ++ language superset. Level 3 (223) shaders can only be implemented in software, but level 3 (223) is the most expressive of the three levels because it includes all the features of C ++. However, given that there are few shaders that require C ++ complexity and that Level 1 (221) has a minimal, general set of possible goals, most shaders have Level 1 (221) and It seems to be written using only level 2 (222).

  Level 1 (221) is considered the smallest subset of MetaSL, but it is also the most common in the type of platform it supports. Level 3 (223) is the largest superset, including all of C ++, making it very powerful and expressive. For applications that need it, level 3 (223) allows complex shaders to be written using all of the features of C ++. The price of using level 223 is that there are limited goals to support it. FIG. 11 is a bar graph 230 showing the level of MetaSL and its applicability to hardware and software rendering.

  Level 1 and 2 shaders (221, 222) have a high level of compatibility, the only difference being that Level 2 shaders (222) use advanced algorithms that cannot be launched on the GPU. . However, the MetaSL compiler removes features that are not supported by Level 1 (221) and replaces them with no-ops so that it looks as if it is a Level 1 shader (221) (and the target hard Level 2 shaders (222) can be used, such as This feature of the MetaSL compiler and its ability to detect a given shader level also allows the MetaSL compiler to generate the shader's hardware and software versions simultaneously (or only the software shader when it is needed) Generation). Hardware shaders can be used for direct feedback to the user via hardware rendering. Software rendering can then add more accurate images.

  Another useful feature of the mental mill is the ability to easily change the purpose of the shader. One important example is found in the integration of video games and feature films. You rarely see video games developed with permission to use their content from successful movies. Increasingly, feature films are also produced based on successful video games. It makes sense to use the same artistic assets for video games and movies based on them, but in the past, movies are rendered using a completely different rendering algorithm than video games So this was a challenge for shaders. The mental mill overcomes this obstacle by allowing the same MetaSL shader to be used in both situations.

  A shader graph model for building shaders also facilitates the reuse of shaders. A shader graph essentially facilitates the construction of shaders in a componentized way. A single metanode implemented by a MetaSL shader can be used in many different ways by many different shaders. In fact, the entire graph sub-tree can be packaged in Phenomenon and reused as a single node.

  The mental mill graphical user interface provides a way to build shaders that do not necessarily involve programming. Thus, an artist or other person who does not feel comfortable writing code will now have the ability to create a shader himself. In the past, creating a shader required relying on the programmer, which was a slow process that could involve many iterations between the programmer and the artist. Giving the artist control over the shader creation process not only frees the programmer for other tasks, but also allows the artist to more freely explore the possibilities that are possible through custom shaders.

  The mental mill user interface also provides a development environment for programmers and technical directors. Programmers can create custom metanodes written in MetaSL, and artists can use these nodes to create new shaders. Technology directors can create complex custom shader graphs that implement functional units and package them into Phenomenon. These different levels of flexibility and complexity give the user broad control over the shader creation process and can involve the user in a wide range of technical and artistic expertise.

  An important aspect of creating shaders is the ability to analyze defects, determine their cause, and find solutions. That is, the shader rieta must be able to debug the shader. Finding and resolving defects in a shader is necessary regardless of whether the shader was created by adding a metanode to form a graph, or by writing MetaSL, or both . The mental mill provides users with the ability to debug their shaders using a high level of visual technology. This allows the shader creator to visually analyze the state of the shader and quickly isolate the cause of the problem. A prototype application was created as a proof of concept for this shader debug system.

  Almost all shader applications, such as offline or real-time interactive rendering, require that the shader achieve the highest possible level of performance. Typically, shaders are activated in the most performance critical section of the renderer, and thus have a significant impact on overall performance. For this reason, it is important that the shader creator can analyze the performance of the shader at a fine point and isolate very expensive parts in the calculation of the shader.

  The mental mill provides an analytical method called profiling with an intuitive graphical representation. This allows the mental mill user to receive visual feedback indicating the relative performance of the shader parts. This profiling information is provided both at the node level for nodes that are part of the graph or phenomenon and at the statement level for MetaSL code contained in the metanode.

  A shader's performance timing can depend on the particular input value that drives the shader. For example, a shader can include a loop where the number of iterations through the loop is a function of a special input parameter value. The mental mill's graphical profiler allows shader performance to be analyzed in the context of a shader graph where the node resides, and the performance results are special input values that drive the node in that situation. Can be relative to.

  Performance information at any particular level of detail is normalized to the overall performance cost of the node, i.e., the entire shader, or to the cost of rendering the entire scene by multiple shaders. For example, the execution time of a MetaSL statement in a metanode can be expressed as a percentage of the total execution time of that metanode or, if the metanode is a member of a graph, the total execution time of the entire shader.

A graphical representation of performance results is provided using multiple visualization techniques. For example, one technique is to present normalized performance costs by mapping percentages to color gradients.
FIG. 12 shows a screenshot 230 illustrating this aspect of the invention. A MetaSL code list 232 appears in the center of the screen 230. A color bar 234 appears to the left of each statement 232 indicating relative performance. The first 10 percent points are mapped to the blue slope and the remaining 90 percent points are mapped to the red slope. Using such a non-linear mapping, the user's attention can be focused on “hot spots” in the MetaSL code. In addition, the user has access to special numerical values used to select a color from the gradient. As the user sweeps the mouse over the color bar, a pop-up displays the statement execution time as a percentage of the total execution time.

  If the shader is part of an animation, its performance characteristics will change over time, either because the overall cost of rendering the scene will change over time, or because the shader itself is a function of time. The graphical representation of performance results is updated as the animation progresses to reflect these changes in the performance profile. In the screen 230 of FIG. 12, the color bar 234 adjacent to each statement 232 of the code changes to reflect the measured performance change as the animation is played.

  FIG. 13 shows a performance graph 240 illustrating another visualization technique. The graph 240 displays performance results for a range of values for special input parameters. In this example, the performance cost of the shader lighting loop is graphed against the number of lights in the scene. In this example, the jump in performance cost indicates that the shader must be broken down into paths in order to meet the graphics hardware constraints.

The mental mill can also display performance results in tabular form. FIG. 14 shows a table 250 in which the performance timing of each Phenomenon node is displayed with respect to the overall performance cost of the entire shader.
Graphical profiling techniques with mental mills, like other features of mental mills, are platform independent. This means that performance timing can be generated for any supported target platform. As new platforms emerge, new back-end plug-ins to the mental mill compiler are provided, and these new platforms can be profiled in the same way. However, any special timing is measured for a selected target platform. For example, the same shader can be profiled when running on hardware versus software, or on a different hardware platform. Different platforms have individual characteristics, so the performance profile of a special shader may look very different when comparing platforms. The ability of shader creators to analyze shaders on different platforms is important for developing shaders that run with considerable performance on all target platforms.

  FIG. 15 shows a diagram of a mental mill library component 260. The mental mill library component 260 is divided into two main categories. They are a graphical user interface (GUI) library 270 and a compiler library 280. The GUI library 270 includes the following components. The Phenomenon Graph Editor 271, Shader Parameter Editor 272, Render Preview Window 273, Phenomenon Library Explorer 274, Metanode Library Explorer 275, and Code Editor and IDE 276. The compiler library 280 includes the following components. They are the MetaSL language compiler 281 and the Phenomenon Phenomenon shader graph compiler 282.

  FIG. 16 shows a more detailed view of the compiler library 280. The mental mill compiler library 280 provides the ability to compile MetaSL shaders into special platforms or shaders targeted to multiple platforms simultaneously. The compiler library 280 also provides the ability to compile shader graphs that implement Phenomenon on a flat monolithic shader. By flattening a shader graph into a single shader, the overhead of call from shader to shader is reduced to nearly zero. This allows a graph constructed from small shader nodes to be used efficiently without incurring much overhead.

  According to a further aspect of the invention, the mental image next generation renderer and Reality Server (R) will be based on MetaSL and monolithic C ++ shaders. For this purpose, they include a copy of the MetaSL compiler that generates executable C ++ and hardware shader code from MetaSL shaders and Phenomenon. FIG. 17 shows a diagram of a renderer 290 according to this aspect of the invention.

  In the configuration of FIG. 17, neither MetaSL nor other shader codes are exported. Note that all language paths are shown in this figure, but a typical renderer does not use all five rendering units shown at the bottom. Typically, only the two most appropriate (one software and one hardware) are used at a time.

  MetaSL compiler extensibility supports multiple target platforms and shading languages. New goals can be supported when they appear in the future. This extensibility is achieved by plug-ins to the compiler backend. The MetaSL compiler handles a lot of processing and provides the back-end plug-in with a high-level shader representation that can be used to generate shader code. The MetaSL compiler currently targets high-level languages, but may directly target the GPU and generate machine code from high-level representations. This allows code generators to work directly from this high level representation and take advantage of the inherent optimizations that are available only to avoid native compilers. Will.

The mental mill GUI library provides an intuitive, easy-to-use interface for building sophisticated shader graphs and phenomena. The library is componentized and implemented in a platform independent manner, allowing some or all of the UI components to be integrated into a third party application.
A standalone Phenomenon Creator application is also provided, which utilizes the same GUI that can be used directly for integration into other applications.

The main GUI components provided by the library are:
Phenomenon graph editor-The graph editor allows shader graphs to be constructed by connecting inputs to metanode outputs.
Shader Parameter Editor—The parameter editor provides intuitive user interface controls for setting parameter values for shader node input.

Render preview window—The preview window provides interactive feedback to the user when building a shader graph and editing shader parameters.
• Phenomenon Library Explorer-allows users to browse pre-built Phenomenon libraries. Users can add their own shader graphs to the library and configure their contents.

Metanode Library Explorer-The Metanode Library provides a metanode toolbox that users can use to build shader graphs. New metanodes can be created by writing MetaSL code and added to the library.
Code editor and IDE-Code editor and integrated development environment (IDE) allow new metanodes and monolithic MetaSL shaders to be written directly in the GUI. When the user writes MetaSL code, the code is automatically compiled and the results can be immediately viewed in the shader graph and preview window. An interactive, visual-based debugger allows users to step through MetaSL code and examine variable values. Through the mental mill GUI, the user can know a particular numerical value of a variable at a particular pixel location, and can examine multiple values that the variable can take on the surface of the object.

  The Phenomenon Graph Editor allows the user to build shaders and Phenomenons by clicking and dragging with the mouse to place nodes and add them together to form a graph. An extensive toolset assists the user in this task and makes it easy to navigate and maintain complex shader graphs. FIG. 18 shows a diagram of the graph editor user interface 300.

Graph nodes are represented at various levels of detail, allowing the user to zoom out to get a large view of its entire graph and zoom in to see all of the details of any particular node Become.
The portion of the shader graph can be incorporated into a Phenomenon that looks like a single node when closed. This allows the user to successfully deal with large complex graphs of many nodes by grouping subgroups into a single node. Phenomenon can be opened so that the user can edit its internal graph.

  Each node in the shader graph has a preview window that shows the state of the shader at that point in the graph. This provides a visualization debugging mechanism for the creation of shaders. The user can follow the data flow of the graph and see the results so far at each node. At first glance, the user can see a visual representation of the shader's construction.

FIGS. 19-22 show a series of screenshots 310, 320, 330, 340, and 350 showing the mental mill Phenomenon graph editor and an integrated MetaSL graphical debugger.
FIG. 24 shows a diagram of the shader parameter editor 360. The parameter editor 360 allows the user to set special values for shader and phenomenon parameters. When creating a new shader type, this allows the user to specify default parameter values for future instances of that type.

It can be added from within the parameter view and the user can follow the attachment path in this view from one node to another. This provides an alternative method for creating and editing shader graphs that is useful in some situations.
A resizable render preview window allows the user to interactively visualize the shader results. This preview window can provide a real-time hardware-accelerated preview of the shader along with high quality software rendering results, including advanced rendering algorithms such as ray tracing and global illumination.

  The Phenomenon / Metanode Library Explorer View allows users to browse and configure Phenomenon and Metanode collections. FIG. 25A shows a thumbnail view 370 and FIG. 25B shows a list view 380. To create a new node in the current graph from one of these libraries, the user simply drags the node and drops it into the graph.

Libraries can be stored and categorized for organization. Users can see the library in list view or icon view and see a collection of samples that show the function of each node.
The code editor allows users to create new metanodes by writing MetaSL code. The code editor is integrated into the rest of the mental mill GUI so that the user edits the shader code and the rest of the user interface updates each other to reflect the changes.

  FIG. 26 shows the code editor and IDE view 390. The code editor is also integrated into the mental mill compiler. When the user edits the shader code, you receive interactive feedback from the compiler. Errors or warnings from the compiler are presented to the user in this view with an option to highlight the part of the code that causes the error or warning.

  The mental mill MetaSL debugger provides the user with a source code listing containing the MetaSL code for the appropriate shader node. The user can then step through the shader's instructions and examine the value of the variable as it changes throughout the execution of the program. However, instead of simply providing the user with a single numeric value, the debugger displays multiple values simultaneously as a color mapped onto the surface of the object.

  Representing the value of a variable as a single image rather than a single number has several advantages. First, the user can immediately recognize the characteristics of the function driving the value of the variable and find the area that is operating incorrectly. For example, the rate of change of a variable through the surface can be seen in an intuitive manner by observing how the color is changing across the surface. This is difficult to recognize if the user is using the traditional method of debugging shaders one pixel at a time.

  The user can also use the visualization debug paradigm to quickly locate input conditions that produce undesirable results. Shader bugs appear only when certain input parameters take special values, and such situations only occur in special parts of the geometric surface. The mental mill debugger allows the user to navigate through the three-dimensional space using the mouse to find a picture around a location on the surface that is an indication of the problem and direct the view there.

  Conventional debugging techniques allow the user to go through the program line by line and examine the state of the program at each statement. Typically, the user can only step forward by one or more lines (in the direction of program execution), but jumping to any statement can generally restart the program. is necessary.

  The mental mill MetaSL debugger allows the user to jump to any statement in the shader code in any order. One particularly nice aspect of this feature is when code statements modify the value of a variable of interest. A shader writer can easily proceed before and after this statement and can exchange the values of variables before and after the statement is executed. This makes it easier for the user to analyze the effect of any special statement on the value of the variable.

  Obviously, displaying the value of a variable as a color mapped on the surface of the object works well when the variable is of color type. This method also works fairly well for scalar and vector values (less than 3 components), but the values must be mapped in the range 0-1 to produce a normal color. The mental mill UI allows the user to specify the range of scalars and vectors that will be used to map those values to colors. Alternatively, the mental mill can automatically calculate the range for any given viewpoint by determining the minimum and maximum values of the variables on the surface when viewed from that viewpoint.

  In addition to viewing the variable as a color mapped on the surface of the object, the user can take advantage of other visualization techniques provided by the mental mill. One such technique for vector values allows the user to sweep the mouse over the surface of the object and the mental mill debugger will point to the direction specified by the variable at that position on the surface. Draw. The debugger also displays numerical values for variables at pixel locations that can be selected with the mouse or specified by the user by providing pixel coordinates. This technique is illustrated in FIGS. 27A-C, which are a series of screen images 400, 410, 420 displayed on the surface of the image corresponding to different positions of the mouse.

  The mental mill shader debugger demonstrates another advantage of mental mill platform independence. The debugger can operate in hardware or software mode and operates independently of any special rendering algorithm or platform. The fact that the shader debugger is tightly integrated into the mental mill Phenomenon creation environment further reduces creation / test cycles and keeps shader creators working at a high level, isolated from platform dependencies. Make it possible.

  The mental mill GUI library is implemented in a platform-independent manner. The shader graph editor component uses a graphics API to ensure smooth performance when editing complex shader graphs. The graphic abstraction layer avoids any dependency on any special API. For example, some applications want to use DirectX over OpenGL, which also simplifies integration issues when applications also use DirectX.

  The rest of the GUI also uses an abstraction layer to avoid dependencies on any special platform operating system user interface libraries. FIG. 28 shows a diagram of a GUI library architecture 430 according to this aspect of the invention. FIG. 28 illustrates how these abstraction layers isolate the application and mental mill user interface from platform dependencies.

  When integrating mental mill GUI components into third-party applications, the look and feel of the components can be customized to better match the standards of the host application. This can involve purely superficial customization, using special colors or fonts, or can include customizing the appearance of the component and its behavior by user interaction. it can.

Below is how the mental mill GUI library allows customization.
Phenomenon graph appearance—Phenomenon graph components, such as metanodes and connecting lines, are drawn by invoking a plug-in callback function. A default drawing function is provided. However, third parties can also provide their own functionality and customize the appearance of the shader graph to better match the application. The callback function also handles mouse point hit testing because the components of the node can be placed at different locations.

• Keyboard shortcuts-All keyboard commands are relocatable.
• Mouse movements-Mouse movements like mouse button mapping can be customized.
Toolbar item—Each toolbar item can be omitted or included.
View windows-Each view window is designed to operate on its own without depending on other windows. This allows third parties to integrate only the Phenomenon graph view into their own application, for example. Each view window can be driven by an API, allowing third parties to include any combination of view windows by replacing some of the view windows with their own user interface.

Mental Mill Shading Language-MetaSL is simple, intuitive, and expressive enough to represent the entire shader spectrum required for a wide range of platforms supported by mental mill .
The MetaSL language generally uses concepts found in other standard shading languages along with programming languages. However, MetaSL is designed for efficient shader programming. Users familiar with other languages will be able to learn MetaSL quickly, while users without programming skills expertise will understand many parts of the MetaSL shader because of its readability. Seem.

Here is a functional overview of MetaSL.
Shader class—The MetaSL shader class declaration describes the interface to the outside of the shader. Shader declarations, along with other member variables, include specification of input and output parameters.
A shader declaration includes a declaration of a method called a shader entry point, main. In addition, an optional event method allows shaders to respond to the start and end of events.

A shader class can also contain declarations of other member variables and methods. Other member variables can hold data used by the shading calculation and can be initialized from the shader's event method. Other member methods can function as helper methods that are called by the shader's main or event methods.
The following is an example shader declaration.

shader Phong
{
input:
Color ambient;
Color diffuse;
Color specular;
Scalar shininess;
output:
Color result;
void main ();
void event ();
};
MetaSL offers a wide range of built-in data types.

-Scalar-Floating point value-Bool-Boolean-String-String-Color-Color-Vector-A vector of length 2, 3, or 4 is provided. In addition, boolean and integer vectors are also supported.

Matrix-N × M size matrix is supported, where N and M are 2, 3, or 4.
• Texture-1d, 2d, 3d, and 3D map texture types are provided.
Shader—Shader instance data type A standard set of mathematical functions and operators that work with vectors and matrices is provided. Mathematical operators are supported for multiplying, dividing, adding, and subtracting matrices and vectors. Thereby, the following compact and easy-to-read expressions are possible.

Vector3 result = pt + v * mat;
A concept called swizzling is also supported. This allows the vector components to be read or written while simultaneously rearranging or replicating the components. For example,
vect.yz
The result is a 2d vector constructed from the y and z components of the vect.

vect.xxyy
The result is a 4d vector with the x component of vect assigned to the first two components and the y component of vect assigned to the last two components.
v1.yz = v2.xx
Assign the x component of v2 to both the y and z components of v1.

Vector types are implicitly converted from one type to another if there is no data loss due to the conversion. The Color type is provided primarily for code readability, otherwise it is synonymous with Vector4.
In addition to the built-in types provided by MetaSL, you can define custom structure types. The structure can be used for both input and output parameters, along with other variables. Both structures and built-in types can be declared as arrays. The array can have either a fixed or dynamic size. Array elements are accessed with parenthesis ([]) syntax.

struct Texture_layer
{
Texture2d texture;
Scalar weight;
};
Texture_layer tex_layers [4];
This example shows a custom structure type with variables declared as a custom type fixed length array.

MetaSL supports well-known programming constructs that control the flow of shader execution. Specifically, there are for; while; do, while: if, else; switch, and case.
The task of iterating over scene lighting and summarizing that lighting is abstracted in MetaSL by light loops and iterators. An instance of light iterator is declared and the foreach statement repeats on each scene light. Within the loop, the light iterator variable provides access to the resulting lighting from each light.

Color diffuse_light (0,0,0,0);
Light_iterator light;
foreach (light) {
diffuse_light + = max (0, light.dot_nl) * light.color;
}
A shader lighter does not have to worry about which lighting contributes to which part of the scene or how many times a given light needs to be sampled. The lighting loop automates the process.

  Within the shader's main method, a set of special state variables is implicitly declared so that shader code can be referenced. These variables hold values that describe both the renderer's current state, as well as information about the intersection that leads to the shader call. For example, normal is the interpolated normal at the intersection point. The state variables are described in detail below.

  There are two constructs that handle light and illumination in MetaSL. One is the BRDF (Bidirectional Reflection Distribution Function) shader type, which allows surface illumination models to be abstracted and rendered in a highly efficient manner. Alternatively, a light iterator provides a more traditional method for iterating over scene lighting and samples within each light.

  The BRDF shader approach is often preferred for several reasons. It allows efficient sampling and global illumination without having to create a separate photon shader. In general, a single BRDF implementation can be used unchanged by different rendering algorithms. It facilitates functions that perform certain lighting calculations, such as tracing shadow rays, with delayed methods that allow significant rendering optimization. It also provides an integrated description of the analysis and acquired lighting models.

In MetaSL, BRDF is a first class shader object. They have the same event method as a normal shader. However, instead of the main method, several other methods must be provided to implement BRDF.
FIG. 29 shows a table 440 that lists the methods required to implement a BRDF shader.

The in_dir and out_dir vectors in these methods are specified by the local coordinate system. This coordinate system is defined by the surface normal as the z-axis and by the tangent vector as the x and y axes.
BRDF is declared like a normal shader, except that the brdf keyword is used instead of the shader keyword. BRDF also differs from regular shaders due to the fact that it does not have an output variable. BRDF itself is its own output. The following is an example for Phong BRDF.

brdf Phong
{
input:
Color diffuse;
Color glossy;
Color specular;
Scalar exponent;
Color eval_diffuse (
Vector3 in_dir,
Vector3 out_dir)
{
return in_dir.z * out_dir.z <0.0?
diffuse: Color (0, 0, 0, 0);
}
Color eval_glossy (
Vector3 in_dir,
Vector3 out_dir)
{
Vector3 r = Vector3 (-in_dir.x, -in_dir.y, in_dir.z);
return pow (saturate (dot (r, out_dir)), exponent) * glossy;
}
Color eval_specular (
Vector3 in_dir,
Int specular_component,
out Vector3 out_dir,
out Ray_type out_type)
{
out_dir = Vector3 (-in_dir.x, -in_dir.y, in_dir.z);
out_type = RAY_REFLECT;
return specular;
}
Int specular_components ()
{
return 1;
}
};
Both surface shader combinations and direct and indirect BRDF shaders define shading for special surfaces. MetaSL provides two functions that surface shaders can use to perform lighting calculations. It is direct_lighting () that loops over some or all lighting and evaluates the given BRDF, and indirect_lighting () that calculates the lighting contribution from the global lighting. These two functions compute illumination as separate diffuse, glossy, and reflection components and store the results as output arguments in the variables passed to them.

void direct_lighting (
out Color diffuse,
out Color glossy,
out Color specular);
void indirect_lighting (
out Color diffuse,
out Color glossy,
out Color specular);
In most cases, the variable passed to the lighting function as an output parameter may be the actual output parameter of the root surface shader node. In other words, a call to the lighting function produces the final shader result. As long as there are no other direct dependencies on these output values, the renderer is free to delay its computation, thereby allowing significant optimization.

Another possibility is that the output of the surface shader is attached to the input of another node. This imposes a direct dependency on the result of the illumination function, which is possible, but eliminates the possibility of delayed illumination calculations and some potential performance gains with it.
To avoid imposing a dependency on the surface shader result, most operations that would have been performed by the output of the surface shader are applied to the BRDF node itself. A limited set of mathematical operators can be applied to BRDF nodes, but these are usually sufficient to achieve the most common use cases. These operations are
• Add 2 or more BRDFs • Multiply BRDF with scalar or color • Dynamically select BRDF from set

  The ability to add BRDF after scaling / enlarging with a scalar or color makes it possible to mix multiple lighting nodes represented by BRDF nodes. The scalar or color factor need not be constant; for example, the two BRDFs can be driven by the output of another shader to mix as a function of the texture of the Fresnel falloff. FIG. 30 shows a diagram of an example configuration 450 where two BRDFs are mixed.

In FIG. 30, a “Mix BRDF” node is a composite BRDF, which is realized by reducing / enlarging the two BRDF inputs by “amount” and “1-amount”, respectively. In this example, the “amount” parameter is attached to the output of the texture that controls the blending of the two BRDFs. The specular reflection of Phong BRDF is attenuated by the Fresnel attenuation function.
The material Phenomenon collects direct and indirect BRDF shaders, along with surface shaders that are themselves represented by shader graphs. When the surface shader activates the illumination function, the BRDF shader in the material Phenomenon is used to iterate over the illumination sample and calculate the result of the illumination function. In this case, since there is no dependency on the surface shader result, the illumination calculation is delayed to the optimal time by the renderer.

The BRDF shader type integrates the BRDF representation represented by an analytical model such as Phong with the acquired BRDF represented by the data generated by the measuring device. The direct_lighting () and indirect_lighting () functions work equally well with acquired or analytic BRDFs, regardless of the BRDF implementation given to them.
The raw data representing the acquired BRDF can be provided in a number of different formats, usually sparse and unstructured. Typically, raw data is provided to a stand-alone utility application where it is preprocessed. This application can organize the data into a regular grid, decompose the data, and / or compress the data to a more practical size. By storing the data in a floating point texture, the measured BRDF can be used for hardware shading.

  FIG. 31 is a diagram illustrating a pipeline 460 for shading using the acquired BRDF. A stand-alone utility application processes raw BRDF data, stores structured data in an XML file, and optionally references a binary file or texture to hold the actual data. The XML file provides a description of the data format and associated model or decomposition. This file and associated data can then be loaded by the BRDF shader at render time and used to define the BRDF. The XML file can be fed back to the utility application for further processing when requested by the user.

By storing the data in a floating point texture, the BRDF based on the data can operate in hardware. In this case, the texture that holds the data and any other parameters that describe the data model can be explicit parameters of the BRDF node.
For software shading with measured BRDF, there are two options for loading data. The first one is to introduce a unique C ++ function into the array that reads the data from the file. This unique function can then be called by Level 2 MetaSL BRDF shaders. Another option is to introduce the entire BRDF shader as a Level 3 MetaSL shader, thereby giving the shader full access to all C ++ features. This shader can read the data file directly, but it loses some of the flexibility of the Level 2 shader. The first option to load data from a native C ++ function is preferred as long as the data can be loaded into a Level 2 compatible representation such as an array. If the data must be represented by a structure that requires a pointer (such as a kd-tree), some of the implementations that require the use of pointers need to be Level 3 shaders.

Technology is one variation of shader realization. Some shaders may need only a single technology, but there are situations where it is desirable to introduce multiple technologies. The language provides a mechanism for declaring multiple technologies within a shader.
The technique can be used when it is desired to execute different code in a hardware or software environment, but often the same shader can be used for both hardware and software. Another use of the technique is to describe alternate versions of the same shader with different quality levels.

Technology is declared in the shader class. Each technology has its own version of the main and event methods, but parameters and other member variables or methods are shared with other technologies.
The following is an example of a technical declaration within a shader.
Shader my_shader {
input:
Color c;
output:
Color result;
technique software {
void event (Event_type event);
void main ();
}
technique hardware {
void event (Event_type event);
void main ();
}
};
The language allows material shaders to express their results as a series of components instead of a single color value. This allows the stored components to separate the image buffer for later synthesis. Each pass may render a subset of all components and combine them with the remaining previously rendered components.

The material shader breaks the result into components by declaring separate outputs for each component. The name of the output variable specifies the name of the current rendering layer.
shader Material_shader {
input:
...
output:
Color diffuse_lighting;
Color specular_lighting;
Color indirect_lighting;
};
This example shows a material shader that specifies three components for diffuse, reflective, and indirect illumination.

  If a scene has multiple material shaders that break down the results into different layers, the total number of layers can be large. The user will not want to allocate a separate image buffer for each of these layers. This mechanism in the scene definition file allows the user to specify compositing rules for combining layers into an image buffer. The user can specify how many image buffers to create and, for each buffer, specify an expression that determines the color to put in that buffer when the pixel is rendered. The expression may be a function of the layer value as follows:

Imagel = indirect_lighting
Image2 = diffuse_lighting + specular_lighting
In this example, the three layers from the shader result structure in the previous example are routed to two image buffers.
In order for shader users to interact with them in the GUI and set their parameter values, the application needs to know some additional information about the shader parameters and the shader itself. MetaSL provides the ability to annotate shader parameters, techniques, and the shader itself with additional metadata.

Shader annotation can describe parameter ranges, default values, and tooltip descriptions, among other things. You can also attach arbitrary data to the shader using a custom annotation type.
MetaSL includes an extensive collection of built-in functions. These include mathematics, geometry, and texture lookup functions, to name a few. In addition, functions that can only be supported by a software rendering platform are also included. An example is a function that casts a reflected ray to a point in space or calculates the amount of global illumination at a point in space.

B. Mental Mill GUI Specification The mental mill (TM) Phenomenon (TM) creation tool allows users to build shaders interactively without programming. Users primarily work in shader graph views, where Metanodes ™ are attached to other shader nodes to build complex effects. Metanodes are simplified shaders and form a building block for building more complex Phenomenon ™.

  The definition of “phenomenon” was introduced by the Mental Ray® renderer to fully understand the concept of “visual effects”. Briefly, a Phenomenon can be a shader, or a set of shader trees, or a collaborative shader tree (DAG), which includes a geometric shader, resulting in a single parameterized region with a defined domain A set of boundary conditions in 3D space, including the boundary conditions created when the renderer is started, along with the boundary conditions given by the geometric objects in the scene.

  Phenomenon is a structure that contains one or more shaders or shader DAGs and various “condition” options that control rendering. In appearance, Phenomenon looks exactly like a shader with input parameters and outputs, but internally, its functionality is not implemented in a programming language, but as a set of shader DAGs with special access to Phenomenon interface parameters. Realized. Additional shaders or shader DAGs can also be included for auxiliary purposes. Phenomenon is attached to a unique root node that serves as an attachment point to the scene. The internal structure is hidden from outside users, but can be accessed by the mental mill Phenomenon creation tool.

  For users who wish to develop shaders by writing code, mental mill uses the mental images' shader language; MetaSL (TM) to create metanodes for integrated development. An environment (IDE) is also provided. Users can develop complete monolithic shaders by writing code or metanodes that provide special features that make them a component of the Phenomenon shader graph.

The mental mill tool also provides an automatically generated graphical user interface (GUI) for Phenomenon and Metanodes. This GUI allows the user to select a value for the parameter and interactively preview the result of the setting.
Before being added to the scene, a parameter value for instantiating the phenomenon must be specified. There are two types of phenomenon that users edit. Phenomenon with no parameter value specified (referred to as free value phenomenon) and Phenomenon with fixed or partially fixed parameters (referred to as fixed phenomenon). When a user creates a new phenomenon by creating a shader graph or by writing MetaSL code (or a combination of both), a new type of free parameter value phenomenon will be created. The user can then create a phenomenon with fixed parameter values based on this new phenomenon type. Typically, there will be a large number of fixed value phenomenons based on the special phenomenon. When a user changes a Phenomenon, all fixed Phenomenon based on it will inherit that change. Changes to fixed Phenomenon are limited to that special Phenomenon only.

  If the user chooses to create a new phenomenon type, the user will have the shader graph built and the mental mill will start by creating a new empty phenomenon. The user can also specify Phenomenon interface parameters that form a public interface to that shader. You can also specify the number of Phenomenon Roots and other options.

Here we describe the mental mill user interface and the features it provides.
The mental mill application UI consists of several different views, each view containing a different set of controls. The view panel is separated by four movable splitter bars. This allows the user to adjust the relative size of the view.

  FIG. 32 is a screenshot 470 showing a basic simplified layout. Although the main view panels are named, the contents of these views are not shown for simplicity. These view panels include: A tool box 472, a phenomenon graph view 474, a code editor view 476, a navigation control 478, a preview 480, and a parameter view 482.

The Phenomenon graph view 474 allows a user to create a new Phenomenon by connecting metanodes or Phenomenon nodes together to form a graph. The output of a node can be connected to one or more inputs, so that the connected node can provide a value for the input parameter to which it is connected.
Phenomenon graph view area 474 may be virtually infinite in size, thereby holding any complex shader graph. The user can navigate this area using the mouse by panning by pressing the middle mouse button and zooming by pressing the right mouse button (button assignment can be changed). The navigation controls described in the following sections provide many ways to control the Phenomenon view.

The user can create a node by dragging the node from the toolbox 472 to the Phenomenon graph view 474 as shown below. Once in the graph view 474, the node can be positioned by the user. The layout command also performs automatic layout of graph nodes.
FIG. 33 shows a graph node 490. The graph node 490 (phenomenon node or metanode) includes several elements.

  Preview—The preview window portion of the node allows the user to see the result of the shader node rendered on the surface. The sphere is the default surface, but other geometric shapes can be specified. All nodes can potentially have a preview window even if they are internal nodes of the shader graph. The preview is generated by considering each node as a complete shader and using that shader to render the sample geometry. This allows the user to see the shader results at each stage of the graph, so that the data flow can be visualized through the shader graph. The node preview portion can also be closed to reduce the size of the node.

Output—Each node has at least one output, but some nodes have more than one output. The user clicks and drags on the output location to add the output to another node's input. An output can be added to two or more inputs.
Input—Each node has no input parameters or one or more input parameters. An input can be added to the output of another node, so that the shader can control the value of the input parameter. Alternatively, the value can be set by the user. An input can be added to only one output. When the user holds the mouse over the input for a moment, a tooltip is displayed that gives a brief description of the parameter. Tooltip text is provided by attributes associated with the shader.

Some of the input or output parameters may be sub-parameter structures. In this case, the input parameter of the node has a “+” or button to open or close the structure. Can be attached to individual elements of the structure. FIG. 34 shows a graph node 500 that includes sub-parameters.
The Phenomenon node itself contains a shader graph. At the top level, users can create multiple Phenomenon nodes, each representing a new shader. The command causes the user to enter the Phenomenon node, which replaces the Phenomenon graph view with a graph that exists inside the Phenomenon. Alternatively, a Phenomenon can be opened directly in the graph where it resides. This allows the user to see the nodes outside the Phenomenon and possibly the nodes connected to it, along with the contents of the Phenomenon itself.

  Inside Phenomenon, the user has access to Phenomenon interface parameters along with Phenomenon output and auxiliary routes. The input (interface parameter) to the Phenomenon appears in the upper left corner of the Phenomenon and operates like an output when viewed from inside the Phenomenon. The Phenomenon output appears in the upper right corner and behaves like an input when viewed from inside.

The shader graph inside a Phenomenon can also contain other Phenomenon nodes. The user can enter these Phenomenon nodes in the same way and repeat the process as long as Phenomenon is nested.
FIG. 35 shows a sample graph view 510 when inside the Phenomenon. Although the entire graph is shown in this figure, it is common to have a sufficiently large graph that the entire graph cannot be seen at one time unless the user zooms out extremely. Note that in this example, all nodes except one node have their preview windows closed.

When opening a Phenomenon in a graph, it can be maximized to cover the entire graph view, or opened at an appropriate size. When opened at the proper size, the user can see the outer graph of Phenomenon as well as the inner graph, as shown in the graph view 520 of FIG.
Phenomenon can be nested inside other Phenomenon, so it is possible to open Phenomenon inside other open Phenomenon and create a new nested Phenomenon. FIG. 37 shows a graph view 530 showing such a case.

  When the user drags a node to the top level, the user will create a phenomenon with a fixed value based on the type of phenomenon that he has chosen to drag. Top-level fixed Phenomenon nodes have editable parameters, but are not attached to other nodes. A fixed Phenomenon, as mentioned above, is a Phenomenon that is created and inherits any changes to that Phenomenon. A command is also available to convert a fixed Phenomenon to a free value Phenomenon, which allows the user to modify the Phenomenon without affecting other instances.

  When the user drags a node into the Phenomenon, a fixed value Phenomenon or metanode is created inside the Phenomenon, depending on the type of node created. A node inside the Phenomenon can be connected to other nodes or Phenomenon interface parameters by wire. If the node that the user dragged into the Phenomenon is itself a Phenomenon node, a fixed value Phenomenon is created. The parameter value can be set or appended to another node, but since it is a fixed phenomenon that is the original phenomenon, the user cannot get inside the phenomenon node and change it. Also, any changes to the original Phenomenon will affect the node. If the user wants to change the phenomenon, a command is available that converts the node to a new free value phenomenon that the user can enter and modify.

To create a shader addition, the user clicks and drags on the output area of one node to place the mouse cursor on the input of another node. When the user releases the mouse, a connection line representing the shader connection is drawn. If the connection is not valid, the cursor will indicate this to the user when the mouse hovers over possible input during the add process.
A type checking system ensures that a shader is attached only to inputs that match its output type. In some cases, if there is an adapter shader that handles the conversion, two parameters of different types can be added. For example, a scalar value can be added to a color input using an adapter shader. The adapter shader can convert the scalar to gray, or can perform other conversions depending on the settings selected by the user. The adapter shader is automatically inserted when it is available. When a user adds a parameter that requires an adapter, the adapter is automatically inserted when the user completes the addition. In addition, the mental mill ensures that the user does not inadvertently create cycles in the graph when making an addition.

  FIG. 38 shows a view 540 showing the result of adding a color output to a scalar input. A “color to scalar” adapter shader node is inserted between them to perform the conversion. The conversion type parameter of the adapter node allows the user to select how colors are converted to scalars. Some options for this parameter are:

Average-averages the red, green and blue components.
NTSC weighted luminance—A weighted average of red, green and blue.
Select component—takes only one of the red, green, blue, or alpha component colors.

Both nodes and connecting lines can be selected and deleted. When deleting a node, all connections to that node are also deleted. When deleting a connection, only the connection itself is deleted.
As the shader graph becomes more complex, the user can construct the graph by packing the portion of the graph into Phenomenon nodes. A command is available that takes the currently selected subgraph and converts it to a Phenomenon node. The result is a new phenomenon with interface parameters for each input of the selected node added to the unselected node. This Phenomenon has an output for each selected node whose output is added to an unselected node. New Phenomenon is added in place of the old subgraph moved within Phenomenon. The result is that the behavior of the shader graph is unchanged, but the graph appears to be simplified because several nodes have been replaced by a single node. The ability of Phenomenon to encapsulate complex operations into a single node is an important and powerful feature of mental mills.

  FIG. 39 shows a view 550 where shaders 2, 3 and 4 are packed into a new Phenomenon node. Connections to external nodes are maintained and the results provided by the graph are unchanged. However, the graph is slightly organized. Since Phenomenon can be nested, this type of grouping of subgraphs into Phenomenon can occur at any depth level.

The preview window displays a sample rendering of the currently selected phenomenon. The image is the same as the image shown in the previous window of the Phenomenon node, but can be enlarged to show more details.
The preview always shows the result of the topmost phenomenon node. Once the user enters a phenomenon, the preview shows the result of that phenomenon, regardless of which node is selected. This allows the user to work on the shader graph in Phenomenon while still previewing the final result.

The navigation window provides controls that allow the user to navigate the Phenomenon graph. The button allows the user to zoom to fit a selected portion of the graph within the Phenomenon graph or to fit the entire graph so that it can be viewed.
The “bird's eye” control shows a small representation of the entire shader graph, with a rectangle showing the portion of the graph shown in the Phenomenon graph view. The user can click and drag on this control to position the rectangle at the part of the graph that the user wants to see. There are also “zoom in” and “zoom out” buttons and a slider that allows the user to control the size of the view rectangle. As the user changes the zoom level, the rectangle becomes larger or smaller. Conformal views are also considered.

FIG. 40 is a view 560 showing a bird's eye view control viewing a sample shader graph. The dark gray rectangle 562 shows the area that can be seen in the Phenomenon graph view.
As the user zooms in and out, the shader node in the graph view becomes larger or smaller. As the user zooms out and the node gets smaller, some elements of the node disappear and simplify the node. 41A-D show a series of views 570, 572, 574, and 576 showing the detailed node level progression.

If the input or output collapses, the user can still make additions. When dragging an attachment to a node, a pop-up list allows the user to select the actual input by releasing the mouse.
As described in the Phenomenon Graph section, the user can enter a Phenomenon node, which replaces the graph view with the entered Phenomenon graph. This process can continue as long as the Phenomenon is nested within other Phenomenon. The navigation window provides back and forward buttons, which allow the user to follow the path that has been navigated through the nested phenomenon.

The toolbox window includes a shader node that creates a building block from which the shader graph is constructed. The user can click and drag a node from the toolbox to the Phenomenon graph view to add the node to the shader graph.
Nodes appear as icons and names in the toolbox. Typically, the icon is a sphere rendered using a shader, but other icons may be used. The list of nodes can also be seen in a tight list without icons, which allows more nodes to fit in the view. Some nodes may be Phenomenon nodes, i.e. nodes defined by shader graphs, and other nodes may be metanodes, i.e. nodes defined by MetaSL code. This is often not important to the user because both types of nodes can be used interchangeably. Phenomenon nodes are colored differently than metanodes, or colored so that they can be visually distinguished otherwise, so that the user can distinguish the two. A Phenomenon node can be edited as a graph by editing its shader graph, but a metanode can only be edited by changing its MetaSL source code.

  The nodes are sorted by category and the user can choose to see a single category or all categories by selecting from the category drop-down list. Some commands result in dialogs that allow the user to configure that category. In this dialog, the user can control to create or delete categories and assign categories to each node type.

  In addition to Phenomenon or Metanodes, toolboxes also contain “actions”. An action is a fragment of a complete shader graph that can be used by the user when building a new shader graph. It is normal for shader nodes and additional patterns to appear in different shaders. The user can select a portion of the shader graph and use it to create a new action. In the future, when the user wants to create nodes with the same configuration, the user can simply drag actions from the toolbox to the shader graph to create those nodes.

42A-B are views 580 and 582, showing the toolbox in two different view modes. The view 580 in FIG. 42A shows the thumbnail view mode, and the view 582 in FIG. 38B shows the list view mode.
The toolbox is packed with nodes defined in the specified shader description file. A user can select one or more shader description files to use as a shader library that can be accessed via the toolbox. There are commands to add node types to this library and to remove nodes. The mental mill tool also provides an initial library of metanodes.

  FIG. 43 shows a partial screenshot 590 illustrating some controls in the parameter view. The parameter view displays controls that allow you to edit the parameters of the selected node. These controls include sliders, color pickers, checkboxes, drop-down lists, text editing fields, and file pickers, to name a few.

When editing a free value Phenomenon interface parameter, what is being edited is the default value. Fixed value Phenomenon overrides those parameter values.
If shader addition is possible, there is a button row that allows the user to retrieve the addition source from within the parameter view. Note that at this stage, adding shaders is not possible when editing top-level phenomena. This button displays a pop-up list that allows the user to retrieve or select a new node from other nodes available in the current graph. The “none” option is provided to remove the addition. When an attachment is made, the normal control over the parameter is replaced with a label indicating the name of the attached node.

Some parameters are structures, in which case controls are created for each element of the structure. According to a further form, the structure is displayed with a “collapsed” shape UI and opened with a + button to open it (repeatedly for nested structures). Alternatively, the structure can always be displayed in an expanded shape.
In the UI, parameters appear in the order in which they are declared in a Phenomenon or Metanode, but attributes in the node can also control the order and grouping of parameters. When editing default parameters for free-valued Phenomenon, commands that group parameters together can be used with commands that change the order of parameters. These commands edit attributes associated with the node.

  Most parameters have hard or soft limitations associated with them. The hard limit is the range that the parameter cannot exceed. A soft restriction is to specify a range for generally valid parameters, but parameters are not strictly limited to that range. The range of the slider control is set to a soft limit of parameters. Commands in the parameter view allow the user to change the slider range beyond the soft limit unless it exceeds the hard limit.

  Controls in the parameter view display a tooltip when the user pauses the mouse for a moment. The text displayed in the tooltip is a brief description of the parameters due to the attributes associated with the shader. Similarly, the buttons at the top of all controls display a relatively short description of the overall shader function. This description is also due to the attributes associated with the node.

  FIG. 44 shows a partial screenshot 600 illustrating a code editor view according to this aspect of the invention. The code editor view allows users who want to create shaders by writing code to do so using MetaSL. Users can create monolithic shaders by writing code if necessary, but users are likely to create new metanodes that are intended to be used as part of the shader graph.

The command allows the user to create a new metanode. The result is a metanode at the top level of a graph (not in Phenomenon) that represents a new type of metanode. The user can create an instance of this new metanode in the Phenomenon whenever desired.
When the user creates a new metanode, the mental mill creates the initial skeleton code for the minimally compilable shader. Users can then immediately focus on implementing the special features that the shader is supposed to provide.

  When a top-level metanode (outside of Phenomenon) is selected, the corresponding MetaSL code appears in the code editor view for user editing. After making changes to the code, a command to compile the shader is available. The MetaSL and C ++ compilers for your specific platform are invoked by a mental mill that compiles shaders. Cross compilation for other platforms is also possible. Any errors are returned to the mental mill so that it displays it to the user. Clicking on the error leads the user to the corresponding line in the editor. If the shader compiles successfully, the metanode will update to show the current set of input parameters and outputs. The preview window also updates to give visual feedback on the shader appearance. This integration of MetaSL and C ++ compilers greatly simplifies the development of metanodes and monolithic shaders.

The parameter view also displays controls for the selected top-level metanode, allowing the user to edit default values for the node's input parameters. This is similar to the editing of interface parameters for free value Phenomenon.
Many users, especially those who are not technically savvy, are not interested in writing code and would only want to use the built-in Phenomenon graph editing method. The UI allows the code editing window to be closed, leaving extra space for the Phenomenon graph view.

The main menu can be configured in a variety of ways, including:
• File-The file menu includes the following items:
1. New Phenomenon-This command is used to create a new free value phenomenon. Once created, other phenomenons with fixed parameter values can be created based on this phenomenon.

2. New Metanode-Create a new metanode type. A top-level metanode is created when selected and its code is editable in the code editor.
3. Open File-Opens a shader description file for editing. The contents of the file appear in the Phenomenon graph view. This can include free or fixed phenomenon with metanode types. You can open and edit files specified as part of the toolbox. Editing a Phenomenon shader graph affects all fixed value Phenomenon based on that Phenomenon. Thus, opening a toolbox file is quite different from dragging a phenomenon or metanode from the toolbox to the phenomenon graph view. Dragging a node from the toolbox creates a fixed value phenomenon that can be modified without affecting the original phenomenon. When you open the description file used by the toolbox, you can modify the original phenomenon.

4. Save-Save the currently open file. If the file has never been saved, this command prompts the user to name the new file.
5. Save as-saves the currently open file, but allows the user to give the file a new name.

6. Exit—Exits the mental mill application.
Edit-Edit menu contains the following items:
1. Undo-Undo the last change. This includes changes to the shader graph, changes to parameter values, or changes to shader code created within the Code Editor view. The mental mill tool has an unlimited level that virtually undoes. The user can set the maximum number of levels to undo by preference, but since this application uses less memory, the number of levels to undo can remain very high.

2. Redo—Repeat the last change that was previously undone.
3. Cut-deletes the selected item, but copies it to the clipboard. The item selected includes the text in the node (or nodes) or code editor view. If the keyboard focus is in the code view, the text will be cut, otherwise the shader graph selection will be cut. This also applies to copy, paste, delete, select all, select none, select invert.

4. Copy-Copies the selected item to the clipboard.
5. Paste-Pastes the contents of the clipboard into the shader graph or code view.
6. Delete-Deletes the selected item.
7. Select All-Select all items. Text in shader node or code editor, depending on keyboard focus.

8. Do not select anything-clear the selection.
9. Select Invert-Select all items that are not selected and deselect all selected items.
10. Preferences—provides a dialog that allows the user to set preferences for the mental mill application.

Help-Help menu includes the following items: 1. Mental Mill Help-Brings up the mental mill help file.
2. Product Information (About)-Brings a product information box.
The toolbar includes tool buttons that allow easy access to common commands. In general, toolbar commands work with shader graph selection. Some of the toolbar commands are duplicates of the commands in the main menu. The list of toolbar items includes the following commands: Open file, Save file, New shader graph (Phenomenon), New shader code-based, Undo, Repeat, Copy, Paste, Close ).

  An important aspect of shader creation is the ability to analyze defects, determine their cause, and find solutions. In other words, the shader creator must be able to debug the shader. Finding and resolving defects in shaders can be created whether shaders are created by adding metanodes to form a graph, created by writing MetaSL code, or both. It is necessary regardless of that. A mental mill provides the ability for a user to debug a user's shader using a high level of visual technology. This allows shader creators to visually analyze the state of the shader and quickly isolate the cause of the problem.

  This form of the invention provides a structure for debugging Phenomenon. The mental mill GUI allows the user to build a phenomenon and form a graph by adding metanodes or other phenomena. Each metanode has a UI representation that includes a preview image that describes the results produced by that node. When viewed as a whole, this network of images provides an illustration of the process that shaders use to calculate their results. FIG. 45 shows a partial screenshot 610 that illustrates this aspect of the invention. The first metanode 612 calculates the illumination on the surface, and another metanode 614 calculates the texture pattern. The third node 616 combines the first two results and creates the result.

  By visually looking around the metanode network, the shader creator can examine its shading algorithm and find out where the results are not what they expected. A node in a network may be a Phenomenon, in which case it contains its own one or more networks. The mental mill allows the user to navigate to the Phenomenon node and visually examine its shader graph using the same technique.

  In some cases, examining the results for each node in Phenomenon may not provide enough information for the user to analyze the shader problem. For example, all inputs to a particular metanode appear to have the correct value, but the result of that metanode may still not appear as expected by the user. Also, when authoring a new metanode by writing MetaSL code, the user may wish to analyze the value of a variable in the metanode when the metanode calculates the resulting value.

  Traditional models for general program debugging are not fully adapted to shader debugging. A typical program that runs on a CPU is essentially linear in nature, and a series of commands that manipulate data are executed. Some programs have multiple threads of execution, but usually have a relatively smaller number of threads, often 12 or less, and each thread represents a separate, independent sequence of instructions.

  A shader program, on the other hand, represents a linear continuous command, but differs in that it operates on multiple data points in parallel. While the shader program executes a continuous list of commands, it appears to be doing it for each data point simultaneously. In some cases, shaders are processed using a SIMD (single command multiple data) model, in which case each command is actually applied to multiple data points simultaneously.

Traditional models that step through the program and examine the value of a variable can have each variable having many different values that span the data points at any particular stage of the shader's execution. It must be corrected to take into account. This makes the traditional way of examining individual values of variables much impractical.
The mental mill extends the visual debugging paradigm to the MetaSL code behind each metanode. The mental mill MetaSL debugger provides the user with a source code listing containing MetaSL code for the shader node in question. The user can then proceed with the shader's command and the value of the variable can be examined as the user changes through the execution of the program. However, instead of just providing the user with a single numeric value, the debugger displays multiple values simultaneously as a mapped color on the surface of the object.

  Representing the value of a variable as an image rather than as a single number has several advantages. First, the user can immediately recognize the feature of the function that drives the value of the variable and find out the area that is working incorrectly. For example, the rate of change of a variable on the surface can be viewed in an intuitive manner by observing how the color is changing on the surface. As long as the user is using the traditional method of debugging shaders one pixel at a time, this is difficult to recognize. Defects in the shader are often due to discontinuities in the function behind the shader. By observing the rate of change of the shader, the user can isolate such discontinuities.

  Users can also use visual debugging paradigms to quickly locate input conditions that produce undesirable results. Shader bugs may only appear when certain input parameters take special values, and such situations may only appear in special parts of the geometric surface. The mental mill debugger allows the user to navigate through the three-dimensional space using the mouse to find the position on the surface that is an indication of the problem and direct his gaze there.

  Mental mill MetaSL debugging provides the user with a list of all variables in scope in the selected statement. FIG. 46 shows a partial screenshot 620 illustrating a variable list according to this aspect of the invention. If the user selects a different statement, new variables enter the scope and appear in the list, while other variables exit the scope and are removed from the list. Each variable in the list has a button next to its name, which allows the user to open the variable and display additional information such as its type and its value on the surface. Can see. If the user passes a statement that modifies a variable, the preview image for that variable is updated to reflect the modification.

In addition, the user can select a variable from the list and display its value in a larger preview window.
Traditional debugging techniques allow the user to step through the program and examine the state of the program at each statement. Typically, the user can only go one or more lines in the direction of execution of the program. In general, jumping to any statement requires a program restart.

  The mental mill MetaSL debugger allows the user to jump to any statement in the shader code in any order. A particularly nice aspect of this feature is that when a code statement modifies the value of a variable of interest, the shader writer can easily return or go ahead from this statement, before and after the statement is executed. The value can be switched. This makes it easy for the user to analyze the effect of any special statement on the value of the variable.

Loops (like for, for each, while loops) and conditional statements (like if and else) create interesting situations within this debug model. Since shader programs operate on multiple data points simultaneously, the statements in the if / else statement may or may not be executed for each data point.
MetaSL debugging provides the user with several options for viewing the value of a variable in a conditional statement. The problem is how to handle data points that do not execute if or else statements that contain the selected statement. These optional modes include the following:

Show final shader result-In this mode, data points that do not reach the selected statement are processed by the complete shader and the final result is generated in the output image.
Show constant color-In this mode, the constant color replaces the final result for data points that do not reach the selected statement. This allows the user to easily identify which data points are and will not be processed by the selected statement. This is an effective debugging tool by itself and by itself.

Discard—In this mode, if the selected statement does not reach a particular data point, the shader is discarded only for that data point and the output image is not modified. This essentially removes the portion of the surface that contains the data points for which the selected statement was not reached.
Similarly, the loop can be executed different times for each data point. In addition, the user can arbitrarily jump to any statement in the shader program, so when selecting a statement in the loop, the user must specify which iteration of the loop is considered by the user. Given that the loop may be executed for each data point a different number of times, some data points may have already exited the loop before reaching the desired number of iterations.

  The mental mill debugger allows the user to specify a loop count value that specifies the desired number of iterations through the loop. The loop counter can be set to any value greater than zero. The higher the value, the more data points will not reach the selected statement, and in fact, given a sufficiently large value, no data points will reach the selected statement. Options that control the situation where the selected statement is not reached also apply to the loop.

  It is clear that expressing the value of a variable as a color mapped on the surface of the object works well if the variable is of color type. This method also works fairly well for scalar and vector values of 3 components or less, but the values must be mapped to the 0-1 range to produce a regular color. The mental mill UI allows the user to specify ranges for scalars and vectors that are used to map those values to colors. Alternatively, the mental mill can automatically calculate the range for that viewpoint by determining the minimum and maximum values of the variables on the surface as viewed from a given viewpoint.

  Mapping scalars and vectors to colors using user-specified ranges is useful, but requires the user to guess the value of the variable by looking at the color of the surface. The mental mill UI provides other techniques for visualizing these data types. For vector types that represent direction (not location), one visualization technique is to use a vector as an arrow located on the surface when the user drags the mouse over the surface, as shown in the illustration below. It is to draw. This visualization technique is illustrated in a series of partial screenshots 630, 632, and 634 shown in FIGS. 47A-C.

The numeric value of the variable is also displayed in the tooltip when the user moves the mouse over the surface of the object.
Matrix type variables impose another challenge on visualization. A 4-by-4 matrix can be viewed as a grid of numbers formatted in standard matrix notation. This visualization technique is illustrated in the partial screenshot 640 shown in FIG.

  A matrix type variable of 3 rows and 3 columns can be thought of as a set of three directional vectors that form the rows of the matrix. A common example of this is the tangent space matrix, which consists of u 'derivatives, v' derivatives, and normals. The three row vectors of the matrix can be drawn as arrows below the mouse pointer, as shown below. Furthermore, the individual values of the matrix can be displayed in a standard matrix layout. This visualization technique is illustrated in the partial screenshot 650 shown in FIG.

  Vector type values that do not represent direction can be viewed using a gauge style display. The same user-specified range that maps these values to colors can be used to set a gauge range or set of gauges that appears when the user selects a position on the surface. As the user moves the mouse over the surface, the gauge illustrates values for the user specified range. This visualization technique is illustrated in the partial screenshot 660 shown in FIG.

When a vector arrow, gauge, or tooltip containing a numeric value is displayed, the user can choose to see the final shader result on the surface of the object instead of the value of the variable mapped to color. This allows the user to know the position of the portion of the surface corresponding to the feature in the final shader result while monitoring the value of the selected variable.
Another useful feature provided by the mental mill debugger is that it locks onto a special pixel location instead of using the mouse pointer to sweep across the surface. The user can select the location of the pixel (depending on whether it is selected with the mouse or given a numeric pixel coordinate) and the value of the variable at that pixel location is displayed as the user proceeds through the statements in the code .

The mental mill shader debugger presents another advantage that is independent of the mental mill platform. A single MetaSL metanode, or an entire Phenomenon, can be created and debugged, and can target multiple platforms.
The debugger can operate in hardware or software mode and functions independently of any special rendering algorithm. The fact that the shader debugger is tightly integrated into the mental mill Phenomenon creation environment further reduces the creation / test cycle, thereby isolating users from platform dependencies at a high level of shader creators. Can continue working.

A prototype application was created as a proof of concept for this shader debug system.
The mental mill application is built on a modular library that contains all the functions used by this application. The API allows third party tools to integrate mental mill technology into their applications.

Future generations of mental mill Ray will have access to several sub-components of the mental mill library, but this is done only to facilitate rendering with mental ray. The complete mental mill library is licensed separately from mental mill ray.
More detailed documentation of the mental mill API will be available in future documents.

When integrating mental mill technology into third-party applications, the mental mill GUI can be customized to match the look and feel of the application. There are several ways to customize the GUI that are possible with the mental mill API.
Phenomenon graph appearance—Phenomenon graph elements such as metanodes and connecting lines can be drawn by invoking a plug-in callback function. A default drawing function is provided, but third parties can also provide their own functions and customize the appearance of the shader graph to better suit their application. The callback function also addresses mouse point hit testing, since the elements of the node can be placed at different locations.

• Keyboard shortcuts-All keyboard commands are reassignable.
• Mouse actions-Mouse actions such as mouse button assignments can be customized.
Toolbar item—Each toolbar item can be omitted or included.
View windows-Each view window is designed to run on its own without depending on other windows. This allows the third party to integrate, for example, only the Phenomenon graph view into the application. Since each view window can be driven by an API, a third party can include any combination of view windows, and some of the view windows can be replaced with its own user interface.

C. MetaSL Design Specification Mental Mill ™ Shading Language (MetaSL ™) provides a powerful interface to achieve custom shading effects and abstraction from any special platform on which the shader is executed Function as.

  Since the rendering can be performed on the CPU or on the graphics processor unit (GPU), the shader itself runs on the CPU or GPU. There is a big difference between the graphics hardware and the API functions that drive it. Thus, shaders written in a language that directly targets graphics hardware will likely have to be rewritten as hardware and APIs change. Furthermore, such shaders do not work in software renderers and do not support features currently only available for software rendering, such as ray tracing.

  MetaSL solves this problem by maintaining independence from any target platform. A shader can be written once in MetaSL, and the MetaSL compiler automatically converts it to any supported platform. As new platforms emerge and graphics hardware functionality changes, shaders written in MetaSL will automatically benefit from the new functionality without the need for rewriting.

  This isolation from the target platform allows the MetaSL shader to operate automatically in software or hardware rendering mode. The same shader can be used to render a software mode on a machine and a hardware mode on another machine. Another use of this feature is the automatic repurpose of shaders for different situations. For example, a shader created for use for visual effects in a movie can also be used for video games based on that movie.

  In many cases, the same MetaSL shader can be used whether rendering is performed in software or hardware and the user is required or not to execute multiple shaders. In some cases, the user may want to customize the shader for the GPU or CPU. The language does this and still provides a mechanism to implement both technologies in MetaSL.

  The hardware shader generated by the MetaSL compiler is only used for rendering by the next generation of Reality Server (R) based on Mental Ray (R) and Neuray (R) There is a restriction that you can't. Mental Mill ™ Phenomenon ™ creation technology provides the ability to generate hardware shaders that can be used externally by mental ray rendering or other applications such as games.

The next section describes the MetaSL language specification. MetaSL is designed to be easy to use, focusing on the programming elements necessary for a common shading algorithm, rather than the wide, sometimes difficult-to-understand features found in common programming languages.
All elements of MetaSL shaders are grouped into shader classes indicated by shader keywords.

shader my_shader {
// Contents of the shader are found here
};
A single source file can have multiple shader definitions. A shader class can also inherit the definition of another shader by writing the name of the parent shader that follows the name of the child shader and is separated by a colon.

shader my_parent_shader {
// ...
};
shader my_shader: my_parent_shader {
// ...
};
As a result, various shader type variants are executed while sharing a shader portion common to all the variants. A shader definition can only be inherited from a single parent shader.

A shader can have zero or more input parameters that the shader uses to determine the resulting value. The parameter can be any built-in type as described below, or it can also be a custom structure type as described below.
When used in a scene, an input parameter may store a character value, or may be appended to the result of another shader, but the shader need not be related to this possibility. A shader can reference an input parameter just as it references any other variable. However, note that input parameters can only be read, not written.

A shader declares its input parameters in the section indicated in the input. A label followed by a declaration for each parameter.
shader my_shader {
input:
Color cO;
Color cl;
};
This example declares a shader with two color input parameters.

  Input parameters may also be fixed or variable sized arrays. Since the size of the dynamic array cannot be known in advance, the array input parameter includes a built-in count parameter. The length of the array named “my array” can be referred to as my_array.count. Fixed length input parameters, unlike dynamic arrays, have a length indicated as part of the declaration.

shader my_shader {
input:
int n [4]; // Fixed size array (count = 4)
int m []; // Variable sized array
};
Arrays of arrays are not supported as input parameter types.

A shader must have at least one output parameter, but may have more than one. The output parameter stores the shader result. The purpose of the shader is to calculate a function with that input parameter and store the result of that function in the output parameter.
The output parameter can be of any type and includes a user-defined structure type. However, it cannot contain dynamic arrays.

A shader declares its output parameters in the section indicated by the output. A label followed by a declaration for each parameter.
shader my_shader {
output:
Color ambient_light;
Color direct_light;
};
This example declares a shader with two color outputs. Many shaders have only a single output parameter, customarily named “result”.

  A shader can declare other variables that are not input or output parameters, but instead store other values that are read by the shader. When initializing before rendering begins, the shader can calculate a value and store it in a member variable. Member variables are designed to hold values that are calculated once before rendering begins, but do not change thereafter. This avoids the redundancy of calculating the value each time the shader is called.

Member variables are declared in the section indicated by the member; keyword. For example,
shader my_shader {
input:
Scalar amount;
output:
Color result;
member:
Scalar noise_table [1024];
};
The main task of a shader is to calculate one or more result values. This is done in the shader class by a method called “main”. This method is called when a renderer needs the result of a shader, or when a shader is appended to another shader's input parameters and the shader needs a parameter value.

The return type of this method is always void, that is, it does not return a value. Instead, the shader result must be placed in its output parameter.
shader mix_colors {
input:
Color cO;
Color cl;
Scalar mix;
output:
Color result;
void main () {
result = co * (1.0-mix) + cl * mix;
}
};
This example shader mixes two colors based on a third mix parameter.

The main method is executed as an inline method that is convenient for simple shaders. If the method is large, it is desirable to execute it separately from the shader definition. To do this, the name of the method must be prepended to the shader name, separated by two colons. For example,
void mix_colors :: main () {
result = co * (1.0-mix) + cl * mix;
}
In addition to the main method, other methods that function as helper methods for the main method can also be defined in the shader class. These methods can return any type of value and allow calls to any type of parameter. The method declaration is placed in the shader class, as in the following example:

Vector3 average_normals (Vector3 norml, Vector3 norm2);
As with the main method, the helper method is executed either directly in the shader class definition or outside of it. When executed outside a shader class definition, the name of the method must precede the name of the shader, followed by two colons.

  MetaSL allows you to define functions that are not directly related to special shader classes, such as methods. Functions are declared like methods, except that the declaration appears outside of any shader class. A function must have a declaration that appears before the reference to the function. The function body can be included as part of the declaration, or the separate function declaration may be later in the source file after the function declaration.

  Both methods and functions can be overloaded by defining another method or function that has the same name but a different set of call parameters. An overloaded version of a function or method must have a different number of parameters and / or different types of parameters. It is not enough for overloaded functions to differ only in return type.

Parameters to functions and methods are always passed by value. A call parameter declaration can be modified by one of the following modifiers, allowing a function or method to modify the call parameter and making the modification visible to the caller.
In—The parameter value is copied to the function being called, but cannot be copied.

Out—The parameter value is not copied to the called function and is not defined when read by the called function. However, the called function can set the parameter value and the result is copied back into the variable passed by the caller.
Inout—The parameter value is copied to the function being called and copied to the variable passed by the caller.

Note that according to this form of the invention, functions and shader methods cannot be called recursively.
Another specially named method in the shader class is a method called “event”. This method is called to allow the shader to perform a task before and after the shading operation. These are sometimes referred to as “init” or “exit” functions.

The event method is passed with a single Event_type parameter that identifies the event. FIG. 51 shows a table 670 describing a list of Event_type parameters according to this aspect of the invention.
All init / exit events have access to threadlocal variables except Module_init / exit and Class_init / exit.

shader my_shader {
input:
Color c;
output:
Color result;
member:
Scalar noise_table [1024];
void main ();
void event (Event_type event) {
if (event == Instance_init) {
for (int i = 0; i <1024; i ++)
noise_table [i] = Random ();
}
}
};
In this example, the Instance init event is processed and the noise table array is initialized to contain random values.

MetaSL includes an extensive set of basic types. These built-in types cover most of the needs that shaders have, but can also be used to define custom structures, as described below.
The following is a list of MetaSL specific types.
Int-single integer value Bool-Boolean value (true or false)
Scalar-floating point value with unspecified precision. This type maps to the highest possible accuracy of the target platform.

-Vector2-vector with two scalar components-Vector3-vector with three scalar components-Vector4-vector with four scalar components-Vector2i-vector with two integer components-Vector3i-vector with three integer components -Vector4i-vector with 4 integer components-Vector2b-vector with 2 Boolean components-Vector3b-vector with 3 Boolean components-Vector4b-vector with 4 Boolean components-Matrix2x2-2 by 2 matrix Matrix3x2-3 rows and 2 columns matrix-Matrix2x3-2 rows and 3 columns matrix-Matrix3x3-3 rows and 3 columns matrix-Matrix4x2-4 rows and 2 columns matrix-Matrix2x4-2 rows and 4 columns matrix-Matrix4x3-4 rows 3 Matrix of columns-Matrix3x4-3 rows by 4 columns-Matrix4x4-4 rows by 4 Color-r, g, b, and scalar component colors Texture1d-one-dimensional texture Texture2d-two-dimensional texture Texture3d-three-dimensional texture Texture cube-cube map texture Shader-reference to shader-String A string that supports the − == and + operators
MetaSL provides functions that define enumerations as a convenient way to represent a set of named integer constants. The keyword “enum” is used before the comma separated list of parenthesized identifiers. For example,
enum {LOW, MEDIUM, HIGH};
An enumeration can also be named, in which case it defines a new type. An enumerator can also be explicitly assigned a value. For example,
enum Detail {
LOW = 1,
MEDIUM = 2,
HIGH = 3
};
This example defines a new type called Detail with possible values LOW, MEDIUM, and HIGH. An enumeration type value is implicitly assigned to an integer, resulting in an integer having an explicitly assigned value. If no explicit value is specified, the value is assigned to the first enumerator starting from zero and then incremented by one.

All vector types have components that can be accessed in a similar manner. Components can be referenced by appending a period to the variable name followed by one of [x, y, z, w]. A vector of length 2 can have only x and y components, a vector of length 3 has x, y, and z, and a vector of length 4 has all four components.
When referring to a vector component, multiple components can be accessed simultaneously, but the result is another vector of the same or different length. The order of the components can also be arbitrary, and the same component can be used more than once. For example, for a vector V of length 3,
・ V.xy returns a two-component vector <x, y> ・ V.zyx returns a three-component vector <z, y, x> ・ V.xxyy returns a four-component vector <x, x, y, y> The same syntax can be used as a mask when writing to the return vector. The difference is that the left component of the assignment cannot be repeated.

V.yz = Vector2 (0.0, 0.0);
In this example, the y and z components are set to 0.0 and the x and y components are not changed.
Vector components can also be accessed using an array index, which can be a variable.
Scalar sum = 0.0;
for (int i = 0; i <4; i ++)
sum + = V [i];
In this example, Vector4 V has components summed using a loop.

A vector can be constructed directly from another vector (or color) if the total number of elements is greater than or equal to the number of elements in the vector being constructed. Elements are taken from the constructor parameters in the order in which they are listed.
Vector4 v4 (1.0, 2.0, 3.0, 4.0);
Vector2 v2 (v4);
Vector3 v3 (0.0, v2);
The above three vector constructor calls are all appropriate. This example results in three vectors initialized with the values listed in Table 680 shown in FIG.

  Standard mathematical operators (+,-, *, /) apply to all vectors and operate on components. Standard mathematical operators are overloaded to allow a mixture of differently sized scalars and vectors in an expression, but in any single expression all vectors must have the same size. When a scalar is mixed with a vector, the scalar is promoted to a vector of the same size, with each element set to a scalar value.

Vector2 vl (a, b);
Vector2 v2 (c, d);
Vector2 result '= (vl + v2) * e;
In this example, the variables a, b, c, d, and e are all scalar values previously declared. The result of the variable has the values <(a + c) * e, (b + d) * e>.

Standard Boolean logic operators can also be applied to Boolean individual or vectors. When applied to a Boolean vector, it operates on the components and yields a vector result. These operators are listed in the table 690 shown in FIG.
Bit operators are not supported. Comparison operators are supported and perform operations on the components on the vectors, yielding a Boolean vector result. A list of supported comparison operators is set forth in table 700 shown in FIG.

Ternary conditional operators can also be applied to vector operands in terms of components. The conditional operand must be a single Boolean expression or a vector Boolean expression with the same number of components as the components of the second and third operands.
Vector2 vl (1.0, 2.0);
Vector2 v2 (2.0, 1.0);
Vector2 result = vl <v2? Vector2 (3.0, 4.0): Vector2 (5.0, 6.0);
In this example, the result variable holds the value <3.0, 6.0>.

An instance of the Color type is identical in structure to that of Vector4, but its members are referenced by [r, g, b, a] instead of [x, y, z, w], respectively Means red, green, blue, and alpha components.
They can be used wherever Vector4 can be used, and all operations applied to Vector4 work equally well with Color. The main purpose of this type is code readability. Otherwise, this type is logically synonymous with Vector4.

  The matrix is defined by rows and columns with sizes ranging from 2 to 4. All matrices are composed of Scalar type elements. Matrix elements can also be referenced using an array notation (row priority order) in which the array index selects a row from the matrix. The resulting row depends on the size of the original matrix and is Vector2, Vector3, or Vector4. The result of matrix indexing is a vector, and the vector type also supports index operators, so that the individual elements of the matrix can be accessed with a syntax similar to that for multidimensional arrays.

Matrix4x3 mat (
1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.5, 0.0);
Vector4 row;
Scalar element;
// row will equal <0.0, 1.0, 0.0> after the assignment
row = mat [I];
// element will equal 0.5 after the assignment
element = mat [3] [1]
As shown by this example, the matrix also has a constructor that accepts the values of the elements in row-first order. A matrix can also be constructed by specifying a row vector, as in the example below.

Vector3 row0 (1.0, 0.0, 0.0);
Vector3 row1 (0.0, 1.0, 0.0);
Vector3 row2 (0.0, 0.0, 1.0);
Vector3 row3 (0.0, 0.0, 0.0);
Matrix4x3 mat (row0, row1, row2, row3);
The number of vector elements passed to the matrix constructor must match the number of elements in the matrix row being constructed.

  The multiplication operator is supported for two matrix or matrix-vector multiplications and performs a linear algebra style multiplication between the two. As expected when multiplying two matrices, the number of columns in the left matrix must be equal to the number of rows in the right matrix. The result of multiplying the N × T matrix by the T × M matrix is an N × M matrix. A vector can be multiplied from either the right side or the left side if the number of elements is equal to the number of rows when the vector is on the left side of the matrix and if the number of elements is equal to the number of columns when the vector is on the right side.

Automatic type conversion allows one type of variable to be assigned to another type of variable. The only limitation is that when implicitly transforming a vector, the transform is for vectors of the same size. To convert between vectors of different sizes or between scalars, you can use constructors or use the .xyzw notation. For example,
Vector3 v3 (0,0,0);
Vector2 v2 = Vector2 (v3.x, v3.y);
or:
Vector3 v3 (0,0,0);
Vector2 v2 = v3.xy;
The .xyzw notation can be applied to Scalar type variables to generate vectors. For example,
Scalar s = 0.0;
Vector3 v3 = s.xxx;
In this situation, only x elements are valid for a scalar that is considered to be one element vector.

Conversions that can result in a loss of precision, such as converting a scalar to an integer, are possible, but a compiler warning is issued. This warning can be disabled if desired by the shader lighter.
MetaSL supports both built-in and user-defined structure arrays, but only fixed-length arrays can be declared in shader functions. There are two exceptions. As mentioned in the Input Parameters section, shader inputs can be declared as an array with a dynamic size. Another exception is a parameter to a function or method, which may be an unspecified size array. In both cases, during rendering, the actual size of the array variable is known by the time the shader is activated. The shader code can refer to the size of the array as name.count, where name is the name of the array variable.

Scalar sum_array (Scalar values []) {
Scalar sum = 0.0;
for (int i = 0; i <values.count; i ++)
sum + = values [i];
return sum;
}
Scalar a_function () {
Scalar foo [8]; ... // initialize members of foo
return sum_array (foo);
}
This simple example loops over the array and sums the components. The code for this feature was written without actually knowing the size of the array, but when shading it also knows the size. Either the array variable comes from an array shader parameter or a fixed size array is declared in the call function.

Custom structure types can be defined to extend the set of types used by MetaSL shaders. The structure type definition syntax looks like the example below.
struct Color_pair {
Color a;
Color b;
};
In this example, a new type called Color pair is defined as consisting of two colors.

The structure member variable can be any built-in type, or another user-defined structure type for creating a nested structure. The structural member may be a sequence.
User-defined structures and enumerations are the only form of user-defined types in MetaSL.
Within the shader's main method, special state variables are declared implicitly and are available for reference by the shader code. These variables hold values that describe both the renderer's current state, as well as information about the intersection that resulted in the shader call. For example, normals refer to normals interpolated at intersection points.

These variables are only available inside the shader's main method. When a shader tries to access one of these state variables in a helper method, the variable must be explicitly passed to that method.
This set of state variables appears to all shaders as an implicit input. The state input data type is a struct that contains all the state variables available to the shader. Special state shaders can connect to this implicit input. The state shader has an input for each state member and outputs its state struct.

  State data members cannot be modified directly, but disclosing the state as an input refers to a state variable while allowing one shader to drive the state value used by that shader. It becomes possible to do. If the shader state input is left unattached, any reference to the state variable from within the shader will revert to a reference to the unmodified state value.

  For example, a shader that calculates illumination may reference a vertical surface at the intersection point. A bump map shader can create modified normals that it computes from a combination of state normals and perturbations derived from grayscale images. A state shader can thus be added to an illumination shader that exposes the normal as input. The bump shader output can then be added to the normal input of the state shader.

The lighting shader includes a lighting loop that repeats over the lighting of the scene, most likely indirectly causing the lighting shader to be evaluated. The state value passed to the lighting shader is the same as the state value provided to the surface shader. If the state is modified by the state shader, the modification also affects the lighting shader.
This system of implicit state input parameters simplifies shader writing. A shader can easily reference a state variable while maintaining the possibility of adding another shader to modify the state variable at the same time. The state itself is not actually corrected, so there is no risk of adversely affecting another shader.

FIG. 55 shows a schematic diagram 710 illustrating bump mapping according to this aspect of the invention. At first, this may seem a bit complicated, but the graph that performs the bump mapping can be packed inside the Phenomenon node and looks like a single shader.
The “Perturb normal” shader uses three samples of a grayscale image to create a turbulence amount. The texture coordinates used to sample the bump map texture are offset in both the U and V directions, making it possible to calculate the gradient of the grayscale image in the U and V directions.

The “amount” input enlarges / reduces the amount of disturbance. The “Perturb Normal” shader adds this perturbation to the state normal and creates a new modified normal.
Three “Texture Lookup” shaders drive the inputs of the “Perturb Normal” shader. Two of these shaders are supplied with modified texture coordinates from the attached state shader. The state shader is itself supplied with modified texture coordinates created by the “Offset coordinates” shader.

  The entire schematic is contained in Phenomenon, so not all users need to be interested in this detail. The Bump Map Phenomenon has a Texture2d type input that feeds all three texture lookups. The “Offset” input allows the user to control the offset in the U and V directions with a single parameter. The “Perturb Normal” shader “Quantity” input is also disclosed as an input to the Bump Map Phenomenon.

FIG. 56 shows a diagram 720 illustrating a bump map phenomenon in use. A phong shader implicitly refers to the state normal when it loops over scene lighting. In this case, the von shader input is added to the state shader, and the modified normal created by the bump shader is added to the normal input of the state shader.
The presence of a state shader makes it very clear to the user what is happening behind the scene. Interested users can open the Bump Map Phenomenon and see a graph that visually depicts the Bump Map Algorithm.

The set of state variables includes: Position, normal, origin, direction, distance, texture_coord_n, and screen_position. This list can be replenished to make it more extensive. Note that a preprocessor can be used to replace these long names with a common short name abbreviation (often a single letter).
In an alternative configuration, state variable parameters are added to the node. According to this aspect of the invention, the set of special state variables is implicitly declared in the shader's main method, and the shader code is available for reference. These variables hold values describing both the renderer's current state, as well as information about the intersection that leads to the shader call. For example, the normal variable refers to the interpolated normal at the intersection point.

  According to this form of the invention, these variables are only available inside the shader's main method. If a shader wants access to one of these state variables in a helper method, the variable must be explicitly passed to that method. Alternatively, the state variable itself may be passed to another method, in which case all the state variables are made available to that method.

  This set of state variables can be considered by default as an implicit input to all shaders attached to the state itself. However, one or more input parameters can be dynamically added to the shader instance whose name corresponds to the state variable. In that case, these inputs are overwritten with the state value, allowing connection to the result of another shader without modifying the original shader source code. In addition to modifying state variables with overwritten input parameters, shaders can also modify state variables directly through assignment statements in the MetaSL implementation.

Disclosing a state variable as input allows one shader to reference the state variable, while another shader can drive the state variable used by that shader. If there is no input parameter for a specially referenced state variable, the variable will continue to reference the original state value.
For example, shaders that calculate illumination typically reference surface normals at intersection points. A bump map shader may generate a modified normal by calculating from a combination of state normals and perturbations derived from grayscale images. By adding a parameter called “normal” to an instance of a lighting shader, normal can be disclosed as input only to that particular instance. The output of the bump shader can then be added to the shader's normal input.

  According to a further aspect of the invention, the lighting shader includes a lighting loop that repeats over the illumination of the scene, allowing the lighting shader to be evaluated directly. The state variables passed to the illumination shader will have the same state values as those provided to the surface shader. If the state variable is overwritten by a parameter or modified in the shader, the modification also affects the lighting shader. However, since all input parameters are evaluated before the shader begins execution, it is not possible to modify state variables that affect the shader attached to the input parameters.

This system of implicit state input parameters simplifies shader writing. A shader can easily reference a state variable, while at the same time maintaining the possibility of adding another shader to modify that state variable.
FIG. 57 shows a schematic diagram 730 of the bump map phenomenon 732. As shown in FIG. 57, the graph performing the bump mapping can be packed inside the Phenomenon node and looks like a single shader.

  The “Perturb Normal” shader 734 uses three samples of the grayscale image to create the perturbation amount. The texture coordinates that sample the bump map texture are offset in both the U and V directions, so that the gradient of the grayscale image in the U and V directions can be calculated. The “amount” input scales the perturbation amount. The “Perturb Normal” shader 734 adds this perturbation to the state “normal” to create a new modification “normal”.

Three “Texture lookup” shaders 736 drive the inputs of the “Perturb Normal” shader 004. Two of these shaders are supplied with modified texture coordinates from an added “Offset coord” shader 738.
The entire schematic is contained in Phenomenon 732, so that not all users have to be related in detail. This Bump Map Phenomenon has a Texture2d type input supplied to three “Texture lookups”. The “offset” input allows the user to control the offset in the U and V directions with a single parameter. The “Perturb Normal” shader “Quantity” input is also disclosed as input to the Bump Map Phenomenon.

  FIG. 58 shows a diagram of a bump map phenomenon 740 in use. The von Shader 742 implicitly references the state normal when it loops over the scene lighting. The illustration shows an instance of a von shader with an added “normal” input, which allows normal to be added to the output of the bump map shader. FIGS. 59A-B show a table 750 listing a complete set of state variables.

The state vector is always provided in “internal” space. The internal space is not defined and can vary between different platforms. If the shader can perform computations independent of the coordinate system, it can work directly with the state vector, otherwise it needs to be converted to a known space.
There are several defined spaces, and states provide matrices and functions that transform vectors, points, and normals between these coordinate systems. FIG. 60 shows a table 760 that lists the transformation matrices.

  There are a number of additional state variables available to lighting and volume shaders, which provide access to lighting characteristics and input values for the volume shader. A shader node that references an illumination or volume shader state variable can only be used in a graph that is used as an illumination or volume shader or itself as an illumination or volume shader.

The light shader can call the state conversion function and pass the value “light” as the from or to parameter.
61 shows a table 770 that lists lighting shader state variables, and FIG. 62 shows a table 780 that lists volume shader state variables.
The ray responsible for the current intersection state is described by ray type, ray shader, and is a ray_dispersal_group () and ray_history_group () state variable and function. These variables and functions use the following strings to describe ray attributes:

Rays have exactly one of the following types:
"Eye"-the first generation ray whose origin is at the eye position-"transparent"-transparent ray to the current object-"refract"-refraction to the current object-"reflect"-from the current object Reflection “shadow” —shadow ray • “occlusion” —ambient blockage ray • “environment” —environment ray ray may be at most one member of the following groups:

-"Specular"-specular transparency, reflection, or refraction-"glossy"-glossy transparency, reflection, or refraction-"diffuse"-diffuse transparency, reflection, or refraction-Shaders are exactly one member of the following groups: .
• “regular” —regular surface or volume • “photon” —global illumination or fire ray photons • “light” —lighting shader call • “displace” —displacement shader call Ray has zero or more of the following history flags be able to.

-"Lightmap"-light map shader call-"final gather"-final integrated ray-"probe"-probe ray that does not call shader-"hull"-path through the fuselage surface-"volume"-the origin of the ray is empty The given Trace_options class holds the parameters used by the trace () and occlusion () functions described in the next section. A shader can declare an instance of this type once and pass it to multiple trace calls.

When a Trace_options instance is declared, all of its member values are initialized to default values. The shader can then call various set methods to change the value to something different from the default. FIG. 63 shows a table 790 that lists the methods of the Trace_options class.
FIGS. 64 and 65 show tables 800 and 810 that list functions that are provided as part of the intersection state and depend on values that are accessible via the state variable. These functions, like state variables, can only be called in shader main methods or any method where state variables are passed as parameters.

MetaSL supports familiar programming constructs that control the flow of shader execution. Specifically, it is as follows.
• Loop statements, for, while, and do, while. The keywords continue and break are supported to control loop execution.
• Branch statements if with optional else close, switch, and case statements.

A return statement to exit a function or method and return a value if the function or method does not have a void type.
The syntax for flow control statements is the same as that used in the standard C programming language.
The Light_iterator class facilitates lighting repetition and does not require explicit lighting list shader input parameters. The lighting iterator implicitly refers to the scene lighting through the state. This iterator instance is declared and specified as part of a foreach statement. The syntax looks like this:

Light_iterator light;
foreach (light) {
// Statements that refer to members of 'light'
}
Most surface shaders loop the illumination source and sum the illumination directly from the illumination in the scene. Furthermore, area illumination requires taking multiple samples at points on the surface of the area illumination. The foreach statement enumerates both the lighting in the scene and the sample points on the surface of the area lighting, if appropriate.

Within the foreach block, you can access the members of the lighting iterator that contain the results from invoking the lighting shader for each lighting. The Light_iterator class is listed in the table 820 shown in FIG.
Shaders don't have to be aware of how many lights or how many samples are accumulating, but only provide the BRDF for a single sample at a time to the renderer driving the enumeration of lighting and samples. responsible. A shader may declare one or more variables outside the loop to store the lighting results. For each trip in the loop, the shader adds the BRDF result to these variables.

Color diffuse_light (0,0,0,0);
Light_iterator light; foreach (light) {
diffuse_light + = light.dot_nl * light.color;
This is a simple example of looping over lighting and summing diffuse lighting.
A powerful feature of MetaSL is the ability to describe shaders independently of special target platforms. This includes the ability to launch MetaSL shaders in software with software-based renderers and in hardware when the GPU is available.

  Software rendering is typically more generalized and flexible, allowing a variety of rendering algorithms including ray tracing and global illumination. At the time of writing this specification, graphics hardware generally does not support these features. Furthermore, different graphics hardware has different functions and resource limitations.

  The MetaSL compiler provides feedback to the shader writer indicating the conditions for any of the special shaders it compiles. This allows the user to know if the shader he wrote can be run on a special piece of graphics hardware. Where possible, the compiler will indicate specifically which part of the shader is causing incompatibility with the graphics hardware. For example, when a shader calls a raytracing function, the compiler indicates that the presence of a raytracing call forces the shader to be software compatible only. Alternatively, the user can specify a switch that causes the compiler to create a hardware shader. Calls to APIs not supported by the hardware are automatically removed from the shader.

MetaSL includes support for the following preprocessor declarations: #define, #undef, #if, #ifdef, #ifndef, #else, #elif, #endif.
These declarations have the same meaning as their equivalents in the C programming language. Macros with arguments are also supported as follows:
#define square (a) ((a) * (a))
#include declarations are also supported to add other MetaSL source files to the current file. This allows classes based on structure definitions and shaders to be shared between files.

Technology is a variation of shader implementation. Some shaders require only a single technology, but there are situations where it is desirable to implement multiple technologies. The language provides a mechanism for declaring multiple technologies within a shader.
A single shader implementation can be mapped to both software and hardware, and the exact same shader can often be used regardless of whether rendering is done on the CPU or GPU. However, in some cases, such as when a software shader uses a function that is not supported by the current graphics hardware, the shader implements another method for the shader to work on the GPU. There is a need to. Different graphics processors also have different functions and limitations, so a shader that runs on a special GPU may be too complex to run on another GPU. The technology allows multiple versions of shaders to support different classes of hardware.

  A shader may also want to implement various shading methods that are used in different situations. For example, a material shader may implement a shadow technique that provides the amount of transparency at a surface point used when tracing shadow rays. Different techniques can also be used to implement faster but lesser quality or slower but higher quality shaders.

  In a sense, the technology appears to be a different version of the shader, but the shader technology is declared within the shader class and shares the shader parameters and other variables. Thus, the technology continues to be grouped into classes. When a shader has only one technology, the shader class need not formally declare that technology.

  The technical declaration looks somewhat similar to the nested class definition within the shader class definition. A technology declaration provides a name that can be used to refer to the technology. The technology must at least define a main method that performs the main function of the shader technology. In addition, the technology can implement event methods that handle init and exit events. The main method and event method are described in the previous section. In addition, the technique can also include other local helper methods used by the two main technical methods.

Shader my_shader {
input:
Color c;
output:
Color result;
technique software {
void event (Event_type event);
void main ();
}
technique hardware {
void event (Event_type event);
void main ();
}
};
void my_shader :: software :: event (Event_type event) {
...
}
void my_shader :: software :: main () {
...
}
void my_shader :: hardware :: event (Event_type event) {
...
}
void my_shader :: hardware :: main () {
...
}
This example shows a shader that implements two different technologies for hardware and software. The technology main and event methods can be implemented inline in the class definition or separately as shown in this example.

At render time, a separate rules file accessible by the renderer informs the renderer how to select different shader technologies. The rules describe criteria for selecting a technology based on the value of a predetermined set of tokens. The token value describes the situation in which the shader is used. Possible token values are:
• Shading quality—surface points that appear in reflections are shaded by a faster but lower quality version of the shader (bumpy, blurred, or otherwise distorted surface points). Often.

Shadow-This token value is true when shading a surface point to determine transparency while tracing a shadow ray.
Energy—This token value is true when calling the lighting shader to determine the energy generated by the lighting and allow the renderer to sort the rays by importance.

Hardware / Software—These token values indicate whether the rendering is done in software on the CPU or in hardware on the GPU.
• Shader version-the highest pixel shader version supported by the current hardware.

Hardware vender chipset—A string that identifies the current hardware chipset. For example, nv30 or r420.
The rule needs to specify an expression to be defined when a particular technology is selected based on the technology name and token value. Multiple rules are suitable for any special set of token values. The process that the renderer uses to select a technique for the shader is as follows. At first, consider only the rules for the technology that exists in the shader. From these rules, each is tested in turn, and the first matching rule selects the technique. If no rule matches, an error / warning is issued or the default technique is used for the shader.

The following shows an example of a rule file.
beauty: software and shade_quality> 1
fast: software and shade_quality = 1
standard: software
fancy_hardware: hardware and shader_vers?} on> = 3.0
nvidia_hardware: hardware and chipset == "nv30"
basic_hardware: hardware
The first three rules are called “standard”, a single technique that handles all of the shading quality levels, or two techniques “beauty” and “fast” for handling two different quality levels of shading separately. Software shaders with shaders with The token value can also be used by the shader when activated, and a shader with a single standard technology can perform optional calculations according to the desired quality level.

  The second three rules are examples of different technologies to support different classes of hardware. Advanced hardware technology can make use of features that are only available in shader model 3.0 or later. nvidia_hardware technology can use features specific to NVIDIA's nv30 chipset. Finally, basic_hardware can be said to be a technology that can handle various types of hardware for handling general hardware.

  The language includes a mechanism that allows material shaders to express their results as a series of components instead of a single color value. This allows the components to be stored to separate the image buffer for later synthesis. Individual passes can also render a subset of all components, which can be combined with the remaining previously rendered components.

  When the shader breaks the result into multiple components, a shader variant can be automatically generated that calculates all components, or a subset of all components at the same time. When only a subset of the components are being calculated, other calculations on which the components depend can be omitted from the automatically generated shader. For example, if a shader uses global lighting to calculate indirect lighting and stores that indirect lighting in another layer, the other layers are re-rendered without recalculating the global lighting, and the indirect lighting layer Can be synthesized. The same shader source code can be used for each pass, but the renderer automatically generates individual shaders that calculate only the necessary components from a single source. This obviously relies on having source code that can be used with a C ++ compiler if any of the layers are related to software rendering.

The material shader breaks the result into components by declaring a separate output for each component. The name of the output variable defines the name of the layer in the current rendering.
shader Material_shader {
input:
...
output:
Color diffuse_lighting;
Color specular_lighting;
Color indirect_lighting;
};
This example shows a material shader that specifies three components for diffusion, specular reflection, and indirect illumination.

  If multiple material shaders exist in a scene that breaks down the results into different layers, the total number of layers can be large. The user may not want to allocate a separate image buffer for each of these layers. A mechanism in the scene definition file allows the user to specify compositing rules for combining layers into an image buffer. The user specifies how many image buffers to create and, for each buffer, an expression that determines the color that should be placed in that buffer when the pixel is rendered. The expression can be expressed as a function of the layer values:

Imagel = indirect_lighting
Image2 = diffuse_lighting + specular_lighting
In this example, the three layers from the shader result structure in the previous example are routed to two image buffers.
The standard MetaSL library provides API functionality that allows shaders to cast rays. Ray tracing is very computationally intensive. To optimize rendering time, the renderer provides a mechanism that delays ray tracing so that multiple shader calls can be grouped. This improves cache coherency and thus improves overall shader performance. The shader has an option to call a function to schedule a ray for ray tracing. This function returns just before the ray is actually traced, allowing the shader and other shaders to continue processing.

  When a shader schedules raytracing, it must also provide factors that help control the synthesis of the results from the raytracing with the shader results. It also provides a weight factor that describes the importance of the ray to the final image, thereby enabling importance based sampling. For example, the factor may be the result of Fresnel attenuation combined with a user-specified amount of reflection. Scheduling a ray implicitly defines a layer. The expression in the layer composition rule can refer to the factors provided when scheduling a ray. For example:

Imagel = diffuse + reflectionFactor * reflection
The shader needs to act directly on the raytracing results and may not schedule the ray for later compositing. In such cases, synchronous ray tracing functions are still available.
Shader parameters are often set by a user in an application that uses a graphical user interface (GUI). In order for the user to interact with the shader in the GUI, the application needs to know the shader parameters and some additional information about the shader itself.

  By writing annotations in the shader source code, information attributes can be added to shaders, parameters, or techniques. Annotations are placed immediately after a shader, parameter, or technology declaration by enclosing a list of attributes in curly brackets. An attribute instance is declared in a class with a similar method, with optional parameters passed to the constructor. The syntax is as follows:

{
attribute_name (paraml, param2, ...);
attribute_name (paraml, ...);
}
Parameters to attribute constructors must be direct constants, except in special cases where state variables are assigned as default values. Some standard attributes are predefined.

Default value (object value)-appended to the parameter to specify the default value for the parameter.
-Value-The value used as the default for the parameter. The type of this value is the same type as the parameter.
Soft range (object min, object max)-specifies the range of valid values for the parameter. This range can be exceeded if the user desires.

-Min- The lower end of the range. The value type is the same as the parameter type.
-Max- The upper end of the range. The value type is the same as the parameter type.
• hard range (object min, object max)-specifies the boundaries for parameters that cannot be exceeded.
-Min- The lower end of the range. The value type is the same as the parameter type.

-Max- The upper end of the range. The value type is the same as the parameter type.
Display name (string name)-the name of the shader, parameter, or technology that should be displayed to the user.
−name—Display name. This can be a more readable name than a typical variable name and can include spaces.

Description (string description) —A description of the shader, parameter, or technology.
-Description-description. An application can use this string to provide a tooltip for the GUI.
Hidden-added to the parameter, indicating that the parameter must not be shown in the application GUI. This attribute has no parameters. When the attribute is present, the parameter is considered hidden or not hidden.

• enum label (int value, string label)-added to a parameter of type enum. Two or more instances of this attribute may be added to enum type parameters, one for each possible enum value.
-Value-one of the possible enum values for which the parameter is set.
-Label-Label to use when presenting the enum value as an option in the GUI.

These attributes and other GUI-related attributes can be assigned to shaders via external files. Filling a shader source code with many annotations is not always desirable.
The following are examples of attributes in use:
shader mix_colors {
input:
Color colorl {
default_value (Color (0,0,0,1));
display_name ("Color 1");
};
Color color2 {
default_value (Color (1,1,1,1));
display_name ("Color 2");
};
Scalar mix {
default_value (0.5);
soft_range (0.0, 1.0);
display_name ("Mix amount");
};
output:
Color result;
void main () {
result = colorl * (1.0-mix) + color2 * mix;
}
} {Description ("Blends two colors using a mix amount parameter")};
The following keywords are not currently used in MetaSL, but are saved for future versions. class, const, private, protected, public, template, this, typedef, union, virtual.

MetaSL includes a standard library of eigenfunctions. The following is a list including lighting and ray tracing functions that can be extended without departing from the scope of the present invention and does not include software-only methods.
・ Mathematical functions: abs, acos, all, any, acos, asin, atan, ceil, clamp, cos, degrees, exp, exp2, oor, frac, lerp, log, log2, log10, max, min, mod, pow , radians, round, rsqrt, saturate, sign, sin, smoothstep, sqrt, step, tan.

Geometric functions: cross, distance, dot, faceforward, length, normalize, reflect, refract.
Texture map function: texture lookup. Texture functions pose an interesting problem for software and hardware shader integration. The hardware texture function usually has several versions, thereby projective texturing (division by w is built into “texture lookup”), explicit filter width, Allows depth texture lookup with depth compare. Cg also has a RECT version of texture lookup that uses texture pixel coordinates instead of normalized coordinates. In this way, functions are provided to both hardware and software. However, it is desirable to provide software-only texture lookup with elliptical filtering.

Derivatives: ddx, ddy, fwidth.
FIG. 67 shows a diagram illustrating an architecture 830 of a MetaSL compiler according to a further aspect of the present invention. The MetaSL compiler handles both the conversion of MetaSL shaders to target formats and the compilation of shader graphs into a single shader. The MetaSL compiler architecture can be extended with plug-ins, which can support future language targets with different input syntax.

  The compiler front end supports pluggable parser modules that support different input languages. MetaSL is expected to be the primary input language, but other languages can be supported by extensions. Thereby, for example, if a parser for a language in which a shader is written is created, the existing code base of the shader can be used.

  The compiler backend can also be extended with plug-in modules. The MetaSL compiler handles a lot of processing, provides a high-level shader representation for backend plug-ins, and can be used to generate shader code. Support for several languages and platforms currently in use is planned, but it is almost certain that new platforms will appear in the future. The main advantage of mill technology is that it isolates the shader from these changes. As new platforms or languages become available, new back-end plug-in modules can be implemented to support these targets.

The MetaSL compiler currently targets high-level languages, but there is also the possibility of generating machine code from high-level expressions with the GPU as the direct target. This allows special hardware to take advantage of the specific optimizations available because the code generator works directly from this high level representation and bypasses the native compiler.
Another component of Mill's MetaSL compiler is the Phenomenon Phenomenon shader graph compiler. The graph compiler processes shader graphs and compiles them into a single shader. These shaders avoid the overhead of shader attachment, thereby allowing the graph to be constructed from a larger number of simple nodes rather than a small number of complex nodes. This makes the shader's internal structure more accessible to users who are not professional programmers.

FIG. 68 shows a diagram illustrating a MetaSL compiler architecture 840 according to an alternative form of the invention.
The example below shows a phone shader implemented in MetaSL.
shader Phong {
input:
Color ambience;
Color ambient;
Color diffuse;
Color specular;
Scalar exponent;
output:
Color result;
void main () {
result = ambience * ambient;
Light_iterator light;
foreach (light) {
result + = light.color * max (0, light.dot_nl);
result + = light.color * specular *
phong_specular (light.direction, exponent);
}
}
};
The phong_specular function called in this example is a built-in function provided by MetaSL. State parameters such as surface normal and ray direction are implicitly passed to the function.

The example below shows a simple checker texture shader implemented in MetaSL.
shader Checker {
Input:
Color colorl;
Color color2;
Vector2 coords;
Vector2 size;
output:
Color result;
void main () {
Vector2 m = fmod (abs (coords), size) / size;
Vector2b b = m <0.5f;
result = lerp (colorl, color2, bx ^^ by);
}
};

D. MetaSL Shader Debugger Here we describe a mental mill MetaSL shader debugger application that provides an implementation of the concept of image-based shader debugging.

  FIG. 69 shows a screenshot of a debugger UI 850 according to a further aspect of the present invention. The shader debugger UI 850 includes a code view panel 852 that displays MetaSL code for the currently loaded shader, a variable list panel 854 that displays all variables in the range in the selected statement, and the value of the selected variable, or When no variable is selected, a 3D view window 856 is displayed that displays the results of the entire shader. Further, an error display window 858 is also provided.

  FIG. 70 shows a screenshot of the debugger UI 860 that appears when loading a shader, and if there are compilation errors, they are listed in an error display window 868. Selecting an error in the list highlights the line of code 862 where the error occurred. The shader file is reloaded by pressing the F5 key.

  FIG. 71 shows a screenshot of the debugger UI 870 that appears once the shader has been successfully loaded and compiled without errors. Debugging is initiated by selecting statement 872 in code view panel 874. The selected statement is highlighted in light green along the line of the selected statement. The variable window displays a range of variables 876 for the selected statement.

  As shown in FIG. 71, a statement is selected by clicking on a line of code clicks on a variable and displays its value in a render window. In the screen shot 870 of FIG. 71, the “normal” variable (Vector3 type) is selected. A vector value is mapped to each color. A line that is a statement has a white background. Lines that are not statements are gray.

FIG. 72 shows a screenshot of the debugger screen 880 illustrating how condition statements and loops are handled. Conditional statements and loops are not executed for some data points, so when the selected statement is a conditional close, the variable is not visible for some data points.
As shown in FIG. 72, when the selected statement 882 is a conditional statement, only the pixel 884 whose condition value is true displays the debug value. The remaining pixels display the original result.

  FIG. 73 shows a screenshot of the debugger screen 890, illustrating what happens when the selected statement is in a loop. In that case, the displayed value represents the first pass through the loop. A loop counter can be added to allow the user to specify which path he wants to debug through the loop.

The user can advance the statement by moving the line of code back and forth using the left and right arrow keys. The up and down arrow keys move through the variable list.
The space bar cycles through the sample object type in the viewport.
FIG. 74 shows a screenshot of the debugger screen 900 and shows how texture coordinates are handled. The user can select and view the texture coordinates as shown in this example. The prototype provides four sets of texture coordinates, each displayed side by side twice as many times as the previous set. U and V derived vectors are also provided.

When a vector value is mapped to a color, if the value exceeds 1, it is set to wrap. In this example, the selected texture coordinate is repeated four times, which can be clearly seen when looking at the variable 902 in the debugger.
In FIG. 74, a set of texture coordinates can be used with double tiling.
FIG. 75 shows a screen shot of the debugger screen 910, in which a depth illusion is generated by transforming texture coordinates according to a parallax map. In the screen shot 920 of FIG. 76, the texture coordinate offset can be clearly seen by looking at the texture coordinates in the debugger.

77 and 78 are screenshots of debugger screens 930 and 940 illustrating other shaders.
Conclusion The above description has been limited to specific embodiments of the present invention. However, it will be apparent that various changes and modifications can be made to the invention to obtain some or all of the advantages of the invention. The purpose of the appended claims is to cover these and other variations and modifications as falling within the true spirit and scope of the invention.

Generation of shader DAGs packaged and encapsulated, each containing one or more shaders, generated to ensure that the shaders in the shader DAG can work correctly during rendering, and constructed in accordance with the present invention FIG. 2 illustrates a computer graphics system that facilitates enhanced collaboration between shaders. FIG. 2 is a functional block diagram of the computer graphic system shown in FIG. 1. Figure 3 shows a graphical user interface for one embodiment of a phenomenon creator used in a computer graphics system, the functional block diagram of which is shown in Figure 2; FIG. 4 illustrates an example Phenomenon generated using the Phenomenon Creator shown in FIGS. 2 and 3. FIG. 3 shows a graphical user interface for one embodiment of a phenomenon editor used in a computer graphics system, the functional block diagram of which is shown in FIG. A and B show details of the graphical user interface shown in FIG. FIG. 7 is a flowchart showing operations performed by the scene image generation unit of the computer graphic system shown in FIG. 2 when generating an image of a scene. FIG. 7A is a flowchart showing operations performed by the scene image generation unit of the computer graphic system shown in FIG. 2 when generating an image of a scene. Fig. 2 shows a flowchart of the overall method according to one form of the invention. FIG. 2 shows a diagram of layers of software illustrating mental mill platform independence. An illustration of the MetaSL level as a subset is shown. A bar diagram illustrating the MetaSL level and its applicability to hardware and software rendering is shown. A screenshot of the graphical performance analysis window is shown. A bar graph of performance results for a range of values for special input parameters is shown. A screen view showing performance results in a tabular format is shown. FIG. 4 shows a diagram of a library module illustrating the main categories of a mental mill library. A diagram of the mental mill compiler library is shown. A diagram of a renderer based on a mental mill is shown. A screen view of the Phenomenon Graph Editor is shown. FIG. 6 shows a series of screenshots illustrating the operation of the Phenomenon graph editor and integrated MetaSL graphical debugger. FIG. 6 shows a series of screenshots illustrating the operation of the Phenomenon graph editor and integrated MetaSL graphical debugger. FIG. 6 shows a series of screenshots illustrating the operation of the Phenomenon graph editor and integrated MetaSL graphical debugger. FIG. 6 shows a series of screenshots illustrating the operation of the Phenomenon graph editor and integrated MetaSL graphical debugger. FIG. 6 shows a series of screenshots illustrating the operation of the Phenomenon graph editor and integrated MetaSL graphical debugger. Fig. 4 shows a diagram of a shader parameter editor. A and B show the thumbnail view and list view of the Phenomenon / Metanode Library Explorer, respectively. Shows code editor and IDE diagrams. A through C show a series of views of the debugger screen where numerical values for variables in pixel positions are displayed based on the mouse position. A diagram of the GUI library architecture is shown. A table 0101 listing the methods required to implement the BRDF shader is shown. An example configuration diagram in which two BRDFs are mixed is shown. It is a figure which illustrates the pipeline for the shading by the acquired BRDF. A screenshot of the basic layout of the GUI is shown. A diagram of a graph node is shown. Fig. 4 shows a diagram of a graph node, including the structure of subparameters. A sample graph view inside Phenomenon is shown. Fig. 5 shows a sample graph view when the Phenomenon is opened in place. Shows a sample graph view when a Phenomenon is opened within another Phenomenon. FIG. 6 shows a diagram illustrating the result of adding a color output to a scalar input. FIG. 6 shows a diagram illustrating a shader boxed into a new Phenomenon node. Figure 7 shows a bird's eye view control for viewing a sample shader graph. A through D show a series of diagrams illustrating the detailed node level progression. A and B show toolboxes in the thumbnail view and the list view. Fig. 5 shows a parameter view for displaying a control that allows editing the parameters of a selected node. Shows a diagram of the code editor that allows MetaSL users to create shaders by writing code. The figure which illustrates the combination which makes two metanodes into the 3rd metanode is shown. A partial diagram of the variable list is shown. A-C show a series of diagrams illustrating a visualization technique in which a vector is drawn as an arrow located on the image surface when the user drags the mouse over the image surface. FIG. 6 shows a diagram illustrating a visualization technique for a matrix. FIG. 4 shows a diagram illustrating a visualization technique for a three-way vector. FIG. 6 shows a diagram illustrating a visualization technique for viewing vector type values using a gauge style display. Shows a table listing the Evnet_type parameters and their descriptions. 2 shows a table illustrating the results of a vector construction method. A table describing a Boolean operator is shown. A table listing comparison operators is shown. A schematic diagram of a bump map phenomenon is shown. FIG. 6 shows a diagram illustrating a bump map phenomenon in use. FIG. 4 shows a schematic diagram of a bump map phenomenon according to a further embodiment of the invention. A diagram of the bump map phenomenon in use is shown. Shows a table listing the set of state variables. Shows a table listing the set of state variables. Fig. 5 shows a table listing the transformation matrices. Fig. 5 shows a table listing lighting shader state variables. Fig. 5 shows a table listing volume shader state variables. Shows a table listing the methods of the Trace_options class. Describes a table that lists functions that depend on values provided as part of the intersection state and accessible via the state variable. Describes a table that lists functions that depend on values provided as part of the intersection state and accessible via the state variable. Shows a table that lists the members of the Light_iterator class. A diagram of the MetaSL compiler is shown. Fig. 4 shows a diagram of a MetaSL compiler according to an alternative form of the invention. Fig. 4 shows a screenshot of a debugger screen according to a further aspect of the invention. Shows a screenshot of the debugger screen when there is a compilation error when loading a shader. Shows a screenshot of the debugger screen when the shader has been successfully loaded and compiled without error, at which point debugging can begin by selecting a statement. Shows a screenshot of the debugger screen when the selected statement is conditional. Shows a screenshot of the debugger screen when the selected statement is in a loop. Shows screenshot of debugger screen where texture coordinates are visible. FIG. 6 shows a screenshot of a debugger screen where parallax mapping produces depth illusion by transforming texture coordinates. The texture coordinate offset shows a screenshot of the debugger screen that can be seen when looking at the debugger texture coordinates. FIG. 6 shows a screenshot of a debugger screen illustrating another shader example. FIG. 6 shows a screenshot of a debugger screen illustrating another shader example.

Claims (38)

In a computer graphics system for generating an image of a scene from a representation to which at least one instantiated phenomenon is added, the instantiated phenomenon comprises an encapsulated shader DAG comprising at least one shader node; The improvement of the system is
A metanode environment comprising component shaders that are operable for the creation of metanodes and that can be combined to build more complex shaders in the network;
A graphical user interface (GUI) for managing the metanode environment to be in contact with the metanode environment and to allow a user to build shader graphs and femininas using the metanode environment; A computer graphic system. The improvement
The computer graphics system of claim 1, further comprising a software language that can be used by a human operator to manage the metanode environment, implement a shader, and integrate a separate shading application. The improvement
The computer graphics system of claim 1, further comprising at least one GUI library that can be used to generate a GUI that builds a shader graph and a phenomina in conjunction with the metanode environment. In the further improvement,
The software language can be configured as a superset of a plurality of selected shader languages for a selected hardware platform; and
The software language includes software code whose compiler function is optimized for a selected hardware platform in a selected shader language from a single reusable description of Phenomenon expressed in the software language. The computer graphics system of claim 2, wherein the computer graphics system is capable of being generated. The improvement
Using the compiler functions specific to the selected shader language, the optimized software code for the selected hardware platform and the selected shader language is converted into a machine language for the selected integrated circuit instantiation. 5. The computer graphic system of claim 4, further comprising converting. The improvement
Being in contact with the GUI, (1) allowing the user to detect and correct potential defects in the shader, and (2) a test scene with the shader, metanode, or phenomenon that is being tested is always rendered The computer graphics system of claim 1, further comprising an interactive visual real-time debugging environment that provides a viewing window. The improvement
Being in contact with the GUI, (1) allowing the user to detect and correct any possible defects in the shader, and (2) a test scene with the shader, metanode, or phenomenon that is being tested is always rendered The computer graphics system of claim 2, further comprising an interactive visual real-time debugging environment that provides a viewing window. A computer graphics system that allows an operator to create a phenomenon with an encapsulated shader DAG comprising at least one shader node, the computer graphics system comprising: (A) a plurality of basic shader nodes including shaders And (B) a Phenomenon creator configured to allow the operator to interconnect the basic shader node from the basic shader node database to a DAG, The Phenomenon Creator for verifying that the interconnection between the basic shader nodes provided by the operator comprises a DAG, and in improving the system,
The Phenomenon Creator is
A metanode environment capable of creating metanodes with element shaders that can be combined to build more complex shaders in the network;
A graphical user interface (GUI) that manages the metanode environment so that the metanode environment is in contact and a user can use the metanode environment to build a shader graph and a phenomina;
A software language that can be used by the operator to manage the metanode environment, implement a shader, and integrate a separate shading application, a superset of a plurality of selected shader languages for a selected hardware platform The compiler function generates optimized software code for a selected hardware platform in a selected shader language from a single reusable description of Phenomenon expressed in the software language A computer graphics system comprising: a software language that enables   9. The computer graphic system according to claim 8, wherein, in the further improvement, the phenomenon creator stores the phenomenon created by the operator in a phenomenon database.   In the further refinement, the Phenomenon Creator is further configured to allow the operator to interconnect the basic shader nodes in a Phenomena with a plurality of cooperating DAGs, and a shader in one of the cooperating DAGs. 9. The node, when used during rendering of an image of a scene, provides at least one value used in conjunction with another one of the cooperating DAGs. Computer graphics system. In a computer graphic system that allows an operator to generate an instantiated Phenomenon from a Phenomenon with an encapsulated shader DAG comprising at least one shader DAG having at least one shader node, the computer graphic The system provides (A) a Phenomenon database configured to store the Phenomenon, and (B) the operator selects the Phenomenon and provides a value for at least one parameter associated with the at least one shader node. A Phenomenon editor configured to allow for the improvement of the system,
The Phenomenon Editor
A metanode environment capable of creating metanodes with component shaders that can be combined to build more complex shaders in the network;
A graphical user interface (GUI) that manages the metanode environment so that the metanode environment is in contact and a user can use the metanode environment to build a shader graph and a phenomina;
A software language that can be used by the operator to manage the metanode environment, implement a shader, and integrate a separate shading application, a superset of a plurality of selected shader languages for a selected hardware platform The compiler function generates optimized software code for a selected hardware platform in a selected shader language from a single reusable description of Phenomenon expressed in the software language A computer graphics system comprising: a software language that enables A computer graphics system for generating an image of a scene from a representation appended with at least one instantiated Phenomenon comprising an encapsulated shader DAG comprising at least one shader node, the computer graphics system comprising: (A) The at least one instantiated Phenomenon determines whether preprocessing associated with the representation is necessary and, if necessary, performs the preprocessing to generate a preprocessed representation of the scene A preprocessor configured, and (B) a renderer configured to generate a rendered image from the preprocessed representation of the scene, the system improvement comprising:
A metanode environment capable of creating metanodes with element shaders that can be combined to build more complex shaders in the network;
A graphical user interface (GUI) that manages the metanode environment so that the metanode environment is in contact and a user can use the metanode environment to build a shader graph and a phenomina;
A software language that can be used by the operator to manage the metanode environment, implement a shader, and integrate a separate shading application, a superset of a plurality of selected shader languages for a selected hardware platform The compiler function generates optimized software code for a selected hardware platform in a selected shader language from a single reusable description of Phenomenon expressed in the software language A computer graphics system comprising: a software language that enables   In the further refinement, at least one type of shader node is a geometric shader node type, and the preprocessor is configured to add to the scene when the at least one shader node is the geometric shader node type. 13. The computer graphic system of claim 12, wherein the computer graphic system is configured to perform the preprocessing operation to define a geometric figure for it.   In the further refinement, the at least one type of shader node is a photon shader node type, and in the further refinement, the preprocessor is configured so that the at least one shader node is the photon shader node type. The pre-processing operation is configured to control a path of photons in a scene and characteristics of photon interaction with an object surface in the scene. 13. The computer graphic system according to 12.   In the further refinement, the at least one type of shader node is a photon emitter shader node type, and the preprocessor is configured such that the at least one shader node is the photon emitter shader node type. 13. The computer graphic system of claim 12, wherein the computer graphic system is configured to perform the preprocessing operation to simulate photon generation with a light source that illuminates the scene.   In the further refinement, at least one type of shader node is a photon volume shader node type, and the preprocessor is configured such that the at least one shader node is the photon volume shader node type. 13. The apparatus of claim 12, wherein the preprocessing operation is performed to simulate the interaction of photons from a light source with a three-dimensional volume of space in the scene. Computer graphic system.   The refinement determines whether the at least one instantiated phenomenon has a post-processing operation associated with the representation and, if necessary, performs the post-processing operation. The computer graphics system of claim 12, further comprising a processor.   In the further refinement, the at least one shader node is an output shader node type, and the post-processor performs the post-processing operation when the at least one shader node is the output shader node type. 13. The computer graphic system of claim 12, wherein the computer graphic system is configured.   In the further refinement, the rendered image comprises a plurality of pixels, each associated with a pixel value, and the post processor is configured to perform the post-processing operation in relation to the pixel value. The computer graphics system of claim 18, wherein: In a computer graphics system for generating an image of a scene from a representation appended with at least one instantiated Phenomenon comprising an encapsulated shader DAG comprising at least one shader node, the system improvement comprises:
A metanode environment capable of creating metanodes with component shaders that can be combined to build more complex shaders in the network;
A graphical user interface (GUI) that manages the metanode environment so that the metanode environment is in contact and a user can use the metanode environment to build a shader graph and a phenomina;
A software language that can be used by the operator to manage the metanode environment, implement a shader, and integrate a separate shading application, a superset of a plurality of selected shader languages for a selected hardware platform The compiler function generates optimized software code for a selected hardware platform in a selected shader language from a single reusable description of Phenomenon expressed in the software language A software language that enables
At least one GUI library for generating a GUI that can be used in conjunction with the metanode environment and that builds shader graphs and fenomina;
An interactive, visual, and real-time debugging environment in contact with the GUI, (1) allowing the user to detect and correct potential defects in the shader; (2) A debugging environment that provides a viewing window in which test scenes from shaders, metanodes, or Phenomenon under test are always rendered;
A device in communication with the compiler function, wherein the optimized software code for the selected hardware platform and the selected shader language is obtained using the intrinsic compiler function for the selected shader language. A computer graphic system comprising: a machine language for the instantiation of the selected integrated circuit. Enables the generation of an image of a scene in a computer graphics system for generating an image of a scene from a representation with at least one instantiated Phenomenon comprising an encapsulated shader DAG comprising at least one shader node A method,
Configuring a metanode environment operable to create a metanode with component shaders that can be combined in the network to build more complex shaders;
Configuring a graphical user interface (GUI) that is in communication with the metanode environment, manages the metanode environment, and allows a user to build shader graphs and femininas using the metanode environment;
A software language that can be used by a human operator as an interface to manage the metanode environment, implement shaders, and integrate separate shading applications, with multiple selected shader languages for a selected hardware platform Optimized software code for a selected hardware platform in a selected shader language from a single reusable description of Phenomenon expressed in the software language, which can be configured as a superset of Providing a software language that generates
Providing at least one GUI library that can be used in conjunction with the metanode environment to generate a GUI for building shader graphs and fenomina;
An interactive, visual, and real-time debugging environment in contact with the GUI, (1) allowing the user to detect and correct potential defects in the shader; (2) Configuring a debugging environment that provides a viewing window in which test scenes from shaders, metanodes, or Phenomenon under test are always rendered;
A device in communication with the compiler function, wherein the optimized software code for the selected hardware platform and the selected shader language is obtained using the intrinsic compiler function for the selected shader language. Configuring the device to convert to machine language for instantiation of the selected integrated circuit. The improvement
9. The computer graphics system of claim 8, further comprising a shader performance analysis via profiling including a graphical representation on the GUI.   In the further refinement, the shader performance for a given node can be analyzed relative to the shader graph where the node resides such that the performance result is relative to the input value driving the node. 23. The computer graphic system of claim 22, wherein:   23. In the further refinement, profiling information is provided at a node level for nodes that are part of a graph or phenomenon, and at a statement level for computer language software code contained in a metanode. The computer graphics system described.   In further refinements, the performance information at a given level of detail is normalized to the overall performance cost of the node, the associated shader, or the cost of rendering the entire scene with multiple shaders. 23. The computer graphic system of claim 22, wherein:   In the further refinement, the execution time of the computer software code statements in the metanode is expressed as a percentage of the total execution time of the metanode or the total execution time of the entire shader if the metanode is a member of a graph. 23. The computer graphics system of claim 22, wherein the computer graphics system is capable.   23. The computer graphics system of claim 22, wherein in the further refinement, the graphical representation of performance can be provided using a plurality of visualization methods.   In the further refinement, one method presents the normalized performance cost information by mapping a percentage associated with the normalized performance cost information to a color gradient. 28. The computer graphics system of claim 27.   29. In the further refinement, the mapping includes presenting a color bar that an operator can identify adjacent to each computer software code statement to be evaluated. Computer graphics system.   28. The computer graphics system of claim 27, wherein, in the further refinement, the graphical representation of the performance result is dynamically updated as the movie scene progresses.   28. The computer graphics system of claim 27, wherein, in the further refinement, the visualization method includes displaying a graph of performance results for a range of values for a given input parameter.   28. The computer graphics system of claim 27, wherein, in the further refinement, the visualization method includes displaying performance results in a tabular format.   28. The computer graphic system of claim 27, wherein in the further refinement, the visualization method includes displaying performance timing of each node of the Phenomenon with respect to the overall performance cost of the entire shader.   9. In the further improvement, one of the shaders is a bi-directional reflection distribution function (BRDF) shader type that allows a surface illumination model to be abstracted and rendered. Computer graphics system.   35. The computer graphics system of claim 34, wherein the improvement further comprises employing a plurality of illumination models represented by each of a plurality of BRDF nodes.   36. The computer graphics system of claim 35, wherein the improvement further comprises enabling scaling of BRDF values by scalar or color.   37. The computer graphics system of claim 36, wherein the improvement further comprises enabling the mixing of a plurality of lighting models represented by each of a plurality of BRDF nodes.   35. The computer graphic system of claim 34, wherein the improvement further comprises the use of acquired BRDF represented by data generated by a sensing device.