[go: up one dir, main page]

WO2019241630A1 - Systèmes et procédés pour plateforme de fusion de données d'intelligence artificielle - Google Patents

Systèmes et procédés pour plateforme de fusion de données d'intelligence artificielle Download PDF

Info

Publication number
WO2019241630A1
WO2019241630A1 PCT/US2019/037181 US2019037181W WO2019241630A1 WO 2019241630 A1 WO2019241630 A1 WO 2019241630A1 US 2019037181 W US2019037181 W US 2019037181W WO 2019241630 A1 WO2019241630 A1 WO 2019241630A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
synapse
dataspace
neuron
artifact
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2019/037181
Other languages
English (en)
Inventor
David W. BAUER Jr.
Joel Branch
Christopher D. Carothers
Justin Lapre
David SALMEN
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Deep Insight Solutions Inc
Original Assignee
Deep Insight Solutions Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US16/010,083 external-priority patent/US20190385052A1/en
Priority claimed from US16/147,135 external-priority patent/US10417558B1/en
Priority claimed from US16/252,422 external-priority patent/US20190392001A1/en
Application filed by Deep Insight Solutions Inc filed Critical Deep Insight Solutions Inc
Publication of WO2019241630A1 publication Critical patent/WO2019241630A1/fr
Priority to US16/900,858 priority Critical patent/US20200342290A1/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/082—Learning methods modifying the architecture, e.g. adding, deleting or silencing nodes or connections
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/098—Distributed learning, e.g. federated learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/0985—Hyperparameter optimisation; Meta-learning; Learning-to-learn
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00—Computing arrangements using knowledge-based models
    • G06N5/02—Knowledge representation; Symbolic representation
    • G06N5/022—Knowledge engineering; Knowledge acquisition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/047—Probabilistic or stochastic networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/049—Temporal neural networks, e.g. delay elements, oscillating neurons or pulsed inputs

Definitions

  • the present disclosure generally relates to cloud computing, and more particularly to methods and systems for an artificial intelligence data fusion platform.
  • An organization may use multiple disparate and changing data sources and data processing systems to accomplish its objectives.
  • Different data sources may include data in different formats, data with different access rights, and data that is logically separate from other data. Because of the differences in these data sources, sharing data between different parts of the organization may be difficult. For example, a person in the organization may wish to compile all the data the organization stores that pertains to a certain topic. However, because of the different data formats, the logical separation of the data, or other barriers, compiling such data may require a lot of work to transform the data into a compatible form.
  • a user may manually look through data in a relational database, file system, or other data storage means (one or more of which may reside in different logical sections), manually determine that the data stored in different places are inconsistent, choose a standardized format, and manually change the data into the standardized format.
  • the relational database or file system store the data in highly rigid and non-flexible ways that may restrict data usage or storage options.
  • an organization may use a cloud system or supercomputer to execute artificial intelligence (Al), machine learning, or data analytics capabilities.
  • Al artificial intelligence
  • machine learning or data analytics capabilities
  • these conventional systems do not provide important features such as input data management, workflow management, or data security.
  • these conventional systems cannot use unstructured data during machine learning or data analytics processes.
  • Conventional systems in response to encountering a portion of data with missing values, either drop the data portion or fill in the missing values with a default data value or some kind of calculated data value (e.g ., an average data value calculated from other data in the dataset).
  • Conventional systems may also use a web-based user interface that leverages Javascript libraries or may use an online coding notebook interface that accepts lines of computer code as input.
  • a system may ingest data from one or more external sources, populating a dataspace.
  • the system may include one or more software applications accessed by distributed users. Some of the software applications may be hosted on a cloud of the system. Some functionality may be distributed at a tactical edge of the system.
  • the system may include one or more infrastructure services, some of which may include large-scale services.
  • the system may support interoperability with other external systems.
  • the system may be deployed in a variety of configurations, such as a data center or a smaller configuration, such as half rack or transit case.
  • the system may support a large number of widely different usages. These usages may include varying operating needs.
  • the system may support multiple users from different organizations using the capabilities of the system simultaneously while keeping these users’ or organizations’ data and processing separate from one another and secure.
  • One embodiment of the disclosure may include a method.
  • the method may include a method for data flow and data fusion.
  • the method may include providing, in a computer-readable storage device, a dataspace.
  • the dataspace may include a plurality of logical segments.
  • the dataspace may include dataspace data stored in the plurality of logical segments.
  • the method may include receiving, from a data source external to the dataspace, a data artifact.
  • the data artifact may include a plurality of data elements.
  • the method may include processing, at a plurality of data filters, the plurality of data elements.
  • a data filter may process a data element based on a structure of the plurality of the data elements.
  • a data filter may process a data element based on a configuration of the data filter.
  • a data filter may process a data element based on the mathematical analysis of the data elements.
  • a data filter may process a data element based on an algorithm based on a mathematical function.
  • a data filter may process a data element based on a method that may include generating processed data. Generating the data may be based on the plurality of data elements. The method may include sending the processed data to a logical segment.
  • a data filter may use artificial intelligence (Al) model, system, or method functionality as part of the data flow and fusion.
  • Al artificial intelligence
  • One aspect of the disclosure may include a computer-implemented method.
  • the computer-implemented method may include an improved hyperparameter search for an Al or machine learning model.
  • the method may include generating a first deep learning model configuration.
  • the method may include calculating a first result metric for the first deep learning model configuration.
  • the method may include selecting a first sample space.
  • the first sample space may be based on the first deep learning model configuration.
  • the method may include generating a second deep learning model configuration.
  • the second deep learning model configuration may be within the first sample space.
  • the method may include calculating a second result metric for the second deep learning model configuration.
  • the method may include, in response to the second result metric exceeding the first result metric, selecting a second sample space.
  • the second sample space may be based on the second deep learning model configuration.
  • the method may include, in response to the second result metric not exceeding the first result metric, reducing the size of the first sample space.
  • the computer-implemented method may include an improved optimistic event processing method for an artificial neural network.
  • the method may include receiving, at a neuron of an artificial neural network, a sequence of synapse messages.
  • Each synapse message may include a timestamp of the time the synapse message was sent.
  • the method may include determining, based on the timestamp of each synapse message, whether the neuron processed the sequence of synapse messages out of order with respect to the timestamps.
  • the method may include in response to the neuron processing the sequence of synapse messages out of order, reverse computing at least one computation performed by the neuron in response to processing the sequence of synapse messages out of order.
  • the method may include performing the at least one computation based on receipt of the sequence of synapse messages in a correct order as determined by the timestamp of each synapse message in the sequence of synapse messages.
  • the computer-implemented method may include a method for roll back in a computer system.
  • the method for rollback may include a method for rolling back data ingested into a dataspace, rolling back an artificial intelligence or machine learning model, rolling back a data model, or rolling back an assertion in a dataspace.
  • FIG. 1 A is a schematic block diagram of one embodiment of a system for an artificial intelligence data fusion platform
  • FIG. 1 B is a schematic block diagram illustrating one embodiment of a mission dataspace
  • FIG. 2A is a schematic block diagram illustrating one embodiment of a cloud environment
  • FIG. 2B is a schematic block diagram illustrating one embodiment of a cloud environment
  • FIG. 3 is a schematic block diagram illustrating one embodiment of an information environment
  • FIG. 4A is a schematic block diagram illustrating one embodiment of a server-side environment
  • FIG. 4B is a schematic block diagram illustrating one embodiment of a portion of a system for an artificial intelligence data fusion platform
  • FIG. 5A is a schematic block diagram illustrating one embodiment of a mission application environment
  • FIG. 5B is a front view illustrating one embodiment of a graphic user interface (GUI) for the system for artificial intelligence data fusion platform;
  • GUI graphic user interface
  • FIG. 5C is a front view illustrating another embodiment of a graphic user interface (GUI) for the system for artificial intelligence data fusion platform;
  • GUI graphic user interface
  • FIG. 5D is a front view illustrating another embodiment of a graphic user interface (GUI) for the system for artificial intelligence data fusion platform;
  • GUI graphic user interface
  • FIG. 5E is a front view illustrating another embodiment of a graphic user interface (GUI) for the system for artificial intelligence data fusion platform;
  • GUI graphic user interface
  • FIG. 6A is a schematic block diagram illustrating one embodiment of a management environment
  • FIG. 6B is a schematic block diagram illustrating one embodiment of a data management application
  • FIG. 7 is a schematic block diagram illustrating one embodiment of a security environment
  • FIG. 8 is a flowchart illustrating one embodiment of a method for data ingestion
  • FIG. 9 is a flowchart block diagram illustrating one embodiment of a method for deep learning optimization
  • FIG. 10A is a schematic block diagram illustrating one embodiment of a hyperparameter design space for deep learning optimization
  • FIG. 10B is a schematic block diagram illustrating one embodiment of a hyperparameter design space for deep learning optimization
  • FIG. 10C is a schematic block diagram illustrating one embodiment of a hyperparameter design space for deep learning optimization
  • FIG. 10D is a schematic block diagram illustrating one embodiment of a hyperparameter design space for deep learning optimization
  • FIG. 10E is a schematic block diagram illustrating one embodiment of a hyperparameter design space for deep learning optimization
  • FIG. 1 1 is a flowchart block diagram illustrating one embodiment of a method for deep learning optimization
  • FIG. 12A is a schematic block diagram illustrating one embodiment of a hyperparameter design space for deep learning optimization
  • FIG. 12B is a schematic block diagram illustrating one embodiment of a hyperparameter design space for deep learning optimization
  • FIG. 12C is a schematic block diagram illustrating one embodiment of a hyperparameter design space for deep learning optimization
  • FIG. 12D is a schematic block diagram illustrating one embodiment of a hyperparameter design space for deep learning optimization
  • FIG. 12E is a schematic block diagram illustrating one embodiment of a hyperparameter design space for deep learning optimization
  • FIG. 13 is a flowchart block diagram illustrating one embodiment of a method for deep learning optimization
  • FIG. 14 is a schematic block diagram depicting one embodiment of an artificial neural network
  • FIG. 15 is a flowchart block diagram illustrating one embodiment of a method for artificial neural network optimistic event processing
  • FIG. 16 is a flowchart block diagram illustrating one embodiment of a method for artificial neural network optimistic event processing
  • FIG. 17A is a schematic block diagram depicting one embodiment of a sequence of synapse message
  • FIG. 17B is a schematic block diagram depicting one embodiment of a sequence of synapse message and anti-message
  • FIG. 18 is a flowchart block diagram illustrating one embodiment of a method for artificial neural network optimistic event processing
  • FIG. 19 is a schematic flow diagram depicting one embodiment of an example process of a synapse group processing for an artificial neural network
  • FIG. 20 is a flowchart block diagram illustrating one embodiment of a method for artificial neural network optimistic event processing.
  • FIG. 21 is a schematic block diagram illustrating one embodiment of a system for neural network optimistic event processing.
  • FIG. 22 is a schematic block diagram illustrating one embodiment, of an echo state network.
  • FIG. 1 A depicts one embodiment of a system 100.
  • the system 100 may include a system for an artificial intelligence data fusion platform.
  • the system 100 may include a plurality of environments.
  • An environment of the system 100 may include a physical or logical subsection of the components, devices, or applications, of the system 100.
  • the plurality of environments may include a cloud environment 102, an information environment 104, a security environment 106, a server-side environment 108, a mission application environment (MAE) 1 10, or a management environment 1 12.
  • MEE mission application environment
  • the system 100 may include a plurality of dataspaces.
  • a dataspace may include a set of data.
  • the dataspace may include one or more logical structures that may store or organize data stored in the system 100.
  • the dataspace may result from data being integrated into the system 100.
  • the dataspace may result from enrichment of the data included in the dataspace.
  • the dataspace may include metadata associated with the data stored in the dataspace.
  • the contents of the dataspace may be accessible by an application executing in the system 100.
  • a dataspace may be implemented on one or more storage devices, such as a cloud storage device.
  • FIG. 1 B depicts one embodiment of a mission dataspace 120.
  • the system 100 may include a mission dataspace 120 and an operational dataspace (not depicted in FIG. 1 B).
  • the mission dataspace 120 may include data that is the subject of processing or that pertains to an application of the system 100.
  • the operational dataspace may include data that supports the operation of the system 100 or the processing of data in the mission dataspace 120.
  • the operational dataspace may include data used or created by infrastructure applications to support their operation or execution, input data (such as data entering the system 100 for the first time from an external system or user), work and messages queues, temporary results, configuration files, or transient data.
  • the mission dataspace 120 may include a plurality of logical segments.
  • the plurality of logical segments may include a first logical segment.
  • the first logical segment may include an artifact semantics segment 122.
  • the plurality of logical segments may include a second logical segment.
  • the second logical segment may include a data semantics segment 124.
  • the plurality of logical segments may include a third logical segment.
  • the third logical segment may include a model semantics segment 126.
  • the plurality of logical segments may include a fourth logical segment.
  • the fourth logical segment may include a data artifact storage segment 128.
  • the data artifact storage segment 128 may include one or more data artifacts.
  • a data artifact may include a digital container of data.
  • a data artifact may include data in a file, data in a relational database, data retrieved via an interface of an application or a computer system, or a data feed or stream (including a real-time data feed).
  • a data artifact may include a plurality of data elements.
  • a data element may include a specifiable portion of the data artifact.
  • a data element may include a string of text located at a certain byte offset in a document, an entry in a relational database table at a certain row and certain column, or a segment of audio in an audio stream from a start time to an end time.
  • the data artifact semantics segment 122 may include metadata about a data artifact.
  • the metadata may include generic metadata. Generic metadata may include a hash, file name, location, data size, file type, or a timestamp of the data artifact.
  • the data artifact semantics segment 122 may include the contents or body of the data artifact.
  • the data artifact semantics segment 122 may include artifact association data. The artifact association data may associate a first data artifact with a second data artifact.
  • the data semantics segment 124 may include data element association data.
  • the data element association data may associate a data element with another data element or a data element with a data artifact.
  • the data semantics segment 124 may include a data element for the text string“Washington.”
  • Data element association data may associate the“Washington” data element with a data artifact including the text string “Washington.”
  • the data element association data may associate the“Washington” data element with a“City” concept or a“Person” concept.
  • disambiguating a data element may include determining whether an instance of a data element pertains to one or more concepts.
  • a text document may include the text string “Washington,” and disambiguating the text string data element may include determining whether the text string refers to the city of Washington or a person named “Washington.”
  • An application executing the disambiguating process may use other data in the data artifact or data in a dataspace to disambiguate.
  • the data semantics segment 124 may include metadata such as a timestamp, source, author, associated artifacts, or other metadata.
  • the data model semantics segment 126 may include a data model and corresponding metadata for a data source or type of data artifact.
  • an email data model may describe the structure of an email data artifact.
  • a second data model may describe the structure of a database used by an external application.
  • a data model may include a plurality of data model elements. The plurality of data model elements for a certain data model may describe the structure of the data source. The plurality of data model elements may include concept data or predicate data.
  • Concept data may include data associated with a representation of a concept.
  • Concept data may include data describing a concept that a data element may pertain to.
  • a concept may include a city, a person, a name, an address, or another concept.
  • Predicate data may include data describing a relationship between data elements and concepts.
  • a predicate may describe a data element as being a concept, having a concept, or another type of relationship.
  • certain data elements of a data artifact may be copied or moved to the data artifact semantics segment 122.
  • Metadata about the data artifact, or data model information associated with the data artifact may be derived from a data artifact stored in the data artifact storage segment 128 and stored in the data artifact semantics segment 122, the data semantics segment 124, or the data model semantics segment 126.
  • a data element may include a mention.
  • a mention may include a portion of data contained within a data artifact at a quantifiable location within the artifact. The location may be expressed as a byte offset, a pixel coordinate, a video or audio frame, or some other location-indicating data.
  • a mention may include a string of text, an object within an image, a segment of audio or video, or some other portion of data in a data artifact. As an example, the string of text“red yellow red blue” may include two mentions of “red.”
  • a data element may include a sign.
  • a sign may include a representation of multiple identical mentions. The multiple mentions of the sign may include the same portion of data, but be located at different locations. The different locations may be within the same data artifact or may be within different artifact. As an example, the sign“red” may represent all mentions of the text“red.”
  • a data element may include a concept.
  • a concept may include a representation of an idea.
  • a concept may be defined by a data model, such as the source data model of a data artifact.
  • the nodes in an ontology, a tag set in an extensible markup language (XML) schema, or attribute/table names in a relational database may define one or more concepts. Examples of a concept may include a city, a person, a name, an address, a photograph, a message, or body text.
  • a data element may include a term.
  • a term may include a disambiguated sign.
  • a sign being disambiguated may include associating a sign with a concept.
  • the mission dataspace 120 may include two terms for the sign“red.” The first term for“red” may associate the sign“red” with the concept of color. The second term for“red” may associate the sign“red” with the concept of a slang word for a communist.
  • a term may include a reference to a data artifact, file, or other object within the mission dataspace 120 or the system 100. For example, in one embodiment, a term may include a pathname of a file in the system 100.
  • a data element may include a predicate.
  • a predicate may include a representation of a relationship between data elements or concepts.
  • a predicate may indicate a relationship between two terms.
  • a relationship may include a wide variety of types of relationships.
  • a relationship may include a first thing being an instance of a second thing, a first thing being a subset of a second thing, a first thing possessing a second thing, or other relationships between two or more things.
  • a data element may include a statement.
  • a statement may include one or more subject terms, one or more predicates, and one or more objects.
  • An object of the one or more objects may include a term or another statement.
  • a statement may include a subject term, a predicate, and an object term.
  • An example notation for such a statement may include [subject term, predicate, object term].
  • Such a statement may include [“red”, is an instance of,“color”].
  • a statement may include a subject term, a predicate, and an object statement.
  • An example notation for such a statement may include [subject term, predicate, [object statement]].
  • Such a statement may include [“John Smith”, is the parent of , [“Alice Smith”, is a member of, “the Smith family”]].
  • Another example, statement may include a subject statement, a predicate, and an object statement. Other combinations of terms and statements may form a statement.
  • Metadata may be associated with a data element.
  • a term or a statement may include metadata describing the sign, concept, term, or statement.
  • the metadata may include a user that generated the data element or inserted the data element into the system 100, a timestamp for when the data element was generated or inserted, or other data about a data element.
  • the operational dataspace may include operational data.
  • the operational data may include temporary data storage for an executing application, data for system events, performance or resource utilization metrics, log messages, a messaging infrastructure for messages sent between infrastructure applications or devices, or other operational data.
  • FIG. 2A depicts one embodiment of a cloud environment 102.
  • the system 100 may include a cloud environment 102.
  • the cloud environment 102 may include one or more components that support cloud computing functionality or cloud storage functionality of the system 100 or another computer system.
  • the cloud environment 102 may include cloud storage that stores the dataspace.
  • the cloud environment 102 may include a Cloud Management service(CMS) 202.
  • CMS 202 may manage, oversee, or coordinate the operation of the one or more components of the cloud environment 102.
  • the CMS 202 may include a centralized mechanism for implementing a cloud computing policy that supports overall elasticity and multi-tenacity of the cloud of the overarching system.
  • a cloud computing policy may include a computing policy (e.g ., the policy may dictate how processes, tasks, jobs, etc. may execute) or a data access policy ⁇ e.g., the policy may dictate how data may be accessed, formatted, stored, managed, etc.).
  • a computing policy e.g ., the policy may dictate how processes, tasks, jobs, etc. may execute
  • a data access policy e.g., the policy may dictate how data may be accessed, formatted, stored, managed, etc.
  • the CMS 202 may manage one or more cloud environment 102 components as services.
  • the CMS 202 may manage a component of the cloud environment 102 by adapting hardware resource allocation to comply with demands of the system 100.
  • One example of software that may control operations of the cloud environment 102 may include HT-CONDOR (provided by the University of Wisconsin - Madison).
  • One example of software that may control monitoring the components of the cloud environment 102 may include NAGIOS (provided by Nagios Enterprises).
  • the CMS 202 may include data center management services or high-level management services.
  • High level management services may include dynamic execution of component services, execution of user tasks in adherence with the cloud computing policy, determining whether tasks adhere to the cloud data model, integration of cloud environment 102 components with security services (such as the components of the Systems Integrity Management Platform (SIMP) (provided by the U.S. National Security Agency), or other services.
  • Monitoring the cloud environment 102 may include monitoring and metering activities executing in the cloud environment 102.
  • monitoring the cloud environment 102 may include complying with one or more data standards.
  • a data standard may include a government standard (e.g ., a National Institute of Standards and Technology (NIST) standard, a Health Insurance Portability and Accountability Act (HIPAA) standard), an organizational standard ⁇ e.g., an International Organization for Standardization (ISO) standard, a Payment Card Industry (PCI) standard), or some other type of compliance standard.
  • NIST National Institute of Standards and Technology
  • HIPAA Health Insurance Portability and Accountability Act
  • ISO International Organization for Standardization
  • PCI Payment Card Industry
  • one or more components of the CMS 202 may execute outside the cloud environment 102. This may help promote reliable execution of the CMS 202. In some embodiments, to encourage reliability, one or more CMS 202 components may not depend on other CMS 202 components. In one embodiment, accessibility to the CMS 202 may be limited to the cloud owner and administrators. Access to different CMS 202 components may be configured based on user roles. In some embodiments, one or more CMS 202 components may include tools or application programming interfaces (APIs) that support distributed monitoring and logging of cloud environment 102 activities. This may improve system performance and allow dedication of system computing resources to other activities such as cloud computing and application execution.
  • APIs application programming interfaces
  • the CMS 202 may log one or more activities the occur in the cloud environment 102. This may allow security auditing, performance auditing, or other types of auditing. Logging the one or more activities may include storing data related to the activities.
  • the CMS 202 may be scalable. The scalability of the CMS 202 may be based on various factors. Such factors may include the number of host devices, the number of cloud compute nodes executable in the cloud environment 102, or other specifications.
  • the cloud environment 102 may include one or more cloud compute nodes (CCN) 204(1 )-(n).
  • a CCN 204 may include a physical computing device.
  • the CCN 204 may include a central processing unit (CPU), graphics processing unit (GPU), memory (such as random-access memory (RAM) or dynamic RAM (DRAM)), non-volatile storage (such as a hard disk drive, flash memory, or other storage).
  • a CCN 204 may include a software-implemented node executing on a host device.
  • a CCN 204 may include a virtual machine.
  • the CCN 204 may execute a job.
  • a job may include one or more logical process that execute on one or more CCNs 204(1 )-(n).
  • a job may be initiated by a user of the system 100 ( e.g ., from a command line interface).
  • a job may be initiated by another component of the system 100.
  • a job may include one or more subsystems, components, services, or the like described herein.
  • the CCN 204 may execute the job using the computing resources of a host device.
  • the CCN 204 may start, suspend, resume, checkpoint, migrate, or stop the job.
  • the CCN 204 may advertise the capabilities, hardware specifications, software specifications, or other information related to the host device to another component of the system 100, such as the CMS 202.
  • the CMS 202 may initiate execution of the CCN 204.
  • the CCN 204 may set up an execution environment on a host device, start a job, monitor the job’s execution, detect completion of the job, and send status information to the CMS 202.
  • setting up an execution environment may include the CCN 204 executing a starter service on the host device.
  • the starter service may be responsible for monitoring a single job.
  • the CCN 204 may be free from monitoring a job and may monitor the CMS 202 for new jobs.
  • the cloud environment 102 using multiple CCNs 204(1 )- (n) may allow the components of the cloud environment 102 to coordinate multiple user jobs even if those user jobs (including the one or more processes that make up each job) have differing priorities that may change over time. Coordinating multiple jobs may include setting up, executing, suspending, checkpointing, migrating, or stopping a job.
  • the CCN 204 may be scalable to multiple host devices. A job running on a CCN 204 may include an application with distributed computing capability.
  • the cloud processing parallelization service 212 and distributed state coordination service 318 may provide some distributed state or parallel processing logic or support to the CMS 202 or one or more CCNs 204(1 )- (n).
  • the cloud environment 102 may include a cloud unstructured storage service (CUSS) 206.
  • the CUSS 206 may provide a scalable data storage service across one or more nodes.
  • the CUSS 206 may execute on a cloud storage node (CSN), which may be a specific type of CCN 204.
  • the CUSS 206 or the CSN may execute as a job on a host CCN 204.
  • the CSN executing the CUSS 206 may include computer storage.
  • the computer storage may include directed attached storage of the host device, such as a hard drive, solid-state drive, optical disc drive, or an external drive.
  • the computer storage may include storage accessed over a computer network.
  • the CUSS 206 may use the computer storage of the CSN to store data for the dataspace.
  • a CUSS 206 may include the HADOOP distributed file system (provided by Apache).
  • the CUSS 206 may include a distributed file system.
  • the CUSS 206 may include a master server and one or more chunk servers. Each of the master server and the chunk servers may execute and store data on their own CSN.
  • each server master or chunk
  • each server may use its respective CSNs’ computer storage to host the distributed file system.
  • Each CSN may be responsible for managing a portion of the distributed file system.
  • Each CSN may contribute computing resources that support the distributed file system such as the CSN’s local disk, network bandwidth, or processing power.
  • the CUSS 206 may add additional CSNs or remove CSNs to scale the distributed file system.
  • dataspace contents may be stored in the storage provided by the CUSS 206.
  • Applications executing the cloud may have access to at least a portion of the dataspace contents.
  • the data storage associated with the CUSS 206 may include large scale, highly available, fast data storage and management.
  • An application may initially communicate with the master server to obtain distributed file system metadata to determine which chunk server(s) to communicate with.
  • the chunk server(s) may store the data for the application, which may reduce latency from having to communicate with the master server each time the application accesses data in the dataspace.
  • data may be replicated across multiple CSNs and coordinated by the CUSS 206. In some embodiments, data may be replicated across three CSNs. This may help data to be available even if a CSN fails, goes offline, or otherwise becomes unavailable.
  • components of the CUSS 206 may communicate with each other via the secure shell protocol (SSH).
  • SSH secure shell protocol
  • the cloud environment 102 may include a cloud structured storage service (CSSS) 208.
  • the CSSS 208 may provide scalable, distributed, persistent, or highly available logical structured storage.
  • the structured storage may include a database.
  • the CSSS 208 may execute on a CSN as part of a job.
  • the CSSS 208 may include ACCUMULO (provided by Apache).
  • the CSSS 208 may use the CUSS 206 to store data in a structured manner.
  • the CSSS 208 may include a master server and one or more tablet servers. Each of the master server and the tablet servers may execute on their own CSN.
  • the master server may coordinate the one or more tablet servers, mange the state of distributed CSSS 208 components, perform authentication and authorization checks, track the status of the tablet servers, coordinating tablet splits or migration, or other may perform other functionality.
  • the master server may assign data regions to tablet servers.
  • One region may include a top-level region that locates meta regions to be assigned.
  • a meta region may map a number of user regions that make up data tables that the CSSS 208 uses. After the meta regions have been assigned, the master server may assign user regions to tablet servers.
  • the master server may monitor the health of the one or more tablet servers. In response to the master server detecting that a certain tablet server is unavailable, the master server may split the unavailable server’s write-ahead log such that one write-ahead log exists for each region the table server used.
  • the master server may handle table administration functions, such as on or off-lining tables, changing a table schema, or other functions.
  • the master server may store a meta table.
  • the meta table may store information about each user region in the structured storage.
  • the information may include a region information object that may include information such as the start and end row keys, whether the region is online or offline, or the address of the table server that uses the region.
  • the master server may include a top-level table.
  • the top-level table may include a single region and may map the regions in the meta table.
  • the top- level table may include region information objects for each meta region.
  • the tablet server may handle CSSS 208 read and write requests.
  • the request may be written to a write-ahead log.
  • the tablet server may store the request in an in-memory cache.
  • the tablet server may check the memory cache for the requested data. If the requested data is not in the cache, the tablet server may search the distributed file system.
  • a tablet server may communicate with the master server to obtain a list of regions to use.
  • the tablet server may transmit its status to the master server.
  • a status message from the master server may include region assignments or other instructions.
  • the cloud environment may include a cloud access point node (CAPN) 210.
  • the CAPN 210 may include a specialized type of CCN 204.
  • the CAPN 210 may run as a job on a CCN 204.
  • the CAPN 210 may provide a secured entry point or interface into the cloud environment 102.
  • the CAPN 210 may assist human interaction with the cloud environment 102.
  • the CAPN 210 may support different interface formats such as web technologies, web services, or Web 2.0 technologies.
  • the CAPN 210 may implement one or more interfaces for accessing the cloud environment 102.
  • the CAPN 210 may communicate over a data network with a user’s computing device via hypertext transfer protocol (HTTP) or HTTP secure (HTTPS).
  • the user’s computing device may include a browser or other user interface that may receive HTTP data, and the browser may send HTTP data to the CAPN 210.
  • the CAPN 210 may user the received HTTP data to manipulate components of the cloud environment 102, such as the cloud storage.
  • the CAPN 210 may interact with one or more cloud security components for authentication, authorization, or audit functionality to control access to the cloud environment 102.
  • an interface for accessing the cloud environment 102 may include a representational state transfer (REST) interface.
  • REST representational state transfer
  • the CAPN 210 may include an access point service.
  • the access point service may include an HTTP web server or servlet container.
  • the access point service may include a web desktop or web operating system.
  • the access point service may include one or more software components that provide an interface for accessing the cloud environment 102.
  • the cloud environment 102 may include a cloud processing parallelization service (CPPS) 212.
  • the CPPS 212 may parallelize the processing of a file or data in a dataspace.
  • the CPPS 212 may leverage processing resources available across multiple CCNs 204(1 )-(n) or CSNs.
  • the CPPS 212 may include a job tracker component.
  • the job tracker component may schedule a job’s component tasks on one or more CCNs 204(1 )-(n), monitor the tasks statuses, and re-execute a task in response to the task failing.
  • the job tracker may schedule tasks according to a first-in-first-out strategy, a priority strategy, or another scheduling strategy.
  • the job tracker component may include a job configuration component.
  • the job configuration component may determine the number of CCNs 204(1 )-(n) to use to execute a job.
  • the job configuration component may determine the number of CCNs 204(1 )-(n) to use by determining the amount of data to be processed for the job, and calculating a number of splits on the data based on a number of task tracker services.
  • the CPPS 212 may include a task tracker service.
  • a task tracker service may be in data communication with the job tracker component.
  • the task tracker service and the job tracker component may communicate over SSH or inter-process communication data.
  • a task tracker may execute an application on a CCN 204 or a virtual machine hosted on a CCN 204.
  • the task tracker service may receive data indicating which split of the data to be processed the task track may work on.
  • the CPPS 212 may include the HADOOP software’s MapReduce functionality.
  • the cloud environment 102 may include a cloud monitoring service 214.
  • the cloud monitoring service 214 may provide one or more manageability functions for the cloud environment 102.
  • the cloud monitoring service 214 may provide a unified or consistent presentation of the health of monitored components.
  • a monitored component may include a component of the cloud environment 102 such as a CCN 204, a CSN, an application running in the cloud environment 102 ( e.g ., on a CCN 204), infrastructure software, an operating system of one or more devices in the cloud environment 102, hardware, or a network devices.
  • the cloud monitoring service 214 may provide policy-based monitoring templates.
  • the cloud monitoring service 214 may distribute views of a presentation layer to various operating centers.
  • the cloud monitor service 214 may support input from other monitored environments such as security events.
  • the cloud monitoring service 214 may restrict what data a user may access based on a role of a user.
  • the cloud monitoring service 214 may provide a user interface for performing one or more management functions.
  • the cloud monitoring service 214 may control or change the behavior of a monitored component.
  • the cloud monitoring service 214 may alert an administrator or other user of the system 100 in response to an event occurring.
  • the cloud monitoring service 214 may gather performance and health data ⁇ e.g., resource utilization data) about monitored components.
  • the cloud monitoring service 214 may store performance or health data or provide reports based on the performance/health data.
  • the cloud monitoring service 214 may user third-party plugin applications.
  • the cloud monitoring service 214 may include a monitor component.
  • the monitor component may collect monitoring data from one or more monitored components through one or more APIs.
  • the monitor component may execute on a monitored device or on a separate device in the cloud environment 102.
  • the monitor component may forward collected monitoring data to a data collector component.
  • the monitor component may forward certain data, such as raw events to a data store.
  • An event engine component may process the raw events.
  • the cloud monitoring service 214 may include a data collector component.
  • the data collector component may collect monitoring data from one or more monitor components.
  • the cloud monitoring service 214 may include an event engine component.
  • the event engine component may receive data from a data collector component.
  • the event engine component may perform event filtering, aggregation, or masking.
  • Event filtering may include discarding events that may be irrelevant as based on a monitoring policy or monitoring configuration.
  • Event aggregation may include merging duplicate events ( e.g ., caused by network delays). Even masking may include discarding events pertaining to a cloud environment 102 component that may be downstream of a failed component.
  • the cloud monitoring service 214 may include an analysis component.
  • the analysis component may analyze dependencies between events, to detect whether an event can be explained by one or more other events.
  • a low service level event for the database may be explained by the overloading of the server.
  • the analysis may be based on a model of the cloud environment 102, a dependency graph of cloud environment components 102, or other data.
  • the data collector component or the event aggregation engine may send collected or processed data to a data store.
  • a user interface may read from the data store.
  • the cloud environment may include a cloud logging service 216.
  • the cloud logging service 216 may log events that occur in the cloud environment 102.
  • the cloud logging service 216 may include an logging agent component.
  • the logging agent component may execute on a CCN 204.
  • the logging agent component may collect log information from the CCN 204 or an application, service, or other software executing on the CCN 204.
  • the logging agent component may send log information to a logging collector component.
  • the logging collector component may aggregate received log information and store the collected information in a data store.
  • a user interface may read the collected data from the data store.
  • a cloud logging service 216 may include CLOUDERA FLUME (provided by Apache).
  • FIG. 3 depicts one embodiment of an information environment 104.
  • the system 100 may include an information environment 104.
  • the information environment 104 may include a workflow and processing management service (WPMS) 302.
  • the system 100 may support multiple business and management workflows.
  • a workflow may include one or more actions arranged in a control dependency direct acyclic graph.
  • a control dependency may include that a first action cannot execute until a second action is completed.
  • a business workflow may be related to ingesting data into the dataspace, enriching the data of the dataspace, exploiting the data, or executing various data management activities.
  • a management workflow may include a workflow related to operating, administrating, configuring, or monitoring the underlying infrastructure of the system 100.
  • the WPMS 302 may support a business or management process.
  • the WPMS 302 may collect analytics about an executing process.
  • the WPMS 302 may execute background data management tasks, which may include indexing or validating data.
  • the WPMS 302 may execute background system operation tasks such as creating backups, assessing security vulnerabilities, or determine computer cluster integrity.
  • the WPMS 302 may schedule some of these tasks.
  • the WPMS 302 may execute user-generated background tasks.
  • the WPMS 302 may support workflow specification, visualization, or execution.
  • the WPMS 302 may support task execution.
  • the WPMS 302 may assist integration with certain cloud services such as a distributed file system.
  • the WPMS 302 may assist in coordinating workflow execution, including simultaneous workflow execution.
  • the WPMS 302 may monitor and log certain portions of workflows.
  • the WPMS 302 may support workflow failure recovery.
  • the WPMS 302 may support multiple task triggers such as user-initiated, time of day, an event (e.g ., a file arriving at the system 100), a timer expiring, or other triggers.
  • the WPMS 302 may include a workflow coordinator.
  • the workflow coordinator may control workflow submissions, resubmissions, monitoring, or cancellations.
  • the workflow coordinator may control one or more workflow nodes.
  • a workflow node may include a control flow node or an action node.
  • a control flow node may define the beginning and end of a workflow and may control the workflow execution path.
  • An action node may trigger the execution of a processing task.
  • the information environment 104 may include an application and system integration service (ASIS) 304.
  • the ASIS 304 may integrate applications executing within the system 100.
  • the ASIS 304 may integrate the system 100 with external systems.
  • the ASIS 304 may execute as a job on a CAPN 210.
  • the ASIS 304 may receive external data from outside the system 100 via the one or more interfaces of the CAPN 210 such as the HTTPS interface. An external system or a user may send the data to the ASIS 304 via the interface.
  • the ASIS 304 may include a file transfer service (FTS).
  • FTS file transfer service
  • the FTS may distribute files across various network configurations.
  • the FTS may receive data from outside the system 100.
  • the FTS may send data from the system 100 to an external system.
  • the FTS may support different protocols designed for different network situations.
  • the FTS may support file or data compression, bandwidth limiting, or a secure transfer protocol (e.g ., a protocol that uses encryption such as Advanced Encryption Standard (AES)).
  • AES Advanced Encryption Standard
  • the FTS may execute as a background service or daemon.
  • the FTS may execute a transmit component that sends files or a receiving component that receives files.
  • configurations for the FTS may be stored in an extensible markup language (XML) file.
  • XML extensible markup language
  • One example of an FTS may include the SafeMove software (provided by the United States government).
  • the ASIS 304 may interact with a dataspace management service (DSMS) 306 to move the external data into the dataspace.
  • the DSMS 306 may include logic and persistence data that implements one or more abstractions.
  • the one or more abstractions may be defined in a data description framework (DDF).
  • the DSMS 306 may operate as an abstraction layer on the dataspace.
  • the DSMS 306 may implement some services associated with relational database management such as automatic generation of indices, inverted indices, referential integrity determination, range determination for certain data columns.
  • the DSMS 306 may include functions to access the dataspace. These functions may include inserting data into the dataspace, reading the dataspace, modifying data in the dataspace, or removing data from the dataspace. In some embodiments, inserting data into the dataspace may include the DSMS 306 receiving the external data from the ASIS 304 and writing the received data to the cloud unstructured storage via the CUSS 206.
  • the system 100 may receive data from an import/export service 308.
  • the import/export server 308 may receive data that has already been formatted, indexed, or otherwise processed such that the data is ready to be stored in a dataspace without further processing or with minimal further processing.
  • the import/export service 308 may interact with the DSMS 306 to move the imported data into a dataspace of the system 100.
  • the information environment 104 may include a tactical edge service (TES) 310.
  • the TES 310 may provide a user access to dataspace data when the user may be unable to access the cloud environment 102 or may experience network connectivity difficulty in accessing the cloud environment 102.
  • the TES 310 may include a proxy service.
  • the proxy service may intercept requests between the user and the cloud environment 102.
  • the proxy service may use a caching strategy to store data locally (e.g ., in the local file system of the device executing the tactical edge application or a shared file server).
  • the proxy service may cache frequently accessed data, recently accessed data, or other cloud data according the caching strategy.
  • the proxy service may cache assertions created by the user or data ingested into the application at the direction of the user.
  • a user when network connectivity is deteriorated, a user may create asynchronous search jobs.
  • the proxy service may execute an asynchronous search job created by the user as the deteriorated network connectivity may allow.
  • the proxy service may receive the search results and present them to the user.
  • the proxy service may provide cached data to the user.
  • the proxy service may receive ingest data or assertions from the user to asynchronously add to a dataspace.
  • the user may execute browsing jobs, search jobs on the cached data of the tactical edge application’s host device.
  • the user may insert assertions into the local data or send data to the proxy service for ingestion and the proxy service may ingest the data into the cached data.
  • the proxy service may forward cached assertions or data for ingestion to the cloud environment 102.
  • the information environment 104 may include an artifact content indexing service (ACIS) 312.
  • ACIS 312 may create an index for a data artifact or other data ingested into the dataspace.
  • the ACIS 312 may search the dataspace for data artifacts using one or more indices.
  • the one or more functions of the ACIS 312 may be implemented by one or more components of the ACIS 312.
  • one or more components of the ACIS 312 may each execute as a job on a CCN 204.
  • the CMS 202 may manage each ACIS 312 component executing on a CCN 204.
  • the ACIS 312 may include an artifact indexer component.
  • the artifact indexer may include a long-running, standalone software application.
  • the artifact indexer may wait for a notification that a data artifact is ready to be indexed.
  • the DSMS 306 in response to receiving a data artifact into the mission dataspace 120 via one or more instances of the server ingest application 402 (discussed below), may send the notification to the artifact indexer.
  • the data artifact may be stored in the data artifact storage segment 128 or the data artifact semantics segment 122 of the mission dataspace 120 using the CSSS 208.
  • the notification may include an identifier that may specify the data artifact to be indexed, its location in the mission dataspace 120, or other identifying information.
  • the artifact indexer may access the data artifact to be indexed via the DSMS 306.
  • the artifact indexer may index the data artifact. Indexing the artifact may include creating an alphabetical list of words, phrases or numerical information with references to where they occur in data artifacts. Indexes may also be inverted, with references being listed to one or more words, phrases or numerical information.
  • LUCENE provided by Apache
  • a standardized programmatic library for indexing may be used to be standardized programmatic library for indexing.
  • the ACIS 312 may include an index publisher component.
  • the index publisher may publish an indexing file generated by one or more artifact indexers.
  • the index publisher may publish the indexing file in response to a predetermined amount of time elapsing since the indexing file was created.
  • the index publisher may publish the indexing file in response to the indexing file reaching a predetermined size.
  • the predetermined amount of time or file size may be configurable by a user or a component of the system.
  • publishing the indexing file may include the index publisher copying the indexing file from the local storage of the host device to a predetermined area of the CUSS 206.
  • the indexing file on the local storage may be deleted by the index publisher, artifact indexer, or other component.
  • the ACIS 312 may include an index searcher component.
  • An index publisher may notify an index searcher that an indexing file has been copied to the CUSS 206.
  • the index searcher may use the copied indexing file to service an index search request.
  • An index searcher may include SOLR (provided by Apache).
  • the ACIS 312 may include an index manager component.
  • the index manager may execute as a long-running job on a CCN 204.
  • the index manager may merge multiple unmerged indexing files that have been published by one or more index publishers.
  • the index manager may merge the unmerged indexing files into one or more resulting merged files.
  • the index manager may apply heuristics or logic to merge the multiple unmerged indexing files.
  • the index manager merging the unmerged indexing files may include copying the unmerged indexing files of the CUSS 206 to the local storage of the host machine executing the index manager.
  • Merging the unmerged indexing files may include the index manager merging the copied unmerged indexing files on the host machine. Merging the copied unmerged indexing files may include using MapReduce.
  • the index manager may copy the resulting one or more merged indexing files to the CUSS 206.
  • the copying back to the CUSS 206 may include some cleanup logic of the previously unmerged indexed files.
  • the index manager may delete the copied unmerged indexing files or the resulting merged indexing files from the local storage of the host machine executing the index manager.
  • an index searcher may copy one or more resulting merged indexing files from the CUSS 206 to the local storage of the host machine executing the index searcher.
  • searching a data artifact in the mission dataspace 120 may include receiving a search request.
  • the DSMS 306 may send the search request to the index searcher.
  • the DSMS 306 may receive a search request from a user (e.g ., via the CAPN 210), external system ( e.g ., via the ASIS 304), or from a job executing within the system 100 ( e.g ., via an instance of the dataspace navigation and query application 506).
  • the search request may include search criteria. Searching a data artifact may include an index searcher master receiving the search request.
  • the index searcher master may obtain a list of index searcher workers, which may include one or more jobs executing one or more CCNs 204(1 )-(n).
  • One or more index searcher workers may include one or more indexing files stored on the local storage of the host device executing the index searcher worker.
  • Searching a data artifact may include an index searcher worker searching the one or more indexing files of its local storage.
  • Searching a data artifact may include the index search worker sending search results to the index worker master.
  • the index worker master may merge the search results and return them to the DSMS 306.
  • the DSMS 306 may apply filtering to enforce one or more authorization policies.
  • the filtering may remove search results that the requester is not authorized to view or access.
  • the ACIS 312 may include an index validator component.
  • the index validator component may compare contents of an indexing file to one or more documents in the artifact semantics segment 122 of the mission dataspace 120. If a comparison of the contents of an indexing file to a document determines that the document has not been indexed or has been index improperly, the index validator may notify another component of the ACIS 312, such as an artifact indexer, to index the document.
  • multiple instances of the index manager may execute.
  • the index manager may merge multiple unmerged indexing files into a single indexing file or into multiple indexing files.
  • the index manager may merge different parts of different unmerged indexing files into the one or more merged indexing files.
  • multiple instances of the artifact indexer may index a data artifact.
  • the CMS 202 may start, monitor, or stop the instances.
  • each instance may include different functionality to index the data artifact.
  • the information environment 104 may include a data surveillance service (DSS) 314.
  • the DSS 314 may analyze ingested data and search for patterns in the data of the mission dataspace 120.
  • the DSS 314 may generate data based on an observed pattern.
  • the DSS 314 may store the generated data in the mission dataspace 120.
  • a user or application may access the stored generated data.
  • the DSS 314 may push the generated data to a user or application.
  • the DSS 314 may include a data watcher component.
  • the data watcher component may observe changes to the mission dataspace 120.
  • the data watcher component may include a pattern definition configuration.
  • the pattern definition configuration may define a pattern the data watcher is to look for.
  • the data watcher may report changes in the dataspace 120 according to the pattern definition.
  • the data watcher component may report changes to a surveillance engine component.
  • the surveillance engine component may identify a pattern in data received from a data watcher component.
  • the data received may include an event that occurred in the system 100.
  • the surveillance engine component may publish the event to other components of the system 100, notify other components, or initiate the execution of another application of the system 100.
  • the information environment 104 may include a web infrastructure service (WIS) 316.
  • the WIS 316 may include one or more web servers.
  • a web server may act as an entry point into the cloud storage of the cloud environment 102.
  • a web server may accept HTTP requests from applications (such as a web browser) and server HTTP responses.
  • One example of a web server may include the APACHE HTTP server provided by Apache.
  • the WIS 316 may include one or more application servers.
  • An application server may host or execute server-side applications (such as the server ingest application 402 described below).
  • An example application server may include TOMCAT (provided by Apache).
  • the WIS 316 may include one or more portal servers.
  • a portal server may include a platform for developing, delivering, or maintaining a web portal that may support personalizing user content or for aggregating information from various web applications.
  • the WIS 316 may include one or more directory servers.
  • a director server may store profiles of users of the system 100, definitions of which users can access which environments, applications, services, or the like in the system 100, or other user information.
  • the information environment may include a testing service 318.
  • the testing service 318 may validate the behavior of one or more components of the system 100 to determine whether the component is behaving properly based on testing criteria.
  • the testing service 318 may provide an environment in the system 100 to test the one or more components.
  • the testing service 318 may evaluate component behavior and performance when data is loaded or enriched. The testing service 318 may determine whether software or system upgrades or modifications function properly. The testing service 318 may test component behavior in response to the detection of a fault in the system 100. The testing service 318 may execute tests to monitor the health of a system 100 component.
  • the testing service 318 may capture test output data for evaluation and comparison.
  • the testing service 318 may play back preconfigured test scenarios.
  • the testing service 318 may generate sample data from existing data for use in testing.
  • the testing service 318 may maintain a library of tests, test drivers, testing data, or other testing information.
  • the information environment 104 may include a distributed state coordination service (DSCS) 320.
  • DSCS distributed state coordination service
  • One or more services or applications in the system 100 may include distributed computing capabilities.
  • the DSCS 320 may coordinate distributed computing information between distributed components. Coordinating distributed computing information may include sharing and broadcasting state information.
  • the DSCS 320 may be initiated and monitored by the CMS 202.
  • the DSCS 320 may use the CUSS 206 to persist state information.
  • Some applications and services that may include distributed computing capabilities may include the CSSS 208, the ACIS 312, or the SIA 402.
  • the DSCS 320 may use one or more security environment 1 12 (discussed below) components to enforce security.
  • the DSCS 320 may include one or more distributed coordination processes to coordinate state among each other. Each distribute coordination process may know about the other distributed coordination process. State information stored by a distributed coordination process may be stored by the CUSS 206 in the operational dataspace. Each distributed coordination process may include a coordination component. The coordination component may coordinate traffic between the multiple distributed coordination processes.
  • a distributed application or service may be in data communication with a distributed coordination process. The data communication may be through a transmission control protocol (TCP) connection.
  • TCP transmission control protocol
  • the distributed application or service may send requests to interact with other distributed applications through their respective distributed coordination processes. The requests may include reading the state of the distributed process, updating the state, or watching the state for changes.
  • the information environment 104 may include a domain name system (DNS) service or a network time protocol (NTP) service.
  • DNS domain name system
  • NTP network time protocol
  • the DNS service may include a hierarchical naming system and may translate domain names into numerical identifiers compatible with the components of the system 100.
  • the NTP service may synchronize the clocks of the devices of the system 100.
  • FIG. 4A depicts one embodiment of a server-side environment 106.
  • the system 100 may include a server-side environment 106.
  • the server-side environment 106 may include one or more software applications that may carry out functions of the system 100.
  • the server-side environment 106 may include a server ingest application (SIA) 402.
  • the SIA 402 may process data stored in the mission dataspace 120.
  • the SIA 402 may process data in various formats such as unstructured data, semi-structured data, or structured data. In some embodiments, different implementations of the SIA 402 may process different types of data.
  • Structured data may include data stored in a relational database; data that adheres to a rigid data or object model; or data that conforms to a semantic ontological representation.
  • Semi-structured data may include text files with structured elements such as tables or lists of information; emails that contain structured header information and unstructured subject or body contents; hypertext markup language (HTML) files that conform to a particular HTML specification and also contain natural language or imagery; extensible markup language (XML) files that contain data segments that are unstructured; or other data.
  • Unstructured data may include imagery, including pictures, geospatial imagery, full motion video, or other imagery related information; data represented in an unknown binary format; audio data represented in a digital form; or other data.
  • the SIA 402 may write certain metadata to the data semantics segment 124.
  • the SIA 402 may write disambiguated entities (or terms) into the data semantics segment 124.
  • the SIA 402 may process unstructured data. Processing unstructured data may include the SIA 402 writing the data artifact containing the unstructured data to the data artifact semantics segment 122.
  • the SIA 402 may remove duplicated data (e.g ., one or more data artifacts being ingested or included in a segment of the mission dataspace 120, where the removed data artifact may contain the same content).
  • the SIA 402 may write data artifact metadata to the data artifact semantics segment 122.
  • the data artifact metadata may include metadata about the data artifact such as a file name, size, data source, date created or modified, or other metadata.
  • the SIA 402 may write disambiguated data elements into the data semantics segment 124.
  • the SIA 402 may process structured data.
  • the SIA 402 processing structured data may include the SIA 402 accessing a relational database system catalog to determine structure of entities being processed.
  • the SIA 402 may write the processed structured data into the data semantics segment 124.
  • the SIA 402 may write the data based on data from the catalog, such as schema, datatype, column, or other structure data.
  • the SIA 402 may write source model elements and corresponding metadata into the data model semantics segment 126 or may write source data and corresponding metadata into the data semantics segment 124.
  • the SIA 402 may maintain one or more performance metrics for processing the structured data or may maintain health data corresponding to ingested data.
  • the SIA 402 may process semi-structured data.
  • Processing semi-structured data may include a combination of processing methods for structured and / or unstructured data, where the semi-structured data is deconstructed into its structured and unstructured elements for processing. For example, if a data artifact comes from a semi-structured data source, the SIA 402 may write source-specific metadata (e.g ., for an email, the “from” tag) into the data semantics segment 124.
  • source-specific metadata e.g ., for an email, the “from” tag
  • the SIA 402 may execute in response to the WPMS 302 executing a workflow that includes the SIA 402 or may executed in response to the CMS 202 executing the WPMS 302 on a CCN 204. Different components of the WPMS 302 may execute on different CCNs 204(1 )-(n).
  • the SIA 402 may implement a Pipe-and-Filter architecture.
  • This SIA 402 may include a pump component.
  • the pump may include a data source or may receive data from a data source.
  • the data source may include the FTS of the ASIS 304, a real-time data feed, or a remote system via an integration bridge.
  • the pump may monitor one or more data sources for available data.
  • the pump may send data to one or more pipe components.
  • the SIA 402 may include a pipe component.
  • a pipe may receive data ⁇ e.g., from the pump or a filter). The pipe may buffer received data until a downstream component ⁇ e.g., a filter) is ready to process the data.
  • a pipe may receive data from one or more components of the SIA 402.
  • a pipe may send data to one or more components of the WPMS 302.
  • the SIA 402 may include a filter component.
  • the SIA 402 may include multiple filter components that execute simultaneously.
  • a filter may wait for data to arrive or may periodically poll another component for available data.
  • a filter may read data from one or more input pipes, perform its function of the data, and send the processed data to one or more output pipes.
  • the filter performing its function may include transforming or filtering the data.
  • Transforming or filtering the data may include message processing, parsing, or tokenizing.
  • Transforming or filtering the data may include populating a portion of the dataspace or indexing the data.
  • a filter may include a long-running logical process that may be started or monitored by the CMS 202.
  • the SIA 402 may include a sink component.
  • the sink may send processed data to a data target.
  • the data target may include a file, database, or an application.
  • each of the pump, pipe, filter, and sink components may execute on a CCN 204 and managed by the CMS 202
  • the SIA 402 may scale to respond to differing data ingest conditions. For example, in response to the system 100 receiving large numbers of external files to be placed in the dataspace, the SIA 402 may coordinate with the CMS 202 to execute multiple pumps, filters, pipes, and sinks to process the external data. Multiple instances of filters with the same functionality may execute to process the data faster than one filter with that functionality.
  • the SIA 402 may implement a new type of pump component or filter component to process the file.
  • the SIA 402 (or another component) may split a file to be ingested into multiple pieces so that the components of the SIA 402 may process different parts of the file simultaneously.
  • the server-side environment 106 may include a data enrichment analytic application 404 (hereinafter,“enrichment application 404”).
  • the enrichment application 404 may process a previously ingested data artifact.
  • the enrichment application 404 processing the data artifact may include extracting data elements from the data artifact and processing the data elements. Processing the data elements may include locating mentions, generating signs based on mentions, generating concepts based on source data models, disambiguating mentions using one or more concepts and generating terms based on the disambiguating, associating one or more data elements from the data artifact with signs, concepts, terms, or statements already in the mission data space 120.
  • the enrichment application 404 may apply syntactic, semantic, and discourse processing to a data artifact or data element.
  • the data artifact may be stored in the artifact semantics segment or the artifacts storage segment.
  • the enrichment application 404 may identify an assertion about data elements, such as entities, events, relations, qualifications, or other factual information included in the data elements of the data artifact.
  • the enrichment application 404 may store such data elements in the data semantics segment of the mission dataspace.
  • the enrichment application 404 may use statistical language models, parameter files, gazetteers, artificial intelligence models, machine learning models, or other processes to enrich data in the dataspace.
  • the enrichment application 404 may identify information about a data artifact such as a classification of the data artifact based on the data elements of the data artifact, may identify whether the data artifact includes information that is relevant to a certain topic, or identify whether the data artifact deserves further, more detailed analysis.
  • the enrichment application 404 may process a data artifact that is a text file.
  • the enrichment application 404 may extract strings of text from the text file as mentions.
  • the enrichment application 404 may determine whether a sign corresponding to the mention is already in the mission dataspace 120. If it is not, the enrichment application 404 may add the sign to the mission dataspace 120.
  • the enrichment application 404 may disambiguate each mention by associating each mention in the text file with a concept.
  • the enrichment application 404 may disambiguate a mention using semantic analysis, machine learning, artificial intelligence (Al), or other techniques.
  • the enrichment application 404 may generate one or more statements based on the disambiguating or other data.
  • the enrichment application 404 may generate metadata about the generated data elements from the text file.
  • the enrichment application may store generated data elements or metadata in the mission dataspace 120.
  • the server-side environment 106 may include a data exploitation analytic application 406 (hereinafter,“exploitation application 406”).
  • the exploitation application 406 may read and analyze data from the data artifact semantics segment 122, data semantics segment 124, or data model semantics segment 126 of the mission dataspace.
  • the exploitation application 406 may analyze the data from these segments to derive one or more patterns in the data, discover data associations or relationships between data elements, and store these derived patterns or discovered associations in the data semantics segment 124 or the data artifact semantics segment 122.
  • the exploitation application 406 may combine or filter data extracted from multiple data artifacts to generate an assertion related to the data.
  • the exploitation application 406 may roll back data generated by the exploitation application’s 406 functionality.
  • the exploitation application 406 may derive a pattern or discover a data association or relationship between data elements of a group of data artifacts and may store data based on the pattern or association or relationship.
  • the exploitation application 406 may perform this functionality in response to a data artifact being added to the group or a data artifact in the group being modified or removed.
  • a data artifact in the group of data artifacts may be rolled back to a previous version or removed from the group or dataspace.
  • the exploitation application 406 may roll back the data based on the pattern, association, or relationship to a previous version of the data that pertains to the rolled back version of the data artifact or group.
  • some of the enrichment or exploitation functionality described in relation to the enrichment application 404 or exploitation application 406 may be performed by the SIA 402 ( e.g ., as part of a filter component). Some of the SIA 402 functionality may be performed by the data enrichment application or exploitation application 406.
  • FIG. 4B depicts one embodiment of a portion 400 of the system 100.
  • the system 100 may include the ASIS 304, the SIA 402, the DSMS 306, and the mission dataspace 120.
  • the SIA 402 may include a plurality of filter components 412.
  • the plurality of filter components 412 may include a first filter 412(1 ), a second filter 412(2), a third filter 412(3), and a fourth filter 412(4).
  • the arrows of FIG. 4B depict an exemplary flow of data between the exemplary components of the system 100.
  • the ASIS 304 may receive a data artifact from an external data source 410.
  • the ASIS 304 may send the received data artifact to the SIA 402.
  • the SIA 402 may send the data artifact to the first filter 412(1 ).
  • the first filter 412(1 ) may generate metadata about the data artifact.
  • the first filter may send the generated metadata to the DSMS 306, which may store the metadata in the mission dataspace 120 in the artifact semantics segment 122.
  • the first filter 412(1 ) may send the data artifact to the second filter 412(2).
  • the second filter 412(2) may remove duplicate data from the data artifact.
  • a filter 412 that removes duplicate data may not send data to the DSMS 306 as is shown in FIG. 4B.
  • the second filter 412(2) may send the data artifact (without the duplicate data) to the third filter 412(3).
  • the third filter 412(3) may process structured data in the data artifact.
  • the third filter 412(3) may generate data model metadata and send the data model metadata to the DSMS 306 for storage in the data model semantics segment 126 of the mission dataspace 120.
  • the second filter 412(2) may also send the data artifact to the fourth filter 412(4).
  • the fourth filter 412(4) may generate disambiguated data elements and store them in the data semantics segment 124 of the mission dataspace 120 via the DSMS 306.
  • FIG. 5A depicts one embodiment of a mission application environment 108.
  • the system 100 may include a mission application environment (MAE) 108.
  • the MAE 108 may include one or more mission applications.
  • a mission application may include an application executing on one or more CCNs 204(1 )-(n).
  • a mission application may include an application that interacts with the mission dataspace 120.
  • a mission application may include an application that may be generated, initiated, or used by a user of the system 100.
  • the MAE 108 may include a user-generated analytics application 502.
  • the user-generated analytics application may process data in the mission dataspace 120, read and analyze the data, and may derive patterns or discover relationships based on a configuration of the user-generated analytics application 502.
  • the MAE 108 may include a dataspace access interface (DAI) 504.
  • the DAI 504 may include an interface used by an end user for accessing information in the mission dataspace.
  • the DAI 504 may include an application that interacts with the DSMS 306 to access the data in the mission dataspace. Accessing the data in the mission dataspace may include reading data from and writing data to the mission dataspace.
  • the DAI 504 may execute on a CCN 204.
  • the CAPN 210 may redirect a request for interact with the mission dataspace to the DAI 504.
  • the DAI 504 may include a REST interface.
  • the DAI 504 may initiate a plurality of REST nodes on a plurality of CCNs 204(1 )-(n).
  • the CMS 202 may manage the plurality of REST nodes.
  • the DAI 504 may include a load balancing component.
  • the load balancing component may direct requests to the DAI 504 to a particular REST node.
  • the DAI 504 may include an HTTP level cache. Identical requests sent to the DAI 504 may be served from the HTTP level cache to reduce response times.
  • the MAE 108 may include a dataspace navigation and query application (DNQA) 506.
  • the DNQA 506 may include an application with a graphical user interface (GUI).
  • the GUI may present data in the mission dataspace 120 in a graphical way.
  • the GUI may allow a user to add data to or modify data in the mission dataspace in a graphical way.
  • the GUI may include a plurality of features, including data visualizations such as charts or graphs (e.g., 2D, 3D or higher scatter graph, Q-Q graph, T-SNE, Pivot chart, Correlation graph); network representations of data; displaying data overlaid on a map, image, video or audio recording; or other visualizations.
  • the GUI may include representations of characteristics of the data displayed as charts or graphs; statistics, or in plain form (e.g., results of a summation across all data features contained within a dataset, or a set of features contained with a search result).
  • the GUI may include features that enable a user to modify a set of data, such as removing duplicate data items; merging similar data items; filling, filtering or removing missing data features, or other data manipulations.
  • the GUI may include representations of data in its original form.
  • the GUI may include the representation of data sets created through the process of searching or querying across larger data sets; created through a process of data manipulation, possibly including joining or splitting one or more data sets; created through a process of data analysis; or other processes.
  • the GUI may include representations of process workflows and lifecycles that can be manipulated by the user, for example, executing the training, testing or serving of neural network models in the platform.
  • FIG. 5B depicts one embodiment of a GUI.
  • the GUI may include a GUI of the DNQA 506 or the GUI may be part of another component of the system 100.
  • the GUI may include one or more interaction components that may allow a user of the system 100 to interact with the data in the mission dataspace 120, search the mission dataspace 120, run analytics on data in the mission dataspace 120, execute Al, machine learning, or data models, or control other functionality of the system 100.
  • the GUI may include a visualization view 550 as shown in FIG. 5B.
  • the visualization view 550 may include a graph 552.
  • the graph 552 may plot data based on data in the mission dataspace 120.
  • the visualization view 550 may include a plurality of plot selection buttons 554.
  • the plurality of plot selection buttons 554 may allow the user to select which kind of graph to display.
  • the kinds of graphs may include a table, a scatter plot, a parallel coordinate plot, a histogram, a box plot, or another type of graph.
  • the visualization view 550 may include a plurality of view configuration buttons 556.
  • a view configuration button may include a button to re-center, reset, or rotate the view of the graph 552.
  • the visualization view 550 may include a plurality of axis selection components 558.
  • An axis selection component may include a drop-down box or other selection mechanism to select what data is associated which axis of the graph 552.
  • FIG. 5C depicts one embodiment of the GUI.
  • the GUI may include a data view 560.
  • the data view 560 may include the graph 552.
  • the data view 560 may include a statistics widget 652.
  • the statistics widget 562 may include a plurality of buttons that allow the user to select which statistic to display about the graph 552, the data associated with the graph 552, or a dataset in the mission dataspace 120.
  • a statistic may include a sum, a standard deviation, a mean, a maximum, a minimum, or another statistic.
  • the statistic widget may display the statistic for one or more dimensions of the graph 552, dataset, or other data.
  • the data view 560 may include an exploratory data analysis (EDA) operations (OPS) widget 564.
  • EDA exploratory data analysis
  • OPS exploratory data analysis
  • the EDA OPS widget 564 may include buttons, drop down boxes, or other interactive components that may allow a user to manipulate the data of the graph 552, the dataset, or other data.
  • the EDA OPS widget 564 may allow a user to filter data based on a value of the data (e.g ., the value being greater than, less than, or equal to a user-selected value) (in FIG. 5C, the“Filter” button), fill empty values with a specified value (“FillNA”), drop data containing an empty value (“DropNA”), mask data that satisfy a user-specified criteria (“Mask”), drop duplicate data, or partition data based on a user-specified criteria.
  • FIG. 5D depicts one embodiment of a query view 570.
  • the query view 570 may allow a user to search the mission dataspace 120, the data being visualized by the graph 552, or other data.
  • the query view 570 may include the graph 552.
  • the query view 570 may include a plurality of query type selection buttons 572.
  • a query type selection button may allow a user to select what kind of data to search.
  • the plurality of query type selection buttons 572 may include a button to search for a data source, search for a data facet, search by keyword, search by date, search by concept, or other search selection functionality.
  • the query view 570 may include a search bar 574. The user may type his or her query into the search bar 574.
  • the query view may include a search suggestion dropdown 576.
  • the search suggestions dropdown 576 may auto-populate with suggested search queries based on what the user has typed in the search bar 574.
  • the query view 578 may include a related content configuration bar 578.
  • the user may select, in the related content configuration bar 578, a threshold cutoff for computing the distance to related content from the selected content. For example, as depicted in FIG. 5D, the user may have selected the“heart rate” concept from the search suggestions dropdown 576.
  • Other concepts may be related to the concept of “heart rate” and the other concepts may each include a calculated distance from the “heart rate” concept. In one embodiment, a higher distance value may indicate that the concept is more closely related.
  • the query view 570 may include a related content search result list 579.
  • the related content search result list 579 may display the results of the related content search.
  • FIG. 5E depicts one embodiment of a models view 580.
  • the models view 580 may include a model selection widget 582.
  • the model selection widget 582 may display the name of an Al, machine learning, or data analytic model that can be used to execute on selected data. The user may select a button in the model selection widget 582 to select which model to execute.
  • the models view 580 may include a dataset selection widget 584.
  • the dataset selection widget 584 may display available datasets that can be used with a model for training, inference calculation, or other model functionality.
  • the dataset selection widget 584 may display a configuration for the dataset.
  • a configuration may include a configuration about dropped values, masked values, filtered values, filled values, or other configurations. The configurations may be based on the EDA OPS widget 564.
  • the model view 580 may include a console display 586.
  • the console display 586 may display one or more lines of computer code that is executing, has been executed, or may be executed.
  • the console display 586 may allow a user to view the computer code for debugging purposes or some other purpose.
  • the model view 580 may include a start training button 588. A user may click the button to start the training of the Al, machine learning, or data analytics model. In some embodiments, the start training button 588 may change to a stop training button after the training begins.
  • the MAE 108 may include an assertion application 508.
  • the assertion application 508 may manage one or more assertions of the mission dataspace.
  • an assertion may include a computer program or service, a set of program instructions, or other computer functionality.
  • the assertion may determine read in data and determine whether the data conforms to a certain condition.
  • the assertion may perform an action.
  • the action may include rejecting the data, notifying another application, service, or other component of the system 100, or another type of action.
  • a user may insert, modify, or remove an assertion via a CAPN 210.
  • the CAPN 210 may provide an interface to the user based on the location of the assertion. For example, if the mission dataspace includes the assertion, then the DNAQ 506 or DAI 504 may provide the assertion insertion/modification/removal functionality.
  • the DSMS 306 may provide an interface for the mission dataspace to the CAPN 210, DNAQ 506, or DAI 504.
  • another component of the system 100 may provide a user with an interface for assertions in the system 100.
  • the assertion may include an assertion on data being ingested into the mission dataspace.
  • a filter component of the SIA 402 may include an assertion.
  • the assertion may include a condition that structured data elements being ingested conforms to a certain data model.
  • the assertion may reject some of the ingested data and notify the user. Rejecting the data may include the data elements or a data artifact including those rejected data elements not being added to the mission dataspace.
  • an assertion may include an assertion on data in the mission dataspace.
  • the assertion may include a condition pertaining to a certain data artifact or one or more data elements. If the data artifact or data elements are modified in the dataspace (e.g ., by a user via the DAI 504 or by another application executing on the system 100) or an attempt to modify the data artifact or data elements is made, the assertion may determine whether the modification conforms with the one or more conditions of the assertion. In response to the modified data not conforming to one or more conditions of the assertion, the assertion may reject the modifications.
  • the enrichment application 404 may add an assertion to the mission dataspace.
  • the assertion may be associated with a disambiguated data element, metadata, a data model, a concept, or some other type of data element.
  • An assertion may be any information associated with a data element. Assertions may be based on implicit or explicit claims or representations of a data element.
  • An assertion may be determined through a process of data extraction, enrichment, enhancement, analysis by mathematical or algorithmic program, or other forms of data analysis.
  • An example of an assertion, for the data term“aspirin”, might be“medicine”.
  • the enrichment application may associate an assertion with data the enrichment application 404 may generate as part of its data enrichment functionality.
  • an assertion may be rolled back.
  • the assertion application 508 may roll back an assertion by returning the assertion to a previous version or by removing the assertion.
  • a previous version of the assertion may include one or more previous conditions that were later modified or removed.
  • the assertion application 508 may roll back an assertion in response to the data associated with the assertion being rolled back.
  • FIG. 6A depicts one embodiment of a management environment 1 10.
  • the system 100 may include a management environment 1 10.
  • the management environment 1 10 may include one or more applications that manage applications executing in the system 100.
  • the management environment 1 10 may include an analytic management application 602.
  • the analytic management application 602 may manage one or more analytic applications.
  • An analytic application may include the enrichment application 404, the exploitation application 406, or a user-generated analytic application.
  • the analytic management application 602 may manage the training of one or more analytical models.
  • An analytical model may include a statistical language model, parameter files, gazetteer, artificial intelligence model, machine learning model, or other type of model.
  • the analytic management application 602 may manage training data (e.g ., add new training data to a training data pool, remove outdated training data), execute a training functionality of an analytical model using the training data, or perform other training data functions.
  • the analytic management application 602 may configure an analytical model based on the deep learning optimization techniques described below. Feedback from an analytical model, configurations of an analytical model, or other data derived from the training process may be stored in the operational dataspace.
  • the management environment 1 10 may include a systems management application 604.
  • the systems management application 604 may gather and process data related to the health, status, and events of one or more components of the cloud environment 102.
  • the systems management application 604 may receive data from the CMS 202.
  • the management environment 1 10 may include a data management application 606.
  • FIG. 6B depicts one embodiment of a data management application 606.
  • the data management application 606 may include a dataset management component 610.
  • the dataset management component 610 may maintain a catalog of datasets.
  • the dataset management component 610 may maintain metadata about a source or nature of a dataset.
  • a dataset may include a standard dataset.
  • the dataset management component 610 may import a dataset into a dataspace.
  • the dataspace may include the mission dataspace or operational dataspace.
  • the dataset management component 610 may generate or update metadata based on the imported dataset.
  • the dataset management component 610 may manage modifications to a dataset.
  • the data management application 606 may include a model management component 612.
  • the model management component 612 may include a library of data models.
  • the model management component 612 may support change control of a data model and related metadata.
  • Related metadata may include a data dictionary.
  • the data management application 606 may include a data integrity component 614.
  • the data integrity component 614 may determine whether certain metadata is present and properly populated for a data artifact, data element, data model, or other data objects.
  • the data integrity component 614 may support maintenance or checking of data integrity in the mission dataspace.
  • the data integrity component 614 may include one or more assertions to perform its functionality.
  • the data management application 606 may include a metadata management component 616.
  • the metadata management component 616 may update metadata.
  • the metadata may include metadata about other metadata.
  • the metadata management component 616 may maintain a catalog of the metadata.
  • the metadata management component 616 may support predicate management.
  • the data management application 606 may include a vocabulary management component 618.
  • the vocabulary management component 618 may define a controlled vocabulary.
  • the vocabulary management component 618 may manage one or more roles or permissions for interacting with the controlled vocabulary. Such roles or permissions may indicate who can access different parts of the vocabulary, which users can modify the vocabulary, or other roles or permissions.
  • the vocabulary management component 618 may import portions of the controlled vocabulary from or export to external systems.
  • the vocabulary management component 618 may import or export via the ACIS 312, import component, or export component.
  • the vocabulary management component 618 may enrich metadata.
  • the data management application 606 may include a data inventory management 620 component.
  • the data inventory management component 620 may generate an inventory of data in a dataspace.
  • the data inventory management component 620 may collate metadata to show a user what a certain data element means or where the data element is located in a data artifact or cloud storage.
  • the data management application 606 may include a data support component 622.
  • the data support component 622 may maintain a knowledge base of data questions and answers or other information about data in a dataspace.
  • the data support component 622 may interact with a help desk component.
  • the data management application 606 may include a data governance component 624.
  • the data governance component 624 may define one or more procedures for handling data in a dataspace.
  • the data governance component 624 may store information on roles or responsibilities for data objects. A user may consult the stored information.
  • the stored information may include data trustees on business- related issues.
  • the data governance component 624 may provider management capabilities to permit communications to be coordinated among users for certain data objects.
  • the management environment 1 10 may include an operations management application 608.
  • the operations management application 608 may manage the one or more testing components of the system 100 (e.g ., the testing service 318).
  • the operations management application 608 may manage one or more security components (such as one or more components of the security environment 1 12).
  • the operations management application 608 may include a software configuration and deployment component (SCDC).
  • SCDC may determine whether a piece of software, such as an application (operational or mission) that executes in the system 100, is up-to-date and current in its configurations.
  • the SCDC may generate one or more reports based on the results of its determinations.
  • the SCDC may retrieve software updates ⁇ e.g., from an internal data store of the system 100 or an external system) and update one or more pieces of software using the retrieved updates.
  • FIG. 7 depicts one embodiment, of a security environment 1 12.
  • the system 100 may include a security environment 1 12.
  • the security environment 1 12 may include one or more services that may provider security functionality for the system 100.
  • the security environment 1 12 may include SIMP.
  • the security environment 1 12 may include an identity and access management service (IAMS) 702.
  • the IAMS 702 may enforce access control policies within the system 100.
  • the IAMS 702 may identify users based on a unique identifier ⁇ e.g., a username or digital certificate).
  • the IAMS 702 may associate a user with a set of access authorizations or privileges of the system 100.
  • the IAMS 702 may use authentication mechanisms ⁇ e.g., passwords or public key infrastructure (PKI)) to establish validity of a user’s identity.
  • PKI public key infrastructure
  • the system 100 may receive an access request from an external entity via the CAPN 210.
  • the system 100 may request a digital certificate from the external entity for authentication purposes.
  • An internal entity of the system 100 such as an application executing in the system 100, may request access and may be authenticated via supplying a user identifier to the IAMS 702.
  • the IAMS 702 may generate an audit record for each data access request.
  • the security environment 1 12 may include a security auditing service 704.
  • the security auditing service 704 may account for actions taken or attempted by an entity (e.g ., user, application, job, component). Such actions may include reading from or writing to the mission dataspace 120.
  • the security auditing service 704 may include similar functionality to the cloud logging service 216. However, the security auditing service 704 may configured to track and report security- related events and not events in general.
  • the security environment 1 12 may include a vulnerability assessment service 706.
  • the vulnerability assessment service 706 may discover, report, and mitigate security vulnerabilities of the system 100. Vulnerabilities may be associated with a physical device, piece of software, a component, an application, a service, or other piece of the system 100.
  • the vulnerability assessment service 706 may determine whether security patches have been applies to one or more components of the system 100.
  • the vulnerability assessment service 706 may perform malware scans, vulnerability scans, or other scans.
  • the vulnerability service 706 may report the results of a scan.
  • the security environment 1 12 may include a threat management service 708.
  • the threat management service 708 may process threats to the system 100 such as viruses, worms, trojans, hacking attempts, or misuse of system 100 components.
  • the threat management service 708 may provide security monitoring, firewalls, security incident and event monitoring, incident response planning, or intrusion detection or prevention.
  • the security environment 1 12 may include a cryptography service 710.
  • the cryptography service 710 may provide cryptographic operations to applications or components in the system 100. Cryptographic operations may include encrypting data or decrypting data.
  • the cryptography service 710 may include one or more configuration files that may configure the cryptography service 710 to use certain encryption standards, cryptography algorithms, or the like.
  • FIG. 8 depicts one embodiment of a method 800.
  • One embodiment of the disclosure may include the method 800.
  • the method 800 may include a method for data flow and data fusion.
  • the method 800 may include providing 802, in a computer-readable storage device, a dataspace.
  • the dataspace may include a plurality of logical segments.
  • the dataspace may include dataspace data stored in the plurality of logical segments.
  • the method 800 may include receiving 804, from a data source external to the dataspace, a data artifact.
  • the data artifact may include a plurality of data elements.
  • the method 800 may include processing 806, at a plurality of data filters, the plurality of data elements.
  • a data filter may process a data element based on a structure of the plurality of the data elements.
  • a data filter may process a data element based on a configuration of the data filter.
  • the method 800 may include generating 808 processed data. Generating 808 the processed data may be based on the processed plurality of data elements. The method 800 may include sending 810 the processed data to a logical segment.
  • the computer-readable stored device may include a CCN 204 or a CSN.
  • the dataspace may include the mission dataspace 120.
  • the plurality of logical segments may include the data artifact semantics segment 122, the data semantics segment 124, the data model semantics segment 126, or the data artifact storage segment 128.
  • the dataspace data may include data stored in one of these segments.
  • receiving 804 a data artifact from a data source external to the dataspace may include the ASIS 304 performing one or more functions.
  • the ASIS 304 may use the FTS to request the external data or data artifact and receive the external data.
  • the ASIS 304 executing on a CAPN 210, may receive the external data from an external application executing on an external computer system.
  • the ASIS 304 via the DSMS 306, may load the received data into the dataspace.
  • the import/export application 308 may import data from an external system and may interact with the DSMS 306 to load the data into the dataspace.
  • processing 806, at a plurality of data filters, the plurality of data elements may include executing a data ingest workflow.
  • the WPMS 302 may execute the data ingest workflow.
  • the WPMS 302 may execute the data ingest workflow in response to a trigger, for example, caused by arrival of the plurality of data elements.
  • the CMS 202 may execute the SIA 402.
  • the SIA 402 may access the received data elements in the mission dataspace 120 through the DSMS 306.
  • a pump component of the SIA 402 may receive the data elements, send the data elements to one or more filter components via one or more pipe components.
  • Each filter component may filter or transform the data elements.
  • a filter component may include an instance of the ACIS 312, which may index the received data elements, data artifacts, or other ingested data.
  • generating 808 the processed data based on the processed plurality of data elements may include assimilating the received data elements.
  • a filter component of the SIA 402 may transform the received data elements to be consistent with other received data or data already in the mission dataspace 120.
  • the SIA 402 may send transformed data elements or analysis data to the mission dataspace 120 via the DSMS 306.
  • transforming the received data elements may include a filter tracking an element count. Tracking an element count may include the filter storing the element count as metadata associated with the received data.
  • the filter tracking the element count may include simple word counting in textual data elements; or tracking the total number of artifacts ingest from a given data source; or tracking the total number of assertions for a given data element; or any counting that can be performed with respect to data elements.
  • transforming the received data elements may include a filter normalizing values included in the received data elements.
  • the received data elements may include structured data with one or more columns that include multiple data values.
  • the filter may normalize the data values in a column.
  • generating 808 the processed data may include a filter modifying the ingested data elements to remove irregularities in the data elements.
  • Removing irregularities may include standardizing the received data elements with data in the mission dataspace 120.
  • the dataspace may refer to global positioning system (GPS) coordinates as“coords,” and received data elements may refer to GPS coordinates as“location.”
  • GPS global positioning system
  • a filter component may modify the received data elements to refer to the GPS coordinates of the received data as “coords.”
  • the filter modifying the received data elements may include the filter modifying metadata associated with the received data elements, a name of a data column, or other data of the received data elements.
  • processing 806, at the plurality of data filters, the plurality of data elements may include receiving, at a first data filter, a first data element. Processing 806 the plurality of data elements may include selecting a portion of the first data element based on the configuration of the first data filter. Processing 806 the plurality of data elements may include sending the portion of the first data element to at least one logical segment. Processing 806 the plurality of data elements may include sending the first data element to second data filter of the plurality of data filters. For example, as discussed above in relation to the SIA 402, a filter component may receive data elements, store a data element in a logical segment of the mission dataspace 120, and forward the data elements to another filter component.
  • generating 808 the processed data may include generating metadata corresponding to the data artifact.
  • a filter component of the SIA 402 may generate metadata and store the metadata in the data artifact semantics segment 122.
  • processing 806 the plurality of data elements may include disambiguating a first data element from a second data element.
  • a filter component of the SIA 402 or the enrichment application 404 may disambiguate ingested data elements.
  • Generating 808 the processed data may include generating first metadata associated with a source data artifact.
  • Generating 808 the processed data may include generating disambiguation data based on the disambiguating the first and second data elements.
  • Sending 810 the processed data to a logical segment may include sending the first metadata to the data artifact semantics segment 122.
  • Sending 810 the processed data to a logical segment may include sending the disambiguation data to a second logical segment, such as the data semantics segment 124.
  • the method 800 may include sending the data artifact to the data artifact semantic segment 122 or the data artifact storage segment 128.
  • receiving 804 the data artifact may include receiving the data artifact from a semi-structured source.
  • Generating 808 the processed data may include generating metadata associated with the semi-structured source.
  • the semi-structured source may include an email ingest and the SIA 402 or other component may generate metadata associated with an email, such as metadata about a sender of an email.
  • Sending 810 the processed data to a logical segment may include sending the generated metadata to the data semantics logical segment 124.
  • receiving 804 the data artifact may include receiving a structured data artifact.
  • Processing 806 the plurality of data elements may include retrieving, from the data source, structure data corresponding to the plurality of data elements.
  • the SIA 402 may access a relational database system catalog to determine the structure of the plurality of data elements, as discussed above.
  • Generating 808 the processed data may include generating metadata associated with the structured data or metadata associated with the data source.
  • the SIA 402 may generate metadata describing the structure of the data in the relational database.
  • Sending 810 the processed data to a logical segment may include sending the structure data and metadata to the data model semantics segment 126.
  • the method 800 may further include sending the plurality of data elements to the data semantics segment 124.
  • receiving the structured data artifact may include receiving data from a relational database, RSS feed, or header section of an email message.
  • receiving 804 the data artifact may include receiving the data artifact from an external application executing on an external system.
  • a an external-facing data interface may receive the data artifact.
  • the external facing interface may include the ASIS 304 or the import/export application 308.
  • receiving 804 the data artifact may include the data artifact from a network location on an external computer system via a file transfer service.
  • a file transfer service of the AS IS 304 may receive the data artifact.
  • the method 800 may include generating a second plurality of data filters. This may be in response to the plurality of data elements including a predetermined side. For example, as discussed above, in response to a growing about of data to be ingested, the SIA 402 may generate more filter components.
  • the method 800 may further include selecting a portion of the dataspace data stored on a data storage device.
  • the method 800 may include indexing the dataspace data.
  • the ACIS 312 may receive a trigger to index some newly ingested data in the mission dataspace 120 stored on a certain CSN.
  • the method 800 may include generating a first indexing file that may include first indexing results, receiving a second indexing file from another data storage device (such as another CSN), sending the first indexing file and second index file to a third data storage device (such as a CCN 204), and merging the first and second indexing results, as described above in relation to the ACIS 312.
  • the system may include a system.
  • the system may be similar to the system 100 discussed above.
  • the system may include a first computing device.
  • the first computing device may include a CSN.
  • the first computing device may include a first processor configured to process a first set of instructions.
  • the first computing device may include a first computer-readable storage medium.
  • the first computer-readable storage medium may include a portion of a dataspace (such as the mission dataspace 120).
  • the system may include a second computing device.
  • the second computing device may include a CCN 204(1 ).
  • the second computing device may include a second processor configured to process a second set of instructions.
  • the second computing device may include a second computer-readable storage medium that may include the second set of instructions.
  • the second set of instructions when executed by the second processor, may cause the second processor to receive a data artifact from a data source external to the dataspace.
  • the data artifact may include a plurality of data elements.
  • the second set of instructions may include the ASIS 304 or the import/export application 308.
  • the system may include a third computing device.
  • the third computing device may include a CCN 204(2).
  • the CCN 204(2) may include the SIA 402.
  • the third computing device may include a third processor that may execute a third set of instructions stored on a third computer-readable storage medium of the third computing device.
  • the third set of instructions may include the SIA 402.
  • the third set of instructions may cause the third processor to perform one or more steps of the method described above.
  • the first computing device may include a cloud storage device (such as a CSN).
  • the cloud storage device may include a distributed file system or a distributed database as described above.
  • the third computing device may include a CCN 206.
  • the CCN 206 may be configured to execute the third set of instructions, such as the SIA 402 in response to receiving a command from a cloud management node, such as a CCN executing a CMS 202 described above.
  • a data artifact may include data file, a portion of a relational database, or a data stream.
  • Another embodiment of the disclosure may include a plurality of non- transitory computer-readable storage media.
  • the plurality of computer-readable storage media may include one or more sets of instructions.
  • the one or more sets of instructions when executed by one or more processors, may cause the system to perform one or more steps.
  • the one or more steps may include one or more steps from the method above.
  • the one or more steps may include one or more functions of the system 100 as described above.
  • the plurality of non-transitory computer readable storage media may include one or more storage devices of one or more CCNs 206(1 )-(n).
  • the system 100 may store data in a dataspace (such as the mission dataspace 120) in unconventional ways.
  • the system 100 may store data elements from unstructured, semi-structured, or structured data artifacts in the different segments of the mission dataspace 120.
  • the data elements from these data artifacts may also be unstructured, semi-structured, or structured.
  • This unconventional flexibility of the data representation of the mission dataspace 120 may improve the functionality of the computing system of the system 100 by providing flexibility in how the data is represented to different applications different and allowing processing to occur on different types of data elements based on the structure of the data elements.
  • the system 100 may configure a data filter 412 based on the structure of the ingested data.
  • the SIA 402 may ingest structured data in a first way (e.g ., based on structure metadata from the source of the structured data or model semantics data from the data model semantics segment 126) and may ingest unstructured data in a different way.
  • One or more data filters 412 may ingest the same data and process it in parallel or in sequence.
  • the system 100 may provide an unconventional automated workflow for the data ingested into the system 100. Whereas conventional computing systems only compute results from ingested data, the system 100 may provide a workflow to ingest the data, process the data, store the data in different segments of the mission dataspace 120, manage the data ⁇ e.g., rolling back data in the mission dataspace 120 when needed, allowing assertions to be placed on the data), execute one or more analytic applications on the data, and optimize those analytic applications for the data. This may improve a computing system by requiring minimal input from a user and performing functionality a user may not even be aware of. Also unconventionally, the system 100 may provide security for the ingested data from the moment the data enters the system 100. The system 100 may provide security for the data during execution of Al or data analytic models, dataspace searches, the enrichment application 404, or at other times while the data exists in the system 100.
  • the system 100 may provide one or more improvements to computer functionality.
  • the system 100 may increase the speed at which Al systems and models receive and process information.
  • the data flow and fusion method 800 of FIG. 8 (or the functionality of a component of the system 100, such as the DMS 306) may reduce the amount of duplicated data in the mission dataspace 120 ⁇ e.g., by dropping duplicate data during data ingest).
  • the system 100 may increase the accuracy of Al systems and models.
  • the step of generating 808 processed data or the functionality of a component of the system 100, such as the enrichment application 404, may, in an unconventional step, extract unstructured data ingested into the dataspace 120 and place the extracted unstructured data into a data model of the mission dataspace 120 to generate structured data.
  • This generated structured data may fill in missing values for data used in data analysis or machine learning capabilities of the system 100. Filling in the missing values and preventing data from being dropped (which occurs in conventional computing systems) may increase the accuracy of the data analysis or machine learning capabilities of the system 100.
  • the system 100 may provide a schema-less data model provided by the implementation of the mission dataspace 120.
  • the data model may provide a uniform representation for data of a wide variety of types.
  • the data elements conforming to that data model may feed into an Al system or model so that the Al system or model spends more time deriving patterns and finding relationships in the data of the system 100 and less time configuring the data to be in a usable format.
  • the system 100 leverages different data from different sources for Al models and systems without a tradeoff in performance.
  • the Al systems and models can derive patterns and find relationships in the data faster and more accurately than prior systems. This enables the Al models and systems to automatically learn and make predictions in ways that previously could not be automated.
  • the system 100 overcomes problems specifically arising in the realm of Al, such as the problem of Al models and systems learning from and using consistent data. Furthermore, the system 100 may provide data security to the data in the mission dataspace 120 and elsewhere in the system throughout an entire workflow.
  • the system 100 also provides an improved interface for a user that allows a user to provide data to the system 100, execute analytics on the data, and manage the data in the mission dataspace 120.
  • the interface may allow a user who has no knowledge of computer programming to interact with the mission dataspace 120 or to execute analytics on the data.
  • the system 100 may provide scaling and high performance to large datasets.
  • the system 100 may provide scaling and performance benefits as part of end-to-end lifecycle for data of the system 100 and not just during one part of the time ingested data spends in the system 100.
  • in response to the parallelization of the data filters 412 or the division of the mission dataspace 120 into multiple segments 122-128 may provide scaling and performance benefits during data ingest.
  • the enrichment application 404 may provide performance benefits during a pre-data analysis phase. Rollback, optimistic processing, and other Al model capabilities may provide performance increases during training or execution of an Al model.
  • Figure 9 depicts one embodiment of a method 900.
  • the method 900 may include a computer-implemented method.
  • the method 900 may include a method for optimizing a deep learning model configuration based on Random Recursive Search (RRS).
  • a“deep learning model” may mean a class of deep learning systems, algorithms, models, or the like.
  • a deep learning model may include one or more parameters.
  • the parameters may include a variety of ways to be configured.
  • a“parameter” of a deep learning model may mean a parameter (or hyperparameter), input feature, configuration, or the like that can be changed, and the behavior of the deep learning model may change based on the changed parameter.
  • Each deep learning model may include one or more parameters, and the details of a parameter may depend on the type of deep learning model.
  • a“deep learning model configuration” may mean an instance of a deep learning model.
  • the deep learning model configuration may include one or more parameters of the deep learning model being configured with a value or the like.
  • a deep learning model configuration may receive data as input (i.e. input data), process the input data based on the deep learning model configuration’s parameters, and produce an output, such as a classification, prediction, value, or the like.
  • a deep learning model may include a hyperparameter design space.
  • the“hyperparameter design space” may include one or more parameters of the deep learning model and the possible values for each of those parameters.
  • the hyperparameter design space may include all the parameters of the deep learning model or a subset of all of the parameters. Which parameters of the deep learning model may be included in the hyperparameter design space may be based on user configuration.
  • a deep learning model may include an artificial neural network (ANN).
  • the ANN may include a variety of parameters.
  • a parameter of the ANN may include a weight of an edge of the ANN, a position of an edge of the ANN, the number of nodes in the ANN, the number of nodes in a layer of the ANN, the number of layers of the ANN, or the like.
  • the hyperparameter design space of the ANN may include the edge positions and the edge weights of the ANN.
  • the hyperparameter design space of the ANN may not include the number of nodes, the number of nodes in a layer, or the number of layers in the ANN. The values of these parameters may be set and may not change during the optimization of the deep learning model.
  • a deep learning model configuration may include a specific instance of the ANN.
  • a deep learning model configuration may include a machine learning model configuration capable of deep learning.
  • a deep learning model may include an artificial neural network.
  • An ANN may include a deep belief network, recurrent neural network (RNN), convolutional neural network (CNN), or the like.
  • a deep learning model may include an estimator.
  • a deep learning model may include other machine learning models capable of deep learning.
  • a deep learning model may include an ensemble of deep learning models.
  • An ensemble may include two or more deep learning models.
  • the two or more deep learning models may be connected in a variety of ways.
  • a parameter of an ensemble may include a type of deep learning model included in the ensemble, a position of a deep learning model in the ensemble, how the deep learning model is connected to another deep learning model, or the like.
  • the method 900 may include generating 902 a first deep learning model configuration.
  • generating 902 the first deep learning model configuration may include generating a new deep learning model configuration.
  • generating 902 the first deep learning model configuration may include receiving a pre-configured deep learning model configuration.
  • a pre-configured deep learning model configuration may include an instance of a deep learning model whose parameters have been previously configured before the start of the method 900.
  • generating 902 the first deep learning model configuration may include generating a value for a parameter of the first deep learning model configuration.
  • Generating a value for a parameter of the first deep learning model configuration may include generating a value for each parameter in the hyperparameter design space of the deep learning model configuration.
  • generating the first deep learning model configuration by generating a value for each parameter in the hyperparameter design space may be called generating the first deep learning model from the hyperparameter design space.
  • generating a value for a parameter may include determining the possible values for the parameter. Determining the possible values may include a user selecting the possible values, a range of possible values, or the like. Determining the possible values for a parameter may include determining the possible values for multiple parameters of a same type. For example, in response to the deep learning model configuration including an ANN, determining the possible values for a weight of the ANN may include determining the possible values for all the weights of the ANN. A user may select that a value of a weight in the ANN may have a value of 0 to 400, and in response, all the weights may have a possible value of 0 to 400.
  • generating 902 the first deep learning model configuration may include generating the deep learning model configuration based on random number generation.
  • Generating the deep learning model configuration based on random number generation may include randomly generating a parameter of the deep learning model configuration.
  • the random generation of a parameter may include randomly selecting a value for the parameter from all possible values of the parameter, a subset of all possible values of the parameter, or the like.
  • the deep learning model configuration may include an ANN.
  • Generating the ANN based on random number generation may include randomly generating a weight of an edge, randomly generating the position of an edge, randomly generating the number of nodes in the ANN, randomly generating the number of layers of the ANN, randomly generating which nodes belong to which layers, or the like.
  • randomly generating deep learning model configurations may be quicker at finding a deep learning model configuration that performs better at solving a problem than using some learning algorithms, such as backpropagation or the like. This may occur because learning algorithms can be slow compared to random generation of parameters. Furthermore, certain learning algorithms, like backpropagation, may seek to learn how the interaction of different parameters can lead to a better deep learning model configuration. This learning may be slow compared to randomly generating parameters and testing if the configuration of parameters leads to better results. In some embodiments, randomly generating parameters may not be affected by local minima as some learning algorithms, like backpropagation, may be.
  • generating 902 the first deep learning model configuration may include generating multiple deep learning model configurations. Generating multiple deep learning model configurations will be discussed in more detail below.
  • the method 900 may include calculating 904 a first result metric for the first deep learning model configuration.
  • a result metric may include a measurement of how well a deep learning model configuration is at performing its purpose.
  • the purpose of a deep learning model configuration may include image classification.
  • a result metric for the deep learning model configuration may include an accuracy of the deep learning model configuration at classifying an image.
  • a result metric for the deep learning model configuration may include overfit, underfit, or the like.
  • calculating 904 the result metric may include testing the deep learning model configuration on a testing dataset.
  • calculating 904 the result metric of the ANN may include inputting the testing dataset into the ANN and determining the accuracy, overfit, or underfit of the ANN.
  • the result metric may include a response surface.
  • the response surface may include multiple dimensions, and each dimension may represent a goal (e.g. accuracy, underfit, overfit, or the like). Each point in the response surface may include a certain value for each of the dimensions.
  • the goals may be competing and the deep learning model configuration doing better in one goal may simultaneously cause the deep learning model configuration to do worse in another goal.
  • the result metric may include a measurement based on the competing goals.
  • the method 900 may include selecting 906 a first sample space.
  • the first sample space may be based on the first deep learning model configuration.
  • a “sample space” may mean a subsection of the hyperparameter design space.
  • Each dimension of the sample space may include a range of possible values for a parameter of the deep learning model configuration.
  • the sample space may be smaller than the hyperparameter design space, thus the range of possible values for a parameter may be smaller in the sample space.
  • selecting 906 the first sample space based on the first deep learning model configuration may include selecting the first sample space based on the first deep learning model configuration having a result metric above a predetermined threshold.
  • the predetermined threshold may include the first deep learning model configuration having a higher result metric than other deep learning model configurations.
  • the sample space being based on the deep learning model configuration may include the sample space being centered on the deep learning model configuration.
  • the sample space being centered on the deep learning model configuration may include the range of possible values for a dimension in the sample space having its center value be the value of the deep learning model configuration’s parameter corresponding to that dimension.
  • the size of the range for each dimension may be different.
  • the method 900 may include calculating an exploitation threshold. Calculating an exploitation threshold may include calculating the threshold based on user configuration, an average result metric, or the like.
  • selecting 906 the first sample space may include selecting the first sample space in response to the first metric exceeding the exploitation threshold.
  • a reason for having an exploitation threshold may include to prevent the method 900 from selecting deep learning model configurations with only a small increase in its result metric than the result metric of the deep learning model around which the first sample space is based. In some embodiments, it may be faster, more advantageous, or the like to generate another deep learning model configuration than to re-center, recalculate, or the like for a sample space.
  • the method 900 may include generating 908 a second deep learning model configuration.
  • the second deep learning model configuration may be within the first sample space.
  • the second deep learning model configuration being within the first sample space may include the parameters of the second deep learning model configuration having values within the ranges of the dimensions of the first sample space.
  • generating 908 the second deep learning model configuration may include generating a new deep learning model configuration with values within the first sample space.
  • generating 908 the second deep learning model configuration may include adjusting the one or more parameters of the first deep learning model configuration.
  • generating 908 the second deep learning model configuration may include generating multiple second deep learning model configurations within the first sample space, as discussed below.
  • generating the second deep learning model configuration by generating or adjusting a value for each parameter in the first sample space may be called generating the first deep learning model from the first sample space.
  • the method 900 may include calculating 910 a second result metric for the second deep learning model configuration. Calculating 910 the second result metric may be similar to calculating 904 the first result metric.
  • the method may include selecting 912 a second sample space.
  • the second sample space may be based on the second deep learning model configuration.
  • selecting 912 the second sample space may include centering the second sample space around the second deep learning model configuration. Centering the second sample space around the second deep learning model configuration may be similar to centering the first sample space around the first deep learning model configuration.
  • the second sample space may be the same size as the first sample space. In another embodiment, the second sample space may be larger or smaller than the first sample space.
  • the second result metric exceeding the first result metric may include the second result metric exceeding the first result metric by a threshold amount.
  • the threshold amount may include an amount based on user configuration. For example, a user configuration may include selecting the second sample space in response to the second result metric exceeding the result metric by 0.00001 .
  • the second result metric does not exceed the first result metric by 0.00001 and the second deep learning model configuration will not be selected to be the center of the second sample space.
  • a reason for having the threshold may include to prevent the method 900 from selecting deep learning model configurations with only a small increase in its result metric than another result metric. In some embodiments, it may be faster, more advantageous, or the like to generate another deep learning model configuration than to re-center, recalculate, or the like a sample space.
  • one result metric exceeding another result metric may not necessarily include one result metric having a higher value than the other.
  • the second result metric exceeding the first result metric may include that the deep learning model configuration corresponding to the second metric performs better than the deep learning model configuration corresponding to the first result metric.
  • a result metric may include number of misclassifications. Thus, the lower the number of misclassifications, the better the deep learning model configuration.
  • a result metric exceeding another result metric may include a result metric being lower than another result metric.
  • the method 900 may include reducing 914 the size of the first sample space. Reducing 914 the size of the first sample space may include reducing the range of possible values for one or more parameters of the first deep learning model configuration. In some embodiments, the first sample space may continue to be centered around the first deep learning model configuration.
  • reducing the size of the first sample space may assist in finding another deep learning model configuration with a higher result metric than the first deep learning model configuration. This may be because the method 900 is searching in a smaller area, and thus, has a higher likelihood of finding something better within that area.
  • the method 900 may include generating a third deep learning model configuration within the new sample space (i.e. the second sample space or the reduced first sample space) and continuing the search process.
  • the method 900 may return to step 908 and use another deep learning model configuration to continue searching for a deep learning model configuration with a higher result metric as depicted in FIG. 9.
  • the method 900 may repeat until stopping criteria is met.
  • a stopping criteria may include not finding deep learning model configuration with a corresponding result metric with an improvement over a current best result metric.
  • a stopping criteria may include not finding a deep learning model configuration with a corresponding result metric with an improvement over a certain threshold.
  • a stopping criteria may include adjusting the threshold after a predetermined number of deep learning model configurations have been generated.
  • the method 900 in response to meeting the stopping criteria, may include selecting the deep learning model configuration with the highest result metric as the output deep learning model configuration. In response to selecting the output deep learning model configuration, the method 900 may cease. The method 900 ceasing may include the method 900 ceasing temporarily.
  • the output deep learning model configuration may perform well on input data that it has not encountered previously. However, the output deep learning model configuration may encounter a set of input data that may be significantly different from the input data the output deep learning model configuration may have used previously or encountered as testing data. In response, the output deep learning model configuration may be less accurate or perform worse on this new set of input data. Thus, in one embodiment, further optimization may assist the output deep learning model configuration to perform well on the new set of input data.
  • the method 900 may include calculating a threshold result metric.
  • a result metric of a deep learning model configuration below the threshold result metric may indicate that the deep learning model configuration may benefit from further optimization.
  • the method 900 may include calculating a third result metric.
  • the third result metric may be for the output deep learning model configuration.
  • the third result metric may be based on additional training set data. Calculating the third result metric may be similar to calculating 904 the first result metric.
  • the method 900 may include, in response to determining that the third result metric is below the threshold result metric, selecting a third sample space.
  • the third sample space may be based on the output deep learning model configuration. Selecting the third sample space may be similar to selecting 906 the first sample space.
  • the method 900 may include continuing one or more steps of the method 900 until a stopping criteria is met and a new output deep learning model configuration is selected.
  • generating a deep learning model configuration at a higher level may include using a first computational approach, and generating a deep learning model configuration at a lower level may include using a second computational approach.
  • a “level” of the hyperparameter design space may include the hyperparameter design space itself, or a subsection of the hyperparameter design space.
  • a level may be higher, lower, or at the same level as another level. In some embodiments, whether a level is higher or lower than another level may be based on a size of the level. For example, a higher level may include a larger size in the hyperparameter space than a lower level, with the hyperparameter design space itself being the highest level.
  • the hyperparameter design space may include a first level, the first sample space may include a second level that is lower than the first level, and the reduced first sample space may include a third level lower than the second level.
  • a computational approach may include software.
  • the software may include a software library, e.g. a dataflow programming library or the like.
  • a computational approach may include an algorithm or the like (e.g. an algorithm that approximates, an algorithm that produces an accurate output, or the like).
  • a computational approach may include computer hardware, e.g. a central processing unit (CPU), a graphics processing unit (GPU), an application-specific integrated circuit (ASIC), or the like.
  • a computational approach may include a portion of software (e.g. a function, subroutine, or the like) or a portion of hardware.
  • a computational approach may include a hardware representation (e.g. a 16 bit integer data type, a 64 bit floating point data type, or the like). Some computational approaches may include different capabilities such as speed, accuracy, memory usage, storage usage, network usage, or the like.
  • generating a deep learning model configuration from a higher level of the hyperparameter design space it may be beneficial to generate the deep learning model configuration quickly. This may be because, initially, it may be more advantageous to generate several deep learning model configurations to increase the probability of generating a deep learning model configuration that may yield a high result metric.
  • generating a deep learning model configuration using a computational approach that may include a lower accuracy capability, but a high-speed capability may be advantageous.
  • a lower level of the hyperparameter design space such as the first sample space, the reduced first sample space, a second reduced sample space, or the like
  • computational accuracy may be more beneficial than speed.
  • generating a deep learning model configuration using a computational approach that may include a high accuracy capability may be advantageous.
  • generating 902 the first deep learning model configuration may include generating the first deep learning model using a first computational approach.
  • Generating 908 the second deep learning model configuration may include generating the second deep learning model using a second computational approach.
  • the first computational approach may include a higher speed capability than the second computational approach.
  • the second computational approach may include a higher accuracy capability than the first computational approach.
  • a GPU may include a high speed capability, but may not include, compared to a CPU, a high accuracy capability.
  • a CPU may include a high accuracy capability, but may not include, compared to a GPU, a high speed capability.
  • generating 902 the first deep learning model configuration may include using a GPU to generate the first deep learning model configuration.
  • generating 902 the first deep learning model configuration may include generating the first deep learning model configuration quickly in order to find an initial first sample space that is likely to yield a good search area in the hyperparameter design space.
  • Generating 908 the second deep learning model configuration may include using a CPU to generate the second deep learning model configuration.
  • generating 908 the second deep learning model configuration may include generating the second deep learning model configuration using more accurate calculations.
  • the method 900 may be balancing improving the deep learning model configuration with computing resources (e.g. processor consumption, memory consumption, non-volatile storage consumption, power consumption, computing time, or the like). Initially, improvements to the deep learning model may be worth the computing resources, but as the rate of improvement slows, it may be more advantageous to stop searching for a better deep learning model configuration than to expending a large amount of computing resources to find a small improvement.
  • computing resources e.g. processor consumption, memory consumption, non-volatile storage consumption, power consumption, computing time, or the like.
  • FIG. 10A depicts one embodiment of a representation of a hyperparameter design space 1000.
  • the hyperparameter design space 1000 may include one or more dimensions and each dimension may include a parameter of the first deep learning model configuration.
  • a dimension may include the range of all possible values for the parameter.
  • a hyperparameter design space 1000 may include a first dimension 1002 and a second dimension 1004.
  • the first dimension 1002 may correspond to a first parameter of a deep learning model configuration and the second dimension may correspond to a second parameter of the deep learning model configuration.
  • the first parameter may have a range of possible values from 0 to 300 and the second parameter may have a range of possible values from 0 to 1000.
  • the hyperparameter design space 1000 may include a point 1006.
  • the point 1006 may correspond to a deep learning model configuration.
  • FIG. 10A may include the result of generating 902 a first deep learning model configuration of method 900.
  • the point may correspond to a deep learning model configuration whose first parameter has a value of 120 and whose second parameter has a value of 73.
  • FIG. 10A depicts a very simple hyperparameter design space 1000.
  • these configurations may include thousands of parameters.
  • a hyperparameter design space 1000 may include a dimension for each weight of an edge, a dimension for each edge position, a dimensions for a number of nodes, or the like.
  • a visual representation of a hyperparameter design space for such configuration would likely be impossible.
  • FIG. 10B depicts one example of a hyperparameter design space 1000.
  • the hyperparameter design space 1000 may include one embodiment of the result of selecting 906 a first sample space of the method 900.
  • the hyperparameter design space 1000 may include a first sample space 1008.
  • the first sample space 1008 may be smaller than the hyperparameter design space 1000.
  • the first sample space 1008 may be centered on the point 1006.
  • the point 1006 corresponding to the first deep learning model configuration may have values for the first dimension 1002 and the second dimensions 1004 of 120 and 73 respectively.
  • the range of the first sample space for each dimension may include 50 possible values.
  • the ranges of possible values for the first and second parameters in the first sample space are 95 to 145 and 48 to 98 respectively.
  • FIG. 10C depicts one embodiment of a hyperparameter design space 1000.
  • FIG. 10C may depict one embodiment of the result of generating 908 a second deep learning model configuration of the method 900.
  • the second deep learning model configuration In response to the first sample space 1008 having a range of 50 values in each dimension centered around the values 120 and 73, the second deep learning model configuration’s first and second parameters may be 132 and 55 respectively. This may correspond to a second point 1010 in the hyperparameter design space 1000.
  • FIG. 10D depicts one embodiment of a hyperparameter design space 1000.
  • FIG. 10D may depict one embodiment of the result of selecting 912 a second sample space of the method 900 of FIG. 9.
  • the method 900 may include selecting 912 a second sample space 1012 centered around the second point 1010.
  • FIG. 10E depicts one embodiment of a hyperparameter design space 1000.
  • FIG. 10E may depict one embodiment of the result of reducing 914 the size of the first sample space of the method 900 of FIG. 9.
  • the range of possible values for the parameters represented by the first sample space 1008, may be reduced from 50 values to 30 values.
  • the range of possible values for the first and second parameters would still be centered around 120 and 73, but the possible values would be 105 to 135 and 58 to 88.
  • the hyperparameter design space 1000 may include a first level of the hyperparameter design space.
  • Generating 902 the first deep learning model configuration may include using a first computational approach.
  • the computation approach may include a high speed capability.
  • the first sample space 1008 may include a second level.
  • the second level may be lower than the first level.
  • Generating 908 the second deep learning model configuration may include using a second computational approach.
  • the second computational approach may include a lower speed capability than the first computational approach, but a higher accuracy capability.
  • the reduced first sample space 1008 of FIG. 10E may include a third level.
  • the third level may be lower than the second level.
  • Generating a third deep learning model configuration from the reduced first sample space 1008 may include using a third computational approach.
  • the third computational approach may include a higher accuracy capability than the second computational approach.
  • FIG. 1 1 depicts one embodiment of a method 1 100.
  • the method 1 100 may include generating 1 102 a plurality of first deep learning model configurations. Generating 1 102 a plurality of first deep learning model configurations may be similar to generating 902 a first deep learning model configuration of the method 900 as described above.
  • generating 1 102 the plurality of first deep learning model configurations may include generating a predetermined number of deep learning model configurations. The predetermined number may be based on a confidence interval. The confidence interval may include a user-defined confidence interval.
  • each of the deep learning model configurations of the plurality of first deep learning model configurations may include different values for each of its parameters.
  • generating multiple deep learning model configurations may include randomly generating two ANNs.
  • the first ANN may include a variety of randomly generated nodes randomly grouped into layers, a variety of edges each with its own randomly generated weight and randomly connected to nodes of the ANN.
  • the second ANN may include randomly generated nodes and edges in randomly generated positions within the ANN with different values for these parameters than the first ANN.
  • generating 1 102 the plurality of first deep learning model configurations may include generating two or more of the first deep learning model configurations in parallel. Generating two or more of the first deep learning mode configurations in parallel may include using a compute cluster, supercomputer, or the like. Generating 1 102 the plurality of first deep learning model configurations may lend itself to generating the deep learning model configurations concurrently since each deep learning model configuration may be generated independently of the others and may not be dependent on the others.
  • the method 1 100 may include calculating 1 104 a first result metric for each of the plurality of first deep learning model configurations. Calculating 1 104 a first result metric for each first deep learning model configuration may be similar to calculating 904 the first result metric for the first deep learning model configuration of method 900 of FIG. 9 as described above. In one embodiment, calculating 1 104 the first result metric for each of the plurality of first deep learning model configurations may include calculating two or more result metrics in parallel. Calculating two or more result metrics in parallel may include using a compute cluster, supercomputer, or the like.
  • the method 1 100 may include selecting 1 106 a deep learning model configuration from the plurality of first deep learning model configurations.
  • selecting 1 106 the deep learning model configuration may include selecting the deep learning model configuration based on the result metric of the selected first deep learning model configuration.
  • selecting 1 106 the deep learning model configuration from the plurality of first deep learning model configurations may include selecting the deep learning model configuration with a result metric above a predetermined threshold.
  • selecting the deep learning model configuration with a result metric above the predetermined threshold may include selecting the deep learning model configuration with highest result metric of the result metrics of the plurality of first deep learning model configurations.
  • the method 1 100 may include selecting 1 108 a first sample space.
  • the first sample space may be based on the selected first deep learning model configuration.
  • selecting 1 108 the first sample space may be similar to selecting the first sample space 906 of the method 900 described above.
  • the method 1 100 may include generating 1 1 10 a plurality of second deep learning model configurations. Each second deep learning model configuration may be within the first sample space.
  • generating 1 1 10 the plurality of second deep learning model configurations may include generating a predetermined number of second deep learning model configurations. The predetermined number may be based on a confidence interval. The predetermined number may include the same number of first deep learning model configurations or may include a different number.
  • 1 1 10 the plurality of second deep learning model configurations may include generating two or more of the second deep learning model configurations in parallel. Generating two or more of the second deep learning model configurations in parallel may include using a compute cluster, supercomputer, or the like.
  • the method 1 100 may include calculating 1 1 12 a second result metric for each second deep learning model configurations of the plurality of second deep learning mode configurations. Calculating 1 1 12 the second result metric for each second deep learning model configuration may be similar to calculating 1 104 the first result metric for each first deep learning model configuration described above. In some embodiments, calculating 1 1 12 the second result metric for each second deep learning model configuration may include calculating two or more second result metrics in parallel. Calculating two or more second result metrics in parallel may include using a compute cluster, supercomputer, or the like.
  • the method 1 100 may include, in response to a second result metric exceeding the first result metric, selecting 1 1 14 a second sample space.
  • the second sample space may be based on the second deep learning model configuration corresponding to the second result metric that exceeds the first result metric. Selecting 1 1 14 the second sample space may be similar to selecting 912 a second sample space of method 900 described above.
  • the method 1 100 may include reducing 1 1 16 the size of the first sample space.
  • reducing 1 1 16 the size of the first sample space may be similar to reducing 914 the size of the first sample space of method 900 described above.
  • generating one or more deep learning model configurations may include efficient generation of a deep learning model configuration.
  • the efficiency may be based on random generation or the like.
  • the generation may be more efficient than tracking parameters of a deep learning model configuration and tracking the effectiveness of certain parameters’ interactions with other parameters.
  • certain steps such as generating deep learning model configurations, testing the deep learning model configurations, or the like may be done in parallel using a compute cluster, supercomputer, or the like.
  • an existing deep learning model configuration as a basis for optimization, an even more optimized version of that deep learning model configuration may be found.
  • FIG. 12A depicts one embodiment of a hyperparameter design space 1200.
  • the hyperparameter design space 1200 may include a hyperparameter design space similar to the hyperparameter design space 1000 of FIG.s 10A-E discussed above.
  • the hyperparameter design space 400 may include a first dimension 1202 and a second dimension 1204 each corresponding to possible values of a parameter of a deep learning model configuration.
  • the hyperparameter design space 1200 may include a plurality of points 1206(1 )-(5).
  • a point 1206 may correspond to a deep learning model configuration of the plurality of first deep learning model configurations.
  • Each first deep learning model configuration may include different values for parameters than other first deep learning model configuration as seen in FIG.
  • FIG. 12A may depict the result of one embodiment of generating 1 102 a plurality of first deep learning model configurations.
  • the result metrics for the deep learning model configurations corresponding to the points 1206(1 )-(5) may be 41 , 36, 86, 44, and 52 respectively.
  • selecting 306 the deep learning model configuration may include selecting the deep learning model configuration corresponding to the point 1206(3).
  • FIG. 12B depicts one embodiment of a hyperparameter design space 1200.
  • FIG. 12B may depict one embodiment of the result of selecting 1 108 a first sample space.
  • the first sample space 1208 may be based on the deep learning model that corresponds to the point 1206(3).
  • the first sample space 1208 may be centered around the point 1206(3).
  • the first sample space 1208 may include other deep learning model configurations of the plurality of first deep learning model configurations.
  • generating 1 1 10 a plurality of second deep learning model configurations may include generating a plurality of deep learning model configurations corresponding to the points 1210(1 )-(4). Each of the points 1210 may be within the first sample space 1208.
  • calculating 1 1 12 a second result metric for each second deep learning model configuration may include calculating a result metric for each deep learning model configuration corresponding to the points 1210(1 )- (5). These result metrics may include, respectively, 77, 65, 85, and 92. Since 92 exceeds the result metric of the deep learning model configuration that corresponds to the point 1206(3) (i.e. 86), in response, the second sample space 1212 may be based on the deep learning model configuration corresponding to the point 1210(4), as is depicted in FIG. 12D.
  • FIG. 12E depicts one embodiment of the hyperparameter design space 400.
  • the result metric for the point 1210(4) may be 27.
  • the second result metrics may include 77, 65, 85, and 27.
  • the first sample space 1208 may reduce in size. The first sample space 1208 may remain based on the selected first deep learning model configuration corresponding to the point 1206(3).
  • FIG. 13 depicts one embodiment of a method 1300.
  • the method 1300 may include a computer-implemented method for improving a deep learning model configuration.
  • the method 1300 may include receiving 1302 a first deep learning model configuration.
  • receiving 1302 a first deep learning model configuration may include receiving a pre-configured deep learning model. For example, a user may have a deep learning model that has been generated through training on training data. The user may want to improve the deep learning model, but training the model on more training data may not be effective. Generating more training data may be difficult and time-consuming. T raining the model on the additional training data may yield minimal increases in the results of the model.
  • the method 1300 may include calculating 1304 a first result metric for the first deep learning model configuration. Calculating 1304 the first result metric may be similar to calculating 904 the first result metric of the method 900 or calculating 1 104 a first result metric for each of a plurality of first deep learning model configurations of the method 1 100. In one embodiment, the method 1300 may include selecting 1306 a first sample space. The first sample space may be based on the first deep learning model configuration. Selecting 1306 the first sample space may be similar to selecting 906 the first sample space of the method 900 or selecting 1 108 the first sample space of the method 1 100.
  • the method 1300 may include generating 1308 a second deep learning model configuration.
  • the second deep learning configuration may be within the first sample space. Generating 1308 the second deep learning model configuration may be similar to generating 908 the second deep learning model configuration of the method 900 or generating 1 1 10 a plurality of second deep learning model configurations of the method 1 100.
  • generating 1308 the second deep learning model configuration may include adjusting a parameter of the deep learning model configuration. Adjusting a parameter of the deep learning model may include changing the value of a parameter without generating a new deep learning model configuration. In some embodiments, it may be more efficient to change the parameters values of a deep learning model configuration than to generate a new deep learning model configuration whose parameter values are equal to the parameter values of the changed deep learning model.
  • adjusting a parameter the first deep learning model configuration may include adjusting a weight of an edge of a neural network of the first deep learning model configuration. The adjusting the weight may include adjusting the weight within a predetermined amount. The predetermined amount may include an amount based on the range of the first sample space in the dimension corresponding to the weight of that edge.
  • adjusting a parameter the first deep learning model configuration may include adding an edge to a neural network of the first deep learning model configuration. Adding the edge may include adding a predetermined number of edges. The predetermined number may be based on the first sample space in the dimension corresponding to a number of edges. The position of an added edge may include a position based on the first sample space in the dimension corresponding to a position of an edge.
  • adjusting a parameter the first deep learning model configuration may include removing an edge from a neural network of the first deep learning model configuration. Removing the edge may include removing a predetermined number of edges. The predetermined number may be based on the first sample space in the dimension corresponding to a number of edges. The position of a removed edge may include a position based on the first sample space in the dimension corresponding to a position of an edge.
  • the first deep learning model configuration may include an ensemble.
  • the first deep learning model configuration may include an ensemble that includes four ANNs, each ANN having a different set of parameters. The ensemble may be configured to output the majority result of the ANNs.
  • adjusting a parameter of the first deep learning model may include removing one ANN from the ensemble. Removing an ANN may include removing the ANN with the least accuracy on a testing dataset.
  • adjusting a parameter of the first deep learning model configuration may include adding a learning model to the ensemble.
  • adding a learning model to the ensemble may include adding an ANN to the ensemble from the example above.
  • the added ANN may include an ANN whose edge positions and edge weights are based on random generation.
  • the parameters of the randomly generated ANN may be within the first sample space’s dimensions corresponding to those parameters.
  • Adjusting a parameter of the first deep learning model configuration may include repositioning a learning model in the ensemble.
  • an ensemble may include multiple learning models and a first learning model may feed its output into a second learning model as input.
  • repositioning a learning model in the ensemble may include positioning the second learning model to feed its output as input to the first learning model.
  • the input and output data formats or the like may adjust so that the different learning models are compatible with each other regarding input and output.
  • the method 1300 may include calculating 1310 a second result metric for the second deep learning configuration. Calculating 1310 the second result metric may be similar to calculating 910 the second result metric of the method 900 or calculating 1 1 12 a second result for each second deep learning model configuration of the method 1 100.
  • the method 1300 may include, in response to the second result metric exceeding the first result metric, selecting 1312 a second sample space, wherein the second sample space is based on the second deep learning model configuration. Selecting 1312 the second sample space may be similar to selecting 912 the second sample space of the method 900 or selecting 1 1 14 the second sample space of the method 1 100.
  • the method 1300 may include, in response to the second result metric not exceeding the first result metric, reducing 1314 the size of the first sample space. Reducing 1314 the size of the first sample space may be similar to reducing 914 the size of the first sample space of the method 900 or reducing 1 1 16 the size of the first sample space of the method 1 100.
  • a deep learning model may include an analytical model, such as the data enrichment model of the MAE.
  • the analytic management model may perform one or more of the methods described above to train the analytical model.
  • FIG. 14 depicts one embodiment of a portion of an artificial neural network (ANN) 1400.
  • the ANN 1400 may include a multi-layer perceptron (MLP), a deep belief network, a recurrent neural network (RNN), a convolutional neural network (CNN), spiking neural network (SNN), or the like.
  • the ANN 1400 may include a plurality of neurons 1402 such as the neurons 1402(1 )-(5) of FIG. 14. As described herein, an unspecified neuron may be referred to as a neuron 1402.
  • the ANN 1400 may include a plurality of layers 1404 of neurons 1402, such as the layers 1404(1 )-(2) of FIG. 14.
  • One or more neurons 1402 of the plurality of neurons 1402 may be grouped a layer 1404.
  • a first layer 1404(1 ) may include the neurons 1402(1 )-(2)
  • a second layer 1404(2) may include the neurons 1402(3)-(5).
  • the ANN 1400 may include an input layer and an output layer (not depicted in FIG. 14). In the exemplary embodiment depicted in FIG. 14, the input layer may be located to the left the layer 1404(1 ), and the output layer may be located to the right of the layer 1404(2).
  • voltage (sometimes called “signal” or a“voltage value”) may flow from the input layer, through one or more hidden layers, and toward the output layer. Voltage may include output from a neuron 1402, synapse 1406, or another component of the ANN 1400.
  • a first layer 1404 (and the one or more neurons 1402 of that layer) may be deemed“downstream” from a second layer 1404 in response to the second layer 1404 being disposed between the input layer and the first layer 1404.
  • the first layer 1404 may include the layer 1404(2) and the second layer 1404 may include the layer 1404(1 ).
  • the first layer 1404(2) may be downstream of the layer 1404(1 ).
  • a first layer 1404 may be deemed“upstream” from a second layer 104 in response to the first layer 1404 being disposed between the input layer and the second layer 1404.
  • the first layer 1404 may include the layer 1404(1 ) and the second layer 1404 may include layer 1404(2).
  • the layer 1404(1 ) may be upstream from the layer 1404(2).
  • the ANN 1400 may include a plurality of synapses 1406.
  • a synapse may connect a neuron 1402 of one layer to a neuron 1402 of another layer.
  • a synapse of the plurality of synapses 1406 may connect the neuron 1402(1 ) to the neuron 1402(3).
  • a synapse may include a weight.
  • the weight may include a value that adjusts a voltage that travels through the synapse. Adjusting the voltage based on the weight may include multiplying the voltage by the weight, or the like.
  • a first neuron 1402(1 ) may connect to each neuron 1402(3)-(5) of the layer immediately downstream from the first neuron. In some embodiments, the first neuron 1402(1 ) may connect to a subset of the neurons 1402(3)-(5) of the layer immediately downstream from the first neuron.
  • a neuron 1402 may include a membrane potential.
  • the membrane potential may include the current amount of voltage stored in the neuron 1402.
  • a voltage integration function of the neuron 1402 may calculate the membrane potential of the neuron 1402.
  • the voltage integration function may use as input one or more voltage amounts received from connected upstream synapses.
  • the voltage integration function may use as input the current membrane potential of the neuron 1402.
  • the voltage integration function may use one or more random values as input.
  • the output of the integration function may include a new membrane potential of the neuron 1402.
  • the integration function may include a linear function, a sigmoid function, or the like.
  • the neuron 1402(3) may receive voltage from the neuron 1402(2) through a connecting synapse of the plurality of synapses 1406.
  • the connecting synapse may adjust the voltage from the neuron 1402(2) based on the weight of the connecting synapse to generate a second voltage.
  • the synapse may send the second voltage to the neuron 1402(3).
  • the neuron 1402(3) may use the received voltage as input to the voltage integration function.
  • the current membrane potential of the neuron 1402(3) may include a value of 0.6.
  • the voltage received from the synapse may include a value of 0.2.
  • a neuron 1402 may include a threshold membrane potential.
  • the threshold membrane potential may include a value.
  • the neuron 102 may produce an output.
  • the output may include a firing message.
  • the firing message may include a voltage.
  • the neuron 1402 may send the firing message to a downstream neuron 1402.
  • the voltage of the firing message may include a numeric value resulting from the integration function, a predetermined value, or the like.
  • the threshold membrane potential of a neuron may be based on a predetermined threshold membrane potential, training, a hyperparameter search, or the like.
  • a message may include a timestamp.
  • the timestamp may include the time the message was generated, sent, or the like.
  • the timestamp may include a date and time (e.g.“1 January 2018 14:03:37”), a time since the beginning of executing the ANN 1400, or the like.
  • the timestamp may include a timestamp from a computing device that generates the message, a timestamp from an overarching computing system that schedules, coordinates, or the like the components of the ANN 1400.
  • a message may include a value.
  • a message from a synapse may include the value of the voltage received by the synapse that has been adjusted based on the weight of the synapse.
  • FIG. 15 depicts one embodiment of a method 1500.
  • the method 1500 may include a computer-implemented method.
  • the method 1500 may include a method for artificial neural network optimistic event processing.
  • the method 1500 may include receiving 1502 a synapse message.
  • a neuron may receive the synapse message.
  • the neuron may store the received synapse message.
  • the neuron may store one or more received synapse messages in a sequence in which they arrive, in which the neuron processes the synapse messages, or the like. Storing the one or more synapse messages in the sequence may allow the neuron to perform reverse computations as explained below.
  • the method 1500 may include storing 1504 a membrane potential of the receiving neuron.
  • the membrane potential may include the membrane potential of the neuron prior to receiving or processing the synapse message.
  • the storing 1504 the membrane potential may include associating the stored membrane potential with the received synapse message. Storing 1504 the membrane potential may include storing the membrane potential value in a data structure or the like.
  • the method 1500 may include adjusting 1506 the membrane potential of the neuron. Adjusting 1506 the membrane potential may be in response to receiving the synapse message, a sequence of synapse messages, or the like. In one embodiment, adjusting 1506 the membrane potential of the neuron may include executing the voltage integration function of the neuron. The receiving neuron may use a voltage value included in the synapse message, sequence of synapse messages, or the like as input for the neuron’s integration function. In one embodiment, adjusting 1506 the membrane potential may include adding a random amount to the membrane potential. Adding a random amount to the membrane potential may include using a randomly generated value as part of the voltage integration function.
  • the method 1500 may include adjusting the membrane potential of the neuron based on a leak voltage amount.
  • the neuron may include a leak function that calculates the leak voltage amount. Adjusting the membrane potential of the neuron based on the leak voltage amount may include adding the leak voltage amount to the membrane potential of the neuron.
  • the leak function may include calculating a leak value amount.
  • the leak value amount may include a constant value, a value that may change each time the leak function executes, or the like.
  • the leak value amount may include a negative value. Whether the leak value amount includes a positive value, a negative value, or a zero value may be based on random number generation.
  • the leak function may include calculating a time difference between the current time and a time the neuron last produced a firing message.
  • the neuron may calculate the time difference in response to the neuron receiving a synapse message, receiving another type of message, a timer elapsing, or the like.
  • the leak function may include calculating a leak voltage amount.
  • the leak voltage amount may be based on the time difference.
  • the leak voltage amount may include multiplying the leak value amount by the time difference.
  • the time difference may be 2 and the leak value amount may include a value of 1 .2.
  • the leak voltage amount may include a value of 2.4.
  • the neuron may execute the leak function without the neuron receiving a synapse message. In another embodiment, the neuron may execute the leak function in response to receiving a synapse message. In some embodiments, the neuron may execute the leak function in response to receiving another type of message, in response to a timer expiring, or the like. For example, in one embodiment, in response to adjusting 1506 the membrane potential of the neuron, the neuron may execute the leak function or may set a timer to execute the leak function in response to the timer expiring.
  • the method 1500 may include determining 1508 whether the membrane potential exceeds a threshold membrane potential of the neuron.
  • the determining 1508 may be in response to a predetermined amount of time expiring since adjusting 1506 the membrane potential of the neuron.
  • the predetermined amount of time may include a time interval, a number of processor cycles, or another way to track time.
  • the method 1500 in response to the membrane potential exceeding the threshold, may include producing 1510 a firing message. In some embodiments, producing 1510 the firing message may include the neuron producing an output. In one embodiment, in response to producing 1510 the firing message, the method 1500 may include reducing the membrane potential of the neuron.
  • a synapse message may include a voltage sent from a first neuron 1402(1 ) to a second neuron 1402(3) that has been calculated using the weight of the synapse.
  • the timestamp of the synapse message may include the time the synapse message was sent.
  • the neuron 1402(3) receiving the synapse message may store its current membrane potential.
  • the neuron 1402(3) may use the voltage of the synapse message as input for its integration function.
  • the neuron 1402(3) in response to the integration function of the neuron 1402(3) resulting in a membrane potential that exceeds the threshold membrane potential of the neuron 1402(3), the neuron 1402(3) may fire.
  • the neuron may send a firing message to each connecting downstream synapse.
  • producing 1510 the firing message may include producing the firing message without receiving a second synapse message from a connecting, upstream synapse.
  • a neuron may connect with multiple upstream synapses. The neuron may receive one or more synapse messages from these upstream synapses at different times. Receiving each synapse message may be similar to the step of receiving 1502 the synapse message.
  • the neuron may adjust its membrane potential based on the received one or more synapse messages, which may be similar to the step of adjusting 1506 the membrane potential.
  • the neuron may determine whether its membrane potential exceeds its threshold membrane potential, which may be similar to the step of determining 1508 whether the membrane potential exceeds the threshold membrane potential.
  • the neuron may make this determination even though it may not have received a synapse message from every connected upstream synapse.
  • the neuron may make this determination even though other neurons in the same layer may not have received one or more synapse messages from their respective connected upstream synapses.
  • producing 1510 the firing message may include scheduling the firing message to be sent at a later time. Scheduling the firing message may not include generating the firing message immediately after the determining 1508.
  • an ANN such as the ANN 1400
  • a random value may be added to the firing message time interval or the synapse message time interval. The random value may prevent two or more messages from arriving at a component of the ANN simultaneously. Preventing simultaneous arrival of messages may prevent ambiguity when performing reverse-computations, or the like as explained below.
  • the method 1500 may include, in response to producing 1510 a firing message, resetting the membrane potential of the neuron.
  • resetting the membrane potential of the neuron may include reducing the membrane potential of the neuron by a predetermined amount.
  • the predetermined amount may include an amount derived from training, a hyperparameter search, or a user configuration.
  • the method 1500 may include determining whether the membrane potential of the neuron is below a second threshold membrane potential.
  • the second threshold membrane potential may be called a “negative threshold,” a “reset threshold,” or the like. Determining whether the membrane potential of the neuron is below the negative threshold may be in response to producing 1510 the firing message, in response to the neuron performing the leak function, or the like.
  • the method 1500 may include setting the membrane potential of the neuron to a predetermined membrane potential, adding a predetermined value to the membrane potential, or the like.
  • the method 1500 may include scheduling a membrane potential determination for the neuron.
  • Scheduling the membrane potential determination may be in response to receiving 1502 the synapse message. Scheduling the membrane potential determination may include selecting a later time, processing cycle, or the like to perform the determining 1508 step of the method 1500. In some embodiments, scheduling the membrane potential determination in response to receiving 1502 the synapse message may result in the neuron determining 1508 whether its membrane potential exceeds the threshold membrane potential only after the neuron has received a synapse message. Scheduling the membrane potential determination may include scheduling the membrane potential determination in response to a membrane potential determination not already being scheduled.
  • scheduling the membrane potential determination may result in more efficient computing resource use, which may improve the functionality of one or more computing devices executing the method 1500. For example, if the neuron has not received a synapse message, the membrane potential of the neuron may not have substantially changed, and checking the membrane potential may use processing power, memory, or the like that may be used for other functions. Furthermore, in an exemplary embodiment, a time interval between a synapse scheduling a synapse message and sending the synapse message may be smaller than a time interval between scheduling a membrane potential determination and performing the membrane potential determination of the step 1508 of the method 1500. This may result in a neuron receiving multiple synapse messages, but only performing the step 1508 once instead of for each message.
  • the method 1500 may include determining whether to remove stored data associated with the neuron.
  • the stored data associated with the neuron may include a stored synapse message, a stored membrane potential, or other stored data regarding the neuron. Removing the stored data may free up memory, computing storage, or other resources of a processor device or computing device the neuron may be executing on so that the resources can be used for other data.
  • determining whether to remove stored data associated with the neuron may include calculating a global time.
  • the global time may include the time of an earliest unprocessed message in the ANN or the overarching computing system executing the ANN.
  • each component of the ANN may send the timestamp of its earliest unprocessed message to the overarching computing system.
  • Each component may send the timestamp in response to a request received from the overarching computing system or may send the timestamp periodically.
  • the overarching computing system may determine which received timestamp is the earliest.
  • the global time may include the earliest timestamp.
  • determining whether to remove stored data associated with the neuron may include the neuron receiving the global time.
  • the overarching computing system may send the earliest received timestamp to each component of the ANN, including the neuron.
  • determining whether to remove stored data associated with the neuron may include comparing each stored message, stored membrane potential, or other stored data to the global time. In response to the timestamp of the stored message, membrane potential, or the other stored data being earlier than the global time, the neuron may remove the stored message, membrane potential, or other stored data.
  • FIG. 16 depicts one embodiment of a method 1600.
  • the method 1600 may include a computer-implemented method.
  • the method 1600 may include a method for artificial neural network optimistic event processing.
  • the method 1600 may include receiving 1602, at a neuron of an ANN (such as the ANN 1400), a sequence of synapse messages.
  • the sequence of synapse messages may include one or more synapses messages and the order in which the one or more synapse messages arrived at the neuron.
  • Each synapse message may include a timestamp of the time the synapse message was sent.
  • the neuron may process the synapse messages in the order in which the synapse messages arrive at the neuron.
  • the synapse messages of the sequence of synapse messages may arrive at the neuron at different times.
  • the method 1600 may include determining 1604, based on the timestamp of each synapse message, whether the neuron processed the sequence of synapse messages out of order with respect to the timestamps.
  • Processing a synapse message may include the neuron performing the voltage integration function based on the voltage of the synapse message, performing the leak function, or the like.
  • the neuron processing the sequence of synapse messages out of order may include the neuron processing one or more synapse messages of the sequence in an order other than the order in which they were sent.
  • the neuron in response to the neuron process the sequence of synapse messages out of order, the neuron may perform the integration function using an incorrect input, which may result in the neuron firing too early, firing too late, or the like.
  • the neuron firing too early, firing too late, or the like may result in an inaccurate output from the ANN.
  • the method 1600 may include reverse-computing 1606 at least one computation performed by the neuron in response to processing the sequence of synapse messages out of order.
  • Reverse-computing the at least one computation may be known as reversing the at least one computation.
  • the at least one computation may include the voltage integration function, the leak function, or the like.
  • Reverse-computing the at least one computation may include undoing an effect of the computation.
  • reverse-computing 1606 at least one computation performed by the neuron may include reverting the membrane potential of the neuron to the value of the membrane potential before the neuron performed the at least one computation.
  • Reverting the membrane potential of the neuron may include restoring the value of the membrane potential to the value that was stored in response to receiving the out-of-order synapse message.
  • reverse computing the at least one computation may include executing a predetermined function that corresponds to the at least one computation.
  • a neuron may be configured with a certain voltage integration function and may also be configured with a corresponding reverse voltage integration function that undoes the effect of the voltage computation function.
  • the method 1600 may include performing 1608 the at least one computation based on receipt of the sequence of synapse messages in a correct order as determined by the timestamp of each synapse message in the sequence of synapse messages.
  • the correct order may include an ascending order of timestamps of the sequence of synapse messages.
  • performing 1608 the at least one computation may include the neuron performing the voltage integration function.
  • the membrane potential of the neuron in response to the neuron processing the sequence of synapse messages in the correct order, may be different after each time the neuron performs the voltage integration function than when the neuron performed the voltage integration function on the sequence of synapse messages out of order.
  • performing 1608 the at least one computation may include the neuron performing the leak function or another function.
  • reverse 1606 computing the at least one computation may include reverse-computing at least one computation corresponding to a first synapse message of the sequence of synapse messages.
  • the first synapse message may include a timestamp after the timestamp of a most-recently received synapse message.
  • Performing 1608 the at least one computation in the correct order may include performing a computation corresponding the most-recently received synapse message and the at least one computation in an order with respect to the timestamps of the most-recently received synapse message and the first synapse message.
  • the method 1600 may include producing at least one anti-message. Producing the at least one anti-message may be in response to reverse-computing a computation. An anti-message may be configured to cancel a message. The anti-message may cancel the message in response to the message having not been yet processed by the receiving component of the ANN. An anti message may be configured to cause a component of the ANN to reverse-compute a computation. In some embodiments, the anti-message may correspond to the reverse- computed computation the caused the anti-message to be produced.
  • the neuron may send one or more anti-messages to two downstream neurons.
  • the first downstream neuron may receive the anti-message, determine that it has not yet processed the message from the upstream neuron, and may cancel the message from the upstream neuron.
  • the cancelled message may correspond to the voltage summation function that was reverse-computed by the upstream neuron (e.g., the upstream neuron produced the message in response to the resulting membrane potential exceeding the threshold membrane potential).
  • Cancelling the message may include removing the message from a queue of messages waiting to be processed by the downstream neuron.
  • the second downstream neuron may have processed the message from the upstream neuron prior to receiving the anti-message from the upstream neuron.
  • the second downstream neuron may reverse-compute a computation that was triggered by the upstream neuron’s message.
  • the second downstream neuron in response to reverse computing the computation, may send one or more of its own anti-messages.
  • FIG. 17A depicts one embodiment of a sequence 1700 of synapse messages received by a neuron.
  • the sequence 1700 may include the synapse messages 1702(1 )-(5).
  • Each synapse message 1702 may include a timestamp 1704 and a voltage value 1706.
  • the sequence 1700 may include the order the neuron received the synapse messages 1702. In the example, the neuron received synapse message 1702(1 ) first, then synapse message 1702(2), then synapse message 1702(3), then synapse message 1702(4), and then the most recently received synapse message 1702(5).
  • the neuron Before the neuron received the synapse message 1702(5), the neuron received the synapse messages 1702(1 )-(4) in a correct order (as indicated by the ascending order of the timestamps 1704(1 )-(4)). However, the synapse message 1702(5) includes the timestamp 1704(5) with a value before the value of the timestamp 1704 of at least one of the previously received synapse messages 1702 (i.e. in the example of FIG. 17A, the synapse messages 1702(3) and 1702(4)). This indicates that the neuron received the sequence 1700 of synapse messages out of order.
  • the neuron may reverse-compute the computations it performed in response to processing the synapse messages 1702(3) and 1702(4).
  • Reverse-computing the computations may include reverting the membrane potential to the value of the membrane potential of the neuron before receiving the synapse message 1702(3).
  • the neuron may perform the computations corresponding to the synapse messages 1702(3)-(5) that it has received in the order indicated by the timestamps 1704(3)-(5).
  • performing the computations may include determining, after performing the integration function, whether the membrane potential of the neuron exceeds the threshold membrane potential of the neuron.
  • FIG. 17B depicts one embodiment of a sequence 1750 of synapse messages 1702 received by a second neuron (e.g ., a different neuron than the first neuron that received the sequence 1700 of synapse messages in FIG. 17B).
  • the sequence 1750 may include the synapse messages 1702(6)-(8).
  • the synapse messages 1702(6)-(8) may have been received by the second neuron. Similar to the synapse messages 1702(1 )-(5) of FIG. 17A, the sequence messages 1702(6)-(8) may each include a timestamp 1704 or a voltage value 1706.
  • the first neuron in response to reverse-computing the computations corresponding to the synapse messages 1702(3) and 1702(4), the first neuron may send one or more anti-messages 1703. For example, in response to receiving the synapse message 1702(3), the first neuron may have used the voltage value 1706(3) as input for its integration function. In response, the first neuron may have determined that its resulting membrane potential exceeded its threshold membrane potential and may have sent a firing message to a second synapse to be received at the second neuron.
  • the first neuron may have sent a firing message at the incorrect time and, thus, may have adjusted its membrane potential with the incorrect voltage value 1706, or the like.
  • the first neuron reverse-computing the computation corresponding to the synapse message 1702(3) may include sending an anti-message 1703.
  • the first neuron may send the anti message 1703 to the second neuron (i.e., a neuron that may have received the firing message at the wrong time or with the wrong timestamp 1704 or wrong voltage value 1706).
  • the anti-message 1703 may include messages data fields left unfilled (e.g., N/A values) corresponding to the firing message sent to the synapse.
  • the second neuron may receive the anti-message, reverse-compute any corresponding computations, and may send one or more anti-messages in response to one or more synapse messages corresponding to the reverse computations.
  • the anti-message 1703 may include data indicating which synapse messages 1702(6)-(8) to reverse computer or data indicating up to which point in the sequence 1750 is correct.
  • a neuron and one or more connecting synapses downstream from the neuron may perform a fan-out message delivery method.
  • the fan-out delivery method may include the following.
  • a first neuron may be coupled to one or more downstream synapses.
  • a downstream synapse may include a synapse that receives a firing message from the first neuron.
  • a downstream synapse may be coupled to a second neuron and the second neuron may be downstream from the first neuron.
  • a method may include producing, at the first neuron, a firing message.
  • the firing message may be for a synapse of the one or more downstream synapses.
  • a“firing message being for a synapse” may include the firing message having the synapse as its destination, target, or the like. Producing the firing message may include the first neuron producing an output.
  • the method in response to the synapse receiving the firing message, may include producing a synapse message for the second neuron.
  • the method may include producing the firing message for a second synapse of the one or more downstream synapses.
  • producing the synapse message for the second neuron and producing the firing message for the second synapse may include a synapse producing the synapse message and the firing message.
  • Each synapse producing these two messages (in some embodiments, with the exception of the last synapse not producing the firing message) may result in fewer total messages being produced and may result in lower memory usage.
  • FIG. 18 depicts one embodiment of a method 1800.
  • the method 1800 may include a computer-implemented method.
  • the method 1800 may include a method for artificial neural network optimistic event processing.
  • the method 1800 may include providing 1802 the ANN (such as the ANN 100).
  • the ANN may include a synapse group.
  • the synapse group may include one or more synapse.
  • the one or more synapse may be coupled to a neuron.
  • the one or more synapse may be downstream of the neuron.
  • the synapse group may include a logical process (explained below).
  • the logical process may include the one or more synapse.
  • the one or more synapse may include synapses downstream from a single neuron or multiple neurons.
  • the synapses may include all synapses downstream of a single neuron or a portion of the synapses downstream of a single neuron.
  • the synapse group logical process may receive one or more messages destined for a synapse included in the synapse grouping.
  • the method 1800 may include receiving 1804, at the synapse group, a firing message from the neuron.
  • the firing message may include an output of the neuron.
  • the firing message may include a value of the output of the neuron.
  • the method 1800 may include, in response to receiving the firing message, producing 1806 a first synapse message for a first synapse of the one or more synapse.
  • a“synapse message for a first synapse” may include a synapse message originating from the first synapse.
  • the first synapse message may include the value of the output of the upstream neuron adjusted based on the weight of the first synapse.
  • the method 1800 may include determining 1808 whether the one or more synapse includes a waiting synapse.
  • the waiting synapse may include a synapse that has not had a respective synapse message produced for it in response to the firing message.
  • the method 1800 in response to determining that the one or more synapse includes the waiting synapse, the method 1800 may include producing 1810 the synapse message for the waiting synapse. Producing 1810 the synapse message may include scheduling the synapse message to be sent at a later time. In some embodiments, producing 1810 the synapse message may be similar to producing 210 the firing message.
  • the method 1800 may include waiting 1812, at the synapse group, for a second firing message from the neuron.
  • waiting 1812 at the synapse group does not necessarily include the synapse group doing nothing, idling, or the like.
  • the synapse group may receive firing messages from other neurons and may process those firing messages.
  • FIG. 19 depicts a flow diagram 1900 of one embodiment of a synapse group processing a firing message received from a neuron.
  • the flow diagram 1900 includes first neuron 1902, a synapse group 1904, a second neuron 1906, a third neuron 1908, and a fourth neuron 1910.
  • the first neuron 1902 is upstream from the second, third, and fourth neurons 1906, 1908, 1910 and is connected to each of these neurons through a first, second, and third, synapse.
  • the synapse group 1904 includes the first, second, and third synapses.
  • the flow begins, and at 1912, the first neuron 1902 sends a firing message to the synapse group 1904.
  • the voltage of the firing message may include a value of 1 .4.
  • the synapse group 1904 may adjust the voltage of the firing message based on the weight of the first synapse of the synapse group 1904. In this embodiment, adjusting the voltage of the firing message may include multiplying the voltage by the weight of the first synapse.
  • the weight of the first synapse may include a value of 0.6.
  • the resulting voltage includes the value of 0.84.
  • the synapse group 1904 may produce a first synapse message for the first synapse.
  • the first synapse message may include the resulting voltage value of 0.84.
  • the first synapse message may be destined for the second neuron 1906.
  • the synapse group 1904 may determine whether the first, second, and third synapses include a waiting synapse. In response to the second and third synapses not having yet sent a synapse message in response to the synapse group 1904 receiving the firing message from the first neuron 1902, the synapse group 1904 may determine that first, second, and third synapses do include a waiting synapse.
  • the synapse group 1904 may adjust the voltage of the firing message based on the weight of the second synapse by multiplying the voltage of the firing message (1 .4) by the weight of the second synapse (in this example, 1 .1 ).
  • the synapse group 1904 may produce a synapse message with the resulting voltage of 1 .54 destined for the third neuron 1908.
  • the synapse group 1904 may determine whether the first, second, and third synapses include a waiting synapse. The synapse group 1904 may determine that the third synapse is a waiting synapse.
  • the synapse group 1904 may adjust the voltage of the firing message based on the weight of the third synapse by multiplying the voltage of the firing message (1 .4) by the weight of the third synapse (in this example, 0.8).
  • the synapse group 1904 may produce a synapse message with the resulting voltage of 1 .12 destined for the fourth neuron 1910.
  • the synapse group 1904 may determine whether the first, second, and third synapses include a waiting synapse.
  • the synapse group 1904 may determine that the first, second, and third synapses do not include a waiting synapse. In response, the synapse group 1904 may wait for a second firing message.
  • using a synapse group logical process instead of one logical process per synapse may reduce the amount of memory used by logical processes. This may allow embodiments with a larger ANN, which may be able to perform more accurately, faster, more efficiently, or the like.
  • Using a synapse group logical process may reduce the number of synapse messages sent between processing cores, over the data network, or the like. This may reduce congestion over the system bus, data network, or the like, which may decrease the chances of a message arriving at a logical process late, and thus, may decrease the number of times a logical process may perform reverse computations.
  • FIG. 20 depicts one embodiment of a method 2000.
  • the method 2000 may include a computer-implemented method.
  • the method 2000 may include a method for artificial neural network optimistic event processing.
  • the method 2000 may include receiving 2002, at a neuron of an artificial neural network, a sequence of synapse messages. Each synapse message may include a timestamp of the time the synapse message was sent.
  • receiving 2002 the sequence of synapse messages may be similar to the step of receiving 1602 the sequence of synapse messages of the method 1600.
  • the neuron may be configured to produce a correct output when the neuron processes the synapse messages in a correct order with respect to the timestamps of the synapse messages.
  • the correct output may include a firing message sent at a certain time and with a certain voltage that would result from the neuron processing the sequence of synapse messages in a correct order as indicated by the timestamps of the synapse messages.
  • the method 2000 may include determining 2004, based on the timestamp of each synapse message, whether the neuron processed the sequence of synapse messages out of order with respect to the timestamps. In one embodiment, determining 2004 whether the neuron processed the sequence of synapse messages out of order may be similar to the step of determining 1604 whether the neuron processed the sequence of synapse messages out of order of the method 1600.
  • the method 2000 may include determining 2006 whether an output produced by the neuron in response to processing the sequence of synapse messages out of order (i.e.“the actual output”) differs from the correct output.
  • An output may include a firing message produced by the neuron.
  • a neuron may process a sequence of synapse messages out of order, but it may not cause the actual output to be substantially different from the correct output. In response to the actual output not being substantially different from the correct output, the neuron may discard the out of order message.
  • a neuron may receive a first synapse message with a timestamp of “4.” The neuron may process the first synapse message by adjusting its membrane potential based on the voltage of the first synapse message and the voltage integration function of the neuron. Next, the neuron may determine that the membrane potential of the neuron exceeds the threshold membrane potential and may produce a firing message.
  • the firing message may include a voltage. The voltage may include a constant value that the neuron includes each firing message and may not be based on the neuron’s membrane potential at the time of producing the firing message.
  • the neuron may receive a second synapse message with a timestamp of “3,” which may indicate that the second synapse message should have been received before the first synapse message.
  • the neuron may discard the second synapse message.
  • processing the sequence of synapse messages may cause the actual output to differ from the correct output.
  • a neuron may receive a first synapse message with a timestamp of“7.” The neuron may process the first synapse message by adjusting its membrane potential based on the voltage of the first synapse message and the voltage integration function of the neuron. Next, the neuron may calculate a leak voltage and add it to the membrane potential. Next, the neuron may determine that the membrane potential of the neuron does not exceed the threshold membrane potential (and, thus, the neuron does not produce a firing message).
  • the neuron may receive a second synapse message with a timestamp of “3,” which may indicate that the second synapse message should have been received before the first synapse message.
  • the second synapse message may include a voltage that, if the neuron had received the second synapse message earlier, would have caused the membrane potential of the neuron to exceed the threshold membrane potential (and, thus, produce a firing message) earlier.
  • the actual output differs from the correct output.
  • the method 2000 may include reverse-computing 2008 at least one computation performed by the neuron in response to processing the sequence of synapse messages out of order.
  • the method 2000 may include performing 2010 the at least one computation based on receipt of the sequence of synapse messages in the correct order to produce the correct output.
  • Reverse-computing 2008 the at least one computation and performing 2010 the at least one computation may be similar to the steps of reverse-computing 306 the at least one computation performed by the neuron and performing 1608 the at least one computation of the method 1600, respectively.
  • the output produced by the neuron in response to processing the sequence of synapse message out of order may differ from the correct output for a variety of reasons.
  • the actual output being different from the correct output may include the neuron producing the output at a time different than a time the neuron would have produced the correct output.
  • the actual output differing from the correct output may include the neuron producing the output with a voltage of a firing message different than a voltage of a firing message of the correct output.
  • a neuron may receive a first synapse message with a timestamp of “5” and a second synapse message with a timestamp of “7.”
  • the neuron may process the first and second synapse messages and adjust its membrane potential based on the voltages of the first and second synapse messages and the voltage integration function of the neuron.
  • the adjusted membrane potential may be 13.4.
  • the neuron may determine that the membrane potential 13.4 exceeds the threshold membrane potential and produce a firing message.
  • the firing message may include the voltage 13.4.
  • the neuron may receive a third synapse message with a timestamp of “6.” Had the neuron received the first, second, and third synapse messages in the correct order, the firing message would have included a voltage of 15.3. Thus, in this example, the actual output includes the voltage 13.4 and the correct output would have included the voltage 15.3, and thus the actual output differs from the correct output.
  • a neuron may discard a received out of order synapse message for a variety of reasons.
  • the neuron may discard a received synapse message in response to the actual output of the neuron not substantially differing from the correct output, as explained above.
  • the neuron may discard a received synapse message in response to a user configuration of the neuron, the ANN, or other component of the ANN.
  • the neuron may discard a received synapse message in response to the time difference between the timestamp of the message and the time the message arrived at the neuron being more than a threshold time difference.
  • the threshold time difference may include 10.
  • the difference may be 15, which is more than the threshold time difference of 10.
  • the neuron may discard the received synapse message.
  • FIG. 21 depicts one embodiment of a system 2100.
  • the system 2100 may include one or more processor devices 2102, such as the processor devices 2102(1 )-(3) of the FIG. 21 .
  • the one or more processor devices 2102 may include at least two processor devices 2102(1 )-(2).
  • a processor device 2102 may be in data communication with at least one other processor device 2102.
  • a processor device 2102 may be in data communication with another processor device 2102 through a communication bus 2104.
  • a processor device 2102 may include a central processing unit (CPU) (including a CPU core), a graphics processing unit (GPU), a multi-core processor, a dataflow engine, or the like.
  • CPU central processing unit
  • GPU graphics processing unit
  • a processor device 2102 may include an application-specific integrated circuit (ASIC).
  • a processor device 2102 may include an asynchronous processor. Examples of an asynchronous processor may include a neuromorphic processor, a non-Von Neumann-type processor, or the like.
  • the system 2100 may include multiple types of processor devices 2102.
  • a processor device 2102 may execute one or more processes, software modules, program instructions, or the like.
  • a logical process may include a computer process, computer program, software module, program instructions, a portion thereof, or the like.
  • a logical process may include a component of the ANN.
  • a logical process may include a neuron.
  • a logical process that includes a neuron may be called a“neuron logical process” 2106.
  • a logical process may include a synapse.
  • a logical process that includes a synapse may be called a“synapse logical process” 2108.
  • a logical process may communicate with another logical process through a message.
  • the message may include data, a timestamp of the time the message was sent, or the like.
  • a message may be similar to a message as described above in relation to synapse messages, firing messages, or the like.
  • a message may be sent between logical processes executing on the same processor device 2102.
  • a message may be sent between logical processes executing on different processor devices 2102 through the communication bus 2104.
  • the communication bus 2104 may include one or more software or hardware components that may communicate a message from one processor device 2102 to another processor device 2102.
  • the communication bus 2104 may include circuitry between two processor devices 2102 that reside on the same computing device, a data network connecting different computing devices, or the like.
  • a logical process may be configured to perform one or more steps, functions, or the like.
  • the logical process may perform these steps, functions, or the like in response to being executed by a processor device 2102.
  • the steps, functions, or the like may include one or more steps of the method 1500, the method 1600, the method 1800, the method 2000, or the like.
  • the system 2100 may include one or more neuron logical processes 2106, such as the neuron logical processes 2106(1 )-(2) of the FIG. 21 .
  • the one or more neuron logical processes 2106 may include at least two neuron logical processes 2106.
  • a neuron logical process 2106 may include a neuron of an artificial neural network (ANN), such as the ANN 100.
  • ANN artificial neural network
  • a processor device 2102 may include a neuron logical process 2106 executing on the processor device 2102.
  • the system 2100 may include one or more synapse logical processes 2108, such as the synapse logical processes 2108(1 )-(2) of the FIG. 21 .
  • a synapse logical process 2108 may include a synapse of an ANN. The synapse may connect to a neuron of a neuron logical process 2106.
  • a processor device 2102 may include the synapse logical process 2108 and may execute the synapse logical process 2108.
  • the synapse logical process 2108 may send one or more synapse messages to a neuron logical process 2106.
  • the synapse logical process 2108 may receive a firing message from a neuron logical process 2106.
  • the system 2100 may include multiple processor devices 2102, each processor device 2102 executing at least one neuron logical process 2106 or synapse logical process 2108.
  • a processor device 2102 may include multiple logical processes executing on the processor device 2102.
  • the processor device 2102(3) may execute the neuron logical process 2106(2) and the synapse logical process 2108(2). More than one logical process of the same type may execute on a single processor device 2102.
  • the number of processor devices 2102, logical processes, or the like of the system 2100 may include tens, hundreds, thousands, millions, or the like.
  • the system 2100 may include a supercomputer, compute cluster, or the like.
  • the system 2100 may be configured to perform one or more functions of an ANN such as training, a hyperparameter search, inference calculations, or the like.
  • a neuron logical process 2106 may be executed by a processor device 2102.
  • the neuron logical process 2106 when executed by the processor device 2102, may be configured to perform one or more steps, functions, or the like.
  • the neuron logical process 2106 may be configured to receive a first synapse message.
  • the first synapse message may include a voltage.
  • the neuron logical process 2106 being configured to receive the first synapse message may be similar to receiving 1502 a synapse message of the method 1500.
  • the neuron logical process 2106 may be configured to adjust a membrane potential of the neuron of the neuron logical process 2106.
  • the logical process 2106 may adjust the membrane potential based on the voltage of the synapse message that the neuron logical process 2106 received.
  • the neuron logical process 2106 adjusting the membrane potential of the neuron may be similar to adjusting 1506 the membrane potential of the neuron of the method 1500.
  • the neuron logical process 2106 may be configured to determine whether the membrane potential of the neuron exceeds a threshold membrane potential of the neuron. In some embodiments, the neuron logical process 2106 may be configured to determine whether the membrane potential of the neuron exceeds a threshold membrane potential of the neuron may be similar to determining 1508 whether the membrane potential exceeds the threshold membrane potential of the method 1500.
  • the neuron logical process 2106 in response to the membrane potential of the neuron exceeding the threshold membrane potential, may be configured to produce an output.
  • the output may include a firing message.
  • the neuron logical process 2106 may be configured to produce the output without receiving a second synapse message from a connecting, upstream synapse.
  • a neuron logical process 2106 may receive one or more synapse messages from one or more synapse logical processes 2108, and the synapses of the one or more synapse logical processes 2106 may be upstream and connected to the neuron of the neuron logical process 2106.
  • the neuron may receive one or more synapse messages from some, but not all, the upstream, connected synapses. However, based on at the voltages of the synapse messages the neuron has received, the neuron may determine that the neuron’s membrane potential exceeds the threshold membrane potential of the neuron. Thus, the neuron may not need to wait to receive a synapse message from the other upstream, connected synapses to produce an output. In one embodiment, a neuron may not need to wait until other neurons in the same layer have determined whether their respective membrane potentials exceed their respective threshold membrane potentials to send a firing message.
  • the system 2100 may include one or more scheduler processes 21 10, such as the scheduler processes 21 10(1 )-(3).
  • Each processor device 2102 may include a scheduler process 21 10.
  • Each scheduler process 21 10 may be configured to determine an execution order for at least one logical process executing on the processor device 2102.
  • a scheduler process 21 10 may include a computer process or the like that schedules one or more processes to execute on the processor device 2102.
  • the scheduling may include ordering the processes in an execution order.
  • the execution order, how long each process runs during its scheduled time, or the like may be based on system 2100 configurations, processor device 2102 configurations, operating system configurations, or the like.
  • multiple logical processes may execute on a single processor device 2102.
  • the processor device 2102(3) may include a neuron logical process 2106(2) and a synapse logical process 2108(2).
  • the scheduler process 21 10(3) of the processor device 2102(3) may determine in what order the neuron logical process 2106(2), the synapse logical process 2108(2), and other processes executing on the process device 2102(3) may execute, how long each process may execute, or the like.
  • a scheduler process 21 10 may be configured to determine, based on a timestamp of a synapse message of a sequence of synapse messages, whether the neuron logical process executing on the processor device of the scheduler process received the sequence of synapse messages out of order with respect to the timestamps.
  • a processor device 2102, a neuron logical process 2106, or the like may receive a sequence of synapse messages.
  • the processor device 2102, a neuron logical process 2106, or the like may receive the sequence of synapse messages faster than the neuron logical process 2106 may process them.
  • the sequence of synapse messages may arrive out of order with respect to the timestamp of each synapse message.
  • the scheduler process 21 10 may sort the sequence of synapse messages received by the neuron logical process 2106 in a correct order as indicated by the timestamps of the sequence of synapse messages.
  • the processor device 2102(1 ) may receive three synapse messages.
  • the timestamp of the first synapse message may include the time“4,” the timestamp of the second synapse message may include the time “7,” and the timestamp of the third synapse message may include the time“5.”
  • the neuron logical process 2106(1 ) may be processing other received messages and may not begin processing the first, second, or third synapse messages immediately upon receipt.
  • the scheduler process 21 10(1 ) may sort the first, second, and third synapse messages.
  • the neuron logical process 2106(1 ) may, in response to finishing processing its previous messages, process the first synapse message, then the third synapse message, and then the second synapse message.
  • the processor device 2102(1 ) may receive three synapse messages.
  • the timestamp of the first synapse message may include the time “4”
  • the timestamp of the second synapse message may include the time“7”
  • the timestamp of the third synapse message may include the time“5.”
  • the processor device 2102(1 ) may receive the third synapse message much later than the first and second synapse messages.
  • the neuron logical process 2106(1 ) may process the first and second synapse messages before processing the third synapse message.
  • the neuron logical process 2106(1 ) in response to determining that the neuron logical process 2106(1 ) received the first, second, and third synapse messages out of order, may reverse-compute the second synapse message and may send a corresponding anti-message as described in relation to the method 1600.
  • an ANN may include an axon.
  • the axon may connect to a neuron and may be downstream from the neuron.
  • the axon may connect to the one or more downstream synapses of the neuron.
  • a logical process for an axon i.e. an“axon logical process”
  • the axon logical process may send the firing message to each connecting downstream synapse logical process, synapse group logical process, or the like.
  • a neuron logical process may not send a message to each connecting downstream neuron, but only to the downstream connecting axon.
  • the ANN discussed herein may differ from a simulated ANN.
  • a simulation is concerned with a subset of the behavior of an ANN. The subset behavior may include behavior being tested to determine whether the theoretical behavior may occur in an actual implementation.
  • a simulated ANN simulates a portion of the data used to execute an implementation of an ANN.
  • a simulated ANN does not include overhead data or overhead processing used to configure the implemented ANN.
  • Overhead data may include data used to create a neuron or a synapse, a neuron logical process or synapse logical process, a firing message or a synapse message, or other components of the implementation of the ANN.
  • Overhead processing may include the processing performed by a processor device to create the aforementioned components of the implemented ANN, the processing performed to switch between processes on the processor device, or other processing performed in the execution of the implemented ANN.
  • a simulated ANN is concerned with the adherence of the simulation to certain conditions in order to test the theoretical behavior.
  • certain functions of an implementation of an ANN are not present in a simulated ANN. This is because the implementation of an ANN is more concerned with obtaining results and is less concerned with testing correct behavior.
  • a neuron would not determine whether the actual output of the neuron differs from the correct output to determine whether to reverse-compute one or more computations.
  • the simulation would reverse-compute each time a sequence of synapse messages was received out of order because the simulation of the ANN is simulating the neurons of the ANN sending the firing messages at the correct times.
  • Other behaviors of a simulated ANN may not be present in an implemented ANN.
  • an ANN of one or more of the embodiments described herein may improve the functionality of a computing device and may include non-conventional steps, functions, components, or arrangements of components.
  • a neuron of an ANN may perform its integration function, leak function, or the like in an asynchronous manner.
  • the neuron executing in an asynchronous manner may include the neuron not waiting to receive input from all connected upstream synapses before producing a firing message, unlike a conventional ANN.
  • the neuron executing in an asynchronous manner may include the neuron not waiting for all of the neurons in the same layer to receive all of their respective inputs, unlike a conventional ANN. In some embodiments, this may result in a neuron of the ANN not expending computing resources when the neuron has not received any input, unlike a conventional, synchronous ANN.
  • a faster execution of one or more functions of the ANN which is an improvement of computer functionality.
  • This improvement in computer functionality may be due to these non-conventional steps, functions, or components and may not be due to faster computer hardware.
  • the asynchronous functionality of the ANN such as may be described above in relation to the methods 200, 300, 500, or 700 may occur during training of the ANN or during inference calculation in the ANN.
  • the neuron process messages from other components of the ANN as they arrive at the neuron, and in the case of a sequence of messages that arrive out of order, the neuron may include the non-conventional steps of reverse-computing the out-of-order messages and processing the messages in a correct order.
  • an analytic model (such as an Al model of the enrichment application 404 or the exploitation application 406 of FIG. 4A) may implement one or more steps of the methods described above or one or more components of the optimistic event processing systems described herein. Other components of the system 100 may use the systems and methods of optimistic event processing.
  • using reservoir computing techniques along with optimistic execution may result in increased scalability of an ANN.
  • the increased scalability may include high levels in scalability.
  • Using reservoir computing techniques with optimistic execution may improve runtime speed or accuracy.
  • a system using the reservoir computing techniques with optimistic execution may use only CPUs, only GPUs, only another types of processor, or a combination of types of processors.
  • FIG. 22 depicts one embodiment of an ANN.
  • the ANN may include an echo state network (ESN) 2200.
  • the ESN 2200 may include an RNN.
  • the RNN may include a sparsely connected RNN.
  • the RNN may include a randomly connected RNN. As can be seen in FIG.
  • the ESN 2200 may include an input layer 2202.
  • the input layer 2202 may include one or more input neurons 2204(1 )- (3).
  • the input neurons 2204 of the input layer 2202 may receive an input signal value, may perform signal integration, and may output a resulting signal based on the signal integration.
  • the input neurons 2204 may transmit resulting signals to a reservoir 2206.
  • the input neurons 2204 may transmit the resulting signals via one or more synapses.
  • the one or more synapses may each include a weight associated with the synapse. The weight may adjust a value of the signal the connected input neuron 2204 may transmit through the synapse, as described above.
  • the signal input to the input neurons 2204, the signal integration of the input neurons 2204, and sending the signal via synapses may be similar to the functions of the ANN 1400 described above.
  • the ESN 2200 may include a reservoir 2206.
  • the reservoir 2206 may include multiple neurons.
  • the reservoir 2206 may include multiple synapses.
  • a synapse of the reservoir 2206 may connect one neuron of the reservoir 2206 to another neuron of the reservoir 2206.
  • a synapse may connect a neuron of the reservoir 2206 to the same neuron.
  • the reservoir 2206 may include neurons arranged in an acyclic or cyclic manner.
  • a reservoir 2206 that includes at least one cyclic portion improves the reservoir’s 2206 capacity to retain values, patterns, or other information.
  • the neurons may perform signal integration and the synapses may perform signal value adjustment based on the weight of the synapse.
  • the weights of the synapses connecting neurons of the reservoir 2206 may not change during training.
  • the weights of the synapses connecting the input layer 2202 to the reservoir 2206 do not change during training.
  • the ESN 2200 may include output synapses 2208.
  • the output synapses 2208 may connect one or more neurons of the reservoir 2206 to an output layer 2210.
  • the output layer 2210 may include one or more output neurons 2212(1 )-(2).
  • the output synapses 2208 may perform signal adjustment based on their respective weights, and the output neurons 2212 may perform signal integration.
  • the weights of the output synapses 2208 may change during training, unlike the other synapses of the ESN 2200.
  • the weights of the output synapses 2208 may change during training based on backpropagation or some other weight-adjustment algorithm. Adjusting only the output synapses 2208 provides time and energy savings. Updating the weights of the other synapses, such as the synapses of the reservoir 2206 or the synapses connecting the input layer 2202 to the reservoir 2206 often does not provide significant improvement in accuracy and uses computing time and resources.
  • the neurons of the reservoir 2206 may be randomly connected to each other.
  • the reservoir 2206 may include a number of neurons n.
  • the one or more input neurons 2204 may be randomly connected to random neurons of the reservoir 2206.
  • the one or more output neurons 2212 may be randomly connected to random neurons of the reservoir 2206.
  • the output neurons 2212 may include one or more synapses from one or more output neurons 2212 back to one or more neurons of the reservoir 2206.
  • the ESN 2200 may execute optimistically similar to the optimistic event execution described above in relation to FIGS. 15, 16, 18, or 20.
  • the ESN 2200 may include an echo state property.
  • the echo state property may prevent the ESN 2200 from perpetuating non-reinforced data that may be based on prior conditions or states of the components of the ESN 2200.
  • the echo state property may reduce the value of an input signal by a predetermined amount.
  • the predetermined amount may include a portion of the input signal value.
  • the portion may include a small portion, for example, less than 50%, of the input signal value, or the portion may include other amounts of the input signal value.
  • signals that are not directly reinforced may have their values reduce to zero (or some other low value) after a predetermined amount of time.
  • the echo state property may prevent perpetuation of non-reinforced data in other ways.
  • the ESN 2200 results in an artificial neural network that is generated faster and that executes faster without sacrificing a significant amount of accuracy. For example, updating only the weights of the synapses connected to the output neurons 2212, as described herein, instead of updating the weights of all neurons of the ESN 2200 generates the ESN 2200 faster. Furthermore, executing the ESN 2200 using optimistic event processing, as described herein, provides a faster execution over previous efforts. The neurons of the ESN 2200 executing optimistically and not waiting on other components of the ESN 2200 to finish computing increases the speed of the ESN 2200. These features reduce the time for training and executing down to a multi-minute level (as opposed to days or weeks). These features provide a transformative leap to near real-time predictions.
  • these features allow computing devices that execute the ESN 2200 to achieve these benefits using only CPUs. Using only CPUs, as opposed to additional GPUs or tensor processing units (TPUs) make this approach even more desirable from a cost-savings standpoint.
  • Increasing the number of computing nodes executing the ESN 2200 results in runtimes that are reduced at a greater degree than efforts of the prior art. In one embodiment, increasing the number of cores by 2x to 10x provides a runtime reduction from 25% to over 50%.
  • a method for rollback in the system 100 may include a state-saving rollback method.
  • the state-saving rollback method may include providing, at a data storage, a first set of data.
  • the state-saving rollback method may include receiving a second set of data.
  • the state-saving rollback method may include saving the first set of data, the first set of data including prior data.
  • saving the first set of data may include incremental state saving or may include full copy saving.
  • the state-saving rollback method may include storing, at the data storage, a third set of data based on the first and second sets of data.
  • the state-saving rollback method may include receiving a trigger to rollback.
  • the state-saving rollback method may include restoring the first set of data to the data storage.
  • the data storage may include the cloud storage of the cloud environment 102.
  • the first set of data may include the dataspace of the system 100 or a portion of the dataspace.
  • the first set of data may include an empty data set or no data in response to, for example, the dataspace being empty.
  • the first set of data may include data in the dataspace being used by or associated with a mission application, management application, or data management application.
  • the data storage may include computer memory (e.g ., the computer memory of a CCN executing an application), or another device of the system capable of storing data.
  • receiving the second set of data may include one or more steps of a method for data flow and data fusion as discussed herein.
  • receiving the second set of data may include the step of receiving external data.
  • the ASIS 304 may receive external data and store the received data in the dataspace via the DSMS 306.
  • the import component may receive data from another instance of a system 100 and insert the received data into the dataspace via the DSMS 306.
  • receiving the second set of data may include the ASIS 304 receiving external data ⁇ e.g., from user input or an external system) and providing the external data to an application executing in the system.
  • receiving the second set of data may include analyzing the second set of data. Analyzing the second set of data may include the step of analyzing the received data of the method for data flow and data fusion. Analyzing the second set of data may include performing an element count on the second set of data or normalizing values of a data column within the second set of data. In one embodiment, receiving the second set of data may include receiving a command to remove data from the first set of data.
  • saving the first set of data may include storing the first set of data in a predetermined data storage location called the“rollback state storage.” Storing the first set of data in the rollback state storage location may include copying or moving the first set of data from the data storage to the rollback state storage location. Storing the first set of data in the predetermined data storage location may include designating the portion of the data storage storing the first set of data as rollback state storage or a previous version of the second set of data.
  • saving the first set of data may include incremental state saving.
  • Incremental state saving may include saving a portion of the dataspace.
  • the saved portion of the dataspace may include the first set of data.
  • Saving the first set of data may include the DSMS 306, CUSS 206, CSSS 208, CSN, or another cloud environment 102 or information environment component copying the first set of data into a rollback state storage portion of the cloud storage of the cloud environment 102 or designating the storage location in the cloud environment 102 of the first set of data as rollback state storage.
  • incremental state saving may include saving metadata about the first set of data.
  • the metadata may include a timestamp for the date and time the first set of data was saved to the rollback state storage, data that corresponds the first set of data with the second set of data (i.e., the data that may have triggered the incremental state save), or other state data for the first set of data.
  • storing, at the data storage, a third set of data based on the first and second sets of data may include updating data stored in the data storage based on the received second set of data.
  • Storing the third set of data may include designating the third set of data as the official or current version of the data.
  • storing the third set of data may include updating the dataspace based on the received second set of data. For example, if the second set of data includes data that is being inserted into the dataspace (e.g ., new data added to the dataspace), then the third set of data may include the first set of data combined with the second set of data.
  • the third set of data may include the first set of data modified by the second set of data. If the second set of data includes data that removes at least a portion of the first set of data, then the third set of data may include an empty data set or the first data set without the removed portion.
  • the DSMS 306 may coordinate the CUSS 206 or the CSSS 208 to store data in, modify the data of, or remove data from the dataspace.
  • restoring the first set of data to the data storage may include copying or moving the first set of data from the rollback state storage location to the data storage.
  • Restoring the first set of data may include removing the third set of data from the dataspace.
  • Restoring the first set of data may include designating the first set of data as the official or current version of the data or removing that designation from the third set of data.
  • restoring the first set of data may include restoring multiple saved first sets of data to the dataspace until a specified saved first set of data is restored.
  • Restoring the multiple first sets of data may include restoring the saved first sets of data in a reverse order of when they were saved to the rollback state storage.
  • restoring the first set of data may include selecting a certain first set of data from the multiple saved first sets of data and restoring the selected first set of data.
  • a method for state-saving rollback may full copy saving.
  • Full copy saving may include saving the entire dataspace as the first set of data.
  • saving the first set of data may include full copy saving.
  • Full copy saving may include saving the entire dataspace.
  • the DSMS 306 may copy the data in the dataspace to the rollback state storage of the cloud storage or designating the storage location of the dataspace as rollback state storage.
  • Full copy saving may include saving metadata about the dataspace.
  • saving the first set of data may include saving metadata associated with the first set of data in a rollback history.
  • the rollback history may track changes to a portion of the dataspace, a file, or the entire dataspace and store data about those changes in an order ( e.g ., from the most recent changes to the least recent).
  • the system 100 may include multiple rollback histories, each rollback history being associated with a different file, portion of the dataspace, a CSN, other data storage.
  • Saving the first set of data may include updating multiple rollback histories that are associated with the first set of data.
  • the rollback history may be stored on a CSN, on a data storage attached to the node executing the DSMS 306, or data storage attached to another device of the information environment.
  • the exploitation application 406 may derive a pattern or discover data associations between data elements in one or more data artifacts.
  • the exploitation application 406 may store data based on the pattern or data associations in the mission dataspace 120, the operational dataspace, or another data storage location in the system 100.
  • the pattern or association data generated by the exploitation model may also be rolled back.
  • the assertion application 508 may rollback an assertion.
  • the assertion may be located in the mission dataspace 120 or some other data storage location.
  • the assertion may be associated with data stored in the mission dataspace 120. In response to the data associated with the assertion being rolled back, the assertion may also be rolled back.
  • the method for state-saving rollback may apply to other data storage.
  • the first set of data may include data associated with an application executing on the system 100, such as a membrane potential of a neuron in an ANN as described above in relation to optimistic event processing.
  • a CCN 204 or a job executing on a CCN 204 may include the one or more rollback histories, rollback state storage, or other rollback data or components described herein.
  • a first set of data may include a text file stored in Cloud Unstructured Storage on a first CSN.
  • the ASIS 304 may have moved the text file from outside the system into the system, the SIA 402 component may have ingested the text file, and the DSMS 306 may have stored the ingested text file in the Cloud Unstructured Storage on the first CSN.
  • the DSMS 306 may receive a second set of data using the same or a similar process.
  • the second set of data may include a new, updated version of the text file of the first set of data.
  • the DSMS 306 may copy or move the original text file from the first CSN to second CSN that includes rollback state storage.
  • the DSMS 306 may store metadata in the rollback state storage that associates the original text file with the updated text file, the first CSN, a storage location of the first CSN, or other data.
  • the DSMS 306 may store metadata in a rollback history associated with the text file.
  • the DSMS 306 may store the updated text file on the first CSN and may update any file references to the text file to the updated text file.
  • the DSMS 306 may receive a trigger to rollback the original version of the text file.
  • the DSMS 306 may determine from one or more rollback histories the location of the original text file on the second CSN.
  • the DSMS 306 may remove the updated text file and move the original text file from the second CSN to the first CSN.
  • the DSMS 306 may update any references to the text file to the original text file in the first CSN.
  • One embodiment of a method for rollback may include a reverse computing rollback method.
  • the method for reverse-computing rollback may include providing, at a data storage, a first set of data.
  • the method for reverse-computing rollback may include receiving a second set of data.
  • the method for reverse- computing rollback may include generating a third set of data based on the first set of data, the second set of data, and a forward computation.
  • the method for reverse computing rollback may include replacing, at the data storage, the first set of data with the third set of data.
  • the method for reverse-computing rollback may include receiving a rollback trigger.
  • the method may include, in response to receiving the rollback trigger, generating the first set of data based on a reverse computation corresponding to the forward computation.
  • the method may include replacing, at the data storage, the third set of data with the generated first set of data.
  • the data storage may include the cloud storage of the cloud environment 102, the dataspace, a computer memory, or another data storage location as described above in relation to the state-saving rollback method.
  • the step of providing the first set of data or receiving the second set of data may include one or more steps that may be similar to one or more steps of the state-saving rollback method.
  • the forward computation may include a set of computer program instructions or may include other computer functionality.
  • the forward computation may receive one or more inputs.
  • the one or more inputs may include the first set of data, the second set or data, or other data.
  • generating the third set of data based on the first set of data, the second set of data, and the forward computation may include using the first and second sets of data as inputs to a set of computer program instructions, and the set of instructions may generate the third set of data.
  • replacing, at the data storage, the first set of data with the third set of data may be similar to the step of replacing the first set of data with the third set of data of the method for state-saving rollback.
  • replacing the first set of data with the third set may include storing metadata associated with the first, second, or third sets of data or storing at least a portion of the first or second sets of data in a rollback history.
  • the reverse computation may include a second set of computer program instructions or other computer functionality.
  • the reverse computation may correspond to or be associated with the forward computation.
  • the reverse computation may be configured such that executing the reverse computation with one or more inputs, such as the third set of data or the second set of data, may generate the first set of data.
  • generating the first set of data based on a reverse computation corresponding to the forward computation may include executing the reverse computation.
  • the reverse computation may use the third set of data as input.
  • the reverse computation may use the second set of data as input.
  • replacing the third set of data with the first set of data may be similar to the step of replacing the first set of data of the method for state-saving rollback.
  • a first set of data may include a set of numbers.
  • the set of numbers may be associated with a mission application executing in the system.
  • the set of numbers may be stored in the memory of a CCN executing the mission application.
  • the second set of data may include a number.
  • the mission application may receive the number from a user via a component of the overarching system such as a tactical edge application, CAPN, or other component.
  • the mission application may be configured ( e.g ., via the mission application’s program code) with a forward computation.
  • the forward computation may include adding the received number to each number of the set of numbers of the first set of data.
  • the mission application may execute the forward computation and multiply each number of the set of numbers by the received number.
  • the CCN executing the mission application may store the received number and metadata in a rollback history.
  • the rollback history may be stored in the memory or attached storage of the CCN executing the mission application, another CCN, or a CSN associated with the CCN executing the mission application.
  • the mission application may generate the third set of data by adding the received number to each number of the set of numbers.
  • the mission application may replace the first set of numbers with the generated set of numbers resulting from the forward computations.
  • the mission application may receive a rollback trigger (e.g., from the user or from another application executing on a CCN). In response to the trigger, the mission application may retrieve the stored number from the rollback history.
  • the mission application may retrieve a reverse computation associated with the forward computation that was used to generate the third set of data.
  • the reverse computation may include subtracting the received number from each number of the resulting set of numbers.
  • the reverse computation may be part of the computer program instructions of the mission application.
  • the mission application may execute the reverse computation using the received number and the resulting set of numbers as input to generate the original set of numbers.
  • the mission application may replace the third set of data with the resulting original set of numbers.
  • the method for reverse-computation rollback may include generating the third set of data based on the first set of data and the forward computation without receiving a second set of data.
  • the first set of data may include a set of numbers associated with an executing application, and the forward computation may include multiplying each number of the set of numbers by 2.
  • the resulting third set of data may include each number of the first set of data multiplied by 2.
  • the reverse computation associated with the forward computation may include dividing by 2.
  • Generating the first set of data based on the reverse computation may include dividing each number of the third set of data by 2.
  • a forward computation may include a logical or software neuron’s (such as a neuron 1402 of the FIG. 14) firing based on integration operations which may be realized as event messages scheduled within the system 100.
  • a forward computation may include a dataspace insertion operation as part of a data collection or insertion task, which then the associated reverse computation may perform the correct dataspace deletion or removal operation.
  • the rollback provided by the system 100 may differ from conventional computing system functionality in a variety of ways. For example, components of an Al, machine learning, or data analytics model of the system 100 may execute without waiting for each other even though these components may be interdependent.
  • the system 100 may perform one or more unconventional steps associated with rollback.
  • unconventional functions of the system 100 may include rolling back data in the mission dataspace 120, rolling back a data model, rolling back an assertion on data in the mission dataspace 120, or rolling back a data analytics model.
  • Such functionality, along with other functionality described herein may improve the functionality of the a computer system such as the system 100. For example, by rolling back data stored in the mission dataspace 120, the system 100 can quickly and efficiently remove data from the system 100.
  • the removed data may include data that was initially ingested into the system, but should not have been based on determining that the ingested data was prematurely ingested and processed.
  • Rolling back data may enable the system 100 to automatically remove data from use by a dataspace or an Al model or system without human intervention. Rollback overcomes problems specifically arising in the realm of Al, such as the problem of Al models and systems using incorrect data as a basis for their functionality.
  • the system 100 may improve the functionality of a computing system.
  • the rollback functionality of the system 100 may enable an Al model to optimistically execute, allowing components of the Al model that are dependent on each other to execute without having to wait on each other for results. This may reduce the amount of time the Al model spends stalled. The frequency an Al model spends rolling back may be small compared to the stalling experienced by a conventional Al model.
  • This optimistic execution improves parallel processing capabilities of the system 100.
  • Rollback also enables recovery operations for an Al model or data in the mission dataspace 120 in response to computer failure due to hardware or software failures.
  • the system 100 may execute an Al or machine learning model, which may be part of the exploitation application 406 or a user-generated analytic application 502, on multiple cores.
  • the system 100 may utilize 65,000 cores or more to increase computational speed by six orders of magnitude or more over conventional computing systems and may increase accuracy by ten orders of magnitude or more. Optimistic execution of the Al or machine learning models may lead to these increases in computing speed and accuracy.
  • the system 100 may utilize the multiple cores such that the system 100 outperforms a conventional computer system that uses special-purpose hardware, such as GPUs.
  • aspects of the present disclosure may be embodied as an apparatus, system, method, computer program product, or the like. Accordingly, aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, micro-code, etc.) or an embodiment combining software and hardware aspects that may all generally be referred to herein as a“circuit,” “module,” or“system.” Furthermore, aspects of the present disclosure may take the form of a computer program product embodied in one or more computer readable medium(s) having program code embodied thereon.
  • a module may be implemented as a hardware circuit comprising custom VLSI circuits or gate arrays, off-the-shelf semiconductors such as logic chips, transistors, or other discrete components.
  • a module may also be implemented in programmable hardware devices such as field programmable gate arrays, programmable array logic, programmable logic devices or the like.
  • Modules may also be implemented in software for execution by various types of processors.
  • An identified module of program code may, for instance, comprise one or more physical or logical blocks of computer instructions which may, for instance, be organized as an object, procedure, or function. Nevertheless, the executables of an identified module need not be physically located together but may comprise disparate instructions stored in different locations which, when joined logically together, comprise the module and achieve the stated purpose for the module.
  • a module of program code may be a single instruction, or many instructions, and may even be distributed over several different code segments, among different programs, and across several memory devices.
  • operational data may be identified and illustrated herein within modules and may be embodied in any suitable form and organized within any suitable type of data structure. The operational data may be collected as a single data set or may be distributed over different locations including over different storage devices, and may exist, at least partially, merely as electronic signals on a system or network.
  • the program code may be stored and/or propagated on in one or more computer readable medium(s).
  • the computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present disclosure.
  • the computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device.
  • the computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • a non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM” or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD- ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read-only memory
  • EPROM erasable programmable read-only memory
  • SRAM static random access memory
  • CD- ROM compact disc read-only memory
  • DVD digital versatile disk
  • memory stick a floppy disk
  • a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon
  • a computer readable storage medium is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.
  • Computer readable program instructions described herein can be downloaded to respective computing/processor devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network.
  • the network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers.
  • a network adapter card or network interface in each computing/processor device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processor device.
  • Computer readable program instructions for carrying out operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the C programming language or similar programming languages.
  • the computer readable program instructions may execute on a supercomputer, a compute cluster, or the like.
  • the computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
  • electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.
  • These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
  • These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
  • the computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
  • each block in the schematic flowchart diagrams and/or schematic block diagrams may represent a module, code segment, or portion of code, which comprises one or more executable instructions of the program code for implementing the specified logical function(s).

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

L'invention concerne un procédé 800 qui peut comprendre la fourniture 802, dans un dispositif de stockage lisible par ordinateur, d'un espace de données 120. L'espace de données 120 peut comprendre une pluralité de segments logiques 122-128, et des données d'espace de données stockées dans la pluralité de segments logiques 122-128. Le procédé 800 peut comprendre la réception 804, en provenance d'une source de données 410 externe à l'espace de données 120, d'un artefact de données. L'artefact de données peut comprendre une pluralité d'éléments de données. Le procédé peut comprendre le traitement 806, au niveau d'une pluralité de filtres de données 412, de la pluralité d'éléments de données. Un filtre de données 412 de la pluralité de filtres de données 412 peut traiter un élément de données de la pluralité d'éléments de données, sur la base d'une structure de la pluralité d'éléments de données ou d'une configuration du filtre de données 412. Le procédé 800 peut comprendre la génération 808 de données traitées. Le procédé 800 peut comprendre l'envoi 810 des données traitées à l'espace de données 120.
PCT/US2019/037181 2018-06-15 2019-06-14 Systèmes et procédés pour plateforme de fusion de données d'intelligence artificielle Ceased WO2019241630A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/900,858 US20200342290A1 (en) 2018-06-15 2020-06-12 Optimistic Execution of Reservoir Computing Machine Learning Models

Applications Claiming Priority (8)

Application Number Priority Date Filing Date Title
US16/010,083 US20190385052A1 (en) 2018-06-15 2018-06-15 Methods for deep learning optimization
US16/010,083 2018-06-15
US16/147,135 US10417558B1 (en) 2018-09-28 2018-09-28 Methods and systems for artificial neural network optimistic event processing
US16/147,135 2018-09-28
US16/252,422 US20190392001A1 (en) 2018-06-15 2019-01-18 Systems and Methods for an Artificial Intelligence Data Fusion Platform
US16/252,422 2019-01-18
US201962860711P 2019-06-12 2019-06-12
US62/860,711 2019-06-12

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US16/252,422 Continuation-In-Part US20190392001A1 (en) 2018-06-15 2019-01-18 Systems and Methods for an Artificial Intelligence Data Fusion Platform

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US16/900,858 Continuation-In-Part US20200342290A1 (en) 2018-06-15 2020-06-12 Optimistic Execution of Reservoir Computing Machine Learning Models

Publications (1)

Publication Number Publication Date
WO2019241630A1 true WO2019241630A1 (fr) 2019-12-19

Family

ID=68842092

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2019/037181 Ceased WO2019241630A1 (fr) 2018-06-15 2019-06-14 Systèmes et procédés pour plateforme de fusion de données d'intelligence artificielle

Country Status (1)

Country Link
WO (1) WO2019241630A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20240231773A9 (en) * 2022-10-20 2024-07-11 Accenture Global Solutions Limited Intelligent adaptive self learning framework for data processing on cloud data fusion

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090222441A1 (en) * 2003-05-30 2009-09-03 International Business Machines Corporation System, Method and Computer Program Product for Performing Unstructured Information Management and Automatic Text Analysis, Including a Search Operator Functioning as a Weighted And (WAND)
US20110161333A1 (en) * 2005-07-05 2011-06-30 Justin Langseth System and method of making unstructured data available to structured data analysis tools
JP2012226705A (ja) * 2011-04-22 2012-11-15 Nippon Telegr & Teleph Corp <Ntt> 非構造型データの構造化処理方法
US20140324393A1 (en) * 2013-04-24 2014-10-30 International Business Machines Corporation Cross-domain data artifacts consolidation in model context
US20160267157A1 (en) * 2015-03-10 2016-09-15 The Boeing Company System and method for large scale data processing of source data

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090222441A1 (en) * 2003-05-30 2009-09-03 International Business Machines Corporation System, Method and Computer Program Product for Performing Unstructured Information Management and Automatic Text Analysis, Including a Search Operator Functioning as a Weighted And (WAND)
US20110161333A1 (en) * 2005-07-05 2011-06-30 Justin Langseth System and method of making unstructured data available to structured data analysis tools
JP2012226705A (ja) * 2011-04-22 2012-11-15 Nippon Telegr & Teleph Corp <Ntt> 非構造型データの構造化処理方法
US20140324393A1 (en) * 2013-04-24 2014-10-30 International Business Machines Corporation Cross-domain data artifacts consolidation in model context
US20160267157A1 (en) * 2015-03-10 2016-09-15 The Boeing Company System and method for large scale data processing of source data

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20240231773A9 (en) * 2022-10-20 2024-07-11 Accenture Global Solutions Limited Intelligent adaptive self learning framework for data processing on cloud data fusion
US12271713B2 (en) * 2022-10-20 2025-04-08 Accenture Global Solutions Limited Intelligent adaptive self learning framework for data processing on cloud data fusion

Similar Documents

Publication Publication Date Title
US20190392001A1 (en) Systems and Methods for an Artificial Intelligence Data Fusion Platform
US20200342290A1 (en) Optimistic Execution of Reservoir Computing Machine Learning Models
US12229642B2 (en) Efficient duplicate detection for machine learning data sets
Kolajo et al. Big data stream analysis: a systematic literature review
Muniswamaiah et al. Big data in cloud computing review and opportunities
US11544623B2 (en) Consistent filtering of machine learning data
AU2016204068B2 (en) Data acceleration
Khan et al. Big data: survey, technologies, opportunities, and challenges
EP3161635B1 (fr) Service d&#39;apprentissage machine
CN106663038B (zh) 用于机器学习的特征处理配方
Pasupuleti et al. Data lake development with big data
Dubuc et al. Mapping the big data landscape: technologies, platforms and paradigms for real-time analytics of data streams
Khan et al. Integrating Changing Data for Advanced Analytics Within Real-Time ETL and Machine Learning Frameworks: Merging ETL with Predictive Analytics
Chen et al. The evolvement of big data systems: from the perspective of an information security application
US20240045896A1 (en) Distributed Entity Re-Resolution Based on Streaming Updates
WO2019241630A1 (fr) Systèmes et procédés pour plateforme de fusion de données d&#39;intelligence artificielle
Alshaer An efficient framework for processing and analyzing unstructured text to discover delivery delay and optimization of route planning in realtime
Demirbaga et al. Big Data Analytics Platforms
Trofimov et al. Distributed Classification of Text Streams: Limitations, Challenges, and Solutions
Kiio Apache Spark based big data analytics for social network cybercrime forensics
Koya Microservice Architecture for Social Media Data Collection, Analysis, and Dashboarding
Mohammed Dynamic Data: Achieving Timely Updates in Vector Stores
US20240412095A1 (en) Feature function based computation of on-demand features of machine learning models
Khan The Convergence of Real-Time ETL and Machine Learning for Predictive Analytics on Dynamic Data
Duvvur AI-Driven Data Migration: Intelligent, Secure, and Scalable Approaches for Modernization

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19819550

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19819550

Country of ref document: EP

Kind code of ref document: A1