WO2015122911A1

WO2015122911A1 - Automatic placement of data center

Info

Publication number: WO2015122911A1
Application number: PCT/US2014/016513
Authority: WO
Inventors: Christopher M. Healey; Jonathan Mark HEALEY
Original assignee: Schneider Electric IT Corp
Current assignee: Schneider Electric IT Corp
Priority date: 2014-02-14
Filing date: 2014-02-14
Publication date: 2015-08-20
Anticipated expiration: 2016-08-14

Abstract

According to various aspects and embodiments, a system and method for optimizing a layout of a plurality of physical modules in a data center is provided. The system and method include receiving data regarding at least one property of each module of the plurality of modules, storing the received data, determining at least one module layout identifying a location for each module in the data center using a routine based on the received data, and displaying at least one module layout corresponding to results from the routine.

Description

AUTOMATIC PLACEMENT OF DATA CENTER

BACKGROUND

Technical Field

The technical field relates generally to data center design, and more specifically, to systems and methods for efficient placement of data center equipment modules using module sequences.

Background Discussion

Data center equipment modules are well suited for quickly constructing and outfitting data centers, either as completely new facilities, or to increase capacity of already existing facilities. From the perspective of a designer, data center modules need to be placed efficiently within a two-dimensional space. This may prove to be a difficult task since there may not only be an exponentially large number of possible arrangements, but an approach that systematically computes certain parameters like footprint or cabling distances may not even exist. Further, grouping modules into more manageable sets for purposes of simplifying computational complexity limits the ability of the designer to maximize the efficiency of a design.

SUMMARY

Aspects and embodiments are directed to optimizing a layout of a plurality of physical modules in a data center. These aspects and embodiments manifest an appreciation that designing the layout of a modular data center design may be a difficult problem because it involves placing large numbers of modules in a space in an efficient manner, but also must satisfy local building codes and maintenance requirements. Planners may need to consider designs that minimize total footprint, rightsize mechanical and electrical equipment, and limit piping, cabling and construction costs. For example, planners may need to consider connections between modules, such as piping or cabling distances necessary to connect IT modules and power/cooling modules. In addition, certain enclosures may have dimensions that vary along a certain width or length, and module elements may comprise one or more different shapes.

Further, modules performing similar tasks may need to be grouped together or otherwise placed in a specific arrangement. Individual modules may also need to be customized or optimized for one or more purposes. Local fire codes may also dictate the placement of equipment, such as by requiring a certain amount of space between certain modules. One or more of these

considerations may need to be incorporated into a particular module layout.

Other aspects and embodiment manifest an appreciation that one or more heuristic approaches, such as those involving genetic or random search processes, can often find approximate optimal arrangements. In fact, for many problems heuristic approaches may generally provide practical, robust results, while expending only a moderate amount of computational effort. In certain environments, such as in engineering or sales, the speed of heuristic methods often outweighs any deficiencies associated with the approximate nature of their solutions.

According to one embodiment, a computer-implemented method for optimizing a layout of a plurality of physical modules in a data center uses a computer system that includes a memory and at least one processor coupled to the memory. The method comprises receiving, by the computer system, data regarding at least one property of each module of the plurality of physical modules, storing the received data, and determining at least one module layout identifying a location for each module in the data center using a routine based on the received data. The routine executes the acts of: arranging the plurality of physical modules into a plurality of sequences, receiving data regarding at least one design objective, calculating a score indicating compliance with the at least one design objective for each sequence of the plurality of sequences, identifying at least one sequence of the plurality of sequences with a score that transgresses a threshold, and storing the at least one sequence. The method may further comprise displaying at least one module layout corresponding to the at least one sequence.

According to at least one embodiment, the plurality of physical modules is arranged into a sequence using a heuristic selection process selected from the group consisting of evolutionary, simulated annealing, neural networks, hill climbing/neighbor, Ant Colony, Particle Swarm, and tabu.

In one embodiment, arranging the plurality of physical modules further comprises converting each sequence of the plurality of sequences into a two-dimensional arrangement using an arrangement method, wherein the two-dimensional arrangement identifies the location of each module in the data center. According to certain embodiments, the arrangement method may be selected from the group consisting of bottom-left, bottom-left-fill, shelf, guillotine, and sequence pair.

In at least one example calculating the score further includes assigning a weighting factor to the at least one design objective. According to at least one embodiment, the at least one design objective includes a first design objective and a second design objective, and the method further comprises assigning a weighting factor to the first design objective and a weighting factor to the second design objective based on the stored sequences for each of the first design objective and the second design objective, determining a comparison result between the score associated with the first design objective and the score associated with the second design objective, and displaying a sequence based on the result of the comparison.

According to another embodiment, receiving the data regarding at least one property of each module includes receiving a first set of received data, and the method further comprises identifying at least one sequence of the plurality of sequences with a score that does not transgress the threshold. In at least one further example, the method further comprises receiving a second set of received data, the second set of received data including at least one of an additional property or a modified property of at least one module of the plurality of physical modules, and determining at least one module layout identifying a location for each module in the data center using the routine based on the second set of received data. In another example, the method further comprises modifying at least one sequence of the plurality of sequences if the at least one sequence has a score that does not transgress the threshold. The sequence may be modified based on at least one of an additional property or a modified property of at least one module of the plurality of modules.

According to certain embodiments, the at least one property is at least one of: module type, shape, size, connection requirements, and clearances. According to one embodiment, the at least one property is a size, and the size includes an enclosure width.

In accordance with at least one embodiment, the at least one design objective is at least one of: footprint size, piping length, cabling length, interior space, pumping machine power, pumping machine size, and maintenance access.

In accordance with some embodiments, each physical module of the plurality of physical modules is at least one of an IT module, a power module, a cooling module, a hydronics module, and a tank module. According to one example, the at least one design objective is a first design objective, and the routine further comprises receiving data regarding a second design objective, calculating a score indicating compliance with the second design objective for each sequence of the plurality of sequences, wherein the threshold associated with identifying at least one sequence of the plurality of sequences is based on a combination of the score associated with the first design objective and the score associated with the second design objective.

In accordance with one or more embodiments, a system for optimizing a layout of a plurality of physical modules in a data center comprises at least one input configured to receive data regarding at least one property of each module of the plurality of physical modules and a programmable device in communication with the at least one input. The programmable device may comprise a memory for storing the received data, and at least one processor coupled to the memory and configured to determine at least one module layout identifying a location for each module in the data center using a routine based on the received data. The routine may include executing acts of arranging the plurality of physical modules into a plurality of sequences, receiving data regarding at least one design objective, calculating a score indicating compliance with the at least one design objective for each sequence of the plurality of sequences, identifying at least one sequence of the plurality of sequences with a score that transgresses a threshold, and storing the at least one sequence. The system may further comprise at least one output in communication with the programmable device and configured to display at least one module layout corresponding to the at least one sequence.

According to one example, the at least one input and the at least one output include an interface configured to receive the data from a user and to display the at least one output to the user.

In accordance with one or more embodiments, a non-transitory computer readable medium stores instructions executable by at least one processor, wherein the instructions are coded to optimize a layout of a plurality of physical modules in a data center and to instruct the at least one processor to receive data regarding at least one property of each module of the plurality of physical modules, store the received data, and execute a routine based on the received data. The routine may include the acts of arranging the plurality of physical modules into a plurality of sequences, receiving data regarding at least one design objective, calculating a score indicating compliance with the at least one design objective for each sequence of the plurality of sequences, identifying at least one sequence of the plurality of sequences with a score that transgresses a threshold and storing the at least one sequence. The processor may further be configured to display at least one module layout associated with the at least one sequence, wherein the layout identifies a location of each module in the data center.

Still other aspects, embodiments, and advantages of these example aspects and embodiments, are discussed in detail below. Moreover, it is to be understood that both the foregoing information and the following detailed description are merely illustrative examples of various aspects and embodiments, and are intended to provide an overview or framework for understanding the nature and character of the claimed aspects and embodiments. Embodiments disclosed herein may be combined with other embodiments, and references to "an embodiment," "an example," "some embodiments," "some examples," "an alternate embodiment," "various embodiments," "one embodiment," "at least one embodiment," "this and other embodiments" or the like are not necessarily mutually exclusive and are intended to indicate that a particular feature, structure, or characteristic described may be included in at least one embodiment. The appearances of such terms herein are not necessarily all referring to the same embodiment.

BRIEF DESCRIPTION OF DRAWINGS

Various aspects of at least one embodiment are discussed below with reference to the accompanying figures, which are not intended to be drawn to scale. The figures are included to provide an illustration and a further understanding of the various aspects and embodiments, and are incorporated in and constitute a part of this specification, but are not intended as a definition of the limits of any particular embodiment. The drawings, together with the remainder of the specification, serve to explain principles and operations of the described and claimed aspects and embodiments. In the figures, each identical or nearly identical component that is illustrated in various figures is represented by a like numeral. For purposes of clarity, not every component may be labeled in every figure. In the figures:

FIG. 1 is a diagram of an example computer system;

FIG. 2 is a diagram of an example computer system architecture;

FIG. 3 is a diagram of an example graphical user interface used for inputting module properties; FIG. 4 is another diagram of an example graphical user interface used for inputting module properties;

FIG. 5 is a flow diagram of a process for determining a layout of equipment in a data center in accordance with one or more embodiments;

FIG. 6 is a diagram illustrating modules that may be structured in accordance with one or more embodiments;

FIG 7 is the flow diagram of process 500, with an additional feedback loop;

FIG 8 is a first optimized layout according to an example computer routine;

FIG 9 is a second optimized layout according to an example computer routine; and

FIG 10 is a third optimized layout according to an example computer routine.

DETAILED DESCRIPTION

By way of introduction, aspects of this disclosure relate to systems and methods through which a user may design one or more data center configurations. These methods may facilitate this design activity by allowing the user to create optimized models of data center configurations using one or more design objectives. For example, according to one embodiment, a computer system is configured to provide an interface through which the computer system receives data regarding one or more properties of a data center module and one or more design objectives of interest to an external entity, such as a user or remote computer system. In response to receipt of the data, the computer system processes the data by executing a computer routine that identifies a location for the data center module based on one or more design objectives.

The aspects disclosed herein in accordance with the present invention, are not limited in their application to the details of construction and the arrangement of components set forth in the following description or illustrated in the accompanying drawings. These aspects are capable of assuming other embodiments and of being practiced or of being carried out in various ways.

Examples of specific implementations are provided herein for illustrative purposes only and are not intended to be limiting. In particular, acts, components, elements, and features discussed in connection with any one or more embodiments are not intended to be excluded from a similar role in any other embodiments.

Also, the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. Any references to examples, embodiments, components, elements or acts of the systems and methods herein referred to in the singular may also embrace embodiments including a plurality, and any references in plural to any embodiment, component, element or act herein may also embrace embodiments including only a singularity. References in the singular or plural form are not intended to limit the presently disclosed systems or methods, their components, acts, or elements. The use herein of "including," "comprising," "having," "containing," "involving," and variations thereof is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. References to "or" may be construed as inclusive so that any terms described using "or" may indicate any of a single, more than one, and all of the described terms. In addition, in the event of inconsistent usages of terms between this document and documents incorporated herein by reference, the term usage in the incorporated reference is supplementary to that of this document; for irreconcilable

inconsistencies, the term usage in this document controls. Moreover, titles or subtitles may be used in the specification for the convenience of a reader, which shall have no influence on the scope of the present invention.

Computer System

Various aspects and functions described herein may be implemented as specialized hardware or software components executing in one or more computer systems. There are many examples of computer systems that are currently in use. These examples include, among others, network appliances, personal computers, workstations, mainframes, networked clients, servers, media servers, application servers, database servers, and web servers. Other examples of computer systems may include mobile computing devices, such as cellular phones and personal digital assistants, and network equipment, such as load balancers, routers, and switches. Further, aspects may be located on a single computer system or may be distributed among a plurality of computer systems connected to one or more communications networks.

For example, various aspects, functions, and processes may be distributed among one or more computer systems configured to provide a service to one or more client computers, or to perform an overall task as part of a distributed system. Additionally, aspects may be performed on a client-server or multi-tier system that includes components distributed among one or more server systems that perform various functions. Consequently, embodiments are not limited to executing on any particular system or group of systems. Further, aspects, functions, and processes may be implemented in software, hardware, firmware, or any combination thereof. Thus, aspects, functions, and processes may be implemented within methods, acts, systems, system elements, and components using a variety of hardware and software configurations, and the examples are not limited to any particular distributed architecture, network, or

communication protocol.

Referring to FIG. 1, there is illustrated a block diagram of a distributed computer system 100, in which various aspects and functions are practiced. As shown, the distributed computer system 100 may include one more computer systems. More specifically, the distributed computer system 100 may include computer systems 102,104, and 106. As shown, computer systems 102, 104, and 106 are interconnected by, and may exchange data through,

communication network 108. Network 108 may include any communication network through which computer systems may exchange data. To exchange data using network 108, computer systems 102, 104, and 106, and network 108 may use various methods, protocols and standards, including, among others, Fibre Channel, Token Ring, Ethernet, Wireless Ethernet, Bluetooth, IP, IPV6, TCP/IP, UDP, DTN, HTTP, FTP, SMTCP/IP, UDP, SNMP, SMS, MMS, SS7, JSON,

SOAP, CORBA, REST, and Web Services. To ensure data transfer is secure, computer systems 102, 104, and 106 may transmit data via network 108 using a variety of security measures including, for example, TLS, SSL, or VPN. While distributed computer system 100 illustrates three networked computer systems, distributed computer system 100 is not so limited and may include any number of computer systems and computing devices that are networked using any medium and communication protocol.

As illustrated in FIG. 1, computer system 102 includes processor 110, memory 112, interconnection element 114, interface 116, and data storage element 118. Processor 110 may be any type of processor, multiprocessor, or controller. Some example processors include commercially available processors such as an Intel Xeon, Itanium, Core, Celeron, or Pentium Processor, an AMD Opteron processor, or Apple A4 or A5 processors, a Sun UltraSPARC or IBM Power5+ processor and an IBM mainframe chip. Processor 110 is connected to other system components, including one or more memory devices 112, by interconnection element 114.

Memory 112 stores programs and data during operation of computer system 102. Thus, memory 112 may be a relatively high performance, volatile, random access memory such as a dynamic random access memory ("DRAM") or static memory ("SRAM"). However, memory 112 may include any device for storing data, such as a disk drive or other non- volatile storage device. Various examples may organize memory 112 into particularized and, in some cases, unique structures to perform the aspects and functions disclosed herein. These data structures may be sized and organized to store values for particular data and types of data.

Components of computer system 102 may be coupled by an interconnection element such as interconnection element 114. Interconnection element 114 may include any communication coupling between system components, such as one or more physical busses in conformance with specialized or standard computing bus technologies such as IDE, SCSI, PCI and InfiniBand. Interconnection element 114 enables communications, such as data and instructions, to be exchanged between system components of computer system 102.

Computer system 102 also includes one or more interface devices 116 such as input devices, output devices, and combination input/output devices. Interface devices may receive input or provide output. More particularly, output devices may render information for external presentation. Input devices may accept information from external sources. Examples of interface devices include keyboards, mouse devices, trackballs, microphones, touch screens, printing devices, display screens, speakers, network interface cards, etc. Interface devices allow computer system 102 to exchange information and communicate with external entities, such as users and other systems.

Storage system 118 includes a computer readable and writeable nonvolatile, or non- transitory, data storage medium in which instructions are stored that define a program or other object that is executed by processor 110. Data storage element 118 may also include information that is recorded, on or in, the medium, and that is processed by the processor 110 during execution of the program. More specifically, the information may be stored in one or more data structures specifically configured to conserve storage space or increase data exchange performance. The instructions may be persistently stored as encoded signals, and the instructions may cause the processor 110 to perform any of the functions described herein. The medium may, for example, be optical disk, magnetic disk or flash memory, among others. In operation, processor 110 or some other controller causes data to be read from the nonvolatile recording medium into another memory, such as memory 112, that allows for faster access to the information by processor 110 than does the storage medium included in data storage element 118. The memory may be located in data storage element 118 or in memory 112, however, processor 110 manipulates the data within the memory, and then copies the data to the storage medium associated with data storage elementl 18 after processing is completed. A variety of components may manage data movement between the storage medium and other memory elements and examples are not limited to particular data management components. Further, examples are not limited to a particular memory system or data storage system.

Although computer system 102 is shown by way of example as one type of computer system upon which various aspects and functions may be practiced, aspects and functions are not limited to being implemented on the computer system 102 as shown in FIG. 1. Various aspects and functions may be practiced on one or more computers having different architectures or components than that shown in FIG. 1. For instance, computer system 102 may include specially programmed, special-purpose hardware, such as for example, an application-specific integrated circuit ("ASIC") tailored to perform a particular operation disclosed herein. While another embodiment may perform the same function using a grid of several general-purpose computing devices running MAC OS System X with Motorola PowerPC processors and several specialized computing devices running proprietary hardware and operating systems.

Computer system 102 may be a computer system including an operating system that manages at least a portion of the hardware elements included in computer system 102. In some examples, a processor or controller, such as the processor 110, executes an operating system. Examples of a particular operating system that may be executed include a Windows-based operating system, such as, Windows NT, Windows 2000 (Windows ME), Windows XP, Windows Vista or Windows 7 operating systems, available from the Microsoft Corporation, a MAC OS System X operating system or an iOS operating system available from Apple

Computer, one of many Linux operating system available from Red Hat Inc., a Solaris operating system available from Sun Microsystems, or a UNIX operating system available from various sources. Many other operating systems may be used, and embodiments are not limited to any particular operating system.

Processor 110 and operating system together define a computer platform for which application programs in high-level programming languages are written. These component applications may be executable, intermediate, bytecode or interpreted code which communicates over a communication network, for example, the Internet, using a communication protocol, for example, TCP/IP. Similarly, aspects may be implemented using an object-oriented programming language, such as .Net, SmallTalk, Java, C++, Ada, C# (C-Sharp), Python, or JavaScript. Other object-oriented programming languages may also be used. Alternatively, functional, scripting, or logical programming languages may be used.

Additionally, various aspects and functions may be implemented in a non-programmed environment. For example, documents created in HTML, XML or other formats, when viewed in a window of a browser program, can render aspects of a graphical -user interface or perform other functions. Further, various embodiments may be implemented as programmed or non- programmed elements, or any combination thereof. For example, a web page may be implemented using HTML while a data object called from within the web page may be written in C++. Thus, the examples are not limited to a specific programming language and any suitable programming language could be used. Accordingly, the functional components disclosed herein may include a wide variety of elements (e.g., specialized hardware, executable code, data structures or objects) that are configured to perform the functions described herein.

In some examples, the components disclosed herein may read parameters that affect the functions performed by the components. These parameters may be physically stored in any form of suitable memory including volatile memory (such as RAM) or nonvolatile memory (such as a magnetic hard drive). In addition, the parameters may be logically stored in a proprietary data structure (such as a database or file defined by a user mode application) or in a commonly shared data structure (such as an application registry that is defined by an operating system). In addition, some examples provide for both system and user interfaces that allow external entities to modify the parameters and thereby configured the behavior of the components.

Example System Architecture

FIG. 2 presents a context diagram including physical and logical elements of distributed system 200. The system structure and content recited with regard to FIG. 2 is for example purposes only and the methods and process disclosed herein are not intended to be limited to the specific structure shown in FIG. 2. As will be apparent to one of ordinary skill in the art, many variant system structures can be architected without deviating from the scope of the methods and processes disclosed herein. The particular arrangement presented in FIG. 2 was chosen to promote clarity. Information may flow between the elements, components and subsystems depicted in FIG. 2 using any technique. Such techniques include, for example, passing the information over the network via TCP/IP, passing the information between modules in memory, and passing the information by writing to a file, database, or some other non-volatile storage device. Other techniques and protocols may be used without departing from the scope of the methods and systems disclosed herein.

Referring to FIG. 2, system 200 includes user 202, interface 204, data center design system 206, communications network 208, and data center database 210. System 200 may allow user 202, such as a data center architect or other data center personnel, to interact with interface 204 to create or modify a model of one or more data center configurations. According to some embodiments, interface 204 may be implemented with specialized facilities that enable user 202 to design, in a drag and drop fashion, a listing or model that includes a representation of each equipment module or groupings of equipment modules that are to be included in the physical layout of a data center. The listing or model may include representations of data center structural components as well as data center equipment. The features of interface 204, as may be found in various embodiments in accordance with the methods and systems disclosed herein, are discussed further below. In at least one embodiment, information regarding a data center is entered into system 200 through the interface, and assessments and recommendations for the data center, including a proposed layout, are provided to the user. Further, in at least one

embodiment, processes or routines may be performed to propose one or more optimized layouts of the data center. According to certain aspects, a graphical user interface may be provided for purposes of exchanging information between the user 202 and one or more other parts of the system, such as interface 204.

As shown in FIG. 2, data center design system 206 presents data design interface 204 to user 202. According to one embodiment, design interface 204 may incorporate input data (not shown).

As illustrated, data center design system 206 may exchange information with data center database 210 via network 208. This information may include any information required to support the features and functions of data center design system 206. For example, in one embodiment, this information may include the type and number of equipment modules as well as one or more properties related to the equipment modules. Data center database 210 may take the form of any logical construction capable of storing information on a computer readable medium including, among other structures, flat files, indexed files, hierarchical databases, relational databases or object oriented databases. The data may be modeled using unique and foreign key relationships and indexes. The unique and foreign key relationships and indexes may be established between the various fields and tables to ensure both data integrity and data interchange performance.

The computer systems shown in FIG. 2, which include data center design system 206, network 208, and data center equipment database 210, may each include one or more real or virtual computer systems. As discussed above with regard to FIG. 1, computer systems may have one or more processors or controllers, memory and interface devices. The particular configuration of system 200 depicted in FIG. 2 is used for illustration purposes only and other contexts are within the scope of this disclosure. Thus, embodiments of the methods and systems disclosed herein are not limited to a specific number of users or systems. Data Center Assessment and Design Objectives

Data centers are facilities that contain servers and other equipment. Data centers may be used to perform various functions, such as hosting web servers, hosting databases, providing back-end support for finance or e-commerce, or any other function that can be performed by computer. Data centers contain large numbers of servers and other IT equipment, as well as mechanical support for that equipment. Mechanical support may include cooling equipment, fire suppression equipment, utility power, backup power, data connectivity, and any other infrastructure that may be used by the IT equipment.

Data centers are often modular, in that the center may be built out of physical modules that contain the equipment to perform various functions. Modularization makes a data center scalable, since the capacity of the data center can be increased or reduced by adding or removing the physical modules. Additionally, modularization allows the components for the data center to be assembled in one place and deployed in another place, since the physical modules may be relatively easy to move.

One way to modularize a data center is to build various function modules, such as an IT module that houses servers and other IT equipment, a cooling module that houses refrigeration equipment, a fire suppression module that houses fire suppression equipment, a power module that houses distribution panels, Uninterruptable Power Supplies (UPSs), transfer switches, backup generators, etc. These modules may further be connected to each other. For example, an IT module that contains servers, routers, disk drives, etc., may be connected to a cooling module that contains refrigeration equipment to produce chilled water. The IT module may then be connected to the cooling module, so that the cooling module keeps the equipment in the IT module below a certain temperature. As used herein, the term "physical module" is intended to mean a data center module that performs one or more function. Non-limiting examples of types of physical modules may include an IT module, a power module, a cooling module, a hydronics module, a tank module, or any combination thereof. For instance, a tank and hydronics functionality may be combined into a single module, or power and IT functionality may be combined into a single module. Other types of physical modules are also within the scope of this disclosure. For example, one or more modules may include functionality related to security, network/communication, fire protection, and any combination thereof.

One issue that arises when modules implement separate functions is that an ΓΓ module often has to include equipment to support its ability to receive another function from one of the functional modules. For example, a cooling module might house the refrigeration equipment that produces chilled water. However, in order for an IT module to be cooled by that

refrigeration module, it may have to have piping to receive and circulate the chilled water, internal cooling units or computer room air conditioners (CRACs) to convert the chilled water into cold air, and fans to distribute the cold air to the servers. Thus, even when the cooling function has been segregated in a separate module, an IT module often has a significant amount of cooling-related hardware in order to use the cooling function provided by another module. Therefore, the connectivity between individual modules is also a factor considered by at least one embodiment disclosed herein, as explained further below.

Various embodiments utilize one or more specially configured computer systems to execute a routine and provide one or more data center layouts. According to some aspects, a user provides input data for a data center module. In some embodiments, the input data is manually entered into the system, however, in other embodiments, the data may be provided electronically from another system. The input data may include one or more properties related to a module or a group of modules, such as the type of module, the shape, the size, the connection requirements, and clearance requirements. For example, the input data may include the type of module, such as whether the module is an IT, power, cooling, hydronics, or tank module. The input data may further include one or more properties related to the module, such as the shape and size, and connection and clearance requirements. For example, according to one or more embodiments, FIG. 3 illustrates an interface that may be used to create and edit a module, such as the chiller or cooling module. Each data center module may be defined by its type, shape, and size. In some embodiments, one or more properties may be grouped together. For example, shape and size may be grouped together to reflect the exterior dimensions of a module. Further, one or more dimensions related to clearances around the object may also be included. For example, local fire codes may require that a certain distance between the module and the ceiling of a room be maintained, or that a certain distance be maintained around the module itself.

Maintenance requirements may also limit the proximity of modules or orientation for placing a module in a certain position. In a further aspect, if two modules are placed immediately adjacent to each other, then some or all of the distance between the two modules may meet the clearance requirements for both of the modules. Therefore, the modules are capable of "overlapping." As depicted in FIG. 3, the chiller may have a size including a total width and height, and clearance requirements in four different directions. One or more module properties may be related or dependent upon one another. For example, if the overall shape of the module is a rectangle, then the size dimensions are those associated with right-angled objects, such as a height, width, and depth. Although some modules may be rectangular in shape, it is within the scope of the systems and methods disclosed herein to also consider modules that are of other shapes, including irregular shapes.

In some embodiments, the input data may further include connectivity information. The connectivity information may include one or more types of utilities that are required for a module as an input or an output, such as power, water, and network capacity. Further, one or more of these connection types may be dependent upon another module type. For example, FIG. 4 illustrates an interface that may be used to specify one or more types of connections that are required in relation to a hydronics module. As shown, a hydronics module may require cold and hot water connections, as well as an electric connection. Further, these connections may be linked to or provided by another module, such as electric power that may be sourced from the power module, cold water that may be sourced from the tank module, and hot water that may be sourced from the chiller or cooler module. In another aspect, users may also specify where connections may be located on the boundaries of each module. For example, a user may wish to place electrical connections above or behind a module to reduce tripping hazards. Although FIGS. 3 and 4 depict specifications made using a graphical user interface, it is also within the scope of this disclosure to include other types of interfaces, such as an adjacency list structure.

According to certain embodiments, once data related to the properties of the data modules is entered, a user may identify one or more design objectives they wish to use to form the basis of the design layout. For example, a user may choose one or more design objectives that in turn may dictate the physical locations of one or more modules in an overall data center layout. In accordance with some embodiments, design objectives may have a minimizing effect, such as minimizing overall footprint, cable and piping distances, and/or pumping machines power and size. Many of these, including cabling and piping distances, may be related to corresponding installation, maintenance, or operating costs. Not all design objectives have to have a

minimizing aspect. For example, other types of objectives may include parameters such as providing maintenance access, providing sufficient power or cooling redundancy, maximizing the IT load capacity for a given design area, maximizing the efficiency of a power or cooling process, or using all possible space within the interior of a module. One or more other related design objectives may be associated with pumping machine power and/or pumping machine size. A user may desire to minimize these design objectives or balance these with other design objectives, such as in situations where it is desired to keep capital costs at a minimum. As discussed above with respect to the module properties, a graphical user interface or other type of interface may be used to receive the information related to one or more design objectives.

In a further aspect, each separate design objective may require its own set of parameters or calculations. For example, footprint may be determined by estimating the maximum boundary points in each cardinal direction. Pumping machine power and size may be calculated as a function of piping distances and the volume of fluid per unit of time requires (such as gallons/minute). In certain instances, a user may be prompted to select additional methods or parameters related to the design objective. For instance, cabling and piping distances may be ascertained by calculating the underground distances between connection points. In the alternative, the aboveground distances may be calculated by using a graph theory shortest path process, such as Dijkstra's shortest path algorithm. Other criteria may require other types of information. For example, maintenance access may include considerations as to how much open space lies between an individual module and the exterior of the entire arrangement or layout.

In certain instances, one or more of the design objectives discussed above may be opposing, meaning that one objective may come at the expense of another. In a further aspect of the methods and systems disclosed herein, a user may be able to specify a hierarchy or preference for one or more design objectives in the layout.

In a related aspect, as discussed further below, multiple design objectives may be chosen by the user, with each design objective given a relative weight based on importance to the user. For example, if both footprint and cabling are of equal importance, then they each may be given a value that represents their equal importance.

In another related aspect, as discussed further below, multiple design objectives may ultimately be chosen by the user by performing a routine that in turn is constructed to calculate through multiple iterations. For example, a first pass may include a first design objective or set of design objectives, and a second pass may include a second design objective or set of design objectives. This situation may occur when the first pass yields multiple layout results, and subsequently, the system may prompt the user to provide further refinement by selecting additional criteria.

Computer Routine

A computer routine may be performed using one or more of the inputs discussed above, such as module properties related to shape and size, and design objectives related to footprint or cabling and/or piping distances. The computer routine uses this information to perform an automatic placement process that places the modules into one or more layouts. In certain instances, the computer routine may also be an optimization routine. When used in reference to the computer routine, the term "optimize" refers to enhancing or improving one or more design objectives, and is not necessarily limited to creating a minimum or maximum value. Although in some instances, comparisons to minimum and maximum values may be used to enhance or improve one or more design objectives.

FIG. 5 shows a flow chart for a routine for automatically placing one or more pieces of data center equipment. In a first stage 502 of the method 500, one or more properties related to the module are imported into the system. For example, properties regarding type, shape, and size of one or more modules may be received by a computer system. This information may be defined by a user or may be provided by another computer system. The information may be provided to the system using interfaces such as those depicted in FIGS. 3 and 4.

Once input data has been received, then at stage 504 modules may be arranged into one or more sequences. A sequence may feature a permutation of module identifiers, such as 15432 for 5 distinct modules. Further, the module identifier may be combined with an orientation for each module. Sequences are easy to generate, store, and manipulate. However, enumerating every single permutation of a sequence may be computationally expensive. Therefore, a selection process may be used to choose and generate a subset or series of sequences to inspect. In some embodiments, the selection process may be random. This type of method includes choosing a random ordering and orientation of all modules. In other embodiments the selection process may be a variation of a random selection. For example, the selection process may utilize a heuristic mathematical approach. As used herein, the term "selection process" refers to both heuristic and random processes. The term "heuristic" refers broadly to selection processes that provide a result that can be compared to a theoretical best result. Non-limiting examples of heuristic problem-solving strategies include: evolutionary processes, such as genetic processes, simulated annealing, neural networks, hill climbing/neighbor, Ant Colony optimization, Particle Swarm optimization, and tabu search. The effectiveness of the selection process may be dependent on the specifics of the problem.

In a third stage 506 the selected module sequences may be converted, or decoded, into two-dimensional arrangements. These arrangements may then be scored based on one or more design objectives, discussed above. According to some aspects, the module sequences may be converted using one or more standard packing placement techniques. Non-limiting examples of arrangement methods include bottom-left, bottom-left-fill, shelf, guillotine, and sequence pair. The bottom-left method places objects as far down as possible and then left justifies the object. Bottom-left-fill is similar to bottom-left, but further includes a check to see if there are any open holes to place objects. Shelf and guillotine processes may yield results more quickly, but may also create spaces between one or more objects. The sequence pair method uses two sequences, one to list objects' locations from right-to-left and one to list locations from front-to-back.

One or more of these selection processes may be extended to handle non-rectangular shapes or overlapping modules. Each process may offer its own set of tradeoffs with respect to efficiency and computational complexity. For example, rectangular objects need to consider four points of intersection with a pre-existing set of already-placed objects in order to establish the bottom-left placement. Different or more complicated shapes may have to consider either more or less (e.g., a triangle) points of intersection with the already-placed objects.

A placement process in accordance with one or more embodiments employs an improved form of a conventional binary tree to represent a compacted placement in which no module can move any further down or to the left. For example, FIG. 6 illustrates four modules listed as a first sequence (a) 1234 and a second sequence (b) 3241. The first and second sequences are then decoded into two arrangements, as shown, using a bottom-left approach and a fixed enclosure width. This approach to arranging the modules may be efficient, since the process only has to check a finite number of locations to place each module. When modules are incapable of overlapping, the list of locations to place modules is easy to update and maintain. However, when modules are capable of overlapping, as discussed above in reference to dimensions related to clearances, the list of locations must be updated for every module about to be placed if some clearances are larger than others.

Once the sequence has been decoded and reflects a two-dimensional arrangement, the sequences may be compared to one another by assigning a score. Scoring may be based on one or more design objectives selected by the user. For example, a user may be interested in optimizing one or more specific properties of the arrangement, such as footprint and/or cabling/piping. According to some embodiments, a multi- objective approach may be used that allows the sequence's score to be a function of each design objective (or objectives) considered, as well as their relative weight (if included). An overall objective score S, may be calculated out of n objectives as a linear combination of the individual objective scores, as described by Equation 1:

where the coefficient describes the weighting factor, or relative importance, of each individual objective, i.

In at least one example, a single sequence that optimizes all design objectives at the same time does not exist. Therefore a Pareto, or non-dominated solution, may be compiled as a collection of sequences that could be optimal for some, but not all, of the sets of criteria. For example, two criteria, such as footprint and piping, may result in three separate sequences: a first that minimizes footprint, a second that minimizes piping, and a third that is the result of giving both criteria equal importance. This scenario is discussed further below in the example.

After decoding and scoring the sequences, the best incumbent design or designs are recorded in stage 508 for each design objective(s). At stage 510, a determination is made as to whether additional sequences need to be tested. If the outcome of determination block 510 is YES, then the routine returns to stage 504 to select additional sequences. For example, if no designs meet the desired design objective, then additional sequences have to be tested. Further, a threshold may be established for determining whether additional sequences should be tested. For example, a target score may be established as a threshold by a user or computer. If the score associated with a certain sequence transgresses the threshold, then the sequence may be stored or otherwise saved, and if the score does not transgress the threshold, then the sequence may be rejected or discarded. According to some examples, the score may transgress the threshold when it exceeds the target value. In other examples, the score may transgress the threshold when it falls below the target value. According to some examples, a certain number of stored sequences may be established as a threshold. If the number of stored sequences transgresses the threshold, then additional sequences do not have to be tested.

If the outcome of determination block 510 is NO, then the routine proceeds to stage 512, where a determination is made as to whether additional inputs need to be tested. If the outcome of determination block 512 is YES, then the process proceeds to stage 516, where additional inputs are entered into the system, such as adjusting one or more modules' enclosure width (discussed below) or connectivity requirements. In a similar approach as discussed above with respect to determination block 510, a threshold may be established for determining whether additional inputs should be tested. For example, a user may desire a design that falls within a certain value (threshold) for footprint and there may be flexibility or different options for one or more modules' enclosure width and/or connectivity requirements. If the routine fails to produce enough layouts that fall within the desired threshold associated with the footprint, then these input values may be adjusted at stage 516. In a second example, a user may wish to compare layouts using several different enclosure widths for one or more modules. Stage 516 allows the user the ability to adjust these values and compare the results. According to some embodiments, one or more properties associated with stage 502 may include a module's enclosure width. Enclosure width is a property that is related to, or considered a component of a module's size. In certain instances, a module may have a varying enclosure width. Due to the complexity, a simple packing approach may be incapable of arranging the modules into a space where multiple boundaries are open. Bin-packing processes may be designed to only place modules within spaces of fixed dimensions. In at least one embodiment, the processes disclosed herein are capable of performing calculations over a range of enclosure widths. For example, enclosure widths of 18, 19, 20, and on up to 25 meters may be included by using stage 516 to feed a different enclosure width back into the process.

If the outcome of determination block 512 is NO, then the process proceeds to stage 514, where the best designs are displayed or otherwise output to a user or system, and the process ends at stage 518. In certain embodiments, stage 514 may be available at other stages in the routine. For example, as discussed above, a user may desire a layout to have a footprint that falls below a certain value for square footage. Therefore, design results may be displayed at stage 512 to assist the user or computer in determining whether module inputs need to be adjusted.

Multi-Scale Optimization

In a further embodiment, the systems and methods disclosed herein may be applied to the content of the module itself. For example, instead of arranging IT, power, or cooling modules within an open space, a user may arrange one or more servers, PDUs, and UPSs within an IT module. In another aspect, similar modules may be grouped into optimized "sets."

According to some aspects, the design optimization for each module or grouping may be performed independently of a high-level arrangement solution. According to other aspects, each module or grouping may be fed into a larger routine. For example, FIG. 7 shows a flow chart 700 of a computer routine similar to that of FIG. 5, with the addition of an enhanced feedback loop. The enhanced feedback loop includes stage 715, which allows the method to utilize new module designs or groupings as additional inputs. For example, a user may wish to group several IT modules together, so that the process produces a layout that places these modules near each other. This can be accomplished by treating the process of generating a single grouping (or subset of groupings) of modules from multiple groupings as its own subroutine, or miniature routine. Groupings to be tested are created by their own set of sequences and then used by the larger routine as an input.

According to an additional aspect, this treatment may also be extended to consider a subset of module properties. For example, one or more enclosure widths may be sampled using the feedback loop featured in FIG. 7 by stages 715 and 716. For example, optimal enclosure widths may not be known in advance of executing the decoding and selection process featured in stage 706. Since only a subset of enclosure widths may be close to optimal, the featured feedback loop can be used to generate a subset of enclosure widths that are close to optimal. These may then be fed back into the larger routine as input values. In some examples, only a few sequences need to be tested to effectively "screen out" inadequate values for enclosure width.

Processes 500 and 700 each depict one particular sequence of acts in a particular embodiment. The acts included in these processes may be performed by, or using, one or more computer systems specially configured as discussed herein. Some acts are optional and, as such, may be omitted in accordance with one or more embodiments. Additionally, the order of acts can be altered, or other acts can be added, without departing from the scope of the embodiments described herein. Furthermore, the acts may be performed on particular, specially configured machines.

The methods and systems disclosed here are also not limited to the arrangement of data center modules within a design space. For example, a similar approach may be used for arranging servers, racks, and/or other pieces of support equipment within an existing structure or module. One or more of the disclosed methods may also be used for placing individual servers within a rack, or may be more broadly applied to creating large-scale data center locations where many different types of computer systems and their associated components are housed within a central location. The functional elements associated with such a design may include

telecommunication, storage, redundant or backup power supplies, redundant data communication connections, environment controls, and security devices. One or more aspects of the methods described here may be used to design an efficient arrangement of these different resources. Examples

According to one example, eight modules were selected for automatic placement into a data center layout using a computer routine according to the systems described herein. The eight modules included two IT modules, one power module, one hydronics module, three cooling or chilling modules, and one water tank. Each module was assumed to be rectangular in shape, with the dimensions and count for each module provided below in Table 1.

Table 1 : List of Example Input Data for Modules

For purposes of this example, the IT and hydronics modules were specified to be connected to the power module using underground cabling, while aboveground pipes were selected to connect the IT and hydronics modules to the water tank, as well as the chillers to the hydronics module.

Three separate design objectives were selected for purposes of creating optimized layouts: minimize footprint, minimize cabling/piping (connection distance), and minimize an approximately equal-weighted combination of footprint and cabling/piping.

For each design objective, a total of 25,000 sequences were tested, with enclosure widths varying from 18 meters to 48 meters. The weighting factors used in the analysis are shown below in Table 2. For example, when minimizing footprint, a token weighting factor (with a value of 0.0001) was included in the calculations for the cabling/piping distance. Further, in some instances multiple layouts may exist that have the same value for footprint. Therefore, according to at least one further embodiment, the method may then further choose the layout with the smallest cabling/piping distance. Using similar logic, when minimizing cabling/piping distance, a token weighting factor may be added to the footprint, as shown below. Further, in instances where multiple layouts yield the same connection distance the method will choose the layout with the smallest footprint. The weighting factors used for minimizing both footprint and connection distance were based on calculations that determined a ratio of footprint to connection distance. For example, when optimizing footprint, this ratio yielded a value of about 7, and when optimizing connection distance, this ratio yielded a value of about 10. Therefore, for this specific example, each additional unit (meter) of connective material used to create the connection distance was determined to be about 7-10 times more expensive than each additional unit (m ) of footprint. The weighting factor value of 8 for the connection distance is therefore an approximation, since it falls in between the values of 7 to 10.

Table 2: Design Objective Weighting Factors

The process found three optimized layouts (one for each objective), with the results of each layout shown in FIGS. 8-10, respectively. Quantitative values for the design objectives are further shown in Table 3 below. In the figures, each module is represented by an element number. For example, in FIG. 8, IT modules are represented by element 818, the power module is represented by element 820, chiller modules are represented by element 822, the hydronics module is represented by element 824, and the tank is represented by element 826. The connection distances are represented by dashed or solid lines in between the respective modules. Each arrangement exhibited certain advantages and disadvantages. For example, FIG. 8 displays an arrangement that achieves a minimal footprint of 594 m , but requires 85.7 m of

cabling/piping. FIG. 9 features 67.5 m of cabling/piping in a footprint of 696 m , which is 16% more than that of FIG. 8. Further, FIG. 10 displays the compromise situation for the two- dimensioned design objective, where only 5% more footprint and 10% more cabling/piping were required over each respective layout of FIGS. 8 and 9. Table 3: Quantified Results for Design Objectives

The methods and systems disclosed herein create an approach that allows automatic generation of many design alternatives, while also possessing the capability to systematically compare alternatives with respect to one or more design objectives such as footprint and cost.

Having thus described several aspects of at least one example, it is to be appreciated that various alterations, modifications, and improvements will readily occur to those skilled in the art. For instance, examples disclosed herein may also be used in other contexts. Such alterations, modifications, and improvements are intended to be part of this disclosure, and are intended to be within the scope of the examples discussed herein. Accordingly, the foregoing description and drawings are by way of example only.

What is claimed is:

Claims

1. A computer-implemented method for optimizing a layout of a plurality of physical modules in a data center using a computer system including a memory and at least one processor coupled to the memory, the method comprising:

receiving, by the computer system, data regarding at least one property of each module of the plurality of physical modules;

storing the received data;

determining at least one module layout identifying a location for each module in the data center using a routine based on the received data, the routine executing acts of:

arranging the plurality of physical modules into a plurality of sequences;

receiving data regarding at least one design objective;

calculating a score indicating compliance with the at least one design objective for each sequence of the plurality of sequences;

identifying at least one sequence of the plurality of sequences with a score that transgresses a threshold;

storing the at least one sequence; and

displaying at least one module layout corresponding to the at least one sequence.

2. The computer- implemented method of claim 1, wherein the plurality of physical modules is arranged into a sequence using a heuristic selection process selected from the group consisting of evolutionary, simulated annealing, neural networks, hill climbing/neighbor, Ant Colony, Particle Swarm, and tabu.

3. The computer- implemented method of claim 1, wherein arranging the plurality of physical modules further comprises converting each sequence of the plurality of sequences into a two-dimensional arrangement using an arrangement method, wherein the two-dimensional arrangement identifies the location of each module in the data center.

4. The computer-implemented method of claim 3, wherein the arrangement method is selected from the group consisting of bottom-left, bottom-left-fill, shelf, guillotine, and sequence pair.

5. The computer- implemented method of claim 1, wherein calculating the score further includes assigning a weighting factor to the at least one design objective.

6. The computer- implemented method of claim 1, wherein the at least one design objective includes a first design objective and a second design objective, and the method further comprises:

assigning a weighting factor to the first design objective and a weighting factor to the second design objective based on the stored sequences for each of the first design objective and the second design objective;

determining a comparison result between the score associated with the first design objective and the score associated with the second design objective; and

displaying a sequence based on the result of the comparison.

7. The computer- implemented method of claim 1, wherein receiving the data regarding at least one property of each module includes receiving a first set of received data, and the method further comprises:

identifying at least one sequence of the plurality of sequences with a score that does not transgress the threshold;

receiving a second set of received data, the second set of received data including at least one of an additional property or a modified property of at least one module of the plurality of physical modules; and

determining at least one module layout identifying a location for each module in the data center using the routine based on the second set of received data.

8. The computer- implemented method of claim 7, wherein the at least one property is a size, and the size includes an enclosure width.

9. The computer- implemented method of claim 1, wherein the at least one property is at least one of: module type, shape, size, connection requirements, and clearances.

10. The computer-implemented method of claim 1, wherein the at least one design objective is at least one of: footprint size, piping length, cabling length, interior space, pumping machine power, pumping machine size, and maintenance access.

11. The computer- implemented method of claim 1, wherein each physical module of the plurality of physical modules is at least one of an IT module, a power module, a cooling module, a hydronics module, and a tank module.

12. A system for optimizing a layout of a plurality of physical modules in a data center, the system comprising:

at least one input configured to receive data regarding at least one property of each module of the plurality of physical modules;

a programmable device in communication with the at least one input, the programmable device comprising:

a memory for storing the received data;

at least one processor coupled to the memory and configured to:

determine at least one module layout identifying a location for each module in the data center using a routine based on the received data, the routine executing acts of:

arranging the plurality of physical modules into a plurality of sequences;

receiving data regarding at least one design objective;

identifying at least one sequence of the plurality of sequences with a score that transgresses a threshold; and

storing the at least one sequence; and at least one output in communication with the programmable device and configured to display at least one module layout corresponding to the at least one sequence.

13. The system of claim 12, wherein the at least one input and the at least one output include an interface configured to receive the data from a user and to display the at least one output to the user.

14. The system of claim 12, wherein the plurality of physical modules is arranged into a sequence using a heuristic selection process selected from the group consisting of evolutionary, simulated annealing, neural networks, hill climbing/neighbor, Ant Colony, Particle Swarm, and tabu.

15. The system of claim 12, wherein arranging the plurality of physical modules further comprises converting each sequence of the plurality of sequences into a two-dimensional arrangement using an arrangement method, wherein the two-dimensional arrangement identifies the location of each module in the data center.

16. The system of claim 15, wherein the arrangement method is selected from the group consisting of bottom-left, bottom- left- fill, shelf, guillotine, and sequence pair.

17. The system of claim 12, wherein calculating the score further includes assigning a weighting factor to the at least one design objective.

18. The system of claim 12, wherein the at least one design objective includes a first design objective and a second design objective, and the routine further comprises:

displaying a sequence based on the result of the comparison.

19. The system of claim 12, wherein receiving the data regarding at least one property of each module includes receiving a first set of received data, and the routine further comprises: identifying at least one sequence of the plurality of sequences with a score that does not transgress the threshold;

20. A non-transitory computer readable medium storing instructions executable by at least one processor, the instructions being coded to optimize a layout of a plurality of physical modules in a data center and to instruct the at least one processor to:

receive data regarding at least one property of each module of the plurality of physical modules;

store the received data;

execute a routine based on the received data, the routine including the acts of:

arranging the plurality of physical modules into a plurality of sequences;

receiving data regarding at least one design objective;

storing the at least one sequence; and

display at least one module layout associated with the at least one sequence, wherein the layout identifies a location of each module in the data center.