User Tools

Site Tools


start

Simulating Data Centers with Redfish-enabled equipment


Introduction

Analytic data, such as temperature and power usage of each data centers’ equipment, are being more and more crucial in High-Performance Computing industry, especially for petascale and future exascale systems consisting of tens of thousands or even hundreds of thousands of servers. These data are very important for accurate and real-time system monitoring and by analyzing them the researchers and data centers’ administrators will be able to find anomalies in their data centers and to anticipate failure as well.

Examining new ideas for data gathering and analyzing them through running different tests against a production data center could be harmful and also could have unintentional consequences. For this reason, a simulated data center, in order to enable researchers and also data center’s administrator to test their developed management and data gathering toolset against it before running that tools in their production environment, is still lacking.

The data center simulator is able to simulate numerous emulated Redfish-enabled equipment such as compute nodes (servers), cooling units, power units, network switches. We have used container technology, Docker Swarm specifically, to simulate a scalable data center and we have also used Redfish Mockup Servers to emulate a Redfish-enable equipment.The proposed Data Center Simulator, by using container technology, increases the efficiency, reusability, simplicity, and most importantly scalability.

Redfish Mockup Servers, on the other hand, has been adjusted to react as similar as possible to a real equipment. We have also used a different kind of Mockup Servers with different datasets and different adjusted response time to make our infrastructure heterogeneous and consequently to add more fidelity to it.


Data Sources and Standards

In a data center the physical parameter data, such as temperatures, power usage, and fan speed could be collected from numerous data sources. These sources range from manufacturer embedded sensor on the server motherboard to the self-contained sensor devices produced by different companies.

  • Linux monitoring sensors (lm-sensors): From Linux kernel version 2.5 upward, Linux kernel offers a new feature called Linux monitoring sensors as a useful and popular open source utility for sensor data acquisition. Nowadays, lm-sensors has been packaged with the majority of Linux popular distributions and could be a reliable source for hardware monitoring chips or sensors such as temperature, voltage, fan speed, humidity, and so forth.
  • Intelligent Platform Management Interface (IPMI): A specification which provides the capability of systems management in hardware layer. For accessing all the manageable features in a system, IPMI provides a common, standardized, and abstracted message-based interface. It also defines standardized records for describing platform management devices and their characteristics.
  • Redfish Scalable Platforms Management API (Redfish): an open industry management standard specification with a data model representation inside of a hypermedia RESTful interface. The data model representation includes a machine-readable schema, expressed in an OData Schema representation, with the messages being expressed in JSON format. Redfish REST-based design, help it to be easier to use and implement than many other solutions.

Emulated Redfish-enabled Equipment

In order to simulate a data center, first of all, we need to emulate Redfish-enabled data center equipment including compute nodes (servers), cooling equipment, power equipment, and network equipment. Different tools provided by DMTF have been used to create mockup datasets for data center equipment.

Redfish Mockup Creator tool

Redfish Mockup Creator is a Python 3.4 program that creates a Redfish Mockup directory structure from a real live Redfish service. The program executes Redfish GET requests to a Redfish service and saves all the responses in a directory structure as a dataset. This dataset could be considered as a snapshot of the target Redfish service. Redfish Mockup Creator has been developed by DMTF and is publicly available from official DMTF’s GitHub page at “https://github.com/DMTF/Redfish- Mockup-Server”

Redfish Mockup Server tool

Redfish Mockup Server tool is Python program that use mockup datasets to serve Redfish requests. As a server, it can serve a REST’s GET request based on the data inside a mockup and is also able to serve REST’s PATCH, POST and DELETE commands to update the corresponding data inside the mockup. Redfish Mockup Server tool has also been developed by DMTF and is publicly available from official DMTF’s GitHub page at “https://github.com/DMTF/Redfish-Mockup-Creator

Mockups based on real data center’s servers

Data Center Simulator supports at least three different kinds of Redfish-enabled servers. Redfish Mockup Creator tool has been used to take snapshots and create mockup datasets for three different kinds of servers in our supercomputer at Texas Tech University High-Performance Computing center, which is called Quanah.They have been listed in the following:

  • Server-PowerEdge-C6320
  • Server-PowerEdge-R740
  • Server-PowerEdge-T630

Mockups based on other data center’s equipment

We have also used at least 15 other different mockup datasets created for different kinds of Redfish-enabled equipment as follows (These mockups are not based on real Redfish-enabled equipment and has been created manually):

  • Power units:
    • Generator
    • PDU
    • RackPDU
    • Rectifier
    • Switchgear
    • TransferSwitch
    • Transformer
    • UPS
    • VFD
  • Cooling units:
    • AirHandlingUnit
    • CRAC
    • CRAH
    • Chiller
    • CoolingTower
  • Network switches
    • Switch-15 (A 15 ports switch)

Packaging everything in a Docker image

For emulating an equipment we need to package all the before mentioned mockups, Redfish Mockup Server, and all necessary tools (including runtime environment, libraries, configuration files and so on) that Redfish Mockup Server needs to run in a Docker image. Docker Swarm will be used to create numerous containers based on this image afterward. This also enables us to scale each equipment out to a specific number, So we will be able to simulate a real data center with a specific number of different Redfish enabled equipment.


Data Center Simulator

Docker

Docker platform has two distinct components: Docker Engine, which is responsible for creating and running containers; and Docker Hub, a cloud service for distributing containers. Further tooling has also been developed by Docker including Docker Swarm which is its native clustering manager.

Docker Swarm

Swarm is the native clustering feature of Docker. A Swarm cluster consists of one or more Docker hosts that act as Swarm Managers and a single or multiple Docker hosts that act as Swarm Workers. Docker Engine on all these hosts has to run in the Swarm mode. Swarm Managers are responsible for cluster management, orchestration, scheduling, and delegation whereas Swarm Workers run containers. To deploy an application to a Swarm cluster (in our case DMTF Redfish Mockup Server tool), a service is submitted to the Swarm Manager and then the Swarm Manager dispatches containers to the Swarm Workers. For creating a service, we also need to determine the number of replicas (the number of containers that need to be run in the service), the port that a service needs to expose to the outside world, and the network that the service’s tasks will connect to.

Docker Networking

Docker includes different kinds of network drivers to support different types of networking. Among all network drivers, we are interested in Docker MacvLan/IPvLan Network Driver that support multi-host networking in Docker Swarm.

Docker IPvLan Network Driver

IPvLan is a network driver supported by Docker that use Linux kernel IPvLan network driver to expose underlay network or host interfaces directly to containers that are running on that host. The Linux implementation of IPvLan is too lightweight because they are easily associated with a Linux ethernet interface to separate networks from connectivity to the physical network. This can finally lead to the ability of removing the bridges that traditionally resides in between the Docker host network interface and container interface and let the container interfaces to attach directly to the Docker host network interface.


Data Center Simulator - Under the hood

There are different technologies that we have put together to create Data Center Simulator. For the operating system layer, we used CentOS Linux release 7.3.1611 (Core). We have also used Docker Engine version 17.06.0-ce, which is the current version of Docker, to support containerization. For the native Docker orchestration, we used Docker Swarm mode that comes out of the box with Docker engine. We also used Docker IPvLan network driver to support networking.

Single Node Vs. Multiple Nodes

How far can we scale out a data center simulation? Although it depends on different aspects of the environment that the simulation runs on, the quick answer is that for a data center simulator with hundreds of emulated equipment, we need the single-node version and for a data center simulator with thousands of emulated equipment, we need the multi-nodes version.

  • Single Node Data Center Simulator: As shown in the following figure, in a single-node data center simulator, Docker Swarm cluster runs on only one machine that acts as both Swarm Manager and Swarm Worker.

  • Multi-node Data Center Simulator: In the multi-node node data center simulator, as shown in the following figure, multiple Docker hosts act as Swarm Worker. Swarm Manager is only responsible for orchestrating the Swarm cluster and Swarm Workers are the hosts that containers (emulated Redfish-enabled equipment) run on. One of the advantages of using multi-node data center simulator is that if there is a need to scale out the simulator up to a certain level beyond the current number of the Swarm Workers achieved, more Swarm workers can be added to increase its simulation capacity.

We have simulated a datacenter with up to 8,500 Redfish-enabled equipment by using 18 servers, as follows:

  • 1 Swarm Manager: Dell PowerEdge 1950 Server (16 GB of RAM)
  • 17 Swarm Workers: Dell PowerEdge 1950 Server (16 GB of RAM)

Each Swarm Worker with 16 GB of memory is able to run up to 500 emulated Redfish-enabled equipment. Theoretically, there is no limit for the simulation scale. If there are enough resources, even huge datacenters can be simulated.


Data Center Simulator clients

Any tool that is able to use Redfish REST API could be considered as a data center simulator client. There is a wide range of tools that could be used as the simulator client, ranging from general tools such as browsers to a very specialized command line tool for accessing the Redfish API called “redfishtool”, provided by DMTF and is publicly available from official DMTF's GitHub page at “https://github.com/DMTF/ https://github.com/DMTF/Redfishtool”.

As shown in the following figure, if the machine that runs the clients is on the same network with data center simulator, then the client will be able to access all the emulated Redfish-enabled equipment in the simulator. In other words, all needed for accessing the data center simulator is assigning an IP address to the client host’s NIC in the range of the emulated Redfish-enabled equipment’s IP addresses.

start.txt · Last modified: 2018/04/10 12:48 by ali