About OMF

Motivation - The need for Experimental Facilities (aka Testbeds)

While the Internet and more generally network technologies improve our lives in many ways, their increasing presence also highlights their various limits. These limits drive the researchers and industries to design and develop new networking technologies. These technologies need to be evaluated and refined before being included in full fledged products.

Simulations allow some rapid first approximations of behaviours, performance and scalability of a new technology. As such, they are valuable (and often inexpensive) initial steps in evaluating a new technology. However, due to design and computational complexities, simulations use simplified models, which fail to capture the dynamics and intricacy of real environments. Emulation is an alternative, which combines the use of implemented prototypes and substitute entities reproducing real-world behaviours. However, these substitutes again often fail to capture the complexity of real environments. As a result, only experimentations using prototype implementations in real and controlled environments (e.g. actual users, wireless radio devices, vehicle movements, building layouts) will provide results close to reality. Such experimental platforms or facilities are referred to as testbeds.

Thus, networking testbeds are a key component in the development of new networking technologies, and this insight led to many large-scale global initiatives aiming at designing, providing and operating such testbeds. The complexity and often high-cost of deploying and operating testbeds make them inaccessible to many experimenters. Moreover, many testbeds are built as part of a specific project and rarely stay operational past the end of that project. This waste of resources has often been quoted as one of the key motivators for dedicated testbed funding and associated systems research. Examples of early successes for this approach are PlanetLab and Orbit. However, both of them are relatively limited in terms of networking experiment capabilities and available technology. The federation of existing and upcoming testbeds under a unified control and management framework would overcome some of these limits, allowing large scale simultaneous resources access and sharing while maintaining administrative control in the hands of the resource owner.

Compared to other scientific research fields, such as physic sciences, the networking field so far has not developed a culture of rigorous peer verification of experimental results. This is mostly due to the fact that even if the same or a similar experimental infrastructure is available (i.e. through a federation of world-wide testbeds), there is currently no unambiguous way to describe and instrument an experiment enabling others to repeat it. Indeed, many high quality publications provide insufficient descriptions of the performed experiments to allow others a fair and valid comparison of different approaches to specific problem in different settings. Thus, to increase the scientific rigour in networking, there is a need for efficient tools and methodologies that provide complete descriptions of experiments, including resources used, and measurements taken.

OMF - A Control and Management Framework for Networking Testbeds

For the last number of years a number of research institutions aroud the world have been developing OMF for networking testbeds, and its companion Measurement Library (OML).

From the experimenter's point of view, OMF provides a set of tools to describe and instrument an experiment, execute it and collect its results. From the testbed operator's point of view, OMF provides a set of services to efficiently manage and operate the testbed resources (e.g. resetting nodes, retrieving their status information, installing new OS image). The following figure present a general overview of OMF from the user's point of view. The user/experimenter describes her experiment in a high-level domain-specific language, and passes it on to OMF. The framework will in turn deploy and configure the experiment on the testbed(s) according to the user's description. Then it will initiate and control the execution of this experiment. Finally, during the experiment execution, the framework will measure and collect data according to the user's description. These measurements are sent to a repository available to the user and can also be used to dynamically steer the experiment control

It is important to node that OMF tools and software are not tied to a specific testbed technology. Indeed, OMF has been deployed and maintained on multiple testbeds with many different types of technologies, at a number of deployments around the world. It has been used regularly by many researchers and students for the past 15 years with some testbeds operating 24/7 catering to experimenters around the world.

The following PDF file contains a short presentation of OMF: OMF-Short-Intro-Web.pdf

Deployment Sites

OMF is currently deployed and used by researchers at the following sites:

• Winlab, Rutgers University, US - Several indoor testbeds (one with 400 nodes!) and 1 outdoor testbed (including nodes in cars)
• NICTA, Sydney, Australia - indoor testbed with 38 nodes (testbed not publicly accessible)
• Witest Lab, Polytechnic institute of NYU, US - 1 indoor testbed (15 nodes)
• NIT Lab, University of Thessaly, Greece - 1 outdoor testbed (50 nodes)
• Thomson Lab, Paris, France (testbed not publicly accessible)
• INRIA, Sophia Antipolis, France (testbed not publicly accessible)
• Heterogeneous Networking Group, Instituto de Telecomunicaes - Aveiro, Portugal (testbed not publicly accessible)
• Consorzio Interuniversitario Nazionale per la Informatica (CINI), Italy (testbed not publicly accessible)
• Networked Media Lab, Gwangju Institute of Science and Technology (GIST), Korea (testbed not publicly accessible)
• Computer Science Department, University of California Los Angeles (UCLA), US (testbed not publicly accessible)
• Shanghai Research Center for Wireless Communications (WiCO), China (testbed not publicly accessible)

Publications

Overview of the ORBIT Radio Grid Testbed for Evaluation of Next-Generation Wireless Network Protocols, D. Raychaudhuri, I. Seskar, M. Ott, S. Ganu, K. Ramachandran, H. Kremo, R. Siracusa, H. Liu and M. Singh, Proceedings of the IEEE Wireless Communications and Networking Conference (WCNC 2005), New Orleans, USA, 2005

ORBIT Testbed Software Architecture: Supporting Experiments as a Service, Maximilian Ott, Ivan Seskar, Robert Siracusa, Manpreet Singh, (TRIDENTCOM 2005), Trento, Italy, Feb 2005

ORBIT Measurements Framework and Library (OML): Motivations, Design,Implementation, and Features, Manpreet Singh, Maximilian Ott, Ivan Seskar and Pandurang Kamat, (TRIDENTCOM 2005), Trento, Italy, Feb 2005

OMF: a Control and Management Framework for Networking Testbeds, Thierry Rakotoarivelo, Max Ott, Ivan Seskar, Guillaume Jourjon, SOSP Workshop on Real Overlays and Distributed Systems (ROADS '09), Big Sky, Montana, Oct. 2009

Mobile Experiments Made Easy with OMF/Orbit, Christoph Dwertmann, Ergin Mesut, Guillaume Jourjon, Max Ott, Thierry Rakotoarivelo, Ivan Seskar, ACM SIGCOMM Demonstration Session, Barcelona, Spain, Aug. 2009

Measurement Architectures for Network Experiments with Disconnected Mobile Nodes, Jolyon White, Guillaume Jourjon, Thierry Rakotoarivelo, Max Ott, International Conference on Testbeds and Research Infrastructures for the Development of Networks and Communities (TRIDENTCOM 2010), Berlin, Germany, May 2010