Changes between Initial Version and Version 1 of OMF/OMF60/0User/2OEDLDoc

Timestamp:

May 17, 2019, 10:42:51 AM (6 years ago)

Author:

seskar

Comment:

--

OMF/OMF60/0User/2OEDLDoc

@@ +1 @@
+== The OMF Experiment Description Language (OEDL) ==
+
+[[TOC(heading=OMF Documentation, OMF/*, reverse, depth=1)]]
+
+[[TOC(heading=OMF 6.0 Documentation, OMF/OMF60/0User*, depth=4)]]
+
+=== Overview ===
+
+OEDL is a domain-specific language for the description of an experiment's execution. OEDL is based on the Ruby language, but provides its own set of experiment-oriented commands and statements. As a new OEDL user you do not need to know Ruby to write experiment description. You can get started with only a few OEDL commands and syntax. However, some general entry-level programming knowledge would be a plus.
+
+An OEDL experiment description is composed of 2 main parts:
+ 1. a first part where we declare the resources that we will use in the experiment, such as applications or computing nodes, and some configurations that we would like to apply to these resources.
+ 1. a second part where we define the events that we would like to re-act to during the experiment's execution, and the associated tasks we would like to execute when these events occur.
+=== Simple examples ===
+ * An experiment with one computing resource, on which we execute the 'hostname' command once the resource is up
+{{{
+# This is the 1st section, where we declare resources and configurations
+#
+defGroup('Actor', "my_resource_name") 
+
+# This is the 2nd part, where we define events and associated tasks
+#
+onEvent(:ALL_NODES_UP) do |event|
+  info "This is my first OEDL experiment" 
+  group('Actor').exec('/bin/hostname')
+  after 5.seconds do
+    Experiment.leave_memberships
+    Experiment.done
+  end
+end
+}}}
+ * An experiment with one computing resource and one associated application resource (ping), which is started once all resources are up and installed
+{{{
+defProperty("a_resource", "replace-with-your-resource-name", "ID of a resource")
+
+defApplication('ping') do |app|
+  app.description = 'Simple Definition for the ping-oml2 application'
+  app.binary_path = '/usr/bin/ping-oml2'
+  app.defProperty('target', 'Address to ping', '', {:type => :string})
+  app.defProperty('count', 'Number of times to ping', '-c', {:type => :integer})
+end
+
+defGroup('My_Pinger', property.a_resource) do |g|
+  g.addApplication("ping") do |app|
+    app.setProperty('target', 'mytestbed.net')
+    app.setProperty('count', 3)
+  end
+end
+
+onEvent(:ALL_UP_AND_INSTALLED) do |event|
+  allGroups.startApplications
+  after 5.seconds do
+    Experiment.leave_memberships
+    Experiment.done
+  end
+end
+}}}
+
+=== Syntax ===
+ * Unless mentioned otherwise, all user-specified names used in the commands described below must be valid Ruby identifiers
+ * Comments within an OEDL experiment description starts with #, as in Ruby
+==== getResources ====
+ * The '''getResources''' command retrieves the list of resources which are associated with the current slice used by the experimenter
+ * Syntax:
+{{{
+list_of_resources = getResources()
+}}}
+ * The result of this command is an array where each element is a resource record.
+ * Such a record is a hash describing the resource. The experiment script can then filter the list of resources based on specific keys and values in such a record (e.g. only retain the resources of type 'node')
+ * An example of such a resource record is:
+{{{
+"client_id": "some_id_for_that_node",
+"status": "ready",
+"omf_id": "the_id_by_which_that_node_can_be_used_in_an_OEDL_script",
+"sliver_id": "urn:publicid:IDN+instageni.foo.edu+sliver+1234",
+"ssh_login": {
+  "hostname": "pc1.instageni.foo.edu",
+  "port": "12345" 
+},
+"interfaces": {
+  "some_id_for_that_node:if0": {
+    "client_id": "some_id_for_that_node:if0",
+    "sliver_id": "urn:publicid:IDN+instageni.foo.edu+sliver+1234",
+    "mac_address": "01234567890a",
+    "ip": [
+      {
+        "address": "10.10.1.1",
+        "type": "ipv4" 
+      }
+    ]
+  }
+},
+"urn": "urn:publicid:IDN+instageni.foo.edu+sliver+1234",
+"type": "node"
+}}} 
+
+==== defProperty & property & ensureProperty ====
+ * The defProperty command defines a new experiment property that can be used throughout the entire experiment description. The value of a property may change through the course of an experiment, and may be either a string of characters, a numerical value, or a boolean value.
+ * The property command is used to access a previously defined property, and can be used anywhere in the experiment description
+ * The ensureProperty command first check if a property exists, if not it defines it in the same way and using the same syntax as defProperty
+ * Syntax:
+{{{
+defProperty(name, default_value, description)
+property.name
+ensureProperty(name, default_value, description)
+
+- name: name of this property
+- default_value: the default value for this property
+- description: some text describing what this property is about
+}}}
+ * Usage Examples
+{{{
+defProperty('rate', 300, 'Bits per second sent from sender')
+defProperty('packetSize', 1024, 'Size of packets sent from sender, in Byte')
+
+some_variable = property.packetSize
+}}}
+ * When a property is defined using the defProperty command, and if you run the experiment with an OMF6 Experiment Controller (EC), you can set the property's initial value from the command line arguments of the EC.
+   * for example when using the EC to run an experiment named 'test.rb' which includes a definition for a property named 'my_resource'
+{{{
+$ omf_rc exec test.rb -- --my_resource node1.mytestbed.net
+}}}
+ * A property can be used in simple arithmetic (i.e. +, -, *, /) and string (i.e. +) operations
+{{{
+defProperty('rate1', 300, 'Bits per second sent from one app')
+defProperty('rate2', 400, 'Bits per second sent from another app')
+
+total_rate = property.rate1 + property.rate2 * 3
+}}}
+
+==== loadOEDL ====
+ * The loadOEDL command fetches an additional OEDL script and loads it at this point in the current OEDL script.
+ * This additional script can be either:
+   * shipped as part of the OMF EC gem package, or any other gem packages
+   * located at a given path on the local filesystem of the machine running the EC
+   * located at a given Web URL
+ * Optional parameters can be passed to this additional OEDL script by mean of experiment properties (see above section)
+ * Syntax:
+{{{
+loadOEDL(location, optional_properties)
+
+- location: a URI which references the OEDL script to load
+    The supported URI schemes are:
+    * system:///foo/bar.rb , which loads the file located at 'foo/bar.rb' in the default Ruby path of this EC
+    * file:///foo/bar.rb , which loads the file located at '/foo/bar.rb' on the local filesystem
+    * http://foo.com/bar.rb , which loads the file located at the URL 'http://foo.com/bar.rb'
+    NOTE: the 'file://' and 'system://' schemes do not support other hosts than localhost, 
+               thus the host part of these URI should be left empty
+- optional_properties: a hash of keys/values for the optional experiment properties to set before loading the OEDL script
+}}}
+ * Usage Examples
+{{{
+loadOEDL('system:///omf_ec/some_shipped_script')
+loadOEDL('system:///omf_ec/some_shipped_script',  { key1: 'value1', key2: 123 })
+
+loadOEDL('file:///home/alice/alice_extra_script.rb')
+
+loadOEDL('http://some.web.server.com/the_script.rb')
+}}}
+
+==== defApplication ====
+ * The defApplication command is used to declare the definition of an application that will be used throughout this experiment description. The details of this definition are provided in a declaration block following the defApplication. Once an application is declared using this command, any resource or group of resources within the experiment description may use it.
+ * Syntax:
+{{{
+defApplication(app_name) do |app|
+  app.declaration1
+  app.declaration2
+  ...
+end 
+
+- app_name: name of this application definition
+- declarationX: some declaration about this application 
+}}}
+ * Usage Example
+{{{
+defApplication('ping') do |app|
+  app.description = "A very simple application definition for ping" 
+  app.binary_path = "/usb/bin/ping" 
+end
+}}}
+ * There are 3 types of declarations that can be made within the declaration block
+   * Generic declaration:
+{{{
+These are declarations about various generic attributes of this application. The valid declarations here are:
+
+app.description = a String giving some human-readable description of this application, 
+                  e.g. what it is, what it does, how it works, etc...
+app.binary_path = a String holding the path to the binary for that application, when it is 
+                  installed on a computing resource
+app.map_err_to_out = a Boolean (true or false), if true then map the standard-error stream 
+                  to standard-output stream for a running instance of this application (default = false)
+app.pkg_tarball = a String holding a URI pointing to a tarball package for this application 
+                  (which will be extracted at the root "/" on the computing resource) 
+app.pkg_ubuntu = a String giving a Ubuntu package name for this application (which will be 
+                  installed using 'apt-get install' on the computing resource) 
+app.pkg_fedora = a String giving a Fedora package name for this application (which will be 
+                  installed using 'yum install' on the computing resource) 
+
+When reading the following declaration, the EC will instructs the Resource to download
+the tarball located at 'http://bar.com/foo.tgz', then uncompress it relatively to the root of the 
+Resource's filesystem. Here it is assumed that this tarball contains a binary which will be placed 
+at '/usb/local/bin/foo'
+
+  defApplication('foo') do |app|
+    app.description = "A very simple application definition for the foo application" 
+    app.binary_path = "/usb/local/bin/foo" 
+    app.pkg_tarball = "http://bar.com/foo.tgz" 
+  end
+}}}
+   * Property declaration:
+{{{
+These are declarations about the parameters that this application supports. These parameters can be either configured on the command line, or dynamically via the application's standard-input stream (if the application supports this). These parameters are declared using a defProperty command with a different syntax than previously described one above. This syntax is as follows:
+app.defProperty(prop_name, description, command_line, options)
+
+- prop_name: name for this parameter within OEDL experiment descriptions
+- description: some text describing what this parameter is about
+- command_line: the prefix that is used to set this parameter when starting the application instance from the command line, this can be an nil or an empty string when no prefix is required
+- options: a Ruby Hash with the set of options to apply to this parameter. The list of valid options are:
+     :order (Fixnum) the appearance order on the command line, default FIFO
+     :dynamic (Boolean) parameter can be dynammically changed, default false
+     :type (Numeric|String|Boolean) this parameter's type
+     :default value given by default to this parameter
+     :value value to set for this parameter
+     :mandatory (Boolean) this parameter is mandatory, default false
+
+Examples:
+
+  app.defProperty('target', 'Address to ping', nil, {:type => :string, :mandatory => true, :default => 'localhost'})
+  app.defProperty('count', 'Number of times to ping', '-c', {:type => :integer})
+}}}    
+   * Measurement declaration:
+{{{
+These are declarations about the Measurement Points (MP) that this application supports, if any. When an application has been instrumented using the OML Instrumentation Library, it can generate measurement streams during its execution. When this application is used within an OEDL experiment description, we can declare the existence of these Measurement Points and selectively enables the ones we are interested in. The declaration of such a MP is done using the following defMeasurement command and associated block:
+app.Measurement(mp_name) do |mp|
+  mp.defMetric(metric_name, metric_type)
+  ...
+end
+
+- mp_name: the name of this measurement point, as defined by the OML-instrumented application
+- metric_name: the name of a metric within this measurement point, as defined by the OML-instrumented application
+- metric_type: the type of that metric, as defined by the OML-instrumented application, e.g. :string, :uint32, :double, ...
+   the full list of supported types is at: http://oml.mytestbed.net/projects/oml/wiki/OML_generic_interface#Data-Types
+
+Examples:
+
+  app.defMeasurement('probe_statistic') do |m|
+    m.defMetric('dest_addr', :string)
+    m.defMetric('ttl', :uint32)
+    m.defMetric('rtt', :double)
+    m.defMetric('rtt_unit', :string)
+  end
+
+  app.defMeasurement('video_stream_statistic') do |m|
+    m.defMetric('frame_number', :uint64)
+    m.defMetric('drop_rate', :uint32)
+    m.defMetric('codec_name', :string)
+    m.defMetric('bitrate', :unit32)
+  end
+}}}
+
+==== defGroup ====
+ * The deGroup command defines a new group of resources that will participate to this experiment. Such a group may include one or many resources.
+ * As a group is itself considered a resource, a group may thus include one or many other groups.
+ * A set of configurations and/or applications to use may be associated to a group. This is optional. In such case, all members of the group will have these configurations and/or run these applications. This association is done via declaration block as in other OEDL commands.
+ * Syntax:
+{{{
+defGroup(group_name, resource1_name, resource2_name, ...)
+
+# or with the optional block:
+defGroup(group_name, resource1_name, resource2_name, ...) do |g|
+  ...
+end
+
+- group_name: a name for this group
+- resourceX_name: the name of the X-th resource to include in this group 
+}}}
+ * Within the optional declaration block, you can:
+   * associate an application to this group of resources, by using the addApplication command
+   * configure the parameters of that application to some values, by using the setProperty command
+   * enable the generation and collection of measurements streams supported by the application, by using the measure command
+   * these commands have the following syntax:
+{{{
+defGroup(group_name, resource1_name) do |g|
+  g.addApplication(app_definition_name, app_definition_location) do |app|
+    app.name = 'optional_name_prefix'
+    app.setProperty(prop_name, "some_value")
+    ...
+    app.measure(mp_name, :samples => 1)
+    ...
+  end
+end
+
+- app.name = 'optional_name_prefix'
+   This is an *OPTIONAL* statement, which you can use to set a custom prefix for the
+   name of this application instance. When this application will be created on the resource
+   side, that instance will get the name: "prefix_UID", where:
+     * prefix: is either this 'optional_name_prefix' or a default constructed one based on
+        the application definition
+     * UID: is a unique ID for that particular app instance on that particular resource. 
+- app_definition_name: the name of the application definition as defined by a previous 
+   defApplication declaration
+- app_definition_location: where to find the OMF definition for this application. This is an 
+   *OPTIONAL* parameter. If not provided, the application definition must be either:
+      * provided as part of the same script as this OEDL experiment script
+      * loaded earlier in the same OEDL script, using the loadOEDL command
+   This value can be either:
+     * a path relative to the OMF packages, e.g. "system:///omf_ec/some_directory/a_app_definition_file" 
+     * a path on the local file system where the EC is running, e.g. "file:///usr/local/shared/omf/a_app_definition_file" 
+     * a web URL, e.g. "http://mytestbed.net/app_definitions/a_app_definition_file" 
+- prop_name: the name of a parameter of the application to configure to some_value, as 
+   defined by a previous defProperty within a block of a defApplication command
+- mp_name: the name of a Measurement Point provided by the application, as defined by a 
+   previous defMeasurement within a block of a defApplication command.
+   The measure command takes an additional parameter to specify the measurement reporting frequency,
+   i.e. every X samples (:samples => X) or every X seconds (:interval => X)
+
+Examples:
+
+defGroup('Two_Resource', 'one_of_my_resource', 'another_of_my_resource')
+
+defGroup('Pinger', 'yet_another_of_my_resource') do |g|
+  g.addApplication("ping") do |app|
+    app.setProperty('target', 'mytestbed.net')
+    app.setProperty('count', 3)
+    app.measure('probe_statistic', :samples => 1)
+  end
+end
+}}}    
+  * Within the optional declaration block, you can also configure some networking parameter for this group of resources, using the following dot syntax:
+defGroup(group_name, resource1_name) do |g|
+  g.net.interface.attribut = some_value
+  ...
+end
+
+- interface: the OEDL shortname for a network interface, which can be either:
+   - e0: the first ethernet interface (often eth0 on Linux resources)
+   - e1: the second ethernet interface (often eth1 on Linux resources)
+   - w0: the first wireless ethernet interface (often wlan0 on Linux resources)
+   - w1: the first wireless ethernet interface (often wlan1 on Linux resources)
+- attribut: the name of a the networking attribut to configure to some_value, which can be either:
+   - ip: the IPv4 network address to use
+   - type: the type of 802.11 network to use, either 'a','b','g' or 'n' (only for 802.11 wireless interface)
+   - essid: the ESSID name to use (only for wireless interface)
+   - mode: the wireless mode to use, either 'adhoc', 'master' (i.e. Access Point), 'managed' 
+      (i.e. Wireless Station) name to use (only for wireless interface)
+   - channel: the channel ID to use, depending on the type of network (only for wireless interface)
+
+Examples:
+
+defGroup('Wireless_Client', 'one_of_my_resource') do |g|
+  g.net.w0.mode = "adhoc" 
+  g.net.w0.type = 'g'
+  g.net.w0.channel = "6" 
+  g.net.w0.essid = "example" 
+  g.net.w0.ip = "192.168.0.2/24" 
+end
+}}}
+
+==== onEvent ====
+ * OEDL adopts an event-based approach to describe the different actions required to perform during an experiment.
+ * There are a basic set of defined events that we supports (e.g. 'ALL_NODES_UP') and users can define their own custom events (using the defEvent command)
+ * The basic set of defined events are:
+   * :ALL_NODES_UP this event is triggered when all the computing resources of type 'node' have confirmed their membership to all their assigned groups in this experiment
+   * :ALL_UP_AND_INSTALLED this event is triggered when all :ALL_NODES_UP is triggered and in addition all the applications associated to all the groups have been installed (if no installation was required then the application is assumed to be installed)
+   * :ALL_APPS_DONE, this event is triggered when all resource of type application have reported that their running instance has exited
+   * :ALL_INTERFACE_UP, this event is triggered when all the resource of type network interface have been created and are ready to receive further commands
+   * :ALL_UP, this event is triggered when all the resources defined in the experiment script have been created and are ready to receive further commands
+ * The set of actions to execute when a given event has been triggered is described in a declarative block associated to that event, using the following syntax:
+{{{
+onEvent(:event_name, consume_event) do 
+  some_command
+  ...
+end
+
+- event_name: the name of the event to associate the enclosed actions with (e.g. :ALL_NODES_UP)
+- consume_event: optional, by default equal to true. If set to false, then the event is not consumed, i.e. it is allowed to be triggered again in the future
+- some_command: a command describing the actions to perform.
+
+Examples:
+
+onEvent(:ALL_NODES_UP)
+  Experiment.done
+end
+}}}
+ * Some available type of actions and their syntax are listed below:
+    * Output some information message on the standard-output of the experiment controller
+{{{
+info "Hello World!" 
+}}}    
+    * Start/stop all applications associated with all the groups
+{{{
+allGroups.startApplications
+allGroups.stopApplications
+}}}    
+    * Start/stop all applications associated with a given group called 'Actors'
+{{{
+group('Actors').startApplications
+group('Actors').stopApplications
+}}}    
+    * Execute another block of actions after a 10 seconds
+{{{
+group('Actor').startApplications
+after 10.seconds do
+    group('Actor').stopApplications
+end
+}}}    
+    * Execute a given shell command (e.g. "/bin/hostname") on all resources within a group called 'Actors'
+{{{
+group('Actors').exec('/bin/hostanme')
+}}}    
+    * Explicitly requests all resources to leave this experiment
+{{{
+Experiment.leave_memberships
+}}}    
+    * Explicitly terminates the experiment's execution. This requires all the resources to release/terminate all their running application, then leave the experiment
+{{{
+Experiment.done
+}}}    
+    * Close the EC instance without terminating the experiment, i.e. resources will keep performing whatever tasks they are assigned to and are still part of this experiment. This would be the case in long-running experiments, where the user would close her EC instance, while letting the experiment still running, then would start a new EC instance later to interact further with the resources.
+{{{
+Experiment.disconnect
+}}}
+    
+==== defEvent ====
+ * As mentioned above, a user can define her own custom events to monitor within an OEDL experiment. The pattern for use this feature is as follows:
+    * the user defines the conditions that would trigger her custom event within a defEvent block
+    * the user then tasks to perform when that event triggers in one or many onEvent block(s)
+{{{
+defEvent(:MY_EVENT) do |state|
+  # put here the conditions to check in order to trigger this event
+end
+...
+onEvent(:MY_EVENT) do
+  # put here the tasks to execute when the event :MY_EVENT has triggered
+end
+}}}
+ * the conditions in the defEvent block can be based on
+    * the state of one or many resources involved in the experiment (e.g. when the application foo has finished)
+    * the value of any measurements collected as part of the experiment (e.g. when loss rate is above 50%)
+    * any time conditions (e.g. at 7am, in 10 second, etc...)
+ * these conditions are checked:
+    * by default, only when a new message arrives to the Experiment Controller
+    * every X second, only if the every: optional parameter is set on defEvent
+ * the condition-checking code in the defEvent block must:
+    * return the boolean true, when the conditions have been met
+    * return false otherwise
+ * Syntax:
+{{{
+defEvent(event_name) do |state|
+... 
+end
+
+# or with the optional periodic timer:
+
+defEvent(event_name, every: time_duration) do |state|
+...
+end
+
+- event_name: a name for this event, we recommend using an all-capital symbol identifier, such as :MY_EVENT 
+- every: the name of the X-th resource to include in this group 
+}}}
+ * '''Example of a custom event based on the state of some resources:'''
+  * the EC keeps record of the states of all resources that are involved in an experiment trial, i.e. for each resource the EC maintain a set of <key,value> hash where key is a property of that resource and value is the last known value for that property
+  * each time that the EC receives an FRCP inform message from a given resource advertising the values of all/some of its properties, the EC updates its corresponding records
+  * thus it is possible for an experimenter to query these state records and trigger an event based on them
+  * these state records are available within the defEvent block through the 'state' variable in the following example
+    * the state variable is an array where each element is the record for a given resource
+    * such an element is a hash of <key, value> as mentioned above
+  * in the following example:
+    * in the defEvent block, we are checking each element of the 'state' array (i.e. each record for a resource)
+    * if that element has a 'type' key with the value 'application' (i.e. the resource is an application) and another 'state' key with the value 'stopped' (i.e. the application stopped), then we trigger the event
+    * the corresponding onEvent defines a block of instructions to execute when the event fired. It also has its consume_event parameter set to false, which allows the event to be fired again in the future
+{{{
+defEvent :APP_EXITED do |state|
+  triggered = false
+  state.each do |resource|
+      triggered = true if (resource.type == 'application') && (resource.state == 'stopped')
+  end
+  triggered
+end
+
+onEvent :APP_EXITED, consume_event= false do
+    info "An application has just finished... should we do something about it?" 
+end
+}}}
+ * '''Example of a custom event based on some collected measurements:'''
+   * when an experiment starts an [http://oml-doc.orbit-lab.org OML-instrumented application] with some of its [http://oml-doc.orbit-lab.org/wiki/MeasurementPoints Measurement Points] enabled, the collected measurements are accessible to the experimenter in the defEvent block. Thus the experimenter can define test conditions on these measurements and trigger the event when these conditions are met.
+   * the measurements are usually stored in an SQL-like database, and there are 2 ways to access these measurements:
+     * the experimenter can directly write a specific SQL query to select the measurements she is interested in, or
+     * she can use a [http://sequel.jeremyevans.net/ Sequel-based syntax] similar to the [wiki:/OMF/OMF60/0User/2OEDLDoc#defGraph defGraph command], to define a more user-friendly request for measurement
+   * in both cases, the request/query is ran periodically against the measurement database
+   * also in both cases, the results of the query are available as an array of hash:
+     * each element of the array is a Measurement Point sample that matches the query/request, and
+     * each <key,value> pair in that element is a metric and its corresponding value
+   * in the following example:
+     * we first use the Sequel-based syntax to define a request for the metrics 'oml_ts_client' and 'values' from the 'my_measurement_point' MP
+     * we then pass this defined request as the argument to the defQuery method, which will run it against the measurement database
+     * the value of the 'every' parameter of the 'defEvent' block is set to 1, which will tell the EC to execute the entire condition checking code in the defEvent block every 1 second
+     * alternatively in the commented line below, we also show how to express this same measurement request using a direct SQL syntax
+     * then if the returned array of measurement is not nil, we take the last one (e.g. pop) and compare its value against an arbitrary threshold to decide if we trigger the event
+{{{
+defEvent(:VALUE_ABOVE_THRESHOLD, every: 1) do
+  # Query for some measurements...
+  # returns an array where each element is a hash representing a row from the DB
+  query = ms('my_measurement_point').select { [ :oml_ts_client, :value ] }
+  data = defQuery(query)
+
+  # Alternatively the above line could also be:
+  # data = defQuery('select oml_ts_client, value from my_measurement_point_table')
+  #
+  # Also if you want to rename 'oml_ts_client' to 'ts'
+  # query = ms('my_measurement_point').select { [ oml_ts_client.as(:ts), :value ] } 
+  # data = defQuery('select oml_ts_client as ts, value from my_measurement_point_table')
+
+  triggered = false
+
+  if !data.nil? && !(last_row = data.pop).nil? # Make sure we have some data
+    triggered = true if last_row[:value].abs > 0.99
+  end
+  triggered
+end
+
+onEvent(:SINE_ABOVE_THRESHOLD) do
+    info "The value just went above the threshold... should we do something about it?" 
+end
+}}}
+  * '''Example of a custom event based on a timer:'''
+{{{
+defEvent(:AFTER_1230, every: 1) do
+  now = Time.now
+  true if now.hour>12 && now.min > 30
+end
+
+onEvent :AFTER_1230 do
+  info "Do something after 12:30..." 
+end
+}}}
+
+==== defGraph ====
+  * This command is used to describe graphs, which !LabWiki will display and update during an experiment execution.
+  * Note that these graphs will only be displayed in !LabWiki for now. If you are running an experiment directly from an OMF command line (i.e. without using !LabWiki at all), graphs will not be displayed. We plan to describe soon the graphing interface between OMF and !LabWiki, so that 3rd party may implement their own visualization tools to display defGraph-defined plots.
+  * Graphs can only be plotted using OML-collected data from applications, which are declared and used in OEDL experiments.
+  * There are many different types of graphs that can be plotted. The currently documented graph types are: line chart, pie chart, and histogram chart.
+  * Syntax:
+{{{
+defGraph 'graph_name' do |g|
+  g.ms('mp_name').select {[ :field_1, :field_2, ... ]}
+  g.caption 'some_caption'
+  g.type 'type_of_graph'
+  ... graph_specitific_options ...
+end
+- graph_name, a human readble name for this graph
+- mp_name, the name of the measurement point (from an application), which generates the measurement we would like to top
+- field_i, the name of the specific field(s) in that measurement point, which we are interested in plotting
+- some_caption, some caption text to add to the graph
+- type_of_graph, the type of graph to plot for now only the following are documented: line_chart3, pie_chart2, histogram2
+}}}
+  * Note that in addition to the field_i defined in by the application declaration. All OML Measurement Points also defined by default addition fields, such as:
+    * oml_sender_id, the id of the source that generated this measurement
+    * oml_ts_client, the timestamp at the source when this measurement was generated (remapped in the collection server time reference)
+    * oml_ts_server, the timestamp at the collection server when this measurment was received
+  * Specific options for line chart
+{{{
+  g.mapping :x_axis => :field_1, :y_axis => :field_2, :group_by => :field_3
+  g.xaxis :legend => 'some_legend_for_x_axis'
+  g.yaxis :legend => 'some_legend_for_x_axis', :ticks => {:format => 's'}
+}}}
+    * the above draws a line chart graph of field_2 as a function of field_1.
+    * optionally, if field_2 contains data from multiple sources, which are differentiated by field_3, then the :group_by option draws a separate line per data source within the same graph
+    * the legend for the X and Y axises can be set, as the format of the ticks to use
+  * Specific options for pie chart
+{{{
+  g.mapping :value => :field_1, :label => :field_2
+}}}
+    * the above draws a pie chart where the values for a pie piece are taken from field_1 and the label to assign to that pie piece taken from field_2.
+  * Specific options for histogram chart
+{{{
+  g.mapping :value => :field_1, :group_by => :field_2
+  g.yaxis :legend => 'some_legend_for_y_axis'
+  g.xaxis :legend => 'some_legend_for_x_axis', :ticks => {:format => ',.2f'}
+}}}
+    * the above chart draws an histogram of the values in field_1, using a automatic algorith to determine the histogram's bin size based on all the currently collected values
+    * optionally, if field_1 contains data from multiple sources, which are differentiated by field_2, then the :group_by option draws a separate histogram bar per data source within the same graph
+  * Examples:
+{{{
+# Assuming an application name 'ping' which defines the following measurement points:
+#  app.defMeasurement('probe') do |m|
+#    m.defMetric('dest_addr',:string)
+#    m.defMetric('ttl',:uint32)
+#    m.defMetric('rtt',:double)
+#    m.defMetric('rtt_unit',:string)
+#  end
+#  app.defMeasurement('rtt_stats') do |m|
+#    m.defMetric('min',:double)
+#    m.defMetric('avg',:double)
+#    m.defMetric('max',:double)
+#    m.defMetric('mdev',:double)
+#    m.defMetric('rtt_unit',:string)
+#  end
+
+# Draw a line chart for RTT values against time, in addition draw a line per source
+# that generated the probe data
+#
+defGraph 'RTT1' do |g|
+  g.ms('probe').select {[ :oml_sender_id, :oml_ts_client, :oml_ts_server, :rtt ]}
+  g.caption "Round Trip Time (RTT) reported by each resource" 
+  g.type 'line_chart3'
+  g.mapping :x_axis => :oml_ts_client, :y_axis => :rtt, :group_by => :oml_sender_id
+  g.xaxis :legend => 'time [s]'
+  g.yaxis :legend => 'RTT [ms]', :ticks => {:format => 's'}
+end
+
+# Draw a pie chart of the average RTT values other an entire ping run, using the sources
+# that generated the measurement as labels for the pieces of the pie
+#
+defGraph 'RTT2' do |g|
+  g.ms('rtt_stats').select {[ :oml_sender_id, :avg ]}
+  g.caption "RTT Comparison Between Resources [ms]" 
+  g.type 'pie_chart2'
+  g.mapping :value => :avg, :label => :oml_sender_id
+end
+
+# Draw a histogram chart for RTT values, if many source generated these values, then 
+# draw adjacent bars for each source for a given bin
+#
+defGraph 'RTT3' do |g|
+  g.ms('probe').select {[ :oml_sender_id, :oml_ts_client, :oml_ts_server, :rtt ]}.where("rtt < 1")
+  g.caption "Histogram of RTT counts [ms]" 
+  g.type 'histogram2'
+  g.mapping :value => :rtt, :group_by => :oml_sender_id
+  g.yaxis :legend => 'Count'
+  g.xaxis :legend => ' ', :ticks => {:format => ',.2f'}
+end
+}}}
+
+==== defPrototype ====
+The defPrototype can be used to define a new prototype that can be reused as is, or customised through a set of parameters.
+  * Usage
+{{{
+# Define a prototype with identifier "agressivePing" 
+
+defPrototype("aggressivePing") do |proto|
+  proto.description = "A node that flood packets to a destination to quickly determine packet drop rates" 
+  # Here we specify which property of the base application we would like to use 
+  # AND what are the default value we would like to give them.
+
+  # 'destination' is a mandatory parameter of ping, thus we do not set any default value
+  proto.defProperty('destination', 'Name or IP address of the destination')
+
+  # This is a agressive ping, so we set the default probe number to 1000, we also set the 
+  # flooding behavior as 'true' by default, along with an interval of 0
+  #
+  proto.defProperty('numPacket', 'Number of probe packets to flood', 1000)
+  proto.defProperty('enableFlood', 'Enable packet flooding', true)
+  proto.defProperty('pktInterval', 'Time interval between packets in a flood', 0)
+
+  # Here we bind this prototype with the "pingApp" application definition. (Assuming "pingApp" has been defined)
+  # And we also bind the properties that we decided to use and gave default values to 
+  #
+  proto.addApplication("ping") do |app|
+    app.bindProperty('dst', 'destination')
+    app.bindProperty('count', 'numPacket')
+    app.bindProperty('flood', 'enableFlood')
+    app.bindProperty('interval', 'pktInterval')
+  end
+end
+}}}
+  * Use A Defined Prototype Within Group Context
+After a prototype has been defined, it could be added to a group
+{{{
+# Define a group, The node(s) within this group will run the "aggressivePing" prototype/application
+#
+defGroup('worker', [node1, node2]) do |group|
+   # Here we decide that the default 1000 probes are not enough in our case
+   # and we give a specific value of 4000 to the 'numPacket' parameter of 
+   # the "aggressivePing" prototype
+   #
+   group.prototype("aggressivePing", {
+     'destination' => "192.168.0.1",
+     'numPacket' => 4000
+   })
+
+   ...
+end
+}}}
+=== Additional OMF EC Built-in Library ===
+The OMF6 OEDL-enabled EC comes with some built-in OEDL libraries (i.e. really additional OEDL scripts) that can be optionally loaded by an experimenter within her own OEDL experiment scripts (using the above load_oedl command.
+
+==== omf_ec/backward/timeout_resources ====
+  * This additional OEDL script implements a timeout timer for resources to checked-in an experiment
+  * When it is loaded as part as another experiment, it will:
+    * wait for the specified time in the experiment property 'oedl_timeout'
+    * when that wait is over, it checks if all resources defined in the experiment have joined all their groups (also as defined in the experiment)
+    * If not, then it stops the experiment.
+  * The default timeout value is set here to 120 s. To modify that you should set in your experiment the oedl_timeout property to the desired timeout in second. This must be done prior to or as you load this script.
+  * Usage example:
+{{{
+...
+# This must be added within your own OEDL experiment
+# (here we set the timeout to 180 s)
+
+loadOEDL('system:///omf_ec/backward/timeout_resources', { oedl_timeout: 180 })
+...
+}}}