-
Shigeru Imai,
Thomas Chestna,
and Carlos A. Varela.
Elastic Scalable Cloud Computing Using Application-Level Migration.
In 5th IEEE/ACM International Conference on Utility and Cloud Computing (UCC 2012),
Chicago, Illinois, USA,
November 2012.
Keyword(s): distributed computing,
distributed systems,
cloud computing.
Abstract:
We present the Cloud Operating System (COS), a middleware framework to support autonomous workload elasticity and scalability based on application-level migration as a reconfiguration strategy. While other scalable frameworks (e.g., MapReduce or Google App Engine) force application developers to write programs following specific APIs, COS provides scalability in a general-purpose programming framework based on an actor-oriented programming language. When all executing VMs are highly utilized, COS scales a workload up by migrating mobile actors over to newly dynamically created VMs. When VM utilization drops, COS scales the workload down by consolidating actors and terminating idle VMs. Application-level migration is advantageous compared to VM migration especially in hybrid clouds in which migration costs over the Internet are critical to scale out the workloads. We demonstrate the general purpose programming approach using a tightly-coupled computation. We compare the performance of autonomous (i.e., COS-driven) versus ideal reconfiguration, as well as the impact of granularity of reconfiguration, i.e., VM migration versus application-level migration. Our results show promise for future fully automated cloud computing resource management systems that efficiently enable truly elastic and scalable general-purpose workloads. |
@InProceedings{imai-chestna-varela-ucc-2012,
author = {Shigeru Imai and Thomas Chestna and Carlos A. Varela},
title = {Elastic Scalable Cloud Computing Using Application-Level Migration},
booktitle = {5th IEEE/ACM International Conference on Utility and Cloud Computing (UCC 2012)},
year = 2012,
address = {Chicago, Illinois, USA},
month = {November},
pdf = {http://wcl.cs.rpi.edu/papers/ucc2012.pdf},
keywords = {distributed computing, distributed systems, cloud computing},
abstract = {We present the Cloud Operating System (COS), a middleware framework to support autonomous workload elasticity and scalability based on application-level migration as a reconfiguration strategy. While other scalable frameworks (e.g., MapReduce or Google App Engine) force application developers to write programs following specific APIs, COS provides scalability in a general-purpose programming framework based on an actor-oriented programming language. When all executing VMs are highly utilized, COS scales a workload up by migrating mobile actors over to newly dynamically created VMs. When VM utilization drops, COS scales the workload down by consolidating actors and terminating idle VMs. Application-level migration is advantageous compared to VM migration especially in hybrid clouds in which migration costs over the Internet are critical to scale out the workloads. We demonstrate the general purpose programming approach using a tightly-coupled computation. We compare the performance of autonomous (i.e., COS-driven) versus ideal reconfiguration, as well as the impact of granularity of reconfiguration, i.e., VM migration versus application-level migration. Our results show promise for future fully automated cloud computing resource management systems that efficiently enable truly elastic and scalable general-purpose workloads.}
}
-
Shigeru Imai and Carlos A. Varela.
A Programming Model for Spatio-Temporal Data Streaming Applications.
In Dynamic Data-Driven Application Systems (DDDAS 2012),
Omaha, Nebraska,
pages 1139-1148,
June 2012.
Keyword(s): programming languages,
cyber physical systems,
data streaming.
Abstract:
In this paper, we describe a programming model to enable reasoning about spatio-temporal data streams. A spatio-temporal data stream is one where each datum is related to a point in space and time. For example, sensors in a plane record airspeeds (va) during a given flight. Similarly, GPS units record an airplane's flight path over the ground including ground speeds (vg) at different locations. An aircraft's airspeed and ground speed are related by the following mathematical formula:, where va and αa are the aircraft airspeed and heading, and vw and αw are the wind speed and direction. Wind speeds and directions are typically forecast in 3,000-foot height intervals over discretely located fix points in 6-12 hour ranges. Modeling the relationship between these spatio-temporal data streams allows us to estimate with high probability the likelihood of sensor failures and consequent erroneous data. Tragic airplane accidents (such as Air France's Flight 447 on June 1st, 2009 killing all 216 passengers and 12 aircrew aboard) could have been avoided by giving pilots better information which can be derived from inferring stochastic knowledge about spatio-temporal data streams. This work is a first step in this direction |
@InProceedings{imai-varela-pilots-dddas-2012,
author = {Shigeru Imai and Carlos A. Varela},
title = {A Programming Model for Spatio-Temporal Data Streaming Applications},
booktitle = {Dynamic Data-Driven Application Systems (DDDAS 2012)},
year = 2012,
pages = {1139-1148},
address = {Omaha, Nebraska},
month = {June},
pdf = {http://wcl.cs.rpi.edu/papers/dddas2012.pdf},
keywords = {programming languages, cyber physical systems, data streaming},
abstract = {In this paper, we describe a programming model to enable reasoning about spatio-temporal data streams. A spatio-temporal data stream is one where each datum is related to a point in space and time. For example, sensors in a plane record airspeeds (va) during a given flight. Similarly, GPS units record an airplane's flight path over the ground including ground speeds (vg) at different locations. An aircraft's airspeed and ground speed are related by the following mathematical formula:, where va and αa are the aircraft airspeed and heading, and vw and αw are the wind speed and direction. Wind speeds and directions are typically forecast in 3,000-foot height intervals over discretely located fix points in 6-12 hour ranges. Modeling the relationship between these spatio-temporal data streams allows us to estimate with high probability the likelihood of sensor failures and consequent erroneous data. Tragic airplane accidents (such as Air France's Flight 447 on June 1st, 2009 killing all 216 passengers and 12 aircrew aboard) could have been avoided by giving pilots better information which can be derived from inferring stochastic knowledge about spatio-temporal data streams. This work is a first step in this direction}
}
-
Shigeru Imai and Carlos A. Varela.
Programming Spatio-Temporal Data Streaming Applications with High-Level Specifications.
In 3rd ACM SIGSPATIAL International Workshop on Querying and Mining Uncertain Spatio-Temporal Data (QUeST) 2012,
Redondo Beach, California, USA,
November 2012.
Keyword(s): programming languages,
cyber physical systems,
data streaming.
Abstract:
In this paper, we describe the design and implementation of PILOTS, a ProgrammIng Language for spatiO-Temporal data Streaming applications. Using PILOTS, application developers can easily program an application that handles spatio-temporal data streams by writing a high-level declarative program specification. Whereas spatio-temporal data is available with various spatial density and time frequency depending on data sources (e.g., weather forecast data can be given hourly/-daily for a vast geographic area, while GPS data can be given every second or at a higher frequency for a specific geographic location), applications often need to process data at a constant frequency. To view such heterogeneous data streams as homogeneous data streams, PILOTS specifically provides first-class support for data selection and interpolation so that applications can get data consistently regardless of the data's original spatio-temporal heterogeneity. To enable reasoning about errors in correlated spatio-temporal data streams, we introduce the notion of error signatures, patterns in output data streams that appear when input data is erroneous. These patterns are produced thanks to a mathematical model that explicitly specifies the redundancy exhibited in the input data. PILOTS applications readily produce error signatures, which can be an important tool to semi-automatically detect data error conditions and enable better decision support systems. As a motivating application, we illustrate a PILOTS program that receives as input data: the airspeed, the ground speed, and the wind speed for a flight. We then compute the error signatures exhibited by failing the airspeed data stream simulating a pitot tube icing scenario (such as the one occurring in Air France flight 447 in June 2009 ultimately killing all people onboard), and by failing the ground speed data stream simulating a GPS constellation shutdown. |
@InProceedings{imai-varela-quest-2012,
author = {Shigeru Imai and Carlos A. Varela},
title = {Programming Spatio-Temporal Data Streaming Applications with High-Level Specifications},
booktitle = {3rd ACM SIGSPATIAL International Workshop on Querying and Mining Uncertain Spatio-Temporal Data (QUeST) 2012},
year = 2012,
address = {Redondo Beach, California, USA},
month = {November},
pdf = {http://wcl.cs.rpi.edu/papers/quest2012.pdf},
keywords = {programming languages, cyber physical systems, data streaming},
abstract = {In this paper, we describe the design and implementation of PILOTS, a ProgrammIng Language for spatiO-Temporal data Streaming applications. Using PILOTS, application developers can easily program an application that handles spatio-temporal data streams by writing a high-level declarative program specification. Whereas spatio-temporal data is available with various spatial density and time frequency depending on data sources (e.g., weather forecast data can be given hourly/-daily for a vast geographic area, while GPS data can be given every second or at a higher frequency for a specific geographic location), applications often need to process data at a constant frequency. To view such heterogeneous data streams as homogeneous data streams, PILOTS specifically provides first-class support for data selection and interpolation so that applications can get data consistently regardless of the data's original spatio-temporal heterogeneity. To enable reasoning about errors in correlated spatio-temporal data streams, we introduce the notion of error signatures, patterns in output data streams that appear when input data is erroneous. These patterns are produced thanks to a mathematical model that explicitly specifies the redundancy exhibited in the input data. PILOTS applications readily produce error signatures, which can be an important tool to semi-automatically detect data error conditions and enable better decision support systems. As a motivating application, we illustrate a PILOTS program that receives as input data: the airspeed, the ground speed, and the wind speed for a flight. We then compute the error signatures exhibited by failing the airspeed data stream simulating a pitot tube icing scenario (such as the one occurring in Air France flight 447 in June 2009 ultimately killing all people onboard), and by failing the ground speed data stream simulating a GPS constellation shutdown.}
}
-
Shigeru Imai.
Task Offloading between Smartphones and Distributed Computational Resources.
Master's thesis,
Rensselaer Polytechnic Institute,
May 2012.
Keyword(s): distributed computing,
mobile computing,
cloud computing.
Abstract:
Smartphones have become very popular. While people enjoy various kinds of applications, some computation-intensive applications cannot be run on the smartphones since their computing power and battery life are still limited. We tackle this problem from two categories of applications. Applications in the first category are single-purpose and moderately slow (more than 10 seconds) to process on a single smartphone, but can be processed reasonably quickly by offloading a single module to a single computer (e.g., a face detection application). Applications in the second category are extremely slow (a few minutes to hours) to process on a single smartphone, but their execution time can be dramatically reduced by offloading computationally heavy modules to multiple computers (e.g., a face recognition application). In this category of applications, management of the server-side computation becomes more important since the intensity of computation is much stronger than in the first category. For the first category, we propose a light-weight task offloading method using runtime profiling, which predicts the total processing time based on a simple linear model and offloads a single task to a single server depending on the current performance of the network and server. Since this model’s simplicity greatly reduces the profiling cost at run-time, it enables users to start using an application without pre-computing a performance profile. Using our method, the performance of face detection for an image of 1.2 Mbytes improved from 19 seconds to 4 seconds. For the second category, we present a middleware framework called the Cloud Operating System (COS) as a back-end technology for smartphones. COS implements the notion of virtual machine (VM) malleability to enable cloud computing applications to effectively scale up and down. Through VM malleability, virtual machines can change their granularity by using split and merge operations. We accomplish VM malleability efficiently by using application-level migration as a reconfiguration strategy. Our experiments with a tightly-coupled computation show that a completely application-agnostic automated load balancer performs almost the same as human-driven VM-level migration; however, human-driven application-level migration outperforms (by 14% in our experiments) human-driven VM-level migration. These results are promising for future fully automated cloud computing resource management systems that efficiently enable truly elastic and scalable workloads. |
@MastersThesis{imais-taskoffloading-2012,
author = {Shigeru Imai},
title = {Task Offloading between Smartphones and Distributed Computational Resources},
school = {Rensselaer Polytechnic Institute},
year = 2012,
month = {May},
pdf = {http://wcl.cs.rpi.edu/theses/imai-master.pdf},
keywords = {distributed computing, mobile computing, cloud computing},
abstract = {Smartphones have become very popular. While people enjoy various kinds of applications, some computation-intensive applications cannot be run on the smartphones since their computing power and battery life are still limited. We tackle this problem from two categories of applications. Applications in the first category are single-purpose and moderately slow (more than 10 seconds) to process on a single smartphone, but can be processed reasonably quickly by offloading a single module to a single computer (e.g., a face detection application). Applications in the second category are extremely slow (a few minutes to hours) to process on a single smartphone, but their execution time can be dramatically reduced by offloading computationally heavy modules to multiple computers (e.g., a face recognition application). In this category of applications, management of the server-side computation becomes more important since the intensity of computation is much stronger than in the first category. For the first category, we propose a light-weight task offloading method using runtime profiling, which predicts the total processing time based on a simple linear model and offloads a single task to a single server depending on the current performance of the network and server. Since this model’s simplicity greatly reduces the profiling cost at run-time, it enables users to start using an application without pre-computing a performance profile. Using our method, the performance of face detection for an image of 1.2 Mbytes improved from 19 seconds to 4 seconds. For the second category, we present a middleware framework called the Cloud Operating System (COS) as a back-end technology for smartphones. COS implements the notion of virtual machine (VM) malleability to enable cloud computing applications to effectively scale up and down. Through VM malleability, virtual machines can change their granularity by using split and merge operations. We accomplish VM malleability efficiently by using application-level migration as a reconfiguration strategy. Our experiments with a tightly-coupled computation show that a completely application-agnostic automated load balancer performs almost the same as human-driven VM-level migration; however, human-driven application-level migration outperforms (by 14% in our experiments) human-driven VM-level migration. These results are promising for future fully automated cloud computing resource management systems that efficiently enable truly elastic and scalable workloads.}
}