PILOTS: Programming Tutorial

Tutorial index (Basic Examples Programming Running Learning Model Overview Linear Regression Example)

Introduction

PILOTS is a programming language designed to enable a high level specification of streaming applications which handle spatio-temporal data. Specifically, PILOTS applications are able to monitor heterogeneous input streams and combine their data to produce output streams. The computational options for combining input data is limited due to the experimental nature of this language. Instead the focus of PILOTS applications should be on the input data and how they are related to each other. Currently as of v0.2.x, the input data for PILOTS applications is limited to the double type and can be generated in two ways: from a file with a special format or from a periodic thread which produces linear data. Input data generation falls under the responsibility of external software components, which will be discussed in the running tutorial.

With a high level specification, PILOTS programmers can rapidly prototype experiments on spatio-temporal objects in real time. The language incorporates a data selection model, and application model, and an error correction model.

The data selection model views discrete data points as a continuous spatio-temporal domain.

The application model designates what is to be done with the input data: how to compute the output stream data.

The data correction model allows input redundancies to be made explicit so that a correction may be made in case of a particular failure.

A PILOTS program is compiled directly into Java (using Java CC) which utilizes TCP/IP sockets to handle input and output. A PILOTS application, which is the compiled version of the program, runs on Java so it is compatible cross-platform as long as Java is installed. This tutorial will explain how to write a PILOTS program. First the PILOTS grammar is introduced, which is similar in style to Pascal. Next a very simple example, called Squared is demonstrated as an equivalent to the famous "Hello World!" program.

Grammar

The grammar of the language is separated into clauses which describe the inputs, outputs, and error correcting mechanics of a program. A formal definition of the grammar can be seen in the figure below. Every PILOTS program must have an inputs and outputs clause. Only programs which incorporate error correcting code need to include the errors, signatures, and correct clauses. The signatures clause enables both error correction and detection. A program without a correct clause is still capable of error detection, but not correction. The Corrects and Correct fields will be removed after version 0.3.1.

Program := program Var;
inputs Inputs;
outputs Outputs;
[errors Errors;]
[signatures Signatures;
[correct Corrects;]]
end

Inputs := [(Input;)* Input]

Input := Vars Dim using Methods

Outputs := [(Output;)* Output]

Output := Vars : Exps at every Time

Errors := [(Error;)* Error]

Error := Vars : Exps

Signatures := [(Signature;)* Signature]

Signature := Var[Const] : Var = Exps [Estimate][String]

Estimate := estimate Var = Exp

Corrects := [(Correct;)* Correct]

Correct := Var[Const] : Var = Exps

Vars := Var | Var, Vars

Var := { a, b, c, ... }

Const := ( Var )

String := { "a", "b", "c" }

Dim := '(t)' | '(t,x)' | '(t,x,y)' | '(t,x,y,z)' | '(t,x,y,z,n)'

Methods := Method | Method, Methods

Method := (closest | euclidean | interpolate | predict) '(' Exps ')'

Time := Number (msec | sec | min | hour | day)

Exps := Exp | Exp, Exps

Exp := Func()Exps | Exp Func Exp | '(' Exp ')' | Value

Func := { +, -, *, /, sqrt, sin, cos, tan, abs, ...}

Value := Number | Var

Number := Sign Digits | Sign Digits'.'Digits

Sign := '+' | '-' | ''

Digits := Digit | Digit Digits

Digit := { 0, 1, 2, ..., 9 }

The input clause is where the spatio-temporal dimensions of each input variable is defined along with associated data selection operations. The possible data selection operations are closest, euclidean, and interpolate. The closest operation takes one dimension as a parameter and selects the closest available data in that dimension. Euclidean is similar to closest, except that it takes multiple spatial dimensions and selects the data that is closest in euclidean distance to the queried location. Interpolate takes an extra parameter (denoted above as n) which denotes the number of nearby points to consider as part of a euclidean average. The specific dimensions that are used (out of {x,y,z,t}) within a PILOTS program are important. They determine whether a PILOTS application's data streams are spatio-temporal, or just temporal. All PILOTS programs must select on at least the 't' dimension, otherwise PILOTS may not be an appropriate language choice. Therefore all PILOTS applications (whether real-time or simulated) have a notion of "current time." If 't' is the only dimension that is specified, then application data is only temporal. However, if 'x','y', or 'z' are used then the application must be told what the "current location" is at the "current time." This is achieved by supplying a text file called "x-y-x.txt" which is processed before the input streams enter the data selection module. The exact format of the "x-y-x.txt" file will be described in the running tutorial.

Basic example

As an initial example we present Squared.plt, which simply squares input data. This example can be found precompiled in $PILOTS_HOME/examples/squared.

program Squared;
inputs
x(t) using closest(t);
outputs
o: x*x at every 1 sec;
end

In this application the selection module has received data for a variable named "x". The output is simply the input squared, which is produced every second. Presumably this application could be a link in a larger chain of similar applications where the output of one is the input of another.

Example with error detection/correction

program Twice;
inputs
a(t) using closest(t);
b(t) using closest(t);
outputs
o: b-2*a at every 1 sec;
errors
e: b-2*a;
signatures
s0: e = 0 "Normal mode";
s1(K): e = 2 * t + K "A failure"
estimate a = b / 2;
s2(K): e = -2 * t + K "B failure"
estimate b = a * 2;
s3(K): e = K, abs(K) > 20 "Out-of-sync";
end

In this example input stream b is intended to always be twice that of stream a. We feature error detection and correction using error signatures depicted in the signatures clause. Each signature has a label followed by a pattern and an optional description string (which is used for debugging purposes). The pattern within the signatures is an expression that may include any input variable or error function. Additionaly, a constant K is used to denote the presence of an unknown. Internally this is handled by executing search over a (small) number of measured data points to find the best estimate for the unknown. Notice that the signature s3 has a constraint after its pattern. Finally, the correct clause describes what actions to take when a signature has been detected. If a signature is not included in the correct clause then it is unrecoverable (or the "Normal mode" signature where no corrective action is necessary).

Example with prediction

program Twice;
inputs
a, b(t) using closest(t);
c(t) using predict(linear_model, a);
outputs
o: b - c at every 1 sec;
end

In this example input stream b is intended to always be twice that of stream a. After the offline training process in machine learning component using existing data containing a and b, we use data stream a as parameter to the linear_model to generate a new data stream c, which is an estimation of b. Finally, we output the difference of b and c to verify the correctness of the trained model linear_model.

Appendix

Example with error detection/correction before v0.3.1

Same function as "Example with error detection/correction" with different grammar

Tutorial index (Basic Examples Programming Running Learning Model Overview Linear Regression Example Bayes Classifier Example)

PILOTS home WCL home

Worldwide Computing Laboratory

Department of Computer Science

Rensselaer Polytechnic Institute

Program	:=	program Var; inputs Inputs; outputs Outputs; [errors Errors;] [signatures Signatures; [correct Corrects;]] end
Inputs	:=	[(Input;)* Input]
Input	:=	Vars Dim using Methods
Outputs	:=	[(Output;)* Output]
Output	:=	Vars : Exps at every Time
Errors	:=	[(Error;)* Error]
Error	:=	Vars : Exps
Signatures	:=	[(Signature;)* Signature]
Signature	:=	Var[Const] : Var = Exps [Estimate][String]
Estimate	:=	estimate Var = Exp
Corrects	:=	[(Correct;)* Correct]
Correct	:=	Var[Const] : Var = Exps
Vars	:=	Var \| Var, Vars
Var	:=	{ a, b, c, ... }
Const	:=	( Var )
String	:=	{ "a", "b", "c" }
Dim	:=	'(t)' \| '(t,x)' \| '(t,x,y)' \| '(t,x,y,z)' \| '(t,x,y,z,n)'
Methods	:=	Method \| Method, Methods
Method	:=	(closest \| euclidean \| interpolate \| predict) '(' Exps ')'
Time	:=	Number (msec \| sec \| min \| hour \| day)
Exps	:=	Exp \| Exp, Exps
Exp	:=	Func()Exps \| Exp Func Exp \| '(' Exp ')' \| Value
Func	:=	{ +, -, *, /, sqrt, sin, cos, tan, abs, ...}
Value	:=	Number \| Var
Number	:=	Sign Digits \| Sign Digits'.'Digits
Sign	:=	'+' \| '-' \| ''
Digits	:=	Digit \| Digit Digits
Digit	:=	{ 0, 1, 2, ..., 9 }