Learning Model Overview


PILOTS home     WCL home
Tutorial index     (Basic ExamplesProgrammingRunningLearning Model OverviewLinear Regression ExampleBayes Classifier Example)

This page details instructions on how to execute a given PILOTS example with machine learning and serves as a guide for constructing and implementing new learning models.

Running a PILOTS program entails updating the machine learning server, generating new estimator files with new learning model definitions (with corresponding data), and using a PILOTS script to interact with the server to predict results using predefined models.

Note: The following commands shown for the command line are assumed to be implemented in the bash shell


Overview of Interaction with Machine Learning Component

The steps for interacting with the machine learning component of PILOTS are as follows:

  1. Installation of necessary packages
  2. Build a PILOTS jar file (from home directory)
  3. Configure aliases (from home directory)
  4. Train any new necessary learning models (offline)
  5. (If using simulation) Add Data for Producers
  6. Run the learning model server
  7. Compile and run the plt program with Producer and Output Handler
  8. (If using simulation) Output producer data to Output Handler

Installation of Necessary Packages

Make sure to install the following before working with the learning model component of PILOTS:

  1. python-pip
  2. virtualenv

Then simply run at the root directory of the project:

virtualenv .
source bin/activate

Finally run the following in the pilots/util/learningmodel/ directory:

pip install -r requirements.txt

Build a PILOTS jar file

In $PILOTS_HOME, run the build script

$ ./build.sh    // (for Windows, use build.bat instead)

After doing so, pilots.jar will be created under the $PILOTS_HOME/lib directory.

Configure aliases

In order to make sure that aliases for the plc and plcsim compiler commands are correctly figured as well as the definition of $PILOTS_HOME, make sure that you are in the root directory of the project and then use the following command:

source setenv

Training a Learning Model (Offline)

Before implementing a new learning model in PILOTS, the model has to be trained offline. In order to train a new model for the machine learning component in the PILOTS project, follow the specifications for the JSON definition file that are detailed in the README in the learningmodel directory.

Also, follow the specifications for the corresponding JSON schema file that are detailed in the README in the learningmodel directory. and add the corresponding data file in the same directory as the schema file. Within this data, make sure that the input data is separated by adjacent columns and that the first row acts as the header.

In server.json, add the loading path of the “.estimator” file with the name to be exposed to the PILOTS program.
For example, look at the linear regression model component of the server.json file.

Finally, make sure that the id used is mapped to a given sklearn learning model in pilots/util/learningmodel/engine/scbase/algorithms.py (for example, look at line 6)

Once this is done, execute the engineIO from the pilots/util/learningmodel/engine/ directory with the new JSON definition file as the command line argument (this will train the model that you have defined based on your definition file, schema, and data):

python engineIO.py <training definition file>

Once the EngineIO is executed, the input data for the learning model with be read, and the learning model with be trained offline. This training will result in a generated estimator file in the learningmodel directory and a message will display saying, "engineIO: Finished Training" if there are no errors in the offline training process.

After running the engineIO, the server.json should be updated with the new model and a new <learning_model>.estimator should be created which corresponds to the given JSON definition file.

Add Data for Producers

Make sure that the data that you intend to use for your producers in a simulation are stored in the data directory.

Follow the format similar to a.txt from the PredictionTest linear regression example.

Note: Notice that the first line of the file must be a header beginning with a pound sign (#). If you are expecting an output for the input, based on the learning model, then make sure to also include the label of the output similar to that defined in the model training data.

Running Learning Model Server

A socket connection must be made with the learning model server client in order to interact with the PILOTS machine learning component. Therefore, before running the plt program, run the learning model server by executing the server.sh script in pilots/util/learningmodel/

./server.sh

When the server is run, any new “.estimator” files will be loaded into the predictive server (loading the models defined in server.json)

(note: you may need to wait a few seconds for the server to start up)

Running the plt program with Producer and Output Handler

Once the previous procedures are completed, follow these 3 main steps:

  1. Run the desired output handler script for the corresponding PILOTS program in a new terminal.
  2. In a separate terminal, follow the instructions in the Compiling and running a program section of the PILOTS: Running Tutorial page. This involves the PILOTS compiler, javac, and execution.
  3. In a separate terminal, run the necessary producer script that will provide the input data.

(If using simulation) Output Producer Data to Output Handler

If a simulation is being run, go back to the terminal running the class generated from the plt file, and press enter. (After this, some warning messages may appear in this terminal, but they do not hinder the simulation.)

Learning Model Examples

For some simple examples of the execution of a PILOTS program with the machine learning component (offline), refer to the:

Additional Notes

(In the following example, the id linear_regression was used in both the ATR72.plt and the server.json file)



Tutorial index     (Basic ExamplesProgrammingRunningLearning Model OverviewLinear Regression ExampleBayes Classifier Example)
PILOTS home     WCL home

Worldwide Computing Laboratory

Department of Computer Science

Rensselaer Polytechnic Institute