1) Create a Source Report
You should begin by building a source report using our visual builder (Go to My Intelliboard > Create New > Create Report). This report will contain all the data needed to train and predict outcomes. We need:
- Unique identifiers from the Learning Management System for each case we want to use as a training example or a prediction input; (e.g. course ID, user ID, and Enrollment ID). You can include the course name and username for our clarity while reviewing the input report, though these values won't be used by the predictive engine.
- The variable that will distinguish current data from historical data. This column distinguishes between rows of the report that will be used to train the model, rows used to generate predictions, and rows to ignore (e.g. courses that have not yet started or students that will not be included in the model training or predictions). A conditional formula (If-type) should be used to make this distinction between current and historical rows.
- Case process criteria: it represents the success/label indicator that we are going to predict; (e.g. define pass or fail if course grade>50). Again here, an If-type formula will be needed.
- Multiple predictor variables or features. (e.g. participation per activity*, for all activities delivered within the course in the LMS; you can also break out submissions per assignment; attempts per quiz, and posts per forum). Please note that some data needs to be standardized: In the case of forum posts, we recommend a natural log of the number of posts since posts per forum are open-ended per user and there is no easily predictable upper boundary. Also, the Intelliboard plugin tracks an estimated time spent per user per activity, again this value can be incorporated into the model as a natural log of seconds spent per activity.
*Our research has shown that active participation - that is for example learner submission of work for feedback by an instructor, peer, or automated system - has the highest predictive power, but you will have the ability to add other predictors using any data stored and connected systems.
INPUT REPORT EXAMPLE
Unique identifiers; Variable about current data vs. Historical data; Success Criteria; Predictors
2) Create a New Model
When the source report has been created and saved, it can be used to create a predictive model. The columns of the report are identified as parameters of the model. You will have to go to Apps > Predictive Models > Create Model. One model is associated with one connection. Model creation is based on 7 steps:
Step 1 - Predictive Model Description: give the model a name and description; choose the source data report you have created for this model; choose your algorithm model.
Step 2 - Select "Case Identifier" columns: these values will be used to distinguish unique and related cases during model training and are also used to relate the results to existing data for reporting. e.g. user id, course id. They should be fields of type "ID". You may choose multiple columns.
Step 3 - Select the "Outcome column": choose the data column containing the outcome the model will predict. This will be populated in historical data and blank in prediction data. It must contain a value of 0 or 1 (e.g. passed)
Step 4 - Select the "Feature columns": choose the columns that will be used to predict the outcome (predictors). These must be numerical. You may choose more than 1.
Step 5 - Column for Prediction Result: insert the column name and click the next button. This will be used as the column name that stores the prediction results.
Step 6 - Select Model Process Criterion: Select a column in the data set that defines whether the data in that row is for training, prediction, or ignore: 1 = prediction, 0 = training, -1 = ignore (e.g current).
Step 7 - Check Prediction Model data: Review the configuration of our model and click create the model. The Model is created and we are able to train the model. This process can take several minutes; Execution occurs on a separate server to minimize disruption to users of the reporting system. A log displays the progress.