Stages#

We organize a POPROX experiment into 5 stages.

  1. Intake

  2. Specification

  3. Testing

  4. Operation

  5. Closeout

Stage 1: Intake#

This is the part of the process where a potential experimenter and POPROX determine if a particular study or research question is a good fit for POPROX’s capabilities at the present time.

More about the Intake stage.

Stage 2: Specification#

Once an experiment has made it through the Intake stage, the next goal is to specify the experiment fully so that it can be executed on the platform. This includes specifying and having approved the human subjects protocol. A key outut from this stage is the experiment manifest, which is a technical document compiled by the POPROX platform to control the experiment’s operation.

More about the Specification stage.

Stage 3: Testing#

The researcher need not have their code implemented at the time the specification stage is complete, but the next stage cannot begin until the experimenter’s endpoint is implemented and able to return recommendations.

At this point, the focus of attention moves to POPROX internally. We will conduct a series of tests to ensure that the experimenter’s code works as intended and working with the experimenter to fix any problems that arise.

More about the Testing stage.

Stage 4: Operational#

With testing complete, the experiment launches, which means that the experimenter’s code is responsible for delivering recommendations to POPROX users until the experiment is complete. POPROX staff will be closely monitoring the progress of the experiment for its duration.

More about the Operational stage.

Stage 5: Closeout#

Once an experiment is complete, the experimenter is free to release any resources committed to their recommender’s operation. POPROX will make data and metrics computed over that data available once they are prepared for release.

More about the Closeout stage.

On this page