POPROX Guide

• Stages
  • Stage 1: Intake
  • Stage 2: Specification
  • Stage 3: Testing
  • Stage 4: Operational
  • Stage 5: Closeout
• Measurement
  • Interaction data
  • Surveys
  • Data exports

The POPROX Guide describes how to design, plan, and implement experiments to run on POPROX. POPROX delivers a personalized daily emailed newsletter to a pool of daily readers who have agreed to be experimental subjects. It has an experimenter backend that researchers can use to conduct experiments.

Capabilities

What POPROX does

POPROX supports researchers in conducting live user studies (aka A/B tests or randomized control experiments) in the area of news recommendation. POPROX allows researchers to evaluate novel means of constructing and presenting recommendations and on exploring important research questions around news recommendation as a medium.

• POPROX has a pool of consented research subjects.

• POPROX has API-based access to news from the Associated Press.

• POPROX has a platform for managing subscriptions, user accounts, and compensation.

• POPROX delivers anonymized user data including interaction data and answers to regular surveys about users’ experience with the system.

What POPROX doesn’t do

POPROX aims to provide a wide range of support for researchers hoping to conduct this type of research. We do have some important limitations to keep in mind.

• POPROX is not a computing platform. Researchers will have to provision the computing resources necessary to calculate recommendations and provide a POPROX-compatible endpoint.

• POPROX is not a subcontractor. We will supply code examples and lot of help, but we cannot write code for you.

• POPROX is not a way around your ethics review board (e.g., IRB). Researchers will still need to create a human subjects protocol for their specific study and have it approved.

• POPROX is not (yet) an app. Daily newsletters are the only modality we support at this time.

• POPROX is (right now) only capable of running simple experimental designs. More complex designs (for example, multi-step within-subjects treatments) are planned for the future.

• At this stage in the project’s evolution, we can support US researchers only. We hope to make the service available to a global audience of researchers in the future, so stay tuned!

Next steps

If you think that POPROX might be a good platform for your work, you can

• read on about the Stages of engagement with POPROX,

• download and fill out our Intake Form, or

• or email our team at poprox@umn.edu.