POPROX Guide#

The POPROX Guide describes how to design, plan, and implement experiments to run on POPROX. POPROX delivers a personalized daily emailed newsletter to a pool of daily readers who have agreed to be experimental subjects. It has an experimenter backend that researchers can use to conduct experiments.

Capabilities#

What POPROX does#

POPROX supports researchers in conducting live user studies (aka A/B tests or randomized control experiments) in the area of news recommendation. POPROX allows researchers to evaluate novel means of constructing and presenting recommendations and on exploring important research questions around news recommendation as a medium.

  • Researchers don’t have to recruit users. POPROX already has a pool of daily readers for its email newsletter. These users have consented to participate in studies and to have their reading behavior tracked by our platform.

  • Researchers don’t have to scrape news content or negotiate with news sites. POPROX already has a daily stream of news articles from our partner, the Associated Press.

  • Researchers don’t have to create a platform for managing subscriptions and user accounts. We’ve done all that.

  • Researchers have access to anonymized user data including interaction data and answers to regular surveys about users’ experience with the system. As an experimenter, you can request other data to be collected before, during, or/and after running your study.

What POPROX doesn’t do#

POPROX aims to provide a wide range of support for researchers hoping to conduct this type of research. We do have some important limitations to keep in mind.

  • POPROX is not a computing platform. By default, we provide users with recommendation produced by a baseline algorithm, but researchers who aim to study novel recommendation algorithms will have to provision the computing resources necessary to calculate recommendations and provide a POPROX-compatible endpoint for the duration of their experiment.

  • POPROX is not a subcontractor. We will supply code examples and lot of help, but we cannot write code for you. If you have an idea for an algorithm or a novel presentation mechanism to explore, but don’t know how implement it, we suggest partnering with a research group or other organization who can help make your idea a reality.

  • POPROX is not a way around your ethics review board (e.g., IRB). While the users of our study have consented to participate in research studies, researchers will still need to create a human subjects protocol for their specific study and have it approved by an institutional ethics review board before an experiment can be run. We do provide assistance with the process.

  • POPROX is not (yet) an app. We do not have the capability of delivering interactive news recommendations in real-time. Daily newsletters are the only modality we support at this time.

  • POPROX is (right now) only capable of running simple experimental designs. More complex designs (for example, multi-step within-subjects treatments) are planned for the future.

  • At this stage in the project’s evolution, we can support US researchers only. We hope to make the service available to a global audience of researchers in the future, so stay tuned!

Next steps#

If you think that POPROX might be a good platform for your work, you can

  • read on about the Stages of engagement with POPROX,

  • download and fill out our Intake Form, or

  • or email our team at intake@poprox.ai.

On this page