POPROX uses a pipelined design for its recommender implementations, built on the
LensKit pipeline. The pipeline works primarily with mutable data;
some components return entirely new data structures, while others add new data
to their input and return the modified structure (e.g. scorers augment a list
of articles with scores).
Note
POPROX is currently using a vendored copy of the pipeline code while it
is in development, exposed as poprox_recommender.lkpipeline.
There are quite a few components here — the recommendation experience is
composed from a set of small components, each of which has a specific
responsibility in the recommendation experience.
Todo
Document the components!
Article Embedder
We use NRMSArticleEmbedder to compute embeddings from article content. There are two instances of this embedder, one for candidate articles and another for articles in the user’s history.
Softmax-Sampled Ranker
The SoftmaxSampler component generates a stochastic ranking from the scored articles with softmax sampling. Many experimental pipelines replace this with a TopkRanker.
This pipeline implements simple top-K ranking using NRMS. This pipeline works
exactly like the default pipeline, except it replaces the
softmax sampler with a TopkRanker.
flowchart LR
subgraph input["Inputs"]
PROFILE[/"User Profile"/]
HISTORY[/"User Clicks"/]
ARTICLES[/"Candidate Articles"/]
end
subgraph primary["Primary Recommender"]
CandEmb["Article Embedder
(Candidates)"]
HistEmb["Article Embedder
(Clicked)"]
UserEmb["User Embedder"]
SCORE["Scorer"]
RANK["Top-K Ranker"]
class RANK changed
end
subgraph fallback["Fallback Recommender (cold users)"]
FILTER["Topic Filter"]
SAMPLER["Sampler"]
end
FILL["Fallback Fill"]
RESULT[\"Final Result"\]
ARTICLES --> CandEmb
HISTORY --> HistEmb
HistEmb --> UserEmb
PROFILE --> UserEmb
UserEmb --> SCORE
CandEmb --> SCORE
SCORE --> RANK
RANK --> FILL
PROFILE --> FILTER
ARTICLES --> FILTER
FILTER --> SAMPLER
SAMPLER -.->|if needed| FILL
FILL --> RESULT
classDef changed stroke-width:4px,stroke:red
The NRMS + MMR pipeline applies maximum marginal relevance to the NRMS scores.
As with the default pipeline, the straight top-K results are available for
comparison and evaluation.
flowchart LR
subgraph input["Inputs"]
PROFILE[/"User Profile"/]
HISTORY[/"User Clicks"/]
ARTICLES[/"Candidate Articles"/]
end
subgraph primary["Primary Recommender"]
CandEmb["Article Embedder
(Candidates)"]
HistEmb["Article Embedder
(Clicked)"]
UserEmb["User Embedder"]
SCORE["Scorer"]
RANK["MMR Ranker"]
class RANK changed
TOPK["Top-K Ranking"]
class TOPK extra
end
subgraph fallback["Fallback Recommender (cold users)"]
FILTER["Topic Filter"]
SAMPLER["Sampler"]
end
FILL["Fallback Fill"]
RESULT[\"Final Result"\]
ARTICLES --> CandEmb
HISTORY --> HistEmb
HistEmb --> UserEmb
PROFILE --> UserEmb
UserEmb --> SCORE
CandEmb --> SCORE
SCORE --> RANK
SCORE --> TOPK
RANK --> FILL
PROFILE --> FILTER
ARTICLES --> FILTER
FILTER --> SAMPLER
SAMPLER -.->|if needed| FILL
FILL --> RESULT
classDef extra stroke-dasharray:5 5,font-style:italic
classDef changed stroke-width:4px,stroke:red,font-style:bold
Alternative POPROX Recommender Pipeline (NRMS+MMR)#