poprox_recommender.evaluation.evaluate

Generate evaluations for offline test data.

For an evaluation EVAL and PIPELINE, this script reads outputs/DATA/PIPELINE/recommendations.parquet and produces ouptuts/DATA/PIPELINE/profile-metrics.csv.gz and ouptuts/DATA/PIPELINE/metrics.json.

Usage:

poprox_recommender.evaluation.evaluate [options] EVAL PIPELINE

Options:

-v, --verbose

enable verbose diagnostic logs

--log-file=FILE

write log messages to FILE

-M DATA, --mind-data=DATA

read MIND test data DATA [default: MINDsmall_dev]

-P DATA, --poprox-data=DATA

read POPROX test data DATA

EVAL the name of the evaluation to measure PIPELINE

the name of the pipeline to measure

Functions

main()
profile_eval_results(eval_data, profile_recs)
rec_profiles(eval_data, profile_recs)	Iterate over rec profiles, yielding each recommendation list with its truth and whether the profile is personalized.

poprox_recommender.evaluation.evaluate.rec_profiles(eval_data, profile_recs)

Iterate over rec profiles, yielding each recommendation list with its truth and whether the profile is personalized. This supports parallel computation of the final metrics.

Parameters:

• eval_data (EvalData)

• profile_recs (DataFrame)

Return type:

Iterator[ProfileRecs]