chemicalchecker.tool.targetmate.nonconformist.evaluation

Evaluation of conformal predictors.

Functions

class_avg_c	Calculates the average number of classes per prediction of a conformal classification model.
class_empty	Calculates the rate of singleton predictions (prediction sets containing only a single class label) of a conformal classification model.
class_mean_errors	Calculates the average error rate of a conformal classification model.
class_mean_errors_one_class	Calculates the average error rate of a conformal classification model, considering only test examples belonging to class c.
class_mean_p_val	Calculates the mean of the p-values output by a conformal classification model.
class_n_correct	Calculates the number of correct predictions made by a conformal classification model.
class_one_c	Calculates the rate of singleton predictions (prediction sets containing only a single class label) of a conformal classification model.
class_one_err	Calculates the error rate of conformal classifier predictions containing only a single output label.
class_one_err_one_class	Calculates the error rate of conformal classifier predictions containing only a single output label.
cross_val_score	Evaluates a conformal predictor using cross-validation.
n_test	Provides the number of test patters used in the evaluation.
reg_max_size
reg_mean_errors	Calculates the average error rate of a conformal regression model.
reg_mean_size	Calculates the average prediction interval size of a conformal regression model.
reg_median_size
reg_min_size
reg_n_correct	Calculates the number of correct predictions made by a conformal regression model.
reg_q1_size
reg_q3_size
run_experiment	Performs a cross-validation evaluation of one or several conformal predictors on a collection of data sets in csv format.

Classes

BaseIcpCvHelper	Base class for cross validation helpers.
ClassIcpCvHelper	Helper class for running the cross_val_score evaluation method on IcpClassifiers.
RegIcpCvHelper	Helper class for running the cross_val_score evaluation method on IcpRegressors.