chemicalchecker.tool.targetmate.nonconformist.evaluation.cross_val_score
- cross_val_score(model, x, y, iterations=10, folds=10, fit_params=None, scoring_funcs=None, significance_levels=None, verbose=False)[source]
Evaluates a conformal predictor using cross-validation.
Parameters
- modelobject
Conformal predictor to evaluate.
- xnumpy array of shape [n_samples, n_features]
Inputs of data to use for evaluation.
- ynumpy array of shape [n_samples]
Outputs of data to use for evaluation.
- iterationsint
Number of iterations to use for evaluation. The data set is randomly shuffled before each iteration.
- foldsint
Number of folds to use for evaluation.
- fit_paramsdictionary
Parameters to supply to the conformal prediction object on training.
- scoring_funcsiterable
List of evaluation functions to apply to the conformal predictor in each fold. Each evaluation function should have a signature
scorer(prediction, y, significance).- significance_levelsiterable
List of significance levels at which to evaluate the conformal predictor.
- verboseboolean
Indicates whether to output progress information during evaluation.
Returns
- scorespandas DataFrame
Tabulated results for each iteration, fold and evaluation function.