Evaluator

class ucca.evaluation.Evaluator(verbose, constructions, units, fscore, errors)

Bases: object

Methods Summary

find_mutuals(m1, m2, eval_type, mutual_tags)
get_scores(p1, p2, eval_type[, r])	prints the relevant statistics and f-scores.

Methods Documentation

static find_mutuals(m1, m2, eval_type, mutual_tags, counter=None)

get_scores(p1, p2, eval_type, r=None)

prints the relevant statistics and f-scores. eval_type can be ‘unlabeled’, ‘labeled’ or ‘weak_labeled’. calculates a set of all the yields such that both passages have a unit with that yield. :param p1: passage to compare :param p2: reference passage object :param eval_type: evaluation type to use, out of EVAL_TYPES 1. UNLABELED: it doesn’t matter what labels are there. 2. LABELED: also requires tag match (if there are multiple units with the same yield, requires one match) 3. WEAK_LABELED: also requires weak tag match (if there are multiple units with the same yield,

requires one match)

Parameters:r – reference passage for fine-grained evaluation Returns:EvaluatorResults object if self.fscore is True, otherwise None