textattack.metrics package

metrics package: to calculate advanced metrics for evaluting attacks and augmented text

• textattack.metrics.attack_metrics package
  • attack_metrics package:
  • Metrics on AttackQueries
  • AttackQueries
    • AttackQueries.avg_num_queries()
    • AttackQueries.calculate()
  • Metrics on AttackSuccessRate
  • AttackSuccessRate
    • AttackSuccessRate.attack_accuracy_perc()
    • AttackSuccessRate.attack_success_rate_perc()
    • AttackSuccessRate.calculate()
    • AttackSuccessRate.original_accuracy_perc()
  • Metrics on perturbed words
  • WordsPerturbed
    • WordsPerturbed.avg_number_word_perturbed_num()
    • WordsPerturbed.avg_perturbation_perc()
    • WordsPerturbed.calculate()
• textattack.metrics.quality_metrics package
  • Metrics on Quality package
  • Perplexity Metric:
  • Perplexity
    • Perplexity.calc_ppl()
    • Perplexity.calculate()
  • USEMetric class:
  • USEMetric
    • USEMetric.calculate()

Metric Class

class textattack.metrics.metric.Metric(**kwargs)

Bases: ABC

A metric for evaluating results and data quality.

abstract calculate(results)

Abstract function for computing any values which are to be calculated as a whole during initialization :param results: Attack results for each instance in dataset :type results: AttackResult objects