textattack.metrics package

metrics package: to calculate advanced metrics for evaluting attacks and augmented text

Subpackages

• textattack.metrics.attack_metrics package
  • attack_metrics package:
  • Submodules
    • Metrics on AttackQueries
    • AttackQueries
      • AttackQueries.avg_num_queries()
      • AttackQueries.calculate()
    • Metrics on AttackSuccessRate
    • AttackSuccessRate
      • AttackSuccessRate.attack_accuracy_perc()
      • AttackSuccessRate.attack_success_rate_perc()
      • AttackSuccessRate.calculate()
      • AttackSuccessRate.original_accuracy_perc()
    • Metrics on perturbed words
    • WordsPerturbed
      • WordsPerturbed.avg_number_word_perturbed_num()
      • WordsPerturbed.avg_perturbation_perc()
      • WordsPerturbed.calculate()
• textattack.metrics.quality_metrics package
  • Metrics on Quality package
  • Submodules
    • BERTScoreMetric class:
    • BERTScoreMetric
      • BERTScoreMetric.calculate()
    • MeteorMetric class:
    • MeteorMetric
      • MeteorMetric.calculate()
    • Perplexity Metric:
    • Perplexity
      • Perplexity.calc_ppl()
      • Perplexity.calculate()
    • USEMetric class:
    • SBERTMetric
      • SBERTMetric.calculate()
    • USEMetric class:
    • USEMetric
      • USEMetric.calculate()

Submodules

Metric Class

class textattack.metrics.metric.Metric(**kwargs)

Bases: ABC

A metric for evaluating results and data quality.

abstractmethod calculate(results)

Abstract function for computing any values which are to be calculated as a whole during initialization :param results: Attack results for each instance in dataset :type results: AttackResult objects

Attack Metric Quality Recipes:

class textattack.metrics.recipe.AdvancedAttackMetric(choices: list[str] = ['use'])

Bases: Metric

Calculate a suite of advanced metrics to evaluate attackResults’ quality.

add_metric(name: str, metric: Metric)

calculate(results) → dict[str, float]

Abstract function for computing any values which are to be calculated as a whole during initialization :param results: Attack results for each instance in dataset :type results: AttackResult objects

property achoices