textattack.constraints.grammaticality.language_models package

non-pre Language Models:

• textattack.constraints.grammaticality.language_models.google_language_model package
  • Google Language Models:
  • Google Language Models from Alzantot
  • Google 1-Billion Words Language Model
  • A library for loading 1B word benchmark dataset.
  • Utils for loading 1B word benchmark dataset.
• textattack.constraints.grammaticality.language_models.learning_to_write package
  • “Learning To Write”
  • AdaptiveSoftmax
  • Language model helpers
  • “Learning To Write” Language Model
  • RNN Language Model

GPT2 Language Models:

class textattack.constraints.grammaticality.language_models.gpt2.GPT2(model_name='gpt2', **kwargs)

Bases: textattack.constraints.grammaticality.language_models.language_model_constraint.LanguageModelConstraint

A constraint based on the GPT-2 language model.

from “Better Language Models and Their Implications” (openai.com/blog/better-language-models/)

Parameters

model_name – id of GPT2 model

get_log_probs_at_index(text_list, word_index)

Gets the probability of the word at index word_index according to GPT-2.

Assumes that all items in text_list have the same prefix up until word_index.

Language Models Constraint

class textattack.constraints.grammaticality.language_models.language_model_constraint.LanguageModelConstraint(max_log_prob_diff=None, compare_against_original=True)

Bases: textattack.constraints.constraint.Constraint, abc.ABC

Determines if two sentences have a swapped word that has a similar probability according to a language model.

Parameters

• max_log_prob_diff (float) – the maximum decrease in log-probability in swapped words from x to x_adv

• compare_against_original (bool) – If True, compare new x_adv against the original x. Otherwise, compare it against the previous x_adv.

extra_repr_keys()

Set the extra representation of the constraint using these keys.

To print customized extra information, you should reimplement this method in your own constraint. Both single-line and multi- line strings are acceptable.

abstract get_log_probs_at_index(text_list, word_index)

Gets the log-probability of items in text_list at index word_index according to a language model.