textattack.constraints.grammaticality.language_models package

non-pre Language Models:

GPT2 Language Models:

class textattack.constraints.grammaticality.language_models.gpt2.GPT2(model_name='gpt2', **kwargs)[source]

Bases: LanguageModelConstraint

A constraint based on the GPT-2 language model.

from “Better Language Models and Their Implications” (openai.com/blog/better-language-models/)


model_name – id of GPT2 model

get_log_probs_at_index(text_list, word_index)[source]

Gets the probability of the word at index word_index according to GPT-2.

Assumes that all items in text_list have the same prefix up until word_index.

Language Models Constraint

class textattack.constraints.grammaticality.language_models.language_model_constraint.LanguageModelConstraint(max_log_prob_diff=None, compare_against_original=True)[source]

Bases: Constraint, ABC

Determines if two sentences have a swapped word that has a similar probability according to a language model.

  • max_log_prob_diff (float) – the maximum decrease in log-probability in swapped words from x to x_adv

  • compare_against_original (bool) – If True, compare new x_adv against the original x. Otherwise, compare it against the previous x_adv.


Set the extra representation of the constraint using these keys.

To print customized extra information, you should reimplement this method in your own constraint. Both single-line and multi- line strings are acceptable.

abstract get_log_probs_at_index(text_list, word_index)[source]

Gets the log-probability of items in text_list at index word_index according to a language model.