non-pre Language Models:
- textattack.constraints.grammaticality.language_models.google_language_model package
- textattack.constraints.grammaticality.language_models.learning_to_write package
GPT2 Language Models:
- class textattack.constraints.grammaticality.language_models.gpt2.GPT2(model_name='gpt2', **kwargs)
A constraint based on the GPT-2 language model.
from “Better Language Models and Their Implications” (openai.com/blog/better-language-models/)
model_name – id of GPT2 model
- get_log_probs_at_index(text_list, word_index)
Gets the probability of the word at index word_index according to GPT-2.
Assumes that all items in text_list have the same prefix up until word_index.
Language Models Constraint
- class textattack.constraints.grammaticality.language_models.language_model_constraint.LanguageModelConstraint(max_log_prob_diff=None, compare_against_original=True)
Determines if two sentences have a swapped word that has a similar probability according to a language model.
max_log_prob_diff (float) – the maximum decrease in log-probability in swapped words from x to x_adv
compare_against_original (bool) – If True, compare new x_adv against the original x. Otherwise, compare it against the previous x_adv.
Set the extra representation of the constraint using these keys.
To print customized extra information, you should reimplement this method in your own constraint. Both single-line and multi- line strings are acceptable.
- abstract get_log_probs_at_index(text_list, word_index)
Gets the log-probability of items in text_list at index word_index according to a language model.