textattack.constraints.grammaticality.language_models package

non-pre Language Models:

• textattack.constraints.grammaticality.language_models.google_language_model package
  • Google Language Models:
  • Google Language Models from Alzantot
  • GoogLMHelper
    • GoogLMHelper.clear_cache()
    • GoogLMHelper.get_words_probs()
    • GoogLMHelper.get_words_probs_uncached()
    • GoogLMHelper.CACHE_PATH
  • Google 1-Billion Words Language Model
  • GoogleLanguageModel
    • GoogleLanguageModel.check_compatibility()
    • GoogleLanguageModel.extra_repr_keys()
  • A library for loading 1B word benchmark dataset.
  • CharsVocabulary
    • CharsVocabulary.encode_chars()
    • CharsVocabulary.word_to_char_ids()
    • CharsVocabulary.max_word_length
    • CharsVocabulary.word_char_ids
  • LM1BDataset
    • LM1BDataset.get_batch()
    • LM1BDataset.vocab
  • Vocabulary
    • Vocabulary.decode()
    • Vocabulary.encode()
    • Vocabulary.id_to_word()
    • Vocabulary.word_to_id()
    • Vocabulary.bos
    • Vocabulary.eos
    • Vocabulary.size
    • Vocabulary.unk
  • get_batch()
  • Utils for loading 1B word benchmark dataset.
  • LoadModel()
• textattack.constraints.grammaticality.language_models.learning_to_write package
  • “Learning To Write”
  • AdaptiveSoftmax
  • AdaptiveLoss
    • AdaptiveLoss.forward()
    • AdaptiveLoss.remap_target()
    • AdaptiveLoss.reset()
    • AdaptiveLoss.training
  • AdaptiveSoftmax
    • AdaptiveSoftmax.forward()
    • AdaptiveSoftmax.log_prob()
    • AdaptiveSoftmax.reset()
    • AdaptiveSoftmax.set_target()
    • AdaptiveSoftmax.training
  • Language model helpers
  • QueryHandler
    • QueryHandler.load_model()
    • QueryHandler.query()
    • QueryHandler.try_query()
  • util_reverse()
  • “Learning To Write” Language Model
  • LearningToWriteLanguageModel
    • LearningToWriteLanguageModel.get_log_probs_at_index()
    • LearningToWriteLanguageModel.CACHE_PATH
  • RNN Language Model
  • RNNModel
    • RNNModel.forward()
    • RNNModel.init_hidden()
    • RNNModel.init_weights()
    • RNNModel.training

GPT2 Language Models:

class textattack.constraints.grammaticality.language_models.gpt2.GPT2(model_name='gpt2', **kwargs)

Bases: LanguageModelConstraint

A constraint based on the GPT-2 language model.

from “Better Language Models and Their Implications” (openai.com/blog/better-language-models/)

Parameters

model_name – id of GPT2 model

get_log_probs_at_index(text_list, word_index)

Gets the probability of the word at index word_index according to GPT-2.

Assumes that all items in text_list have the same prefix up until word_index.

Language Models Constraint

class textattack.constraints.grammaticality.language_models.language_model_constraint.LanguageModelConstraint(max_log_prob_diff=None, compare_against_original=True)

Bases: Constraint, ABC

Determines if two sentences have a swapped word that has a similar probability according to a language model.

Parameters

• max_log_prob_diff (float) – the maximum decrease in log-probability in swapped words from x to x_adv

• compare_against_original (bool) – If True, compare new x_adv against the original x. Otherwise, compare it against the previous x_adv.

extra_repr_keys()

Set the extra representation of the constraint using these keys.

To print customized extra information, you should reimplement this method in your own constraint. Both single-line and multi- line strings are acceptable.

abstract get_log_probs_at_index(text_list, word_index)

Gets the log-probability of items in text_list at index word_index according to a language model.