TextAttack & AllenNLP

This is an example of testing adversarial attacks from TextAttack on pretrained models provided by AllenNLP.

In a few lines of code, we load a sentiment analysis model trained on the Stanford Sentiment Treebank and configure it with a TextAttack model wrapper. Then, we initialize the TextBugger attack and run the attack on a few samples from the SST-2 train set.

For more information on AllenNLP pre-trained models: https://docs.allennlp.org/models/main/

For more information about the TextBugger attack: https://arxiv.org/abs/1812.05271

Open In Colab

View Source on GitHub

[4]:
!pip install allennlp allennlp_models > /dev/null
[7]:
!pip3 install textattack[tensorflow]
Requirement already satisfied: textattack[tensorflow] in /usr/local/lib/python3.7/dist-packages (0.3.3)
Requirement already satisfied: language-tool-python in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (2.6.1)
Requirement already satisfied: terminaltables in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (3.1.0)
Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (3.0.12)
Requirement already satisfied: numpy>=1.19.2 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (1.19.5)
Requirement already satisfied: scipy>=1.4.1 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (1.4.1)
Requirement already satisfied: word2number in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (1.1)
Requirement already satisfied: lru-dict in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (1.1.7)
Requirement already satisfied: nltk in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (3.2.5)
Requirement already satisfied: pandas>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (1.1.5)
Requirement already satisfied: num2words in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (0.5.10)
Requirement already satisfied: PySocks!=1.5.7,>=1.5.6 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (1.7.1)
Requirement already satisfied: transformers>=3.3.0 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (4.5.1)
Requirement already satisfied: lemminflect in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (0.2.2)
Requirement already satisfied: more-itertools in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (8.8.0)
Requirement already satisfied: editdistance in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (0.5.3)
Requirement already satisfied: bert-score>=0.3.5 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (0.3.10)
Requirement already satisfied: torch!=1.8,>=1.7.0 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (1.9.0+cu111)
Requirement already satisfied: tqdm<4.50.0,>=4.27 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (4.49.0)
Requirement already satisfied: flair in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (0.9)
Requirement already satisfied: datasets in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (1.11.0)
Requirement already satisfied: tensorflow>=2 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (2.6.0)
Requirement already satisfied: tensorflow-hub in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (0.12.0)
Requirement already satisfied: tensorboardX in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (2.4)
Collecting tensorflow-text>=2
  Downloading tensorflow_text-2.6.0-cp37-cp37m-manylinux1_x86_64.whl (4.4 MB)
     |████████████████████████████████| 4.4 MB 5.4 MB/s
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from bert-score>=0.3.5->textattack[tensorflow]) (2.26.0)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from bert-score>=0.3.5->textattack[tensorflow]) (3.2.2)
Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.7/dist-packages (from bert-score>=0.3.5->textattack[tensorflow]) (21.0)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.9->bert-score>=0.3.5->textattack[tensorflow]) (2.4.7)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.0.1->textattack[tensorflow]) (2.8.2)
Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.0.1->textattack[tensorflow]) (2018.9)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.7.3->pandas>=1.0.1->textattack[tensorflow]) (1.15.0)
Requirement already satisfied: grpcio<2.0,>=1.37.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (1.41.0)
Requirement already satisfied: tensorflow-estimator~=2.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (2.6.0)
Requirement already satisfied: protobuf>=3.9.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (3.17.3)
Requirement already satisfied: google-pasta~=0.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (0.2.0)
Requirement already satisfied: gast==0.4.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (0.4.0)
Requirement already satisfied: flatbuffers~=1.12.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (1.12)
Requirement already satisfied: termcolor~=1.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (1.1.0)
Requirement already satisfied: tensorboard~=2.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (2.6.0)
Requirement already satisfied: keras~=2.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (2.6.0)
Requirement already satisfied: clang~=5.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (5.0)
Requirement already satisfied: absl-py~=0.10 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (0.12.0)
Requirement already satisfied: wrapt~=1.12.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (1.12.1)
Requirement already satisfied: astunparse~=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (1.6.3)
Requirement already satisfied: wheel~=0.35 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (0.37.0)
Requirement already satisfied: typing-extensions~=3.7.4 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (3.7.4.3)
Requirement already satisfied: opt-einsum~=3.3.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (3.3.0)
Requirement already satisfied: keras-preprocessing~=1.1.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (1.1.2)
Requirement already satisfied: h5py~=3.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (3.1.0)
Requirement already satisfied: cached-property in /usr/local/lib/python3.7/dist-packages (from h5py~=3.1.0->tensorflow>=2->textattack[tensorflow]) (1.5.2)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (0.6.1)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (0.4.6)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (3.3.4)
Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (57.4.0)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (1.8.0)
Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (1.0.1)
Requirement already satisfied: google-auth<2,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (1.35.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (4.7.2)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (4.2.4)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (1.3.0)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (3.10.1)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (0.4.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.7/dist-packages (from requests->bert-score>=0.3.5->textattack[tensorflow]) (2.0.6)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->bert-score>=0.3.5->textattack[tensorflow]) (1.25.11)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->bert-score>=0.3.5->textattack[tensorflow]) (2.10)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->bert-score>=0.3.5->textattack[tensorflow]) (2021.5.30)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (3.1.1)
Requirement already satisfied: sacremoses in /usr/local/lib/python3.7/dist-packages (from transformers>=3.3.0->textattack[tensorflow]) (0.0.46)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers>=3.3.0->textattack[tensorflow]) (2019.12.20)
Requirement already satisfied: tokenizers<0.11,>=0.10.1 in /usr/local/lib/python3.7/dist-packages (from transformers>=3.3.0->textattack[tensorflow]) (0.10.3)
Requirement already satisfied: xxhash in /usr/local/lib/python3.7/dist-packages (from datasets->textattack[tensorflow]) (2.0.2)
Requirement already satisfied: fsspec>=2021.05.0 in /usr/local/lib/python3.7/dist-packages (from datasets->textattack[tensorflow]) (2021.10.1)
Requirement already satisfied: multiprocess in /usr/local/lib/python3.7/dist-packages (from datasets->textattack[tensorflow]) (0.70.12.2)
Requirement already satisfied: huggingface-hub<0.1.0 in /usr/local/lib/python3.7/dist-packages (from datasets->textattack[tensorflow]) (0.0.19)
Requirement already satisfied: dill in /usr/local/lib/python3.7/dist-packages (from datasets->textattack[tensorflow]) (0.3.4)
Requirement already satisfied: pyarrow!=4.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from datasets->textattack[tensorflow]) (3.0.0)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from huggingface-hub<0.1.0->datasets->textattack[tensorflow]) (3.13)
Requirement already satisfied: segtok>=1.5.7 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (1.5.10)
Requirement already satisfied: hyperopt>=0.1.1 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (0.1.2)
Requirement already satisfied: langdetect in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (1.0.9)
Requirement already satisfied: konoha<5.0.0,>=4.0.0 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (4.6.5)
Requirement already satisfied: bpemb>=0.3.2 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (0.3.3)
Requirement already satisfied: gdown==3.12.2 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (3.12.2)
Requirement already satisfied: ftfy in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (6.0.3)
Requirement already satisfied: conllu>=4.0 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (4.4.1)
Requirement already satisfied: scikit-learn>=0.21.3 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (0.22.2.post1)
Requirement already satisfied: sqlitedict>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (1.7.0)
Requirement already satisfied: mpld3==0.3 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (0.3)
Requirement already satisfied: lxml in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (4.2.6)
Requirement already satisfied: janome in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (0.4.1)
Requirement already satisfied: gensim<=3.8.3,>=3.4.0 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (3.6.0)
Requirement already satisfied: sentencepiece==0.1.95 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (0.1.95)
Requirement already satisfied: tabulate in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (0.8.9)
Requirement already satisfied: wikipedia-api in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (0.5.4)
Requirement already satisfied: deprecated>=1.2.4 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (1.2.13)
Requirement already satisfied: smart-open>=1.2.1 in /usr/local/lib/python3.7/dist-packages (from gensim<=3.8.3,>=3.4.0->flair->textattack[tensorflow]) (5.2.1)
Requirement already satisfied: networkx in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.1.1->flair->textattack[tensorflow]) (2.6.3)
Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.1.1->flair->textattack[tensorflow]) (0.16.0)
Requirement already satisfied: pymongo in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.1.1->flair->textattack[tensorflow]) (3.12.0)
Requirement already satisfied: overrides<4.0.0,>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from konoha<5.0.0,>=4.0.0->flair->textattack[tensorflow]) (3.1.0)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->markdown>=2.6.8->tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (3.6.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->bert-score>=0.3.5->textattack[tensorflow]) (1.3.2)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->bert-score>=0.3.5->textattack[tensorflow]) (0.10.0)
Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.21.3->flair->textattack[tensorflow]) (1.0.1)
Requirement already satisfied: wcwidth in /usr/local/lib/python3.7/dist-packages (from ftfy->flair->textattack[tensorflow]) (0.2.5)
Requirement already satisfied: docopt>=0.6.2 in /usr/local/lib/python3.7/dist-packages (from num2words->textattack[tensorflow]) (0.6.2)
Requirement already satisfied: click in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers>=3.3.0->textattack[tensorflow]) (7.1.2)
Installing collected packages: tensorflow-text
Successfully installed tensorflow-text-2.6.0
[8]:
from allennlp.predictors import Predictor
import allennlp_models.classification

import textattack


class AllenNLPModel(textattack.models.wrappers.ModelWrapper):
    def __init__(self):
        self.predictor = Predictor.from_path(
            "https://storage.googleapis.com/allennlp-public-models/basic_stanford_sentiment_treebank-2020.06.09.tar.gz"
        )
        self.model = self.predictor._model
        self.tokenizer = self.predictor._dataset_reader._tokenizer

    def __call__(self, text_input_list):
        outputs = []
        for text_input in text_input_list:
            outputs.append(self.predictor.predict(sentence=text_input))
        # For each output, outputs['logits'] contains the logits where
        # index 0 corresponds to the positive and index 1 corresponds
        # to the negative score. We reverse the outputs (by reverse slicing,
        # [::-1]) so that negative comes first and positive comes second.
        return [output["logits"][::-1] for output in outputs]


model_wrapper = AllenNLPModel()
textattack: Updating TextAttack package dependencies.
textattack: Downloading NLTK required packages.
[nltk_data] Downloading package averaged_perceptron_tagger to
[nltk_data]     /root/nltk_data...
[nltk_data]   Unzipping taggers/averaged_perceptron_tagger.zip.
[nltk_data] Downloading package stopwords to /root/nltk_data...
[nltk_data]   Unzipping corpora/stopwords.zip.
[nltk_data] Downloading package omw to /root/nltk_data...
[nltk_data]   Unzipping corpora/omw.zip.
[nltk_data] Downloading package universal_tagset to /root/nltk_data...
[nltk_data]   Unzipping taggers/universal_tagset.zip.
[nltk_data] Downloading package wordnet to /root/nltk_data...
[nltk_data]   Unzipping corpora/wordnet.zip.
[nltk_data] Downloading package punkt to /root/nltk_data...
[nltk_data]   Unzipping tokenizers/punkt.zip.
textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings/paragramcf.
100%|██████████| 481M/481M [00:14<00:00, 33.6MB/s]
textattack: Unzipping file /root/.cache/textattack/tmp7xfefu5f.zip to /root/.cache/textattack/word_embeddings/paragramcf.
textattack: Successfully saved word_embeddings/paragramcf to cache.
Plugin allennlp_models could not be loaded: No module named 'nltk.translate.meteor_score'
downloading: 100%|##########| 37033341/37033341 [00:01<00:00, 27735821.99B/s]
[9]:
from textattack.datasets import HuggingFaceDataset
from textattack.attack_recipes import TextBuggerLi2018
from textattack.attacker import Attacker


dataset = HuggingFaceDataset("glue", "sst2", "train")
attack = TextBuggerLi2018.build(model_wrapper)

attacker = Attacker(attack, dataset)
attacker.attack_dataset()
Downloading and preparing dataset glue/sst2 (download: 7.09 MiB, generated: 4.81 MiB, post-processed: Unknown size, total: 11.90 MiB) to /root/.cache/huggingface/datasets/glue/sst2/1.0.0/dacbe3125aa31d7f70367a07a8a9e72a5a0bfeb5fc42e75c9db75b96da6053ad...
Dataset glue downloaded and prepared to /root/.cache/huggingface/datasets/glue/sst2/1.0.0/dacbe3125aa31d7f70367a07a8a9e72a5a0bfeb5fc42e75c9db75b96da6053ad. Subsequent calls will reuse this data.
textattack: Loading datasets dataset glue, subset sst2, split train.
textattack: Unknown if model of class <class 'allennlp.models.basic_classifier.BasicClassifier'> compatible with goal function <class 'textattack.goal_functions.classification.untargeted_classification.UntargetedClassification'>.
Attack(
  (search_method): GreedyWordSwapWIR(
    (wir_method):  delete
  )
  (goal_function):  UntargetedClassification
  (transformation):  CompositeTransformation(
    (0): WordSwapRandomCharacterInsertion(
        (random_one):  True
      )
    (1): WordSwapRandomCharacterDeletion(
        (random_one):  True
      )
    (2): WordSwapNeighboringCharacterSwap(
        (random_one):  True
      )
    (3): WordSwapHomoglyphSwap
    (4): WordSwapEmbedding(
        (max_candidates):  5
        (embedding):  WordEmbedding
      )
    )
  (constraints):
    (0): UniversalSentenceEncoder(
        (metric):  angular
        (threshold):  0.8
        (window_size):  inf
        (skip_text_shorter_than_window):  False
        (compare_against_original):  True
      )
    (1): RepeatModification
    (2): StopwordModification
  (is_black_box):  True
)

more-to-come:

class:

stderr

0%| | 0/10 [00:00&lt;?, ?it/s]Using /tmp/tfhub_modules to cache modules.

Downloading TF-Hub Module ‘https://tfhub.dev/google/universal-sentence-encoder/4’. Downloaded https://tfhub.dev/google/universal-sentence-encoder/4, Total size: 987.47MB Downloaded TF-Hub Module ‘https://tfhub.dev/google/universal-sentence-encoder/4’. [Succeeded / Failed / Skipped / Total] 1 / 1 / 0 / 2: 20%|██ | 2/10 [01:27&lt;05:48, 43.58s/it] </pre>

0%| | 0/10 [00:00<?, ?it/s]Using /tmp/tfhub_modules to cache modules.

Downloading TF-Hub Module ‘https://tfhub.dev/google/universal-sentence-encoder/4’. Downloaded https://tfhub.dev/google/universal-sentence-encoder/4, Total size: 987.47MB Downloaded TF-Hub Module ‘https://tfhub.dev/google/universal-sentence-encoder/4’. [Succeeded / Failed / Skipped / Total] 1 / 1 / 0 / 2: 20%|██ | 2/10 [01:27<05:48, 43.58s/it] end{sphinxVerbatim}

0%| | 0/10 [00:00<?, ?it/s]Using /tmp/tfhub_modules to cache modules.

Downloading TF-Hub Module ‘https://tfhub.dev/google/universal-sentence-encoder/4’. Downloaded https://tfhub.dev/google/universal-sentence-encoder/4, Total size: 987.47MB Downloaded TF-Hub Module ‘https://tfhub.dev/google/universal-sentence-encoder/4’. [Succeeded / Failed / Skipped / Total] 1 / 1 / 0 / 2: 20%|██ | 2/10 [01:27<05:48, 43.58s/it]

--------------------------------------------- Result 1 ---------------------------------------------
[[Negative (95%)]] --> [[Positive (93%)]]

[[hide]] new secretions from the parental units

[[concealing]] new secretions from the parental units


--------------------------------------------- Result 2 ---------------------------------------------
[[Negative (96%)]] --> [[[FAILED]]]

contains no wit , only labored gags


[Succeeded / Failed / Skipped / Total] 1 / 2 / 1 / 4:  40%|████      | 4/10 [01:27<02:11, 21.91s/it]
--------------------------------------------- Result 3 ---------------------------------------------
[[Positive (100%)]] --> [[[FAILED]]]

that loves its characters and communicates something rather beautiful about human nature


--------------------------------------------- Result 4 ---------------------------------------------
[[Positive (82%)]] --> [[[SKIPPED]]]

remains utterly satisfied to remain the same throughout


[Succeeded / Failed / Skipped / Total] 1 / 3 / 1 / 5:  50%|█████     | 5/10 [01:28<01:28, 17.62s/it]
--------------------------------------------- Result 5 ---------------------------------------------
[[Negative (98%)]] --> [[[FAILED]]]

on the worst revenge-of-the-nerds clichés the filmmakers could dredge up


[Succeeded / Failed / Skipped / Total] 1 / 4 / 1 / 6: 60%|██████ | 6/10 [01:28&lt;00:59, 14.75s/it]

</pre>

[Succeeded / Failed / Skipped / Total] 1 / 4 / 1 / 6: 60%|██████ | 6/10 [01:28<00:59, 14.75s/it]

end{sphinxVerbatim}

[Succeeded / Failed / Skipped / Total] 1 / 4 / 1 / 6: 60%|██████ | 6/10 [01:28<00:59, 14.75s/it]

--------------------------------------------- Result 6 ---------------------------------------------
[[Negative (99%)]] --> [[[FAILED]]]

that 's far too tragic to merit such superficial treatment


[Succeeded / Failed / Skipped / Total] 2 / 5 / 1 / 8:  80%|████████  | 8/10 [01:29<00:22, 11.24s/it]
--------------------------------------------- Result 7 ---------------------------------------------
[[Positive (98%)]] --> [[Negative (62%)]]

[[demonstrates]] that the [[director]] of such [[hollywood]] blockbusters as patriot games can still [[turn]] out a [[small]] , personal [[film]] with an emotional [[wallop]] .

[[shows]] that the [[directors]] of such [[tinseltown]] blockbusters as patriot games can still [[turning]] out a [[tiny]] , personal [[movies]] with an emotional [[batting]] .


--------------------------------------------- Result 8 ---------------------------------------------
[[Positive (90%)]] --> [[[FAILED]]]

of saucy


[Succeeded / Failed / Skipped / Total] 2 / 6 / 1 / 9:  90%|█████████ | 9/10 [01:30<00:10, 10.03s/it]
--------------------------------------------- Result 9 ---------------------------------------------
[[Negative (99%)]] --> [[[FAILED]]]

a depressed fifteen-year-old 's suicidal poetry


[Succeeded / Failed / Skipped / Total] 3 / 6 / 1 / 10: 100%|██████████| 10/10 [01:30<00:00,  9.05s/it]
--------------------------------------------- Result 10 ---------------------------------------------
[[Positive (79%)]] --> [[Negative (65%)]]

are more [[deeply]] thought through than in most ` right-thinking ' films

are more [[seriously]] thought through than in most ` right-thinking ' films



+-------------------------------+--------+
| Attack Results                |        |
+-------------------------------+--------+
| Number of successful attacks: | 3      |
| Number of failed attacks:     | 6      |
| Number of skipped attacks:    | 1      |
| Original accuracy:            | 90.0%  |
| Accuracy under attack:        | 60.0%  |
| Attack success rate:          | 33.33% |
| Average perturbed word %:     | 17.94% |
| Average num. words per input: | 9.5    |
| Avg num queries:              | 35.11  |
+-------------------------------+--------+

[9]:
[<textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7fab125a1810>,
 <textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7fab19420c10>,
 <textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7fab0bb0f4d0>,
 <textattack.attack_results.skipped_attack_result.SkippedAttackResult at 0x7fab0f3cd610>,
 <textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7fab192d4790>,
 <textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7fab0b7ba190>,
 <textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7fab0bd2ee90>,
 <textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7fab09731590>,
 <textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7fab097e9610>,
 <textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7fab13265450>]