TextAttack & AllenNLP
This is an example of testing adversarial attacks from TextAttack on pretrained models provided by AllenNLP.
In a few lines of code, we load a sentiment analysis model trained on the Stanford Sentiment Treebank and configure it with a TextAttack model wrapper. Then, we initialize the TextBugger attack and run the attack on a few samples from the SST-2 train set.
For more information on AllenNLP pre-trained models: https://docs.allennlp.org/models/main/
For more information about the TextBugger attack: https://arxiv.org/abs/1812.05271
[4]:
!pip install allennlp allennlp_models > /dev/null
[7]:
!pip3 install textattack[tensorflow]
Requirement already satisfied: textattack[tensorflow] in /usr/local/lib/python3.7/dist-packages (0.3.3)
Requirement already satisfied: language-tool-python in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (2.6.1)
Requirement already satisfied: terminaltables in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (3.1.0)
Requirement already satisfied: filelock in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (3.0.12)
Requirement already satisfied: numpy>=1.19.2 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (1.19.5)
Requirement already satisfied: scipy>=1.4.1 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (1.4.1)
Requirement already satisfied: word2number in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (1.1)
Requirement already satisfied: lru-dict in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (1.1.7)
Requirement already satisfied: nltk in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (3.2.5)
Requirement already satisfied: pandas>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (1.1.5)
Requirement already satisfied: num2words in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (0.5.10)
Requirement already satisfied: PySocks!=1.5.7,>=1.5.6 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (1.7.1)
Requirement already satisfied: transformers>=3.3.0 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (4.5.1)
Requirement already satisfied: lemminflect in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (0.2.2)
Requirement already satisfied: more-itertools in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (8.8.0)
Requirement already satisfied: editdistance in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (0.5.3)
Requirement already satisfied: bert-score>=0.3.5 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (0.3.10)
Requirement already satisfied: torch!=1.8,>=1.7.0 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (1.9.0+cu111)
Requirement already satisfied: tqdm<4.50.0,>=4.27 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (4.49.0)
Requirement already satisfied: flair in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (0.9)
Requirement already satisfied: datasets in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (1.11.0)
Requirement already satisfied: tensorflow>=2 in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (2.6.0)
Requirement already satisfied: tensorflow-hub in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (0.12.0)
Requirement already satisfied: tensorboardX in /usr/local/lib/python3.7/dist-packages (from textattack[tensorflow]) (2.4)
Collecting tensorflow-text>=2
Downloading tensorflow_text-2.6.0-cp37-cp37m-manylinux1_x86_64.whl (4.4 MB)
|████████████████████████████████| 4.4 MB 5.4 MB/s
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from bert-score>=0.3.5->textattack[tensorflow]) (2.26.0)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from bert-score>=0.3.5->textattack[tensorflow]) (3.2.2)
Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.7/dist-packages (from bert-score>=0.3.5->textattack[tensorflow]) (21.0)
Requirement already satisfied: pyparsing>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging>=20.9->bert-score>=0.3.5->textattack[tensorflow]) (2.4.7)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.0.1->textattack[tensorflow]) (2.8.2)
Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.0.1->textattack[tensorflow]) (2018.9)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.7.3->pandas>=1.0.1->textattack[tensorflow]) (1.15.0)
Requirement already satisfied: grpcio<2.0,>=1.37.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (1.41.0)
Requirement already satisfied: tensorflow-estimator~=2.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (2.6.0)
Requirement already satisfied: protobuf>=3.9.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (3.17.3)
Requirement already satisfied: google-pasta~=0.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (0.2.0)
Requirement already satisfied: gast==0.4.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (0.4.0)
Requirement already satisfied: flatbuffers~=1.12.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (1.12)
Requirement already satisfied: termcolor~=1.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (1.1.0)
Requirement already satisfied: tensorboard~=2.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (2.6.0)
Requirement already satisfied: keras~=2.6 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (2.6.0)
Requirement already satisfied: clang~=5.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (5.0)
Requirement already satisfied: absl-py~=0.10 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (0.12.0)
Requirement already satisfied: wrapt~=1.12.1 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (1.12.1)
Requirement already satisfied: astunparse~=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (1.6.3)
Requirement already satisfied: wheel~=0.35 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (0.37.0)
Requirement already satisfied: typing-extensions~=3.7.4 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (3.7.4.3)
Requirement already satisfied: opt-einsum~=3.3.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (3.3.0)
Requirement already satisfied: keras-preprocessing~=1.1.2 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (1.1.2)
Requirement already satisfied: h5py~=3.1.0 in /usr/local/lib/python3.7/dist-packages (from tensorflow>=2->textattack[tensorflow]) (3.1.0)
Requirement already satisfied: cached-property in /usr/local/lib/python3.7/dist-packages (from h5py~=3.1.0->tensorflow>=2->textattack[tensorflow]) (1.5.2)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (0.6.1)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (0.4.6)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (3.3.4)
Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (57.4.0)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (1.8.0)
Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (1.0.1)
Requirement already satisfied: google-auth<2,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (1.35.0)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (4.7.2)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (4.2.4)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (1.3.0)
Requirement already satisfied: importlib-metadata in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (3.10.1)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (0.4.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.7/dist-packages (from requests->bert-score>=0.3.5->textattack[tensorflow]) (2.0.6)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->bert-score>=0.3.5->textattack[tensorflow]) (1.25.11)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->bert-score>=0.3.5->textattack[tensorflow]) (2.10)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->bert-score>=0.3.5->textattack[tensorflow]) (2021.5.30)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (3.1.1)
Requirement already satisfied: sacremoses in /usr/local/lib/python3.7/dist-packages (from transformers>=3.3.0->textattack[tensorflow]) (0.0.46)
Requirement already satisfied: regex!=2019.12.17 in /usr/local/lib/python3.7/dist-packages (from transformers>=3.3.0->textattack[tensorflow]) (2019.12.20)
Requirement already satisfied: tokenizers<0.11,>=0.10.1 in /usr/local/lib/python3.7/dist-packages (from transformers>=3.3.0->textattack[tensorflow]) (0.10.3)
Requirement already satisfied: xxhash in /usr/local/lib/python3.7/dist-packages (from datasets->textattack[tensorflow]) (2.0.2)
Requirement already satisfied: fsspec>=2021.05.0 in /usr/local/lib/python3.7/dist-packages (from datasets->textattack[tensorflow]) (2021.10.1)
Requirement already satisfied: multiprocess in /usr/local/lib/python3.7/dist-packages (from datasets->textattack[tensorflow]) (0.70.12.2)
Requirement already satisfied: huggingface-hub<0.1.0 in /usr/local/lib/python3.7/dist-packages (from datasets->textattack[tensorflow]) (0.0.19)
Requirement already satisfied: dill in /usr/local/lib/python3.7/dist-packages (from datasets->textattack[tensorflow]) (0.3.4)
Requirement already satisfied: pyarrow!=4.0.0,>=1.0.0 in /usr/local/lib/python3.7/dist-packages (from datasets->textattack[tensorflow]) (3.0.0)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from huggingface-hub<0.1.0->datasets->textattack[tensorflow]) (3.13)
Requirement already satisfied: segtok>=1.5.7 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (1.5.10)
Requirement already satisfied: hyperopt>=0.1.1 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (0.1.2)
Requirement already satisfied: langdetect in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (1.0.9)
Requirement already satisfied: konoha<5.0.0,>=4.0.0 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (4.6.5)
Requirement already satisfied: bpemb>=0.3.2 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (0.3.3)
Requirement already satisfied: gdown==3.12.2 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (3.12.2)
Requirement already satisfied: ftfy in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (6.0.3)
Requirement already satisfied: conllu>=4.0 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (4.4.1)
Requirement already satisfied: scikit-learn>=0.21.3 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (0.22.2.post1)
Requirement already satisfied: sqlitedict>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (1.7.0)
Requirement already satisfied: mpld3==0.3 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (0.3)
Requirement already satisfied: lxml in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (4.2.6)
Requirement already satisfied: janome in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (0.4.1)
Requirement already satisfied: gensim<=3.8.3,>=3.4.0 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (3.6.0)
Requirement already satisfied: sentencepiece==0.1.95 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (0.1.95)
Requirement already satisfied: tabulate in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (0.8.9)
Requirement already satisfied: wikipedia-api in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (0.5.4)
Requirement already satisfied: deprecated>=1.2.4 in /usr/local/lib/python3.7/dist-packages (from flair->textattack[tensorflow]) (1.2.13)
Requirement already satisfied: smart-open>=1.2.1 in /usr/local/lib/python3.7/dist-packages (from gensim<=3.8.3,>=3.4.0->flair->textattack[tensorflow]) (5.2.1)
Requirement already satisfied: networkx in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.1.1->flair->textattack[tensorflow]) (2.6.3)
Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.1.1->flair->textattack[tensorflow]) (0.16.0)
Requirement already satisfied: pymongo in /usr/local/lib/python3.7/dist-packages (from hyperopt>=0.1.1->flair->textattack[tensorflow]) (3.12.0)
Requirement already satisfied: overrides<4.0.0,>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from konoha<5.0.0,>=4.0.0->flair->textattack[tensorflow]) (3.1.0)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata->markdown>=2.6.8->tensorboard~=2.6->tensorflow>=2->textattack[tensorflow]) (3.6.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->bert-score>=0.3.5->textattack[tensorflow]) (1.3.2)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->bert-score>=0.3.5->textattack[tensorflow]) (0.10.0)
Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn>=0.21.3->flair->textattack[tensorflow]) (1.0.1)
Requirement already satisfied: wcwidth in /usr/local/lib/python3.7/dist-packages (from ftfy->flair->textattack[tensorflow]) (0.2.5)
Requirement already satisfied: docopt>=0.6.2 in /usr/local/lib/python3.7/dist-packages (from num2words->textattack[tensorflow]) (0.6.2)
Requirement already satisfied: click in /usr/local/lib/python3.7/dist-packages (from sacremoses->transformers>=3.3.0->textattack[tensorflow]) (7.1.2)
Installing collected packages: tensorflow-text
Successfully installed tensorflow-text-2.6.0
[8]:
from allennlp.predictors import Predictor
import allennlp_models.classification
import textattack
class AllenNLPModel(textattack.models.wrappers.ModelWrapper):
def __init__(self):
self.predictor = Predictor.from_path(
"https://storage.googleapis.com/allennlp-public-models/basic_stanford_sentiment_treebank-2020.06.09.tar.gz"
)
self.model = self.predictor._model
self.tokenizer = self.predictor._dataset_reader._tokenizer
def __call__(self, text_input_list):
outputs = []
for text_input in text_input_list:
outputs.append(self.predictor.predict(sentence=text_input))
# For each output, outputs['logits'] contains the logits where
# index 0 corresponds to the positive and index 1 corresponds
# to the negative score. We reverse the outputs (by reverse slicing,
# [::-1]) so that negative comes first and positive comes second.
return [output["logits"][::-1] for output in outputs]
model_wrapper = AllenNLPModel()
textattack: Updating TextAttack package dependencies.
textattack: Downloading NLTK required packages.
[nltk_data] Downloading package averaged_perceptron_tagger to
[nltk_data] /root/nltk_data...
[nltk_data] Unzipping taggers/averaged_perceptron_tagger.zip.
[nltk_data] Downloading package stopwords to /root/nltk_data...
[nltk_data] Unzipping corpora/stopwords.zip.
[nltk_data] Downloading package omw to /root/nltk_data...
[nltk_data] Unzipping corpora/omw.zip.
[nltk_data] Downloading package universal_tagset to /root/nltk_data...
[nltk_data] Unzipping taggers/universal_tagset.zip.
[nltk_data] Downloading package wordnet to /root/nltk_data...
[nltk_data] Unzipping corpora/wordnet.zip.
[nltk_data] Downloading package punkt to /root/nltk_data...
[nltk_data] Unzipping tokenizers/punkt.zip.
textattack: Downloading https://textattack.s3.amazonaws.com/word_embeddings/paragramcf.
100%|██████████| 481M/481M [00:14<00:00, 33.6MB/s]
textattack: Unzipping file /root/.cache/textattack/tmp7xfefu5f.zip to /root/.cache/textattack/word_embeddings/paragramcf.
textattack: Successfully saved word_embeddings/paragramcf to cache.
Plugin allennlp_models could not be loaded: No module named 'nltk.translate.meteor_score'
downloading: 100%|##########| 37033341/37033341 [00:01<00:00, 27735821.99B/s]
[9]:
from textattack.datasets import HuggingFaceDataset
from textattack.attack_recipes import TextBuggerLi2018
from textattack.attacker import Attacker
dataset = HuggingFaceDataset("glue", "sst2", "train")
attack = TextBuggerLi2018.build(model_wrapper)
attacker = Attacker(attack, dataset)
attacker.attack_dataset()
Downloading and preparing dataset glue/sst2 (download: 7.09 MiB, generated: 4.81 MiB, post-processed: Unknown size, total: 11.90 MiB) to /root/.cache/huggingface/datasets/glue/sst2/1.0.0/dacbe3125aa31d7f70367a07a8a9e72a5a0bfeb5fc42e75c9db75b96da6053ad...
Dataset glue downloaded and prepared to /root/.cache/huggingface/datasets/glue/sst2/1.0.0/dacbe3125aa31d7f70367a07a8a9e72a5a0bfeb5fc42e75c9db75b96da6053ad. Subsequent calls will reuse this data.
textattack: Loading datasets dataset glue, subset sst2, split train.
textattack: Unknown if model of class <class 'allennlp.models.basic_classifier.BasicClassifier'> compatible with goal function <class 'textattack.goal_functions.classification.untargeted_classification.UntargetedClassification'>.
Attack(
(search_method): GreedyWordSwapWIR(
(wir_method): delete
)
(goal_function): UntargetedClassification
(transformation): CompositeTransformation(
(0): WordSwapRandomCharacterInsertion(
(random_one): True
)
(1): WordSwapRandomCharacterDeletion(
(random_one): True
)
(2): WordSwapNeighboringCharacterSwap(
(random_one): True
)
(3): WordSwapHomoglyphSwap
(4): WordSwapEmbedding(
(max_candidates): 5
(embedding): WordEmbedding
)
)
(constraints):
(0): UniversalSentenceEncoder(
(metric): angular
(threshold): 0.8
(window_size): inf
(skip_text_shorter_than_window): False
(compare_against_original): True
)
(1): RepeatModification
(2): StopwordModification
(is_black_box): True
)
0%| | 0/10 [00:00<?, ?it/s]Using /tmp/tfhub_modules to cache modules.
Downloading TF-Hub Module 'https://tfhub.dev/google/universal-sentence-encoder/4'.
Downloaded https://tfhub.dev/google/universal-sentence-encoder/4, Total size: 987.47MB
Downloaded TF-Hub Module 'https://tfhub.dev/google/universal-sentence-encoder/4'.
[Succeeded / Failed / Skipped / Total] 1 / 1 / 0 / 2: 20%|██ | 2/10 [01:27<05:48, 43.58s/it]
--------------------------------------------- Result 1 ---------------------------------------------
[[Negative (95%)]] --> [[Positive (93%)]]
[[hide]] new secretions from the parental units
[[concealing]] new secretions from the parental units
--------------------------------------------- Result 2 ---------------------------------------------
[[Negative (96%)]] --> [[[FAILED]]]
contains no wit , only labored gags
[Succeeded / Failed / Skipped / Total] 1 / 2 / 1 / 4: 40%|████ | 4/10 [01:27<02:11, 21.91s/it]
--------------------------------------------- Result 3 ---------------------------------------------
[[Positive (100%)]] --> [[[FAILED]]]
that loves its characters and communicates something rather beautiful about human nature
--------------------------------------------- Result 4 ---------------------------------------------
[[Positive (82%)]] --> [[[SKIPPED]]]
remains utterly satisfied to remain the same throughout
[Succeeded / Failed / Skipped / Total] 1 / 3 / 1 / 5: 50%|█████ | 5/10 [01:28<01:28, 17.62s/it]
--------------------------------------------- Result 5 ---------------------------------------------
[[Negative (98%)]] --> [[[FAILED]]]
on the worst revenge-of-the-nerds clichés the filmmakers could dredge up
[Succeeded / Failed / Skipped / Total] 1 / 4 / 1 / 6: 60%|██████ | 6/10 [01:28<00:59, 14.75s/it]
--------------------------------------------- Result 6 ---------------------------------------------
[[Negative (99%)]] --> [[[FAILED]]]
that 's far too tragic to merit such superficial treatment
[Succeeded / Failed / Skipped / Total] 2 / 5 / 1 / 8: 80%|████████ | 8/10 [01:29<00:22, 11.24s/it]
--------------------------------------------- Result 7 ---------------------------------------------
[[Positive (98%)]] --> [[Negative (62%)]]
[[demonstrates]] that the [[director]] of such [[hollywood]] blockbusters as patriot games can still [[turn]] out a [[small]] , personal [[film]] with an emotional [[wallop]] .
[[shows]] that the [[directors]] of such [[tinseltown]] blockbusters as patriot games can still [[turning]] out a [[tiny]] , personal [[movies]] with an emotional [[batting]] .
--------------------------------------------- Result 8 ---------------------------------------------
[[Positive (90%)]] --> [[[FAILED]]]
of saucy
[Succeeded / Failed / Skipped / Total] 2 / 6 / 1 / 9: 90%|█████████ | 9/10 [01:30<00:10, 10.03s/it]
--------------------------------------------- Result 9 ---------------------------------------------
[[Negative (99%)]] --> [[[FAILED]]]
a depressed fifteen-year-old 's suicidal poetry
[Succeeded / Failed / Skipped / Total] 3 / 6 / 1 / 10: 100%|██████████| 10/10 [01:30<00:00, 9.05s/it]
--------------------------------------------- Result 10 ---------------------------------------------
[[Positive (79%)]] --> [[Negative (65%)]]
are more [[deeply]] thought through than in most ` right-thinking ' films
are more [[seriously]] thought through than in most ` right-thinking ' films
+-------------------------------+--------+
| Attack Results | |
+-------------------------------+--------+
| Number of successful attacks: | 3 |
| Number of failed attacks: | 6 |
| Number of skipped attacks: | 1 |
| Original accuracy: | 90.0% |
| Accuracy under attack: | 60.0% |
| Attack success rate: | 33.33% |
| Average perturbed word %: | 17.94% |
| Average num. words per input: | 9.5 |
| Avg num queries: | 35.11 |
+-------------------------------+--------+
[9]:
[<textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7fab125a1810>,
<textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7fab19420c10>,
<textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7fab0bb0f4d0>,
<textattack.attack_results.skipped_attack_result.SkippedAttackResult at 0x7fab0f3cd610>,
<textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7fab192d4790>,
<textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7fab0b7ba190>,
<textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7fab0bd2ee90>,
<textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7fab09731590>,
<textattack.attack_results.failed_attack_result.FailedAttackResult at 0x7fab097e9610>,
<textattack.attack_results.successful_attack_result.SuccessfulAttackResult at 0x7fab13265450>]