This repository contains the data and code for paper:

Extractive Opinion Summarization in Quantized Transformer Spaces,
Stefanos Angelidis, Reinald Kim Amplayo, Yoshihiko Suhara, Xiaolan Wang, Mirella Lapata,
To appear: In Transactions of the Association for Computational Linguistics (TACL).
arXiv

The paper introduces SPACE, a large-scale opinion summarization benchmark for the evaluation of unsupervised summarizers. SPACE is built on TripAdvisor hotel reviews and includes a training set of approximately 1.1 million reviews for over 11 thousand hotels.

For evaluation, we created a collection of human-written, abstractive opinion summaries for 50 hotels, including high-level general summaries and aspect summaries for six popular aspects: building, cleanliness, food, location, rooms, and service. Every summary is based on 100 input reviews, an order of magnitude increase compared to existing corpora. In total, SPACE contains 1,050 gold standard summaries. You can view the full instructions for out multi-stage annotation procedure here.

Download the corpus from this google drive url.

Here is an example drawn from SPACE of one general and 3 aspect-specific summaries for the same hotel:

The Quantized Transformer (QT) is inspired by Vector-Quantized Variational Autoencoders, which we repurpose for popularity-driven summarization. It uses a clustering interpretation of the quantized space and a novel extraction algorithm to discover popular opinions among hundreds of reviews, a significant step towards opinion summarization of practical scope. In addition, QT enables controllable summarization without further training, by utilizing properties of the quantized space to extract aspect-specific summaries.

• Python version: python3.6

• Dependencies: Use the requirements.txt file and conda/pip to install all necessary dependencies. E.g., for pip:

    pip install -U pip
    pip install -U setuptools
    pip install -r requirements.txt 
  

• ROUGE: To ensure replicability and future research, we used the original ROUGE perl implementation and the pyrouge wrapper. Please follow the instructions in this guide to setup ROUGE and pyrouge correctly. Make sure to you have activated your conda/virtualenv environment when installing pyrouge

• SPACE training set: The training set is not included in this repo. Download SPACE via the above google drive link (405MB) and copy the file space_train.json into the ./data/json/ directory.

To train QT on a subset of the training set using a GPU, go to the ./src directory and run the following:

python3 train.py --max_num_entities 500 --run_id run1 --gpu 0

This will train a QT model with default hyperparameters (for general summarization), store tensorboard logs under ./logs and save a model snapshot after every epoch under ./models (filename: run1_<epoch>_model.pt). Our model achieves high summarization performance, even when trained on reviews from just 500 entities, as shown here.

For explanations of the available parameters for training the model, please see train.py.

To perform general opinion summarization with a trained QT model, go to the ./src directory and run the following:

python3 extract.py --model ../models/run1_20_model.pt --sample_sentences --run_id general_run1 --gpu 0

This will store the summaries under ./outputs/general_run1 and also the output of ROUGE evaluation in ./outputs/eval_general_run1.json. The --sample_sentences flag enables 2-step sampling.

For aspect opinion summarization, run:

python3 aspect_extract.py --model ../models/run1_20_model.pt --sample_sentences --run_id aspects_run1 --gpu 0

Outputs stored similarly to the general opinion summarization example. For explanations of the available parameters for summarizing with the model, please see extract.py and aspect_extract.py.

Check this page for details on the hyperparameters used in our paper's main experiments.

If you want to use QT with a summarization corpus other than SPACE, please follow the instruction on this page.