InteractiveLREC-COLING 2024

CLAP: the same AMR parser, five times faster

Abelardo Carlos Martínez Lorenzo, Roberto Navigli · Sapienza NLP

AMR parsers had been getting bigger and slower. We asked a simpler question: what if the graph the model writes were shorter? A compact way of writing graphs, plus a leaner parser, cut training and inference time by about 80% with no loss in accuracy.

Start the story ACL Anthology PDF Code

tokens for the example graph: Penman versus Compact: 83 → 28tokens for the example graph: Penman versus Compact
training time versus SPRING (BART-large): −82%training time versus SPRING (BART-large)
inference time versus SPRING: −78%inference time versus SPRING
SMATCH on AMR 3.0, versus 83.0 for SPRING: 83.1SMATCH on AMR 3.0, versus 83.0 for SPRING

Every token is a decoding step

Modern AMR parsers, starting with SPRING, are sequence-to-sequence models: they read a sentence and write the graph as text, one token at a time. The longer that text, the slower both training and inference.

Meanwhile the field kept adding weight: structural pre-training, extra tasks, ancestor-aware decoding, graph adapters, ensembles. Accurate, but increasingly out of reach for researchers with limited compute.

Our example, from the paper: In 1891, Pablo Picasso moved from his hometown Málaga to A Coruña.

The AMR graph Figure 1

“In 1891, Pablo Picasso moved from his hometown Málaga to A Coruña”

Writing the graph with fewer tokens

The same graph can be written in several ways. Penman is the official format; SPRING's depth-first (DFS) format made it model-friendly. The Compact linearisation keeps only what is needed to rebuild the graph:

only the closing parenthesis, when returning to a parent node;
no variables: nodes are written as their concept, with an index for repeats (city#2);
collapsed names (:name Pablo Picasso) and de-reified structures (have-location-91 → :location);
no Wikipedia links, added later by an entity linker.

Three ways to write the same graph Figures 1–3

(z0 / move-01
   :ARG0 (z1 / person
      :name (z2 / name
         :op1 "Pablo"
         :op2 "Picasso"))
   :ARG1 (z3 / hometown
      :poss z1
      :ARG0-of (z4 / have-location-91
         :ARG1 (z5 / city
            :name (z6 / name
               :op1 "Malaga"))))
   :ARG2 (z7 / city
      :name (z8 / name
         :op1 "A"
         :op2 "Coruña"))
   :time (z9 / date-entity
      :year 1891))

Penman

DFS (SPRING)

Compact (CLAP)

Figure 1

The format AMR 3.0 ships in. Slashes, quotation marks and variables like z0 are redundant or awkward for a tokenizer.

highlight what gets removed next

Token counts as reported in Section 2 of the paper. Red marks what the next linearisation removes or rewrites.

An adaptable parser

SPRING adds more than 3,000 special tokens to its vocabulary: one for every pointer, relation and PropBank frame. That grows the model, shrinks the batches, and ties SPRING to one tokenizer and a set of hand-written repair rules.

CLAP adds only the 25 AMR relations, index tokens (#1, #2), -of for inverse relations and frame suffixes (-01, -02). Everything else goes through the model's original tokenizer, so any seq2seq model can be plugged in: BART, T5, mBART, LongT5.

Decoding reads triplets straight from the output instead of rebuilding Penman with rules, and training uses Adafactor with batches sized by token count.

T5 vs BART tokenizers (Table 1) training tokens

PenmanT5 generates 29% more

7.15M

BART

5.56M

DFST5 generates 18% more

5.19M

BART

4.38M

CompactT5 generates 2% more

2.53M

BART

2.48M

T5 uses a Unigram tokenizer with a 32K vocabulary; BART uses BPE with 50K, so it splits text into fewer pieces. Compact linearisation nearly erases the difference.

Results

Training time

−82%

SPRING

45h

CLAP

Inference time

−78%

SPRING

134s

CLAP

29s

Tokens to generate (test set)

−42%

SPRING

180,086

CLAP

103,732

Head to head on BART-large, CLAP with the Compact linearisation trains in 8 hours instead of 45 and parses the test set in 29 seconds instead of 134, at the same SMATCH. Across all five models, Compact trains about 60% faster than Penman and 50% faster than DFS, with SMATCH within 0.1 of DFS. Explore every configuration:

Tokens, time and accuracy (Table 1) AMR 3.0

Training tokens

CLAP · PM5.56M

SPRING (DFS)4.04M

CLAP · DFS4.38M

CLAP · CMP2.48M

Training time (50 epochs)

CLAP · PM17 h

SPRING (DFS)45 h

CLAP · DFS14 h

CLAP · CMP8 h

Inference on the test set

CLAP · PM147 s

SPRING (DFS)134 s

CLAP · DFS67 s

CLAP · CMP29 s

SMATCH

CLAP · PM81.5

SPRING (DFS)83.0

CLAP · DFS83.0

CLAP · CMP83.1

BART-large: 400M parameters, vocabulary 50,459. Bars start at zero; SMATCH uses a zoomed axis (80–85).

CLAP went on to power other work: it is the parser behind the multilingual AMR layer of MOSAICo and the parsing experiments of MSL.

Scope

Evaluated on AMR 3.0 with BART and Flan-T5, from 60M to 770M parameters.
Compared against SPRING, since later parsers build on it and mostly change the training strategy.
No entity-linking post-processing was applied, to keep the comparison fair.

Cite

@inproceedings{martinez-lorenzo-navigli-2024-efficient,
    title = "Efficient {AMR} Parsing with {CLAP}: Compact Linearization with an Adaptable Parser",
    author = "Martinez Lorenzo, Abelardo Carlos  and
      Navigli, Roberto",
    booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)",
    month = may,
    year = "2024",
    address = "Torino, Italia",
    publisher = "ELRA and ICCL",
    url = "https://aclanthology.org/2024.lrec-main.495/",
    pages = "5578--5584"
}

All publications

Linearisations from Figures 1–3; numbers from Table 1.