# Welcome to FewRel

a Few-shot Relation classification dataset

We have now moved to [Codalab competition].

FewRel is a Few-shot Relation classification dataset, which features 70, 000 natural language sentences expressing 100 relations annotated by crowdworkers.

Baseline models and a series of toolkits are released in this repo:

It is fairly easy for you to get started with FewRel. First, you can download a copy of the dataset by the following links. The dataset is distributed under the CC BY-SA 4.0 license:

Once you have built a model that works to your expectations on the validation set, you can submit it to get official scores on the hidden test set. To preserve the integrity of test results, we do not release the test set labels to the public. Instead, we require you to submit your predicted results so that we can evaluate it. You can submit to the automatic test benchmark:

@inproceedings{han-etal-2018-fewrel,
title = "{F}ew{R}el: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluation",
author = "Han, Xu and Zhu, Hao and Yu, Pengfei and Wang, Ziyun and Yao, Yuan and Liu, Zhiyuan and Sun, Maosong",
booktitle = "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing",
month = oct # "-" # nov,
year = "2018",
publisher = "Association for Computational Linguistics",
url = "https://www.aclweb.org/anthology/D18-1514",
doi = "10.18653/v1/D18-1514",
pages = "4803--4809"
}


# Model 5-way-1-shot 5-way-5-shot 10-way-1-shot 10-way-5-shot

#

Human 92.22 - 85.88 -

1

Matching the Blanks[paper]
(Feb 26, 2019)
93.8697.0689.2094.27

2

Anonymous Pony

(Jan 17, 2020)
91.6196.0186.7891.63

3

GTP
Anonymous
(Aug 15, 2020)
89.4397.0284.4193.84

4

Anonymous Groundhog

(Nov 3, 2019)
90.3194.2885.4890.51

5

PAMN
Anonymous
(Jun 8, 2020)
90.1095.2284.1390.58

6

Anonymous Python

(Nov 3, 2019)
88.3095.9481.1092.67

7

CTEG[paper]
JD AI Research
(Nov 20, 2020)
88.1195.2581.2991.33

8

BERT-PAIR[paper][code]
THUNLP, Tsinghua University
(Nov 3, 2019)
88.3293.2280.6387.02

9

MLMAN[paper][code]
NEL-SLIP, University of Science and Technology of China
(Jan 10, 2019)
82.9892.6673.5987.29

10

PLATO
Primer.ai
(Nov 19, 2019)
82.9391.7674.4185.91

11

Anonymous Rabbit

(Mar 3, 2020)
83.4792.3173.6485.30

12

TDCN
Anonymous
(Jul 23, 2019)
81.9392.0872.1885.98

13

Prototypical Network (LM-Transformer)
LAVIS, University of Applied Sciences RheinMain
(Jan 18, 2019)
81.4092.1172.5186.03

14

LM-ProtoNet (FGM)
Anonymous
(Feb 15, 2019)
76.6089.3165.3182.10

15

bert+maml(CNN)
Yanshan ISE
(Mar 17, 2020)
77.2188.0765.9080.06

16

Prototypical Network (CNN)

74.5288.4062.3880.45

17

anonymous venetian
Anonymous
(Jun 13, 2020)
82.7182.7367.5368.22

18

SNAIL (CNN)

67.2979.4053.2868.33

19

Meta Network (CNN)

64.4680.5753.9669.23

20

GNN (CNN)

66.2381.2846.2764.02

21

kNN (PCNN)

60.2872.4146.1559.11

22

kNN (CNN)

54.6768.7741.2455.87

23

Finetune (CNN)

44.2168.6627.3055.04

24

Finetune (PCNN)

45.6457.8629.6537.43