Shared Tasks

A shared task is an international competition among researchers on a pre-defined dataset. We participate regularly to keep up with the state-of-the-art technologies.

We are proud winners of SemEval 2016 and Evalltalia 2016

Sentiment Analysis

SemEval 2017 – Task 4B

 

Goal: Topic-Based Sentiment Classification

Technology: Distant-Trained Convolutional Neural Network (CNN)

Result: Recall 84.6 (best score: 88.2)

Rank: 4th place out of 23

 

Paper:

TopicThunder at SemEval-2017 Task 4: Sentiment Classification Using a Convolutional Neural Network with Distant Supervision

Simon Müller, Tobias Huonder, Jan Deriu, and Mark Cieliebak

 

EVALITA 2016 – SENTIPOLC – Task 2

 

Goal: Message Level Sentiment Classification for Italian Tweets

Technology: Multi-task trained Convolutional Neural Network (CNN) with weakly labelled distant learning phase

Result: F1-Score 68.28 (was best score)

Rank: 1th place out of 26

 

Paper:

Sentiment Detection using Convolutional Neural Networks with Multi-Task Training and Distant Supervision on Italian Tweets

Jan Deriu and Mark Cieliebak

 

SemEval 2015 – Task 10

 

Goal: Message Level Sentiment Classification for Tweets

Technology: Meta-Classifier on several flipout-regularized Support Vector Machines

Result: F1-Score 62.61 (best score: 64.84)

Rank: 8th place out of 40

 

Paper:

Swiss-Chocolate: Combining Flipout Regularization and Random Forests with Articially Built Subsystems to Boost Text-Classication for Sentiment

Fatih Uzdilli, Martin Jaggi, Dominic Egger, Pascal Julmy, Leon Derczynski and Mark Cieliebak

 

SemEval 2014 – Task 9B

 

Goal: Message Level Sentiment Classification for Tweets

Technology: Regularized Support Vector Machine (SVM) with hand-crafted features.

Result: F1-Score 67.54 (best score: 70.96)

Rank: 8th place out of 50

 

Paper:

Swiss-Chocolate: Sentiment Detection using Sparse SVMs and Part-Of-Speech n-Grams

Martin Jaggi, Fatih Uzdilli and Mark Cieliebak

 

 

SemEval 2014 – Task 9B

 

Goal: Message Level Sentiment classification for Tweets

Technology: Combining 12 sentiment classification systems with a meta-classifier

Result: F1-Score 66.79 (best score: 70.96)

Rank: 12th place out of 50
Paper:

JOINT_FORCES: Unite Competing Sentiment Classifiers with Random Forest

Oliver Dürr, Fatih Uzdilli, and Mark Cieliebak

Author Profiling

CLEF 2017 – PAN

 

Goal: Gender and Language Variety Detection from Tweets

Technology: Bi-directional Recurrent Neural Network (RNN) with attention mechanism

Results:

Gender Classification: Accuracy 75.31% (best score: 82.53)

Language Variety: Accuracy 85.22% (best score: 91.84)

Rank: 12th place out of 22

 

Paper:

Author Profiling with Bidirectional RNNs using Attention with GRUs – Notebook for PAN at CLEF 2017

Don Kodiyan, Florin Hardegger, Stephan Neuhaus, and Mark Cieliebak

Named Entity Recognition

CAp 2017

 

Goal: Named Entity Recognition on French Tweets

Technology: Deep learning with partially annotated data

Result: F-score 50.05 (best score: 58.89)

Rank: 5th place out of 8

 

Paper:

Swiss Chocolate at CAp 2017 NER Challenge: Partially Annotated Data and Transfer Learning

Nicole Falkner, Stefano Dolce, Pius von Däniken, and Mark Cieliebak

 

WNUT 2017

 

Goal: Named Entity Recognition on Tweets

Technology:

Result: F1-Score 40.78 (best score: 41.86)

Rank: 2nd place out of 7

 

Paper:

Transfer Learning and Sentence Level Features for Named Entity Recognition on Tweets

Pius von Däniken and Mark Cieliebak.

Adverse Drug Reactions

PSB 2016 – Task 1

 

Goal: Detect Mentions of Adverse Drug Reactions (ADR) in Tweets

Technology: Adaption of a feature-based sentiment classifier to ADR

Result: F1-Score 31.74 (best score: 41.95)

Rank: 5th place out of 8

 

Paper:

Adverse Drug Reaction Detection using an Adapted Sentiment Classifier

Dominic Egger, Fatih Uzdilli, and Mark Cieliebak

Question Answering

SemEval 2017 – Task 3A

 

Goal: Finding Relevant Responses to never-before seen Questions

Technology: Siamese Convolutional Neural Network (CNN) with attention mechanism

Result: MAP-Score 86.24 (best score: 88.43)

Rank: 7th place out of 13

 

Paper:

SwissAlps at SemEval-2017 Task 3: Attention-based Convolutional Neural

Network for Community Question Answering

Jan Deriu and Mark Cieliebak

Natural Language Generation

E2E NLG Challenge 2017

 

Goal: Generate Restaurant Reviews from Structured Data

Technology: Character-based Semantically Controlled Long Short-term Memory Network (SC-LSTM) with first-word control

Rank: 2nd rank out of 4 clusters

 

Paper:

End-to-End Trainable System for Enhancing Diversity in Natural Language Generation

Jan Deriu and Mark Cieliebak

Resources and Datasets

We provide datasets, such as textual corpora and word embeddings for sentiment analysis, to give back to the research community, as well as for commercial usage.

Our Publications

Together with our research partners, we regularly publish our achievements in distinguished conferences and journals:

Our resources are free and open to the public!

Contact Us

captcha