Skip to content

Code and Data for our Findings of ACL 2021 paper titled 'Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentation. Varun Gangal *, Harsh Jhamtani *, Eduard Hovy, Taylor Berg-Kirkpatrick'

Notifications You must be signed in to change notification settings

harsh19/Diverse-Reference-Augmentation

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
 
 
 
 
 
 

Repository files navigation

Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentation

Code and Data for our Findings of ACL 2021 paper titled 'Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentation. Varun Gangal *, Harsh Jhamtani *, Eduard Hovy, Taylor Berg-Kirkpatrick'

Data

  • Relevant original and augmented reference files in are present in 'ref_files/' in the required format
  • Human ratings file: 'human_rating_correlation/mturk_rating_processed_output.csv'. Please consider citing Gupta et al if you use the human ratings file.

Code

Code and script to compute metric correlations with human ratings can be found in 'human_rating_correlation/' directory

Requirements

  • Python 3.7.5
  • bert_score (0.3.7)
  • nlgeval(Accessed: December 2020)
  • scipy 1.1.0

Citation

@inproceedings{acl2021dialogeval, 
title={Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentation}, 
author={Gangal, Varun and Jhamtani, Harsh and Hovy, Ed and Berg-Kirkpatrick, Taylor}, 
booktitle={Findings of ACL}, 
year={2021} 
}

About

Code and Data for our Findings of ACL 2021 paper titled 'Improving Automated Evaluation of Open Domain Dialog via Diverse Reference Augmentation. Varun Gangal *, Harsh Jhamtani *, Eduard Hovy, Taylor Berg-Kirkpatrick'

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published