*** News: Invited Speaker - Prof Kathy McKeown, Columbia University ***
INLG 2006 Special Session on Sharing Data and Comparative Evaluation
--------------------------------------------------------------------
Final Call for Short Paper Contributions
----------------------------------------
WHAT THIS IS ABOUT
Unlike many other areas within natural language processing, Natural
Language Generation (NLG) does not have a tradition of sharing data
resources, and no tradition of comparative evaluation of alternative
techniques. We believe the introduction of a shared task for NLG
would be of benefit to the community, both internally and in terms of
making the field more visible to those outside the community.
We would like to move discussion on this topic forward by means of a
special session at INLG'06. The focus of the session will be to move
us closer to an answer to the question: what would a shared task for
NLG look like?
BACKGROUND
Over the past year, in particular at ENLG'05 and UCNLG'05, NLG
researchers have started discussing ways in which the field might move
towards a situation where sharing data and other resources becomes
possible and comparative evaluation results can be produced. It is
clear that the wide range of tasks and issues of interest to
researchers in NLG makes it difficult to identify a single set of data
that can be used by many, or a single task that could serve as the
basis for comparative evaluation of techniques. For this reason we
believe it is important to see if we can establish a consensus view
amongst NLG researchers as to the content and focus of a shared task.
We invite submissions of two kinds of contributions to the INLG'06
Special Session on Sharing Data and Comparative Evaluation:
(1) Short papers for inclusion in the main INLG'06 proceedings:
(a) Position papers that explore or make proposals regarding
shared data and shared tasks, covering topics including (but
not necessarily limited to) the following:
* standardisation of NLG tasks
* what kind of data people are interested in sharing
* mechanisms for sharing
* evaluation metrics
* human evaluation methods
* evaluation campaigns
* lessons from evaluation campaigns in MT and document
* summarisation
(b) Reports on the results of experimenting with sharable data
sets. Contributors may use any data set at all, but for anyone
looking for a data set we recommend the SumTime Meteo data set
(created by the SumTime project group at Aberdeen), available
from the SIGGEN webpage.
(2) Abstracts for oral presentations during an Open Mic session on the
topics of the special session. We hope to involve a good number
of NLG researchers, and particularly encourage contributions likely
to stimulate discussion.
SUBMISSION
Paper submissions should follow the two-column format of ACL
proceedings (http://www.acl2006.org/program/style/), should be
in PDF format, and should not exceed three (3) pages, including
references. As reviewing will be blind, please do not include the
authors' names and affiliations, and avoid revealing their identity in
the text of the paper. Please follow the submission instructions for
the main INLG'06 session (details will be available on the Coling/ACL
web site: http://www.acl2006.org), but note that you should submit
your paper to the SPECIAL SESSION TRACK (not the short paper track or
any of the others). The deadline for submissions is Friday April 28th
(9 days later than the deadline for the main INLG'06 session).
Abstracts for the Open Mic part of the special session should not
exceed 300 words in length and preferably be in plain text
format. Please send your abstract to Anja Belz (asb@brighton.ac.uk)
and Robert Dale (rdale@ics.mq.edu.au) by no later than Friday May
27th.
TIMETABLE
Submission of papers: April 28th, 2006
Notification of acceptance for papers: May, 22nd, 2006
Submission of abstracts for Open-Mic session: May, 27th, 2006
Submission of camera-ready copy of papers: June 6th, 2006
Notification of acceptance for Open-Mic session: June, 16th, 2006
INLG 2006: July 15-16th, 2006
Questions regarding the special session, its topics and submission
procedure, should be directed to the organisers, Anja Belz
(asb@brighton.ac.uk) and Robert Dale (rdale@ics.mq.edu.au).
This archive was generated by hypermail 2b29 : Fri Apr 21 2006 - 11:42:53 MET DST