Tuesday 5 September 2017

A Comparison of Nuggets and Clusters for Evaluating Timeline Summaries


There is growing interest in systems that generate timeline summaries by filtering high-volume streams of documents to retain only those that are relevant to a particular event or topic. Continued advances in algorithms and techniques for this task depend on standardized and reproducible evaluation methodologies for comparing systems. However, timeline summary evaluation is still in its infancy, with competing methodologies currently being explored in international evaluation forums such as TREC. One area of active exploration is how to explicitly represent the units of information that should appear in a 'good' summary. Currently, there are two main approaches, one based on identifying nuggets in an external 'ground truth', and the other based on clustering system outputs. In this paper, by building test collections that have both nugget and cluster annotations, we are able to compare these two approaches. Specifically, we address questions related to evaluation effort, differences in the final evaluation products, and correlations between scores and rankings generated by both approaches. We summarize advantages and disadvantages of nuggets and clusters to offer recommendations for future system evaluations.

Gaurav Brauah, Richard McCreadie and Jimmy Lin.
A Comparison of Nuggets and Clusters for Evaluating Timeline Summaries
In Proceedings of CIKM, 2017.

PDF
Dataset

10 comments:

Unknown said...

Nice information. Thank you for sharing such post

I Digital Academy said...

Thank u for sharing.... its a nice post.
Best Web Designing Training in Bangalore
Best SAP Training in Bangalore

Unknown said...

Awesome article

Unknown said...

Really useful and nice post.
Best Oracle Training in Bangalore

Unknown said...

Nice Post.
Digital Marketing Courses in Bangalore

Unknown said...

It's Really A Great Post. Looking For Some More Stuff.



shriram break free

Anonymous said...
This comment has been removed by the author.
Realtime Experts said...

It’s great blog to come across a every once in a while that isn’t the same out of date rehashed material. Fantastic read.best oracle training in bangalore

TNK Design Desk said...

Nice blog! i'm also working with a Digital marketing company in gurgaon
website designing in gurgaon
web company in delhi
web desiging company
web design & development banner
web design & development company
web design & development services
web design agency delhi
web design agency in delhi
web design and development services
web design companies in delhi
web design company delhi
web design company in delhi
web design company in gurgaon
web design company in noida
web design company list
web design company services
web design company website
web design delhi
web design development company
web design development services
web design in delhi
web design service
web design services company
web design services in delhi
web designer company
web designer delhi
web designer in delhi
web designers delhi
web designers in delhi
web designing & development
web designing advertisement
web designing and development
web designing and development company
web designing and development services
web designing companies in delhi
web designing company delhi
web designing company in delhi
web designing company in gurgaon
web designing company in new delhi
web designing company in noida
web designing company logo

Ananad said...
This comment has been removed by the author.

Post a Comment

older post Home