Groundtruth data

Description of the Contents

This page contains the experimental results for our WSDM 2017 paper "Modeling Event Importance for Ranking Daily News Events" we will call it CREST for simplicity. It is divinded into the following:
If you are using this data please cite our paper
@inproceedings{setty2017modeling,
title={Modeling event importance for ranking daily news events},
author={Setty, Vinay and Anand, Abhijit and Mishra, Arunav and Anand, Avishek},
booktitle={Proceedings of the Tenth ACM International Conference on Web Search and Data Mining},
pages={231--240},
year={2017},
organization={ACM}
}
1. For the different settings of CREST over the whole dataset as discussed in the paper is given here. The folders contain verbose files which inturn contain top k(here k=10) results for a given day across all days in the data sets. The result files shows the results for each individual split and average over all splits for each settings.
2. For different setting of CREST the experimental results are only for the last splits in the data set(both GDelt and Stics).
3. The results for the competitor is shown here. As discussed in the paper there are 2 settings for the competitor, with/without historical feature. Both experimental results on GDelt and Stics for those setting are shown.
4. Differnt results of Online vs Retrospective approaches are shown here.

Split Details

GDelt Data Set

Split-1 : 28th June 2013 to 16th October 2013
Split-2 : 17th October 2013 to 1st January 2014
Split-3 : 2nd January 2014 to 28th March 2014
Split-4 : 29th March 2014 to 13th June 2014
Split-5 : 14th June 2014 to 29th August 2014
Last Split : 1st July 2014 to 31th August 2014

Stics Data Set

Split-1 : 2nd January 2014 to 19th April 2014
Split-2 : 20th April 2014 to 5th August 2014
Split-3 : 6th August 2014 to 21th November 2014
Split-4 : 22th November 2014 to 9th March 2015
Split-5 : 10th March 2015 to 25th June 2015
Last Split : 1st May 2015 to 29th June 2015

1. Overall results in each of the settings for CREST on different data sets(Summarized results in Table 1 in the paper)

GDelt Data Set

[ICO]NameLast modifiedDescription
[DIR]CREST with Stuctural Feature(CREST-SF) 2015-10-14 09:09 Contains verbose files for CREST only using Structural Feature
[DIR]CREST with Stuctural Feature and Authority(CREST-SF-AUTH) 2015-10-14 09:09 Contains verbose files for CREST using Structural Feature and Diversity
[DIR]CREST with Stuctural Feature, Authority and Diversity(CREST-SF-AUTH-DIV) 2015-10-14 09:09 Contains verbose files for CREST using Stuctural Feature, Authority and Diversity
[DIR]CREST with Stuctural Feature, Authority, Diversity and History(CREST-SF-AUTH-DIV-HIST) 2015-10-14 09:09 Contains verbose files for CREST using Stuctural Feature, Authority, Diversity and History
[TXT]CREST-SF Result File 2015-10-14 09:09 Result file for CREST only using Structural Feature
[TXT]CREST-SF-AUTH Result File 2015-10-14 09:09 Result file for CREST using Structural Feature and Diversity
[TXT]CREST-SF-AUTH-DIV Result File 2015-10-14 09:09 Result file for CREST using Stuctural Feature, Authority and Diversity
[TXT]CREST-SF-AUTH-DIV-HIST Result File 2015-10-14 09:09 Result file for CREST using Stuctural Feature, Authority, Diversity and History

Stics Data Set

[ICO]NameLast modifiedDescription
[DIR]CREST with Stuctural Feature(CREST-SF) 2015-10-14 09:09 Contains verbose files for CREST only using Structural Feature
[DIR]CREST with Stuctural Feature and Authority(CREST-SF-AUTH) 2015-10-14 09:09 Contains verbose files for CREST using Structural Feature and Diversity
[DIR]CREST with Stuctural Feature, Authority and Diversity(CREST-SF-AUTH-DIV) 2015-10-14 09:09 Contains verbose files for CREST using Stuctural Feature, Authority and Diversity
[DIR]CREST with Stuctural Feature, Authority, Diversity and History(CREST-SF-AUTH-DIV-HIST) 2015-10-14 09:09 Contains verbose files for CREST using Stuctural Feature, Authority, Diversity and History
[TXT]CREST-SF Result File 2015-10-14 09:09 Result file for CREST only using Structural Feature
[TXT]CREST-SF-AUTH Result File 2015-10-14 09:09 Result file for CREST using Structural Feature and Diversity
[TXT]CREST-SF-AUTH-DIV Result File 2015-10-14 09:09 Result file for CREST using Stuctural Feature, Authority and Diversity
[TXT]CREST-SF-AUTH-DIV-HIST Result File 2015-10-14 09:09 Result file for CREST using Stuctural Feature, Authority, Diversity and History

2. CREST results in each setting for the last splits on different data sets (Summarized results in Table 2 in the paper)

GDelt Data Set

[ICO]NameLast modifiedDescription
[DIR]CREST with Stuctural Feature(CREST-SF) 2015-10-14 09:09 Contains verbose files for CREST only using Structural Feature
[DIR]CREST with Stuctural Feature and Authority(CREST-SF-AUTH) 2015-10-14 09:09 Contains verbose files for CREST using Structural Feature and Diversity
[DIR]CREST with Stuctural Feature, Authority and Diversity(CREST-SF-AUTH-DIV) 2015-10-14 09:09 Contains verbose files for CREST using Stuctural Feature, Authority and Diversity
[DIR]CREST with Stuctural Feature, Authority, Diversity and History(CREST-SF-AUTH-DIV-HIST) 2015-10-14 09:09 Contains verbose files for CREST using Stuctural Feature, Authority, Diversity and History
[TXT]CREST-SF Result File 2015-10-14 09:09 Result file for CREST only using Structural Feature
[TXT]CREST-SF-AUTH Result File 2015-10-14 09:09 Result file for CREST using Structural Feature and Diversity
[TXT]CREST-SF-AUTH-DIV Result File 2015-10-14 09:09 Result file for CREST using Stuctural Feature, Authority and Diversity
[TXT]CREST-SF-AUTH-DIV-HIST Result File 2015-10-14 09:09 Result file for CREST using Stuctural Feature, Authority, Diversity and History

Stics Data Set

[ICO]NameLast modifiedDescription
[DIR]CREST with Stuctural Feature(CREST-SF) 2015-10-14 09:09 Contains verbose files for CREST only using Structural Feature
[DIR]CREST with Stuctural Feature and Authority(CREST-SF-AUTH) 2015-10-14 09:09 Contains verbose files for CREST using Structural Feature and Diversity
[DIR]CREST with Stuctural Feature, Authority and Diversity(CREST-SF-AUTH-DIV) 2015-10-14 09:09 Contains verbose files for CREST using Stuctural Feature, Authority and Diversity
[DIR]CREST with Stuctural Feature, Authority, Diversity and History(CREST-SF-AUTH-DIV-HIST) 2015-10-14 09:09 Contains verbose files for CREST using Stuctural Feature, Authority, Diversity and History
[TXT]CREST-SF Result File 2015-10-14 09:09 Result file for CREST only using Structural Feature
[TXT]CREST-SF-AUTH Result File 2015-10-14 09:09 Result file for CREST using Structural Feature and Diversity
[TXT]CREST-SF-AUTH-DIV Result File 2015-10-14 09:09 Result file for CREST using Stuctural Feature, Authority and Diversity
[TXT]CREST-SF-AUTH-DIV-HIST Result File 2015-10-14 09:09 Result file for CREST using Stuctural Feature, Authority, Diversity and History

3. Competitor Results(Summarized results in Table 1(overall results) and Table 2(Last split Results))

Competitor Results with GDelt data Set

[ICO]NameLast modifiedDescription
[TXT]LM-BASED W History last split 2015-10-14 09:09 Competiror results using historical feature on Last Split
[TXT]LM-BASED WO History last split 2015-10-14 09:09 Competiror results without historical feature on Last Split
[TXT]LM-BASED W History overall 2015-10-14 09:09 Competiror results using historical feature on overall data set
[TXT]LM-BASED WO History overall 2015-10-14 09:09 Competiror results without historical feature on overall data set

Competitor Results with Stics data Set

[ICO]NameLast modifiedDescription
[TXT]LM-BASED W History last split 2015-10-14 09:09 Competiror results using historical feature on Last Split
[TXT]LM-BASED WO History last split 2015-10-14 09:09 Competiror results without historical feature on Last Split
[TXT]LM-BASED W History overall 2015-10-14 09:09 Competiror results using historical feature on overall data set
[TXT]LM-BASED WO History overall 2015-10-14 09:09 Competiror results without historical feature on overall data set

4. Online vs Retrospective(Summarized results in Table 3)

[ICO]NameLast modifiedDescription
[TXT]Retrospective vs Online GDelt 2015-10-14 09:09 Retrospective vs Online in GDelt data set
[TXT]Retrospective vs Online Stics 2015-10-14 09:09 Retrospective vs Online in Stics data set