News Ranking Review Panels

NewsQ Activity Area: Ideation Through Data

How should news ranking and recommendation systems work? One answer has been news quality: higher quality news should rise to the top.

Yet, news quality itself is a complicated, and not clearly understood concept. There are aspects around which experienced journalists may disagree, depending on the context in question. At the same time, news ranking and recommendation systems are built around defined processes and rules, which require technical specificity and sometimes trade-offs.

For example,

  • if the presence of local news is important in news feeds, what exactly counts as local news?
  • what are the places of opinion pieces in breaking news feeds?
  • should the rules governing the ranking and recommendation for a science news feed be the same as another topic, such as politics?

We could many other questions such as these.

To support constructive engagement on these questions, NewsQ can convene review panels. Composed of journalists, journalism experts, and technologists, the panels delve into specific cases of news ranking and recommendation to aid their deliberation. The goal is to understand whether generalizable guidance or principles regarding quality journalism—interpretable both by algorithms and by journalists—might emerge.

2021 Panels

2021 panels focused on the quality of news feeds through two main questions:

  • How can the separation of opinion from news in a news feed actually work?
  • How important is it for a [quality] news feed to be representative?

This year’s panel cohort included:

  • Opinion: Nancy Ancrum, Karen Attiah, Maria Bustillos, Sewell Chan, Jen Granito, Elena Jaya Gooray, Arwa Mahdawi.
  • Representativeness: Amara Aguilar, Matthew Ballinger, Megan Garvey, Shaun Griswold, Sarah Gustavus, Ilia Rodriguez, Gabriella Schwarz.

Reflections from both panels can be found in two reports, available here for download:

  • “Improving Representativeness in Online News Ranking and Recommendation Systems” by Sarah Gustavus Lim and Ilia Rodríguez, with contributions from Amara Aguilar, Matthew Ballinger, Russell Contreras, Megan Garvey, Shaun Griswold, Gabriella Schwarz, and Andrea Chong Brás.
  • “Separating Quality News from Quality Opinion by Kate Harloe, with contributions from Nancy Ancrum, Karen Attiah, Maria Bustillos, Sewell Chan, Elena Gooray, and Arwa Mahdawi.
  • 2020 Panels

    Our first panel discussions over collected cases of ranked recommended news results included:

    • the place of opinion versus other kinds of news genres;
    • how science and health news might be ranked or recommended;
    • the priority of local news.

    Members of our inaugural cohort included:

    • Opinion: David Agraz, Jon Allsop, Leona Allen, Patricia Lopez, Subramaniam Vincent, Rebecca Traister, Rochelle Riley.
    • Science and Health: Caroline Chen, Jennifer Kahn, Katherina Thomas, Randi Hutter Epstein, Rick Weiss, Wudan Yan, Zoë Triska.
    • Local: Al Cross, Claudia Irizarry Aponte, David Kroman, Gabriel Kahn, Jeff Jarvis, Mandy Jenkins, Meredith Clark, Natalie Mao.

    Reflections from the panels can be found in three different panel reports here:

    • “The Local News Landscape is Broken: NewsQ Panel Review of Platform Products” by Gabriel Kahn with Meredith Clark, Al Cross, Claudia Irizarry Aponte, Mandy Jenkins, David Kroman
    • “Our Opinion: Recommendations for Publishing Opinion Journalism on Digital Platforms” by Subramaniam Vincent and Patricia Lopez with David Agraz, Leona Allen Ford, Jon Allsop, Rochelle Riley, and Rebecca Traister
    • “Towards Healthier Science and Health News Feeds: NewsQ Panel on Science and Health Journalism” by Caroline Chen, Randi Hutter Epstein, Jennifer Kahn, Katherina Thomas, Rick Weiss, Wudan Yan, and Yemile Bucay