A content audit case study: AI-assisted processes for student-produced local newsrooms

Abstract

This case study uses machine learning and generative AI to audit content diversity and source representation in The Scope, a Boston-based hyperlocal news outlet focused on underrepresented communities. Text analysis reveals geographic coverage, topical focus and quoted source demographics. Student journalists quoted more women and fewer public officials compared to professional norms. This study demonstrates AI’s potential to support small newsrooms in enhancing diversity, while addressing the ethical use of commercial AI in journalism.

Keywords

content/coverage community journalism minorities/diversity editing ethics new media/technological advances source analysis women

Get full access to this article

View all access options for this article.

References

Abernathy

(2022). The state of local news 2022. Northwestern Medill School of Journalism. https://localnewsinitiative.northwestern.edu/projects/state-of-local-news/

Armstrong

(2004). Influence of reporter gender on source selection in newspaper stories. Journalism & Mass Communication Quarterly, 81(1), 139–154. https://doi.org/10.1177/107769900408100110

Artwick

C. G.

(2014). News sourcing and gender on Twitter. Journalism, 15(8), 1111–1127. https://doi.org/10.1177/1464884913505030

Asr

F. T.

Mazraeh

Lopes

Gautam

Gonzales

Rao

Taboada

(2021). The gender gap tracker: Using natural language processing to measure gender bias in media. PLOS ONE, 16(1), Article e0245533. https://doi.org/10.1371/journal.pone.0245533

Bandy

Diakopoulos

(2020). Auditing news curation systems: A case study examining algorithmic and editorial logic in Apple News. Proceedings of the International AAAI Conference on Web and Social Media, 14(1), 36–47. https://doi.org/10.1609/icwsm.v14i1.7277

Barberá

Boydstun

A. E.

Linn

McMahon

Nagler

(2021). Automated text classification of news articles: A practical guide. Political Analysis, 29(1), 19–42. https://doi.org/10.1017/pan.2020.8

Bender

E. M.

Gebru

McMillan-Major

Shmitchell

(2021). On the dangers of stochastic parrots: Can language models be too big?

. In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency (pp. 610–623). Association for Computing Machinery.

Berkowitz

(2019). Reporters and their sources. In Wahl-Jorgensen

Hanitzsch

(Eds.), The handbook of journalism studies (pp. 165–179). Routledge.

Bhargava

Jespersen

M. H.

Ndulue

E. B.

Dsouza

(2025). Impacts of racial bias in historical training data for news AI. Computation + Journalism Symposium. https://doi.org/10.48550/arXiv.2512.16901

10.

Bhargava

Suresh

Dogan

A. L.

Suárez Val

Fumega

D’Ignazio

(2022). News as data for activists: A case study in feminicide counterdata production. Computation + Journalism Symposium. https://github.com/browninstitute/c-plus-j-website/raw/main/proceedings/Session9Group2.pdf

11.

Boyle

(2023). Artificial intelligence and the future of journalism: Where do we go from here? Newspaper Research Journal, 44(1), 3–5. https://doi.org/10.1177/07395329231158682

12.

Chen

N. T. N.

Ognyanova

Zhang

Wang

Ball-Rokeach

S. J.

Parks

(2015). Causing ripples in local power relations: The meso-level influence of a hyperlocal news website. Journalism Studies, 18(6), 710–731. https://doi.org/10.1080/1461670X.2015.1078738

13.

Clegg

Kennedy

(2021, May 26). The local news crisis will be solved one community at a time. Nieman Lab. https://www.niemanlab.org/2021/05/the-local-news-crisis-will-be-solved-one-community-at-a-time/

14.

Clegg

Kennedy

(2024). What works in community news: Media startups, news deserts, and the future of the fourth estate. Beacon Press.

15.

Coppins

(2021, October 14). A secretive hedge fund is gutting newsrooms. The Atlantic. https://www.theatlantic.com/magazine/archive/2021/11/alden-global-capital-killing-americas-newspapers/620171/

16.

Daud

Ullah

Rehman

Saba

Damaševičius

Sattar

(2023). Topic classification of online news articles using optimized machine learning models. Computers, 12(1), 16. https://doi.org/10.3390/computers12010016

17.

North Country Public Radio. (2022). Diversity Statement. https://www.northcountrypublicradio.org/pdfs/2022DiversityStatement.pdf

18.

Fatima

Shin

(2024, February 29). We asked an AI to map our stories across NYC. The City. https://www.thecity.nyc/2024/02/29/chatgpt-map-stories-nyc/

19.

Ferrier

Sinha

Outrich

(2016). Media deserts: Monitoring the changing media ecosystem. In Lloyd

Friedland

(Eds.), The Routledge companion to media and human rights (pp. 215–232). Palgrave Macmillan. https://doi.org/10.1057/978-1-349-94925-0_14

20.

Finneman

Heckman

M. E.

Walck

(2022). Reimagining journalistic roles: How student journalists are taking on the US news desert crisis. Journalism Studies, 23(3), 338–355.

21.

(2021, August 12). New tool allows NPR to track source diversity in real time. Poynter. https://www.poynter.org/reporting-editing/2021/new-tool-allows-npr-to-track-source-diversity-in-real-time/

22.

Godinho

(2018). Extraction, attribution, and classification of quotations in newspaper articles [Thesis, Técnico Lisboa, Lisboa, Portugal]. https://fenix.tecnico.ulisboa.pt/downloadFile/281870113704383/Thesis_Report.pdf

23.

Hare

(2021, July 21). Can J-schools fix news deserts? Poynter. https://www.poynter.org/reporting-editing/2021/can-j-schools-fix-news-deserts/

24.

Hayes

Lawless

J. L.

(2018). The decline of local news and its effects: New evidence from longitudinal data. The Journal of Politics, 80(1), 332–336. https://doi.org/10.1086/694105

25.

Hendrickx

Ballon

Ranaivoson

(2022). Dissecting news diversity: An integrated conceptual framework. Journalism, 23(8), 1751–1769. https://doi.org/10.1177/1464884920966881

26.

Huan

(2024). Politicized or popularized? News values and news voices in China’s and Australia’s media discourse of climate change. Critical Discourse Studies, 21(2), 200–217.

27.

Jehangir

Radhakrishnan

Agarwal

(2023). A survey on Named Entity Recognition—Datasets, tools, and methodologies. Natural Language Processing Journal, 3, 100017. https://doi.org/10.1016/j.nlp.2023.100017

28.

Keyes

(2018). The misgendering machines: Trans/HCI implications of automatic gender recognition. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW), 1–22. https://doi.org/10.1145/3274357

29.

Khanom

Kiesow

Zdun

Shyu

(2023). The news crawler: A big data approach to local information ecosystems. Media and Communication, 11(3), 318–329. https://doi.org/10.17645/mac.v11i3.67809

30.

Lacy

Coulson

D. C.

(2000). Comparative case study: Newspaper source use on the environmental beat. Newspaper Research Journal, 21(1), 13–25. https://doi.org/10.1177/073953290002100102

31.

Lee

Peirsman

Chang

Chambers

Surdeanu

Jurafsky

(2011). Stanford’s multi-pass sieve coreference resolution system at the CoNLL-2011 shared task. In Proceedings of the CoNLL-2011 (pp. 28–34). Association for Computational Linguistics.

32.

Martin

G. J.

McCrain

(2019). Local news and national politics. American Political Science Review, 113(2), 372–384. https://doi.org/10.1017/S0003055418000965

33.

Mathews

Ali

(2023). Desert work: Life and labor in a news and broadband desert. Mass Communication and Society, 26(5), 727–747. https://doi.org/10.1080/15205436.2022.2093749

34.

Matthews

Zhao

Jackson

Thorsen

Mellado

Abuali

Glück

(2024). Sourcing UK covid-19 news: An analysis of sourcing patterns of 15 UK news outlets reporting on covid-19 across Facebook, Twitter, and Instagram. Health Communication, 39(1), 173–182.

35.

Monroe

Wenzel

(2021). 2020 diversity and inclusion audit: The Philadelphia inquirer. https://drive.google.com/file/d/1MJB8IaP4MC_kpP47ZGsVo5y1cAR3VByR/view

36.

Moorhead

(2024). Who gets a voice in homelessness? A content analysis of quotations used by journalists. Journalism Practice, 19(9), 2127–2154.

37.

Center for Community News. (n.d.). More than 14 million: The impact of student reporting. https://www.uvm.edu/ccn/impact

38.

Mozzherina

(2013). An approach to improving the classification of the New York Times annotated corpus. In Klinov

Mouromtsev

(Eds.), Knowledge engineering and the semantic web (pp. 83–91). Springer. https://doi.org/10.1007/978-3-642-41360-5_7

39.

Mulupi

Zirugo

(2023). Who gets to speak? Sources in Covid-19 news cverage by Kenyan and Zimbabwean press. Feminist Media Studies, 23(7), 3157–3174.

40.

Newell

Cowlishaw

Man

(2018). Quote extraction and analysis for news. In Proceedings of the Workshop on Data Science, Journalism and Media, KDD (pp. 1–6). Association for Computing Machinery (ACM).

41.

Nielsen

R. K.

(2015). Introduction: The uncertain future of local journalism. In Nielsen

R. K.

(Ed.), Local journalism: The decline of newspapers and the rise of digital media (pp. 1–30). I.B. Tauris & Co.

42.

Nishikawa

Towner

Clawson

Waltenburg

(2009). Interviewing the interviewers. Howard Journal of Communications, 20(3), 242–259. https://doi.org/10.1080/10646170903070175

43.

Pareek

(2019). Non-binary gender and data. In Tandon

Mewa

Chattapadhyay

(Eds.), Handbook of gender and open data. https://cis.pubpub.org/pub/non-binary-gender-data

44.

Pavllo

Piccardi

West

(2018). Quootstrap: Scalable unsupervised extraction of quotation-speaker Pairs from large news corpora via bootstrapping. In Proceedings of the 12Th International AAAI Conference on Web and Social Media (ICWSM-18). AAAI Press.

45.

Poesio

Stuckardt

Versley

(2016). Anaphora resolution. Springer.

46.

Pouliquen

Steinberger

Best

(2007). Automatic detection of quotations in multilingual news. In Proceedings of Recent Advances in Natural Language Processing (487–492). INCOMA Ltd.

47.

Roberts

Bhargava

Valiukas

Jen

Malik

M. M.

Bishop

C. S.

Ndulue

E. B.

Dave

Clark

Etling

Faris

Shah

Rubinovitz

Hope

D’Ignazio

Bermejo

Benkler

Zuckerman

(2021). Media cloud: Massive open source collection of global news on the open web. Proceedings of the International AAAI Conference on Web and Social Media, 15, 1034–1045. https://doi.org/10.1609/icwsm.v15i1.18127

48.

Ross

(2007). The journalist, the housewife, the citizen and the press: Women and men as sources in local news narratives. Journalism, 8(4), 449–473. https://doi.org/10.1177/1464884907078659

49.

Rubinovitz

(2017). News matter: Embedding human intuition in machine intelligence through interactive data visualizations [Thesis]. Massachusetts Institute of Technology.

50.

Sandhaus

(2008). The New York Times annotated corpus. https://hdl.handle.net/11272.1/AB2/GZC6PL, Abacus Data Network, V1

51.

Sebo

(2021). Performance of gender detection tools: A comparative study of name-to-gender inference services. Journal of the Medical Library Association, 109(3), 414–421. https://doi.org/10.5195/jmla.2021.1185

52.

Seuri

Koivunen

Levola

Mäkelä

(2024). Enduring elites in quoted sources: Institutional alignment in Finnish media, 1999–2018. Nordicom Review, 45(S1), 193–215. https://doi.org/10.2478/nor-2024-0013

53.

Shine

(2021). Gender and sourcing in student journalism from Australia and New Zealand. Australian Journalism Review, 43(2), 227–243. https://doi.org/10.1386/ajr_00070_1

54.

Simon

F. M.

(2024). Artificial intelligence in the news: How AI retools, rationalizes, and reshapes journalism and the public arena. Tow Center for Digital Journalism, University of Oxford.

55.

Smethers

J. S.

Mwangi

S. C.

Bressers

(2021). Signal interruption in Baldwin City: Filling a communication vacuum in a small town “news desert.” Newspaper Research Journal, 42(3), 379–396. https://doi.org/10.1177/07395329211030687

56.

Smith

(2024, February 6). Q&A: How we used large language models to identify guests on popular podcasts. Pew Research Center. https://www.pewresearch.org/short-reads/2024/02/06/how-we-used-large-language-models-to-identify-guests-on-popular-podcasts/

57.

Sukthanker

Poria

Cambria

Thirunavukarasu

(2020). Anaphora and coreference resolution: A review. Information Fusion, 59, 139–162.

58.

Tenor

(2018). Hyperlocal news and media accountability. Digital Journalism, 6, 1064–1077. https://doi.org/10.1080/21670811.2018.1503059

59.

Tiffen

Jones

P. K.

Rowe

Aalberg

Coen

Curran

Soroka

(2014). Sources in the news: A comparative study. Journalism Studies, 15(4), 374–391.

60.

Usher

(2023). The real problems with the problem of news deserts: Toward rooting place, precision, and positionality in scholarship on local news and democracy. Political Communication, 40(2), 238–253. https://doi.org/10.1080/10584609.2023.2175399

61.

Usher

Darr

J. P.

Napoli

P. M.

Miller

M. L.

(2023). Media policy for an informed citizenry: Revisiting the information needs of communities for democracy in crisis. The ANNALS of the American Academy of Political and Social Science, 707(1), 8–20. https://doi.org/10.1177/00027162231219550

62.

Usnewsdeserts.com. (n.d.). https://www.usnewsdeserts.com/

63.

Vychegzhanin

Kotelnikov

(2019). Comparison of named entity recognition tools applied to news articles. In Ivannikov Ispras Open Conference (ISPRAS) (pp. 72–77). IEEE. https://doi.org/10.1109/ISPRAS47671.2019.00017

64.

Yến-Khanh

(2023). Representation of autism in Vietnamese digital news media: A computational corpus and framing analysis. Communication Research and Practice, 9(2), 142–158.

65.

Zeldes

G. A.

Fico

(2010). Broadcast and cable news network differences in the way reporters used women and minority group sources to cover the 2004 presidential race. Mass Communication and Society, 13, 512–527.