Applying artificial intelligence to automate clinical data extraction from veterinary electronic health records: A practical guidance and comparative scoping review

Review in Veterinary Medicine

Literature search

Search terms were generated based on the following concepts to cover our areas of interest: (1) the use of AI, (2) the desired data type, and (3) research in the veterinary field. Keywords and synonyms were iteratively developed to capture studies that apply AI tools to extract information from veterinary EHRs (Table 3). The search was performed on June 3, 2025, covering all publications up to that date.

OPEN IN VIEWER

Table 3.

Key concepts and associated search terms for the veterinary medicine literature.

Key Concepts	Search Terms
1. Artificial intelligence	Artificial intelligence, machine learning, deep learning, transformer, large language model, natural language processing
2. Data type	Text, notes, medical record, health record
3. Veterinary medicine	Veterinary, veterinarian, animal, equine, horse, canine, dog, feline, cat, bovine, cattle, cow, swine, pig, porcine, ovine, sheep

The terms within each concept were connected using Boolean operators “OR,” and concepts were combined using “AND.” The final search strategies were applied across 3 databases: PubMed, CAB Abstracts, and Web of Science. In addition, we included the ACL Anthology as a supplementary database focusing on computational linguistics research. Search strings were customized on each platform depending on its available options. The ACL Anthology does not provide direct export searches on the platform but instead stores all publication metadata as a Python library. A code script was created to perform the search and compile results into the Research Information Systems format and import into the same paper screening tool with the other database search results. As the ACL Anthology database is already NLP-specific by nature, the search terms were solely focused on “veterinary.” The full search strategies are provided in Supplemental Material S2.

Selection process

Paper screening was performed on Covidence, an online review management platform (https://www.covidence.org/). Once search results across all 4 databases were imported into Covidence, duplicates were automatically detected and removed before manual review. The screening was carried out independently by 2 reviewers (YC and CPC). The review process consisted of 2 phases (Fig. 1). In phase I, each reviewer screened independently and was blinded from the other’s assessments. Each title and abstract were examined against the preestablished inclusion and exclusion criteria (Table 4). Once the review was completed, disagreements were resolved through discussion. The selected studies will be advanced to the next phase. In phase II, full texts of selected studies in phase I were retrieved and assessed against the same inclusion criteria (Table 4) to ensure the studies fit our scope. Papers without accessible full texts were excluded. This phase was also conducted independently on Covidence by 2 reviewers (YC and CPC). Disagreements were resolved through discussion.

OPEN IN VIEWER

Figure 1.

PRISMA 2020 flow diagram of veterinary medicine paper selection.

OPEN IN VIEWER

Table 4.

Inclusion and exclusion criteria for the veterinary literature.

Criteria	Inclusion	Exclusion
Peer review	Peer-reviewed studies, conference papers, proceeding	Preprints
Domain	Veterinary medicine, animal health	Other disciplines (eg, human medicine, chemistry, agriculture)
Study type	Original research articles	Review papers, book chapters
NLP tasks	IE (NER, RE, negation detection), IR, classification	Summarization, question answering, benchmarking, annotation
Applications	Diagnosis coding, disease surveillance, mortality prediction, clinical sign classification	Report generation and other applications not related to IE (NER, RE, negation detection), IR, and classification
Data set type	EHRs, EMRs, radiology reports, pathology reports, progress notes	Social media posts, biomedical literature, medical case reports, drug residue reports
Language	Manuscripts written in English and trained on English data sets	Manuscripts not written in English or trained on non-English data sets

Abbreviations: NLP, natural language processing; IE, information extraction; NER, named entity recognition; RE, relation extraction; IR, information retrieval; EHR, electronic health record; EMR, electronic medical record.

Review in Human Medicine

Selection process

Independent screening was performed by 2 reviewers (YC and CPC) on Covidence, as described in for the review of the veterinary literature, but with a strict focus on IE using LLMs with prompts. Additional relevant studies that were not captured in the initial literature search were identified by manually reviewing the reference lists of included studies (snowball sampling) and were assessed for eligibility using the same criteria. See Fig. 2 and Table 5 for the full diagram and inclusion and exclusion criteria.

OPEN IN VIEWER

Figure 2.

PRISMA 2020 flow diagram of human medicine paper selection. IE, information extraction.

OPEN IN VIEWER

Table 5.

Inclusion and exclusion criteria for human literature.

Criteria	Inclusion	Exclusion
Peer review	Peer-reviewed studies, conference papers, proceeding	Preprints
Domain	Human medicine	Other disciplines (eg, veterinary medicine, chemistry, agriculture)
Study type	Original research articles	Review papers, book chapters
NLP tasks	IE (NER, RE, negation detection)	Classification, IR, summarization, annotation
Applications	Adverse events detection, social determinants of health information extraction, tumor staging extraction	Medical error detection, diagnosis coding, risk prediction
Model type	Prompt-based LLMs	Nonprompt-based models
Data set type	EMRs, EHRs, radiology reports, pathology reports, progress notes	Social media posts, biomedical literature, medical case reports
Prompts	Provided	Not provided
Language	Manuscripts written in English and trained on English data sets	Manuscripts not written in English or trained on non-English data sets

Abbreviations: NLP, natural language processing; IE, information extraction; NER, named entity recognition; RE, relation extraction; IR, information retrieval; LLM, large language model; EHR, electronic health record; EMR, electronic medical record.

Data charting

The data charting process followed the same approach as previously described, but the charting template was tailored for the human medicine scope. NLP task, downstream application, and prompt availability were handled at the eligibility-screening stage and therefore were not charted. An item documenting data set privacy was added because prompt-based LLMs often require external servers, which raises concerns about data security:

The charted data are presented in Supplemental Table S2 and synthesized in the “Results” section by grouping studies into broader categories for descriptive analysis.

Data Sets

In both domains, some research data came from shared or public databases, while the majority of the included studies utilize institution-specific data sets. In veterinary medicine, research predominantly relied on 3 large data sets, including the Small Animal Veterinary Surveillance Network (SAVSNET) (5/23), VetCompass (6/23), and institutional data sets from Colorado State University (CSU) (4/23). Of these, only SAVSNET (https://www.liverpool.ac.uk/savsnet/) offers publicly available sample data and shares research-ready data upon request. Similarly, a handful of benchmark data sets in human medicine were used more than once, including the Medical Information Mart for Intensive Care (MIMIC) (4/31), the Informatics for Integrating Biology and the Bedside (i2b2) (2/31), MTSamples (2/31), the Vaccine Adverse Event Reporting System (VAERS) (2/31), and The Cancer Genome Atlas (TCGA) (2/31). A significant proportion of included studies in both fields (8/23 in veterinary, 22/31 in human) relied on unique data sets cited only once, suggesting a shared challenge of data isolation. While the primary corpus in both disciplines was EHRs (18/23 in veterinary, 23/31 in human), niche data types, such as pathology and necropsy reports (3/23), oncology records (1/23), and shelter records (1/23), were also utilized in veterinary domain, which was comparable with human studies that used pathology (4/31), radiology (3/31), and echocardiogram reports (1/31).

In veterinary medicine, data sets often spanned from hundreds to millions of records, reflecting reliance on supervised ML methods that require extensive expert annotation (Fig. 3a). By contrast, in human medicine, data set sizes more commonly fell within a modest range of hundreds to thousands of records (Fig. 3b), likely influenced by the recent adoption of prompt-based LLMs, which reduces the need for large volumes of annotated training data.

OPEN IN VIEWER

Figure 3.

Distribution of data set sizes in the included veterinary and human medicine studies. The histograms show the number of data sets grouped by size, with the x-axis on a logarithmic scale (powers of 10). (a) In veterinary medicine (N = 22; one study did not report data set size), the distribution is wide, with a primary concentration in the hundreds-to-thousands range and also includes several large-scale data sets with over 1 million records. (b) In human medicine (N = 30; one study did not report data set size), the distribution is more constrained, with data sets predominantly containing hundreds to thousands of records.

Privacy

The use of LLMs in human medicine introduces significant privacy challenges. Due to their immense scale, many state-of-the-art LLMs are difficult to run locally, compelling researchers to rely on commercial APIs or hosted platforms. This dependency introduces the risk of transmitting sensitive patient information outside institutional boundaries. To mitigate this risk, most studies (23/31) employed data deidentification or anonymization, a process that itself demands considerable time and manual effort to ensure accuracy. A smaller subset of studies pursued alternative strategies, either using secure servers like Microsoft Azure or Amazon Web Services (5/31) to create enclosed environments or by running open-source LLMs such as LLaMA locally (3/31). This latter approach avoids sharing data with external providers but requires substantial computational resources to implement.

NLP Tasks and Downstream Application

Across the 23 included veterinary studies, the use of AI for automated clinical data extraction demonstrated considerable methodological diversity. Classification tasks dominated the field (13/23), and their applications ranged from disease coding to mortality prediction and obesity detection. IE (4/23) and NER (3/23) were less common but provided foundations for downstream applications such as surveillance, drug labeling, and resource development. Information retrieval (2/23) and negation/speculation detection (1/23) were rarely reported, suggesting these areas remain underexplored compared with classification-oriented approaches. By contrast, all 31 included human studies focused on IE, as determined by the eligibility criteria.

Ground Truth and Validation

Although some studies adopted labels directly from existing benchmarks, most defined study-specific tasks and therefore developed tailored label sets. In both veterinary and human medicine, ground truth was typically established by 1 to 6 domain experts, though their specific backgrounds (Supplemental Tables S1 and S2) and annotation protocols were inconsistently reported.^36,50,72 When reported, the annotation guidelines could be granular and extensive. For example, Vithanage et al ⁷² published an annotation protocol ⁸³ prior to their LLM-based extraction study, specifying annotator training hours and providing a comprehensive list of 120 dementia and malnutrition-relevant terms to extract. Similarly, Ralevski et al ⁵⁰ explicitly defined 9 conditions that should be labeled as “unstably housed.” Keloth et al ³⁶ employed a 2-level annotation scheme by first capturing the presence of key factors and then specifying finer-grained attributes such as temporality and assertion status.

Other studies described more general procedures in which 2 reviewers annotated independently, and disagreements were resolved through internal consensus and/or adjudication by an additional expert to finalize the labels. Label quality was most often assessed via interannotator agreement, typically Cohen’s kappa, with reported values ranging from approximately 0.55 to 1.0 (Supplemental Tables S1 and S2). Conventional interpretive thresholds for Cohen’s kappa, according to Landis and Koch, are as follows: 0.00–0.20, slight; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, substantial; 0.81–1.00, almost perfect. ³⁹ This range indicates moderate to near-perfect agreement among the included studies. However, comparisons across studies are constrained by differences in task complexity and labels. Although essential for mitigating subjectivity, the reliance on multireviewer consensus creates a significant resource bottleneck in NLP research, particularly for annotation tasks requiring deep domain expertise (eg, oncology or pathology). Apart from human consensus, one study used GPT-4 outputs as an auxiliary reference to flag missed annotations before finalizing the gold standard. ⁵⁰ In this workflow, the use of an LLM as a quality assurance layer suggests its potential not only to be evaluated but also to assist in the creation of ground truth.

Model Evolution

The evolution of models in veterinary medicine illustrates a trajectory from traditional algorithms to modern neural architectures (Fig. 4). Earlier studies relied on statistical and supervised ML algorithms such as support vector machine, random forest, and logistic regression, which demonstrated moderate performance (F1-scores typically ranging from 60% to 80%).^6,46,57 Progressively, DL models like long short-term memory and convolutional neural network architectures were adopted, then transformer-based models such as PetBERT, ²¹ VetBERT, ³² and PubMedBERT ⁵⁷ became the dominant approach. In the human medicine studies, while the primary focus was on LLMs, encoder-only architectures such as RoBERTa^3,20,27 and BioClinicalBERT^20,30,31 were also used as comparative baselines. Decoder-only transformers, more widely known as generative LLMs, represent the current emphasis in literature. In veterinary medicine, 4 studies applied models such as GPT,^24,33,76 LLaMA, ³³ and VetLLM, ³⁴ exhibiting competitive or superior performance in tasks like overweight detection and drug label extraction. In human medicine, adoption is more widespread, with models from the GPT series appearing in 17 studies and the LLaMA family in 13 studies. The emerging Mistral family and other models like Gemma were also reported (Supplemental Tables S1 and S2).

OPEN IN VIEWER

Figure 4.

Trends in model selection for veterinary information extraction by publication year (2017–2025). Each rectangle corresponds to the most advanced model reported in a single study. The darker the color, the more advanced the model. The chart illustrates a clear progression from rule-based and early deep learning models toward the increasing dominance of transformer-based architectures from 2020 onward.

Evaluation Metrics and Performance

The variety of models complicates direct performance comparison and evaluation. Even for the same model, such as LLaMA-2-13B, reported outcomes varied widely, ranging from 0% accuracy in one study ⁵⁶ to an F1-score of 73% in another, ²⁸ indicating how performance is shaped by individual study design.

Reported metrics are summarized in Supplemental Tables S1 and S2, where we prioritize the F1-score for comparability, although some studies reported only accuracy, precision, or recall. The performance of a model is fundamentally tied to the nature of the information it is tasked to extract. The evaluated data across studies differed significantly and could be broadly grouped into categorical and noncategorical outputs, each presenting distinct evaluation requirements.

Categorical extraction was most common and was typically operationalized as binary, multilabel, or multiclass classification. Many studies framed extraction as a series of yes/no questions, such as the presence or absence of a disease (eg, canine parvovirus, ⁵⁷ pulmonary embolism), ¹¹ a finding (eg, necrosis, ⁶⁹ endplate pathology), ⁸⁴ or a patient attribute (eg, vaccinated, ³⁵ biomarker positive). ⁵⁵ While for multilabel extraction, a single note could be positive for multiple binary targets simultaneously. A common veterinary example is International Classification of Diseases (ICD) coding, where each disease code is treated as a separate yes/no label and the model may assign more than one disease class per case.^{9,21,34,46,71,79} Similarly, metastatic site identification can be framed as selecting applicable sites from a fixed list of candidate organs (eg, 12 sites). ²³ By contrast, multiclass settings required choosing among more than 2 options. Examples include classifying the primary reason for an actions (eg, antibiotic indication across 52 diseases), ³² selecting among multiple temporal-frequency categories (eg, daily, weekly, and monthly patterns), ²⁸ social variables (eg, marital status categories),^27,36,50 imaging progression (worsening, stable, or improving), ²³ and histopathology features including laterality, histologic type, and staging.^31,55,69

Noncategorical data required extracting values that were not drawn from a closed label set, which presented a greater evaluation challenge. These tasks include numerical outputs (eg, tumor size in mm,^23,55 administration frequency, body weight in kg) ³³ and free-text fields (eg, breed, patient name, and descriptive tumor-location phrases). ⁴⁸ Such targets are inherently more difficult to evaluate with strict string matching and are more sensitive to formatting differences.

Given the generative nature of LLMs, this distinction between data types is central to the challenge of evaluation. For categorical data, the model’s output can be directly compared against a finite list of predefined labels (eg, specific disease class). However, for noncategorical data where items cannot be exhaustively listed, exact match becomes overly stringent. Semantically equivalent outputs may differ lexically (eg, “anorexia” vs “decreased appetite”), making performance sensitive to minor variations in tokenization and spelling.

To address this, one study explored Recall-Oriented Understudy for Gisting Evaluation (ROUGE) and Jaccard similarity, which are commonly used in text generation to accommodate semantically similar paraphrases. The authors emphasized that ROUGE and Jaccard are not standard IE evaluation measures, so that their values are not directly comparable with conventional entity-level F1. Supplemental Tables S1 and S2 prioritizes average F1-scores when provided by original studies, which may differ from individual scores reported here. In their results, Jaccard ranged from 0.43 to 0.62, ROUGE from 0.27 to 0.75, and entity-level F1 from 33% to 41% when compared with a board-certified oncologist. ⁴⁸ Another study even used a novel approach with BERTScore to measure lexical similarity, reporting F1-scores from 64.7% to 89.74%. ⁷² Two additional studies compared relaxed versus exact approaches, where relaxed matching permits partial overlap between predicted and reference spans and treats partial matches as correct. In one of the studies, relaxed F1-scores ranged from 67.6% to 90.1% and exact match F1-scores ranged from 47.4% to 78.5%. ³⁰ In the other, relaxed F1-scores ranged from 63.7% to 85.4% and exact match F1-scores from 63.6% to 83.1%. ¹⁰

At the other end of the spectrum, other studies took exact match to another level by applying it strictly to whole documents. For instance, in one study extracting 42 descriptors from pathology reports, an extraction was only considered correct if all 42 items were correct per morphology group, resulting in exact match rates from 19.57% (Mistral-7B) to 92.24% (Deductive Mistral-7B). ¹⁴ Similarly, in the VetTag study, while the per-prediction F1-score was 48.6%–66.2%, the fraction of notes where predicted diagnoses exactly matched the expert diagnoses was only 9.2%–26.2%. ⁷⁹

Regarding metrics across these varied evaluation approaches, most studies relied on a common set of metrics as summarized in the “Introduction” section. Model outputs could be mapped to standard error categories: correct extraction (true positive), incorrect extraction (false positive, often termed “hallucination” in LLM settings), and missed extraction (false negative). True negatives are only well defined when the label schema explicitly specifies absence (eg, “no” in binary classification or “not mentioned”). The majority of the studies emphasized performance on positive or specific target classes and reported precision and recall, which do not include true negatives.

Beyond metrics, performance is also influenced by the quality of the input text. Ambiguity and inconsistencies commonly produce errors that even human reviewers find difficult to resolve.^27,50 This issue was similarly noted in a veterinary study, ⁷⁶ suggesting that the inherent complexity of EHRs challenges both human assessment and model evaluation.

Finally, the impact of the “temperature” setting in LLMs has been investigated in both fields. Temperature is a parameter controlling randomness in the content generation of LLMs. Jiang et al ³⁴ fixed the setting at 0 to ensure reproducibility. Wulcan et al ⁷⁶ varied temperature from 0 to 1 and found no significant effect on classification performance, but a temperature of 0 yielded the highest reproducibility and the best compliance with citation instructions. Lower temperatures (eg, 0) were also generally preferred in human medicine to ensure deterministic outputs.^28,75 However, a contrasting finding showed that higher values (eg, 0.7) can improve outcomes in tasks requiring inference when explicit answers are absent from the source text. ³⁸

Prompt Engineering

To facilitate the adoption of LLMs for automated data extraction, links to the original prompts from studies in both veterinary and human medicine are provided in Supplemental Material S3. The foundational structure of these prompts typically begins with a task definition, which may be supplemented with input-output examples for zero-, one-, or few-shot learning. The strategy of few-shot prompting was shown to be effective across both domains. In the veterinary field, Hur et al ³³ conducted a direct comparison using GPT-4. The accuracy for dose extraction fell from 85% under a few-shot setting to just 24% in a zero-shot setting. In human medicine, few-shot prompts showed similar benefits. In one study, adding examples to GPT improved the microaverage F1-score from 69.9% (no example) to 82.3% with 3-shot prompting. ²⁶ Another study directly compared prompting strategies and found that few-shot yielded a 19% increase in the F1-score and a 17% increase in accuracy over zero-shot. ⁷² Similarly, a study reported that performance improved as the number of examples increased, with the relaxed-match F1-score for GPT-3.5 on the Vaccine Adverse Event Reporting System data set rising from 43.1% (zero-shot) to 67.6% at its maximum tested 5-shot setting. ³⁰

Apart from example-based prompting, studies in human medicine explored more versatile strategies. Role-playing was occasionally employed to increase the contextual relevance of responses,^{4,10,14,64,77} and chain-of-thought prompting, which asks the model to generate intermediate reasoning steps, was used to frame outputs by creating self-checkpoints.^31,50 Furthermore, as noted in the above, the variable formatting of generative outputs is a challenge for standardized evaluation and consistent extraction. To address this, some studies imposed explicit output constraints, such as fixed templates,^29,78 binary yes/no answers, ⁵⁸ or multiple-choice formats, ⁷⁷ to improve consistency. The significant impact of structured prompt design was illustrated by Arsenyan et al. ³ They compared a basic prompt against more advanced versions. Their “instruction prompt” employed role-playing (“act as a medical extractor”), while a “guided prompt” enforced strict output constraints, such as a fixed format and a default “I do not know” answer. This evolution from a basic to a guided prompt improved the precision for treatment extraction from 43% to 98%. ³ A similar effort was seen in Fu et al’s ²⁶ study, which focused on extracting social determinants of health from pediatric clinical notes. They compared a single-step approach (extracting all event components at once) against a 2-step question-answering pipeline. They found that GPT-4’s 2-step question-answering prompt slightly improved trigger extraction F1-score from 69.9% to 71.3%. ²⁶

A significant focus was also directed toward reducing hallucination and improving traceability. In human medicine, for example, Huang et al ³¹ proposed requiring models to provide supporting evidence and indicate confidence levels. Hu et al ³⁰ suggested incorporating error examples identified during iterative prompt refinement to reduce repeated mistakes, while Zhang et al ⁷⁸ recommended initiating a new conversation for each record to avoid contextual contamination. In a similar effort from the veterinary domain, Wulcan et al ⁷⁶ enhanced traceability by adding a secondary citation task to the primary goal of classifying clinical signs, ensuring the model’s outputs could be linked back to the source text.

Beyond the prompt content, prompt development is another critical but often underreported aspect. In contrast to the well-defined training and testing phases of supervised ML, many of the reviewed studies did not specify how many records were used to develop or iteratively refine their prompts. Among those that did, the development sets were typically small, ranging from 10 to 78 records, representing 1%–10% of total records.^4,31,64

Practical Guidelines for Generative Large Language Models in Clinical Information Extraction

To turn findings in this review into actions, and inspired by the structured approach of Tam et al, ⁶⁶ we propose comprehensive guidelines for adopting LLMs in clinical settings for IE tasks, which include 3 phases: (1) planning phase, (2) adjudication and refinement phase, and (3) testing phase.

Testing phase

Apply the finalized model and prompt to the held-out test set to calculate final performance metrics. If a fully annotated test set is unavailable or infeasible to create, use statistical sampling techniques (eg, stratified sampling based on report type or diagnosis) to select a representative subset for manual annotation and evaluation. Finally, determine whether the model’s performance meets the predefined objectives, ensuring it is accurate and reliable enough for its intended application.

We herein demonstrated a use case by extracting 62 clinical variables from dogs with suspected pancreatitis, following the previously outlined guidelines (Fig. 5). A detailed embedded prompt example, which adheres to the standardized components proposed in the guidelines, is provided as an annotated example in Table 6.

OPEN IN VIEWER

Figure 5.

Guidelines of the proposed information extraction framework and the corresponding applications in a canine pancreatitis study. API, application programming interface; HIPAA, Health Insurance Portability and Accountability Act; LLM, large language model.

OPEN IN VIEWER

Table 6.

An annotated example prompt used in a canine pancreatitis study.

Prompt	Corresponding Prompt Component (Prompt Engineering Section)
Medical Record Data Extraction	1
This task involves extracting structured information from medical record PDFs and populating a CSV file while ensuring accuracy, consistency, and adherence to predefined rules.	4
General Extraction Rules	1
- Extract information only from the corresponding file for each row in the CSV. Do not pull data from other PDFs.
- Numerical values: Enter as digits *without units* (eg, report 10.2 instead of 10.2 mg/dL).	2
- Categorical values: Use predefined choices (eg, gender = male or female).	2
- Binary indicators: Use 1 (present) or 0 (absent). Provide exact source terms in the details column.	5
- Lab values: Extract only numerical values. *Preserve symbols such as “<” and “>”.* Ignore reference ranges or flagged indicators like “H” or “L”	1
- Convert all NaN values to 0 before updating the CSV file.	1
Demographics and Signalment Extraction (From signalment_physical.pdf)	7
- age: Extract as a numerical value in years. *discard “years” or other unit indicators.*	1
- breed: Extract exactly as stated in the report.	1
- gender: Enter male or female.	1
- neuter_status:	1
- If “neutered”, “spayed”, or “castrated” is explicitly mentioned, enter 1 and quote the term in details. - If “intact” is mentioned, enter 2. - If the information is absent, enter 0.	5
Clinical Signs Extraction (From signalment_physical.pdf)	7
- If symptoms are inferred, enter 1, provide supporting terms in details.	5
- If symptoms are completely absent, enter 0.	1
Symptom Extraction Table	1
- vomit_nausea: Vomiting, nausea, regurgitation, emesis	1
- lethargy_weakness: Lethargy, weakness, dull, depressed	1
- appetite_loss: Inappetence, anorexia, hyporexia, not eating	1
- diarrhea_melena: Diarrhea, melena, loose stools, tarry stools	1
- abdominal_pain: Abdominal pain, prayer position, cranial abdominal pain	1
- weight_loss: Weight loss, cachexia	1
- duration: - If any of the following clinical signs (vomit_nausea, lethargy_weakness, appetite_loss, diarrhea_melena, abdominal_pain, or weight_loss) are present AND persist >14 days, enter 1 and specify the duration and the associated clinical sign(s) in details. - Otherwise, enter 0.	5
Vital Signs Extraction (From signalment_physical.pdf)	7
- temp (temperature): Enter as °F. *Extract only the number, discard “F” or other unit indicators.*	1
- bw (body weight): Extract numerical value. If provided in lbs, convert to kg.	1
- hr (heart rate): Enter the numerical value.	1
- rr (respiratory rate): Enter the numerical value.
- bcs (body condition score): Enter as provided (eg, 4/9 or 3/5).
Laboratory Results Extraction (From cbc.pdf, chem.pdf, and cpli.pdf)	7
- Extract only numerical values.	1
- Don’t extract HCT (hematocrit) for PCV (packed cell volume).	1

Limitations

This review has several limitations. First, our search of the human medicine literature was restricted to studies of prompt-based LLMs and those that published their prompts, a criterion intended to facilitate adoption by veterinary researchers. Therefore, earlier rule-based or traditional ML tools, such as cTAKES ⁵⁴ or CLAMP, ⁶¹ and many BERT-based models specifically developed for IE, were excluded. Although some included studies mentioned BERT variants as baselines, a substantial body of BERT-focused literature was not captured. Also, several studies did not provide full prompts and were excluded, which may have omitted otherwise valuable insights. Second, this review intentionally focused on prompt engineering and did not deeply explore alternative methods such as knowledge graphs, fine-tuning, or retrieval-augmented generation. While a full comparison is beyond our scope, an important direction for future work is to systematically evaluate the trade-offs among these approaches. In general, fine-tuning is the most computationally expensive, followed by retrieval-augmented generation, whereas in-context learning is typically the least resource-intensive, making it a highly accessible entry point for clinical data extraction. ² Third, only English-language manuscripts and data sets were included, and some potentially relevant non-English studies may have been missed. Finally, given the rapid pace of progress in NLP for medicine, especially with veterinary AI applications emerging, newer articles may be published during manuscript preparation. Readers should consult the most recent literature for the latest advancements.

The proposed guidelines, while grounded in the synthesis of the included literature, may not be applicable in all settings. First, the guidelines emphasize prompt-based LLM workflows and may be less relevant for fine-tuned encoder models, rule-based systems, or hybrid pipelines. Second, several recommendations (eg, 10% of records for prompt development) may be infeasible for resource-limited teams and could introduce practical bottlenecks for large projects. Third, the guidelines do not recommend a single best evaluation strategy. Choices such as exact versus relaxed matching, variable-level versus case-level scoring, and acceptable performance thresholds remain task- and risk-dependent, which may limit comparability across implementations. Finally, because LLM performance is sensitive to model versions and deployment settings, periodic re-validation is likely necessary as models and clinical documentation evolve.