Abstract
Abstract
Aims:
There exists a learning curve (LC) with the adoption of any minimally invasive surgical (MIS) technique with implications for training, implementation, and evaluation. A standardized approach to describing and analyzing LCs in pediatric MIS is lacking. We sought to determine how pediatric MIS LCs are quantified and present a framework for reporting.
Methods:
Systematic search of MEDLINE and EMBASE 1985–October 2015 for articles describing MIS in the pediatric population and presenting formal analysis of the LC. Articles screened by two independent reviewers.
Results:
Twenty-nine articles (n = 17 general abdominal/thoracic, n = 12 urological) from an 18-year period (1997–2015) were included representing 3345 procedures (n = 3116 laparoscopic, n = 10 thoracoscopic, n = 219 robotic). Seven (24%) were prospective, three multicenter. Twenty-two (76%) presented data pertaining to >1 operating surgeon. Operative time was the most commonly employed surrogate of proficiency (n = 26 [90%] studies). Twenty (69%) described >1 LC outcome measure. Sixteen additional measures were described, including conversion (n = 12 studies); blood loss (n = 4 studies); complications (n = 10 studies); and postoperative outcomes (n = 14 studies). Three studies assessed impact of LC on trainees and one considered economic impact. LCs were presented in tabular form (n = 14 studies) and graphically (n = 19). Eleven (38%) studies undertook statistical appraisal utilizing comparative statistics (n = 8 studies) and regression analysis (n = 4 studies).
Conclusions:
Multiple outcome measures of proficiency are employed in reporting pediatric MIS experience and analysis of LCs is inconsistent. A standardized multioutcome approach to reporting should be encouraged. In addition, attempts should be made to quantify the impact on trainee involvement. We present an idealized framework for reporting.
Introduction
I
It is widely accepted that with the adoption of any novel technique, there comes a period of learning characterized initially by poorer outcomes that diminish with experience.5–7 This phenomenon is termed the “learning curve” and is recognized as comprising three components: a starting point (which varies with individual experience); a slope (wherein performance improves with experience); and a plateau (at which the outcome measured no longer changes significantly with greater experience). 8
When reporting the adoption of any novel technique, often the learning curve (LC) is afforded only cursory consideration. 9 However, LCs have broad implications impacting not only upon patient outcomes but also on trainee experience, resource utilization, and institutional costs. 8 In pediatric surgery, case volume and frequency are typically low and the adverse impacts of learning are more acutely felt as a significantly greater time is required to reach a plateau of competence. 5 It is therefore critical that when reporting the adoption of a novel technique in pMIS, the LC is afforded a detailed consideration and, as an ideal, reported in a standardized and readily recognizable format.
Objectives
We undertook a systematic review of the pediatric surgical literature with the aim of determining how LCs in pMIS are described and quantified and sought to produce a reporting framework for LCs in the pMIS literature.
Methods
This review was undertaken in accordance with PRISMA guidance. 10
Information sources
The MEDLINE and EMBASE databases from 1985 were systematically searched. The last set of searches was performed in October 2015. A deliberately broad three-dimensional search strategy was employed comprising three separate search strings of Medical subject heading (MeSH) terms to define the elements of “pediatric surgery,” “minimally invasive surgery,” and “learning curves.”
Data saturation was achieved by hand searching the contents section of the Journal of Pediatric Surgery (JPS) and the Journal of Laparoendoscopic & Advanced Surgical Techniques (JLAST), as well as the reference lists of included full text articles.
Study selection
Original articles in English describing MIS in the pediatric population and presenting outcome measures in the context of an LC and/or graphical representation and/or formal analysis of an LC were included. Exclusion criteria comprised studies with mixed (adult/pediatric) patient populations lacking a defined pediatric subgroup; studies reporting several discrete pMIS techniques where the outcome measures for a particular technique were not presented as a subgroup; and studies where the presence of an LC was acknowledged (e.g., stating “there was an LC for this procedure” in the article full text), but a specific outcome measure was not linked to the LC.
Article selection was undertaken by two reviewers (A.L.M. & S.A.C.) with backgrounds in pediatric minimally invasive surgery.
Data collection process
Data were extracted and abstracted by a single author (A.L.M.) utilizing a standardized proforma, which included article title; article type; authors; country of origin; year of publication; pediatric surgical subspecialty; sample size; patient pathology; pMIS technique; outcome measures; description of LC in article text; graphical and/or tabular representation of LC; and LC data.
Results
Study selection
Systematic search identified 460 articles. Hand searching of the contents pages of JPS and JLAST yielded a further 34 articles. Four hundred ninety-four articles were thus screened by title, identifying 154 for abstract review. Abstract review identified 68 articles appropriate for full text evaluation and 29 of these met inclusion/exclusion criteria and were included in the review11–39 (Fig. 1).

Article selection flow diagram. JLAST, Journal of Laparoendoscopic & Advanced Surgical Techniques; JPS, Journal of Pediatric Surgery.
Study characteristics
Included studies represented the 18-year period 1997–2015. The majority (n = 22, 76%) had a retrospective design and only three were multicentered. Eleven studies presented outcome data pertaining to more than one primary operating surgeon. Only seven studies explicitly stated that the outcome data and corresponding LC were that of a single surgeon. In the remaining 11 studies, it was not clear how many individual surgeons were represented in the outcome data.
An aggregate total of 3345 cases (3116 laparoscopic, 10 thoracoscopic, and 219 robotic) were reported in the 29 included studies. Median study sample size was 51 [2–703] cases. Thirty-nine discrete pMIS procedures were described (n = 31 general pediatric surgery, n = 8 pediatric urological surgery). Frequently described general surgical pMIS procedures included laparoscopic pyloromyotomy (n = 5 studies); laparoscopic/robotic fundoplication (n = 4 studies); laparoscopic (n = 3 studies); and laparoscopic inguinal hernia repair (n = 3 studies). Frequently described urological pMIS procedures included laparoscopic pyeloplasty (n = 4 studies) and laparoscopic nephrectomy (n = 4 studies). Two studies included more than one pMIS procedure.
LC outcomes
A number (splenectomy n = 17) of clinical outcome measures were employed and utilized to define the LC (Table 1). Four measures (inguinal hernia recurrence rate; perforation; resection margins; mobility) utilized in five studies were procedure specific, that is, could not be universally applied to all pMIS studies. No studies presented any patient-/parent-reported outcome data and only three studies presented technical data (number of sutures used [n = 1]; setup time [n = 2]). The majority of studies (n = 20, 69%) described more than one outcome measure with a median of 2 [1–7] measures. Operative time (n = 26, 90% of studies) was the most commonly utilized LC outcome measure followed by conversion (n = 12, 41% of studies) and “complications” (n = 10, 34% of studies).
CUSUM, cumulative sum; LC, learning curve; N/S, not specified in article; PO, Per Oral; SILS, single incision laparoscopic surgery.
Presentation of LCs
All 29 studies presented the LC in the narrative, that is, they described the LC within the text of the article. Fourteen (48%) studies additionally presented the LC data in the form of a table. Nineteen (66%) studies represented the LC graphically. Graphical representation of the LC comprised column charts (n = 2 studies); scatter plots (n = 6 studies); line charts (n = 9 studies); cumulative sum (CUSUM) charts (n = 1 study); and best-fit curve (n = 1 studies). Eleven studies (38%) undertook statistical analysis of their LC data. Statistical analysis comprised comparative statistics (n = 7 studies); regression analysis (n = 4 studies); and CUSUM analysis (n = 1 study).
A minority (n = 4, 14%) of studies considered the impact of the LC in the context of anything other than the patient. Three studies considered the impact of the LC on surgical trainee involvement and one study considered the economic impact of the LC on the institution.
Discussion
This review has demonstrated that in the pMIS literature, LCs lack comprehensive measurement and are poorly defined. They are often portrayed inappropriately in a manner—such as a table or column chart—that is not befitting their nature as a curvilinear phenomenon. Analysis of LCs is often rudimentary and this limits their usefulness to the reader. However, if LCs are given due and detailed consideration as part of the published reporting of novel pMIS techniques, then they have far greater potential to inform the adoption of novel techniques.
Measuring LCs
In defining the LC, the majority of studies employed more than one outcome measure. Outcome measures, such as “complications,” were often ambiguous terms lacking clear definition and open to interpretation. However, one measure, operative time, was unambiguous and was coincidentally by far the most commonly utilized metric.
As a surrogate of proficiency, the appeal of operative time is easy to understand, is a continuous variable that is easy to measure, has significant pedigree in the field, and can be readily applied to any pMIS procedure. However, a degree of caution ought to be exercised with its use in isolation as an absolute indicator of proficiency, particularly in pediatric surgical practice where owing to the broad nature of the specialty there may be considerable variability between cases in terms of patient characteristics (e.g., age/size, comorbidities) and pathology. Furthermore, to assume that a greater degree of competency is being exhibited if a case is completed faster may often be presumptive, particularly if the caseload of more than one surgeon is represented in the series. Thus, where operative time is to be employed as a surrogate of proficiency, it ought to be very clear that the outcome data originate from a series of sequential cases with broadly similar patient and pathological characteristics performed by a single primary surgeon. Where confounding factors do exist (e.g., multiple surgical teams, broad patient age/size range), such factors should be adjusted for with appropriate statistical modeling.
While a reasonable variety (17) of outcome measures were employed, there was a notable lack of the kind of sophisticated technical procedural measures of surgical learning such as instrument path length and movement range, which are prevalent in the simulation literature. 40 An element of this observed difference in approach to measuring learning is that there is perhaps considerably greater onus in the simulation literature to define the LC in detail as often a primary outcome of simulation is to shorten the LC. 41 Hence when designing the methodology of a study appraising a simulated technique, considerable consideration is likely given to metrics, which will robustly define the LC. However, in the broader clinical literature, the LC is perhaps of less interest as it is likely considered inevitable and its nature is rarely seen as being directly related to the study outcomes. LC measurement is possibly afforded considerably less consideration when designing the study methodology. Beyond authors' intentions, technical procedural metrics can be hard to measure reliably in the clinical setting, discouraging study authors from attempting to employ such measures. For example, to attempt to derive technical metrics from even a simple short procedure, such as a laparoscopic inguinal hernia repair, would require considerable preoperative planning, additional setup time, and specific technical resources (e.g., dedicated cameras, software) that may not be available in many institutions. However, the issues are not insurmountable, but certainly prior planning and a prospective study methodology would be required, which were seen to be lacking in the majority of studies included in this review. We observed that no single study sought to include technical procedural measures as they are clearly a much more direct and potentially sensitive measure of learning and procedural proficiency. However, should they be included in future clinical pMIS series, they ought not to be employed in isolation, but in tandem with other more global outcome measures. Certainly, it could be argued that it is possible to achieve technical procedural competency while global outcomes still remain poor, implying that learning is still occurring and a plateau of competency has not yet been achieved. For example, laparoscopic fundoplication could be held as an example of a complex procedure where it could be possible for a surgeon to have achieved a plateau of competency with regard to technical scores (e.g., instrument path length, movement range), but still remain on the slope of the curve with regard to more global outcome measures such as wrap failure and symptom persistence.
In the majority of studies, little or no consideration was afforded to portraying the impact of the LC on agents other than the patient such as trainees and institutional costs. This is notable as the extent of the possible impact of an LC on an institution's operating costs and its trainees' educational experience can and should be an important factor in considering the case for adopting a novel pMIS technique. This is particularly important in pediatric surgical practices where often the evidence for the clinical efficacy of a particular novel technique may be sparse and others factors (such as costs and trainee impact) can play an important role, when considering the merits of adopting a particular novel technique. Detailed knowledge of the impact of learning on factors such as costs and trainees' experience does not, of course, remove these inevitable initial burdens of learning, but it does permit a level of prior planning that can go some way toward managing and minimizing such negative impacts on an LC.
When measuring and defining the LC, the majority of studies employed outcome data that either represented the work of more than one primary operating surgeon or failed to specify the number of primary surgeons involved. It is acknowledged that individuals learn at different rates, and thus, it is presumptive to assume that their LCs will be equivalent. Compounding this, few studies adequately described the background (or lack of) in pMIS of the included surgeons. The prior experience of a learner, if relevant, can impact not only their starting point on the curve but also the rate at which they learn. Thus, if the case work of multiple surgeons with different levels of prior experience is represented in the outcome data set of a novel pMIS experience, then that data ought to be statistically adjusted/modeled to take account of such confounding factors before it is presented as an aggregate LC for that procedure. A number of techniques/models exist—the individual suitability of which is beyond the scope of this review—and authors ought to strive to employ one in appraising their data.
Presenting LCs
The nature of the generic LC as a curve ought to demand graphical representation in the form of a best-fit curve or at the most rudimentary level, a line plot, with the X axis representing case numbers and the Y whichever learning outcome measure(s) is being utilized. This allows the reader to readily identify the stages of learning and the experience/caseload required to establish a plateau of competency.
Data indicative of an LC in the form of a table has the potential to be useful, but ultimately, its supplemental and graphical presentation ought to be mandatory. The omission entirely by some authors of any graphical representation of the LC could be considered indicative of the only cursory consideration that the LC is often afforded. The use by other authors of a variety of inappropriate graphs and tables perhaps suggests a lack of understanding of the process of experiential learning, that is, it is a curvilinear in nature.
A framework for reporting
If LCs in the pMIS literature continue to be regarded in the rudimentary manner evident from this review, then the opportunity to utilize them to inform adoption, define competency, and permit cross-study comparison is lost. A universal tool for reporting pMIS LCs has the potential to drive standardization and significantly increase the utility of LC data. Thus, encompassing the ideals discussed in the preceding sections, we present a framework for “ideal” reporting of LCs.
Our framework (Fig. 2.) comprises three domains: measuring; presenting; and interpreting. In the first domain (“measuring”), we advocate a minimum of four outcome measures. Both pMIS procedures and pediatric surgical patients are complex and varied and this mandates multiple measures of outcomes as surrogates for competency, not just operative time alone. We suggest that at least one measure be universal (e.g., operative time) and applicable to any pMIS procedure in any patient population and as such comparable across studies. We suggest one should be procedure specific (e.g., mucosal perforation), patient orientated (e.g., time to feeds), and resource related (e.g., length of stay).

Framework for reporting learning curves in pediatric minimally invasive surgery.
In the second domain (“presenting”), we advocate that the LC be graphically represented as a best-fit curve with appropriate statistical interpretation and, where feasible, modeling be undertaken. Specifically, if more than one primary surgeon is represented in the data set and/or patient groups are pathologically diverse, then this should be adjusted for.
Finally, in the third domain (“interpreting”), the impact of the LC should be addressed. The relevance of the LC to clinical practice should be considered in the context of its impact upon patients, surgeons (including those in training), and the operational costs of the institution. A reader of a novel pMIS clinical series should be able to relatively easily ascertain the answer to the basic and pertinent question, “if I as a novice adopt this procedure at my institution, how many cases will it take me to reach a plateau of competency with regard to the impact of my learning on patient outcomes, trainee experience, and institutional costs?.” In satisfactorily answering these questions, the case for locally adopting a novel pMIS technique can be made all the more readily.
Limitations
This review has a number of limitations. There is the possibility that, given the breadth of the topic, the search strategy, while deliberately broad and inclusive, missed articles appropriate for inclusion. The reference lists of included articles were hand searched and an additional review of the contents' sections of JPS and JLAST was undertaken to minimize the possibility of overlooking articles. Furthermore, two reviewers participated in article selection to minimize inconsistencies and bias. The variability and lack of detail in the methodology and results reporting of the included studies limited the ability in this review to undertake meaningful synthesis and comparative analysis of LC outcome measures. This is a limitation that could be overcome in future reviews if standardized reporting of LCs in pMIS was to be embraced.
Conclusions
At present, multiple proxies of proficiency are employed in the reporting of pediatric MIS experience and analysis of LCs is inconsistent. A standardized multioutcome approach to reporting pMIS LCs along with graphical representation and statistical analysis should be encouraged.
Such meaningful analysis of the LC is critical to ascertain the impact of adoption of novel pMIS techniques on patients, trainees, and institutional resources.
Footnotes
Disclosure Statement
No competing financial interests exist.
