Abstract
Abstract
The characterization of proteins via liquid chromatography–mass spectrometry (LC-MS) and tandem MS is a challenge due to the large dynamic range and the high complexity of the molecules of interest. In LC-MS experiments, the inconsistent variation in the travel time of analytes in the LC column results in nonlinear shifts in the LC retention time (RT). This variability must be corrected to accurately match corresponding peptide features across samples in LC-MS experiments. Standard methods for RT alignment applied to the raw data are computationally expensive, making it impractical to process a large number of samples. More successful algorithms perform the alignment on features that matched across experiments based on pre-specified mass and RT windows. Features that match across multiple experiments are more likely to be true positives and, therefore, will be more suitable to drive the alignment correction. However, depending on the feature matching algorithm, ambiguities can arise when more than one candidate feature match falls within the specified windows which might affect the alignment performance. In addition, some of the feature-based alignment algorithms do not correct for nonlinear RT shifts. We propose a novel feature matching algorithm that incorporates wavelet-based shape information about the features. We tested our algorithm on two different applications of MS. First, we combined the feature matching algorithm with a robust nonparametric kernel-type regression to form a nonlinear feature-based alignment framework for LC-MS experiments. We validated our alignment framework on LC-MS data from complex samples with known spiked-in proteins, demonstrating our ability to correctly identify each of them with higher reproducibility and probability score when comparing with the SuperHirn software. In addition, by using our feature-based alignment framework, we were able to increase the number of matched features and improve the correlation between replicates. Second, we tested our feature matching algorithm on MALDI MS with MS/MS acquisitions. We found that using only features that matched across replicates of tandem mass spectra we could improve the identification of peptides compared with the current state-of-the-art software. Supplementary Material is available online at www.libertonline.com/cmb.
1. Introduction
The identification of peptides is performed via shotgun tandem MS in which only a small subset of peptide features present in the sample (usually the high abundance ones) are selected for the analysis. An alternative approach that spans a wider dynamic range is the analysis of peptide ion currents directly from the LC-MS maps using the peptides m/z, charge, and abundance as well as their RT information. This analysis is complicated by high levels of noise and variability among different systems and experiments (Listgarten and Emili, 2005). In particular, for several reasons, there is an inconsistent variation in the travel time of the analytes in the LC column. The common approach for analyzing LC-MS maps involves several major steps: (1) low-level signal processing attempts to remove various types of noise from the data; (2) two-dimensional (2D) feature extraction aims to detect peptides at each LC-MS map; (3) alignment correction along the LC retention time axis attempts to make RT comparable across samples; (4) feature matching compares peptide sets across the LC-MS maps in order to match recurring components. The output of these steps is a matrix in which the columns are samples, the rows are peptides detected across all samples, and the entries represent the desired property (e.g., the intensity/area) of a given peptide in a particular sample. Finally, statistical analysis and pattern detection are performed to classify the samples according to different biological conditions and diseases.
In both approaches, the peptides of interest are identified through tandem MS using search tools such as SEQUEST, MASCOT, and X!Tandem to match sets of peptide tandem mass spectra with their corresponding protein sequences.
The alignment correction is a crucial step in LC-MS experiments—the inconsistent variation in the LC column can cause nonlinear shifts in the LC RT axis which has to be corrected across different LC-MS maps in order to accurately match corresponding features.
Various alignment methods have been proposed to correct the RT axis variability across LC-MS maps. While some algorithms are based on the raw data (Listgarten et al., 2006; Prakash et al., 2006; Prince and Marcotte, 2006; Christin et al., 2008), others use only the total ion current (TIC) for the alignment (Tomasi et al., 2004; Listgarten and Emili, 2005). In general, methods for LC-MS maps alignment based on raw data tend to be more accurate. However, they are computationally intensive and therefore may not be applicable for the multiple alignment of a large number of samples typically seen in LC-MS experiments. TIC data on the other hand is more simple, but it does not take into consideration the information at the MS dimension which also contains information about the RT shifts. Current state-of-the-art algorithms are based only on the features detected at each LC-MS map (Lange et al., 2007; Suits et al., 2008; Mueller et al., 2007; Podwojski et al., 2009). These algorithms depend on the feature extraction and matching steps and are much faster to compute. In this procedure, corresponding peptide features are matched across samples and an alignment correction is applied to their RT coordinates. Current methods for feature matching are based on the mass and RT deviation thresholds which can result in ambiguous features, where a feature has more than one match. As a result, different feature matching algorithms can result in different alignment performance. In addition, some of the feature-based alignment algorithms do not correct nonlinear RT shifts (Lange et al., 2007), which are ubiquitous in LC-MS experiments (Podwojski et al., 2009).
In this work, we propose a novel feature matching algorithm that incorporated wavelet-based shape information about the MS features. We propose to use this algorithm in the alignment correction step in LC-MS experiments. More specifically, we present a nonlinear feature-based alignment framework in which the nonparametric kernel-type regression estimator, also called a locally weighted scatter plot smoother (LOWESS), is used for the correction function, and is calculated based on the detected and matched features. This function is then used to correct the RT of the matched features across LC-MS maps. The proposed shape-based feature matching algorithm is an extension of our previous method for feature matching in MS experiments (Fasulo et al., 2006).
We also propose a new approach for the identification of proteins via tandem MS based on our feature matching algorithm, in which only features that were matched across replicates of tandem mass spectra (MS/MS) are used for peptide identification.
There are several advantages to our approach. Our feature matching algorithm incorporates wavelet-based shape information about the signal as well as the location of the features in the mass and time domain. For the alignment algorithm, this is combined with a proven approach to robust, nonparametric regression estimation which does not require prior knowledge about the data. The correction function, LOWESS, was previously proven to perform well in aligning LC-MS maps (Mueller et al., 2007; Podwojski et al., 2009). In a nonlinear feature-based alignment framework, LOWESS is calculated based on the detected features that matched across LC-MS experiments. Features that match across multiple experiments are more likely to be true positives and therefore, will be more suitable to drive the correction function. Finally, our feature matching method applies equally to both 2D MS typically seen in LC-MS experiments and 1D MALDI MS with MS/MS acquisitions.
2. Methods
We propose a novel feature matching algorithm to be incorporated in RT alignment in LC-MS experiments. Our alignment procedure is comprised of a nonparametric kernel-type regression estimation and an extended, shape-based MS feature matching. In Subsections 2.1 and 2.2, we summarize our previously developed feature matching algorithm and describe its extension to include wavelet-based shape information. In Subsection 2.3, we describe the alignment procedure and the way the regression estimation and the feature matching step are tied together. Finally, in Subsection 2.4, we describe the similarity metric used to evaluate the performance of the alignment correction procedure.
2.1. n-dimensional feature matching in mass spectrometry
The feature matching step aims to match corresponding peptide features across different MS spectra and LC-MS maps to ensure that the same peptide intensities are correctly identified. For the LC-MS alignment, we use this step twice: (1) to find the corresponding features across pairs of LC-MS maps to which a nonparametric kernel-type regression estimator is applied for the RT alignment correction, and (2) once all the maps are aligned, to match recurring features across the aligned maps. In previous work, we have developed a method for feature matching in MS (Fasulo et al., 2006). For completeness, we briefly summarize the method here.
Our feature matching method is similar to clustering and is based on clique finding and optimization. In LC-MS experiments, this method can be used to cluster features based on multiple dimensions, using m/z, RT, charge state, and/or other dimensions representing additional peptide feature properties if applicable.
Our method is comprised of three main steps. First, a parameter estimation step in which our method requires one simple parameter for each dimension, which is the stochastic measurement error; these parameters can be estimated from the data in many cases, thus requiring no manual parameter selection by the user. Second, clique finding in which a computational geometry approach is used to find all groups of features that lie within given error intervals; this step is based on the location of the features in the mass and RT domains and can result in ambiguous features when a feature has more than one match. Third, an optimization step in which a simulated annealing process is used to place ambiguous features.
Here, in order to improve the assignment of ambiguous features, we incorporate shape-based information into the geometric information used in the second step. For this, we use the Haar wavelet transform signals of the features. More specifically, we extended both our 1D and 2D feature extraction algorithms (Noy and Fasulo, 2007a,b) to provide the 1D and 2D Haar wavelet transform of the raw signal for each detected peptide feature as well as other feature properties (see Subsection 2.2. for more information about the Haar wavelet transform). For each pair of cliques, we use the correlation between the wavelet transform of the signals of corresponding peptide features as a similarity measure in order to correct the assignment of ambiguous features.
2.2. Haar-wavelet transform of mass spectrometry features
We use the Haar wavelet to represent MS features for the feature matching step. Haar wavelets belonging to Daubechies wavelet family are commonly used because they are easy to implement and fast to compute (Ramsey and Schafer, 2002). By recursively averaging and differencing adjacent elements in a pairwise manner at different resolutions, they retain the most important information about a signal which greatly simplifies the analysis. More information about wavelets can be found in various reviews and textbooks (Mallat, 1998).
The Haar wavelet transform is used extensively in image processing for compression. In addition, this method has been shown to perform well in image registration and matching analysis of multiresolution features (Netanyahu et al., 2004). To date, among the MS community, the discrete wavelet transform is used for denoising MS data (Li et al., 2006; Coombes et al., 2005) and feature detection (Schulz-Trieglaff et al., 2008; Bellew et al., 2006), and there was no attempt, to the best of our knowledge, to use it for feature matching.
We extended our 1D and 2D feature extraction algorithm (Noy and Fasulo, 2007a,b) to calculate the wavelet transform for the detected features. In both cases, we calculate the wavelet transform of a region around the feature. Ideally, this region should be large enough to capture trends that flank the features in question. In addition, we set the region size to be divisible by 2 J (where J is the number of decompositions) as required for the Haar wavelet calculation. For the 1D features, we picked a region size in the m/z space and calculated the nearest power of 2 to use in the sampling. For the 2D features, the region size depends on the number of scans found containing the feature. In order to calculate a 2D Haar transform for each LC-MS feature, we build a matrix for each cluster of 1D features where the rows are the 1D wavelet transform of each feature. Then, we applied 1D Haar transform to each column to create the 2D Haar transform. In general, more wavelet coefficients means more detailed description; however, it also means more data and more noise. Thus, the low-frequency coefficients of each MS feature wavelet transform were retained in our analysis. For both the 1D and 2D features, best results were obtained with the first 16 coefficients (data not shown). Figure 1 in the Supplementary Material illustrates the 1D wavelet transform of ambiguous features in two replicates of MS/MS spectra (see www.liebertonline.com/cmb for online Supplementary Material). For the example shown, in the raw signal, the correlation across replicates is higher between the two distinct features than between the matched features. In the wavelet transform, the correlation across replicates is higher for the matched features.
There are several advantages of using wavelet transform over the raw signal in feature matching. First, only the low-frequency coefficients are retained in the analysis which makes it much faster to compute. Second, high-frequency features such as noise are discarded. Third, by calculating the sums and differences of adjacent elements, the Haar wavelet transform is more robust to differences in LC-MS feature signals and alignment errors.
2.3. Retention time correction in liquid chromatography–mass spectrometry
In LC-MS experiments, variability in the retention time of analytes results in nonlinear shifts in the RT axis. This has to be corrected to account for inconsistent deviations in peptide elution times across different LC-MS maps in order to allow the usage of RT (in addition to other peptide feature properties) in the feature matching step.
For RT correction, we utilize the locally weighted scatter plot smoother (LOWESS) (Cleveland, 1979), which is based on nonparametric kernel regression estimator. We start by applying our 2D feature extraction method in which a feature list is provided for every LC-MS map (Noy and Fasulo, 2007b). The feature list is comprised of the following peptide feature properties: m/z, RT, charge state and the wavelet transform of the raw signal. Then, for each target LC-MS map and a reference, denoted A and B respectively, we applied our feature matching algorithm to find all the common MS features (
The advantage of using LOWESS for fitting the regression parameters for the alignment correction is its robustness in dealing with nonlinear shifts that might occur between the corresponding RT values of the target and the reference. In particular, LOWESS is less influenced by outliers compared to classical linear least-squares regression fitting algorithms (Cleveland and Devlin, 1988). The advantage of using the ranking order of RT in the feature matching step is that it does not depend on the actual RT values which can vary across LC-MS experiments. Rather, it depends on the rank of the RT values when set in order which is expected to be more robust for corresponding peptides across LC-MS experiments.
2.4. Assessing the similarity between liquid chromatography–mass spectrometry maps
In order to evaluate the performance of our alignment correction algorithm, we assessed the similarity between LC-MS maps before and after the alignment. For this, we adopt the strategy used in Mueller et al. (2007) in which an LC-MS similarity score, SSIM, is assigned pairwise to LC-MS maps. SSIM is based on the overlap of the detected features and the reproducibility of their intensity values:
More specifically, it comprised of feature overlap score:
where
where RA/B,x represents the intensity rank of common feature x in LC-MS samples A and B, respectively. SSIM is between 0 to 1 such that a score close to 1 represents high similarity.
3. Results and Discussion
3.1. Validation on a benchmark liquid chromatography–mass spectrometry dataset of six standard proteins
We tested our methods on a benchmark dataset of six standard proteins in a complex sample background (Mueller et al., 2007). More specifically, this dataset consists of two-fold dilution series of the six proteins myoglobin, carbonic anhydrase, cytochrome c, lysozyme, alcohol dehydrogenase, and aldolase A, spiked into a complex sample background of human peptides. Each dilution step was analyzed in triplicate on an FT-LTQ instrument to give 18 LC-MS maps, and MS/MS scans were obtained. Note that the dilution profiles are not sequential; we refer to each dilution series as a target profile in the text.
First, each LC-MS map was subjected to our 2D feature extraction routine (Noy and Fasulo, 2007b) to give a peptide list with the m/z, charge state, retention time, intensity and the wavelet transform signal for every peptide feature. Then, the detected features of each LC-MS map were matched and aligned with a reference LC-MS map using our feature alignment procedure (see Subsection 2.3 for more details). The final outcome of this procedure was a matrix with a format analogous to gene expression studies in which the columns are samples and rows are peptides. Since the data set contains three LC-MS replicates for each of the six dilution runs, we averaged the abundance of corresponding peptides among replicates to give a six-column matrix. Missing values were ignored when calculating the average.
Our validation analysis proceeded as follows. First, performance measurements were made by assessing the similarity between the LC-MS samples in the dataset in order to evaluate the reproducibility of our results. Then, we followed the protein profile analysis used by Mueller et al. (2007) in order to evaluate whether our methods were able to correctly identify the six proteins in the dataset. More specifically, we correlated the peptide abundance profiles of our results with the known dilution profiles. We compared our results with the SupeHirn software (Mueller et al., 2007).
3.1.1. Performance measurements and reproducibility of peptide feature abundance across LC-MS experiments
In order to qualitatively evaluate our methods, we assigned LC-MS similarity scores SSIM to pairs of LC-MS samples (see Subsection 2.4 for more details). The LC-MS similarity score is based on the unification of the detected features and the reproducibility of their intensity values. As expected, we found that the reproducibility of the intensity values was very high across the samples and in particular within replicates (see Fig. 3 in Supplementary Material). On average, the Spearman correlation coefficient was 0.9 across all samples and 0.93 within replicates. It should be noted that, for the SINT score, we used the log intensity values.
In order to evaluate our alignment algorithm, we calculated the SSIM pairwise for every LC-MS map and a reference before and after applying our alignment procedure. On average, SSIM before and after alignment was 0.62 and 0.83, respectively.
The alignment and matching algorithms were implemented in Java, and the program was run on a typical PC running Microsoft Windows XP (3GB RAM). The feature matching and alignment routine required 2 min CPU time for all 18 samples (this is approximately 7 sec per one pair of LC-MS maps).
3.1.2. Unsupervised feature profiling
In order to evaluate whether our methods were able to correctly identify the proteins involved in the experiments, we followed a similar approach to the one presented by Mueller et al. (2007). We first extracted feature abundance profiles from our final expression matrix. Then, we calculated the correlation between our results and the target profiles.
More specifically, we used the K-means clustering method (MacQueen, 1967) to extract the feature profiles. Since our final matrix contains missing values as well, we consider profiles that were detected in at least four of the six runs, allowing two missing values per profile. In addition, for this analysis, the missing values were set to zero. We used the gap statistics to estimate the optimal number of K-means start cluster centers (Tibshirani et al., 2000). By gap statistic analysis, we found that seven clusters were required for the K-means clustering (see Fig. 4 in Supplementary Material) with our analysis as opposed to 12 using the SupeHirn software (Mueller et al., 2007). As expected, one cluster profile was pretty much constant reflecting the detected peptides in identical samples that were not spiked in. The remaining six clusters correlate well with their target profiles with an average correlation of 0.78. Figure 1 illustrates the constructed K-means clustering profiles together with their standard deviations.

Unsupervised feature profiling. The seven constructed K-means clustering profiles together with their standard deviations. Six profiles correlate well with the target profiles (blue lines).
3.1.3. Targeted protein profiles
In order to evaluate whether our methods were able to correctly detect the known proteins that were identified by Mueller et al. (2007) using MS/MS, we followed the protein profiling analysis presented in the study. In this analysis, we used the peptide list provided by Mueller et al. (2007) containing the peptide ID from the MS/MS analysis (when applicable) along with the usual feature properties (e.g., m/z, retention time, charge state). First, we extracted a subset of deconvoluted peptides from our feature matrix. Deconvoluted peptides are a set of LC-MS features that belong to the same peptide but appear with different charge states. Then, from the subset of the deconvoluted peptides, we extracted the peptides that matched those identified by Mueller et al. (2007). Only those peptides for which the m/z and RT difference from the identified peptides fall within fixed m/z and RT tolerance windows were extracted. Protein consensus profiles were built by averaging the LC-MS feature profiles of all deconvoluted peptides associated with a given protein.
We used the statistical method presented by Mueller et al. (2007) to assign high correlation peptide profiles to a given target profile. The following was applied for each of the six known proteins. First, profile scores, based on the Manhattan distance between all deconvoluted peptides and the target profile, were calculated to build a distribution of scores. This distribution is then modeled by a mixture of two Gaussian curves which are used to compute a profile probability of a true correlation between the deconvoluted peptide profile and the target profile. The parameters of the two Gaussian distributions are found by using the expectation maximization (EM) procedure (Hastie et al., 2001). Figure 2 illustrates the distributions of profile scores obtained by calculating the distance between each target profile and the profiles of all its deconvoluted peptides. As obtained in Mueller et al. (2007), a clear partition of the histogram into two subdistributions is observed; the left side of the histogram reflects the population of high similarity profiles, and the right side defines the population of low similarity scores. The shapes of these bimodel distributions differ from experiment to experiment, and in order to adapt the calculations to the data, a two-component Gaussian model was fitted to the data and peptide profile probabilities were calculated. These probabilities were used to remove deconvoluted peptides with low profile similarity to the protein consensus profile. Only peptides with profile probability higher than 0.5 were retained in the analysis. We compare our results with the SuperHirn software used in the study. We compared the average profile probabilities of the peptides assigned to each of the target profiles, the average number of identified peptide features per protein, and in how many replicates a protein has been assigned to the correct target profile. Both methods found the correct proteins in all replicates. Over all, using our methods, we got more identified pe ptides per protein with higher profile probabilities and higher reproducibility across replicates. On average, 10.22 ± 3.28 peptides were identified per protein with profile probability of 0.99 ± 0.01 using our methods, and 9.33 ± 1.3 peptides with profile probability of 0.95 ± 0.08 using the SuperHirn software. Table 1 shows the averaged profile probabilities of the assigned protein for both methods.

Evaluation of protein profile correlation to the target profile. Evaluation of protein profile correlation to the target profile. A two-component Gaussian model was fitted to the histogram of profile scores from deconvoluted peptide features to a given target profile.
For every target profile, the assigned protein with a profile probability above a threshold of 0.5 is shown. Probability scores and standard deviations are averaged across replicates.
Then, a protein consensus was calculated and correlated with the corresponding target profile. On average, the protein consensuses were highly correlated with their corresponding target profiles (R = 0.89; Pearson correlation).
In order to test the sensitivity of the correlation results to small subsets, we performed bootstrapping analysis with 1000 random samples (Efron and Tibshirani, 1993) for each protein profile. We found that these correlation values were rather robust. Subsets of each protein profile have similar median correlations and only slightly lower average correlations, and the majority are statistically significant (p < 0.05; Table 2).
The percentages of the statistically significant correlation coefficients (p-value 0.05) are indicated in the last column. The correlation coefficients for all proteins were added for completeness.
3.2. Replicate feature matching improves protein identification via MALDI-TOF MS/MS
In this section, we present a new approach for the identification of proteins via MS/MS based on feature matching. Feature extraction is performed on replicates of tandem mass spectra and only those features that were matched across replicates are used for peptide identification. In previous work, we have already shown that our 1D feature extraction algorithm can identify more significant peptides and fewer false positives in tandem MS (Noy and Fasulo 2007a). Here, we perform a similar analysis in order to evaluate our feature matching algorithms for the same application. We use the Aurum data set (Falkner et al. 2007) for this purpose. This high-resolution data set of known purified and trypsin-digested protein samples was generated on an ABI 4700 MALDI TOF/TOF with MS/MS acquisitions via shotgun tandem MS. Spectra were acquired for the eight most intense ions of four replicate wells. More specifically, in a replicate well, after excluding the seven most intense ions, the next eight most intense ions were analyzed. Similarly, the next set of eight ions was analyzed for wells 3 and 4 (Falkner et al., 2007). This process resulted in acquisition of a maximum of 32 spectra per digest, or theoretically 29 unique spectra. We randomly picked 20 proteins from this study and analyzed their corresponding set of tandem MS using our feature extraction algorithm (Noy and Fasulo, 2007a). Then, we used our feature matching algorithm (see Subsection 2.1 for more details) to match the extracted features across these non unique spectra; non-matched features were excluded from the downstream analysis. The rational behind this step is that features that were detected in more than one spectra and are matched across multiple experiments are more likely to be true positives.
The features detected by using our methods were submitted to the X!Tandem search engine for peptide identification (Craig and Beavis, 2004). We compared the results from X!Tandem before and after applying our feature matching algorithm. We also compare our results with the X!Tandem results for the peak lists provided in this study, generated using the Applied Biosystems GPS software (Falkner et al., 2007). Combining our feature extraction and matching methods, 167 peptides were identified by X!Tandem with the best score of −64.4. Using our feature extraction method, 173 peptides were identified by X!Tandem with the second best score of −63.9, while 162 peptides with the lowest score of −63.3 were identified using peak lists from the GPS software. Although all methods found the correct proteins, we found some evidence of increased sensitivity and specificity. In terms of sensitivity, three additional contaminant proteins—Trypsin precursor, KERATIN 1, and KERATIN 10—that are usually seen in MS experiments were detected by our methods. With the GPS software, only KERATIN 1 was found. In terms of specificity, four apparent false positive proteins were also found with the GPS software, and only three with our feature extraction algorithm and two with the feature extraction coupled with the matching algorithm. More detailed information and search engine results can be found in the Supplementary Material.
4. Conclusion
Feature matching and alignment are crucial in the analysis of MS experiments. In 2D LC-MS experiments, the inconsistent variation in the LC column results in nonlinear shifts in the LC RT axis which has to be corrected across different LC-MS maps in order to accurately match recurring components. Algorithms that can solve this problem efficiently and accurately will greatly benefit large-scale proteomics efforts, reducing the amount of “missing data” arising from poor alignment and matching across maps, and thus improving the statistical power of the data.
This study presents a novel shape-based feature matching algorithm that can be used for different MS applications. We tested our algorithm on an LC-MS data set in the context of a nonlinear alignment correction framework and on tandem mass spectra. First, we applied this procedure to the analysis of LC-MS experiments, which is particularly difficult due to the complexity of the data; we showed that we were able to find the expected spiked-in proteins in a complex sample background of human peptides. We compared our results with the SuperHirn software and showed that using our methods we were able to assign more peptides to the target proteins with higher probability and reproducibility. We also tested the performance of our feature matching algorithm on protein identification via tandem MS. Using this approach, we identified more significant peptides and fewer false positives than using the state-of-the-art software. This suggests that replicate feature matching can improve the identification of proteins via shotgun tandem MS.
Finally, the advantage of the feature matching strategy presented in this study is that it allows an easy incorporation of additional information like the shape information using the wavelet transform of the MS signal. In the future, this could be extended to incorporate additional information, such as the MS/MS spectra associated with the signal.
Footnotes
Acknowledgments
We thank Prof. Boaz Shaanan and Dr. Danny Barash from Ben Gurion University for their guidance and support.
Disclosure Statement
No competing financial interests exist.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
