VisExpress : Visual exploration of differential gene expression data

Discover stage —- problem characterization and abstraction

Starting with interviews and observations of the current workflows of the BIO team, the Liaison (first author) subsequently collected relevant state-of-the-art systems based on her professional expertise as a bioinformatician and VIS expert. In the second step, the drawbacks of these systems were discussed and the problem characterization was refined. In the third step, the VIS team discussed these, concretized tasks and requirements, and improved the problem abstraction. The Liaison (first author) ensured in the whole process that the problem abstraction was still valid from the domain users’ perspectives.

Initial prototyping and expert feedback

The Liaison (first author) created a low-resolution prototype to receive feedback from the BIO team. This initial design enabled the Bio team to precisely point out important aspects that the system should cover which were translated and merged with the identified requirements.

Design refinements

Based on experts’ feedback, we stepped back to the design phase. In order to fully exploit the expertise of the four VIS team members, we took the following approach to create and implement design ideas: (1) every team member created a set of alternative solutions as paper mock-ups; (2) these solutions were selected, merged, and refined in a critique-and-creation round; and (3) we discarded or refined ideas by evaluating them against tasks and requirements. This entire process iterated until all VIS team members were satisfied. The matching of the mental model is one important point to support the gaining of insights with a visualization system. ⁴ The Liaison (first author), therefore, ensured in this process that the design matched the mental model of the domain experts.

Formative assessment and final design implementation

In this process, the VIS team improved design details based upon formative assessment conducted by the Liaison (first author) with one member of the BIO team. Functionalities of the system were explained and demonstrated. The constructive feedback led to design improvements and an optimized user interface to resolve some usability issues.

Summative assessment and design refinement

For the evaluation of our design, we performed a Pair Analytics study ⁵ with the BIO team in order to verify our design decisions for target tasks. Thereby, the Liaison (first author) acted as analysis partner in the collaborative analysis parts of the study. Based upon the evaluation results, we refined our system designs and reflected our findings.

Fingerprinting

Based on these considerations, we decided to represent the GAR patterns of each gene as a single entity (glyph-like) which we will name gene fingerprint. Our design goal of gene fingerprints is to provide a visual summary of a gene which can be used to compare the GAR patterns effectively (R1). The idea of fingerprinting is based upon the work of Keim and Oelke ²⁶ of literature fingerprinting. Each gene consists of a tuple of a GAR r k , l ( g i ) and a quality q k , l ( g i ) as well as functional description (plain text) for contextual information. Gene fingerprints should support identification and comparison of GAR patterns (R0, R1) and the assessment of quality (R6). Therefore, we discussed dividing the tuple into measure and quality in order to focus the visualization on the GAR measure.

The quality could be handled by threshold-filtering and/or details on demand such that only GAR patterns with a high quality are visualized. However, the BIO team preferred to see all genes and to perform quality aware analysis (R6). Even patterns with low quality can be interesting and there is no fixed threshold that can define interestingness which rejects the idea of threshold-filtering. The challenge is to find visual metaphors that can encode both GAR value and quality and also satisfy R0, R1, and R3 (interpret, compare, and explore GAR patterns). In the following, we discuss design alternatives for gene fingerprints.

Design of gene fingerprints

Due to the exploration requirement (R3), the visualization design has to be scalable. Highly scalable techniques are pixel-based visualizations such as recursive patterns ²⁷ or pixel bar charts. ²⁸ Therefore, the VIS team discussed several alternatives to visualize GAR patterns with pixel-based or pixel-cell-based techniques such as circular, ring, or matrix representations. As in the linear arrangement of a heatmap, identification of the involved comparisons is not effective for circular or ring representations which violates the interpretability requirement (R0) (see Figures 2 and 4). Matrices support the identification of the involved conditions since the matrix element at row x and column y indicates the activity ratio value of the xth condition and the yth condition (see Figures 4 and 5). Biologists can, therefore, interpret the GAR pattern of a single gene by inspecting elements of a matrix (R0). Subsequently, they can compare the GAR patterns between multiple genes by inspecting the distribution of patterns across multiple matrices (R1).

OPEN IN VIEWER

Figure 5.

Design alternatives for matrix visualizations of gene fingerprints. Four different measures to characterize a gene are illustrated for each design (see (c)). (a) Two triangular portions in a matrix representing the value (bottom left) and the quality (upper right) of a gene. (b) and (c) Stacked matrices with inner and outer rectangles encoding value and quality, respectively. (d) and (e) Two-dimensional colormaps for normal and dichromatic visions, respectively. The color mapping in (c) highlights high values and low quality.

Two symmetric or triangular matrices for value and quality

One solution is to visualize the quality of each gene as an additional matrix juxtaposed to the corresponding value matrix. Although this design may ensure more accurate perception of both values, there are some significant drawbacks: (1) it wastes valuable display space and (2) it is hard to visually align value–quality pairs. Therefore, this design does not guarantee effective inspection on the GAR and the quality (R6) by burdening biologists with cognitive efforts to find and check two locations for a single comparison. The VIS team, therefore, excluded this design.

Value and quality triangles

Similar to the aforementioned design, Figure 5(a) shows a design where each of two triangular portions represents the activity ratio and its quality, respectively. This solution was discussed among the VIS team and with the BIO team as well. We concluded that the cognitive efforts to find and check two locations for a single comparison are still a burden for the analysis.

Resizing matrix

A further possibility to encode the quality would be to encode the GAR ratio with color and quality with the size of matrix cells. However, this solution is not scalable and the saliency of patterns is highly dependent on the size and, thereby, on quality which might suppress important patterns in the data. The VIS team, therefore, excluded this design.

Stacked matrix

Another approach is to use a Stacked Matrix. This approach is inspired by work of Oelke et al., ²⁹ where a stacked resizing matrix is used to represent user opinions on printers. The Stacked Matrices in Figure 5(b) and (c) use the outer rectangle for encoding the quality and the inner rectangle for encoding the value. The size of the inner rectangle is fixed. The Stacked Matrix with two different colormaps perceptually separates the inner and outer rectangles. This design is different from Oelke et al. ²⁹ since the inner and the outer rectangle do not represent the same measure in our design and the size is fixed. The proximity between two values enables biologists to read the activity ratio and its quality accurately and, thus, it supports the interpretability (R0) and quality requirement (R6). However, this design may suffer when many fingerprints are shown in a small space. Thus, zooming and panning interactions should be used when the task requires exploration of many genes (T1–T4; see also sections “Limitations and future work” and “The size of gene fingerprints”).

Colormap design of gene fingerprints

In addition to the matrix structure, colormaps should be carefully selected because they encode the activity ratios and qualities in our design. The selection of colormaps impacts upon the performance of all tasks (T1–T4) because our visual cognition system is steered by several attention effects. Our vision tends to focus on strong contrasts especially when colors are fully saturated and intense on dark backgrounds. ³⁰ Warm colors will suppress cold ones if they are spatially close. ³¹ Therefore, lightness, saturation, and temperature of colors must be considered. ³⁰ For interpreting (R0) and comparing (R1) GAR patterns as well as to assess the quality, the analyst performs the elementary analysis task of comparing encolored values and qualities. Following the guidelines of ColorCAT ³⁰ for specific, as well as combined, analysis tasks, we use perceptually uniform colormaps (value: black to green; quality: grayscale) for this elementary comparison task. This colormap choice supports to pre-attentively perceive value and quality differences. Furthermore, the values appear more prominently in comparison to the qualities which are encoded with a perceptually uniform grayscale.

One might also consider using the same colormap for activity ratios and qualities (see Figure 5(c) upper matrices). Due to the Gestalt Laws of Similarity and Pragnanz, we perceive regions of similar color as a whole large rectangle, instead of several stacked rectangles with different shades of green (see Figure 5(c)). This supports the detection of row and column patterns (R3) which are important in the tasks of building and associating groups (T1–T4). This design alternative of a Stacked Matrix has a higher scalability and can, therefore, be used in overviews with larger amounts of fingerprints.

Two-dimensional colormap matrix

Two-dimensional (2D) colormaps can also be used as illustrated in Figure 5(d) and (e). However, 2D colormaps are not suited for accurate value perception, ³² but these colormaps support the quick assessment of quality differences between different genes (R6) in data exploration (R3). Thus, it is recommended to use this where biologists want to quickly estimate values of multiple genes with a reasonable accuracy (R3). Furthermore, one should note that two-dimensional colormaps fail to function as intended for people with color vision deficiencies. Addressing this issue, we used opponent chromatic channels to encode the dimensions (normal: red-green, dichromatic: blue-yellow). As illustrated in Figure 5(d) and (e), the lower left matrix is clearly different from the other matrices. This is extremely useful to compare GAR patterns with the quality in mind (R6) which is only partially supported by other designs. Furthermore, this design is highly scalable in overviews of vast amounts of fingerprints (see Figure 14).

Triangle versus symmetric matrices and reordering

The Stacked Matrix and the Two-dimensional Colormap Matrix designs can be used with a full (symmetric) matrix or even a triangle matrix since half of the matrix comparisons are redundant. The advantage of a triangle matrix would be to save the space of redundant information. However, after a series of discussions among the VIS team and a consultation of the BIO team, we concluded that a symmetric matrix strengthens the visual saliency of patterns. The BIO team perceived the pattern in Figure 8(b), for example, less salient than that in (a) even though the two figures show the same pattern. Furthermore, some patterns might appear more interesting than others with the symmetric layout (e.g. the cross in (a) appeared more interesting than in (d) for the biologists on the first sight). However, the BIO team always reflected the meaning of a pattern and had no concern to realize that (d) reflects the same pattern as (a) (one condition is different to all others; condition 1 for (d) and condition 4 for (a)). The rows and columns represent specific experimental conditions which need to be maintained as references in order to assess other matrices. Therefore, the idea of the VIS team to use ordering emphasizing interesting patterns was rejected. Inconsistent ordering may confuse biologists to interpret the comparison of results between multiple genes (R0, R1).

Alternatives for cluster overviews

In order to build sets of genes with similar GAR patterns, heatmap-based approaches such as used by TM4 ¹⁰ , Mayday ¹¹ , and INVEX ¹⁴ use clustering. Genes naturally form hierarchical clusters if the genes operate with the same regulatory mechanism (regulon). In heatmap-based visualizations, the hierarchical clustering is used to order rows and a dendrogram is visualized next to the heatmap (see Figure 2). However, this representation does not clearly show which different clusters exist in the data set since (1) clustering is ill-defined and, therefore, clusters are often not visually separable and (2) small clusters might be overlooked. Thus, these approaches do not fulfill the comparison and exploration requirements (R1, R3).

There are space-filling visualization techniques such as self-organizing maps (SOMs) or treemaps that can be used to overview gene clusters. However, SOM clustering does not preserve the natural hierarchy. Large clusters will span over large parts of the map, whereas small clusters are suppressed. Furthermore, the creation of cluster centroids will refine the centroids of big clusters; however, suppress centroids of small clusters such that interesting GAR patterns of small clusters are consumed. This violates R0, R1, and R3 (interpretability, comparison, and exploration).

Treemap overview

We choose to visualize groups of genes with a squarified treemap ³⁵ showing the hierarchical clusters. The number of cluster items is encoded by its node size. This enables to assess the importance of clusters and also small clusters are preserved. Inside the treemap, either a centroid gene fingerprint of the corresponding cluster or a textual representation of the gene functions in this cluster is shown (see Figure 9). The representation of centroid gene fingerprints allows an overview of GAR patterns as well as their comparison (R1). The textual representation allows relating the GAR patterns with the gene functions (R2) and to relate clusters with hypotheses (T2).

OPEN IN VIEWER

Figure 9.

Treemap visualization for representing the hierarchical clusters of the genes. The clusters are either characterized by the centroid fingerprint or a word cloud of the functional categories of the genes (“no COG”: no functional categories assigned). The saturation of the cluster colors encodes how much variation exists within the cluster. Users can interactively drill-down the hierarchy or open the Gene Fingerprint View to explore one cluster.

A straightforward solution to visualize gene functions would be a list of words ranked by frequency. However, there is a large number of different functions in gene clusters which need to be summarized (R2). Therefore, a scalable approach is required. Word clouds are frequently used as visualization technique to aggregate and visualize textual data (e.g. see Wordle ³⁶ or Bateman et al. ³⁷ for guidelines). Furthermore, word clouds have already found their way in the biology domain.^19,38 We use the R package wordcloud. ³⁹ The BIO team preferred the encoding of the word frequency by size in word clouds as they could easily spot the most prominent words (functions) as well as get an overview of the distribution of functions (including outliers) which is important to derive a conclusion (R2).

The clusters and hierarchies are separated with categorical colors that share equal lightness and saturation based on guidelines of Healey ⁴⁰ and Harrower and Brewer ¹³ to prevent any attentional steering effects. We also provide a linear blend around borders to offer cushions to guide users’ attention through the hierarchy according to van Wijk and van de Wetering. ⁴¹ In order to indicate the quality of the current clustering, we encode the variance within the clusters with saturation of the categorical colors. Saturated colors indicate high quality (low variance) and gray colors indicate low quality (high variance) which implies that these clusters should be refined. We enable the user to drill-down the cluster hierarchy interactively (see section “Interaction design of VisExpress”).

Alternative layouts for gene fingerprint overviews

One way to structure the view is a sorting by interestingness function. For instance, by sorting gene fingerprints by their GAR values and/or their qualities, or the similarity of GAR patterns. The selection of the interestingness function depends on the analysis task and can be changed by the user on-the-fly (see section “Interaction design of VisExpress”).

Using an interestingness function allows several alternatives for a structured layout. The most straightforward alternative is, for instance, to layout fingerprints line by line according to the interestingness. However, this does not preserve local proximity (e.g. the two first objects of the first and second rows are spatially close but very distant in the interestingness or data similarity). Hilbert curves ⁴² preserve local proximity but cannot guarantee a globally ordered layout since curves might start and also end at the top depending on the number of objects. This violates intuition because intuitively all interesting genes are on the top and the least interesting ones are on the bottom.

Layout of gene fingerprints

We used the recursive pattern algorithm of Keim et al. ²⁷ that is particularly suitable to arrange sorted data points in dense pixel displays. This algorithm lays out the pixels with recursive levels of arrangements (hierarchical “Z”-arrangements) that have specific widths and heights. Thereby, recursive patterns can preserve local proximity and global (intuitive) interpretation. Recursive patterns can guarantee to show the interesting GAR patterns on the top area and similar patterns in proximity.

As shown in Figure 10, the system arranges the fingerprints on the first level by four columns to the right, one row down, four columns left, one row down, and four columns right to complete the “Z.” This pattern is then repeated three times to the right and then three times to the left in the lower row. In each level, the ordering of the interestingness is preserved which preserves local proximity and (intuitive) interpretation of the whole layout (top: the most interesting ones; bottom: the least interesting ones). A disadvantage of the technique is that parameters of the algorithm have to be selected in advance. The problem is to find a good combination of widths and heights (e.g. four steps in the example above) for each recursive level. Keim et al. ²⁷ suggest determining the arrangements by interaction. However, this would disturb the exploration process and we decided to determine the parameters automatically by applying an optimization algorithm to this combinatorial problem (see Supplement Material).

OPEN IN VIEWER

Figure 10.

(a) Overview of gene fingerprints. Matrices are sorted according to the interest of the user and layouted in recursive patterns. ²⁷ (b) Users can add interesting genes to a remember list for later inspection.

Instruction

Each participant entered the user study room separately which was reserved within experts’ workplace. The participant sat down next to the experimenter with a notebook and one monitor (24″ LCD). The experimenter provided detailed instructions through a slideshow presentation. Details such as visual representations, underlying data, measures, and interaction capabilities were covered so that participants could use the functions later on.

Introduction to the system

In the introduction, the experimenter asked a set of predefined easy questions for each level of VisExpress (Cluster View, Gene Finperprint View, and Gene Board) to make sure that the participants understood the views, graphical representations, and interactions. For example, the participants were asked “Which cluster has the largest GAR?” Furthermore, design adjustment possibilities were demonstrated. In this step of the study, the experimenter operated the system and participants were allowed to ask questions to clarify any uncertain areas. See Supplement Material for more details.

Open-ended exploratory part

After participants had completed all given tasks, we asked them to freely explore the data set which was the main part of the study. The participants were asked to verbally formulate, confirm, or reject hypotheses during the analysis process and to report interesting or unexpected findings along the way. The experimenter encouraged the domain experts also to focus on patterns which appeared interesting to her as a bioinformatician to facilitate a more collaboratively exploration of the given data and to generate deeper conversation about the biologists’ analytic activities, their reasons, and intentions. However, the experimenter made sure not to unduly influence the analysis by only suggesting a deeper look in a few cases and, otherwise, only acting as an active listener who did not initiate conversation unless she wanted to clarify unclear motivation or action (e.g. “why?” or “how?”). As participants had no issues using VisExpress and since user interaction was quite high, the experimenter decided to let the domain experts operate the system themselves.

Coding procedure

We followed a top–down and a bottom–up approach. Our goals were (1) to reveal the domain expert’s workflows with the VisExpress system, (2) to clarify expert tasks, and (3) to specify areas for improvements. First, the experimenter of the Pair Analytics study formulated findings from study impressions and verified them with corresponding clips of the video material. A second author checked against these findings with the corresponding clips. Second, the experimenter coded the whole video material. The video material was first annotated and split into clips according to the different used views (Cluster View, Gene Fingerprint View, Gene Board). For each clip, the experimenter coded the participants’ analytic and visualization activities. In particular, the attempt was to reveal the reason behind the participant’s actions and workflows that lead to findings. From this analysis, the experimenter formulated further findings. The findings were verified with the clips by the second author.

B1 discovered that membrane proteins are disseminated between different clusters

B1 started the analysis in the treemap Cluster View with the inspection of cluster centroids and the according word clouds (by hovering over the clusters one by one). Participant B1 observed many membrane proteins in cluster “condition 4 high” and cluster “condition 1, 5, and 6 high”. Such patterns (relations of different conditions) are strikingly visible with our gene fingerprints which are easily overlooked in the state-of-the-art representations where just (1:n) comparisons are shown. After looking for the gene product names by hovering over the gene fingerprints in the Gene Fingerprint View (see tool-tip in Figure 13), B1 concluded that in the cluster “condition 4 high” more transporter genes are present. Transporters are located in the membrane to transport, for instance, nutrients into the cell. An increase in transporters is reasonable since condition 4 is a stationary phase condition and, thus, nutrients are reduced in the medium run and it would be important for the bacteria to increase membrane transporters to get a better yield. In the cluster “condition 1, 5, and 6 high,” B1 observed more membrane proteins related to stress. This is an unexpected finding since condition 1 is the control/reference condition. B1 had no explanation why these membrane proteins should react as in conditions 5 and 6 but mentioned that it would be interesting to analyze this surprising fact in detail. To rule out false positives, B1 tried to reject the finding by inspecting the genes in the Gene Board (e.g. B1 tried to verify if the expression signal is just an artifact and the gene is not active under all conditions). Since this finding seems not to be an artifact, further analysis steps are required beyond VisExpress, for example, a literature analysis about the genes in this cluster to check whether such a correlation was observed before.

B2 quickly discovered low pH-responding genes

B2 discovered in the treemap Cluster View several cluster representatives with gene fingerprint patterns which indicate that several genes are similarly regulated in low pH (acidic) conditions but have no or negligible differences between other conditions (conditions 5 and 6 are low pH (acidic) conditions). By concentrating on this pattern, B2 discovered several genes annotated as “hypothetical” by browsing the tool-tips and GAR patterns of genes in the Gene Fingerprint View of the corresponding cluster. He added interesting representatives of this finding in each cluster to the journal for later inspection. His aim for further analysis was to examine these genes for their low pH (acidic) response. In order to rule out false positives, B2 analyzed the functions of genes with the same pattern (located in the same cluster) and inspected the genes in the Gene Board. In summary, the regulation of the acid-responsive genes appears to be more significant than expected by B2 based on today’s literature. The advantage of VisExpress for this finding was that the world clouds allowed an intuitive relation of the cluster to the gene functions. Thus, the word cloud allowed identifying that some genes in the cluster are annotated as “hypothetical” which was then analyzed further by B2 in the Gene Fingerprint View. The further required analysis step beyond VisExpress is a literature analysis to verify the finding of B2. Furthermore, a BLAST search could be performed to check whether related sequences in other species have been annotated with an acid-responsive function.

B3 rediscovered that there is a relation between experimental conditions 5 and 6 and iron

The data set has been analyzed by B3 before. One aim of B3 was to analyze which genes are influenced by supplement B (condition 5 vs. condition 6). In order to rediscover findings related to this supplement, B3 explored the treemap Cluster View for clusters with differences between conditions 5 and 6. By inspecting genes in the corresponding Gene Fingerprint Views, B3 discovered several genes related to iron. B3 reported that she had checked this correlation in the literature and found studies describing this correlation as well in Escherichia coli which is a species related to the analyzed species Salmonella Typhimurium. A relation to iron is reasonable since condition 6 is a stress condition which affects iron–sulfur cluster-containing proteins.

Ribosomal genes are enriched in a cluster with down-regulated GAR values in condition 4

This enrichment was observed by all participants. In the treemap Cluster View (see Figure 13), three clusters with a cross-pattern of condition 4 are revealed. By mouse over and inspection of the corresponding word clouds, the participants discovered that “Translation” stands out in one of the clusters. A closer inspection of this cluster in the Gene Fingerprint Views and the Gene Board revealed a down-regulation of a high number of ribosomal genes (belonging to the functional class “Translation”). This finding is not surprising because condition 4 is a stationary phase (see Figure 14). Bacteria move into stationary phase if their habitat does not allow a further increase in the population size due to space and low nutrient availability. In this state, bacteria slow their metabolisms to conserve energy. Consequently, less ribosomes are needed which produce proteins (encoded by genes). This cluster of down-regulated ribosomal genes could be excluded from now on, reducing the data set to more interesting and biologically relevant functions (other than growth speed).

Participants found several patterns they could not explain

B1 observed that several genes with the same function occurred in a cluster where condition, 1, 2, and 5 stand out. Detailed analysis with the Gene Board revealed that condition 5 is up-regulated, conditions 1 and 6 are slightly up-regulated, and condition 2 is down-regulated. The genes are related to a substance which is added in conditions 3, 4, and 6. The reaction pattern was, therefore, not explainable and surprising for B1. Such complex patterns were intuitively perceived by our experts due to the gene fingerprint design. Furthermore, VisExpress enables to inspect the functions of genes by demanding word clouds or the detailed Gene Board. The experts can query for a comprehensive view of such unexpected patterns more efficiently than in the state-of-the-art tools which would require the analyst to perform additional workflows. Such findings are especially interesting in an open-ended/hypotheses free data exploration because they are starting points for new hypotheses and further research.

General workflow of participants

We observed the same general workflow among the three participants. They started with the Cluster View and selected a cluster to analyze further. In order to decide for a deeper analysis of a cluster, the cluster representatives were inspected as well as the corresponding word clouds. In the Gene Fingerprint View, participants selected genes to analyze in detail with the Gene Board. Genes were selected according to their GAR patterns, their quality, and their functional category provided by tool-tips. With the Gene Board, participants assessed the trustworthiness of the GAR pattern. For example, if the pattern is surprising for the function, a closer look can reveal that the strength of the gene activity levels is too low to trust the GAR pattern. After the inspection of all interesting genes in the Gene Fingerprint View, participants switched to the Cluster View and looked for the next cluster for further exploration. The outcome of an analysis session is a list of genes of interest which can be checked with literature research and database comparisons. B2 states about next steps: “I would look up the genes at NCBI, Uniprot, perform a similarity search with BLAST and do a literature research” (ft). Consolidated hypotheses could then be verified by further experiments.

All participants used the quality to reduce the search space. They did not pay much attention to genes where all GARs had low quality after they were convinced that the quality is really an indicator for trustworthiness (checked with the Gene Board). However, they still inspected low-quality genes later on if the pattern was of interest.

Individual analysis processes and findings

The following paragraphs quote and describe different examples of the analysis processes and findings of each participant in detail.

B1 said: “I will successively look at all clusters.” The word clouds were used to get an idea about the included functional categories in a cluster. For example, B1 said: “In this cluster should be [supplement A] depended genes.” and for the corresponding word cloud: “Energy production and conversion stands out. This is reasonable. [Supplement A] is an energy supplier.” (fts). B1 also systematically checked gene functions by hovering over at least the first lines of gene fingerprints in each cluster (high-quality ones). B1 explained: “I am looking for the gene functions. It is striking that most genes have a functional annotation, this was not the case for some other clusters.” (ft). Genes with interesting functions were inspected with the Gene Board. B1 tried to gather findings for each cluster and explained whether he had expected them. For example, B1 said: “Many genes are related to the cell membrane. I interpret this as extrinsic stress. I am surprised that condition 1 and condition 5 and 6 are similar.” (ft; remark: conditions 5 and 6 are stress conditions but condition 1 is not a stress condition).

B2 built a hypothesis about the data set at the beginning and looked for the respective patterns. A hypothesis about, for example, only small differences between conditions 1 and 2 was rejected: “It is a surprising finding that [supplement A] has an effect on quite a number of genes. […] I have not expected this.” The word clouds were less frequently used by B2. After he had checked a few hypotheses, he checked random clusters with interesting patterns and /or interesting word clouds. B2 also compared similar clusters, by arranging the Gene Fingerprint Views of two clusters next to each other. In the Gene Fingerprint View, B2 randomly hovered over genes to get the functional categories; he tended to focus more on varying patterns. For example, B2 said: “These are the acid genes. However, this gene stands out. This is obviously a gene reacting on acid and [supplement B] stress only.” Genes with interesting patterns or functions were inspected with the Gene Board. B2 gathered findings for some inspected clusters and explained whether they confirm or reject his hypothesis. For example, B2 said: “I have no explanation for this pattern. Standard condition and a condition in stationary phase [1 and 4] behave similar. I have no idea what these genes should have in common.” and about the corresponding word cloud: “Ah … mostly no functional prediction. Thus, also others could not classify these genes.” (ft; see Figure 13).

B3 had analyzed the data set before. On one hand, she tried to rediscover her findings and on the other hand, she inspected clusters with an interesting pattern or an interesting word cloud. For example, B3 said for one cluster: “Here we have no difference between conditions 4 and 5 but between most others. I also realized that in my former analysis.” and for one gene in this cluster: “I found exactly this gene in my own analysis. A database and literature analysis revealed that this function has not yet been experimentally verified for this organism. The annotation is only based on a low sequence similarity.” B3 also looked more systematically at the genes in the Gene Fingerprint View and hovered over at least the first part of the genes in each cluster (high-quality ones) to check the functional categories.

Recursive pattern layout

The recursive pattern layout preserves a global ordering as well as local proximity. We found that local proximity was appreciated by the domain experts in the Pair Analytics study to search for neighboring genes with similar gene fingerprints. In these cases, participants browsed the names of all surrounding gene fingerprints of a gene of interest in spiral fashion. It was, therefore, intuitive for the biologist to search for the most similar gene fingerprints in the surrounding instead of browsing down the line in line-by-line layouts.

However, we found in our Pair Analytics study that participants often inspected all gene fingerprints or they inspected all gene fingerprints with good quality. The global ordering preserved by the recursive pattern layout helped the participants to concentrate on the gene fingerprints with a good quality. However, in the case that domain experts inspect all genes of interest sequentially, it is more intuitive to layout the gene fingerprints line by line than following similar neighborhoods in z-fashion. Therefore, we provide an option to arrange the gene fingerprints line by line such that the recursive pattern algorithm layouts the fingerprints sequentially to the right and on the next line sequentially to the left. This avoids that the analyst’s eye has to jump from line to line (right to left) and allows to sequentially browse the fingerprints.

Gene Board increases trust in the visualization metaphors

The direct integration of quality in the data analysis process was new to the participants. In order to assess the trustworthiness of the quality representation within the gene fingerprint matrices, participants inspected several Gene Boards in which the gene fingerprint matrix indicated low quality. Due to the detailed view of the raw sequencing data provided by the Gene Boards, participants concluded that the visual representation is correct.

Expressiveness of the evaluation

We evaluated the design of VisExpress with a Pair Analytics study with three domain experts and a real data set. This evaluation demonstrated the usefulness of the system. We claim, furthermore, that the results of the Pair Analytics study demonstrate that the design is in-line with the mental model of the domain experts. This comprises the general top–down workflow from treemap overview (Cluster View) to the detailed Gene Board view as well as the single visualizations. Especially, the specific request to add the functionality to drill-down further in the cluster hierarchy by clicking on a cell in the treemap showed how well the abstraction and aggregation of the Cluster View and gene fingerprints design was in-line with the mental model of the domain experts. From this point of view, the design is effective for our domain experts for whom the system was designed. However, we cannot claim that the design is effective in general since each domain has its own mental model.

Synergy effects

During this design study, the first author gained a deep understanding on the NGS data preparation process. She was able to estimate and formulate sources of errors from the computer science point of view that lead to data uncertainties. This understanding and description led to a biological research project proposal dealing with uncertainty introduced in the NGS data preparation process and is now funded the German Research Foundation. Furthermore, the VIS team had doubts about the common practice to calculate gene activity and gene expression values instead of analyzing the read coverage data directly (see Figure 1(b)). Here, for instance, methods for comparing time series could be applied which is again an interesting topic for VIS experts.

Do not underestimate biologists

Visualization experts often suggest fancy visualizations in the first place and have to realize in the end that a combination of state-of-the-art techniques is sufficient and gains a better acceptance by domain experts. However, a first refusal of sophisticated visualizations does not mean per se that everything should be simple. Domain experts can often surprise how well they also understand complex concepts. For instance, in a series of discussions, the first author realized that the BIO team had no issue with understanding the cluster hierarchy in treemaps; they even wanted to interactively drill-down in the cluster hierarchy and explicitly suggested splitting clusters on demand to identify interesting GAR patterns deeper in the cluster hierarchy (R3) (see section “Components of VisExpress”). This was surprising since even VIS students have often problems to understand the hierarchy in treemaps in the beginning. Furthermore, the experts demanded to sketch a GAR pattern to search for similar patterns. The first author had suggested a search by sketch functionality in another context. B2 remembered this and remarked he would like to look for patters that match his (sketched) hypothesis. B1 and B3 agreed that this would be a helpful functionality.

Find a good level of abstraction

To tackle complex problems, abstractions are needed. However, it is often hard to find a good level of abstraction. For instance, in this design study, we visualize GAR values with a green color scale. However, one biologist mentioned in the Pair Analytics study that a binary representation (green: any value, black: no value) would also be sufficient for some tasks. This would ease some analysis processes. However, after we had explained the bias of automatic thresholding, the domain expert saw the danger of this approach and withdrew the request. We, therefore, argue to work closely with domain expert to determine a good level of abstraction. One should ask for all parts whether a further abstraction is reasonable. However, one should also question abstraction requests for their meaningfulness since biologists sometimes tend to abstract too much. Furthermore, one should keep in mind that a reasonable abstraction level might also depend on tasks in mind.

A visualization expert with application domain knowledge helps to bridge the gap

Misunderstandings in the requirement analysis lead to high costs if they are recognized late in the design process. However, avoiding misunderstandings is challenging especially in design studies with molecular biology. The large knowledge gap between molecular biology and visualization leads to an interdisciplinary communication issue. ³ The language between both domains differs strongly, and to learn the language of the other domain to bridge the knowledge gap requires a lot of time. A cooperation partner with application domain knowledge can reduce learning time by directly abstracting and translating application problems to visualization terms. The Liaison role ³ of the first author highly reduced misunderstandings and development time in this design study and, furthermore, led to a comprehensive understanding of the problem domain. Moreover, the background knowledge of the first author was beneficial for the Pair Analytics study. She acted as an informed analysis partner, facilitating proficient discussions with the domain experts.

Dimensionality

We have applied VisExpress on a data set with a maximum of six experimental conditions resulting in six rows and columns in the matrices. This is a reasonable number but also data sets of experiments with a higher number of conditions exist. In these cases, it is harder for the analyst to determine which conditions form the patterns due to the high number of rows and columns. To test which number of conditions can still be distinguished, we performed a perceptual study in section “The size of gene fingerprints”. From this study, we can conclude that 12 conditions can still be distinguished with reasonable accuracy. For further conclusions, a more rigorous user study, including different patterns, matrix sizes and, for example, the influence of matrix cell borders, is desired. However, such a study is beyond the scope of this article. Alternatively, details on demand, revealing the involved conditions in the analyst’s focus, could be integrated.

Scalability

Regarding the limits of scalability of the Gene Fingerprint View, the existing interaction possibilities can reduce the number of gene fingerprints per Gene Fingerprint View. First, users can select gene fingerprints and open the selection in a new Gene Fingerprint View. Second, users can split large clusters in the treemap Cluster View (see Figure 12 arrows 2 and 8, as well as section “Interaction design of VisExpress”). Regarding perception, the readability of gene fingerprint matrices is reduced below a cell size of approximately 0 . 1 ∘ of the visual angle per matrix cell (see section “The size of gene fingerprints”). This threshold, however, depends on the display size, the resolution, the viewing distance, and the contrast sensitivity of the user. Since this is different for each user and setting, we integrated in our latest version of VisExpress that the minimum cell size can be adapted by the user. We then determine the size of matrix cells in advance and split large clusters in the Cluster View if the cell size in the according Gene Fingerprint View is below this threshold.

Analysis of gene function enrichments

We focused for this design study on the visual exploration of DGE patterns. The relation of GAR patterns to the functions of genes is revealed in VisExpress by word clouds. To further enhance the analysis of gene functions, we plan to integrate functional and gene set enrichment analysis (see Hung et al. ²⁰ for an overview). Besides statistical analysis of “unexpectedness” of gene functions, this also requires a tightly integrated expert for justification with visual analysis tools since “expectedness” depends also on implicit domain expert knowledge and is, therefore, ill-defined. A similar problem and solution was presented by Mittelstädt et al. ⁴⁸ that required a tightly integrated physician for adverse drug event detection.

Support of collaborative analysis

During our Pair Analytics study, we observed that some findings were interpreted and judged differently by the domain experts. A possibility to facilitate the individual differences is to capture, present, and communicate analysis results among the colleagues. This would also support the verification loop of the knowledge generation model for visual analytics. ⁴⁵ Also the externalization of findings and insights in general plays an important role in knowledge generation.

Support for bottom–up analysis

Our design specifies a top–down analysis for exploration. Analysts start with a cluster hierarchy and narrow down the subject of analysis. Participants stated that they would also like to have the opportunity to start an analysis with a set of interesting genes, for example, genes that are known to respond on acid. The system should import a list of genes with a similar reaction (provided by the analysts) and expand this set of genes with new similar candidates. A similar approach was presented by Bertini et al. ⁴⁹ to explore large chemical libraries and van den Elzen and van Wijk ⁵⁰ to explore multivariate networks.