Abstract
In this article we present, the conceptual and technical background of a tool for mapping and geo-computing, Painting with Data (PWD). PWD was inspired by early computer mapping algorithms that focused on drawing as an intuitive interface between spatial data and architectural practice. PWD embraces the potential of such techniques, coupled with alternative computational representations, modes of interaction, and computational interfaces, to encourage public participation in planning and design. Essential to this task, is PWD’s integration of open-source, collaborative, interactive, and web-based technologies to create an online software with a visually based approach to spatial analysis and mapping that dramatically reduces the steep learning curve required for GIS software. As a high-level graphical interface, PWD allows users to iterate by intuitively creating spatial models on-the-fly based on their subjective understanding of information. In PWD, we deploy voxels, a data structure that organizes information as 3D pixels, which allows users to compute with spatial information visually, to potentially inform the ways in which users build quantitative models. In PWD, multiple layers of information can be visualized concurrently, and visual correlations can be made instantly. Users build spatial models by directly manipulating the map itself instead of manipulating a database that then produces a map. PWD’s high-level interaction is made possible by custom data structures that leverage GPU processing, which makes them significantly faster than traditional topological data structures. Computationally, user interactions generate visualization specifications and declarative queries that are compiled and executed by the platform. Lastly, PWD introduces a visual programming language, which enables intuitive geospatial modeling and visualization. Practical work has shown the value of PWD’s approach to design students, planning agencies and community non-profits. Throughout the process, PWD’s open-source spatial models generated a user community with more than 3000 users, including designers, students, and educators.
Keywords
Major limitations of current tools for geo-computing
For planners and designers, current Geospatial Information System (GIS) tools present limitations in terms of accessibility, user interface, and data representation. For the last 40 years in North America, enterprise GIS have been developed to support information management practices and technically skilled users (Sandoval Olascoaga, 2021).
Enterprise GIS software is robust and high-performance, but its data structures and interfaces have made it difficult to express non-standard concepts, or methods that are common in day-to-day urban design and planning practice. For designers and planners, current GIS programs are generic, complex, inflexible, incompatible with pragmatic planning problems, and oriented towards technology rather than real world problems (Batty et al., 1999; Kahila-Tani et al., 2019). With interfaces that provide wrappers around databases, enterprise GIS programs present technical barriers to entry to visually trained practitioners, which has resulted in planning and design education slowly adopting spatial computing tools in their curriculums; even if taught, current approaches focus on using GIS programs to manage a spatial database, or program analytical applications (Chen, 1998).
Enterprise GIS licensing costs represent a significant financial barrier to entry for smaller companies or individuals; this is especially relevant for smaller city governments, and non-profits with limited financial resources (Göçmen and Ventura, 2010). Over the years, open-source alternatives like QGIS have become available, but those alternatives lack the amount of support that planners and designers require to adapt the software to their needs and day-to-day practice or require the use of commercial support (Yeh, n.d.).
In a design context, in the past 10 years, users have developed custom plugins for parametric CAD tools, such as Grasshopper, a visual programming language interface for McNeel’s Rhino, to integrate GIS datasets into design software. However, due CAD’s unstructured data model, its limited functionality to define data properties, and lack of spatial projection, these efforts have been limited to cartographic representation or involve technically complex operations to perform GIS-like tasks and analysis.
Painting with data: An alternative tool for urban computing
PWD is a GIS web-software that implements an alternative spatial data structure to improve accessibility and to provide an information model for computing visually with spatial information. PWD was created to provide visually trained users like designers and planners a tool that couples visual intuitions with functional computing. PWD’s visual-first methodology enables users to construct spatial models based on their personal knowledge and visual perception, which facilitates civic engagement in design, planning, and policymaking.
Voxels as an alternative geo-computational device method
PWD deploys a 3D voxel data structure that retains specific analytical capabilities of vector-based representations, while preserving the granularity and graphic expression of raster-based representations. This information model is visual-first: what you see is also what you compute with. Voxels represent multiple layers or dimensions of information as transparent, stratified pixel layers, which enables users to develop visual intuitions of spatial correlations among layers.
Conceptually, voxels are a 3D grid of points with an attribute value that is used to define the voxel’s graphic representation. Every distinct dataset is represented as a pixel grid with a Z-position defined by the order of the layer in the set of layers; in time-based datasets, the Z-position of the dataset is defined by the time dimension. Computationally, the data structure is represented as a multidimensional matrix, with a set of X, Y, and Z transformations (Foley et al., 1990). Graphically, PWD uses a direct-volume rendering to represent the voxel data structure, which is rendered as a 3D sphere with a color and a radius. Additionally, PWD’s voxels provide analytical functions to compute with 3D and 4D spatial datasets (Mennis, 2010; Mennis et al., 2005; Raper, 1989).
Voxel layers are built by sampling spatial information from vector and raster-based layers into a regular-field of points (Kaufman, 1994); in vector datasets, polygon-types and points can be used to construct voxels. This pipeline includes steps that (1) enable users to select a set of datasets to create a voxel; (2) transform every spatial dataset selected into the same spatial projection, which the user selects based on one of the datasets or specifies directly; (3) transform the datasets into a high-dimensional raster image; (4) sample the pixels of the high-dimensional image into 3D point grid; and (5) parse the values into GeoJSON. 1
Designing information
In PWD, mapping and analysis models are built by users designing and computing with information visually. The quantitative attributes of every dataset are used to parametrically define the radius of every voxel in every layer. At the same time, users assign every voxel layer a different color, Z-position to define the order in which layers are displayed in 3D, and an opacity value, all of which allow layers to visually merge with each other; users can come up with rules about how different layer colors combine (Knight, 1989). This allows users to compute arithmetic and Boolean operations visually and functionally like sum, intersection, and difference, among different layers, which blend the colors of different voxel layers into a new layer.
Through PWD’s GUI, users can interactively modify the parameters defining the layers’ graphic representations; the same datasets can yield different visual representations and help users arrive to personal visual conclusions. Users select and query voxels, modify colors, sizes, and opacity values to come up with personal interpretations of information. PWD’s GUI translates user’s interactions into complex spatial queries to select, highlight, or transform spatial features (Supp Figure 1).
Parallel coordinates and data drive charts GUIs
PWD implements a parallel coordinates GUI to visually explore relationships in high-dimensional data, and to perform data-driven queries. Users can interactively query the parallel coordinates graph to highlight in real-time the voxels that correspond to values selected. Additionally, the user can interactively reorganize the order of the different axes to discover correlations among different voxel layers (Supp Figure 2). For example, when most of the graph’s lines are parallel to each other, there is a positive correlation among data dimensions; when lines create an “X-like” shape, the correlation is negative (Inselberg, 1997).
Similarly, a bar chart and density graph GUI visualizes the voxel layers’ quantitative information and enables users to brush ranges of values in the charts to produce data queries and table joins (Supp Figures 3 and 4).
Visual programming language
PWD’s visual programming language (VPL) builds upon a lineage of graph-based interfaces to facilitate programming in CAD software and provides data-driven functions to transform existing voxel layers or create new ones. The VPL’s functions transform layers on-the-fly and result in immediate visual feedback of the operations performed by the user. The VPL constructs a graph that records the sequential rules and transformations defined by the user; these graphs can be reused in other PWD projects. The VPL is extensible through a pipeline that allows users to compile new nodes.
The VPL is composed by a series of nodes that represent voxel layers or function operators. The properties of a node are inherited and transformed by each children node in the graph. Every time a new node is connected to existing nodes, the new node creates a new layer with new visual and data properties (Supp Figure 5).
The VPL implements “Map Algebra” (MA) operations, which currently include arithmetic computations, statistical calculations, relational operations, and logic gates (Supp Figure 6). Voxels can be aggregated and computed in local, focal, zonal, or global neighborhoods.
In addition to 2D MA operations, the VPL implements 3D and 4D functions. The 3D and 4D functions allow users to work with spatiotemporal datasets; supports datasets include: one-dimensional in time, two dimensional in space and one-dimensional in time, three-dimensional in space, and three-dimensional in space and one-dimensional in time (Mennis et al., 2005).
Application outputs
Users can output: (1) graphics and drawings to aid in the design and planning process; (2) spatial models and VPL graphs; and (3) cross-platform spatial files. PWD model’s data can be exported as two different filetypes: GeoJSON, and CSV files. 2 Maps can be exported as static PNG, SVG, or GeoJSON files. GeoJSON files can be loaded into a variety of mapping software, or in design software. These files can be used to perform more in-depth spatial analytics or to inform data-driven design processes.
Web application implementation
PWD’s interface was developed through an iterative user-centered design process. Over the course of 3 years, we held formative interviews with intended users, including architectural designers, planners, academics, and non-profit stakeholders, to understand their design process and the limitations of their existing tools. These user evaluations asserted the importance of visual and real-time user interaction to create an intuitive interface.
PWD is implemented as web application with two different codebases running side-by-side in a user’s client (frontend), and in cloud servers (backend). This allows users to access the application without having to locally install any additional software, or code libraries. The application is an open-source project hosted on GitHub.
The backend servers manage the interaction with the database, create information, and perform heavy analytical computations triggered by the user. This codebase contains modules for data input, data-processing, and handling API requests and responses. The data input module checks the integrity and security of the geospatial files that are uploaded to the tool and saves them into a relational database, which manages the geospatial files, handles Coordinate Reference Systems, and constructs the Voxel data structure.
The frontend application is a reactive Vue.js and React.js application, which provides an interface for users to upload spatial information, create custom voxel projects, share models and data, and build spatial models through the main PWD application. The frontend module implements the PWD user interface, and functions to make API and database calls.
Finally, the frontend contains a custom voxel geometry library, which provides geometry functions to construct, display, and manipulate geometries in WebGL. The geometry library implements R-Tree indexing and renders geometries (Guttman, 1984). Users can blend the values of individual voxels based on a K-Nearest Neighbor algorithm. Other computational geometry functions include geometry translations, spatial Boolean operations, and spatial arithmetic.
The primary component of the frontend application is the PWD GUI, which combines a 3D voxel visualization with a tiled map-service. The 3D visualization is controlled by a series of UI elements and widgets that allow the user to manipulate the display properties of the voxels, to subset the layers, and to create new voxel layers. The frontend leverages GPU processing to enable real-time user interaction. Currently, PWD supports three main spatial datatypes: rasters, shapefiles, and GeoJSONs.
Reflections
PWD was designed to support intuitive, collaborative, and accessible geo-computing practices: users do not require technical expertise to construct spatial models. User feedback has suggested that in the design and planning disciplines, PWD fills in a gap left by current GIS interfaces. With PWD, users quickly learn how to interact with spatial models without any training in just a few minutes. Users can transform the graphic representations of many overlapping layers on-the-fly, and iteratively build complex graphic and analytic models through real-time feedback. Additionally, the voxel data structure offloaded the process of data aggregation, or spatial reprojection from the user.
Users have provided positive reviews for PWD’s interface, highlighting that: (1) voxel visualizations are intuitive and enabled users to make sense of multiple layers of information (Supp Figure 7); (2) visual charts are very easy to perform queries with; (3) the visual programming language provides an intuitive interface to construct analytical models (Supp Figure 8); and (4) the interface’s online and open nature allows users to easily share their data and models. In general, users have appreciated PWD’s very low-bar to entry and highlighted the potential of PWD’s open-source model to overcome usage barriers for organizations with limited budget for spatial modeling and visualization (Supp Figure 9). Finally, users highlighted that PWD’s output formats allow them to seamlessly share results between different types of design and GIS software and collaborate with other users. For example, different users have “forked” the same project to create their own models.
Current limitations and future work
PWD’s visual and interactive interface can introduce limitations in terms of visual and motor accessibility to users. Future work to improve accessibility includes interfaces to select high-contrast color schemes for voxel layers, informative tooltips, and increasing the keyboard usability of the interface.
Most PWD users don’t have a GIS background and are unfamiliar with potential data and graphic biases that can misrepresent information. For example, layer colors can have different meanings to different users, or combining multiple layers with diverging scales and units together can lead to false conclusions; to address this issue work in progress includes first, a graphic linter, that highlights model construction errors such as differing scales and areas, missing data, and perceptually ineffective color schemes; second, a library of models using the same datasets for users to compare; and a set of tutorials of effective graphic techniques for voxel mapping.
Voxels present a sampling and scale issue: the data structure is created by sampling values from a regularly spaced point grid. This poses two main issues: (1) after the algorithm is run, the point density, or spacing between points, can’t be modified, which result in resolution limitations; and (2) differing densities of voxels are analogous to the “modifiable areal unit problem” (Openshaw, 1984), where depending on the sampling density, voxels can result in varying data values and interpretations. Future work to test the validity of the visualizations will include backend approaches to construct different voxel densities interactively; work in progress provides interfaces for users to ground truth datasets and models by allowing them to modify datasets, and to upload mixed media and personal observations. Altogether, this future work could enable users to provide feedback of the biases and assumptions of datasets and models through PWD’s visual approach.
In terms of backend functionality, users expressed needs for larger file-size uploads, which is currently limited to reduce cost of operation. The cost of the backend infrastructure is a limitation for the long-term sustainability of the application; we anticipate that different models for hosting the application, such as a federated software architecture, can be implemented to distribute these costs.
Finally, to expand the ways in which users can participate in the planning and design process, we are currently developing an interface for users to draw and record spatial features easily and intuitively.
Supplemental Material
Supplemental Material - Painting with Data: Visually based open-source tool for geo-computing
Supplemental Material for Painting with Data: Visually based open-source tool for geo-computing by Carlos Sandoval Olascoaga in Environment and Planning B: Urban Analytics and City Science.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Massachusetts Institute of Technology (Sandbox Innovation Fund).
Supplemental Material
Supplemental material for this article is available online.
Notes
Appendix
Carlos’ current book project uncovers the computational and political history of computational tools for design and geo-computing and provides a critical framework to develop computational design tools to support community participation in design. Carlos received his PhD and SMArchS in Computation from MIT, a Master of Architecture from UC Berkeley, and a Bachelor of Architecture from UNAM in Mexico.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
