Neo-Fashion: A Data-Driven Fashion Trend Forecasting System Using Catwalk Analysis

Fashion Trend Forecasting and Catwalk Analysis

A trend is defined as “a pattern or direction of change: a way of behaving or dressing that is developing or becoming more visible” (Holland & Jones, 2017, p. 50). A trend could be a popular item, a way of dressing or styling, or color combinations that are popular at a given time. Trend forecasting is gleaning what will inspire designers and other key influencers; and determining how these inspirations will then be turned into products that appears in stores and in consumers’ wardrobes. Trend forecasters mainly contribute at the beginning of the creative process. Based on trend forecasting research, fashion designers, producers, and buyers are able to predict what is likely to be “in fashion,” then create the product (Jackson, 2007). In addition, short-term and long-term forecasting are commonly mentioned when companies predict fashion trends. While long-term forecasting creates a shared vision for a brand or company’s development, short-term forecasting involves color forecasting, textile development, and style development throughout the whole product life cycle (Keiser et al., 2017).

Most trend forecasting services today focus on short-term forecasting, with trend forecasters usually working 1–2 years ahead of the season for most fashion collections (Kim et al., 2013). With the development of digital technology, especially the emergence of social media, some fashion-forward brands move quickly from the catwalk to the stores to meet the new trends within just a few months or even a few weeks (Stein, 2019). Trend forecasters use their knowledge of fashion design and their understanding of fashion history to conduct in-depth research and identify past and future trends. A variety of information sources are involved in trend forecasting research. From catwalk shows, popular culture, street style, retail data, art, and lifestyle, to digital culture and technology, trend forecasters look for ideas and inspiration to find out what is new and next (Keiser et al., 2017). The catwalk remains a crucial part of fashion delivery, and seasonal catwalk shows always play an important role in providing information on which designs are resonating most with the press and buyers. At the same time, catwalk reports and analysis are the most valuable trend services used by many trend agencies in the early inspiration stage (Holland & Jones, 2017).

Fashion Trend Forecasting in the Digital Age

In the past few years, fashion trend forecasting has received increasing attention within computer vision, machine learning, data mining and multimedia communities. Most studies related to trend forecasting utilize social media or google search to identify and predict trends in the fashion industry. For example, one previous study looked at posts and tags of the top-twenty fashion brands on Twitter and Instagram to reveal diversification of trends after linguistic and visual clustering analysis (Manikonda et al., 2015). Gabale and Subramanian (2018) identified Indian social media trends using an improved object detector model, which was pre-trained on unlabeled images from Facebook and Instagram in an unsupervised manner and then trained on Open Images V4 dataset. In addition, Checco et al. (2017) envisioned a fashion trend forecast system fed by images from Instagram’s well-known fashion bloggers. After processing data by combining image analysis algorithms with human crowdsourcing, the system was able to predict trends using times series analysis. Researchers also detected fashion trends and seasonal cycles by modeling clients’ reactions on websites to changes in style units over time (Sanchis-Ojeda et al., 2016). Queries related to the apparel category in Google search have also been used to identify fashion trends by time series clustering (Silva et al., 2019).

Trends in fashion are reflected through a variety of design elements, including color, style, silhouette, etc. However, most studies only focus on the prediction of one or two aspects of trends. Choi et al. (2011) built mathematical models to predict color trends. With the help of an object detection model, Gabale and Subramanian (2018) predicted the potential popular color and styles in India. Getman et al. (2020) visualized the patterns of baseball caps and tracked the trend frequency from 2000 to 2018. For fashion practitioners, it is more crucial, although more difficult, to predict the trending colors for each individual fashion item (e.g., color trends for shirts versus color trends for jean), rather than only predicting the overall trending color for an upcoming cycle. However, there has been little research up to this point that provides a systematic pipeline for efficiently extracting specific attributes and holistically predicting trends through given images.

Another fascinating research stream is related to popular image processing and classification technologies, although most studies in this area aim to classify and categorize fashion items, not necessarily to predict fashion trends. These methods have opened the door for summarizing key design elements. For example, one previous study used a supervised deep convolutional model to discover a “vocabulary” of latent styles using non-negative matrix factorization. Then a forecasting model was trained to represent trends in the latent styles over time and to predict their popularity in the future. Even though this work mainly focused on long-term forecasting, it provides a holistic view of the life cycle of visual styles in fashion (Al-Halah et al., 2017). Other researchers have presented novel approaches to recognize the category, sub-category and attributes of fashion images, from the high-end fashion website Farfetch through an end-to-end architecture by embedding the hierarchical nature of the annotations directly into the model (Ferreira et al., 2018). This new method identifies visual information in fashion images thoroughly, thus potentially helping in trend forecasting. Despite the importance of rich catwalk data, none of the previous studies or systems have focused on forecasting seasonal trends via catwalk analysis. A systematic tool is needed to categorize and analyze such images by following fashion forecasting principles.

Analyzing Catwalk Images With Computational Approaches

To analyze catwalk images, it is crucial to recognize and retrieve clothing items from the given images first. Since most of the original images have complex backgrounds and the clothing is worn by fashion models, it is challenging to distinguish clothing items from other objects in an image. This process is quite time-consuming and sometimes subjective if done manually. Therefore, computer-assisted image segmentation is brought into this type of research as an important data pre-processing step. Image segmentation is a process of partitioning a digital image into multi segments, typically used to locate objects and boundaries. Recently, with the development of deep learning in object detection, deep learning object detectors, such as Single Short Detector (SSD) (Liu, Anguelo et al., 2016), “You Only Look Once” object detector (YOLO) (Redmon & Farhadi, 2017) and Region Convolutional Neural Network (RCNN) (Sun et al., 2018), have been widely used in multiple domains. The quantity of labeled images for classification are much more than that of object-detection images. Therefore, YOLO uses a hierarchical view of object classification to utilize the classification dataset to detect objects (Redmon & Farhadi, 2017). SSD, a single-shot detector, detects objects in images by predicting bounding box offsets for a fixed set of bounding boxes (Liu, Anguelo, et al., 2016). The algorithm in RCNN attempts to selectively search the region proposals, and predicts the presence of an object within a region proposal, which usually is a bounding box (Sun et al., 2018). All these models have fast and accurate performance in object-detection tasks. However, to generate a more robust system for fashion trend forecasting, a general object detector may not be a promising solution since most of them only provide a bounding box of each object and it may keep the complexity of background within each bounding box. To solve this problem, Mask Region Convolutional Neural Network (Mask RCNN) (He et al., 2017) was developed by computer researchers to not only mark out object regions with bounding boxes but also extract object regions from the background at the pixel level. Yu et al. (2019) have proposed a strawberry fruit detection algorithm based on Mask RCNN, and the result has overcome the difficulties of poor universality and robustness. Vuola et al. (2019) have utilized Mask RCNN in nuclei segmentation tasks and found promising future application of this algorithm in the biomedical domain. The above research demonstrated that Mask RCNN showed a preferable performance for target detection in various unstructured scenes and areas. However, to the best of our knowledge, this cutting-edge image segmentation method has not been used in fashion image processing yet. This approach is very useful to analyze catwalk data and precisely recognize fashion items in a given image.

Trends in fashion are reflected through design elements. That is, trends have to be broken down to color, texture, fabric, silhouette, etc. For catwalk images, once clothing items are recognized and categorized, some key fashion elements can be extracted from these items. In this study, colors, styles, clothing combinations, and key attributes were analyzed and interpreted.

Module 1: Data Collection and Labeling Process

In this study, we mainly focus on analyzing catwalk images from 2019 collections across various famous brands in big four fashion capitals. Generally, fashion companies will show their collections for next year in fashion weeks. In total, 32,702 catwalk images were gathered from the WGSN database in 2018 to predict trends in 2019. To better forecast the seasonal trends, we separated all the images into four groups based on their corresponding seasons, including spring-summer, autumn-winter, pre-summer and pre-fall.

The first step is to recognize fashion items in catwalk images. Most original catwalk images feature the fashion models walking on the runway with images in the background blurred, including the audience and other models. Each image usually contains more than one fashion item. For example, one outfit may have a coat, a sweater, a skirt, a pair of boots and other accessories. Therefore, without any data preparation process, it is impossible to implement further analysis. In this light, Neo-Fashion system first develops a module to automatically segment fashion items from the original images. To achieve this goal, a sample dataset was prepared with 2,000 random selected images from the 2019 autumn/winter collection, to be labeled. All of the labeled data would be split into training data and validation data. For machine-learning projects, algorithms learn to find patterns from the input data. The input is referred to as training data. Once algorithms have learnt the underlying patterns of the training data and build the model, it needs to be validated on the validation data. In our case, if the model performs well on the validation data, the Neo-Fashion segmentation module could be considered as a machine-learning model that can segment fashion items from the whole dataset (32,702 catwalk images) and could be generalized to a much larger dataset.

Fashion major students who are knowledgeable about fashion styles were recruited for the labeling process to prepare the training dataset. LabelMe (Torralba et al., 2010), an open annotation tool, was used to assist labeling process. A detailed instruction and short training were provided to students who participated in the labeling process. For each given image, students were asked to outline each fashion item, and label the category and corresponding attributes of that fashion item, including color, texture, shape, and type. In total, out of 2,000 random selected images, 769 labeled images with accepted quality in 23 categories were returned to our server. These categories included: sneakers, leggings, shoes, vests, boots, skirts, belts, blouses, bags, cardigans, shirts, dresses, pants, hats, heels, accessories, trench coats, shorts, blazers, jumpsuits, coats, and jackets. Figure 2 shows the number of images for each season and indicates how students use LabelMe to complete labeling process.

OPEN IN VIEWER

Figure 2.

Data preparation stage.

Module 2: Instance Segmentation

For the Neo-Fashion system, we use RCNN, a state-of-the-art object detection algorithm in computer vision to detect the clothes on the model for further analysis. RCNN is a deep neural network aimed at solving instance segmentation problems. There are two stages of Mask RCNN. First, it generates proposals about the regions where there might be an object in the given image. Second, it predicts the class of the object, refines the bounding box, and segments the object (Vuola et al., 2019). Therefore, there are three types of output in Mask RCNN, namely, the bounding box, segment, and category of each fashion item in each image.

To apply this algorithm, first, we needed labeled training data to build the model and labeled validation data to prove the accuracy of the model. In Neo-Fashion system, out of 769 human-labeled catwalk images, 669 were used to train the Mask RCNN model and the rest 100 images were used for the validation of the robustness of model. The architecture of the network remains the same as the original Mask RCNN from (He et al., 2017). To generate segmentations more accurately, we doubled the weight of mask loss in the loss function of network. Meanwhile, some hyper-parameters were also adjusted. Optimizer was modified to Adam optimizer with 0.001 as initial learning rate. Anchors sizes were set smaller than original ones to catch small objects, such as shoes and heels. Other hyper-parameters were kept the same (He et al., 2017).

The computer scanned each training images, differentiated the outlined fashion items and the remaining background region, and “learned” the patterns of training images through the Mask RCNN algorithm. The patterns were then stored in the Mask RCNN model for our Neo-fashion system. Next, 100 validation images were tested to demonstrate the accuracy of the segmentation process of our system. Finally, 32,702 catwalk images were input into the Neo-Fashion system to acquire the segment, classification and bounding box of each fashion item, which was then utilized in the next analysis module. Figure 3 shows a sample result of Instance Segmentation Module. In the images of the generated bounding box part, the red dotted rectangles frame each fashion items, including coats, pants and dresses. The segments from two images consist of various categories of fashion items, such as dresses, shoes, coat, etc.

OPEN IN VIEWER

Figure 3.

A sample result of segmentation module.

Module 3: Fashion Trend Analysis and Forecasting

In the trend analysis module, two submodules are included. One is to forecast fashion trends based on 32,702 catwalk images, while the other is to analyze attributes of recognized fashion items based on 769 human-labeled images. As aforementioned, Neo-Fashion mainly focuses on three crucial aspects of fashion trend forecasting based on design principles, including color, style, and clothing combination.

For color forecasting, Neo-Fashion identifies the most prevalent color of each category and then decides the most trending color for each product category in a given season. Unlike traditional trend forecasters who mainly rely on their experiences and intuitions to analyze color trends, Neo-Fashion system helps reduce research time and bias through computer assisted analysis. To recognize the dominant color in each product category, a K-means clustering algorithm was applied to summarize the top colors across all images provided in 2019 collections. K-means clustering is a popular cluster analysis tool in data mining. In our research, each image consists of a large number of pixels, and each pixel has a color. We first choose a product category, then input the segments of fashion items in this category into our system. The K-means clustering algorithm then partitions all pixel colors of these segments into K clusters, in which each pixel color belongs to the cluster with the nearest means. The mean of each cluster serves as a prototype of the whole cluster (Klimenko, 2018). Through this way, the 10 most popular colors for each category of fashion items were identified.

Style and cloth combination are the other two major functions provided in Neo-Fashion system. To predict the potential popular styles for each season, we calculated the occurrence frequency of each category, based on the category classification result of 32,072 catwalk images from Instance Segmentation module. Given the input catwalk image, instance segmentation module using Mask RCNN outputs styles included in this image. Then the automatic summarization of frequency for each style is generated. In addition, with styles demonstrated in each image, if two or three fashion categories appear simultaneously with a higher frequency across the whole season, we consider it as a popular clothing combination in that season. To calculate the clothing combination, the tabulate data were prepared and generated by season. Then the co-occurrence frequency of the categories in dataset across all images in each season were calculated and sorted in the prepared tabulate data. The top clothing combinations then can be generated for each season. In the Neo-Fashion system, the dominant clothing combinations for each fashion season can be demonstrated to fashion researchers.

Even though Neo-Fashion aimed at three major domains to forecast fashion trends, at this stage of the study, it did not yet fully capture all aspects of fashion trends. Materials, patterns, prints, structures, textures, and other details still needed to be investigated. Therefore, fashion students were asked to label key attributes for fashion items as well during the data preparation stage. Because of the exploratory nature of this study, we only used selected catwalk images to demonstrate the potential application of analyzing attributes of fashion items in trend forecasting. To provide more accurate and understandable recommendation to researchers, the Neo-Fashion system generated word clouds of image attributes for each category of fashion item based on 769 labeled images. In a word cloud, the larger and bolder the word appears, the more it is mentioned in the dataset and the more important it is. Therefore, the attribute word clouds provide an intuitive way to visualize the trending attributes of each fashion category.