Fashion fingerprint is a compact feature vector for fashion items that can be used for tasks such as recognition, clustering, and retrieval of similar items. It is equally useful for both online fashion retailers as well as for physical apparel stores (with or without their online extensions). A related problem is understanding the apparel preferences of an individual from the dresses that they wear while visiting the physical store. One of the challenges of fashion study different from others is lack of enough accurate annotation. Available datasets have either limited number of images or very noisy annotation.

Currently we have successfully trained a fashion item localization model based on SSD[1]. The model is able to localize upper clothes, bottom clothes and one-pieces and has been tested on the Clothing Parsing dataset[2]. It achieves an F-score of 0.887 on upper clothes localization. For other clothing items, errors occur because the model may focus on too local regions and thus gets confused between skirt and dress. Prior location of human body will be incorporated in our model to solve this problem.

In the future we will further refine our localization model and also work on two directions. One is to recognize the garments based on our localization. The other one is to automatically label more images to enlarge the size of datasets.

[1] Liu, Wei, et al. “SSD: Single shot multibox detector.” European Conference on Computer Vision. Springer International Publishing, 2016.
[2] Liang, Xiaodan, et al. “Deep human parsing with active template regression.” IEEE transactions on pattern analysis and machine intelligence 37.12 (2015): 2402-2414.