Hello everyone!

I am very excited to announce Marqo-FashionCLIP & Marqo-FashionSigLIP – two new state-of-the-art multimodal search and recommendation models in the fashion domain. The models have outperformed the current SOTA models FashionCLIP2.0 and OpenFashionCLIP on 7 fashion evaluation datasets including DeepFashion and Fashion200K by as much as 57%.

Marqo-FashionCLIP and Marqo-FashionSigLIP are 150M parameter embedding models that:

• Outperforms FashionCLIP2.0 and OpenFashionCLIP on all benchmarks (up to +57%).
• Are 10% faster for inference than FashionCLIP2.0 and OpenFashionCLIP.
• Use Generalized Constrastive Learning (GCL) with SigLIP to optimize seven specific fashion aspects including descriptions, titles, colors, details, categories, keywords, and materials.
• Benchmarks were performed on 7 publicly available datasets and 3 tasks.

https://preview.redd.it/8kkyn2e61mid1.png?width=1459&format=png&auto=webp&s=e8bfa42faca752538b92e06e3dba3a7780007981

We release Marqo-FashionCLIP and Marqo-FashionSigLIP here under the Apache 2.0 license.

Benchmark results

Here are the results for the 7 datasets. All values ​​represent the relative improvement in precision/recall over the FashionCLIP2.0 baseline. You can find more details and the code to reproduce here https://github.com/marqo-ai/marqo-FashionCLIP.

Average recall/precision @1 results across 7 datasets (compared to FashionCLIP2.0 baseline)

Let me know if you have any feedback or if there are other models you would like to see developed!

GitHub: https://github.com/marqo-ai/marqo-FashionCLIP
Blog: https://www.marqo.ai/blog/search-model-for-fashion