Can Augmented Reality Change How You Shop for Clothes?

Personal avatars, real-time clothing simulation and virtual closets could alter fashion retailing.

Instead of stepping into a fitting room to try on new clothes, shoppers may soon activate an augmented reality body double on a retail kiosk or their personal device. Parents could carry around avatars of their children, making it easy to buy clothes that fit and match with other clothes already hanging in the closet at home.

Intel Labs researcher Nola Donato demonstrates augmented reality dressing room

Bringing mirror-like augmented reality to the shopping experience will require realistic clothing simulation that allows people to see how new clothes might fit and move on their body according to Intel Labs researcher Nola Donato.

Experiments with so-called digital dressing room technology have been around for years, including the multimedia bathroom mirror from the New York Times Research and Development Lab and the Magic Mirror avatar experience developed by Intel Labs researcher Nola Donato.

Recently, Nordstrom’s CEO said technology innovations such as digital fitting rooms will change the way people shop for clothes. It’s a trend that European retailers Tesco and Zalando are pushing ahead, as is Bodymetrics, which is using Microsoft Kinect technology to create full-body scanners to help Bloomingdale’s shoppers find the perfect-fitting pair of jeans.

However, it remains a technological challenge to create digital clothing that looks and moves as if the person were actually wearing it, according to Donato. The graphics software architect is tinkering with perceptual computing hardware and software algorithms that could someday let people put clothes on their own lifelike avatar using a personal device and do it with a fidelity that re-creates an augmented reality experience like looking in a mirror. In a recent interview, Donato described the difficulties of creating real-time clothing simulation and how the technology could make possible new shopping experiences.

Being able to touch fabrics, try them on and feel how they fit before you buy is hard to beat. Why would shoppers be attracted to using 3-D augmented reality shopping technology instead?

Today, when you go into a store you have to find what you like, try it on and sometimes you have to wait in a long line. That experience, for some, is unpleasant. When you look at the alternative of shopping online, you see little 2-D images and you have to make a buying decision based on that with no idea for how the items look on you. A lot of returns happen because people buy three sizes to find the one that fits them and then return the others. It would be nice if you could figure out what size fits you by first trying on the clothing.

Using clothing simulation technology you might be able to say, “I have this shirt at home and here I found a couple of pairs of pants … let’s try them on and see which one goes best with my shirt.”

You can also make shopping social and collaborative. You can shop with your friends who aren’t with you or your mom who lives in another city. Say you’re shopping for a dress and you want your mom’s opinion while you’re at the store in front of the kiosk. Wouldn’t it be nice if you could say, “Mom, take a look at this on my avatar and tell me what you think and should I buy it?”

The cloth simulator we’re working on basically takes dress patterns as a seamstress would make them and allows you to put them on to see how it fits and it matches with other clothes. The software drapes it on the avatar without using collision geometries, so you’ll be able to drape jackets over skirts over sweaters and layer the clothing just like you would in real life.

Stores like Lego have already brought 3-D augmented reality experiences to retail. Why is shopping for clothes still stuck in the 2-D world?

One of the most challenging things is to upgrade the quality to where you really think it [the kiosk or personal device screen] is a mirror. Tracking with a single-depth camera does not give you enough fidelity and cannot track you when you turn around. The [real-time experience] just completely falls apart and it doesn’t give you any way to show grace of body movement. Lower-quality rendering may be fine in a game where you’re fighting or something, but for [retail clothing] use we’ll need more than one camera and a better-performance tracking system.

Intel 3-D graphics architect Nola Donato shows augmented reality dressing room

We want to create a real-time experience where you can answer, "How does this look on me when I'm standing, twirling or walking, and does this style suit me, does this go with something that I already have?" said Nola Donato, a 3-D graphics architect at Intel Labs.

The solution that we’re working on this year involving photo-realistic rendering and very high-quality cloth simulation probably won’t run on a typical [mobile computing] client. In order to get a version [for tablet or smartphone] to have all of the capabilities, we have to stream the video from a server to a client. However, inside a store we can do all of the simulation calculation on the kiosk because it’s a very powerful client.

There are a lot of offline cloth simulators that although they’re not real time, produce very accurate results. But to get real time, you need very highly parallel [computer] architecture [that can process lots of information quickly]. We’re collaborating with UC Berkeley, James O’Brien’s group, to make it a parallel cloth simulator that will run in real time with fashion-quality results so that you can get an idea of what it would look like on you by modeling it virtually.

We want to create a real-time experience where you can answer, “How does this look on me when I’m standing, twirling or walking, and does this style suit me, does this go with something that I already have?”

And in the future we want to give access to a personal virtual closet that has clothes that you actually bought. You can mix and match to find the right personal look, then click a button and say, “Send me the stuff that I don’t already have.”

Computer scientists such as Michael Black have spent years creating mathematically accurate depictions of human body types, shapes and movements. If each of us had a lifelike avatar, where would we use it?

The first thing we’re going to do is to make a kiosk where you stand in front of it and it does what you do. It’s your body, your motion and the avatar lets you model outfits like you’re looking in a mirror. You’d be able to use gestures to select or use speech to ask it to show merchandise. It will be more like a personal style concierge or fashion consultant. We see this being in a retail establishment or in a mall.

The second thing we want to do is to have a version that lets you do online shopping. You can use your smartphone, tablet or laptop to find merchandise, see how it looks on you or somebody — say if you’re shopping for your son, daughter, wife or husband and you have their avatars already stored then you’d be able to shop and see how things look on them. You might even have access to their virtual closet so that you could make it match stuff they already have.

Can Augmented Reality Change How You Shop for Clothes?

Personal avatars, real-time clothing simulation and virtual closets could alter fashion retailing.

Instead of stepping into a fitting room to try on new clothes, shoppers may soon activate an augmented reality body double on a retail kiosk or their personal device. Parents could carry around avatars of their children, making it easy to buy clothes that fit and match with other clothes already hanging in the closet at home.

Intel Labs researcher Nola Donato demonstrates augmented reality dressing room

Bringing mirror-like augmented reality to the shopping experience will require realistic clothing simulation that allows people to see how new clothes might fit and move on their body according to Intel Labs researcher Nola Donato.

Experiments with so-called digital dressing room technology have been around for years, including the multimedia bathroom mirror from the New York Times Research and Development Lab and the Magic Mirror avatar experience developed by Intel Labs researcher Nola Donato.

Recently, Nordstrom’s CEO said technology innovations such as digital fitting rooms will change the way people shop for clothes. It’s a trend that European retailers Tesco and Zalando are pushing ahead, as is Bodymetrics, which is using Microsoft Kinect technology to create full-body scanners to help Bloomingdale’s shoppers find the perfect-fitting pair of jeans.

However, it remains a technological challenge to create digital clothing that looks and moves as if the person were actually wearing it, according to Donato. The graphics software architect is tinkering with perceptual computing hardware and software algorithms that could someday let people put clothes on their own lifelike avatar using a personal device and do it with a fidelity that re-creates an augmented reality experience like looking in a mirror. In a recent interview, Donato described the difficulties of creating real-time clothing simulation and how the technology could make possible new shopping experiences.

Being able to touch fabrics, try them on and feel how they fit before you buy is hard to beat. Why would shoppers be attracted to using 3-D augmented reality shopping technology instead?

Today, when you go into a store you have to find what you like, try it on and sometimes you have to wait in a long line. That experience, for some, is unpleasant. When you look at the alternative of shopping online, you see little 2-D images and you have to make a buying decision based on that with no idea for how the items look on you. A lot of returns happen because people buy three sizes to find the one that fits them and then return the others. It would be nice if you could figure out what size fits you by first trying on the clothing.

Using clothing simulation technology you might be able to say, “I have this shirt at home and here I found a couple of pairs of pants … let’s try them on and see which one goes best with my shirt.”

You can also make shopping social and collaborative. You can shop with your friends who aren’t with you or your mom who lives in another city. Say you’re shopping for a dress and you want your mom’s opinion while you’re at the store in front of the kiosk. Wouldn’t it be nice if you could say, “Mom, take a look at this on my avatar and tell me what you think and should I buy it?”

The cloth simulator we’re working on basically takes dress patterns as a seamstress would make them and allows you to put them on to see how it fits and it matches with other clothes. The software drapes it on the avatar without using collision geometries, so you’ll be able to drape jackets over skirts over sweaters and layer the clothing just like you would in real life.

Stores like Lego have already brought 3-D augmented reality experiences to retail. Why is shopping for clothes still stuck in the 2-D world?

One of the most challenging things is to upgrade the quality to where you really think it [the kiosk or personal device screen] is a mirror. Tracking with a single-depth camera does not give you enough fidelity and cannot track you when you turn around. The [real-time experience] just completely falls apart and it doesn’t give you any way to show grace of body movement. Lower-quality rendering may be fine in a game where you’re fighting or something, but for [retail clothing] use we’ll need more than one camera and a better-performance tracking system.

Intel 3-D graphics architect Nola Donato shows augmented reality dressing room

We want to create a real-time experience where you can answer, "How does this look on me when I'm standing, twirling or walking, and does this style suit me, does this go with something that I already have?" said Nola Donato, a 3-D graphics architect at Intel Labs.

The solution that we’re working on this year involving photo-realistic rendering and very high-quality cloth simulation probably won’t run on a typical [mobile computing] client. In order to get a version [for tablet or smartphone] to have all of the capabilities, we have to stream the video from a server to a client. However, inside a store we can do all of the simulation calculation on the kiosk because it’s a very powerful client.

There are a lot of offline cloth simulators that although they’re not real time, produce very accurate results. But to get real time, you need very highly parallel [computer] architecture [that can process lots of information quickly]. We’re collaborating with UC Berkeley, James O’Brien’s group, to make it a parallel cloth simulator that will run in real time with fashion-quality results so that you can get an idea of what it would look like on you by modeling it virtually.

We want to create a real-time experience where you can answer, “How does this look on me when I’m standing, twirling or walking, and does this style suit me, does this go with something that I already have?”

And in the future we want to give access to a personal virtual closet that has clothes that you actually bought. You can mix and match to find the right personal look, then click a button and say, “Send me the stuff that I don’t already have.”

Computer scientists such as Michael Black have spent years creating mathematically accurate depictions of human body types, shapes and movements. If each of us had a lifelike avatar, where would we use it?

The first thing we’re going to do is to make a kiosk where you stand in front of it and it does what you do. It’s your body, your motion and the avatar lets you model outfits like you’re looking in a mirror. You’d be able to use gestures to select or use speech to ask it to show merchandise. It will be more like a personal style concierge or fashion consultant. We see this being in a retail establishment or in a mall.

The second thing we want to do is to have a version that lets you do online shopping. You can use your smartphone, tablet or laptop to find merchandise, see how it looks on you or somebody — say if you’re shopping for your son, daughter, wife or husband and you have their avatars already stored then you’d be able to shop and see how things look on them. You might even have access to their virtual closet so that you could make it match stuff they already have.