Large language models (LLMs) and other generative artificial intelligence (AI) technologies are becoming increasingly important as tools that revolutionize the user experience and provide new value. AI Listing Support, a feature released by Mercari on September 10, 2024, is one example of such technology. AI Listing Support helps save the user time and effort by automatically filling out the item information and setting a price after the user takes a photo of the item they want to list and selects an item category.

While Mercari has developed a number of initiatives that improve the experience of listing an item, this project in particular was born from a desire to revolutionize the listing experience. We talked to Yasuo Hishii (@hisshy), the project manager who oversaw this project, and Max Frenzel (@maxfrenzel), Head of AI/LLM, about the story behind the development of the feature, the new user experience it offers, and their vision for the future.

How AI Listing Support revolutionizes item listing

——Could you start by telling us about the background behind developing AI Listing Support?

@hisshy：One important indicator for Mercari is how successful we are at getting users to list their unwanted items, in other words, the number of items listed by users. However, we realized that taking a picture of an item, inputting the item information, and writing a description are time-consuming tasks that may make users hesitant to list an item.

We have developed a number of initiatives in the past to try and improve the listing experience. This time, we wanted to make the listing experience dramatically easier than before. That’s when we thought to leverage AI, which was the start of AI Listing Support.

——What were those past initiatives?

@hisshy：Our most recent initiatives include a feature that suggests a category and brand for an item so the user only needs to choose from lists, and a feature that shows users information about similar listings on Mercari when they list an item. These features have made listing somewhat easier.

Now, however, leveraging AI allows users to list items in as few as three steps: 1. Take or select a photo; 2. Select the item category; 3. Check that the AI-generated information is correct, and then list the item for sale. This can save users a lot of time and effort. We believe that this new system will make Mercari easier to use than ever before, not only for novice users but also for frequent users.

An example of AI Listing Support in action. Currently, the page says “The item name and description will be input automatically.”

Easy listings = listings that sell

——What challenges—if any—did you have during development?

@Maxfrenzel：There were many. When we first started development, we weren’t even sure if what we wanted to achieve would be possible. One particular problem was striking a balance between AI accuracy, speed, and cost.

The more you enhance an LLM to improve its accuracy, the slower and more costly it becomes. While our idea of using generative AI to automatically input information was possible with the LLMs that were available at the time, it was not practical in terms of cost and speed.

However, AI evolves at a tremendous pace and we could not afford to fall behind. We took a leap of faith that the technology would catch up with our needs and began developing a new model for implementation. In the end, the LLM became much faster and the cost gradually decreased, just as we had expected. Eventually, we were able to make the model generate a title and description in a very short amount of time while overcoming the cost issue.

——Your belief that the technology would evolve really paid off!

@Maxfrenzel：One other challenge was defining what a good listing actually is. First, we needed to input “good listings” into the AI, but the definition of a “good listing” varies from person to person. We needed a large amount of training data to determine whether information was correct or not and to decide on the format and style of the item description.

We recruited a variety of employees and ran one of the largest dogfooding tests (a trial where a company tests a new feature they have developed by using it themselves) that we’ve ever done at Mercari. Through this test, we were able to gather enough trustworthy data to define what a “good listing” is.

@hisshy：AI-related projects come with the challenge of judging at what point the quality is high enough for release. For example, even if we are able to improve the quality of automatic item description to a satisfactory level, users won’t use the feature if they’re not happy with the overall quality.

In the dogfooding test, we recruited 60 employees to try out a demo tool. We took a simple approach in which we had every user grade the quality of the AI-generated item descriptions. That helped us obtain the level of quality we needed for release.

——When you say “good listing,” do you mean a good listing from the user’s perspective, that is, a listing that sells?

@hisshy：The goal of the user is to sell their item, so a “good listing” can be regarded as a listing that sells. But, for this project in its current stage, we focused on whether the user would be able to list an item in the way that they want to, rather than whether it would sell.

@Maxfrenzel：This may be obvious, but the information that the seller wants to include and the information that makes the buyer want to buy the item are not always the same. For example, even if the information in the item description is technically perfect, buyers will refrain from buying if the description sounds like it was written by AI and lacks warmth.

Ideally, listings include all the information the seller wants to include and also appeal to the buyer. In order to get to that stage, it will be important for us to work with teams that deal with search and recommendations and update the feature to incorporate the opinions and actions of buyers.

——Speaking of teams working together, did you run into any challenges collaborating between the product team and the LLM development team during this project?

@Maxfrenzel：Well, we didn’t have a lot of time, so we had some difficulty finding a solution that incorporated both teams’ ideal outcomes. On the LLM development side, for instance, we believed that we could more reliably generate “good listings” by limiting the number of categories that would use AI Listing Support.

But on the product side, @hisshy’s team wanted both quality and quantity. (laughs) It’s incredible that we were actually able to achieve both quality and quantity thanks to that push from @hisshy’s team, but it involved various discussions about what to prioritize.

@hisshy：When two teams work together, they each have their respective fields of expertise, so delivering both ideals at the same time is a difficult task in any project. However, what led to success under those constraints was that the team leaders always made sure to align with each other and share the direction of what they really wanted to achieve with their team members through frequent communication.

Using seller and buyer data for a mutually beneficial feature

——How have users responded to the feature since its release in September?

@hisshy：We haven’t conducted any fixed-scale surveys just yet, but the people I know like it. The data tells us that more and more users are using AI Listing Support when listing an item. So, it seems we were right in thinking that users found the listing process to be too much effort. To get more people using the feature, we will work on making incremental improvements going forward.

——What kind of improvements do you have in mind?

@Maxfrenzel：Right now, users are listing a lot of items and we are starting to accumulate data on user behavior. We’ll use that data to think of ways to improve the feature in the short term and mid-to-long term.

In the short term, we want to analyze how well listings that use AI Listing Support sell compared to those that don’t and which categories are most successful, and use that information to improve the feature. We would also like to make small improvements to the user experience, such as changing the placement of buttons for canceling AI-generated information to prevent accidental cancellations.

——What are some of the mid- to long-term improvements?

@Maxfrenzel：Broadly speaking, I would like to enhance Mercari’s competitive advantage based on the data we have accumulated. This will include technical challenges, such as evolving LLMs using learned data, and also thinking about “AI transparency.”

Also, in terms of the user experience, we would like to make it more interactive. At the current stage, you take a picture of the entire item, press a few buttons, and a little later, information such as the item description is added. It would be great if, for example, information could be generated by simply taking a picture of an item’s label, or if AI guidance could begin while the picture is being taken.

@hisshy：Ideally, it would be great if the listing process would be complete as soon as the user takes a picture of the item. However, that’s still a big challenge in terms of the technology involved. Keeping that option in mind, we’ll start working on the most feasible features first.

——Lastly, could you share what initiatives you have in mind to help listings sell more easily?

@Maxfrenzel：We’ve been thinking of a few. One is to collect and utilize item metadata to improve search and discovery algorithms. We would do this by extracting metadata from item data and using it to help the search algorithm get the exact information it needs. In doing so, we hope to optimize the current search algorithm to make it easier for users to find the items they are looking for, while at the same time displaying items more attractively.

Also, as I mentioned earlier, we believe it is important to understand and incorporate the buyer’s experience as well. To do this, we will use data to capture search keywords and item trends to continuously update the definition of what a “good listing” is based on the time of year and the current trends.

@hisshy：It will also be important to provide feedback to sellers to create listings that sell well. We will collect and analyze data from the buyer side, and based on that data, we will suggest improvements to the item information, the way the description is written, the price, and so on. We hope that these efforts will make the feature beneficial to both sellers and buyers.