Mercari is releasing new features utilizing AI technology. What can producers do when faced with incorporating new technology requiring specialized knowledge?

This is the seventh entry in a series of interviews by Takeo Iyo, Mercari JP’s product owner, to get a feel for the projects, results, and backgrounds of producers working at Mercari. In this interview, he speaks with Tairo Moriyama, a producer working in product management using search technology and AI at Mercari.

*At Mercari, the terms “producer” and “product manager (PM)” refer to the same position.

Tairo Moriyama (right)

After graduating from Waseda University, Moriyama launched a recruiting/HR startup before joining BizReach Inc., where he worked on developing a search engine for job postings and improving search features using natural language processing technology. After joining Mercari in November 2016, he worked as a product manager to improve search features in both the US and JP versions of the Mercari app, and released new features using personalization and AI. In January 2018, he established the CRE Team to use technology to improve CS at Mercari.

---

Takeo Iyo (left)

After graduating from university, Iyo worked at Matsushita Electric Industrial Co., Ltd. (currently Panasonic Corporation) and Nomura Research Institute, Ltd. before joining Recruit Holdings Co., Ltd. in 2006. After leading numerous projects, including determining mid- to long-term strategies and developing next-generation media, he joined Mercari, Inc. in March 2015, and was appointed as VP in August 2016. He was in charge of product management for the US version of the Mercari app before assuming the role of product owner for the Japanese app in April 2017.

Combining AI technology and search technology with Kando Listing

Iyo: Your field of expertise is search improvement, and recently you’ve been employing AI and machine learning.

Today, I’d like to ask you about three things: the item name recognition feature that uses AI to guess the name of the item based on images, timeline personalization features, and the team established in January of this year, CRE (Customer Reliability Engineering).

First, let’s talk about the item name recognition feature. How did that come about?

Moriyama: Mercari has always had the issue of taking too much effort to list items. Sellers need to look up information like the official name of the item and how much it usually goes for, and then type it all out… It’s a lot of work. So we thought, why don’t we use image search technology to do some of that automatically and make it easier for people to list items? That’s how this idea was born.

Internally, we call this feature “Kando Listing,” where “kando” means impressed in Japanese. (laughs) It analyzes the images users take when listing items, and automatically fills in fields like the item name, category, and brand.

It’s really nice for the user to have all the required fields filled in automatically just by taking a picture of the item, and it’s extremely impressive to see it work. That’s why we named it “Kando.”

Information is extracted from the image data and used to automatically fill in the item name and category (the fields in yellow)

Iyo: It’s a great way to combine AI technology and search technology. What made you think you could pull off a feature like this?

Moriyama: I actually experimented with image search in the past, and so I had some experience that gave me a feel for how this Kando Listing would go.

Some e-commerce services overseas have image search features where you can take a picture of something and search for similar items. I thought about implementing this at Mercari and did some trial and error testing, but then I realized, couldn’t this be used on the seller side to make listing items easier for users? That’s when I started working on this as a feature.

This is a combination of natural language processing technology and machine learning technology called deep learning. In my last job, I worked to create a search engine, so I was really happy to release a feature like Kando Listing combining these two technologies.

Iyo: So the idea for it came from the technology side. How did it go once it was released?

Moriyama: Both the listing completion rate for new users and the number of listings per person improved by a few percent. I think this was a combination of multiple factors: both the listing process becoming easier, and users getting curious and listing multiple items to try out the Kando Listing.

In the second half of last year, we had a period where new AI features at Mercari were booming. I think the most representative of those features is Kando Listing. When I heard that Prime Minister Abe was going to demo the Kando Listing feature the other day, I was really nervous. (laughs) But seeing it used in a situation like that really shows just how much of an impact Mercari has made with AI, and I’m glad we did it.

mercan.mercari.com

What should we show users when they first open Mercari?

Iyo: You also released a feature to personalize the timeline on the first screen users see when they open the app. How did this come about?

Moriyama: This is a feature I’ve always wanted to try.

So many items are listed every day on Mercari. Some users want to explore and see what kinds of items have been listed recently, but other users know exactly what they want and go on Mercari to search for that item specifically. So when we were considering what to show on the timeline, some of us thought it would be better to show a variety of items from different categories and some of us thought it would be better to personalize the items and show things the user might be interested in.

Personally, I wanted to personalize the timeline. At the time, there were more items listed by women, so when men opened Mercari, they were seeing things like lipstick and perfume.

As a Mercari user myself, I wanted to show users things that matched their individual interests. And as luck would have it, someone with a lot of experience in personalization joined around that time, so we thought, let’s do it!

Someone who was browsing children’s clothing sees listings from the same category on their timeline

Iyo: How do you personalize the timeline?

Moriyama: We determine categories the user is interested in based on their search history and browsing history, and put items from those categories on their timeline.

It seems simple, but it was extremely effective. It had such a large impact that it boosted our gross transaction value level by a few percent.

What’s interesting is that you might expect purchases made through searching to go down, because items users might want are displayed on their timeline without having to search for them, but it was actually the opposite—the number of purchases made through searching went up. I guess users see items they want on their timeline and think “oh, right, I was looking for that!” and run the search again.

Now, we’re working on making the personalization feature even better. We’re looking at things like no longer showing the same item on the timeline after the user buys it, and optimizing how the timeline is displayed on the screen.

Using AI technology to resolve inquiries and reports from users with CRE

Iyo: CRE (Customer Reliability Engineering) was established as a way to use AI technology to better meet the needs of users. I know it’s still a work in progress, but what’s the goal of this team?

tech.mercari.com

Moriyama: CRE has two main goals.

• Resolve problems before users have to contact us
• Monitor for listings users are likely to report

When users have problems with their transactions, they send us an inquiry. Here are some examples: “When I went to ship the item I listed, it was broken.” “I’d like to cancel my transaction.” Customer Support (CS) members handle inquiries like that.

When users come across things they think are violations of our terms, they send us a report. Mercari is used by a large number of people. Some of those people accidentally list items that are against our terms of use. A report is what allows other users to notify us about it when it happens.

While we keep releasing new products and features at a rapid pace, CS has tended to fall behind, which is something the Mercari Group as a whole is trying to fix. CRE is a project to symbolize our decision to prioritize and focus on CS in order to significantly reduce the burden on users and resolve situations threatening the safety and security of Mercari with the power of technology.

The decision producers must make: is AI technology necessary?

Iyo: The work you’ve been doing, combining AI technology and search technology, is in a pretty specialized field, even for a producer.

The producer’s job is to take an idea and turn it into a business, but…if you’re in a situation where you have to incorporate this kind of technology, or if you yourself decide you want to, what should you do?

Moriyama: Basically, the most important thing in developing features using AI is the ability to identify tasks. Honestly, the fastest way to develop this ability is to get a lot of experience.

Of course, the best situation is having knowledge about the system. But even if a producer without much knowledge were to enter the CRE Team, they would develop the ability to identify tasks in the course of releasing product improvements over and over.

For AI, they would be able to identify whether AI is necessary to solve a task or not, and whether AI is appropriate for the task.

Iyo: So basically, you can learn how to best utilize AI by getting experience working with it.

Moriyama: Exactly. This might not be the best way to put it, but I think producers should be doing the same things as the members that actually do the work.

For example, suppose you’re trying to decide whether AI technology can be used to identify counterfeit brand items. If you look at the actual CS team in the workplace, you can see that even the team members with expertise in this area can’t always tell whether an item is counterfeit at first glance. If that’s the case, even if you train an AI with a huge amount of data, the accuracy will eventually reach a limit. Looking at how things are done now and how the data is structured will help improve your ability to accurately make these decisions.

Iyo: AI can only make decisions based off of data, after all.

Moriyama: Right. AI is heavily dependant on how data is structured. In my experience, I can say that releasing a new feature that uses AI is very different from releasing a new feature with regular system development.

In traditional system development, you decide the specifications to meet the requirements you want to implement. You test to make sure the program works according to those specifications, and then release it. You may have some things you missed in the specifications because you didn’t expect certain cases to show up when you were deciding the specs, but it works exactly how you decided it should.

But for features using AI, since you’re basically using a lot of past data to guess at an unknown result, it may not go 100% according to what you expected. There is always a chance it will guess incorrectly. You need to change your way of thinking from determinism to probability theory.

Producers, engineers, and even the higher-ups and management need to understand this special characteristic of working with AI and, in a sense, prepare themselves for the worst in order to proceed with the project. If not, you could end up with meaningless debates over errors of only a few percent, and even make it all the way to the release only for it to be shut down at the last second.

There are some unique points to consider when introducing AI to a project, and it requires some getting used to, but as you go, it gets easier and easier to understand.

If you’re a fan of simulation games, you may like working with AI

Iyo: What are some qualities that might make someone a good fit for a producer working with AI and other new technology?

Moriyama: AI can only get the answer right a certain percent of the time, so someone who’s up for a challenge. Also, people who like simulation games where you raise a pet or other character might be a good fit.

Iyo: Why is that?

Moriyama: When working with AI, you need to keep adding new data even after release. If you don’t constantly train it, it won’t be able to recognize new products.

Take Mercari’s Kando Listing feature. When Google Home was first released, the Kando Listing AI couldn’t guess how to fill in the information from pictures, because it was a new product and had never been sold on Mercari before. But after we updated the model with new data, it was able to recognize it in pictures.

Sometimes I use the term “feeding” to mean training the AI with new data, which makes it feel like I’m raising a pet or something. (laughs)

Iyo: That does sound like a simulation game. (laughs) The demand for machine learning engineers is getting higher. Do you feel the need for it in the workplace?

Moriyama: I do. But like I said earlier, we need people who can look at the big picture and identify when AI is and isn’t necessary, and who can work with engineers to lead projects.

Iyo: You do interviews in the hiring process for producers. What do you look for?

Moriyama: I look for two points. First, can this person structure their ideas logically? Second, are they passionate? Do they have intellectual curiosity and strong interest? I look at their emotions.

Basically, I can tell whether someone has the potential to be a good producer by looking at whether they can use logic to structure things they feel strongly about. If they can’t, they won’t be able to explain what they want to do. Even if they can explain something logically, it won’t be interesting if they don’t put their own ideas and thoughts into it, and they won’t be able to get those around them to agree.

Iyo: That’s the base for becoming a producer.

Moriyama: Right. Are they passionate about wanting to do something, and can they turn that passion into a plan to make it into a reality? If so, even if they’re faced with working with new technology like AI, they’ll be able to keep learning and make a big impact.

mercari – Jobs: Product Manager – Apply online