image: SMU Associate Professor Hady Lauw’s NRF Fellowship research produces publications, software programmes, and commercialisation of technologies. view more
Credit: Singapore Management University
By Grace Segran
SMU Office of Research & Tech Transfer – After you’ve watched the movie ‘Titanic’, Netflix suggests a list of movies coincidentally containing many of your favourites and new ones that you may like. When you click a link to Adele’s latest music video on YouTube, you see a personalised collection of music videos on the sidebar that’s eerily accurate. Have you ever wondered how a website knows what you want and like?
E-commerce websites use a recommender system that employs artificial intelligence (AI) and data science. It is a data filtering tool using machine learning algorithms to recommend the most relevant items to a particular user to increase user engagement and drive purchases.
This is an area of research that SMU Associate Professor Hady W. Lauw is working on. He was awarded a 5½-year National Research Foundation (NRF) Fellowship that just concluded in September 2021.
“The main thrust of the research is making use of all data of various modalities such as images, text, and graphs to enhance the performance and interpretability of recommender systems,” he told the Office of Research and Tech Transfer. “We have a number of publications coming out of the research and we have developed a series of software programmes. We are also looking into potential commercialisation of some technologies coming out of the project.”
The professor of computer science says he’s fascinated by two things in the work he does.
One is how the Web is a treasure trove of information, essentially a form of collective intelligence if only we know how to tease out the insights from it. From the beginning of his research career, he has always looked at the Web as a source of data to be mined and it will probably continue to be so for a long time.
The other thing is the depth and diversity of user preference, how simultaneously we are all unique in having our own personal preferences. “Yet we are not all that so different,” he says. “One of the methodology in personalising recommendations is to find another similar person and to peek at their preferences. Given the size of human population, there is always someone similar to us in that sense. The challenge is in identifying who that is.”
The research
The overall objective of Professor Lauw’s NRF Fellowship research was to investigate how signals of user preferences that surface through various modalities (such as text, ratings, links, images, etc.) may be integrated holistically into a comprehensive knowledge base that can be used to supplement recommendation algorithms with crucial information on the relationships among entities in a given domain.
There were a couple of reasons to focus on multiple modalities. “For one, the kind of modality we get depends on a specific data source. On social media, we may be able to find images. But on e-commerce sites, we are more likely to find text reviews,” he explains. “For another, the kind of modality that is important depends on product category. For things that are visual in nature such as fashion items, image modality would be useful. For book recommendations, the text content is the primary information.”
When the research team started out, the original objective was to improve the accuracy of predicted recommendations. Over time they realised that given the nature of modalities that they were considering, these were also great at improving the interpretability of recommendations. “Someone may be recommended a product, and being informed by a textual explanation or a visual example of why the product is being recommended would make the recommendation itself more persuasive,” opined Professor Lauw.
Distinguished Paper Award
One of the research papers arising from the NRF Fellowship project clinched the Distinguished Paper Award at IJCAI-20, a top AI conference.
The objective of the paper entitled “Synthesising Aspect-Driven Recommendation Explanations from Reviews” was to provide a textual explanation to a product recommendation.
“Generating a realistic customised textual passage is still challenging for current technologies,” says Professor Lauw. “The key insight of the paper was to focus not on generating an explanation sentence by sentence, word by word, but rather, to show that it would be more efficient to put together human-created sentences from disparate reviews and then to make them gel together as a single coherent passage.”
Multimodal recommender library
The research team created Cornac, an open-source multimodal recommender library. It was recently showcased in a tutorial at ACM Conference on Recommender Systems 2021. It is a library of recommendation algorithms. Currently there are more than 40 algorithms built into the library with a focus on multimodal recommendation.
“The cool thing is how easy this software makes it not only to build recommendation models, but also to compare them side by side to see which is more effective under which dataset and which metric,” enthuses Professor Lauw. “The reason why this is important is because different algorithms work differently depending on specific recommendation datasets. We might not know beforehand what is best, and so a comparison capability is important. It even leads to surprising findings, such as an algorithm that was designed with image modality may work better with text or graph modality.”
Current adopters are probably researchers running experiments on various recommendation algorithms, but the researchers hope to promote it more broadly among industry players as well.
Commercialisation
There is potential commercialisation of some technologies coming out of the project. One is ThriftCity, an offer comparison service that is currently undergoing proof of concept study. This is currently being pursued as a project funded by MOE Decentralised Gap Fund, which is run by SMU Institute for Innovation and Entrepreneurship. The main capability is the underlying AI and machine learning algorithms that facilitate going to the Web to find various products and their information.
“Diverse as these products are, we devise algorithms to match them to allow apple-to-apple comparison,” says Professor Lauw. “This is a difficult problem because sellers represent their products by describing them in different ways. So finding matches in the face of such diversity is the technical challenge being solved.”
One use case which the researchers are exploring is price comparison for consumers to find best offers, as well as price intelligence by sellers to price their products more competitively. They are currently working with a company to run a prototype.