In today’s fast-paced world, driven by artificial intelligence, it’s easy to believe that machines can do a good job of analyzing information into actionable intelligence. It isn’t too far off in the future and we do believe there is progress being made. But in the meantime, companies need to leverage machine learning technologies and AI driven solutions as best they can to enhance individuals’ own intelligence. Augmented human intelligence can be given to teams to assist them in making better decisions, when analyzing the right content themselves.

But what are some of the challenges when it comes to developing augmented intelligence applications such as a content recommender system, specifically in the corporate world?

As unstructured data continues to increase and overwhelm, knowledge departments and employees need to find ways to filter out the noise, and gather information to be able to extract actionable artificial intelligence in an efficient manner. The real challenge, therefore, is for augmented intelligence companies to assist in developing systems and processes at a quicker speed than their customer’s competitors. This way, the company can always stay one step ahead.

Machine learning based recommenders are not new concepts, in fact, many implementations are currently being used for commercial systems such as Amazon. These available models are optimized towards structured data sets and product category classification of data items aimed at helping the shopper find information and products they might also enjoy in an effort to sell more.  But when trying to apply these methods on short-lived unstructured data, unfortunately these concepts fall short.

The unstructured content challenge

In the past, commercial systems would recommend products based on user purchase and data ratings to compute the similarity between users and items. The typical approach is often the nearest neighbor product based on recommendations in a trust-based network of people. The basic idea with a trust-based network model, is that users use their social network to reach information and their trust relationships to filter it. On an e-commerce site, these people do not know each other but form a temporary network in the form of a customer group, in relation to the vendors and products being sold.

Unlike shopping websites that have products in store for several months and quite few classification terms, a content recommender must be designed to work with much shorter time spans and be able to handle vast quantities of new content. A content recommender might work with tagged and content rich documents, such as internal and external analysis reports, with a life span of months, and short-lived news items which might only be available for a few days at the most, or even hours at worst. In addition to this, users’ consuming patterns are very different from buying or searching for products in an online store.

When an online website shop can collect usage patterns for a specific product over a long time and from many users, a content recommender must work with short timeframes and in comparison, few users and interactions.

News items and classification with augmented intelligence software

There are a lot of possible data points to explore when it comes to news. The number of clicks can, of course, indicate a point of interest to a user, but does not necessarily indicate the quality of the information, but merely an attractive document title.

A rating system, such as thumbs-up, likes or stars rating, might also boost the quality and relevancy rank, but corporate systems usually find it challenging to encourage people to upvote and recommend information in a scale that would be really helpful in its own right. Users’ additional contribution suffers from the same problem but commenting on articles and tagging content should boost relevancy in general.

The amount of time spent on an article can also be an important indicator but depending on the length of the content accessed, these numbers might vary a lot. A threshold value for the time spent seems to be a reasonable factor. To solve these challenges, a content recommender should be built on several parts, making suggestions based on all these numbers.

Collaborative filtering based on clicks, deep readings, ratings, sharing and other actions such as printing, exporting content or user hash-tagging content is important.

Content based filtering

Another approach that can be applied is analyzing the data classification and metatags provided by vendors and added by users. For example, Netflix does a great job of providing recommendations based on genre, actors or directors that accompanies a “like” system.

In 2017, Netflix simplified the user feedback function, when they found it was hard to collect user feedback with the old rating system based on one to five-stars. According to Netflix, ratings increased by 200% when the new binary rating system (thumbs-up) was implemented.

When news is imported into a system like this, we can apply metadata enrichment with a taxonomy, either by manual or automated classification based on simple rules.

Manual efforts are usually not very efficient when the amount of news monitored every day by a larger corporation counts in many thousands. The resources for doing this would simply be too high and many companies apply automatic Boolean classifications to cope with the data streams.

The more metadata we have available the better, and some premium news providers are already assigning additional metadata about articles. However, the bulk of news feeds does not provide metadata and in addition, the text should be analyzed by machine learning technologies to enrich the content at the time of processing.

As many data points as possible should be generated by content enriched machine augmented intelligence. For example, augmented intelligence solutions should identify people and organizations mentioned, detect geographic location and used language in the article.

In addition to analyzing the content, data about the users, interests and behaviors should be collected and analyzed too. This is a challenge when the usage pattern varies over time. But this problem is not unique – on internet syndications of data from searches and visits to e-commerce websites, targeted commercials often run for weeks after a purchase, even when the potential customer is no longer interested in this information.

Summary

In conclusion, augmented intelligence in business is computational algorithms helping to put the human end user in the best position to make informed decisions. The purpose of a content recommender is to provide one of the tools for augmented intelligence, that meets the challenges of filtering overwhelming unstructured content. 

Organizations should apply a hybrid model of collaborative filtering and several machine learning technologies to provide up to date recommendations to knowledge workers and this can start with Intelligence2day®. The applied models must consider users metadata, the content metadata, and the interactions between them.