AI & ML

How Intercom navigated the AI paradigm shift

Intercom’s Des Traynor and Fergal Reid share field notes on what other builder teams should know as they transform their products with language models.

GPT-3 sparked the flame, and then GPT-4 set the software world on fire. For SaaS builders, capabilities enabled by the recent advancements in large language models mean that anywhere there is natural language processing (NLP)—within the application, within the product workflow, and within the ecosystem—there is an opportunity to leverage these massive predictive calculators to reimagine SaaS solutions.

“The first and most important thing for SaaS leaders to understand is what is now possible that wasn't before,” Des Traynor, co-founder and chief strategy officer of Intercom said. “With GPT-4, for example, it is now possible to generate text, generate images, infer meaning, expand, contract, and take actions. So how can builders make software products easier, faster, and more productive, across different use cases?”

Long before ChatGPT became a household name, Intercom, a leading customer service platform, had been building Artificial Intelligence (AI) into their product since 2018 and experimenting with how it could transform customer support. As of today, the Intercom team has launched two products using OpenAI’s technology, Inbox AI features, which among other things allows support reps to compose, summarize, and expand entire conversations with customers with the click of a button, and Fin, their GPT-4 powered customer support bot.

Amidst this advanced computing era, the tech ecosystem is exploring applications and asking tactical questions on how exactly to reimagine products and drive value with LLMs.

On Atlas, Des Traynor, co-founder and chief strategy officer of Intercom, and Fergal Reid, senior director of machine learning, share their field notes for other product and machine learning (ML) teams. Together, they distill five lessons they learned while building AI products that other SaaS leaders can apply to their future product roadmaps.

Pick a low-risk, high-potential problem

“Selection is the hardest challenge,” said Des Traynor, co-founder and chief strategy officer of Intercom. “Finding an area that is relatively exploitable by generative AI, but also provides room for imperfection, is the challenge. That’s why our first foray into generative AI was on the agent assist side (Smart Replies).”

In this case, Intercom was simply augmenting a workflow by suggesting answers to support agents, whether it was to condense, summarize, expand, or even change the tone of the answers they were already writing. “If it didn't work, there was zero downside. If it did work, there was significant upside,” Des shared.

“No matter the use case, builders should initially focus on an asymmetric opportunity. And whatever you select, remember that there needs to be a somewhat regular and active use case. With low frequency, you’ll never get a sense if it’s working or not.”

However, with this advice, Des warns that builders must assess and forecast downstream costs before building. A worst case scenario would look like adding expensive generative AI to an extremely high frequency and reasonably low-value feature. With millions of users, this experiment could bankrupt a business without driving real customer value.

Start with a small Skunk Works team to gain traction

Though Intercom had a long held belief that the future of customer support would one day be automation working alongside humans, there was still a lot to assess before building. The team had to define clear goals, identify use cases, and prioritize features.

Initially, Intercom kept this early team tiny so they could stay as agile as possible; in other words they gathered a Skunk Works team—a group of dedicated people interdisciplinary in their thinking and collaboration.

“Driven mostly by the ML team, we deliberately ran a very small operation with some product and beta support from outside,” said Fergal Reid, leader in machine learning who has been with Intercom for over seven years. “This reduced overhead of internal communications in such tight timeframes. “As the technology was so new, we didn’t want to waste any precious time with a large group ramping up and trying to get on the same page.”

Truthfully, there were still plenty of internal obstacles around trying to run such a high velocity project. “However, with Des, we had great executive stakeholder support to help reduce the friction when it came to getting legal in place, beta terms and conditions, etc,” said Fergal. Now in full production, and two launches later, there are many more teams surrounding AI launches, but early success started with a tight-knit team.

Figure out a ship-fast-and-learn framework

“For context, we’ve been building (non-generative) AI products for years,” said Fergal. “Our end-user facing bot, Resolution Bot, has been resolving support queries automatically since 2020. Our team used neural networks to detect topics, so we were already utilizing AI features in the Inbox to make support reps more efficient.”

This was just one early example—and as the efforts of their Skunk Works team continued, they had to create a framework for prioritization.

“Intercom has a culture around shipping fast and learning, which is a major asset in times of technological disruption. Initially, we set out to build a series of features that required either thin or medium depth integrations, so we could deploy them quickly, but that would still deliver real value,” Fergal explained.

The team thought language models would only extend to features that kept a human support rep in the loop. So, they started working on the “Inbox” features, building several generative features to help support reps write text in the Inbox, as well as to summarize conversations before handover.

“If anything, we underestimated the value of these AI features."

“If anything, we underestimated the value that our customers would get from these early features.”

“As mentioned, our first feature of this type, Smart Replies, shipped in June 2022. The Smart Reply feature makes it quicker and easier for a support rep to reach out to customers by offering predictive text—based on common greetings used in the past—which the customer can accept, reject, or alter for personalized engagement at speed,” said Des.

The team was in the middle of building a next-gen version of the Smart Replies model, with the intention that it would run on in-house generative models, when ChatGPT launched.

“ChatGPT, and text-davinci-003, completely blew us away in terms of quality, so we instantly started to focus on these models, rather than our internal models,” Fergal shared.

The team knew this was a moment to pivot. Swiftly they moved production from their proprietary models into using OpenAI’s technology, leveraging GPT-3.5, GPT-4 and Plugins for better and faster outcomes.

Now that the team could outsource its model to GPT-4, their development could kick into high gear. “We could now use this foundational model to build a bot that talks to end users.” This was the start of their latest product Fin, one of the first GPT-powered customer support bots on the market.

Centralize a machine learning team

“We are four to five months into a hurricane of opportunity, and it is very early days,” said Des, “but I do think every software company will need significant expertise in AI and ML to identify the best opportunities specific to every business and to know how to explore and exploit.”

Translation: If you’ve yet to map out an ML team, the time is nigh.

“I don't think it's realistic for every engineer in a tech company to overnight become an AI and ML expert. They will probably all become very familiar with the various APIs that are out there, but I think the research and exploration around opportunities is where there is the most strategic opportunity.”

Intercom made it a priority to ensure the best practices of AI implementation extended across product, technology, and strategic teams and executive leaders alike.

Here’s how Intercom breaks down the ML vs. Product and Engineering teams:

Team

Role

Focus

Centralized Machine Learning

— Explore, exploit, present opportunities

— Partner with teams to suggest ways in which AI could influence the roadmap or significantly rewrite certain features or change long-held assumptions about the world.

Be aware of the changing capabilities, challenge assumptions, and identify ways to effectively distribute capabilities and learnings across the organization.

Product and Engineering

— Execute based on opportunities identified by ML teams

— Build prototypes and new solutions that go into production

Find ways to build a new reality; tasked with taking a feature they previously deemed non-automatable and automating it.

The framework lightly outlines the breakdown between the different teams, but that doesn’t mean it won’t change in the future.

Still, there are must-have qualities that all emerging ML teams must possess: Curiosity, the ability to apply research to different contexts, and to think like a customer. “A ML team must be very deep in the culture of R&D, meaning they are used to working with new ambiguous technology, but also care deeply about solving customer problems, and moving and shipping extremely fast. That combination is powerful,” said Des. “In 2022, we were already deeply working on the relevant customer problems, trying to use other technologies, and so when GPT-3.5 came along, we were really well positioned to move rapidly.”

And according to Fergal, even the sharpest of ML teams will only be as successful as a company’s long-term vision: “At Intercom, we see a future where AI will drive the arc of our technology progress—and this is the accelerant of our productivity. Because all teams were aligned with this company vision, it created a great deal of internal excitement and momentum.”

Reimagine language as the new UI for software

Transforming your product suite with AI requires first-principles thinking and assessing the repetitive, predictable, and expandable tasks and workflows your business facilitates. “Once you understand the scope of your product you can see the extent to which AI can change your product,” explains Des.

“Enterprise software has a history of clunky UIs; one massive question we’ll be answering in the industry over the next two years is: Is text the new UI? And is voice the new UI?”

“During the little chatbot renaissance we saw in ~2015, people would’ve laughed at the notion that chat could be the next interface, but OpenAI, GPT-3.5+, and whisper, have allowed natural language to evolve human-computer interaction.”

As a simple example, Des poses the power of text-driven UI for a tool like Google Analytics. “It would make it simpler and easier for new users to get productive within Google Analytics immediately because they don't need to learn all the dropdowns, settings, filters and segments, etc. And that would be extremely disruptive to enterprise web analytics because now everyone can use the product. With language, all you need to do is ask the right questions. Very shortly, and within the year (2023), we will see a lot of complex tooling that offer plain English UI (or any language) across not just written text, but also synthetic voice, real-time audio translation, and transcription.”

"Voice-drive UI is next."

“Voice-driven UI is next and will have long-lasting transformation for how and where software is considered.” As Des explains, voice works in places where access to screens limited.

“We’ll see the resurgence of the ‘place-ona’ vs. the persona, meaning that UI will be truly multi-modal and anyone can engage or direct software through text, voice, and other mediums.”

What Des is describing for SaaS builders is a future where software can break from devices and become more relational and conversational.

“We’re going to continue to use a complex sales dashboard or a CRM or product analytics, but instead now, you can do this work aloud while you’re doing the dishes, getting out of your driverless car, and in a whole new set of contexts throughout our lives.”

Are you a builder or entrepreneur in AI? Learn more from Bessemer: