Tried and Tested: The 13 Pillars of Success with Conversational AI


Based on my research and what I could (or rather couldn’t!) find, I’m setting out my stall that this is an industry-first article. 

I have had the privilege of working in the automation space for the majority of my career. Combining this with being a technologist at heart, I’ve had front-row seats to the change and evolution of one of the fastest-paced industries in the world. 

My responsibilities today can be described as helping join the dots between the art-of-the-possible with Conversational and Generative AI, and the strategies of the enterprises we work with. In practice, that means helping clients engage early in their digital transformation process, understanding their business, and serving as an advisor long-term in bringing a step change in digital transformation within their organisation.

To make these journeys successful for our customers we’ve created guides, playbooks, and operating models. Yet for the benefit of the wider adopters of Conversational AI, I wanted to explain some of the foundational steps and milestones that will help ensure your organisation’s success when adopting this rapidly evolving technology. 


1. Level Up Knowledge 

I know what you’re thinking, why not start with strategy? Well, we’re bucking that trend! First, let’s focus on knowledge. 

If you’re a business leader who is exploring Conversational AI or someone who has been given this directive then you first need to immerse yourself in the state of the market. Conversational AI is a crowded market with over 3,000 vendors placing some type of stake in this ground. Furthermore, with the introduction of Generative AI research firms, for example, Gartner has suspended their assessment programs (like the Magic Quadrant) in anticipation of Conversational AI and Generative AI being part of some new category. 

However, there are a variety of ways to level up your knowledge based on what is known today; this can be in the form of webinars, events, customer (competitors potentially) case studies, and online courses (for all types of roles). But what is important is that you get a baseline knowledge from which to build and educate from. Connect with me on LinkedIn, and I’ll post key items of research as they are published!


2. Strategy, Objectives, and Goals

For a business to benefit from any technological advancement, it must be clear on the problems or opportunities it supports and understand how it can translate into a solution. 

First, start with what you want to achieve and work back from there. In my career, I’ve witnessed how Conversational AI can not only reduce operating costs and increase revenue, but also improve employee and customer satisfaction, provide access to new markets, and disrupt existing markets. But these value levers can only be achieved when there is a clear direction to the business in which to mobilise an initiative. Being clear of the end achievement will act as a source of clarity to both your organisation and the potential vendors who may be supporting you in this aim.


3. Stakeholder Mapping and Engagement

Let’s be clear, this is business transformation! I would question any vendor who claims otherwise. Having worked in automation most of my career, there are varying levels of positive impact. 

If you’re working automation into the back or middle office, your customers, policyholders, patients, or end users are likely abstracted from the change that occurs. When you deliver Conversational AI into your business you’re more than likely making a change to your front office which is the way your products and services are accessed, and how your brand is perceived. Take time to understand who is impacted, who needs a voice within your business, and who should be consulted. 

Whilst a Conversational AI program can be business-led, it should be inclusive of the affected departments and those who play a role in ensuring its success. By ensuring you have a clear understanding of the strategic goal, you can mobilise the business around these outcomes so you all row in the same direction. Conversational AI as with other drivers for business change can leverage traditional methodologies for stakeholder mapping; with a 4 box approach for measuring stakeholder level of interest vs. level of influence – then categorising who needs regular yet minimal contact, who needs to be kept completely informed, understanding whos needs you may need to anticipate, and those who you must be conscientiously managed.  


4. Brand and Experience

Your digital agents – the manifestation of Conversational AI – are part of your workforce and as such are representative of your brand and the experience you wish to provide your customers, partners, and employees. 

We mentioned above the need to ensure you involve the right stakeholders from the business, however, extra focus should be paid to the target persona of your digital agent or agents – and the experience you wish them to provide. 

If they are operating in a customer service capacity, consider the tone of voice they should use in line with your customer needs and demographics. Are you comfortable in the use of colloquial language, or emojis? You’ll likely already have guidelines for your human workforce in these areas which can be evolved as required. Lastly, consider the visibility of your virtual agents in line with your objectives and goals. Do your virtual agents sit passively and support your human workforce? Or do they have an image of their own, perhaps an avatar even – and do they reinforce your brand in validating your investment in the innovative ways you deliver industry-leading products and services? These are just some questions to consider. 


5. Legal and Ethics

Now, more than ever, it’s crucial to consider and prioritise legal and ethical considerations. Begin by ensuring virtual agent guidelines are set in compliance with the data protection laws you adhere to today. Do this by extending those which you already have. 

Transparency and explainability should be emphasised, providing clear information to users about data handling practices and system operations. Choose a vendor who can help and evidence how biases are mitigated to avoid discriminatory outcomes. Consider how you obtain user consent for data processing and offer easy opt-out options.

Whilst accessibility may be overlapping with the ‘experience’ mentioned earlier, you should ensure you can accommodate users with specific needs. Extend processes for system failures or ethical breaches and periodically audit and assess compliance alongside your border programs. Conversational AI vendors who specialise in this space, like OpenDialog, should be able to clearly articulate how your existing business policies, procedures, and rules can be applied to your virtual agents and monitored for ongoing governance purposes. 


6. Evaluate Need, Scalability, and Operational Costs

Conversation AI manifests itself as digital agents – effectively its digital labour within your business. And, in the same way you would capacity assess your human workforce against the tasks at hand, so should you assess the scalability of your digital workforce. 

Look to understand how many contacts or interactions you expect your digitall agent(s) to manage at initial release and over time. Consider the volume of work against any infrastructure of licence costs. The introduction of Conversational AI like any new technology will require some form of business case to articulate the cost vs. value. Whilst most business cases are underpinned by being able to deliver an outcome cheaper, better, and faster than it is achieved today – inspect the longer-term costs as this scales, such as the skills required to support and manage.


7. Integration Assessment

The bottom line is that digital agents are delivering an outcome. This may be supporting an existing use case or giving you the ability to deliver new products and services that simply could be achieved with people alone. 

Whether it’s a Patient Appointment Management use case or Mid-Term Adjustments for an insurance policy, more than likely there are systems and applications that need to be accessed. Again, look for vendors who specialise in your space. Integration can be north of 50% of the overall effort to stand up a use case so using pre-built integrations will drastically cut the effort of the initial service and simplify the in-life operationalization of your digital agents as you expand across your business. 


8. Data Governance

The footnote of this section is that the approach to data governance needs to align with the existing practices that you have today. If the introduction of Conversational AI causes you to introduce silo methods of operation, you are going to fail. The considerations should include; data ownership and accountability, quality and integrity, retention and deletion at a minimum – and again, with the emphasis on evolving your existing practices as opposed to setting up new ones.


9. Operationalization 

You likely want control and ownership of your virtual agents – though there may be a case for more of a managed service type approach and so, you’ll need skills within your business. 

Start by understanding what these skills are, and then where they need to sit. How do you want to identify new use cases, who is responsible for teaching the virtual agents, how will they be managed, how do you approach continuous improvement, and who needs to be provided with reports and analytics, how does this fit in with your wider change management policy? Vendors who have helped clients achieve scale should be equipping you with a means to industrialise, with an ability to align training and enablement for each of the specific roles. This should be something that evolves of course, but ensure you have a view on what good looks like 12, 24, 36 months down the line. 


10. Measure and Analyze

Your digital agents will only get better over time. There are robust approaches to testing and release which will lean on a blend of subject matter expert-defined testing and test scenarios defined through Generative AI to simplify the process. However, there will also be a need to iterate based on real-world experiences. You will want to define an ongoing means of measuring the virtual agents and feeding back to those stakeholder groups as defined in step 3. It’s possible to get more than 1 million data points per day on a use case – and whilst this will grant insights into your customer and user base like never before, there can be too much information if not assessed against wider criteria. Look back at your initially defined strategy, objectives, and goals to ensure the interpretation of the data and insights you collect are adding value to the business, and effectively communicate this need to those responsible for the operation of your virtual agents. 


11. Establish Feedback Protocols

This is a multi-faceted requirement. There will be multiple internal voices who should be consulted, but also consider the voice of your customers, end users, etc. Along with the responsibility of your virtual agents in executing the necessary steps of a process, consider how they can be adapted to gather feedback from the user during or at the end of a process. As humans, we are much better equipped to deliver feedback to machines, so exploit this behaviour when soliciting input on how you can improve. 


12. Vendor Relationship Management

Unless you’ve opted to build your own Conversational AI platform, there is likely a vendor involved in supporting your success. A lot of the common principles for vendor management apply here so I won’t duplicate the well-established protocols out there. That being said, do ensure that any vendors are aware of how you measure success, your objective, and your overall strategy. Once again, if you choose a vendor that specialises in your space, they will understand your language; be it patient experience, policy profitability, loss ratios, profit per bed, etc. Ask vendors to share how they support you in real life, their approaches to customers’ success, and what options there are to collaborate as part of a wider community. 


13. Iterative Adoption

The final point of success is how deployment is approached across your enterprise. You will likely start your journey with a single use case or area in which to prove the value delivered to your business, whilst exposing the initial impact of this transformation. In parallel to this initial activity, you should build a pipeline of additional areas in which conversational AI can support in line with your goals and objectives. 

And there it is – the vital milestones to consider when successfully approaching and deploying Conversational AI within your business. For further details on our Operating Model and how we can help you transform your business – get in touch!

Share This Post

More To Explore

Fred Dust Podcast
OpenDialogues Podcast

Episode 7 – Making conversations

Welcome to the seventh episode of OpenDialogues, the podcast hosted by Maaike Coppens and Ronald Ashri discussing everything Conversation Design and OpenDialog.

See How it works!

Get in touch for a showcase of how OpenDialog can help your business Deploy Conversational AI, at scale.

Transformation eBook download
The Generative AI Transformation Guide

Stay ahead of the curve with Generative AI and automation

Download our free eBook to see how to leverage the power of Generative AI and automation safely and thrive in this new age.