Recent months have seen remarkable advances in conversational AI technology. Companies like OpenAI, Hume, and ElevenLabs have made significant breakthroughs in synthetic voice technology, while new low-latency language models (LLMs) enable near human-level conversations that far exceed traditional chatbot capabilities.
With the industry buzz around these new AI capabilities, we wanted to test a specific hypothesis: Would users prefer these conversational AI agents over Video Canvas's visual approach for consuming financial information? We conducted a head-to-head comparison to find out.
We designed a comparative study focusing on pension communications - a critical area where clear understanding is essential. Ten participants were asked to review their annual pension information, including current values and fees, using two different approaches:
Our research focused on how effectively users could understand and retain their pension information through each UX. The findings revealed significant differences in information absorption and comprehension.
AI agent experience
Despite the sophisticated conversational capabilities, users encountered several challenges. Key information was often missed even when the AI prompted for it, and the lack of visual reference points made it harder to retain numeric information. Users had to actively remember to ask about specific topics, and many felt uncertain whether they had covered everything important. While the conversational format felt natural, it didn't guarantee comprehensive coverage.
As one participant noted: "It was a little bit difficult to take in the information with the AI because it's just audible." Another mentioned having to "keep that question and then ask it later on," highlighting the cognitive load of managing the conversation.
Video Canvas experience
The structured visual approach showed clear advantages for information comprehension. Users could see and hear information simultaneously, with numbers and data clearly displayed while being explained. The guided journey ensured comprehensive coverage, and visual references made it easier to remember key points and review information.
User feedback strongly supported these benefits:
"It broke everything down in a really visual way, which made it a lot easier to actually see that information as well as hearing it."
"The data and the numbers are visible... it's more specific and less likely for any type of mistakes."
"Really liked how detailed it was... very aware of what's going on, very aware of the performance, and very aware of how much it's gonna cost."
The results showed an overwhelming preference for Video Canvas, with 7 out of 10 participants strongly favouring the Video Canvas. Notably, these participants expressed clear and decisive reasons for their choice, emphasising the benefits of visual data presentation and structured information flow.
AI agent feedback
While 3 participants preferred the AI agent, their preference was notably less emphatic. Those who selected the AI agent over Video Canvas expressed only a slight preference, primarily citing a single feature: the ability to ask additional questions. Key feedback included the ability to "ask it all sorts of questions" and get "on the fly" answers. However, these participants acknowledged that this advantage was somewhat theoretical - the ability to ask anything was appealing, but in practice, they weren't always sure what to ask or whether they had covered everything important.
Video Canvas supporters
In contrast, those who preferred Video Canvas expressed strong enthusiasm for multiple aspects of the platform, from its "very clearly structured" approach to how it "broke everything down in a really visual way." They particularly appreciated how the experience combined visual data, clear structure, and comprehensive coverage into a cohesive experience.
Our research uncovered several significant challenges with using conversational AI for financial communications.
Complex technical requirements
The AI agent required multiple third-party services to function: voice recording in the browser, ElevenLabs for voice processing, Google Gemini for natural language understanding, and custom knowledge constraints to prevent AI hallucinations. This technical complexity isn't just a development challenge - it directly impacts the user experience and raises privacy concerns.
Privacy and security barriers
The multi-vendor approach created several privacy challenges. Voice recordings must be sent to multiple third parties, users must explicitly accept privacy terms before using the system, and some users rejected the privacy terms entirely, making the system unusable. Financial institutions also face increased compliance complexity.
Technical dependencies
The reliance on multiple third-party services introduces risks around service availability, potential cost fluctuations, version compatibility challenges, and integration complexity.
Our research set out to test whether the latest conversational AI technology could provide a better experience for financial communications than Video Canvas's visual approach. The findings were clear:
While AI technology will continue to evolve, this research reveals a fundamental truth about financial communications: users need to not just hear their financial information, but see it, understand it, and feel confident they haven't missed anything important. As one participant succinctly put it:
"If a company was to communicate with me, my strong preference would be the Video Canvas format. It communicates more clearly and gives me something to hand that I can repeat over and over."
The future of financial communications will involve AI, but our research suggests it will be as a complement to, rather than a replacement for, clear visual communication that puts user understanding first.
Money Alive Ltd 2025
