Common Mistakes in the Development of AI Assistants

How AI assistants are often developed for companies, why they fail and what we can learn from them.

“The fault-finder will find faults even in paradise in the development of AI assistants.”
(Henry David Thoreau ChatGPT)

How fortunate that people make mistakes: because we can learn from them and improve. We have closely observed how companies around the world have implemented AI assistants in recent months and have, unfortunately, often seen them fail. We would like to share with you how these failures occurred and what can be learned from them for future projects: So that AI assistants can be implemented more successfully in the future!

How AI assistants have been developed in companies so far

A Story in 5 acts: Pitfalls in AI assistant development

Chapter 1: The vision

Every project involving AI assistants starts with an ambitious vision: to develop an assistant that can master specific challenges - from answering complex questions to analyzing business data.

The goals are clear: to increase process efficiency, speed up decision-making and ultimately drive the company's success.

Chapter 2: The research

After thorough research, development teams choose advanced AI tools and platforms, convinced by their maturity and the many success stories in the market.

Popular options such as LangChain, LlamaIndex and QDrant provide the necessary technology to realize ambitious AI assistants.

The development phase is characterized by innovation and creativity. Teams immerse themselves in the world of vector databases, develop intelligent agents and experiment with complex algorithms to bring the first prototypes to life.

The initial results are promising: the AI assistants provide precise and well-founded answers, a strong indication of the project's potential.

Chapter 3: The problems become apparent

However, unexpected difficulties arise after the successful launch.

The systems are prone to errors and "hallucinations", especially for specific queries, which undermines confidence in the technology.

The realization that even advanced AI systems have their limitations leads to the search for solutions to improve accuracy and reliability.

Chapter 4: In search of a solution

Fortunately, public resources offer many ways to improve the accuracy of a misbehaving AI assistant. Well, at least that's what they claim... Common ideas include:

  • Better prompting

  • Tweaking retrieval and fine-tuning embeddings

  • Query transformations (expansion)

  • Chunk reranking

  • Adding multiple agents building routers

What actually results from this? Often an unnecessarily complex architecture of the AI assistant. Despite attempts to improve it, it will not always be able to answer simple questions. It will lack accuracy and occasionally generate incorrect information: The system will still hallucinate.

What were the pitfalls that caused the team to fail?

At this point, we have good news: The mistakes that many teams make when developing AI assistants, as described here, can be traced back to three frequently incorrect assumptions:

3 common, incorrect assumptions in AI assistant development

False Assumption 1

"Publicly available materials and articles tell the full truth about how cutting-edge AI systems actually work in business."

False Assumption 2

"If we break our documents into small pieces and then insert them into a vector database, the AI will be able to create meaning from them as if by magic."

False Assumption 3

"We need to build an all-powerful AI assistant with complex architecture to bring real value to the business."

What concrete measures can be derived to successfully implement AI assistants?

Discover our tried-and-tested tips & tricks to help you avoid common pitfalls during implementation and build good AI assistants for companies cost-effectively.

8 tips for developing AI assistants

Would you like to use KI Assistants in your company?

Then we look forward to hearing from you.

* required

We use the data you send us only for contacting you in connection with your request. You can find all further information in our privacy policy.

Solve captcha, please!

captcha image
Martin Warnung
Sales Consultant TIMETOACT GROUP Österreich GmbH +43 664 881 788 80
Contact
Blog 5/17/24

8 tips for developing AI assistants

8 practical tips for implementing AI assistants
Wissen 4/30/24

GPT & Co: The best language models for digital products

Our analysis based on real benchmark data reveals which solutions excel in document processing, CRM integration, external integration, marketing support and code generation. Find your ideal model!

Wissen 7/30/24

LLM-Benchmarks July 2024

This LLM Leaderboard from July 2024 helps to find the best Large Language Model for digital product development.

Wissen 6/30/24

LLM-Benchmarks June 2024

This LLM Leaderboard from june 2024 helps to find the best Large Language Model for digital product development.

Wissen 4/30/24

LLM-Benchmarks April 2024

This LLM Leaderboard from april 2024 helps to find the best Large Language Model for digital product development.

Wissen 8/30/24

LLM-Benchmarks August 2024

Instead of our general LLM benchmarks, we present the first benchmark of different AI architectures in August.
Wissen 5/30/24

LLM-Benchmarks May 2024

This LLM Leaderboard from may 2024 helps to find the best Large Language Model for digital product development.

Blog 2/3/25

ChatGPT & Co: LLM Benchmarks for January

Find out which large language models outperformed in the January 2025 benchmarks. Stay informed on the latest AI developments and performance metrics.
Blog 10/1/24

ChatGPT & Co: LLM Benchmarks for September

Find out which large language models outperformed in the September 2024 benchmarks. Stay informed on the latest AI developments and performance metrics.
Blog 1/7/25

ChatGPT & Co: LLM Benchmarks for December

Find out which large language models outperformed in the December 2024 benchmarks. Stay informed on the latest AI developments and performance metrics.
Blog 11/12/24

ChatGPT & Co: LLM Benchmarks for October

Find out which large language models outperformed in the October 2024 benchmarks. Stay informed on the latest AI developments and performance metrics.
Blog 12/4/24

ChatGPT & Co: LLM Benchmarks for November

Find out which large language models outperformed in the November 2024 benchmarks. Stay informed on the latest AI developments and performance metrics.
Navigationsbild zu Data Science
Service

AI & Data Science

We offer comprehensive solutions in the fields of data science, machine learning and AI that are tailored to your specific challenges and goals.
Headerbild zur Logistik- und Transportbranche
Branche

AI & Digitization for the Transportation and Logistics Indus

Digitale, transparente Prozesse und automatisierte Optimierung helfen Logistikunternehmen, Kosten und Leistung besser auszubalancieren, für eine starke, zukunftsfähige Rolle als Partner der Wirtschaft
Wissen 5/2/24

Unlock the Potential of Data Culture in Your Organization

Are you ready to revolutionize your organization's potential by unleashing the power of data culture? Imagine a workplace where every decision is backed by insights, every strategy informed by data, and every employee equipped to navigate the digital landscape with confidence. This is the transformative impact of cultivating a robust data culture within your enterprise.
Branche

Digitization of the energy industry

The energy sector is undergoing an unstoppable process of change. Progressive digitization and the energy transition are causing traditional system and process boundaries to disappear.
News 1/26/21

The IPG Group becomes part of the TIMETOACT GROUP

The TIMETOACT GROUP acquires the majority of the shares of IPG Information Process Group Holding AG, based in Winterthur. Through the acquisition, the competencies for Identity and Access Management (IAM) solutions in the DACH market are combined.
Headerbild zur AI Factory for Insurance
Service 7/5/21

AI Factory for Insurance

The AI Factory for Insurance is an innovative organisational model combined with a flexible, modular IT architecture. It is an innovation and implementation factory to systematically develop, train and deploy AI models in digital business processes.
News

Proof-of-Value Workshop

Today's businesses need data integration solutions that offer open, reusable standards and a complete, innovative portfolio of data capabilities. Apply for one of our free workshops!
Referenz

Consulting on the ivv collaboration strategy

The future collaboration of ivv is characterized by modern communication and collaboration tools. It is defined for cross-organizational work in association and with external parties.