A spark of intelligence

The renewable energy sector has a big problem. It has the political and financial backing but lacks the ability to put new plants online fast enough to satisfy demand. My company, Nordic Smart Grid Solutions, provides software solutions and services to speed up capacity buildout. I will introduce one of our solutions that builds on generative AI to solve a key problem in the sector. Although the context, in this case, is the renewable energy sector, the architecture and the techniques are equally applicable to all kinds of different businesses across many industries.

Why build an AI assistant?

In this day and age, people work in highly specialized roles with expert domain knowledge in one area and rely on other experts to put complex projects together. AI agents are already widely used to fill in the gaps in certain contexts to speed up communication. This could be via directly answering user queries or by assisting experts to answer incoming questions more quickly by providing an initial draft to review before sending out.

We have found ourselves in a similar situation where we have a well-identifiable context with an already existing, large knowledge base that we want to interface with. To 'talk to the documents', so to speak. A classic use case for generative AI.

The meat of the problem then comes down to how we do this in a way that produces the outputs we want, not just slop. And more importantly, how can we achieve a real productivity increase rather than just creating a technical demonstration?

Understanding the users

As in any good product design process, we start with the end-user's perspective.

• Who is the target audience?
• What are their pain points?
• What is their current workflow?

No need to overcomplicate it. There are many more aspects we could look at, but this is a fine starting point.

Based on our analysis, we want a factually accurate, digestible, and quick text response.

Limitations and constraints

We identified the following main constraints:

• LLMs are known to hallucinate. In our use case, inaccurate or wrong answers can have serious consequences beyond losing a user's trust. We have to pay special attention to how we ground our AI.
• The corpus of available documentation the AI should draw from is vast. We need to provide context selectively and with precision.
• The user base is multi-lingual and has different qualification levels.
• They are also not tech enthusiasts with an existing workflow that often consists of email messages.
• For compliance reasons, we want to provide authorized, first-party references to mitigate the risk of inaccuracies.

Simple architecture

From these objectives and limitations, a simple, tried-and-proven LLM + RAG architecture is a natural starting point. To populate the RAG, we will use reliable documents made available to us by an authorized party. We will need to establish a ground truth dataset we can test and evaluate our solution against. This will be sourced from our in-house experts. The ground truth is a key component of the development lifecycle. The complexity of the solution can be increased almost indefinitely, but testing against the ground truth will guide our efforts in improving the solution to achieve our desired outcome while minimizing development time and system complexity.

User interface

To deploy our solution with the lowest user friction, we opted to create an email interface. Similar to a chat application, the email service will provide the question-and-answer format while easily integrating into existing workflows. It simplifies the complexities of authentication too, making it suitable for an MVP. It also makes user sessions (email threads) shareable and archivable as an added bonus.

Putting it all together

To summarize, we are building an AI assistant with an email front-end. Upon receiving a user query, the assistant generates a response based on its own world knowledge and additional contextual documents. These documents are vetted to ensure reliable grounding. As development progresses, the AI assistant is benchmarked against a ground truth dataset curated by industry experts.

With the motivation in place, in the following posts, I will go deeper into the technical challenges and our solutions to them. Stay tuned.