In this blog, we will explore the process of crafting a GenAI solution architecture and uncover the essential building blocks for success.
Every new technology or framework/product experiences a “Honeymoon phase,” characterized by the allure of novelty and excitement, coupled with minimal expectations as users familiarize themselves with the technology and its potential business impact.
At the beginning, we hit the right industry terms and concepts, talk about fancy solutions that can impact business positively without having to go into details on how you actually can achieve it in real world, it is a matter of time before business asks, “How is it helping me?”
GenAI has reached a stage where the honeymoon is about to be over; it’s time to focus on real-world architectures and solutions that we can implement across customers.
While some of the big organizations have already started to show case the business value on GenAI and its related tools, others are struggling to realize the same.
My R&D
Based on my last 3 months of learning, ideation, R&D and my curiosity, i have tried to put across a set of key components that you need to have in a solution which you can call a GenAI-driven solution.
Just using Azure OpenAI, ChatGPT, Google Gemini, or other ready-to-use platforms, doesn’t make it a GenAI solution.
I am intentionally not putting a block diagram of the architecture, as it deserves a detailed post for itself.
But Before
Before going into technical architecture, there are two important topics that you need to have clarity on. The Business problem and what specific area of the problem you are trying to solve.
Clarity on Business Problem
The first and only thing you need to have clarity on is what problem you are trying to solve. Has someone already solved this problem, or is it still unresolved? It’s crucial to have a clear understanding of the problem and whether it can be addressed without relying on GenAI.
Answers to these questions are key to ensuring we define the right expectations for the outcome.
Focus Area on Business Problem
Once you identify the problem, depending on whether it’s already solved with the existing system or if you’re attempting to solve it for the first time (which is usually the case), you need to identify the key focus on the solution outcome.
- Cost
- Volumn
- Performance
- Accuracy
The technical details that you design, depends on answers to the above questions and focus.
With the business problem identified and focus on key outcome of the GenAI solution, below are key components that you need to include in your architecture to really get to a GenAI architecture instead of platform architecture,
Data Sources
You need to include problem specific data sources, including format of the data, the data attributes and different sources where you can retrieve the data and inject into GenAI driven solution.
The data source can be database, pdf, excel, web content or any other format that the business sees as fit and relevant to use.
Prompt Templates
Mapping of prompts to data to identify prospective prompt templates based on user personas and business domains. The prompts can be specific or templatized with place holder values for dynamic data.
SLM – LLM Combo
By now, you would have heard “Large Language Models” quite often to get used to it, but rarely hear what are known as SLMs (Small Language Models).
SLMs allows business to utilize for the results which getting similar results compared to LLMs at optimized cost and performance.
Some of the SLMs such as Phi3 – Mini is so powerful yet, small, it can run. in your laptops.
A typical architecture should include a combination of SLMs and LLMs for optimized approach for cost and performance. Cost and Performance optimization is key for any architecture.
Chaining
A GenAI solution cannot in most cases be achieved with single SLMs or LLM. It will be combination of different LMs working together with specific input and output to reach the specific results. This is known as prompt chaining and other names. Identify the chaining approach and sequence, sometimes make the solution delivery intended results at most optimized cost.
Platform to bring all these together.
Last but not the least, you need identify a platform to connect all together to ensure development, monitoring and go live at ease.
Here is where platforms such as OpenAI, Azure OpenAI, Gemini, Lang chain comes into picture. These platforms make the solution development faster and easy with ready to use models.
Conclusion
As I said, at the beginning, a GenAI solution is not about using the most popular platform. It is about going back to basics and design a solution that utilizes various building blocks of the technology and customize the same for your business needs. This approach is what best stands chance to provide the value that we all trying to achieve to us customers.
Feel free to correct me or add anything i missed. Happy to share and learn.