Develop Copilot Your Way

In the early days of app development, the focus was primarily on functionality and usability. Developers aimed to create apps that addressed specific user needs, with straightforward designs and user interfaces. However, with the advent of AI, the paradigm of app development has shifted dramatically. AI technologies such as machine learning, natural language processing, and computer vision have opened new possibilities for creating intelligent and responsive apps. Microsoft Copilot exemplifies how AI can be seamlessly integrated into business applications to provide real-time assistance, automate tasks, and offer actionable insights. These AI-driven assistants, or “AI agents,” enhance productivity and streamline workflows, transforming the way users interact with software. However, organizations often face a critical decision: should they purchase an off-the-shelf solution (ex. Microsoft 365 Copilot), build a custom copilot tailored to their specific needs using low-code or pro-code approaches? This decision involves several key considerations that must be carefully weighed. 

“With the advent of AI, the paradigm of app development has shifted dramatically”

Strategic Selection Process

When guiding customers at Hitachi Solutions, we assess an organization’s unique needs and assist them to make informed decisions about implementing AI agents. Some of the key factors we consider include:

• Specific problems or challenges AI agents are intended to address
• The types of data available (for grounding), and its location
• The nature of the data, its privacy and security
• Compliance or regulatory requirements
• The need for integration with existing systems
• Required technical expertise to build and maintain the AI solution
• Total cost of ownership

Technology Stack

In the past couple of years, Microsoft has introduced several tools and services that can accelerate an organization’s AI development journey. The technology stack to build custom Copilots or AI agents encompasses a range of advanced AI technologies, cloud services, and development tools. These components can be broken out into multiple layers as represented on the following diagram.

The UX layer is responsible for interpreting user requests and providing responses. The user interface can be part of a new application or an enhancement to an existing enterprise application.

The orchestration layer in the middle is responsible for coordinating various activities such as execution of business logic, communication with LLMs and extending capabilities via modular add-ons. Langchain, Semantic Kernel and Prompt flow are some of the examples of orchestration engines.

Note: Copilot Studio is a Saas offering, which offers limited control over selection of the backend models, but it provides a lot of great options to build UI for conversational chatbots. In comparison, Azure AI Studio does not have any features to target UX layer, but it offers other features to support pro-code development.

For instance, developers can select popular models from OpenAI, Microsoft, Hugging Face and Databricks to build their applications. One of the other key features of Azure AI Studio is the ability to customize an LLM through training, if needed. 

Tailored Approaches

The approach adopted to build and deploy AI agents can broadly fall into one of the following categories:

• Low-code approaches for rapid deployment and ease of use
• Pro-code approaches for maximum customization and control
• Hybrid approaches that balance flexibility and efficiency

Each approach has its advantages, and the best choice depends on factors such as skills, time constraints, budget, and the specific problem you aim to solve.

Low Code

A low code approach could comprise of enabling Microsoft first party-solutions such as Microsoft 365 Copilot. If the base functionality of a first party Copilot is not adequate, it can be extended via plugins making it a suitable option for organizations seeking  to minimize the custom development effort and embrace  the associated licensing costs.

Microsoft Copilot Studio is a SaaS offering from Microsoft to extend the capabilities of first party Copilots or build a custom copilot from the ground up. Copilot Studio allows developing the user experiences in a browser without the need to install any development tools. Since the provisioning and hosting of the associated resources is managed by Microsoft, the organizations can create functional prototypes quickly and deploy them via various supported channels (ex. Teams, Slack, Facebook, SharePoint or custom website). Copilot Studio also allows business analysts and citizen developers to extend the functionality by leveraging Microsoft Power Platform.   

Though enabling first party Copilots and/or building custom copilots using Copilot Studio are great options, they may not be a perfect fit for some organizations that have specific requirements around data security, data residency, large language model selection, network security, expected performance, etc. There are several options available in the pro-code space for creating custom AI solutions with extensive control and flexibility. What development tools you select to build a bespoke solution depends on the core requirements.

Pro Code

Azure AI Studio is a comprehensive platform provided by Microsoft to streamline the entire AI development lifecycle. Developers and data scientists can select a variety of large language models (llms) from OpenAI, Hugging Face or Microsoft, and host them within their organization’s Azure ecosystem.

The tools available in Azure AI studio allow developers and data scientists to build and test custom workflows using Prompt flow. The debugging tools available with Prompt flow can help developers trace interactions with models, evaluate quality and performance with larger datasets before promoting the features to production environment.

With a pro code approach, not only  is there plenty of choice in selecting appropriate AI services from Microsoft, but organizations can also leverage platforms such as Databricks to train custom models or implement Retrieval Augmented Generation (RAG) technique. The procode approach also allows organizations to inject intelligence into their existing enterprise applications and ensure consistency in data handling, user interface, and minimize the learning curve.   

Hybrid Approach

A hybrid approach for building copilots refers to combining the best of both methodologies and technologies from low-code and pro-code approaches. The combination brings together the ease of low-code platform with the power of pro-code tools, to build sophisticated AI agents that can potentially be quicker to build and easier to maintain. In the following reference architecture, the front-end chat interface is implemented using Copilot Studio while the backend could implement a sophisticated RAG architecture integrating enterprise data sources for grounding responses from LLMs. The pre-built components and templates available in Copilot Studio can allow for rapid development of the chat interface. The in-built functionality to process natural language, extract intent can be configured to invoke the custom backend for specific scenarios.

Deciding whether to enable, extend or build a copilot involves a careful evaluation of multiple factors, including cost, time to market, customization needs, available expertise, integration requirements, security considerations, and long-term vision. By thoroughly assessing these key considerations, organizations can make an informed decision that aligns with their strategic objectives and operational capabilities. Whichever path you choose, the integration of an AI-driven copilot can significantly enhance productivity, streamline workflows, and provide valuable insights, driving your organization toward greater success in the digital age.

In a recent webinar presentation,  we discuss this topic in much more detail.

View the full webinar now available on-demand for an expert-led presentation of this topic. If you’re interested in hands-on learning for Copilot Studio, check out our free training here.