In an era where personalized, AI-driven experiences are becoming the norm, the way we interact with APIs is undergoing a significant transformation. GraphQL has emerged as the hot API technology, revolutionizing how enterprises build and manage APIs. As the Founder and CTO of Apollo GraphQL—the open-source company behind GraphQL Federation—I want to share insights into how GraphQL Federation is poised to integrate seamlessly with AI, shaping the future of API interactions.
The Rise of GraphQL in the Enterprise
Over the past few years, we’ve witnessed a remarkable adoption of GraphQL Federation across various industries—from retail and media to banking and beyond. Enterprises are leveraging GraphQL Federation to combine hundreds, even thousands, of business and domain APIs. This unified approach enables them to build world-class customer experiences rapidly and cost-effectively.
Driving factors such as personalization, omnichannel strategies, and mergers and acquisitions have underscored the need for a more efficient way to manage APIs. Now, the advent of AI is amplifying this need, pushing companies to rethink their API strategies to stay competitive.
The Challenge: Integrating AI with APIs
AI represents the next frontier of personalized experiences. Customers are beginning to expect AI-enabled interactions with their favorite brands and will gravitate towards those that offer them. Since apps are built on APIs, it’s imperative that we develop an AI strategy for our APIs.
However, a straightforward approach where Large Language Models (LLMs) interact directly with APIs—similar to the plugin model—isn’t scalable or secure. This method poses two significant challenges:
- Policy Enforcement: AI introduces novel workflows that traditional application code or integration APIs aren’t equipped to regulate. Without proper policy enforcement, users could make unauthorized API calls or access restricted data.
- LLM Limitations: LLMs lack the precise reasoning capabilities required to sequence API calls correctly, especially when dealing with complex transactions where the output of one call becomes the input for another. This limitation can lead to inefficiencies or errors that undermine the user experience.
The Solution: Federated GraphQL Layer
To address these challenges, we need an AI orchestration architecture that seamlessly integrates AI into the API stack. While Retrieval Augmented Generation (RAG) has been effective for integrating models with private data sources like knowledge bases, it falls short when dealing with the dynamic, transactional nature of APIs.
This is where the federated GraphQL layer comes into play. GraphQL Federation abstracts the complexities of underlying domain and business APIs, allowing developers to compose queries that fetch precisely the data they need. This self-service platform eliminates the need for custom integration code every time a new combination of APIs is required.
At the heart of this system is the GraphQL query planner. Similar to how SQL databases optimize queries, the query planner interprets the intent of a GraphQL query and translates it into an efficient execution plan. It sequences API calls, applies necessary joins and filters, and ensures that policies are enforced throughout the process.
How the Query Planner Enhances AI Capabilities
The query planner excels at tasks that LLMs struggle with, such as:
- Policy Enforcement: It can require specific scopes or credentials before executing certain operations, ensuring that only authorized actions are performed.
- Optimizing API Calls: By understanding dependencies between different data sources, the query planner can execute API calls in parallel where possible, improving performance.
- Handling Complex Queries: It efficiently manages intricate data fetching scenarios without the need for verbose, imperative code.
By leveraging the strengths of both LLMs and the query planner, we can create a powerful AI orchestration system.
Introducing Text-to-Query: Bridging AI and APIs
Our approach, known as Text-to-Query, harnesses the generative capabilities of LLMs and the execution prowess of the query planner. Here’s how it works:
- Natural Language Input: The user provides a request in plain English.
- LLM Processing: The LLM, informed by the GraphQL schema, generates a corresponding GraphQL query.
- Query Planning: The query planner validates the query, enforces policies, and devises an optimized execution plan.
- Execution: The query is executed, and the desired data is retrieved and returned to the user.
This division of labor allows the LLM to focus on understanding and generating human-like requests, while the query planner handles the technical execution.
Practical Examples
GitHub GraphQL API
Consider a request: “Get the title and author of the 10 most recent issues filed against the OpenAPI Specification repository.”
- LLM Output: Generates a GraphQL query that correctly splits the repository name into owner and name, selects the appropriate fields, and constructs a valid query.
- Execution: The query planner executes the query, and the data is retrieved efficiently.
Yelp GraphQL API
Another example: “Find three highly-rated sushi restaurants and include some reviews for each.”
- LLM Output: Produces a query that filters restaurants based on rating and fetches reviews.
- Execution: The federated GraphQL layer handles the data fetching across different services, returning a seamless response to the user.
These examples demonstrate how Text-to-Query can handle complex data retrieval tasks that would be challenging for an LLM alone.
The Architecture Behind Text-to-Query
The system comprises two main components:
- Schema Embedding System: This involves breaking down the GraphQL schema into manageable chunks and creating embeddings stored in a vector database. This setup allows the LLM to stay updated with schema changes, which is crucial in enterprise environments where APIs evolve rapidly.
- LLM Query Generation: When a user makes a request, relevant schema parts are retrieved and fed into the LLM along with the prompt. The LLM then generates a GraphQL query tailored to the user’s needs.
This architecture ensures that the LLM has the necessary context to generate accurate queries, while the query planner maintains control over execution and policy enforcement.
Envisioning AI-Enabled Experiences
Imagine a future where interacting with businesses is as simple as having a conversation:
- “I have two weeks free in June and want to visit Taiwan and Japan with my two kids. Can you suggest itineraries with available flights and hotels?”
- “Book me tickets for the next Thursday showing of Oppenheimer in 70mm IMAX, with four seats together at least five rows from the front.”
These personalized, dynamic experiences become possible when AI is integrated effectively with APIs through GraphQL Federation.
Path to Adoption: From Low Trust to High Trust
Implementing this vision involves a phased approach:
- Low Trust: Start by using the system internally to accelerate application development. Developers can generate code and queries quickly, reviewing them before deployment.
- Medium Trust: Gradually introduce customer-facing AI features, initially focusing on read-only experiences. As confidence grows, expand to include transactional capabilities.
- High Trust: Eventually, enable customers to bring their own LLMs, requiring robust policy enforcement and architectural safeguards to handle untrusted models.
This progression allows organizations to adopt AI integration responsibly, building trust and refining the system over time.
The integration of AI and APIs is not just a technological advancement but a shift towards more human-centric interactions with digital services. GraphQL Federation provides the architecture to make this integration seamless, efficient, and secure. By combining the generative strengths of LLMs with the precision of the query planner, we’re unlocking new possibilities for customer experiences.
At Apollo GraphQL, we’re excited about this future and invite you to join us on this journey. If you’re interested in exploring how GraphQL Federation can enhance your AI strategy, we’d love to collaborate.