Streaming has become a must-have element for the development of modern artificial intelligence applications, where fluid interaction and responsiveness are essential. In a context where LLMs (Large Language Models) are central, providing an interface that reacts instantly to user actions is now an expected standard. LangGraph, a powerful extension of LangChain designed around the concept of graphs, offers built-in streaming capabilities that allow for the creation of more transparent, interactive, and efficient agents.
Thanks to its advanced features, LangGraph enables real-time tracking of internal processing, whether it involves text generation, calling an external tool, or any other business process orchestrated by the agent. This live feedback not only enhances the user experience but also facilitates the development, debugging, and monitoring of complex systems.
In this article, we will dive into the different streaming modes offered by LangGraph and see how to best leverage them to build dynamic and highly informative interfaces.
Why use streaming in an AI application?
Integrating streaming into your AI agents offers numerous benefits for both the end user and the developer:
-> Reduction of perceived latency: by displaying responses word by word or step by step, the user feels that the system responds more quickly, even if the entire execution takes time.
-> Execution logic tracking: each step of the agent's reasoning can be displayed, which is useful for traceability, verifying the accuracy of responses, and understanding the agent's behavior.
-> Interface enrichment: thanks to real-time updates sent from tools or the graph, we can display progress indicators, explanatory logs, or intermediate states.
-> Thanks to real-time updates sent from tools or the graph, we can display progress indicators, explanatory logs, or intermediate states.
These benefits allow for the design of more engaging and participatory user experiences, in which the machine reveals part of its reasoning, thereby increasing user trust.
The different streaming modes in LangGraph
LangGraph offers three main types of streams, which can be activated independently or in combination, depending on your application's needs:
This mode allows emitting an event at each step taken in the graph. It is particularly useful for getting an overview of the agent's journey through different nodes:
python
In this example, we can receive successively:
An action generated by the LLM, such as a request to invoke a tool;
The return of a tool with its results;
The final message constructed from all the processed information.
This type of stream is ideal for monitoring the behavior of a complex agent and ensuring a good understanding of each phase of its execution.
2. Streaming of tokens generated by the LLM (stream_mode="messages")
This mode allows receiving tokens as they are generated by the model, simulating the real-time writing of a response.
python
This method is ideal for conversational interfaces, real-time chats, writing assistants, or any other case where the immediacy of the generated text enhances interaction. It also allows stopping a stream at any moment if the user changes their mind, which reduces processing costs.
3. Custom Streaming from tools (stream_mode="custom")
LangGraph allows inserting custom emission points within business functions (called "tools") using the get_stream_writer() function. This gives developers the power to transmit any data in real-time:
python
This type of stream is ideal for sending log messages, business progress indicators (e.g., "fetching page 3/10"), or specific alerts. It enhances execution transparency and enriches user feedback.
Combination of streaming modes
LangGraph allows you to enable multiple streaming modes simultaneously, paving the way for a very comprehensive visualization of the agent's behavior:
python
This hybrid approach is recommended in use cases where transparency and interactivity are priorities, such as monitoring interfaces, analysis tools, or AI demonstrators.
If you are still unsure or need specific support, our team is here to help
Disabling streaming: when and why?
It is sometimes useful to disable streaming, especially in multi-agent architectures or distributed systems where too many events could saturate communication channels or overload interfaces. Some models may also not handle streaming well or require specific configurations.
In these cases, LangGraph allows precise targeting of which agents or tools should stream their data and which should opt for a complete response once processing is finished.
Concrete example: testing the different streaming modes
To better understand the differences between streaming modes, here is an example that you can run yourself. It uses a LangGraph agent with a Tavily search tool to answer a contextual question. The goal is to modify the variable <code>STREAM_MODE to observe the behavioral differences based on the activated streaming mode ("messages", "updates", or a combined list).
python
You can change the value of <code>STREAM_MODE and observe what changes in the display:
With "messages", you will see the model tokens display as they are generated.
With "updates", each logical step of the graph will be visible.
With "custom", this will produce nothing here, as emission points would need to be inserted into the agent.
With a combined list, you will get a complete view of the different layers of processing.
This approach is ideal for experimenting, debugging, or designing dynamic interfaces.
Conclusion
With its modular and powerful streaming capabilities, LangGraph provides you with the necessary tools to build rich, fluid, and transparent user experiences. Whether you're looking to make your applications more responsive, provide better visibility into your agents' decisions, or create more natural interactions, the different streaming modes adapt to your needs.
By integrating these live feedback possibilities from the design stage, you enhance both the technical efficiency of your application and the satisfaction of your end users.
Need a more interactive and understandable AI? Adopt streaming with LangGraph today and bring your conversational agents to life.
Envie d’aller plus loin ? Formez vos équipes à la conception et au déploiement d’agents LLM
Want to go further?
This topic is part of our Generative AI for Developers course. Browse the full programme, or get it by email.
Share with
Romain DE LA SOUCHÈRE
Tech Lead, CTO AXI Technologies
Expert Data Engineering et Cloud, Romain affiche plus de 11 ans d'expérience, dont plusieurs années comme Lead Developer sur des solutions Smart Building haute performance. Il y a conçu et mis en production des moteurs de traitement capables d'absorber des centaines de milliers de données de capteurs par minute, ainsi que des bases clusterisées gérant plus de 10 millions de données dynamiques. Certifié Microsoft Azure DevOps Engineer Expert, il maîtrise aussi bien le développement back-end (Python, C#) que le DevOps (Docker, Kubernetes, Terraform) et les agents LLM. Formateur en Python, cloud, DevOps et IA générative appliquée, il forme avec une obsession : Amener chaque apprenant à concevoir et déployer des architectures réellement scalables en production.
