Want more intelligent visions of your inbox? Subscribe to our weekly newsletters to get what is concerned only for institutions AI, data and security leaders. Subscribe now
Valuable broadcast data can be valuable for many applications and purposes across industries. in case New York Stock Exchange (NYSEData flow is literally money.
The New York Stock Exchange is one of the largest financial stock exchanges in the world and has a long history of the ability to share financial market data.
A hundred years ago, the Telegraph Index Rub used the document to Telegraph to share information. In the modern era, high -performance high -performance techniques have developed their own that can communicate with them.
Now takes the next step forwardEmbrace a model based on APache Kafka Open Source Flowing technology that brings NYSE BEST and Trades (BQT) data to AWS Cloud.
Artificial intelligence limits its limits
Power caps, high costs of the symbol, and inference delay are reshaped. Join our exclusive salon to discover how the big difference:
- Transforming energy into a strategic advantage
- Teaching effective reasoning for real productivity gains
- Opening the return on competitive investment with sustainable artificial intelligence systems
Securing your place to stay in the foreground: https://bit.ly/4mwngngo
To do this, NYSE has made a partnership with the data platform seller RedbandaAnd her own developed Implementation of Kafka Written in programming C ++.
NYSES’s posting for the Nyses-based broadcast system from Redpanda performance improvements 4-5X On the traditional Kafka competitors, they offer basic restrictions on how most institutions deal with data data work.
This performance gap becomes decisive as institutions make AI applications that require steady access from low technology data. The Kafka -based data flow also has the possibility to enable communications to the agent, and compete with other curricula Google’s A2A It can also be extended to enable the form of the context of the context of the model (MCP).
“The market thesis is that all large foundation models have already indexed general data groups, and the following borders are private data sets, and that Redpanda really opens private data sets to access the agent,”
What the New York Stock Exchange is building in the cloud
NYSE has built its cloud flow platform for customer service who cannot access its data centers directly. Exchange targets Fintech companies and retailers who need to access to AWS to market data in actual time.
“Not every consumer of our market data is the ability to access our data center, take a summary and use this feeding,” said Vinil Bhandari, President of Cloud and Full Stack Engineering in Nyse Venturebeat. “But you know that a small store in Hong Kong has access to their AWS account, for example, the fans we are trying to meet their needs.”
NYSE flows BQT (Best Quotes and Trades), which collects data in actual time from all seven NYSE exchanges. Publishing requires building a new infrastructure instead of extending the current systems.
Why did the New York Stock Exchange chose Redpanda and how it is important to choose the programming language
The New York Stock Exchange processes more than 500 billion messages per day through seven exchanges. During the fluctuation of the market, The size of the message can rise 1000X above the average inside MicroseConds.
Traditional Java applications are struggling with these patterns because collecting garbage creates unpredictable cumin nails.
“The classic Kafka app has been written in the Java programming language, which makes this type of traffic this type of visits, as you know, is not very good with the Java’s Garbage collection that occurs in the programming language,” Bhandari explained. “Redpanda carried out Kafka by rewriting the Kafka Protocol in C ++, so the more we get a traffic from our market activity, fluctuation, we can better manage these data.”
The choice of programming language is the reason why Nyse has gone with Redpanda to flow data instead of other options such as Confluent or Amazon that manages Kafka (MSK) broadcasts.
This technical decision has led to performance improvements.
“We are safe to prove that We are at least four to five times faster to deliver our data using Redpanda Compared to some of our competing with large tickets who use Kafka technology for similar data flow. “
For institutions that evaluate broadcasting platforms, this comparison sheds light on cash consideration: Java -based applications for data flow may be struggled during traffic rising, while C ++ alternatives can maintain a fixed performance.
The observation of the observation is very important to publish the task
Bhandari emphasized the observation as necessary to spread production flow. Remar measurement capabilities in Redpanda provided an immediate operational value.
“The more the publication is like this, it can have the possibility of observation and remote measurement of what is happening under the cover, the better the data product and consumers of data,” he explained by Handari.
This observation allows the disclosure of issues and pre -emptive resolve before problems affect customers. Without comprehensive monitoring, institutions risk discovering performance problems only after they affect the burdens of production work and customer experience.
Architectural engineering philosophy: Amnesty International Foundation
NYSE will use broadcast data capabilities in a fairly traditional way, at least in the beginning. These are the data from its market exchange that is provided to users for consumption.
RDPanda’s direction refers to the future of more than AIC, which is likely to embrace users such as NYSE in the coming years. Redpanda Gallego CEO argues that institutions must display the structure of flow differently in the era of artificial intelligence.
“The flow has the correct architectural style, not for speed, but because it is the correct structure of interactive applications and factors,” Gallowo explained.
In addition to solving traditional broadcasting problems, RDPanda has re -set itself to what Gallego’s Agentic Gallego calls. The company has concluded its data connections in MCP (the form of the context of the context of the model), allowing artificial intelligence agents to access directly to the Foundation’s data sources.
This approach solves the problem of arithmetic complexity that appears as institutions that publish multiple Amnesty International factors.
“Without the Kafka applications interface, you have a square communication problem where each agent must access each other agent,” said Gallowo. “And when API Kafka offers, it reduces the rhetoric complicate to the linear.”
According to Gallego, banks are already publishing hundreds of agents. A Redpanda customer plans to build 1,000 agents over the next two years. Another is currently building 130 agents to spread production within 18 months. These size requirements make the agent coordination intention important for the success of the long -term artificial intelligence strategy.
What does this mean for the institution’s data strategy
The broadcast data has been set in actual time to become an increasingly important aspect of the operations of many organizations.
The NYSE evaluation process reveals critical decision standards for institutional decision makers who evaluate the flow infrastructure:
Kafka is headed by Java, the walls of performance under traffic. Organizations that deal with unexpected work burden must evaluate the C ++ alternatives before scaling production spread. The difference in performance 4-5X is not a marginal improvement but the basic power gap.
The first cloud broadcasting strategies can achieve the production category. This enables access to global data that was previously impractical due to cumin restrictions, which opens the new market opportunities for data -based companies.
The coordination of the agent requires the structure of the flow. With the expansion of the spread of artificial intelligence beyond individual agents, broadcasting platforms become a basic infrastructure rather than performance improvements. The advantages of the arithmetic complexity become widely decisive.
For institutions that plan for artificial intelligence applications, it is very important to give priority to broadcasting platforms that support MCP integration and the coordination of the agent. The advantages of the arithmetic complexity become very important and re -update the structure of coordination after spreading multiple factors, which proves more difficult to build them properly from the beginning.
Organizations awaiting artificial intelligence must realize that the decisions of the broadcasting structure that have been taken today will restrict the capabilities of artificial intelligence more than most leaders.
https://venturebeat.com/wp-content/uploads/2025/08/data_streaming_NYSE-SMK2.jpg?w=1024?w=1200&strip=all
Source link