At Quix, we say that real-time stream processing is human
Humans can’t survive without continuously processing and acting on data as it arrives. We take in information from our environment, transform it with various biological processes and react immediately. Most human activities involve real-time stream processing, even if our bodies run systems without our conscious awareness.
Imagine if humans didn’t do this. What if we operated on batch processing? What would happen if we took photographs every thirty seconds to analyze once a day rather than constantly looking around? How would it go if we relied on historical data to navigate grocery stores? Life would be a lot like herding cats. (Well, certainly a lot worse than that.)
We need our technology to rely on stream processing so that it can better support human activity. Data tech increasingly supports our personal and business lives, and that’s not going to stop anytime soon. The products and tools we build to help people make financial decisions for their families and companies, doctors to diagnose patients, and smart cities to operate sustainably cannot succeed if they function slower than the markets, biology, and vehicles. Without stream processing, data products and tools can’t keep up with humans and human society.
Thankfully, the fact that streaming is human makes streaming architecture a little less overwhelming to conceptualize, build, maintain and improve. We think that individuals and teams can overcome streaming challenges with a stack of the right microservices. Challenges might still exist, but they don’t stem from fundamental ideas.
Learn how to build a sentiment analysis Python service that connects, transforms and delivers data in real time
At the London Python User Group meetup in April, Tomas Neubauer and Javier Blanco built a chat application that keeps out unwanted language and tracks the positive or negative quality of each message let in. You can build it too.
Kafka Summit London 2022: The Full Recap
Kafka Summit returned in all its glory to welcome over 1,200 people — real human beings — back to discuss steam processing in person.
Building a real-time chat app with React, Laravel and Ably
The Stream is out and about in May
Meet Quix and other streamers at the World Data Summit in Amsterdam, DevOps Pro Europe and Machine Learning Prague 2022.