Machine Learning
Behind the Scenes: Delivering Real-Time News to Apps
In today’s fast-paced digital world, delivering news in real-time has become a fundamental expectation for users. This is especially true in the context of sports apps, where up-to-the-minute updates can make or break the user experience. But what does it take to build and maintain a system capable of delivering seamless news updates, push notifications,…
From Logs to Insights: Leveraging Load Balancer Data for Smarter Monitoring
In a complex infrastructure, where load balancers are key to ensuring smooth traffic flow, gaining actionable insights from traffic patterns is critical. The challenge? Turning raw log data into meaningful information. Recently, we embarked on a project to extract valuable data from our load balancer logs, integrate it into our monitoring system, and prepare for…
·
From Time Series to Insights: Building Machine Learning Models with InfluxDB
In this blog, we will walk you through a practical process of extracting data from an InfluxDB time-series database, preparing it for analysis, and leveraging it to train machine learning models. InfluxDB is a powerful time-series database designed for metrics and events. Along the way, we’ll also explore how to identify the best features for your models.…
The Human Side of Technology: Learning, Growing, and Creating
With 35 years of experience in programming and managing complex systems, I thought I had seen it all. But the past few months have shown me there is always something new to learn. While I have a strong foundation in IT, technologies like MongoDB, InfluxDB, SQLite, Python, and machine learning were uncharted territory for me.…
By 2030, Our Monitoring System Decided We Weren’t Needed Anymore
Six years ago, it was just a humble monitoring project. We wrote a few scripts, set up agents, and proudly collected CPU and RAM metrics. But by 2030, that “simple system” has evolved into something far beyond what we ever imagined—or intended. Somewhere along the way, it decided it didn’t need us anymore. Spoiler alert:…
Proactive Monitoring: Building a Smarter IT Infrastructure
Managing an IT environment goes beyond simply keeping the lights on. As infrastructures grow, so do the complexities of managing them. Reacting to issues after they occur is no longer enough. That’s why we built a proactive monitoring system—one that doesn’t just detect problems but helps predict and prevent them. The Monitoring Challenge With dozens…
From Logs to Intelligence: A Day in the Life of Building a Smarter Monitoring System
What started as a normal workday quickly turned into an epic adventure filled with logs, tables, APIs, and the occasional “database locked” error. Today, I teamed up with my trusted assistant (yes, that’s you, Binky!) to build a smarter, leaner monitoring infrastructure. Along the way, we faced mysterious InfluxDB issues, tangled SQL logic, and the…
Scaling Our Monitoring System with Dynamic Agents
In our previous blog, we explored how we integrated data collection and predictive modeling into our API. Since then, we’ve taken our monitoring system to the next level with a dynamic agent-based approach. Here’s what we’ve accomplished: The Problem We Solved As we scaled up, we realized that each server’s monitoring requirements could differ. Some…
From Data to Predictions – Building a Smarter Monitoring System
In the ever-evolving tech landscape, predictive monitoring isn’t just a luxury – it’s a necessity. What if your servers could tell you when they need updates, resources, or attention? With data collected via our API and some machine learning magic, we’re taking the first steps into smarter, data-driven monitoring. In this blog, we’ll explore how…
Building a Secure, Configurable, and Scalable Monitoring API
In a world where infrastructure monitoring is vital, having a flexible and secure way to collect and process custom data is essential. In this blog post, we’ll take you through building an API that not only allows for dynamic field configurations but also includes robust security with API keys tied to specific servers. We’ll walk…
Reviving Backbone Legacy Systems with Modern Automation Tools
Legacy systems are often the backbone of long-standing organizations, housing valuable data and complex workflows that modernize at a snail’s pace. But with today’s automation tools, breathing new life into these systems is easier than ever. In this blog, we’ll explore strategies to enhance and streamline legacy systems using automation, without the need for a…
Extract SNMP and Syslog Data from InfluxDB
After setting up our tools for monitoring in the first series of our Machine Learning Project we have gathers a lot of data. The next step is to query InfluxDB for the relevant SNMP metrics and syslog events. We’ll use Python with the InfluxDB client to connect and extract data into Pandas DataFrames. Set Up…
