From Time Series to Insights: Building Machine Learning Models with InfluxDB

In this blog, we will walk you through a practical process of extracting data from an InfluxDB time-series database, preparing it for analysis, and leveraging it to train machine learning models. InfluxDB is a powerful time-series database designed for metrics and events. Along the way, we’ll also explore how to identify the best features for your models. Whether you’re an InfluxDB enthusiast or a data science beginner, this guide will provide you with actionable steps to move from raw data to predictive insights.

1. Why InfluxDB?

As mentioned, InfluxDB is a powerful time-series database designed for metrics and events, making it an ideal choice for use cases like monitoring, IoT, and real-time analytics. Its native support for Flux, a functional query language, allows us to retrieve and manipulate data efficiently.

Machine learning models often benefit from time-series data for predictions and anomaly detection. However, raw data needs cleaning, transformation, and preparation before it can fuel predictive models. Here’s how you can make the most of it.

2. Retrieving Data from InfluxDB

First, let’s query and export data from InfluxDB. We’ll use the InfluxDB Python client to connect to the database and retrieve data using Flux queries.

Flux Query Example

Let’s query the net_eth0_out measurement from the past 7 days:


Copied!from influxdb_client import InfluxDBClient

client = InfluxDBClient(url="http://localhost:8086", token="your-token", org="your-org")
query = '''
from(bucket: "bigdata")
  |> range(start: -7d)
  |> filter(fn: (r) => r._measurement == "net_eth0_out")
  |> filter(fn: (r) => r._field == "value")
'''
tables = client.query_api().query(query=query)
data = []
for table in tables:
    for record in table.records:
        data.append({"time": record.get_time(), "value": record.get_value()})
from influxdb_client import InfluxDBClient

client = InfluxDBClient(url="http://localhost:8086", token="your-token", org="your-org")
query = '''
from(bucket: "bigdata")
  |> range(start: -7d)
  |> filter(fn: (r) => r._measurement == "net_eth0_out")
  |> filter(fn: (r) => r._field == "value")
'''
tables = client.query_api().query(query=query)
data = []
for table in tables:
    for record in table.records:
        data.append({"time": record.get_time(), "value": record.get_value()})

This query fetches the time and value fields, which we’ll process further.

3. Cleaning and Preparing the Data

Now that we have our data, let’s load it into Pandas for exploration and preparation.

Loading the Data


Copied!import pandas as pd

df = pd.DataFrame(data)
df['time'] = pd.to_datetime(df['time'])
df.set_index('time', inplace=True)
print(df.head())
import pandas as pd

df = pd.DataFrame(data)
df['time'] = pd.to_datetime(df['time'])
df.set_index('time', inplace=True)
print(df.head())

Visualizing the Raw Data

We can use Matplotlib to inspect the time-series data and look for patterns or anomalies:


Copied!import matplotlib.pyplot as plt

df.plot()
plt.title("Raw Data Visualization")
plt.show()
import matplotlib.pyplot as plt

df.plot()
plt.title("Raw Data Visualization")
plt.show()

Handling Missing Values

Time-series data often has missing values due to network glitches or logging delays. Let’s clean this up:


Copied!df = df.dropna()  # Drop rows with missing values
df = df.dropna()  # Drop rows with missing values

Feature Engineering

Features are the backbone of machine learning models. For time-series data, we can calculate:

Differences between consecutive values to capture trends:


Copied!df['diff'] = df['value'].diff()
df['diff'] = df['value'].diff()

Rolling averages to smooth out short-term fluctuations:


Copied!df['rolling_avg'] = df['value'].rolling(window=5).mean()
df['rolling_avg'] = df['value'].rolling(window=5).mean()

4. Training Machine Learning Models

With our dataset cleaned and enriched with features, let’s split it for training and testing.

Splitting the Data


Copied!from sklearn.model_selection import train_test_split

X = df[['diff', 'rolling_avg']].dropna()  # Features
y = df['value'].shift(-1).dropna()  # Target: the next value
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
from sklearn.model_selection import train_test_split

X = df[['diff', 'rolling_avg']].dropna()  # Features
y = df['value'].shift(-1).dropna()  # Target: the next value
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

Training Multiple Models

We’ll test different models like Linear Regression, Random Forest, and Gradient Boosting to find the best fit.

Random Forest Example


Copied!from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_squared_error

model = RandomForestRegressor()
model.fit(X_train, y_train)
predictions = model.predict(X_test)

print("MSE:", mean_squared_error(y_test, predictions))
from sklearn.ensemble import RandomForestRegressor
from sklearn.metrics import mean_squared_error

model = RandomForestRegressor()
model.fit(X_train, y_train)
predictions = model.predict(X_test)

print("MSE:", mean_squared_error(y_test, predictions))

5. Identifying the Best Features

Some features contribute more to the model’s accuracy than others. Let’s identify the most important ones.

Feature Importances in Random Forest


Copied!importances = model.feature_importances_
for feature, importance in zip(X.columns, importances):
    print(f"{feature}: {importance}")
importances = model.feature_importances_
for feature, importance in zip(X.columns, importances):
    print(f"{feature}: {importance}")

Using SHAP for Feature Insights

For deeper insights, tools like SHAP (SHapley Additive exPlanations) can explain how each feature affects predictions.

6. Conclusion

Time-series data from InfluxDB is a treasure trove for machine learning, offering valuable insights when processed correctly. In this blog, we:

Retrieved raw data from InfluxDB.
Cleaned and prepared the data for analysis.
Engineered meaningful features.
Trained and tested multiple machine learning models.
Identified the most influential features.

This pipeline serves as a foundation for predictive analytics, anomaly detection, or any ML-based time-series application.

Next Steps

Explore other time-series features, such as seasonality and trends.
Integrate this workflow with Grafana dashboards for real-time predictions.
Scale the system for larger datasets.

You can find the complete code for this blog on our download pages. If you have questions or ideas, let us know in the comments!

Happy learning!