Unit 9 Summary
| Lesson | Topic | Key content covered |
|---|---|---|
| Lesson 9.1 | Data pipelines and preprocessing | 7-stage IoT analytics pipeline. ThingsBoard REST API retrieval to pandas DataFrame. Missing value imputation (forward-fill, interpolation, median). Outlier detection (Z-score, IQR, domain bounds). Resampling and normalisation. Feature engineering (temporal, rolling stats, lag, derived). |
| Lesson 9.2 | EDA and statistical analysis | Descriptive statistics for IoT data. Time-series dashboard visualisation with matplotlib. Diurnal pattern analysis. Correlation matrix. Z-score anomaly detection. Rolling average deviation (edge-deployable). Automated analytics report generation. |
| Lesson 9.3 | Machine learning for IoT | ML workflow for IoT (6 steps). Isolation Forest unsupervised anomaly detection. Random Forest classification for fault prediction. Evaluation metrics (accuracy, precision, recall, F1, ROC-AUC, RMSE, MAE). Temperature regression (30-min prediction). Flask REST API deployment. ESP32 linear regression inference without libraries. |
ML Algorithm Selection Guide
| Algorithm | Use case and strengths | IoT application |
|---|---|---|
| Isolation Forest | Anomaly detection (no labels needed). Learns normal patterns; flags outliers. | Sensor fault detection, quality control, cyber attack detection |
| Random Forest | Classification or regression with tabular features. Robust, interpretable feature importance. | Equipment fault prediction, occupancy classification, HVAC control |
| Linear Regression | Predict a continuous value. Embeddable in ESP32 firmware as constants. | Temperature forecasting, energy consumption prediction |
| K-Nearest Neighbours | Classification or regression. No training phase; classify by similarity to known examples. | Activity recognition, pattern matching |
| LSTM (Deep Learning) | Time-series forecasting with long-range dependencies. Requires more data. | Multi-step energy price prediction, complex environmental models |
| One-Class SVM | Anomaly detection with fewer samples than Isolation Forest. | Detecting novel events in low-data IoT deployments |
Key Terms Glossary
Anomaly detection
Identifying data points that deviate significantly from normal patterns.
Classification
Predicting which category (class) a data sample belongs to.
Confusion matrix
A table showing true positives, false positives, true negatives, and false negatives for a classifier.
Data leakage
When information from the test set accidentally influences training, producing falsely optimistic performance estimates.
EDA
Exploratory Data Analysis — initial analysis to summarise characteristics, find patterns, and check assumptions.
F1 Score
Harmonic mean of precision and recall; useful when class distribution is imbalanced.
Feature engineering
Transforming raw data into more informative input features for ML models.
Imputation
Filling in missing data values using statistical methods.
Isolation Forest
An unsupervised ML algorithm for anomaly detection based on isolating outliers in random decision trees.
MAE
Mean Absolute Error — average absolute difference between predicted and actual values.
RMSE
Root Mean Square Error — square root of mean squared prediction errors; penalises large errors.
Random Forest
An ensemble ML algorithm that combines many decision trees for robust prediction.
Recall
Fraction of actual positives correctly identified (also: Sensitivity, True Positive Rate).
Regression
Predicting a continuous numeric value (as opposed to a class label).
Resampling
Aggregating irregularly-spaced time-series data into regular intervals (e.g. 5-minute bins).
ROC-AUC
Area under the Receiver Operating Characteristic curve; a threshold-independent classifier performance metric.
Rolling statistics
Statistics computed over a sliding time window (mean, std, max) to capture local trends.
TFLite
TensorFlow Lite — a lightweight ML runtime for embedded and mobile devices.
Z-score
Number of standard deviations a value is from the mean; used for outlier detection and anomaly flagging.
Further Reading & Resources
- Google Colab — colab.research.google.com — free cloud Jupyter notebooks with GPU. Run all Unit 9 code without local installation.
- pandas documentation — pandas.pydata.org/docs — comprehensive guide to time-series data manipulation.
- scikit-learn documentation — scikit-learn.org/stable — full algorithm reference with examples.
- ThingsBoard REST API — thingsboard.io/docs/reference/rest-api — telemetry retrieval and data export.
- TensorFlow Lite for Microcontrollers — tensorflow.org — deploying neural networks on ESP32 and Arduino.
- Towards Data Science — towardsdatascience.com — practical ML tutorials including time-series and anomaly detection.
You may also like
Nyereka Tech and Tumba College Explore Opportunities for Collaboration
September 25, 2026
The Launch of our IoT Innovation Hub
May 7, 2026
Partnership with MakerSpace Kigali
May 7, 2026
