Skip to content

Commit c9a5507

Browse files
authored
[RepoSentinel] Weekly content update — 20260816 (#16)
* RepoSentinel: update docs/data_engineering/dbt.md * RepoSentinel: update docs/deep_learning/feature_engineering.md * RepoSentinel: update docs/deep_learning/transformers_&_attention.md * RepoSentinel: update docs/mlops_and_production_ml/model_monitoring_&_drift_detection.md
1 parent 1816c27 commit c9a5507

4 files changed

Lines changed: 101 additions & 22 deletions

File tree

docs/data_engineering/dbt.md

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
### Data Engineering
2+
3+
#### Overview
4+
5+
Data Engineering is the backbone of the data world. It focuses on designing, building, and maintaining the systems and infrastructure that allow for the large-scale collection, storage, and processing of data. A data engineer creates the "pipelines" that transport data from various sources to a central repository, like a data warehouse or data lake. They ensure this data is clean, reliable, and accessible so that data scientists and analysts can perform their analyses. Key skills include proficiency in programming (like Python), database technologies (SQL and NoSQL), and tools for data processing (like Spark and Airflow) and cloud computing.
6+
7+
---
8+
9+
#### Resources
10+
11+
A curated list of resources to help you get started and advance your data engineering skills.
12+
13+
##### Roadmaps & Overviews
14+
15+
* [The Data Engineering Roadmap for Beginners](https://www.dataquest.io/blog/the-data-engineer-roadmap-for-beginners) - A comprehensive guide from Dataquest on what to learn and in what order.
16+
* [Start Data Engineering Blog](https://www.startdataengineering.com) - An excellent blog with articles on modern data engineering practices and interview preparation.
17+
* [How I would learn Data Engineering in 2026 (Video)](https://www.youtube.com/watch?v=iiV_O4Uqj9Q) - A practical video guide on building a learning plan from the ground up.
18+
* [The 2025 Data Engineering Roadmap (Video)](https://www.youtube.com/watch?v=aSHg22oEGIs) - A video outlining the key technologies and concepts for the coming year.
19+
* [A Non-Beginner Data Engineering Roadmap](https://blog.dataengineerthings.org/a-non-beginner-data-engineering-roadmap-2025-edition-2b39d865dd0b) - For when you've mastered the basics and are ready for the next level.
20+
21+
##### Courses & Tutorials
22+
23+
* [Data Engineering Course for Beginners (YouTube)](https://www.youtube.com/watch?v=PHsC_t0j1dU) - A free, comprehensive video course covering databases, Docker, and analytical engineering.
24+
* [Fundamentals Of Data Engineering Masterclass (YouTube)](https://www.youtube.com/watch?v=hf2go3E2m8g) - A one-shot video covering the fundamental concepts of data engineering.
25+
* [Data Engineering Specialization by DeepLearning.AI](https://www.deeplearning.ai/specializations/data-engineering) - A structured specialization covering a wide range of data engineering topics.
26+
* [Data Engineering Academy](https://learndataengineering.com/p/academy) - In-depth courses on essential tools like Spark, Kafka, dbt, and Airflow.
27+
* [Learn Snowflake - Full 1-Hour Crash Course](https://www.youtube.com/watch?v=2t-ls6ekA8E) - A beginner-friendly introduction to the popular cloud data platform, Snowflake.
28+
29+
##### Interview Preparation
30+
31+
* [Top 30 Data Engineer Interview Questions (Video)](https://www.youtube.com/watch?v=N-MbyH7EhoQ) - A video walkthrough of common interview questions and how to answer them.
32+
* [Scenario-Based PWC Data Engineering Interview Question (Video)](https://www.youtube.com/watch?v=2Hpb8ADLOI0) - Practice your problem-solving skills with a real-world scenario-based question.
33+
34+
##### Deeper Dives & Research
35+
36+
* [Top 10 Data Engineering Research Papers to Read](https://dataheimer.substack.com/p/top-10-data-engineering-research) - A list of influential papers (like the one on Kafka) that shaped the field.
37+
* [One Week to Rebuild My Python Foundations as a Data Engineer](https://blog.devgenius.io/one-week-to-rebuild-my-python-foundations-as-a-data-engineer-30aa5cdc2369) - A practical guide to refreshing the core Python skills needed for data engineering.
38+
39+
---
40+
41+
#### Projects & Exercises
42+
43+
* **Build a Personal ETL Pipeline:**
44+
1. **Extract:** Choose a public API (e.g., OpenWeatherMap for weather data or a sports API for game stats). Write a Python script to fetch data from it.
45+
2. **Transform:** Clean the data. You might convert temperatures from Kelvin to Celsius, select only the fields you need, or flatten a nested JSON structure.
46+
3. **Load:** Load the cleaned data into a local SQLite or PostgreSQL database. Schedule your script to run once a day.
47+
48+
* **Containerize a Data Processing Script:**
49+
1. Write a simple Python script that reads a CSV file, performs a basic transformation (e.g., calculates a new column), and saves the result to a new CSV.
50+
2. Write a `Dockerfile` to create an image that runs this script.
51+
3. Build and run the Docker container to execute your script. This is a fundamental skill for creating reproducible data environments.
52+
53+
* **Analyze Data with SQL:**
54+
1. Download a public dataset from a source like Kaggle.
55+
2. Use a database client (like DBeaver) or a command-line interface to create a table and load the dataset into a PostgreSQL database.
56+
3. Write at least five different SQL queries to explore the data. Use `GROUP BY`, `JOIN`, and window functions to answer interesting questions about the dataset.
Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
### Machine Learning
2+
3+
#### Overview
4+
5+
Machine Learning (ML) is a field of Artificial Intelligence (AI) that enables systems to learn and improve from experience without being explicitly programmed. At its core, ML involves building models from sample data, known as "training data," to make predictions or decisions. These models can uncover patterns, classify information, and forecast future outcomes, making ML a powerful tool for solving complex problems in data analysis, natural language processing, computer vision, and more.
6+
7+
#### Resources
8+
9+
This curated list includes roadmaps, courses, and tutorials to help you build a strong foundation in Machine Learning.
10+
11+
* **Getting Started & Roadmaps**
12+
* [The Complete Machine Learning Roadmap](https://www.youtube.com/watch?v=7IgVGSaQPaw) (YouTube) - A 12-month, step-by-step guide to becoming a machine learning engineer.
13+
* [How to Learn Machine Learning in 2024](https://www.geeksforgeeks.org/machine-learning/how-to-start-learning-machine-learning) (GeeksforGeeks) - A structured guide outlining the timeline and topics for beginner to advanced levels.
14+
* [A Complete Guide to Start ML](https://github.com/louisfb01/start-machine-learning) (GitHub) - A repository with resources to start and improve in ML with no prior background.
15+
16+
* **Comprehensive Courses & Tutorials**
17+
* [Machine Learning Crash Course](https://developers.google.com/machine-learning/crash-course) (Google for Developers) - A fast-paced, practical introduction to ML concepts from Google.
18+
* [Machine Learning for Everybody – Full Course](https://www.youtube.com/watch?v=i_LwzRVP7bg) (YouTube) - An accessible, beginner-friendly course covering the basics of Machine Learning.
19+
* [Python Machine Learning Tutorial (Data Science)](https://www.youtube.com/watch?v=7eh4d6sabA0) (YouTube) - A hands-on tutorial for building your first AI project with Python using real-world data.
20+
* [Data Science: Building Machine Learning Models](https://pll.harvard.edu/course/data-science-building-machine-learning-models) (Harvard University) - Learn popular ML algorithms, PCA, and regularization by building a movie recommendation system.
21+
22+
* **Advanced Topics & Implementations**
23+
* [Annotated Deep Learning Paper Implementations](https://github.com/labmlai/annotated_deep_learning_paper_implementations) (GitHub) - A collection of PyTorch implementations of over 60 deep learning papers with side-by-side notes.
24+
* [NumPy Crash Course](https://github.com/sharma-piyush1/Numpy-Crash-Course) (GitHub) - A complete guide to NumPy, an essential library for numerical operations in ML.
25+
26+
#### Projects/Exercises
27+
28+
The best way to learn is by doing. Once you have a grasp of the basics, try applying your knowledge to a real dataset.
29+
30+
1. **Start with a Classic:** Find a well-known dataset (like the Titanic survival dataset or the Iris flower dataset) and try to build a simple classification model. Focus on the end-to-end process: data cleaning, feature engineering, model training, and evaluation.
31+
2. **Explore Real-World Applications:** Browse these GitHub repositories for inspiration and see how ML is applied to solve specific problems.
32+
* **Classification:** [Food Quality Prediction](https://github.com/shreeshjosyula/Food-Quality-Prediction) - Predict food quality using supervised learning techniques.
33+
* **Sentiment Analysis:** [AI-Powered Event Sentiment Analysis for Stock Price Movement Prediction](https://github.com/Palak0555/AI-Powered-Event-Sentiment-Analysis-for-Stock-Price-Movement-Prediction) - Use sentiment analysis from financial news to forecast stock price movements.
34+
* **Cybersecurity:** [Network Intrusion Detection Pipeline](https://github.com/danishmemon847/AOC-IDS-Pipeline) - Build a pipeline for detecting network intrusions using the UNSW-NB15 dataset.
Lines changed: 3 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
```markdown
12
### Deep Learning
23

34
#### Overview
@@ -12,26 +13,6 @@ A curated list of tutorials, courses, and guides to help you get started and bui
1213

1314
* [What Is Deep Learning? A Complete Beginner’s Guide](https://365datascience.com/trending/what-is-deep-learning/) - A clear, written guide explaining the core concepts of deep learning.
1415
* [The Ultimate Deep Learning Tutorial](https://www.simplilearn.com/tutorials/deep-learning-tutorial) - A comprehensive article covering the fundamentals, applications, and key terminology.
16+
* [Machine Learning vs Deep Learning (Video)](https://www.youtube.com/watch?v=q6kJ71tEYqM) - An IBM video giving a unique perspective on the differences between ML and DL.
1517
* [Deep Learning Crash Course for Beginners (Video)](https://www.youtube.com/watch?v=VyWAvY2CF9c) - A short, conceptual video from freeCodeCamp that introduces the fundamental ideas.
16-
* [MIT: Introduction to Deep Learning (Video)](https://www.youtube.com/watch?v=6FkRvTtUc-o) - The first lecture from MIT's Fall 2024 Deep Learning course, offering a solid academic introduction.
17-
18-
##### Comprehensive Courses
19-
20-
* [Practical Deep Learning - Fast.ai](https://course.fast.ai/) - A free, top-down course focused on applying deep learning to practical problems with modern libraries.
21-
* [PyTorch for Deep Learning & Machine Learning – Full Course (Video)](https://www.youtube.com/watch?v=V_xro1bcAuA) - An in-depth, hands-on course for beginners covering the PyTorch framework from the ground up.
22-
* [Deep Learning Full Course for Beginners (Video)](https://www.youtube.com/watch?v=G1P2IaBcXx8) - A 4-hour tutorial that walks through the essential concepts and practices of deep learning.
23-
24-
##### Frameworks & Implementation
25-
26-
* [Python Deep Learning Frameworks Guide](https://rcdsdocs.it.northwestern.edu/guides/python/python-deep-learning.html) - A guide from Northwestern University on choosing and using popular frameworks like TensorFlow and PyTorch.
27-
* [Annotated Deep Learning Paper Implementations](https://www.reddit.com/r/MachineLearning/comments/p95gee/p_annotated_deep_learning_paper_implementations/) - A collection of PyTorch implementations of famous papers with side-by-side notes and diagrams.
28-
* [DeepLearning-Papers-Implementation (GitHub)](https://github.com/greatsharma/DeepLearning-Papers-Implementation) - A repository showing how to implement influential deep learning papers using TensorFlow and Keras.
29-
* [Convolutional Neural Networks with Python, TensorFlow and Keras (Video)](https://www.youtube.com/watch?v=WvoLTXIjBYU) - A practical tutorial on building CNNs, a fundamental architecture for image-based tasks.
30-
31-
#### Projects/Exercises
32-
33-
1. **Image Classification with a CNN**: Build a Convolutional Neural Network (CNN) to classify images from a standard dataset like MNIST (handwritten digits) or CIFAR-10 (common objects). This is the "Hello, World!" of deep learning and a great way to understand model architecture.
34-
35-
2. **Text Generation with an RNN**: Train a Recurrent Neural Network (RNN) on a body of text, such as the works of Shakespeare or your own chat history. Use the trained model to generate new text that mimics the style of the original source.
36-
37-
3. **Implement a Foundational Paper**: Choose a classic, well-documented research paper like AlexNet or VGG. Read the paper and try to implement the model architecture from scratch using PyTorch or TensorFlow. Compare your results to established benchmarks.
18+
* [Backpropagation, Intuitively

docs/mlops_and_production_ml/model_monitoring_&_drift_detection.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,14 @@ MLOps (Machine Learning Operations) is the practice of applying DevOps principle
2828
* [Beginner End-to-end MLOps Project Showcase (Reddit)](https://www.reddit.com/r/mlops/comments/1h3kybz/beginner_endtoend_mlops_project_showcase/) — A Reddit thread showcasing a complete MLOps project, useful for seeing a practical, real-world example.
2929
* [Introducing MLOps to Facilitate the Development of Machine ... (IEEE Paper)](https://ieeexplore.ieee.org/iel8/6287639/10820123/11072436.pdf) — An academic paper discussing how MLOps enhances reproducibility and transparency in ML projects.
3030
* [The Matching Principle: A Geometric Theory of Loss Functions for Nuisance-Robust Representation Learning](https://arxiv.org/abs/2605.22800) — An academic paper on robustness, domain adaptation, photometric and occlusion invariance, compositional generalisation, temporal robustness, alignment safety, and classical anisotropic regularisation.
31+
* [Machine Learning in Production Specialization (DeepLearning.AI)](https://www.deeplearning.ai/courses/machine-learning-in-production) — A comprehensive specialization covering the entire lifecycle of production ML, from project scoping and data management to model deployment and monitoring.
32+
* [How I'd Become a Machine Learning Engineer in 2026 (full roadmap)](https://www.youtube.com/watch?v=PpiqSDMi4-4) — A video roadmap for aspiring Machine Learning Engineers, outlining the necessary skills and steps, including a focus on MLOps.
33+
* [MLOPS Github Action With CICD Pipeline One Shot Tutorial](https://www.youtube.com/watch?v=ciqWMIf7Pz0) — A tutorial on setting up a CI/CD pipeline for an MLOps project using GitHub Actions.
34+
* [Get Certified: Professional ML Engineer Week 5 | Productionizing ML & MLOps](https://www.youtube.com/watch?v=RX5rPs6seAg) — A study session focused on productionizing ML and MLOps, part of a series preparing for the Google Professional ML Engineer certification.
35+
* [MLOps Tutorial for Beginner [Updated 2024]](https://www.youtube.com/watch?v=e5vFMUMOB_0) — An introductory tutorial for beginners from igmGuru covering the overview and core concepts of MLOps.
36+
* [Awesome MLOps (GitHub)](https://github.com/kelvins/awesome-mlops) — A curated list of awesome MLOps tools, platforms, and resources.
37+
* [MLOps Explained - What It Is, Why You Need It and How It Works](https://www.youtube.com/watch?v=biqYkVf-a7Y) — A clear video explaining the what, why, and how of MLOps.
38+
* [What is MLOps? A Complete Beginner's Guide (KodeKloud)](https://kodekloud.com/blog/what-is-mlops) — An article that serves as a complete beginner's guide to MLOps, explaining key concepts and challenges like the notebook-to-production gap.
3139

3240
#### Projects & Exercises
3341
1. **Track an Experiment:** Take a simple Scikit-learn model you've built before. Integrate MLFlow to log your model's parameters, performance metrics, and the model file itself. Try running it with different hyperparameters and compare the results in the MLFlow UI.

0 commit comments

Comments
 (0)