Skip to content

Commit ec4c73b

Browse files
committed
Update: Data Mining(kmeans), About me(Senior), Friends(Senuchy)
1 parent 7b3b8c0 commit ec4c73b

7 files changed

Lines changed: 227 additions & 15 deletions

File tree

‎source/_posts/Hexo-Theme-Build.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
---
22
title: Hexo主题搭建日志 <LETS GO HEXO>
33
date: 2024-08-04 23:00:00
4-
categories: "Biuld Notes"
4+
categories: "Build Notes"
55
tags:
66
- Note
77
- Project

‎source/_posts/Intro-to-Data-Mining.md‎

Lines changed: 222 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,9 @@ In-class assessment 20% + One assignment 40% + A final examination 40%
1717
- W12-16 Data Streams, Social Networks, Web Data
1818
- Group Assignment due 6 June 2026
1919

20-
# 1 Intro
20+
# W1-4
21+
22+
## 1 Intro
2123

2224
> Data mining is a process of extracting previously unknown,
2325
valid, and actionable information from large databases and
@@ -35,10 +37,25 @@ Given an n×d training data matrix D (database D), and a class label value in {1
3537

3638
**1. Classification and regression methods are usually described as supervised learning methods. Explain why. And describe the difference between the two.**
3739

40+
> Because they learn from labeled training data. The model learns the relationship between inputs and outputs.
41+
> The diff is that classification **predicts discrete class labels**, while regression **predicts continuos numeric values**.
42+
43+
分类是离散类别,回归是连续数值。
44+
3845
**2. What do you need in order to produce a classification model?**
46+
47+
> A labeled training dataset, input features (attributes), class lables, a classification algorithm, a training process to learn the model.
48+
3949
**3. How do you know that you have produced a good model?**
50+
51+
> It gains great score when using test data.
52+
53+
如 Accuracy,Precision,Recall,F1-score.
54+
4055
**4. Why is a classification model also called a generalized model?**
4156

57+
> Because it learns general patterns from training data and applies them to unseen data.
58+
> The goal is not to memorize the training data but to generalize to new instances.
4259
4360
### Def 2 Data Clustering (数据聚类)
4461

@@ -49,9 +66,20 @@ Given a data matrix D, partition its rows (records) into sets/clusters C1 . . .
4966
### Dis 2
5067

5168
**1. Explain how a clustering task differs from a classification task.**
69+
70+
> Classification is a supervised learning task while clustering task is an unsupervised learning task.
71+
5272
**2. What is the core operation/measurement required in a clustering task?**
73+
74+
> Similarity or distance calculation between data points.
75+
76+
聚类的核心只有一个问题:两个数据点有多相似?
77+
因此聚类算法最重要的是距离或相似度计算。
78+
5379
**3. How do you know that you have produced a good clustering outcome?**
5480

81+
> High similarity with clusters and low similarity between clusters.
82+
5583
### Def 3 Outlier Detection (异常检测)
5684

5785
Given a data matrix D, determine the few rows of the data matrix that are very different from the majority of rows in the matrix.
@@ -61,9 +89,20 @@ Given a data matrix D, determine the few rows of the data matrix that are very d
6189
### Dis 3
6290

6391
**1. Explain how an anomaly detection task differs from a classification task.**
92+
93+
> Anomaly detection aims to identify rare or unusual instances that significantly deviate from the majority of the data. It is often unsupervised or semi-supervised because anomalies are usually unknown beforehand.
94+
> Classification is a supervised learning task.
95+
6496
**2. Explain how it differs from a clustering task.**
97+
98+
> Clustering groups data points into clusters based on similarity, while anomaly detection focuses on identifying points.
99+
100+
它们都是无监督学习。聚类找到数据中的多个群体结构,异常检测关注偏离群体的点。
101+
65102
**3. How do you know that you have produced a good anomaly detection outcome?**
66103

104+
> A good anomaly detection result correctly identifies rare abnormal instances while minimizing false positives. Evaluation may use labeled anomalies, precision/recall, or domain knowledge.
105+
67106
### Def 4 Frequent Pattern Mining (频繁模式挖掘)
68107

69108
**Definition 4 (Frequent Pattern Mining)** Given a binary n×d data matrix D, determine all subsets of columns such that all the values in these columns take on the value of 1 for at least a fraction s of the rows in the matrix. The relative frequency of a pattern is referred to as its support. The fraction s is referred to as the minimum support.
@@ -79,9 +118,17 @@ Given a data matrix D, determine the few rows of the data matrix that are very d
79118
### Dis 4
80119

81120
**1. How does association pattern mining differ from other data mining tasks?**
121+
122+
> Association pattern mining is an unsupervised task that discovers relationships (co-occurrence patterns) among items in data.
123+
82124
**2. How does association pattern mining differ from rule-based classification?**
125+
126+
> Association pattern mining discovers rules based on co-occurrence without a target variable, while rule-based classification learns rules specifically to predict a predefined class label.
127+
83128
**3. What is a search space?**
84129

130+
> The set of all possible candidate solutions.
131+
85132
### Ass i.i.d
86133

87134
Every data point in a dataset is independent and identically distributed
@@ -106,10 +153,18 @@ Is there any issue in using a classification model trained from P to classify a
106153
- Transductive (a special semi-supervised) learning
107154

108155
### Ques
109-
1. Does a semi-supervised learning refer to supervised or unsupervised learning or something completely different?4
110-
2. How does semi-supervised learning impact on model evaluation?
111156

112-
# 2 Data
157+
**1. Does a semi-supervised learning refer to supervised or unsupervised learning or something completely different?**
158+
159+
> Hybrid. It uses a small amount of labeled data (supervised) together with a large amount of unlabeled data (unsupervised) to improve learning.
160+
161+
**2. How does semi-supervised learning impact on model evaluation?**
162+
163+
> Evaluation in semi-supervised learning must focus on labeled data, since only labeled points provide ground truth. The presence of unlabeled data affects training but does not directly provide evaluation metrics, which makes careful validation crucial.
164+
165+
评估仍然依赖 labeled 数据,但训练过程可以利用大量 unlabeled 数据提高性能,需要谨慎验证以避免偏差。
166+
167+
## 2 Data
113168

114169
### (Def 5) Types of Attributes
115170

@@ -120,14 +175,170 @@ Is there any issue in using a classification model trained from P to classify a
120175

121176
### Dis 5
122177

123-
1. What is the attribute type of each of the following kinds of attributes (a)Age,(b) Salary, (c) Zip Code, (d) Province of Residence, (e) Height, (f) Weight.
124-
2. An analyst sets up a sensor network in order to measure the temperature of different locations over a period. What is the data type of the data collected?
125-
3. An analyst processes Web logs in order to create records with the ordering information for Web page accesses from different users. What is the type of this data?
126-
4. Describe the difference between interval and ratio attribute type.
127-
5. Suppose that you are employed as a data mining consultant for an Internet search engine company. Describe how data mining can help the company by giving specific examples of how techniques, such as clustering, classification, association rule mining, and anomaly detection can be applied.
128-
6. Distinguish between noise and outliers. Consider the following questions.
178+
**1. What is the attribute type of each of the following kinds of attributes (a)Age,(b) Salary, (c) Zip Code, (d) Province of Residence, (e) Height, (f) Weight.**
179+
180+
> | Attribute | Type | 说明 |
181+
> | --------------------- | ------- | ------------------------ |
182+
> | Age | Ratio | 有零点,可比较大小和比率 |
183+
> | Salary | Ratio | 有零点,可比较大小和比率 |
184+
> | Zip Code | Nominal | 只是标识,没有大小顺序 |
185+
> | Province of Residence | Nominal | 类别标签,没有顺序 |
186+
> | Height | Ratio | 有零点,可比较大小和比率 |
187+
> | Weight | Ratio | 有零点,可比较大小和比率 |
188+
189+
**2. An analyst sets up a sensor network in order to measure the temperature of different locations over a period. What is the data type of the data collected?**
190+
191+
> Interval + Ratio.
192+
193+
**3. An analyst processes Web logs in order to create records with the ordering information for Web page accesses from different users. What is the type of this data?**
194+
195+
> Complex data object. Nominal
196+
197+
**4. Describe the difference between interval and ratio attribute type.**
198+
199+
> | Aspect | Interval | Ratio |
200+
> | ------------------- | -------------------------- | ---------------------------------- |
201+
> | Zero point | Arbitrary | Absolute zero |
202+
> | Can compute ratios? | No | Yes |
203+
> | Example | Celsius temperature, dates | Kelvin temperature, weight, height |
204+
205+
**5. Suppose that you are employed as a data mining consultant for an Internet search engine company. Describe how data mining can help the company by giving specific examples of how techniques, such as clustering, classification, association rule mining, and anomaly detection can be applied.**
206+
207+
> Clustering: Group users by search behavior to personalize recommendations.
208+
> Classification: Predict whether a search query will lead to a click or conversion.
209+
> Association rule mining: Find frequent co-occurrences of search queries (e.g., “laptop → laptop bag”).
210+
> Anomaly detection: Detect unusual search behavior indicating bots or fraudulent activity.
211+
212+
**6. Distinguish between noise and outliers. Consider the following questions.
129213
(a) Is noise ever interesting or desirable? Outliers?
130214
(b) Can noise objects be outliers?
131215
(c) Are noise objects always outliers?
132216
(d) Are outliers always noise objects?
133-
(e) Can noise make a typical value into an unusual one, or vice versa?
217+
(e) Can noise make a typical value into an unusual one, or vice versa?**
218+
219+
> (a) noise: not intersting. Outliers: often
220+
> (b) yes
221+
> (c) No, noise may just add small variations without being extreme
222+
> (d) No, outliers may be valid, rare patterns
223+
> (e) Yes, noise can distort data to appear as unusual (false outlier) or mask a real outlier
224+
225+
## 3 Distances
226+
227+
### Def 6 K-Means Clustering(K-均值聚类)
228+
229+
Given a data matrix $D$ with $n$ rows (records) and $d$ columns (features), partition the rows into $k$ clusters $C\_1, …, C\_k$ such that the sum of squared distances between each point and the centroid of its assigned cluster is minimized. Specifically:
230+
1. For each cluster $C\_j$ , compute the centroid:
231+
$$ x_i \in C_j \quad \text{if} \quad j = \arg \min_{l \in [1,k]} \|x_i - \mu_l\|^2 $$
232+
2. Assign each point $x\_i \in D$ to the nearest centroid:
233+
$$ \mu_j = \frac{1}{|C_j|} \sum_{x \in C_j} x $$
234+
235+
> 给定 n×d 数据矩阵 D,把数据分成 k 个簇,每个点分配到最近的簇中心,使得簇内平方距离总和最小。
236+
Distance function 使用位置:在分配点到簇中心时,用距离(通常欧氏距离)计算“最近簇”。
237+
238+
### Def 7 DBSCAN: Density-based Clustering(基于密度的聚类)
239+
240+
DBSCAN Given a data matrix D, cluster the data based on density rather than predefined centroids.
241+
The steps are:
242+
1. Estimate the density of each point using its $ε$-neighborhood:
243+
$$ N_\varepsilon(x) = \{ y \in D \mid d(x, y) \le \varepsilon \} $$
244+
2. Identify all high-density points where $|N\_\varepsilon(x)| \geq MinPts$
245+
3. Link neighboring high-density points to form clusters; points not belonging to any cluster are considered noise.
246+
247+
> 根据每个点的邻域密度将数据分簇,不需要指定簇数。
248+
> Distance function 使用位置:计算每个点的 $ε$-邻域时,用距离函数判断哪些点在半径内。
249+
250+
### Def 8 Data-independent distance/similarity measures(独立于数据分布的距离/相似度度量)
251+
252+
Lp Norm / Minkowski Distance:
253+
$$ d_{p}(x, y) = \left( \sum_{i=1}^{d} |x_i - y_i|^p \right)^{1/p} $$
254+
255+
Cosine Similarity:
256+
$$ \text{cosine}(X_1, X_2) = \frac{\langle X_1, X_2 \rangle}{\|X_1\| \|X_2\|} $$
257+
258+
Gaussian Kernel:
259+
$$ K(x, y) = \exp(-\gamma \|x-y\|^2) $$
260+
261+
### Dis 6
262+
263+
![k-means vs DBSCAN.png](\../imgs/Data-Mining/k-means vs DBSCAN.png)
264+
265+
> K-means assumes convex, spherical clusters and minimizes distance to centroids.
266+
> So:
267+
> - For ring-shaped data, it cuts across rings because rings are non-convex → wrong grouping.
268+
> - For two moons / curved shapes, it splits by straight boundaries → fails to capture the true structure.
269+
270+
核心原因:k-means 的“模型假设”不适合这些数据形状。
271+
k-means 的本质是:
272+
- 用质心(centroid)表示每个簇
273+
- 最小化每个点到其质心的 欧氏距离平方
274+
- 等价于假设:每个簇是“凸的、近似球形的”
275+
276+
> DBSCAN groups points by density connectivity instead of distance to centroids.
277+
> So:
278+
> - It can detect arbitrary-shaped clusters (rings, curves)
279+
> - Points connected through dense regions form one cluster → correct grouping
280+
281+
核心原因:DBSCAN 是“基于密度”的,而不是基于中心点。
282+
DBSCAN 的关键思想:
283+
定义一个半径 $ε$(邻域)
284+
如果某点附近有足够多点($minPts$),它就是“核心点”
285+
相互“密度可达”的点会被连成一个簇
286+
287+
288+
### Def 9 Isolation Kernel (隔离核)
289+
290+
Isolation Kernel of any two points $x, y \in \mathbb{R}^d$ is defined to be the expectation taken over the probability distribution $P\_{H\_\psi(D)}$ on all partitionings $H \in H\_\psi(D)$ such that both $x$ and $y$ fall into the same isolating partition $\theta[z] \in H$:
291+
$$
292+
K_\psi(x, y \mid D)
293+
= \mathbb{E}_{H \sim P_{H_\psi(D)}}
294+
\left[
295+
\mathbf{1}\big( x, y \in \theta[z] \mid \theta[z] \in H \big)
296+
\right],
297+
$$
298+
where $\mathbf{1}(\cdot)$ is an indicator function.
299+
300+
> 如果两个点经常被分在同一个区域 → 它们相似
301+
> 如果很容易被分开 → 它们不相似
302+
> 隔离核描述了两个点有多相似。
303+
304+
Definition 9.1 Isolation Kernel $K\_\psi$ has a feature map $\Phi : x \mapsto \{0,1\}^{t \times \psi}$.
305+
$K_\psi$ has no closed-form expression and is expressed in terms of $\Phi$ as follows:
306+
$$
307+
K_\psi(x, y \mid D)
308+
\simeq \frac{1}{t} \langle \Phi(x \mid D), \Phi(y \mid D) \rangle.
309+
$$
310+
311+
Isolation Kernel is a data dependent kernel without closed form expression.
312+
313+
> 表示每个点所属的区域,并比较向量。
314+
> 看它们的向量有多相似(点积),用“向量相似度”去近似那个概率。
315+
316+
![distance function affects result.png](\../imgs/Data-Mining/distance%20function%20affects%20result.png)
317+
318+
距离函数(distance)和相似度度量方式,会直接影响聚类效果;尤其是“数据依赖(data-dependent)”的度量在处理不同密度的数据时更好。
319+
320+
### Dis 7
321+
322+
![kernel trick.png](\../imgs/Data-Mining/kernel%20trick.png)
323+
324+
**既然 kernel trick 能让线性模型变非线性,为什么在 k-means 上效果并不好?**
325+
326+
> **kernel ≠ data-dependent**
327+
> Kernel trick 解决的是“线性 vs 非线性”,但聚类的难点是“结构 vs 密度”。
328+
329+
**为什么说从“非线性”?**
330+
331+
> 模型始终是线性的,但“线性是相对于谁而言”发生了改变。是对 $\phi(x)$ 线性,而 $\phi(x)$ 是非线性映射。
332+
> Kernel 并没有让“模型变非线性”,而是把数据映射到一个更高维空间,在那里用线性模型解决问题,从而在原空间表现为非线性。
333+
> Kernel 相当于把数据“弯曲/抬升”到更高维空间,让原本弯曲的分界变成平的,但它只改变“几何形状”,不改变“簇的统计结构”。
334+
335+
- Kernel k-means 的本质没变:
336+
- 假设每个簇是“球状”的
337+
- 用“中心点”表示
338+
- Kernel 只是将其换到了高维空间中,可以看作是曲线距离
339+
- 即使在高维空间,簇仍然被假设为:凸的、均匀密度、中心可表示
340+
- 同时,Kernel 是全局的,σ 是全局参数,不能同时适应稠密区和稀疏区
341+
- 而 density-based 方法是“局部”的,比如 DBSCAN 思想:
342+
- 每个点有自己的邻域
343+
- 密度是局部计算的
344+
- 能处理:变密度、非均匀结构

‎source/about/index.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,8 +11,8 @@ date: 2024-03-04 15:40:31
1111

1212
ID: Shotsuke
1313

14-
就读南京大学,三年级本科生。
15-
A Junior in Nanjing University
14+
就读南京大学,四年级本科生,直博
15+
A senior undergraduate admitted to a PhD program in Nanjing University
1616

1717
计算机科学与技术系,信息计算与科学专业
1818
Computer Science and Technology, Infomation Computing and Science

‎source/friends/index.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,4 +7,5 @@ date: 2024-03-29 9:13:26
77
- [Eric Li](https://www.ericli.vip)
88
- [detect](https://detect.wiki/)
99
- [IAmDanny](https://blog.iamdanny.online)
10-
- [NamelessYing](https://www.namelessying.xin/)
10+
- [NamelessYing](https://www.namelessying.xin/)
11+
- [Senuchy](https://senuchy.github.io/)
432 KB
Loading
292 KB
Loading
271 KB
Loading

0 commit comments

Comments
 (0)