02 Neural Network Classification -tried using circle equation instead of ReLU but didn't work

[Dark07light]

Hello guys, I am a beginner in ml and have a small doubt
This is the time stamp of 25 hours course of PyTorch on youtube -https://youtu.be/Z_ikDlimN6A?t=38698
I tried using equation of circle instead of ReLU but don't know where i am wrong, even gemini couldn't solve it
As no. of epochs increased the accuracy became less and less
This is the code-

`#Neural Network classification
#First prepare data
from sklearn.datasets import make_circles
n_samples = 1000
X,y = make_circles(n_samples,noise=0.03,random_state=42)

#let's display our data
import pandas as pd
circles = pd.DataFrame({"X1":X[:,0],"X2":X[:,1],"Label":y[:]})
circles.head(10)

#let's draw our data
import matplotlib.pyplot as plt
plt.scatter(x=X[:,0],y=X[:,1],c=y,cmap=plt.cm.RdYlBu)

#let's turn our data into tensors
import torch
from torch import nn
X=torch.from_numpy(X).type(torch.float)
y = torch.from_numpy(y).type(torch.float)
X[:5]

#let's split our data using train_test_split function from sklearn.model_selection
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.2,random_state=42)
type(X_train)

#let's make device agnostic code
device = 'cuda' if torch.cuda.is_available() else 'cpu'
device

import torch
from torch import nn
class classification(nn.Module):
def init(self):
super().init()
self.h=nn.Parameter(torch.randn(1)) # Center x-coordinate
self.k=nn.Parameter(torch.randn(1)) # Center y-coordinate
self.rad=nn.Parameter(torch.randn(1)) # Base parameter for radius

def forward(self,x:torch.Tensor):
return (x[:,0]-self.h)**2 + (x[:,1]-self.k)2 -self.rad2

model_0=classification().to(device)
model_0

#let's enter data into our untrained model
untrained_pred=model_0(X_test.to(device))
print(untrained_pred.shape)
print(untrained_pred.squeeze().shape)
y_test.shape

#let's setup loss function and optimizer
loss_fn = nn.BCEWithLogitsLoss() #it has sigmoid built-in
optimizer = torch.optim.SGD(params=model_0.parameters(),lr=0.01)

#let's create an evaluation metric or function to evaluate the performance of our model
def eval(y_true,y_pred):
correct = torch.eq(y_true,y_pred).sum().item()
acc = (correct/len(y_pred))*100
return acc

#let's create our training and testing loop
torch.manual_seed(42)
epochs = 500
X_train,y_train = X_train.to(device),y_train.to(device)
X_test,y_test = X_test.to(device),y_test.to(device)

for epoch in range(epochs):
model_0.train()
y_logits = model_0(X_train).squeeze()
y_pred = torch.round(torch.sigmoid(y_logits))

loss = loss_fn(y_logits,y_train)
acc = eval(y_true=y_train,y_pred=y_pred)
optimizer.zero_grad()
loss.backward()
optimizer.step()

#testing part
model_0.eval()
with torch.inference_mode():
test_logits = model_0(X_test).squeeze()
test_pred = torch.round(torch.sigmoid(test_logits))
test_loss = loss_fn(test_logits,y_test)
test_acc = eval(y_test,test_pred)

#printing what's happening
if epoch%10==0:
print(f"Epoch:{epoch} |loss:{loss} , Accuracy:{acc}% |test_loss:{test_loss} , test_acc:{test_acc}%") `

Last cell execution image-

This is the notebook -
nn_classification_test.ipynb

[Yigtwxx]

The circle equation is fine. The sign is backwards.

In make_circles, label 1 is the inner ring and label 0 is the outer one. Quick check:

X, y = make_circles(1000, noise=0.03, random_state=42)
d = np.sqrt((X**2).sum(1))
d[y == 0].mean()   # 0.998
d[y == 1].mean()   # 0.801

Your forward returns d**2 - r**2, which gets larger the further a point sits from the center. BCEWithLogitsLoss reads a larger logit as "more likely to be class 1", so you're telling it the outer ring is class 1. No value of (h, k, rad) can satisfy that, so gradient descent keeps pushing the boundary further into the wrong place. That's why accuracy falls off instead of climbing.

I ran your code as posted to be sure. After 500 epochs I get 19% test accuracy. Anything well below 50% is the giveaway that a binary model is inverted rather than just failing to learn.

Flip the subtraction:

def forward(self, x):
    return self.rad**2 - ((x[:, 0] - self.h)**2 + (x[:, 1] - self.k)**2)

That took me to 79%. It stalls there for a second reason that's easier to miss. The two radii are roughly 0.8 and 1.0, so rad**2 - d**2 only ever spans about ±0.2. sigmoid(±0.2) is 0.45 to 0.55, so every prediction sits right on the fence and the loss has almost nothing to push against. Give it a learnable steepness:

class classification(nn.Module):
    def __init__(self):
        super().__init__()
        self.h = nn.Parameter(torch.randn(1))
        self.k = nn.Parameter(torch.randn(1))
        self.rad = nn.Parameter(torch.randn(1))
        self.scale = nn.Parameter(torch.tensor([1.0]))

    def forward(self, x):
        d2 = (x[:, 0] - self.h)**2 + (x[:, 1] - self.k)**2
        return self.scale * (self.rad**2 - d2)

SGD(lr=0.01) is still too slow to get anywhere in 500 epochs with this. Swapping to Adam(lr=0.1) gave me 100%.

What I measured, same seed, 500 epochs each:

version	test acc
as posted	19%
sign flipped	79%
sign flipped + scale + Adam(lr=0.1)	100%

One caveat on the whole approach: this works because you already know the data is two rings, so you hardcoded the shape of the boundary and left the model three numbers to fit. The ReLU stack in the course is doing something different. It has no idea the data is circular and has to build that boundary out of straight pieces. Yours will beat it on make_circles and lose badly on anything that isn't a circle.

[Dark07light]

Thanks brother, i understood it around 80-90% . Now, will do some R&D by myself and thanks for warning me. I knew about ReLU but just wanted to improve my concepts and make a forward function by myself that's why i tried doing it by a circle equation.
Again thank you