Help with homework

[Meowmeow]

I have 40 columns of data with 10,000 rows. I want to reduce the dimensionality to 2 or 3 dimensions, such that the clustering is not by Euclidean distance but by the probability distribution of the cluster. How to do this? R or Python is ok. 

[Scada]

Ma roomate gadini adigi vadi backup number ivvamantava ?

[ring_master]

2 minutes ago, Meowmeow said:

I have 40 columns of data with 10,000 rows. I want to reduce the dimensionality to 2 or 3 dimensions, such that the clustering is not by Euclidean distance but by the probability distribution of the cluster. How to do this? R or Python is ok. 

PCA (Principal Component Analysis)

[Meowmeow]

Just now, ring_master said:

PCA (Principal Component Analysis)

That's Euclidean distance no?

[Meowmeow]

1 minute ago, Meowmeow said:

That's Euclidean distance no?

Nevermind, that's not even Euclidean distance, but based on covariance. I need to probability distribution of the cluster. 

[ring_master]

1 minute ago, Meowmeow said:

That's Euclidean distance no?

S. Are all the columns numerical or there are categorical columns in it?

[Meowmeow]

Just now, ring_master said:

S. Are all the columns numerical or there are categorical columns in it?

All are just numbers. 

[kathanayaka]

2 minutes ago, Meowmeow said:

That's Euclidean distance no?

Try PCA with n_components=2 but depends on R2 how much of data is explained

eucledian distance is used in KNN

[ring_master]

Just now, Meowmeow said:

Nevermind, that's not even Euclidean distance, but based on covariance. I need to probability distribution of the cluster. 

Clustering and dimensionality reduction are two different things.. Do you want to reduce 40 columns into 2 or 3 columns 

or you want to assign each row in the 10K rows to a cluster number that you specify?

[Meowmeow]

Just now, kathanayaka said:

Try PCA with n_components=2 but depends on R2 how much of data is explained

eucledian distance is used in KNN

PCA is only for the variance in the data. tSNE is for Euclidian distance between clusters. I need something for the probability distribution of the cluster. 

[kathanayaka]

your wording is not correct 

I want to reduce the dimensionality to 2 or 3 dimensions -- Does this mean you want to divide the dataset into clusters like 2 or 3?

[Meowmeow]

1 minute ago, ring_master said:

Clustering and dimensionality reduction are two different things.. Do you want to reduce 40 columns into 2 or 3 columns 

or you want to assign each row in the 10K rows to a cluster number that you specify?

I am doing clustering, the distance between clusters denoting the probability distribution of finding the cluster. 

[Sarvapindi]

Data scientists vaa meeru

[Meowmeow]

1 minute ago, kathanayaka said:

your wording is not correct 

I want to reduce the dimensionality to 2 or 3 dimensions -- Does this mean you want to divide the dataset into clusters like 2 or 3?

The number of clusters should depend on how different the data is (ideally 40 columns would become 7-8 clusters) Atleast that's my solution to the problem based on Euclidian distance. 

[ring_master]

Just now, Meowmeow said:

I am doing clustering, the distance between clusters denoting the probability distribution of finding the cluster. 

K- means vaadu but K-means assigns cluster to it's members based on euclidean distance between clusters