ClusterEval

ClusterEval provides external clustering evaluation metrics, including homogeneity, parsimony, Q-measure, purity, and inverse purity scores.

API Reference

clustereval.homogeneity_score(labels_true, labels_pred)

Calculate the homogeneity score for a clustering.

Homogeneity asks whether class members \(C\) are assigned to the same cluster \(K\). Homogeneity is defined as

\[h(C, K) = 1 - \frac{H(C|K)}{H(C)}\]

Homogeneity is maximal (\(h = 1\)) when all members of a cluster share the same class label, and minimal (\(h = 0\)) when cluster assignments provide no information about the classes.

Parameters:

  • labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.

  • labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

homogeneity – The homogeneity score for the clustering.

Return type:

float

clustereval.parsimony_score(labels_true, labels_pred)

Calculate the parsimony score for a clustering.

Parsimony asks whether the \(N\) objects are clustered as simply as possible given the class \(C\). Formally, the parsimony score is defined based on the conditional entropy of clusters \(K\) given classes \(C\):

\[p(C, K) = 1 - \frac{H(K|C)}{\log(N) - H(C)}.\]

Parsimony is maximal (\(p = 1\)) when all class members share an identical cluster label, and minimal (\(p = 0\)) when the clustering fully fragments each class.

Parameters:

  • labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.

  • labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

parsimony – The parsimony score for the clustering.

Return type:

float

clustereval.q_measure_score(labels_true, labels_pred, *, beta=1.0)

Calculate the Q-measure for a clustering.

The Q-measure is defined as the weighted harmonic mean of the normalized homogeneity and parsimony scores:

\[Q_\beta(C, K) = \frac{(1 + \beta) h p}{\beta h + p}\]

As a harmonic mean, the Q-measure is sensitive to low values in either homogeneity or parsimony. The weight parameter beta controls the relative importance of homogeneity vs parsimony. Larger beta puts greater weight on parsimony. The Q-measure is maximal (\(Q_\beta = 1\)) for clusterings that perfectly recover the class partition, and minimal (\(Q_\beta = 0\)) when either the clustering is completely uninformative about the classes or when the clustering fully fragments each class.

Parameters:

  • labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.

  • labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate. beta : float, default=1.0

  • beta (float, default=1.0) – Weight parameter controlling importance of homogeneity vs parsimony. Larger beta puts greater weight on parsimony. Must be >= 0.

Returns:

q_measure – The Q-measure for the clustering.

Return type:

float

See also

homogeneity_score

Homogeneity of clustering relative to class partition

parsimony_score

Parsimony of clustering relative to class partition

clustereval.purity_score(labels_true, labels_pred)

Calculate the purity score for a clustering.

This score is the set-matching analogue of class-conditional clustering entropy.

When \(C\) is the class partition and \(K\) is the clustering, the purity score is defined as:

\[\mathrm{purity}(C, K) = \frac{1}{N} \sum_k \max_c \left| C_c \cap K_k \right|\]
Parameters:

  • labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.

  • labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

purity – The purity score for the clustering.

Return type:

float

See also

normalized_purity_score

Purity normalized to [0, 1]

inverse_purity_score

Corresponding score equivalent to parsimony

clustereval.inverse_purity_score(labels_true, labels_pred)

Calculate the inverse purity score for a clustering.

When \(C\) is the class partition and \(K\) is the clustering, the inverse purity score is defined as:

\[\mathrm{inv\text{-}purity}(C, K) = \frac{1}{N} \sum_c \max_k \left| C_c \cap K_k \right|\]
Parameters:

  • labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.

  • labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

inverse_purity – The inverse purity score for the clustering.

Return type:

float

clustereval.normalized_purity_score(labels_true, labels_pred)

Calculate the normalized purity score for a clustering.

This is a min-max normalized version of the purity score, which is defined as:

\[\text{norm. purity} = \frac{\mathrm{purity}(C, K) - \max_c P(c)}{1-\max_c P(c)}\]
Parameters:

  • labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.

  • labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

normalized_purity – The normalized purity score for the clustering.

Return type:

float

clustereval.normalized_inverse_purity_score(labels_true, labels_pred)

Calculate the normalized inverse purity score for a clustering.

This is a min-max normalized version of the inversepurity score, which is defined as:

\[\text{norm. inv. purity} = \frac{\mathrm{inv.\text{-}purity}(C, K) - |\mathcal{C}|/N}{1-|\mathcal{C}|/N}\]
Parameters:

  • labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.

  • labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

normalized_inverse_purity – The normalized inverse purity score for the clustering.

Return type:

float

clustereval.pair_specificity_score(labels_true, labels_pred)

Calculate the pair specificity score for a clustering.

Pair-based evaluation metrics are based on a binary classification of pairs of samples. This score is the specificity of this binary classifier, defined as the true negative rate.

This score is the pair-based analogue of the homogeneity score.

Parameters:

  • labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.

  • labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

pair_specificity – The pair specificity score for the clustering.

Return type:

float

clustereval.pair_sensitivity_score(labels_true, labels_pred)

Calculate the pair sensitivity score for a clustering.

Pair-based evaluation metrics are based on a binary classification of pairs of samples. This score is the sensitivity of this binary classifier, defined as the true positive rate.

This score is the pair-based analogue of the parsimony score.

Parameters:

  • labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.

  • labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

pair_sensitivity – The pair sensitivity score for the clustering.

Return type:

float

clustereval.completeness_score(labels_true, labels_pred)

Calculate the completeness score for a clustering.

Completeness like parsimony penalizes clustering that fragment classes. However, completeness uses a normalization that depends on the entropy of the specific clustering under evaluation \(H(K)\) rather than its maximum over all possible clusterings \(\log(N) - H(C)\). Formally, completeness is defined as

\[c(C, K) = 1 - \frac{H(K|C)}{H(K)}\]
Parameters:

  • labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.

  • labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

completeness – The completeness score for the clustering.

Return type:

float

clustereval.conditional_entropies(labels_true, labels_pred)

Calculate conditional entropies between two partitions.

Parameters:

  • labels_true (array-like of shape (n_samples,)) – Ground truth class labels C to be used as a reference.

  • labels_pred (array-like of shape (n_samples,)) – Cluster labels K to evaluate.

Return type:

entropy_joint, entropy_CK, entropy_KC, entropy_C, entropy_K