ClusterEval: External Clustering Validation by the Homogeneity-Parsimony Trade-Off

ClusterEval is a lightweight package for external clustering validation.

ClusterEval provides reference implementations for the homogeneity-parsimony scores proposed in arXiv 2607.20799. Evaluation of clustering quality on these two objectives provides a unified framework for assessing clustering agreement with ground truth class labels.

The package also provides set-matching variants of the homogeneity and parsimony scores, called normalized purity, and normalized inverse purity. These scores fix the definition of purity and inverse purity so that the full range of [0, 1] is attainable. The pair-based equivalents of these scores are pair specificity and pair sensitivity, exactly the familiar binary classifier metrics used by the receiver operating characteristic (ROC) curve.

Validation Approach	Criterion 1	Criterion 2	Entropy measure
Information-theoretic	Homogeneity Score	Parsimony Score	Shannon Entropy
Set-matching	Normalized Purity Score	Normalized Inverse Purity Score	min-Entropy
Pair-counting	Pair Specificity	Pair Sensitivity	Collision Entropy

Illustration of validation approaches

All scores are implemented in a way that is compatible with evaluation metrics defined in Scikit-learn’s sklearn.metrics to allow easy replacement within existing pipelines.

Installation

ClusterEval can be installed via pip:

pip install clustereval

The package depends on numpy and scikit-learn.

Documentation and examples

API documentation is provided through docstrings.

Jupyter example notebooks can be found in the examples folder:

You can create a local copy of the API documentation in the docs folder by running:

make html

All metrics require non-empty label arrays of equal length and raise ValueError otherwise.

Support and contributing

For bug reports and enhancement requests use the Github issue tool, or (even better!) open a pull request with relevant changes.

When preparing a pull request, please run the testsuite using pytest to ensure none of the existing functionality breaks.

API Reference

clustereval.homogeneity_score(labels_true, labels_pred)

Calculate the homogeneity score for a clustering.

Homogeneity asks whether class members $C$ are assigned to the same cluster $K$. Homogeneity is defined as

\[h(C, K) = 1 - \frac{H(C|K)}{H(C)}\]

Homogeneity is maximal ($h = 1$) when all members of a cluster share the same class label, and minimal ($h = 0$) when cluster assignments provide no information about the classes.

Parameters:

labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.
labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

homogeneity – The homogeneity score for the clustering.

Return type:

float

Raises:

ValueError – If the two label arrays have different lengths or are empty.

See also

parsimony_score: Parsimony of clustering relative to class partition

clustereval.parsimony_score(labels_true, labels_pred)

Calculate the parsimony score for a clustering.

Parsimony asks whether the $N$ objects are clustered as simply as possible given the class $C$. Formally, the parsimony score is defined based on the conditional entropy of clusters $K$ given classes $C$:

\[p(C, K) = 1 - \frac{H(K|C)}{\log(N) - H(C)}.\]

Parsimony is maximal ($p = 1$) when all class members share an identical cluster label, and minimal ($p = 0$) when the clustering fully fragments each class.

Parameters:

labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.
labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

parsimony – The parsimony score for the clustering.

Return type:

float

Raises:

ValueError – If the two label arrays have different lengths or are empty.

See also

homogeneity_score: Homogeneity of clustering relative to class partition
completeness_score: Completeness is a closely related score, which depends on the entropy of the clustering under evaluation rather than its maximum over all possible clusterings.

clustereval.q_measure_score(labels_true, labels_pred, *, beta=1.0)

Calculate the Q-measure for a clustering.

The Q-measure is defined as the weighted harmonic mean of the normalized homogeneity and parsimony scores:

\[Q_\beta(C, K) = \frac{(1 + \beta) h p}{\beta h + p}\]

As a harmonic mean, the Q-measure is sensitive to low values in either homogeneity or parsimony. The weight parameter beta controls the relative importance of homogeneity vs parsimony. Larger beta puts greater weight on parsimony. The Q-measure is maximal ($Q_\beta = 1$) for clusterings that perfectly recover the class partition, and minimal ($Q_\beta = 0$) when either the clustering is completely uninformative about the classes or when the clustering fully fragments each class.

Parameters:

labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.
labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate. beta : float, default=1.0
beta (float, default=1.0) – Weight parameter controlling importance of homogeneity vs parsimony. Larger beta puts greater weight on parsimony. Must be >= 0.

Returns:

q_measure – The Q-measure for the clustering.

Return type:

float

Raises:

ValueError – If the two label arrays have different lengths or are empty.

See also

homogeneity_score: Homogeneity of clustering relative to class partition
parsimony_score: Parsimony of clustering relative to class partition

clustereval.purity_score(labels_true, labels_pred)

Calculate the purity score for a clustering.

This score is the set-matching analogue of class-conditional clustering: entropy.

When $C$ is the class partition and $K$ is the clustering, the purity score is defined as:

\[\mathrm{purity}(C, K) = \frac{1}{N} \sum_k \max_c \left| C_c \cap K_k \right|\]

Parameters:

labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.
labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

purity – The purity score for the clustering.

Return type:

float

Raises:

ValueError – If the two label arrays have different lengths or are empty.

See also

normalized_purity_score: Purity normalized to [0, 1]

clustereval.inverse_purity_score(labels_true, labels_pred)

Calculate the inverse purity score for a clustering.

When $C$ is the class partition and $K$ is the clustering, the inverse purity score is defined as:

\[\mathrm{inv\text{-}purity}(C, K) = \frac{1}{N} \sum_c \max_k \left| C_c \cap K_k \right|\]

Parameters:

labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.
labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

inverse_purity – The inverse purity score for the clustering.

Return type:

float

Raises:

ValueError – If the two label arrays have different lengths or are empty.

See also

normalized_inverse_purity_score: Inverse purity normalized to [0, 1].

clustereval.normalized_purity_score(labels_true, labels_pred)

Calculate the normalized purity score for a clustering.

This is a min-max normalized version of the purity score, which is defined as:

\[\text{norm. purity} = \frac{\mathrm{purity}(C, K) - \max_c P(c)}{1-\max_c P(c)}\]

Parameters:

labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.
labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

normalized_purity – The normalized purity score for the clustering.

Return type:

float

Raises:

ValueError – If the two label arrays have different lengths or are empty.

clustereval.normalized_inverse_purity_score(labels_true, labels_pred)

Calculate the normalized inverse purity score for a clustering.

This is a min-max normalized version of the inverse purity score. Given the number of unique classes, $L = mathrm{supp}(C)$, the normalized score is defined as:

\[\text{norm. inv. purity} = \frac{\mathrm{inv.\text{-}purity}(C, K) - L/N}{1-L|/N}\]

Parameters:

labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.
labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

normalized_inverse_purity – The normalized inverse purity score for the clustering.

Return type:

float

Raises:

ValueError – If the two label arrays have different lengths or are empty.

clustereval.pair_specificity_score(labels_true, labels_pred)

Calculate the pair specificity score for a clustering.

Pair-based evaluation metrics are based on a binary classification of pairs of samples. This score is the specificity of this binary classifier, defined as the true negative rate.

This score is the pair-based analogue of the homogeneity score.

Parameters:

labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.
labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

pair_specificity – The pair specificity score for the clustering.

Return type:

float

Raises:

ValueError – If the two label arrays have different lengths or are empty.

clustereval.pair_sensitivity_score(labels_true, labels_pred)

Calculate the pair sensitivity score for a clustering.

Pair-based evaluation metrics are based on a binary classification of pairs of samples. This score is the sensitivity of this binary classifier, defined as the true positive rate.

This score is the pair-based analogue of the parsimony score.

Parameters:

labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.
labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

pair_sensitivity – The pair sensitivity score for the clustering.

Return type:

float

Raises:

ValueError – If the two label arrays have different lengths or are empty.

clustereval.completeness_score(labels_true, labels_pred)

Calculate the completeness score for a clustering.

Completeness like parsimony penalizes clustering that fragment classes. However, completeness uses a normalization that depends on the entropy of the specific clustering under evaluation $H(K)$ rather than its maximum over all possible clusterings $\log(N) - H(C)$. Formally, completeness is defined as

\[c(C, K) = 1 - \frac{H(K|C)}{H(K)}\]

Parameters:

labels_true (array-like of shape (n_samples,)) – Ground truth class labels to be used as a reference.
labels_pred (array-like of shape (n_samples,)) – Cluster labels to evaluate.

Returns:

completeness – The completeness score for the clustering.

Return type:

float

Raises:

ValueError – If the two label arrays have different lengths or are empty.

See also

parsimony_score: Related score that uses a normalization based on the maximum possible entropy across all possible clusterings, which behaves more intuitively under refinement.

clustereval.conditional_entropies(labels_true, labels_pred)

Calculate conditional entropies between two partitions.

Parameters:

labels_true (array-like of shape (n_samples,)) – Ground truth class labels C to be used as a reference.
labels_pred (array-like of shape (n_samples,)) – Cluster labels K to evaluate.

Returns:

NamedTuple with joint_entropy, class_given_cluster_entropy, cluster_given_class_entropy, class_entropy, and cluster_entropy fields.

Return type:

ConditionalEntropyResult

Raises:

ValueError – If the two label arrays have different lengths or are empty.

Notes

The logarithm used is the natural logarithm (base-e), so returns are in units of nats.

class clustereval.ConditionalEntropyResult(joint_entropy, class_given_cluster_entropy, cluster_given_class_entropy, class_entropy, cluster_entropy)

Bases: NamedTuple

joint_entropy: float: Alias for field number 0

class_given_cluster_entropy: float: Alias for field number 1

cluster_given_class_entropy: float: Alias for field number 2

class_entropy: float: Alias for field number 3

cluster_entropy: float: Alias for field number 4