You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
**Percentify is a niche data science library for practitioners and learners alike, drawing its main dependencies from pandas and numpy, and including everyday statistics.**
10
-
11
-
Following the **Pareto principle**, Percentify brings the 20% of operations that make up 80% of daily data work to the forefront, each as a single, readable function call. No more digging through six-line recipes and hard-to-remember import paths for the checks you run on every dataset.
12
-
13
-
Percentify **does not aim to compete** with pandas, scipy, statsmodels, or scikit-learn; it stands on their shoulders and works *alongside* them. The goal is to make the core concepts easy to learn, quick to use, and simple to remember. Every function names the underlying library it draws from, so the moment you need the full, configurable version, you know exactly where to go.
14
-
15
-
Every function takes a pandas `DataFrame` (or `Series`) and hands back a clean `DataFrame` you can read, sort, or feed straight into the next step.
16
-
17
-
---
18
-
19
-
## 📦 Installation
20
-
```
21
-
pip install percentify
22
-
```
23
-
24
-
Requires `numpy` and `pandas`.
13
+
**80% of the checks you run on every dataset. 20% of the code.**
25
14
26
-
---
15
+
Exploratory stats and data-quality diagnostics for pandas and **Polars** DataFrames. One call each.
27
16
28
-
## 📊 The Toolkit
17
+
> [!TIP]
18
+
> **⚡ Polars is first-class, not an afterthought.** Pass a Polars DataFrame or Series and get the same kind straight back, with no flag and no manual conversion. Every function works on both backends, which is what sets Percentify apart from the pandas-only tools.
29
19
30
-
### `change`: Percentage Change
31
-
Two numbers, two columns, or a whole series at once.
32
-
```python
33
-
from percentify import change
20
+
Where a function wraps an existing library (pandas, scipy, statsmodels, scikit-learn), it names it, so you always know where to dig deeper.
34
21
35
-
change(100, 150) # → 50.0 (a 50% increase)
22
+
## ⭐ `profiler`
36
23
37
-
change(df["forecast"], df["actual"]) # element-wise % change between two columns
24
+
**pandas `.describe()` tells you what your data _is_. `profiler()` tells you what to _do_ about it:** every issue ranked worst-first, each with its fix.
38
25
39
-
change(df["revenue"]) # period-over-period % change down the column
vif(df, flag=5.0) # only rows above the threshold (your problem columns)
31
+
report.to_frame() # every finding, ranked worst-first, with a suggested fix
32
+
report.errors # just the blocking issues
33
+
report.health # a 0 to 100 data-health score
34
+
assertnot report.errors # drop it straight into a CI data-quality gate
60
35
```
61
36
62
-
### `missing`: Missing Data Profiling
63
-
No more `df.isnull().sum() / len(df) * 100`.
64
-
> _Underlying library:_`pandas.DataFrame.isna`
65
-
```python
66
-
from percentify import missing
67
-
68
-
missing(df)
69
-
# column missing_pct
70
-
# 0 salary 12.40
71
-
# 1 age 3.10
72
-
# 2 name 0.00
73
-
```
37
+
Point it at any messy DataFrame, pandas or Polars, and see what it flags before you model. [Try it on your own data →](https://data-centt.github.io/percentify/documentation/#profiler)
74
38
75
-
### `cv`: Coefficient of Variation
76
-
Relative variability (std ÷ mean), as a percentage.
77
-
> _Underlying library:_`scipy.stats.variation`
78
-
```python
79
-
from percentify import cv
39
+
## 📖 Documentation
80
40
81
-
cv(df["salary"]) # → 34.2 (a single Series returns a number)
82
-
cv(df) # → DataFrame of every numeric column, most variable first
83
-
```
41
+
**Full guide, every function, and live examples → [data-centt.github.io/percentify](https://data-centt.github.io/percentify/)**
84
42
85
-
### `outliers`: Percentage of Outliers (IQR Method)
86
-
Stop rewriting the IQR bounds from scratch.
87
-
> _Similar Concept:_`scipy.stats.iqr`
88
-
```python
89
-
from percentify import outliers
43
+
## 📦 Installation
90
44
91
-
outliers(df["salary"]) # → 4.7
92
-
outliers(df) # → DataFrame of every numeric column
45
+
```bash
46
+
pip install percentify
93
47
```
94
48
95
-
### `r_squared`: R-Squared
96
-
```python
97
-
from percentify import r_squared
49
+
Requires Python 3.10+, `numpy`, and `pandas` 2.0+.
98
50
99
-
r_squared(y_true, y_pred) # → 87.3
100
-
```
101
-
> _Similar Concepts:_`sklearn.metrics.r2_score`
51
+
## Quick example
102
52
103
-
### `pca_variance`: PCA Variance Breakdown
104
-
Columns are standardized by default, so a feature measured in large units (e.g.
105
-
dollars) can't dominate the result just because of its scale. Pass
106
-
`standardize=False` for covariance-based PCA on the raw values.
One import, one line. A clean, sorted DataFrame you can read or feed into the next step.
71
+
72
+
## What's inside
73
+
74
+
| Function | What it answers |
75
+
|---|---|
76
+
|`profiler`| What is wrong with this dataset, and how do I fix it? |
77
+
|`change`| Growth as numbers, columns, or a whole series |
78
+
|`vif`| Which features are collinear? |
79
+
|`missing`| How much of each column is missing? |
80
+
|`cv`| How variable is each column, relative to its mean? |
81
+
|`outliers`| What percentage of each column are outliers? |
82
+
|`pca_variance`| How much variance does each principal component explain? |
83
+
|`pca_loadings`| What does each principal component consist of? |
84
+
|`imbalance`| How skewed are the classes in a target column? |
85
+
|`correlate`| Which features move together, and is it significant? |
86
+
|`skew_report`| How skewed is each column, and what transform helps? |
87
+
|`bootstrap_ci`| What is the confidence interval for a statistic? |
88
+
|`permutation_test`| Are two groups really different? (a p-value) |
89
+
|`effect_size`| How big is the difference, not just whether it is significant? |
90
+
|`difference`| How far apart are two values or columns? |
91
+
|`split`| How does a total divide across weights or groups? |
92
+
|`display`| Format numbers or a column as clean "%" strings |
93
+
94
+
→ See the **[documentation](https://data-centt.github.io/percentify/)** for a worked, real-output example of every function.
121
95
122
96
## 🛟 Friendly by design
123
97
124
-
Built for early-career analysts as much as seasoned ones:
125
-
126
-
-**No cryptic tracebacks.** Hand it a text column where numbers are needed and you get a clear `PercentifyWarning` ("numeric columns required") plus an empty result — not an Arrow/NumPy stack trace.
127
-
-**Sensible defaults.** Results come back sorted worst-first (most collinear, most missing, most variable), and PCA is standardized out of the box.
128
-
-**DataFrames everywhere**, so the output drops straight into your notebook, your next filter, or your model.
-**No cryptic tracebacks**; Hand a function a text column where numbers are needed and you get a clear PercentifyWarning, not an Arrow/NumPy stack trace.
99
+
-**Sensible defaults**; Results come back sorted worst-first, and PCA is standardized out of the box.
100
+
-**DataFrames everywhere**; so the output drops straight into your notebook, your next filter, or your model.
101
+
-**Pandas or polars**; pass either a pandas or polars object and you get the same kind back, no flag needed.
137
102
138
-
---
139
103
140
104
## 🤝 Contributing
141
105
142
-
Contributions are welcome — but they must follow the repo's guiding principle:
106
+
Contributions are welcome but they must follow the repo's guiding principle:
107
+
> Keep each method as direct-to-output as possible. A percentify function should return the single most common answer in one line, and point users to the underlying library (pandas, scipy, statsmodels, scikit-learn) for the full, configurable version when the simplest output isn't what they're after.
143
108
144
-
> **Keep each method as direct-to-output as possible.**A percentify function should return the single most common answer in one line, and point users to the underlying library (pandas, scipy, statsmodels, scikit-learn) for the full, configurable version when the simplest output isn't what they're after.
109
+
**It must support polars.**Every function accepts both pandas and polars objects (via the `@_backend_aware` decorator) and returns the same kind, so any new contribution must keep that parity.
145
110
146
111
If your idea keeps things that simple and direct:
147
112
- Open an issue first to discuss it
@@ -150,4 +115,5 @@ If your idea keeps things that simple and direct:
150
115
- Commit your changes
151
116
- Open a pull request
152
117
153
-
> Anything that adds knobs and options for their own sake, or duplicates what the parent libraries already do well, is out of scope please those cases should point to the source library instead.
118
+
> Anything that adds knobs and options for their own sake, or duplicates what the parent libraries already do well, is out of scope; those cases should point to the source library instead.
119
+
I try to keep it compact on purpose, to discuss big new features first.
0 commit comments