Skip to content

Commit 1312374

Browse files
github-actions[bot]Copilot
github-actions[bot]
and
Copilot
authored
Iteration 142: standalone isna/notna/fillna/dropna module
Added `src/stats/notna_isna.ts` — module-level missing-value utilities mirroring pandas' `pd.isna()`, `pd.notna()`, `pd.isnull()`, `pd.notnull()`. Functions work across all input types (scalar, array, Series, DataFrame): - `isna` / `isnull` — detect null/undefined/NaN - `notna` / `notnull` — inverse of isna - `fillna(obj, {value})` — replace missing with fill value - `dropna(obj, opts)` — remove missing; DataFrame supports how/axis - `countna` / `countValid` — count missing / non-missing values Tests: 50+ unit tests + 3 property-based tests (fast-check). Playground: `playground/notna_isna.html` Run: https://github.com/githubnext/tsessebe/actions/runs/24178594390 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent a98d36f commit 1312374

6 files changed

Lines changed: 1165 additions & 0 deletions

File tree

‎playground/index.html‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -284,6 +284,11 @@ <h3><a href="where_mask.html" style="color: var(--accent); text-decoration: none
284284
<p>Element-wise conditional selection: <code>seriesWhere</code> / <code>seriesMask</code> and <code>dataFrameWhere</code> / <code>dataFrameMask</code>. Accepts boolean arrays, label-aligned boolean Series/DataFrame, or callables. Mirrors <code>pandas.Series.where</code>, <code>pandas.DataFrame.where</code>, and their <code>.mask()</code> inverses.</p>
285285
<div class="status done">✅ Complete</div>
286286
</div>
287+
<div class="card">
288+
<h3><a href="notna_isna.html" style="color: var(--accent); text-decoration: none;">🔍 isna / notna</a></h3>
289+
<p>Module-level missing-value detection: <code>isna</code>, <code>notna</code>, <code>isnull</code>, <code>notnull</code> work on scalars, arrays, Series, and DataFrames. Plus standalone <code>fillna</code>, <code>dropna</code>, <code>countna</code>, and <code>countValid</code>. Mirrors <code>pandas.isna</code>, <code>pandas.notna</code>, <code>pandas.isnull</code>, <code>pandas.notnull</code>.</p>
290+
<div class="status done">✅ Complete</div>
291+
</div>
287292
</div>
288293
</section>
289294
</main>

‎playground/notna_isna.html‎

Lines changed: 242 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,242 @@
1+
<!DOCTYPE html>
2+
<html lang="en">
3+
<head>
4+
<meta charset="UTF-8" />
5+
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
6+
<title>tsb · isna / notna — Missing Value Detection</title>
7+
<style>
8+
*, *::before, *::after { box-sizing: border-box; }
9+
body { font-family: system-ui, sans-serif; max-width: 900px; margin: 2rem auto; padding: 0 1rem; color: #1a1a1a; line-height: 1.6; }
10+
h1 { font-size: 1.8rem; margin-bottom: .25rem; }
11+
.subtitle { color: #666; margin-bottom: 2rem; }
12+
h2 { font-size: 1.2rem; margin-top: 2rem; border-bottom: 2px solid #e5e7eb; padding-bottom: .3rem; }
13+
.demo { background: #f8fafc; border: 1px solid #e2e8f0; border-radius: 8px; padding: 1.2rem; margin: 1rem 0; }
14+
.demo label { font-size: .85rem; font-weight: 600; color: #475569; display: block; margin-bottom: .4rem; }
15+
textarea, input[type=text] { width: 100%; padding: .5rem .7rem; font-family: monospace; font-size: .9rem; border: 1px solid #cbd5e1; border-radius: 5px; background: #fff; }
16+
textarea { resize: vertical; min-height: 80px; }
17+
button { margin-top: .6rem; padding: .45rem 1rem; background: #3b82f6; color: #fff; border: none; border-radius: 5px; cursor: pointer; font-size: .9rem; }
18+
button:hover { background: #2563eb; }
19+
.output { font-family: monospace; font-size: .88rem; background: #1e293b; color: #e2e8f0; padding: 1rem; border-radius: 6px; margin-top: .7rem; min-height: 40px; white-space: pre-wrap; }
20+
.tag { display: inline-block; background: #dbeafe; color: #1e40af; padding: .15rem .5rem; border-radius: 999px; font-size: .75rem; font-weight: 600; margin-right: .3rem; }
21+
nav { margin-bottom: 1.5rem; font-size: .9rem; } nav a { color: #3b82f6; text-decoration: none; } nav a:hover { text-decoration: underline; }
22+
table { border-collapse: collapse; width: 100%; margin-top: .5rem; }
23+
th, td { border: 1px solid #e2e8f0; padding: .4rem .7rem; font-size: .88rem; }
24+
th { background: #f1f5f9; font-weight: 600; text-align: left; }
25+
tr:nth-child(even) td { background: #f8fafc; }
26+
.api-row td:first-child { font-family: monospace; font-size: .82rem; }
27+
code { background: #f1f5f9; padding: .1rem .3rem; border-radius: 3px; font-size: .88rem; }
28+
</style>
29+
</head>
30+
<body>
31+
<nav><a href="index.html">← Back to tsb playground</a></nav>
32+
33+
<h1>isna / notna</h1>
34+
<p class="subtitle">Module-level missing-value detection — mirrors <code>pd.isna()</code>, <code>pd.notna()</code>, <code>pd.isnull()</code>, <code>pd.notnull()</code> from pandas.</p>
35+
36+
<h2>What is "missing"?</h2>
37+
<p>In tsb, the following values are considered missing:</p>
38+
<ul>
39+
<li><code>null</code></li>
40+
<li><code>undefined</code></li>
41+
<li><code>NaN</code> (IEEE 754 not-a-number)</li>
42+
</ul>
43+
<p>Everything else — <code>0</code>, <code>false</code>, <code>""</code>, <code>new Date(...)</code> — is <em>not</em> missing.</p>
44+
45+
<h2>API Overview</h2>
46+
<table>
47+
<tr><th>Function</th><th>Input</th><th>Output</th><th>Pandas equivalent</th></tr>
48+
<tr class="api-row"><td>isna(v)</td><td>Scalar</td><td>boolean</td><td>pd.isna(v)</td></tr>
49+
<tr class="api-row"><td>isna(arr)</td><td>Scalar[]</td><td>boolean[]</td><td>pd.isna(arr)</td></tr>
50+
<tr class="api-row"><td>isna(series)</td><td>Series</td><td>Series&lt;boolean&gt;</td><td>pd.isna(series)</td></tr>
51+
<tr class="api-row"><td>isna(df)</td><td>DataFrame</td><td>DataFrame</td><td>pd.isna(df)</td></tr>
52+
<tr class="api-row"><td>notna(v)</td><td>any of above</td><td>same shape, inverted</td><td>pd.notna(v)</td></tr>
53+
<tr class="api-row"><td>isnull / notnull</td><td>any of above</td><td>same as isna/notna</td><td>aliases</td></tr>
54+
<tr class="api-row"><td>fillna(obj, {value})</td><td>Scalar/array/Series/DataFrame</td><td>same type, no missing</td><td>pd.Series.fillna()</td></tr>
55+
<tr class="api-row"><td>dropna(obj, opts?)</td><td>array/Series/DataFrame</td><td>missing entries removed</td><td>pd.Series.dropna()</td></tr>
56+
<tr class="api-row"><td>countna(obj)</td><td>array or Series</td><td>number</td><td>series.isna().sum()</td></tr>
57+
<tr class="api-row"><td>countValid(obj)</td><td>array or Series</td><td>number</td><td>series.count()</td></tr>
58+
</table>
59+
60+
<h2>🔬 Try it: isna on scalars</h2>
61+
<div class="demo">
62+
<label>Test values (comma-separated, use "null", "NaN", "undefined")</label>
63+
<input type="text" id="scalar-input" value="1, null, NaN, 0, false, undefined, &quot;hello&quot;" />
64+
<button onclick="runScalar()">Run isna</button>
65+
<div class="output" id="scalar-output">Click "Run isna" to see results.</div>
66+
</div>
67+
68+
<h2>🔬 Try it: isna on arrays</h2>
69+
<div class="demo">
70+
<label>Array values (JSON array, use null for missing, "NaN" string for NaN)</label>
71+
<textarea id="array-input">[1, null, null, 42, "NaN", "hello", 0, false]</textarea>
72+
<button onclick="runArray()">Run isna</button>
73+
<div class="output" id="array-output">Click "Run isna" to see results.</div>
74+
</div>
75+
76+
<h2>🔬 Try it: fillna on arrays</h2>
77+
<div class="demo">
78+
<label>Array (JSON, use null for missing)</label>
79+
<textarea id="fillna-input">[1, null, null, 4, 5]</textarea>
80+
<label>Fill value</label>
81+
<input type="text" id="fillna-value" value="0" style="width:80px; display:inline-block; margin-left:.5rem" />
82+
<button onclick="runFillna()">Run fillna</button>
83+
<div class="output" id="fillna-output">Click "Run fillna" to see results.</div>
84+
</div>
85+
86+
<h2>🔬 Try it: dropna on arrays</h2>
87+
<div class="demo">
88+
<label>Array (JSON, use null for missing)</label>
89+
<textarea id="dropna-input">[10, null, 20, null, 30]</textarea>
90+
<button onclick="runDropna()">Run dropna</button>
91+
<div class="output" id="dropna-output">Click "Run dropna" to see results.</div>
92+
</div>
93+
94+
<h2>📝 Code examples</h2>
95+
<div class="demo">
96+
<pre style="margin:0;font-size:.87rem;overflow-x:auto">
97+
import { isna, notna, isnull, notnull, fillna, dropna, countna, countValid } from "tsb";
98+
import { Series, DataFrame } from "tsb";
99+
100+
// ── scalar ──────────────────────────────────────────────────
101+
isna(null); // true
102+
isna(undefined); // true
103+
isna(NaN); // true
104+
isna(0); // false — zero is not missing
105+
isna(false); // false — false is not missing
106+
isna(""); // false — empty string is not missing
107+
108+
// ── array ───────────────────────────────────────────────────
109+
isna([1, null, NaN, 3]); // [false, true, true, false]
110+
notna([1, null, NaN, 3]); // [true, false, false, true]
111+
112+
// ── Series ──────────────────────────────────────────────────
113+
const s = new Series({ data: [1, null, NaN, 4] });
114+
isna(s).values; // [false, true, true, false]
115+
notna(s).values; // [true, false, false, true]
116+
117+
// ── DataFrame ───────────────────────────────────────────────
118+
const df = new DataFrame(new Map([
119+
["a", new Series({ data: [1, null, 3] })],
120+
["b", new Series({ data: [NaN, 5, 6] })],
121+
]));
122+
isna(df).col("a").values; // [false, true, false]
123+
isna(df).col("b").values; // [true, false, false]
124+
125+
// ── aliases ─────────────────────────────────────────────────
126+
isnull(null); // true (same as isna)
127+
notnull(42); // true (same as notna)
128+
129+
// ── fillna ──────────────────────────────────────────────────
130+
fillna([1, null, NaN, 4], { value: 0 }); // [1, 0, 0, 4]
131+
fillna(s, { value: -1 }).values; // [1, -1, -1, 4]
132+
fillna(df, { value: 0 }).col("b").values; // [0, 5, 6]
133+
134+
// ── dropna ──────────────────────────────────────────────────
135+
dropna([1, null, NaN, 3]); // [1, 3]
136+
dropna(s).values; // [1, 4]
137+
dropna(df).shape; // [2, 2] (row 0 dropped because b[0]=NaN, row 1 dropped because a[1]=null)
138+
dropna(df, { how: "all" }).shape; // drops only rows where ALL values are missing
139+
dropna(df, { axis: 1 }).columns.values; // drops columns that contain any missing value
140+
141+
// ── countna / countValid ─────────────────────────────────────
142+
countna([1, null, NaN, 3]); // 2
143+
countValid([1, null, NaN, 3]); // 2
144+
</pre>
145+
</div>
146+
147+
<script type="module">
148+
// Minimal inline implementations for the playground (no bundler)
149+
150+
function scalarIsna(v) {
151+
return v === null || v === undefined || (typeof v === "number" && Number.isNaN(v));
152+
}
153+
154+
function isna(obj) {
155+
if (Array.isArray(obj)) return obj.map(scalarIsna);
156+
return scalarIsna(obj);
157+
}
158+
159+
function notna(obj) {
160+
if (Array.isArray(obj)) return obj.map(v => !scalarIsna(v));
161+
return !scalarIsna(obj);
162+
}
163+
164+
function fillna(arr, { value }) {
165+
return arr.map(v => scalarIsna(v) ? value : v);
166+
}
167+
168+
function dropna(arr) {
169+
return arr.filter(v => !scalarIsna(v));
170+
}
171+
172+
function parseVal(s) {
173+
s = s.trim();
174+
if (s === "null") return null;
175+
if (s === "undefined") return undefined;
176+
if (s === "NaN" || s === '"NaN"') return NaN;
177+
if (s === "true") return true;
178+
if (s === "false") return false;
179+
if ((s.startsWith('"') && s.endsWith('"')) || (s.startsWith("'") && s.endsWith("'"))) return s.slice(1,-1);
180+
const n = Number(s);
181+
return isNaN(n) ? s : n;
182+
}
183+
184+
window.runScalar = function() {
185+
try {
186+
const raw = document.getElementById("scalar-input").value;
187+
const vals = raw.split(",").map(parseVal);
188+
const lines = vals.map(v => {
189+
const display = v === null ? "null" : v === undefined ? "undefined" : typeof v === "number" && isNaN(v) ? "NaN" : JSON.stringify(v);
190+
const r = isna(v);
191+
return `isna(${display.padEnd(12)}) → ${r}`;
192+
});
193+
document.getElementById("scalar-output").textContent = lines.join("\n");
194+
} catch(e) { document.getElementById("scalar-output").textContent = "Error: " + e.message; }
195+
};
196+
197+
window.runArray = function() {
198+
try {
199+
let raw = document.getElementById("array-input").value;
200+
// Replace "NaN" strings with actual NaN placeholder
201+
const arr = JSON.parse(raw.replace(/"NaN"/g, "null")).map((v, i) => {
202+
// Check if original had "NaN"
203+
return v;
204+
});
205+
// Re-parse to handle "NaN" strings
206+
const arr2 = JSON.parse(raw).map(v => v === "NaN" ? NaN : v);
207+
const naFlags = isna(arr2);
208+
const notnaFlags = notna(arr2);
209+
const lines = arr2.map((v, i) => {
210+
const disp = v === null ? "null" : typeof v === "number" && isNaN(v) ? "NaN" : JSON.stringify(v);
211+
return `[${i}] value=${String(disp).padEnd(10)} isna=${String(naFlags[i]).padEnd(6)} notna=${notnaFlags[i]}`;
212+
});
213+
document.getElementById("array-output").textContent = lines.join("\n") || "(empty)";
214+
} catch(e) { document.getElementById("array-output").textContent = "Error: " + e.message; }
215+
};
216+
217+
window.runFillna = function() {
218+
try {
219+
const raw = document.getElementById("fillna-input").value;
220+
const arr = JSON.parse(raw);
221+
const valRaw = document.getElementById("fillna-value").value.trim();
222+
const value = parseVal(valRaw);
223+
const result = fillna(arr, { value });
224+
const before = `Before: ${JSON.stringify(arr)}`;
225+
const after = `After: ${JSON.stringify(result)}`;
226+
document.getElementById("fillna-output").textContent = before + "\n" + after;
227+
} catch(e) { document.getElementById("fillna-output").textContent = "Error: " + e.message; }
228+
};
229+
230+
window.runDropna = function() {
231+
try {
232+
const raw = document.getElementById("dropna-input").value;
233+
const arr = JSON.parse(raw);
234+
const result = dropna(arr);
235+
const before = `Before: ${JSON.stringify(arr)} (${arr.length} elements, ${arr.filter(scalarIsna).length} missing)`;
236+
const after = `After: ${JSON.stringify(result)} (${result.length} elements)`;
237+
document.getElementById("dropna-output").textContent = before + "\n" + after;
238+
} catch(e) { document.getElementById("dropna-output").textContent = "Error: " + e.message; }
239+
};
240+
</script>
241+
</body>
242+
</html>

‎src/index.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -125,3 +125,5 @@ export type {
125125
SeriesWhereOptions,
126126
DataFrameWhereOptions,
127127
} from "./stats/index.ts";
128+
export { isna, notna, isnull, notnull, fillna, dropna, countna, countValid } from "./stats/index.ts";
129+
export type { IsnaInput, FillnaOptions, DropnaOptions } from "./stats/index.ts";

‎src/stats/index.ts‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -50,3 +50,14 @@ export type {
5050
SeriesWhereOptions,
5151
DataFrameWhereOptions,
5252
} from "./where_mask.ts";
53+
export {
54+
isna,
55+
notna,
56+
isnull,
57+
notnull,
58+
fillna,
59+
dropna,
60+
countna,
61+
countValid,
62+
} from "./notna_isna.ts";
63+
export type { IsnaInput, FillnaOptions, DropnaOptions } from "./notna_isna.ts";

0 commit comments

Comments
 (0)