Skip to content

Commit dd5ac8c

Browse files
NucsNucs
committed
feat(dtypes): NumPy 2.x type-alias alignment + np.dtype parser rewrite
Completes the NumPy 2.4.2 parity pass started with Rounds 1-15 by aligning np.* class-level type aliases, rewriting np.dtype(string) with a full FrozenDictionary lookup, extending finfo/iinfo to the new dtypes, adding TypeError / IndexError throwing at NumPy-canonical rejection sites, and plumbing Complex through matmul + UnmanagedMemoryBlock fills. ~1,912 new test LoC across 9 new files + updates to 6 existing test files. np.* type aliases (src/NumSharp.Core/APIs/np.cs) ------------------------------------------------ Breaking changes to match NumPy 2.4.2: np.byte byte (uint8) -> sbyte (int8) NumPy C-char convention np.complex64 complex128 -> throws NSE no silent widening np.csingle complex128 -> throws NSE no silent widening np.uint uint64 -> uintp (ptr) NumPy 2.x np.intp nint -> long on 64-bit (nint has NPTypeCode.Empty which breaks dispatch) np.uintp nuint -> ulong on 64-bit np.int_ long -> intp NumPy 2.x (int_ == intp) Added aliases: np.short, np.ushort, np.intc, np.uintc, np.longlong, np.ulonglong, np.single, np.cdouble, np.clongdouble Platform-detected (C-long convention: 32-bit MSVC / 64-bit *nix LP64): np.@long, np.@Ulong np.dtype(string) parser (src/NumSharp.Core/Creation/np.dtype.cs) ----------------------------------------------------------------- Regex parser replaced with a FrozenDictionary<string, Type> built once at static init. Platform-detection helpers (_cLongType, _cULongType, _intpType, _uintpType) declared BEFORE the dictionary since static initializers run top-down and BuildDtypeStringMap reads them. Covers: - Single-char NumPy codes: ? b B h H i I l L q Q p P e f d g D G - Sized forms: b1 i1 u1 i2 u2 i4 u4 i8 u8 f2 f4 f8 c16 - Lowercase names: bool int8..int64 uint8..uint64 float16..float64 complex complex128 half single double byte ubyte short ushort intc uintc int_ intp uintp bool_ int uint long ulong longlong ulonglong longdouble clongdouble - NumSharp-friendly: SByte Byte UByte Int16..UInt64 Half Single Float Double Complex Bool Boolean boolean Char char decimal Unsupported codes throw NotSupportedException: - Bytestring (S / a), Unicode (U), datetime (M), timedelta (m), object (O), void (V) - NumSharp has no equivalents - complex64 / 'F' / 'c8' - NumSharp only has complex128 np.finfo + np.iinfo (src/NumSharp.Core/APIs/np.{finfo,iinfo}.cs) ---------------------------------------------------------------- np.finfo gains: - Half (IEEE binary16: bits=16, eps=2^-10, smallest_subnormal=2^-24, maxexp=16, minexp=-14) - Complex (reports underlying float64 values with dtype=float64 per NumPy parity: finfo(complex128).dtype == float64) np.iinfo gains SByte (int8) with signed min/max and 'i' kind. IsSupportedType extended to accept Half, Complex, SByte. find_common_type table (src/NumSharp.Core/Logic/np.find_common_type.cs) ----------------------------------------------------------------------- ~30 table entries swapped from np.complex64 -> np.complex128 to reflect NumPy 2.4.2 rules and avoid relying on the now-throwing alias. No behavioral change for callers: the previous complex64 alias pointed at Complex anyway. NDArray implicit/explicit casts ------------------------------- src/NumSharp.Core/Casting/Implicit/NdArray.Implicit.ValueTypes.cs Added implicit scalar -> NDArray for `sbyte` and `Half`. Added explicit NDArray -> `sbyte` scalar. Common validation factored into EnsureCastableToScalar(nd, targetType, targetIsComplex): - ndim != 0 -> IncorrectShapeException - non-complex target + complex source -> TypeError Python's `int(complex(1, 2))` raises TypeError; NumSharp matches. NumPy's ComplexWarning (silent imaginary drop) treated as a hard error since NumSharp has no warning mechanism. NumPy-parity error types at rejection sites -------------------------------------------- - Default.Shift.ValidateIntegerType: NotSupportedException -> TypeError ("ufunc 'left_shift' not supported for the input types, ... safe casting") - NDArray.Indexing.Selection.{Getter,Setter}: ArgumentException -> IndexError ("only integers, slices (':'), ellipsis ('...'), numpy.newaxis ('None') and integer or boolean arrays are valid indices") - np.repeat: permissive Half/Complex truncation -> TypeError ("Cannot cast array data from dtype('float16') to dtype('int64') according to the rule 'safe'") New exception (src/NumSharp.Core/Exceptions/IndexError.cs): public class IndexError : NumSharpException Mirrors Python's IndexError. Raised for out-of-range subscripts and invalid index types (e.g. float/complex index on an ndarray). UnmanagedMemoryBlock.Allocate cross-type fill --------------------------------------------- src/NumSharp.Core/Backends/Unmanaged/UnmanagedMemoryBlock.cs Replaced direct boxing casts `(Half)fill` / `(Complex)fill` / etc with Utilities.Converts.ToXxx(fill) dispatchers. Previously `fill = 1` passed to a Half array threw InvalidCastException because a boxed int cannot unbox to Half. Now follows the same NumPy-parity wrapping path as the rest of the casting subsystem (int -> Half, double -> Complex, etc). Complex matmul -------------- src/NumSharp.Core/Backends/Default/Math/BLAS/Default.MatMul.2D2D.cs MatMulMixedType<TResult> now short-circuits to MatMulComplexAccumulator when TResult is Complex - the double-precision accumulator was dropping imaginary parts for Complex outputs. The dedicated path accumulates in Complex across K and writes Complex-precision results. np.asanyarray ------------- src/NumSharp.Core/Creation/np.asanyarray.cs Half and System.Numerics.Complex added to the scalar-detection branch. Previously fell through to "Unable to resolve asanyarray for type Half/Complex" because neither matched IsPrimitive. Test coverage (~1,912 new LoC) ------------------------------ NpTypeAliasParityTests 174 LoC - every np.* alias vs NumPy 2.4.2 np.finfo.NewDtypesTests 262 LoC - Half / Complex finfo np.iinfo.NewDtypesTests 95 LoC - SByte iinfo UnmanagedMemoryBlockAllocateTests 226 LoC - cross-type fills ComplexToRealTypeErrorTests 170 LoC - Complex -> int/float scalar cast NDArrayScalarCastTests 384 LoC - 0-d cast matrix (implicit + explicit) Complex64RefusalTests 116 LoC - complex64 / csingle throw DTypePlatformDivergenceTests 166 LoC - 'l'/'L'/'int' platform behavior DTypeStringParityTests 319 LoC - every dtype string vs NumPy Updates to existing tests: - ConvertsBattleTests.cs: [Misaligned] tags removed from Half/Complex repeat/shift/index cases; assertions aligned to NumPy-parity TypeError - ShiftOpTests.cs: NotSupportedException -> TypeError - np.finfo.BattleTest / np.iinfo.BattleTest: "float" now -> 64 bits (alias for float64); "int" now -> intp (64 on 64-bit) - np.dtype.Test: split into Case1_ValidForms / renamed classes - np.find_common_type.Test: complex64 -> complex128; added Case4b_c8_ThrowsNotSupported guard Docs ---- docs/website-src/docs/NDArray.md 663 LoC - user-facing NDArray guide docs/website-src/docs/dtypes.md 610 LoC - dtype reference docs/website-src/docs/toc.yml NDArray + Dtypes added to TOC docs/plans/REVIEW_FINDINGS.md 306 LoC - review notes docs/releases/RELEASE_0.51.0-prerelease.md - release notes for the branch
1 parent bacf59d commit dd5ac8c

34 files changed

Lines changed: 4356 additions & 427 deletions

‎docs/plans/REVIEW_FINDINGS.md‎

Lines changed: 306 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/releases/RELEASE_0.51.0-prerelease.md‎

Lines changed: 228 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/website-src/docs/NDArray.md‎

Lines changed: 663 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/website-src/docs/dtypes.md‎

Lines changed: 610 additions & 0 deletions
Large diffs are not rendered by default.

‎docs/website-src/docs/toc.yml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,10 @@
22
href: ../index.md
33
- name: Introduction
44
href: intro.md
5+
- name: NDArray
6+
href: NDArray.md
7+
- name: Dtypes
8+
href: dtypes.md
59
- name: Broadcasting
610
href: broadcasting.md
711
- name: Buffering & Memory

‎src/NumSharp.Core/APIs/np.cs‎

Lines changed: 65 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
using System;
22
using System.Numerics;
3+
using System.Runtime.InteropServices;
34

45
namespace NumSharp
56
{
@@ -16,50 +17,105 @@ public static partial class np
1617
/// <remarks>https://numpy.org/doc/stable/user/basics.indexing.html<br></br><br></br>https://stackoverflow.com/questions/42190783/what-does-three-dots-in-python-mean-when-indexing-what-looks-like-a-number</remarks>
1718
public static readonly Slice newaxis = new Slice(null, null, 1) {IsNewAxis = true};
1819

20+
// Platform-detected C-type sizes. See np.dtype.cs for the same detection logic
21+
// used by string parsing ('l', 'L', 'long', 'ulong' follow C long, which is
22+
// 32-bit on Windows/MSVC (LLP64) and 64-bit on 64-bit Linux/Mac (LP64)).
23+
private static readonly Type _np_cLong =
24+
RuntimeInformation.IsOSPlatform(OSPlatform.Windows)
25+
? typeof(int)
26+
: (IntPtr.Size == 8 ? typeof(long) : typeof(int));
27+
private static readonly Type _np_cULong =
28+
RuntimeInformation.IsOSPlatform(OSPlatform.Windows)
29+
? typeof(uint)
30+
: (IntPtr.Size == 8 ? typeof(ulong) : typeof(uint));
31+
1932
// https://numpy.org/doc/stable/user/basics.types.html
2033
public static readonly Type bool_ = typeof(bool);
2134
public static readonly Type bool8 = bool_;
2235
public static readonly Type @bool = bool_;
2336

2437
public static readonly Type @char = typeof(char);
2538

26-
public static readonly Type @byte = typeof(byte);
39+
// NumPy: np.byte = int8 (signed, C char convention). NumSharp follows NumPy.
40+
// For .NET-style uint8 use np.uint8 / np.ubyte.
41+
public static readonly Type @byte = typeof(sbyte);
42+
public static readonly Type int8 = typeof(sbyte);
43+
public static readonly Type @sbyte = typeof(sbyte);
44+
2745
public static readonly Type uint8 = typeof(byte);
2846
public static readonly Type ubyte = uint8;
2947

30-
public static readonly Type @sbyte = typeof(sbyte);
31-
public static readonly Type int8 = typeof(sbyte);
32-
48+
public static readonly Type @short = typeof(short);
3349
public static readonly Type int16 = typeof(short);
3450

51+
public static readonly Type @ushort = typeof(ushort);
3552
public static readonly Type uint16 = typeof(ushort);
3653

54+
// 'intc' / 'uintc' are NumPy's aliases for C 'int' / 'unsigned int' (always 32-bit in practice).
55+
public static readonly Type intc = typeof(int);
56+
public static readonly Type uintc = typeof(uint);
3757
public static readonly Type int32 = typeof(int);
38-
3958
public static readonly Type uint32 = typeof(uint);
4059

60+
// 'long' / 'ulong' follow C long convention — platform-dependent (32-bit on Windows).
61+
// Access as np.@long / np.@ulong because `long` / `ulong` are C# keywords.
62+
public static readonly Type @long = _np_cLong;
63+
public static readonly Type @ulong = _np_cULong;
64+
65+
// 'longlong' / 'ulonglong' are C 'long long' / 'unsigned long long' — always 64-bit.
66+
public static readonly Type longlong = typeof(long);
67+
public static readonly Type ulonglong = typeof(ulong);
68+
69+
// NumPy 2.x: int_ and intp are pointer-sized (int64 on 64-bit platforms).
70+
// On 64-bit OS typeof(long) is the correct choice — NOT typeof(nint), which is
71+
// System.IntPtr and has NPTypeCode.Empty (breaks np.zeros/np.empty dispatch).
4172
public static readonly Type int_ = typeof(long);
4273
public static readonly Type int64 = int_;
43-
public static readonly Type intp = typeof(nint);
44-
public static readonly Type uintp = typeof(nuint);
74+
public static readonly Type intp = IntPtr.Size == 8 ? typeof(long) : typeof(int);
75+
public static readonly Type uintp = IntPtr.Size == 8 ? typeof(ulong) : typeof(uint);
4576
public static readonly Type int0 = int_;
4677

4778
public static readonly Type uint64 = typeof(ulong);
4879
public static readonly Type uint0 = uint64;
49-
public static readonly Type @uint = uint64;
80+
public static readonly Type @uint = uintp; // NumPy 2.x: np.uint == np.uintp (pointer-sized)
5081

5182
public static readonly Type float16 = typeof(Half);
5283
public static readonly Type half = float16;
5384

5485
public static readonly Type float32 = typeof(float);
86+
public static readonly Type single = float32;
5587

5688
public static readonly Type float_ = typeof(double);
5789
public static readonly Type float64 = float_;
5890
public static readonly Type @double = float_;
5991

92+
// ---- Complex ----
93+
// NumSharp's Complex = System.Numerics.Complex = two 64-bit floats (complex128).
94+
// There is NO complex64 in NumSharp — any attempt to use it throws.
6095
public static readonly Type complex_ = typeof(Complex);
6196
public static readonly Type complex128 = complex_;
62-
public static readonly Type complex64 = complex_;
97+
public static readonly Type cdouble = complex_; // NumPy alias for complex128
98+
public static readonly Type clongdouble = complex_; // NumPy: long-double complex collapses to complex128
99+
100+
/// <summary>
101+
/// NumSharp does not support <c>complex64</c> (two 32-bit floats). The only complex
102+
/// type available is <see cref="complex128"/> (two 64-bit floats, backed by
103+
/// <see cref="System.Numerics.Complex"/>). Accessing this property throws
104+
/// <see cref="NotSupportedException"/>; use <see cref="complex128"/> or
105+
/// <see cref="complex_"/> instead.
106+
/// </summary>
107+
public static Type complex64 => throw new NotSupportedException(
108+
"NumSharp does not support complex64 (two 32-bit floats). " +
109+
"Use np.complex128 (System.Numerics.Complex, two 64-bit floats) instead.");
110+
111+
/// <summary>
112+
/// NumPy alias for complex64. Same as <see cref="complex64"/> — throws
113+
/// because NumSharp does not support complex64.
114+
/// </summary>
115+
public static Type csingle => throw new NotSupportedException(
116+
"NumSharp does not support csingle (= complex64, two 32-bit floats). " +
117+
"Use np.complex128 / np.cdouble instead.");
118+
63119
public static readonly Type @decimal = typeof(decimal);
64120

65121
public static Type chars => throw new NotSupportedException("Please use char with extra dimension.");

‎src/NumSharp.Core/APIs/np.finfo.cs‎

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,10 +87,30 @@ public finfo(NPTypeCode typeCode)
8787
if (!IsFloatType(typeCode))
8888
throw new ArgumentException($"data type '{typeCode.AsNumpyDtypeName()}' not inexact", nameof(typeCode));
8989

90-
dtype = typeCode;
90+
// NumPy parity: np.finfo(np.complex128).dtype == np.float64.
91+
// The finfo represents the precision of the underlying real component, so
92+
// we report float64's machine limits with dtype set to the real type.
93+
// System.Numerics.Complex is 2 × float64 → underlying dtype is Double.
94+
dtype = typeCode == NPTypeCode.Complex ? NPTypeCode.Double : typeCode;
9195

9296
switch (typeCode)
9397
{
98+
case NPTypeCode.Half:
99+
// IEEE 754 binary16: 1 sign + 5 exponent + 10 mantissa bits.
100+
bits = 16;
101+
eps = 0.0009765625; // 2^-10
102+
epsneg = 0.00048828125; // 2^-11
103+
max = (double)Half.MaxValue; // 65504
104+
min = (double)Half.MinValue; // -65504
105+
smallest_normal = 6.103515625e-05; // 2^-14
106+
smallest_subnormal = 5.960464477539063e-08; // 2^-24 (= (double)Half.Epsilon)
107+
tiny = smallest_normal;
108+
precision = 3; // decimal digits of precision
109+
resolution = 1e-3; // 10^-precision
110+
maxexp = 16; // bias+1 = 2^15*(2-eps) = MaxValue
111+
minexp = -14; // 2^-14 = smallest normal
112+
break;
113+
94114
case NPTypeCode.Single:
95115
bits = 32;
96116
// float.Epsilon is the smallest subnormal
@@ -110,6 +130,7 @@ public finfo(NPTypeCode typeCode)
110130
break;
111131

112132
case NPTypeCode.Double:
133+
case NPTypeCode.Complex: // NumPy: finfo(complex128) reports float64 values
113134
bits = 64;
114135
eps = Math.BitIncrement(1.0) - 1.0; // ~2.22e-16
115136
epsneg = 1.0 - Math.BitDecrement(1.0);
@@ -165,8 +186,10 @@ private static bool IsFloatType(NPTypeCode typeCode)
165186
{
166187
return typeCode switch
167188
{
189+
NPTypeCode.Half => true,
168190
NPTypeCode.Single => true,
169191
NPTypeCode.Double => true,
192+
NPTypeCode.Complex => true, // reports underlying float precision
170193
NPTypeCode.Decimal => true, // Partial support - no subnormals
171194
_ => false
172195
};

‎src/NumSharp.Core/APIs/np.iinfo.cs‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -81,7 +81,8 @@ private static bool IsIntegerType(NPTypeCode typeCode)
8181
{
8282
return typeCode switch
8383
{
84-
NPTypeCode.Boolean => true, // NumPy treats bool as integer-like for iinfo
84+
NPTypeCode.Boolean => true, // NumSharp extension — NumPy 2.x throws ValueError
85+
NPTypeCode.SByte => true,
8586
NPTypeCode.Byte => true,
8687
NPTypeCode.Int16 => true,
8788
NPTypeCode.UInt16 => true,
@@ -99,6 +100,7 @@ private static (int bits, long min, long max, ulong maxUnsigned, char kind) GetT
99100
return typeCode switch
100101
{
101102
NPTypeCode.Boolean => (8, 0, 1, 1, 'b'),
103+
NPTypeCode.SByte => (8, sbyte.MinValue, sbyte.MaxValue, (ulong)sbyte.MaxValue, 'i'),
102104
NPTypeCode.Byte => (8, 0, byte.MaxValue, byte.MaxValue, 'u'),
103105
NPTypeCode.Int16 => (16, short.MinValue, short.MaxValue, (ulong)short.MaxValue, 'i'),
104106
NPTypeCode.UInt16 => (16, 0, ushort.MaxValue, ushort.MaxValue, 'u'),

‎src/NumSharp.Core/Backends/Default/Math/BLAS/Default.MatMul.2D2D.cs‎

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -296,11 +296,19 @@ private static unsafe void MatMulContiguous(long* a, long* b, long* result, long
296296
/// <summary>
297297
/// General path for mixed types or strided arrays.
298298
/// Converts to double for computation, then back to result type.
299+
/// For Complex result type, routes to a dedicated Complex accumulator that preserves imaginary.
299300
/// </summary>
300301
[MethodImpl(MethodImplOptions.AggressiveOptimization)]
301302
private static unsafe void MatMulMixedType<TResult>(NDArray left, NDArray right, TResult* result, long M, long K, long N)
302303
where TResult : unmanaged
303304
{
305+
// NumPy parity: Complex matmul must preserve imaginary components (double accumulator would drop them).
306+
if (typeof(TResult) == typeof(System.Numerics.Complex))
307+
{
308+
MatMulComplexAccumulator(left, right, (System.Numerics.Complex*)result, M, K, N);
309+
return;
310+
}
311+
304312
// Use double accumulator for precision
305313
var accumulator = new double[N];
306314

@@ -341,6 +349,40 @@ private static unsafe void MatMulMixedType<TResult>(NDArray left, NDArray right,
341349
}
342350
}
343351

352+
[MethodImpl(MethodImplOptions.AggressiveOptimization)]
353+
private static unsafe void MatMulComplexAccumulator(NDArray left, NDArray right, System.Numerics.Complex* result, long M, long K, long N)
354+
{
355+
var accumulator = new System.Numerics.Complex[N];
356+
var leftCoords = new long[2];
357+
var rightCoords = new long[2];
358+
359+
for (long i = 0; i < M; i++)
360+
{
361+
Array.Clear(accumulator);
362+
363+
leftCoords[0] = i;
364+
for (long k = 0; k < K; k++)
365+
{
366+
leftCoords[1] = k;
367+
System.Numerics.Complex aik = Converts.ToComplex(left.GetValue(leftCoords));
368+
369+
rightCoords[0] = k;
370+
for (long j = 0; j < N; j++)
371+
{
372+
rightCoords[1] = j;
373+
System.Numerics.Complex bkj = Converts.ToComplex(right.GetValue(rightCoords));
374+
accumulator[j] += aik * bkj;
375+
}
376+
}
377+
378+
System.Numerics.Complex* resultRow = result + i * N;
379+
for (long j = 0; j < N; j++)
380+
{
381+
resultRow[j] = accumulator[j];
382+
}
383+
}
384+
}
385+
344386
#endregion
345387
}
346388
}

‎src/NumSharp.Core/Backends/Default/Math/Default.Shift.cs‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -35,6 +35,7 @@ public override NDArray RightShift(NDArray lhs, NDArray rhs)
3535

3636
/// <summary>
3737
/// Validate that the array is an integer type.
38+
/// Raises TypeError to match NumPy's ufunc dtype rejection.
3839
/// </summary>
3940
private static void ValidateIntegerType(NDArray arr, string opName)
4041
{
@@ -44,7 +45,7 @@ private static void ValidateIntegerType(NDArray arr, string opName)
4445
typeCode != NPTypeCode.Int32 && typeCode != NPTypeCode.UInt32 &&
4546
typeCode != NPTypeCode.Int64 && typeCode != NPTypeCode.UInt64)
4647
{
47-
throw new NotSupportedException($"{opName} only supports integer types, got {typeCode}");
48+
throw new TypeError($"ufunc '{opName}' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''");
4849
}
4950
}
5051

0 commit comments

Comments
 (0)