Merge pull request #312 from SKaiNET-developers/release/0.8.0

Release/0.8.0

Author: michalharakal

CHANGELOG.md

@@ -1,5 +1,56 @@
 # Changelog
 
+## [0.8.3] - 2026-01-18
+
+### Changed
+- Updated version to 0.8.3.
+
+## [0.8.2] - 2026-01-18
+
+### Added
+- **KLlama (Llama 2 port)**: Initial version ported from `llama2-kmp`, supporting GGUF models.
+- **GGUF Enhancements**:
+    - Support for `mmap` for zero-copy GGUF tensor loading.
+    - Embedded tokenizer support in GGUF.
+    - New quantization formats: `Q8_0`, `Q4_K`, and BitNet/Ternary support (`TQ1_0`, `TQ2_0`).
+    - Improved loading and bug fixes for quantization and mapping.
+    - Added `int64` support for GGUF.
+    - Improved GGUF metadata loading.
+- **Streaming Support**: Added streaming support for GGUF and ONNX models.
+- **Advanced Operations**:
+    - New activations: `LeakyReLU`, `ELU`.
+    - New pooling: `AvgPool2d`.
+    - New convolutions: `Conv1d`, `Conv3d`.
+- **Optimizers & Training**:
+    - Added `Adam` and `AdamW` optimizers.
+    - Comprehensive loss function library.
+    - New `Metric` interface with `Accuracy` implementation.
+    - KSP-based DSL generator for Network activations.
+- **Data & Datasets**:
+    - Support for `CIFAR-10` and `Fashion-MNIST` datasets.
+    - New `Data Transform API` and `Image Transform DSL`.
+- **Testing & Documentation**:
+    - `skainet-test-groundtruth` module for validation against PyTorch.
+    - Integration tests for quantized inference and `KvCache`.
+    - Shadow JAR support for JVM fat JAR builds.
+    - New documentation for testing architecture with Mermaid diagrams.
+- **WASM/JS**: Initial version of a simple WASM/JS sample.
+
+### Changed
+- Simplified model support to **GGUF-only** (removed legacy Karpathy `.bin` format support).
+- Improved KLlama loading and robustness.
+- Updated roadmap with Phase 1 completion and multi-backend storage abstraction plans.
+- Improved I/O system and overall robustness.
+
+### Fixed
+- Fixed various bugs in quantization and memory mapping.
+- Resolved compilation errors and failing tests in CIFAR-10 support.
+- Fixed KSP and TracingWrapperProcessor tests to match updated log messages.
+- Fixed GGUF metadata loading issues.
+
+## [0.8.1] - 2026-01-18
+- Initial release of 0.8.x series.
+
 ## [0.7.1] - 2026-01-14
 
 ### Added

README.md

@@ -18,8 +18,23 @@ SKaiNET uses a hybrid backend strategy that separates development iteration from
 
 ### SKaiNET is Data
 
-- Data loaders (MNIST, JSON datasets, image helpers)
-- Type‑safe tensors across JVM/JS/Native
+- **Built-in Data Loaders**: `MNIST`, `Fashion-MNIST`, `CIFAR-10`
+- **I/O Formats**: `GGUF`, `ONNX`, `JSON`, `Image` (JPEG, PNG, etc.)
+- **Type-safe Tensors**: Unified API across JVM, JS, and Native
+- **Data Transforms**: Fluent API for data preprocessing, including image resizing, normalization, and tensor conversion.
+- **Transformation DSL**: Compose complex preprocessing pipelines using a type-safe Kotlin DSL.
+
+```kotlin
+// Data Transformation Pipeline
+val transform = transforms<PlatformBitmapImage, Tensor<FP32, Float>> {
+    resize(224, 224)
+    centerCrop(200, 200)
+    toTensor(ctx)
+    normalize(ctx, mean = floatArrayOf(0.485f, 0.456f, 0.406f), std = floatArrayOf(0.229f, 0.224f, 0.225f))
+}
+
+val processedTensor = transform.apply(rawImage)
+```
 
 ```kotlin
 // data loaders
@@ -96,12 +111,18 @@ println(ds.describe())
 
 ### SKaiNET is Compiler
 
-- MLIR/StableHLO based lowering (modules provided in `SKaiNET-compile-*`)
+- **MLIR/StableHLO Backend**: Lowering from high-level Kotlin DSL to MLIR StableHLO dialect.
+- **Optimization Passes**: Extensible transformation API for optimizing the compiled IR.
+    - `ConstantFoldingPass`: Folds arithmetic operations with constant operands.
+    - `OperationFusionPass`: Fuses multiple ops (e.g., Add + ReLU) into efficient kernels.
+    - `DeadCodeEliminationPass`: Removes unused computations.
+
 ```kotlin
-// Illustrative: export graph to JSON/StableHLO IR
-val ir = Compile.toStableHlo(model)
-println(ir.pretty())
+// Applying Compiler Optimizations
+val optimizer = StableHloOptimizer.createDefault()
+val optimizedModule = optimizer.optimize(mlirModule)
 ```
+
 - **Arduino C Code Generation**: Export models to standalone, optimized C99 code with static memory allocation.
 
 ```kotlin
@@ -124,8 +145,8 @@ Read the [Deep Technical Explanation](docs/arduino-c-codegen.md) for more detail
 
 ```kotlin
 dependencies {
-    implementation("sk.ainet.core:SKaiNET-lang-core:0.7.1")
-    implementation("sk.ainet.core:SKaiNET-backend-cpu:0.7.1")
+    implementation("sk.ainet.core:SKaiNET-lang-core:0.8.3")
+    implementation("sk.ainet.core:SKaiNET-backend-cpu:0.8.3")
 }
 // Ready to build & run in ~8 minutes
 ```
@@ -150,15 +171,15 @@ dependencyResolutionManagement {
 
 dependencies {
     // minimal dependency with simple CPU backend
-    implementation("sk.ainet.core:SKaiNET-lang-core:0.7.1")
-    implementation("sk.ainet.core:SKaiNET-backend-cpu:0.7.1")
+    implementation("sk.ainet.core:SKaiNET-lang-core:0.8.3")
+    implementation("sk.ainet.core:SKaiNET-backend-cpu:0.8.3")
 
     // simple model zoo
-    implementation("sk.ainet.core:SKaiNET-lang-models:0.7.1")
+    implementation("sk.ainet.core:SKaiNET-lang-models:0.8.3")
 
     // Optional I/O (e.g., GGUF loader, JSON)
-    implementation("sk.ainet.core:SKaiNET-io-core:0.7.1")
-    implementation("sk.ainet.core:SKaiNET-io-gguf:0.7.1")
+    implementation("sk.ainet.core:SKaiNET-io-core:0.8.3")
+    implementation("sk.ainet.core:SKaiNET-io-gguf:0.8.3")
 }
 ```
 
@@ -168,7 +189,7 @@ Maven:
 <dependency>
   <groupId>sk.ainet.core</groupId>
   <artifactId>SKaiNET-lang-core</artifactId>
-  <version>0.7.1</version>
+  <version>0.8.3</version>
 </dependency>
 ```
 
@@ -177,25 +198,25 @@ Maven:
 - Sample app: https://github.com/SKaiNET-developers/SKaiNET-samples/tree/feature/MNIST/SinusApproximator
 - Kotlin Notebook: https://github.com/SKaiNET-developers/SKaiNET-notebook
 
-## 0.7.1 highlights (with tiny snippets)
+## 0.8.3 highlights (with tiny snippets)
 
-- **Sine Approximation CLI**: Added `skainet-sine-approx-cli` for training models.
-- **Autograd Engine**: Initial support for automatic differentiation and reverse-mode gradients using `DefaultGradientTape`.
-- **Optimization & Training**: New `SgdOptimizer` and training DSL to build and run training loops.
-- **Loss Functions**: Added `MSELoss` and `CrossEntropyLoss` with configurable reduction strategies.
+- **KLlama (Llama 2 port)**: Initial version supporting GGUF models with `mmap` for zero-copy loading.
+- **Quantization & BitNet**: Support for `Q8_0`, `Q4_K`, and BitNet/Ternary (`TQ1_0`, `TQ2_0`) formats.
+- **Streaming & I/O**: Added streaming support for GGUF/ONNX and improved GGUF metadata loading.
+- **Advanced Operations**: Added `LeakyReLU`, `ELU`, `AvgPool2d`, `Conv1d`, and `Conv3d`.
+- **Optimizers & Metrics**: New `Adam`, `AdamW` optimizers and `Accuracy` metrics.
+- **Datasets & Transforms**: Support for `CIFAR-10`, `Fashion-MNIST`, and a new `Data Transform API`.
 
 ```kotlin
-// Example training step with Autograd
-val loss = MSELoss()
-val optimizer = sgd(lr = 0.01)
-
-val (tape, l) = record { loss.forward(model.forward(x, ctx), y, ctx) }
-tape.computeGradients(targets = listOf(l), sources = model.parameters())
-optimizer.step()
+// Example: Streaming inference with KLlama (GGUF)
+val llama = KLlama.load("path/to/model.gguf")
+llama.generate("Once upon a time") { token ->
+    print(token) // streaming output
+}
 ```
 
-- Improved **Graph DSL** with better wiring and recording support.
-- Stability improvements for StableHLO and CUDA backends.
+- **WASM/JS**: Initial support for web-based deployments.
+- **GGUF-only**: Simplified I/O by focusing on GGUF (removed legacy formats).
 
 See [CHANGELOG.md](CHANGELOG.md) for the full list.
 

gradle.properties

@@ -1,5 +1,5 @@
 GROUP=sk.ainet.core
-VERSION_NAME=0.8.0-SNAPSHOT
+VERSION_NAME=0.8.3
 
 POM_DESCRIPTION=SKaiNET
 
@@ -18,7 +18,7 @@ POM_DEVELOPER_URL=https://github.com/SKaiNET-developers/
 
 mavenCentralPublishing=true
 mavenCentralAutomaticPublishing=true
-signAllPublications=false
+signAllPublications=true
 
 #Gradle
 org.gradle.jvmargs=-Xmx2048M -Dfile.encoding=UTF-8 -Dkotlin.daemon.jvm.options\="-Xmx2048M"

skainet-apps/skainet-kllama/gradle.properties

@@ -0,0 +1,4 @@
+POM_ARTIFACT_ID=skainet-apps-kllama
+POM_NAME=skainet neural network scripting API
+
+kotlin.mpp.applyDefaultHierarchyTemplate=false

skainet-compile/skainet-compile-onnx/build.gradle.kts

@@ -5,7 +5,6 @@ import org.jetbrains.kotlin.gradle.dsl.JvmTarget
 plugins {
     alias(libs.plugins.kotlinMultiplatform)
     alias(libs.plugins.androidLibrary)
-    alias(libs.plugins.vanniktech.mavenPublish)
     alias(libs.plugins.kover)
 }
 

skainet-compile/skainet-compile-onnx/gradle.properties

@@ -0,0 +1,2 @@
+POM_ARTIFACT_ID=skainet-data-transform
+POM_NAME=skainet neural basic datasets

skainet-data/skainet-data-media/build.gradle.kts

@@ -5,7 +5,6 @@ import org.jetbrains.kotlin.gradle.dsl.JvmTarget
 plugins {
     alias(libs.plugins.kotlinMultiplatform)
     alias(libs.plugins.androidLibrary)
-    alias(libs.plugins.vanniktech.mavenPublish)
     alias(libs.plugins.kover)
 }
 

skainet-data/skainet-data-transform/gradle.properties

@@ -0,0 +1,2 @@
+POM_ARTIFACT_ID=skainet-test-groundtruth
+POM_NAME=skainet test groundtruth