Skip to content

Commit db85668

Browse files
michalharakalmichalharakal
authored
Merge branch 'develop' into fxi/0.5-io-reader
2 parents 29038ce + 8118704 commit db85668

53 files changed

Lines changed: 42717 additions & 31 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/ISSUE_TEMPLATE/darc_feature_request.md‎

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
---
2+
name: "DARC Feature Proposal"
3+
about: "Propose a new feature, improvement, or component using the DARC methodology"
4+
title: "[Feature]: "
5+
labels: enhancement
6+
assignees: ""
7+
---
8+
9+
# 🧠 D: DEFINE — Problem & Opportunity
10+
11+
**What is the problem, limitation, or opportunity? Why does this matter for SKaiNET?**
12+
13+
Explain the context in clear terms:
14+
- What functionality is missing or insufficient?
15+
- Why is this important now?
16+
- Who benefits (users, developers, researchers)?
17+
- What is the high-level goal of this proposal?
18+
19+
**Summary (1–2 sentences):**
20+
> _Provide a concise overview of what this issue aims to address._
21+
22+
---
23+
24+
# 🔍 A: ASSESS — Feasibility & Impact
25+
26+
Provide an evaluation of the proposal. Address the following:
27+
28+
### ✔️ Feasibility
29+
- Is the feature straightforward to implement?
30+
- Are there architectural constraints?
31+
32+
### ✔️ Expected Impact
33+
- How does this improve SKaiNET?
34+
- Does it unlock new capabilities?
35+
36+
### ✔️ Risks / Constraints
37+
- Technical challenges?
38+
- Numerical stability concerns?
39+
- API consistency?
40+
41+
### ✔️ Dependencies
42+
- Does this rely on an existing SKaiNET module?
43+
- Does it require third-party libraries or standards?
44+
45+
---
46+
47+
# 📚 R: RESEARCH — What Must Be Understood First?
48+
49+
Document research tasks or open questions that must be answered before implementation:
50+
51+
### Research Tasks
52+
- [ ] Review existing frameworks (PyTorch, JAX, TensorFlow, etc.)
53+
- [ ] Identify relevant algorithms or formulas
54+
- [ ] Compare design patterns for modularity
55+
- [ ] Investigate numerically stable or optimized variants
56+
- [ ] Check for relevant academic papers or benchmarks
57+
58+
### Open Questions
59+
> _List unknowns that contributors should discuss or resolve._
60+
61+
---
62+
63+
# 🛠️ C: CONTRIBUTE — Implementation Plan
64+
65+
Break down actionable steps required to deliver this feature:
66+
67+
### Development Tasks
68+
- [ ] Design module interface
69+
- [ ] Implement core functionality
70+
- [ ] Write unit tests (correctness + edge cases)
71+
- [ ] Add configuration integration
72+
- [ ] Update training loop or pipeline components (if applicable)
73+
- [ ] Add documentation & examples
74+
- [ ] Create or update benchmarks (optional)
75+
- [ ] Submit PR referencing this issue
76+
77+
### Acceptance Criteria
78+
- [ ] Feature works as intended and passes all tests
79+
- [ ] Fully documented with clear examples
80+
- [ ] Follows SKaiNET coding style & modular design
81+
- [ ] Reviewed and approved by maintainers
82+
- [ ] No regressions introduced elsewhere
83+
84+
---
85+
86+
# 💬 Additional Notes
87+
88+
> _Add diagrams, pseudo-code, references, or implementation notes that may help future contributors._

‎.github/workflows/documentation.yml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ jobs:
3232
distribution: 'temurin'
3333

3434
- name: Cache Gradle packages
35-
uses: actions/cache@v4
35+
uses: actions/cache@v5
3636
with:
3737
path: |
3838
~/.gradle/caches
@@ -51,7 +51,7 @@ jobs:
5151
run: ./gradlew generateDocs --stacktrace
5252

5353
- name: Upload generated documentation
54-
uses: actions/upload-artifact@v5
54+
uses: actions/upload-artifact@v6
5555
with:
5656
name: operator-documentation
5757
path: |
@@ -61,7 +61,7 @@ jobs:
6161

6262
- name: Upload documentation preview (PR only)
6363
if: github.event_name == 'pull_request'
64-
uses: actions/upload-artifact@v5
64+
uses: actions/upload-artifact@v6
6565
with:
6666
name: documentation-preview-${{ github.event.number }}
6767
path: |
@@ -77,7 +77,7 @@ jobs:
7777

7878
steps:
7979
- name: Download documentation artifacts
80-
uses: actions/download-artifact@v6
80+
uses: actions/download-artifact@v7
8181
with:
8282
name: documentation-preview-${{ github.event.number }}
8383
path: ./docs-preview

‎.github/workflows/schema-validation.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@ jobs:
2424
distribution: 'temurin'
2525

2626
- name: Cache Gradle packages
27-
uses: actions/cache@v4
27+
uses: actions/cache@v5
2828
with:
2929
path: |
3030
~/.gradle/caches
@@ -44,7 +44,7 @@ jobs:
4444

4545
- name: Upload validation artifacts
4646
if: failure()
47-
uses: actions/upload-artifact@v5
47+
uses: actions/upload-artifact@v6
4848
with:
4949
name: validation-logs
5050
path: |

‎README.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENCE)
2-
[![Maven Central](https://img.shields.io/maven-central/v/sk.ainet.core/SKaiNET-lang-core.svg)](https://central.sonatype.com/artifact/sk.ainet.core/skainet-lang-core)
2+
[![Maven Central](https://img.shields.io/maven-central/v/sk.ainet.core/skainet-lang-core.svg)](https://central.sonatype.com/artifact/sk.ainet.core/skainet-lang-core)
3+
[![DeepWiki](https://img.shields.io/badge/DeepWiki-View%20Docs-blue?logo=readthedocs&logoColor=white)](https://deepwiki.com/sk-ai-net/SKaiNET)
34

45
<img src="docs//SKaiNET-logo.png" alt="SKaiNET logo" width="150">
56

‎gradle/libs.versions.toml‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
[versions]
2-
agp = "8.12.3"
2+
agp = "8.12.0"
33
jacksonDatabind = "2.20.1"
4-
jsonSchemaValidator = "2.0.0"
4+
jsonSchemaValidator = "3.0.0"
55
jsonSchemaValidatorVersion = "0.5.3"
66
junit = "4.13.2"
77
kotlin = "2.2.21"
@@ -11,8 +11,8 @@ android-compileSdk = "36"
1111
kotlinxSerializationJson = "1.9.0"
1212
ktorClientCore = "3.3.3"
1313
ktorClientPlugins = "3.1.1"
14-
logbackClassic = "1.5.21"
15-
kover = "0.9.3"
14+
logbackClassic = "1.5.23"
15+
kover = "0.9.4"
1616
binaryCompatibilityValidator = "0.18.1"
1717
ksp = "2.2.21-2.0.4"
1818
kotlinpoet = "2.2.0"

‎settings.gradle.kts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -58,4 +58,4 @@ include("skainet-models:skainet-model-yolo")
5858

5959
// ====== APPS
6060
//include("skainet-apps:skainet-KGPChat")
61-
include("skainet-apps:skainet-tensor-tools")
61+
include("skainet-apps:skainet-tensor-tools")

‎skainet-apps/skainet-KotlinGPT/build.gradle.kts‎

Lines changed: 122 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,122 @@
1+
import org.jetbrains.kotlin.gradle.ExperimentalKotlinGradlePluginApi
2+
import org.jetbrains.kotlin.gradle.ExperimentalWasmDsl
3+
import org.jetbrains.kotlin.gradle.dsl.JvmTarget
4+
5+
plugins {
6+
alias(libs.plugins.kotlinMultiplatform)
7+
alias(libs.plugins.androidLibrary)
8+
alias(libs.plugins.vanniktech.mavenPublish)
9+
alias(libs.plugins.kover)
10+
alias(libs.plugins.binary.compatibility.validator)
11+
}
12+
13+
kotlin {
14+
jvmToolchain(21)
15+
16+
androidTarget {
17+
@OptIn(ExperimentalKotlinGradlePluginApi::class)
18+
compilerOptions {
19+
jvmTarget.set(JvmTarget.JVM_11)
20+
}
21+
}
22+
23+
iosArm64()
24+
iosSimulatorArm64()
25+
macosArm64 ()
26+
linuxX64 ()
27+
linuxArm64 ()
28+
29+
jvm()
30+
31+
js {
32+
browser()
33+
}
34+
35+
@OptIn(ExperimentalWasmDsl::class)
36+
wasmJs {
37+
browser()
38+
}
39+
40+
sourceSets {
41+
commonMain.dependencies {
42+
implementation(project(":skainet-lang:skainet-lang-core"))
43+
implementation(project(":skainet-compile:skainet-compile-core"))
44+
implementation(project(":skainet-lang:skainet-lang-ksp-annotations"))
45+
46+
}
47+
48+
commonTest.dependencies {
49+
implementation(libs.kotlin.test)
50+
implementation(project(":skainet-lang:skainet-lang-models"))
51+
}
52+
53+
val jvmMain by getting
54+
val jvmTest by getting {
55+
dependencies {
56+
implementation(libs.kotlin.test)
57+
}
58+
}
59+
val androidMain by getting
60+
val wasmJsMain by getting
61+
62+
val commonMain by getting
63+
64+
val nativeMain by creating {
65+
dependsOn(commonMain)
66+
}
67+
68+
val linuxMain by creating {
69+
dependsOn(nativeMain)
70+
}
71+
72+
val iosMain by creating {
73+
dependsOn(nativeMain)
74+
}
75+
76+
val macosMain by creating {
77+
dependsOn(nativeMain)
78+
}
79+
80+
val iosArm64Main by getting {
81+
dependsOn(iosMain)
82+
}
83+
84+
val iosSimulatorArm64Main by getting {
85+
dependsOn(iosMain)
86+
}
87+
88+
val macosArm64Main by getting {
89+
dependsOn(macosMain)
90+
}
91+
92+
val linuxX64Main by getting {
93+
dependsOn(linuxMain)
94+
}
95+
96+
val linuxArm64Main by getting {
97+
dependsOn(linuxMain)
98+
}
99+
}
100+
}
101+
102+
103+
tasks.withType<Test>().configureEach {
104+
jvmArgs("--enable-preview", "--add-modules", "jdk.incubator.vector")
105+
}
106+
107+
tasks.withType<JavaExec>().configureEach {
108+
jvmArgs("--enable-preview", "--add-modules", "jdk.incubator.vector")
109+
}
110+
111+
android {
112+
namespace = "sk.ainet.backend.cpu"
113+
compileSdk = libs.versions.android.compileSdk.get().toInt()
114+
115+
defaultConfig {
116+
minSdk = libs.versions.android.minSdk.get().toInt()
117+
}
118+
compileOptions {
119+
sourceCompatibility = JavaVersion.VERSION_11
120+
targetCompatibility = JavaVersion.VERSION_11
121+
}
122+
}

‎skainet-apps/skainet-KotlinGPT/src/commonMain/kotlin/sk/ainet/apps/knanogpt/CharTokenizer.kt‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
package sk.ainet.apps.knanogpt
2+
3+
class CharTokenizer(text: String) : Tokenizer<Int> {
4+
private val chars: List<Char> = text.toSet().sorted()
5+
private val stoi: Map<Char, Int> = chars.mapIndexed { index, c -> c to index }.toMap()
6+
private val itos: Map<Int, Char> = chars.mapIndexed { index, c -> index to c }.toMap()
7+
8+
override fun encode(text: String): List<Int> {
9+
// Tokenizes the given text into tokens.
10+
return text.map { c -> stoi[c] ?: error("Character not found") }
11+
}
12+
13+
override fun decode(tokens: List<Int>): String {
14+
// Reconstructs the text from the tokens.
15+
return tokens.map { i -> itos[i] ?: error("Token not found") }.joinToString("")
16+
}
17+
}

‎skainet-apps/skainet-KotlinGPT/src/commonMain/kotlin/sk/ainet/apps/knanogpt/Tokenizer.kt‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
package sk.ainet.apps.knanogpt
2+
3+
interface Tokenizer<T> {
4+
5+
fun encode(text: String): List<T>
6+
7+
fun decode(tokens: List<T>): String
8+
}
9+
10+
fun main() {
11+
// Step 1. Import ..
12+
/*
13+
// Step 2. Initialisierung-Block
14+
//how many independent sequences will we process in parallel?
15+
val batchSize = 4
16+
// what is the maximum context length for predictions?
17+
val blockSize = 8
18+
19+
20+
// Step 3. 4. Load & tokenize data
21+
val dataProvider = ResourcesDataProvider("input.txt")
22+
val data = dataProvider.load()
23+
assert(
24+
data.shape.dimensions.size == 1
25+
) { "`shape.dimensions.size` must be ${data.shape.dimensions.size}" }
26+
27+
// Step 5. Train & Test Spilt
28+
val splitter = TrainTestSplitter(data)
29+
val (trainData, testData) = splitter.split(0.9f)
30+
println(trainData.shape.volume)
31+
println(testData.shape.volume)
32+
assert(
33+
trainData.shape.dimensions.size == 1
34+
) { "`shape.dimensions.size` must be ${trainData.shape.dimensions.size}" }
35+
// Step 6. get batch
36+
val batch = BatchProvider(trainData, testData, blockSize, batchSize)
37+
val (xb, yb) = batch.getRandomBatch(true)
38+
39+
print("inputs:")
40+
print(xb.shape)
41+
print(xb)
42+
print("targets:")
43+
print(yb.shape)
44+
print(yb)
45+
46+
repeat(batchSize) { b ->
47+
repeat(blockSize) { t ->
48+
val context = xb[b..b, 0..t]
49+
val target = yb[b, t]
50+
println("when input is $context the target: $target")
51+
}
52+
}
53+
54+
*/
55+
}

0 commit comments

Comments
 (0)