Skip to content

Commit e8ecc11

Browse files
committed
Feature: Add load-from-string primitive
1 parent ee4d122 commit e8ecc11

10 files changed

Lines changed: 259 additions & 11 deletions

File tree

README.md

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -206,6 +206,7 @@ There may be rare occasions in which you don't want the NW extension to remember
206206
[`nw:save-graphml`](#nwsave-graphml)
207207
[`nw:load-graphml`](#nwload-graphml)
208208
[`nw:load`](#nwload)
209+
[`nw:load-from-string`](#nwload-from-string)
209210
[`nw:save`](#nwsave)
210211

211212
### Centrality Measures
@@ -1337,6 +1338,37 @@ Limitations:
13371338

13381339

13391340

1341+
### `nw:load-from-string`
1342+
1343+
```NetLogo
1344+
nw:load-from-string format data default-turtle-breed default-link-breed optional-command-block
1345+
```
1346+
1347+
1348+
Load `data` from an in-memory string rather than from a file. This is useful in NetLogo Web and other contexts where the data comes from somewhere other than the file system, such as the `fetch` or `resource` extensions.
1349+
1350+
Because there is no file name to infer the file-type from, the first argument, `format`, names the format explicitly. It is case-insensitive and an optional leading dot is ignored, so `"gml"`, `"GML"`, and `".gml"` are all equivalent. The supported formats are:
1351+
1352+
- `dl`
1353+
- `gdf`
1354+
- `gexf`
1355+
- `gml`
1356+
- `graphml`
1357+
- `matrix`
1358+
- `vna`
1359+
1360+
Aside from taking its data from a string and its format from the `format` argument, `nw:load-from-string` behaves exactly like the corresponding `nw:load-*` primitive, including matching node and edge attributes to turtle and link variables and honoring `breed`.
1361+
1362+
For example, to load a GML network fetched from a URL:
1363+
1364+
```
1365+
fetch:url-async "https://example.com/network.gml" [ [gml] ->
1366+
nw:load-from-string "gml" gml turtles links
1367+
] [ [err] -> print err ]
1368+
```
1369+
1370+
1371+
13401372
### `nw:save`
13411373

13421374
```NetLogo

documentation.conf

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1147,6 +1147,36 @@ Limitations:
11471147
"""
11481148
},
11491149

1150+
{
1151+
name: load-from-string,
1152+
type: command,
1153+
tags: [ io ],
1154+
arguments: [ {name: format, type: string}, {name: data, type: string}, {name: default-turtle-breed, type: turtleset}, {name: default-link-breed, type: linkset}, {name: optional-command-block, type: "optional command block"} ],
1155+
description: """
1156+
Load network `data` from an in-memory string rather than from a file. This is useful in NetLogo Web and other contexts where the data comes from somewhere other than the file system, such as the `fetch` or `resource` extensions.
1157+
1158+
Because there is no file name to infer the file-type from, the first argument, `format`, names the format explicitly. It is case-insensitive and an optional leading dot is ignored, so `"gml"`, `"GML"`, and `".gml"` are all equivalent. The supported formats are:
1159+
1160+
- `dl`
1161+
- `gdf`
1162+
- `gexf`
1163+
- `gml`
1164+
- `graphml`
1165+
- `matrix`
1166+
- `vna`
1167+
1168+
Aside from taking its data from a string and its format from the `format` argument, `nw:load-from-string` behaves exactly like the corresponding `nw:load-*` primitive, including matching node and edge attributes to turtle and link variables and honoring `breed`.
1169+
1170+
For example, to load a GML network fetched from a URL:
1171+
1172+
```
1173+
fetch:url-async "https://example.com/network.gml" [ [gml] ->
1174+
nw:load-from-string "gml" gml turtles links
1175+
] [ [err] -> print err ]
1176+
```
1177+
"""
1178+
},
1179+
11501180
{
11511181
name: save,
11521182
type: command,

documentation.yaml

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1221,6 +1221,44 @@ primitives:
12211221
tags:
12221222
- io
12231223
type: command
1224+
- arguments:
1225+
- name: format
1226+
type: string
1227+
- name: data
1228+
type: string
1229+
- name: default-turtle-breed
1230+
type: turtleset
1231+
- name: default-link-breed
1232+
type: linkset
1233+
- name: optional-command-block
1234+
type: optional command block
1235+
description: |2
1236+
1237+
Load network `data` from an in-memory string rather than from a file. This is useful in NetLogo Web and other contexts where the data comes from somewhere other than the file system, such as the `fetch` or `resource` extensions.
1238+
1239+
Because there is no file name to infer the file-type from, the first argument, `format`, names the format explicitly. It is case-insensitive and an optional leading dot is ignored, so `"gml"`, `"GML"`, and `".gml"` are all equivalent. The supported formats are:
1240+
1241+
- `dl`
1242+
- `gdf`
1243+
- `gexf`
1244+
- `gml`
1245+
- `graphml`
1246+
- `matrix`
1247+
- `vna`
1248+
1249+
Aside from taking its data from a string and its format from the `format` argument, `nw:load-from-string` behaves exactly like the corresponding `nw:load-*` primitive, including matching node and edge attributes to turtle and link variables and honoring `breed`.
1250+
1251+
For example, to load a GML network fetched from a URL:
1252+
1253+
```
1254+
fetch:url-async "https://example.com/network.gml" [ [gml] ->
1255+
nw:load-from-string "gml" gml turtles links
1256+
] [ [err] -> print err ]
1257+
```
1258+
name: load-from-string
1259+
tags:
1260+
- io
1261+
type: command
12241262
- arguments:
12251263
- name: file-name
12261264
type: string

src/main/org/nlogo/extensions/nw/NetworkExtension.scala

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,6 +94,7 @@ class NetworkExtension extends api.DefaultClassManager with GraphContextManager
9494
add("load-graphml", new LoadGraphML())
9595

9696
add("load", new prim.Load())
97+
add("load-from-string", new prim.LoadFromString())
9798
add("load-dl", new LoadFileType(".dl"))
9899
add("load-gdf", new LoadFileType(".gdf"))
99100
add("load-gexf", new LoadFileType(".gexf"))

src/main/org/nlogo/extensions/nw/NetworkExtensionUtil.scala

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,10 +9,19 @@ import org.nlogo.{agent, api, nvm}
99
import org.nlogo.api.MersenneTwisterFast
1010
import scala.language.implicitConversions
1111
import scala.reflect.Selectable.reflectiveSelectable
12+
import java.io.{ ByteArrayInputStream, InputStreamReader, Reader }
13+
import java.nio.charset.StandardCharsets
1214
import java.util.Locale
1315

1416
object NetworkExtensionUtil {
1517

18+
// Wraps an in-memory string in a Reader for feeding to the file importers. We deliberately do NOT use StringReader:
19+
// its `ready()` always returns true, even at EOF, whereas the Gephi importers loop on `reader.ready()` and rely on it
20+
// going false to stop. Backing the reader with a ByteArrayInputStream gives FileReader-like `ready()` semantics
21+
// (false at EOF), so the importers terminate. -Jeremy B July 2026
22+
def readerForString(data: String): Reader =
23+
new InputStreamReader(new ByteArrayInputStream(data.getBytes(StandardCharsets.UTF_8)), StandardCharsets.UTF_8)
24+
1625
implicit def functionToTransformer[I, O](f: Function1[I, O]): org.apache.commons.collections15.Transformer[I,O] =
1726
new org.apache.commons.collections15.Transformer[I, O] {
1827
override def transform(i: I) = f(i)

src/main/org/nlogo/extensions/nw/gephi/GephiImport.scala

Lines changed: 16 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
package org.nlogo.extensions.nw.gephi
22

33
import java.awt.Color
4-
import java.io.{ File, FileReader, IOException }
4+
import java.io.{ File, FileReader, IOException, Reader }
55
import java.lang.{ Boolean => JBoolean, Double => JDouble }
66
import java.util.Collection
77

@@ -48,10 +48,23 @@ object GephiImport {
4848
if (!file.exists) {
4949
throw new ExtensionException("The file " + file + " cannot be found.")
5050
}
51+
load(new FileReader(file), file.toString, world, defaultTurtleBreed, defaultLinkBreed, initTurtles, importer)
52+
}
53+
54+
def loadString(data: String, extension: String, world: World,
55+
defaultTurtleBreed: AgentSet, defaultLinkBreed: AgentSet,
56+
initTurtles: IterableOnce[Turtle] => Unit): Unit = GephiUtils.withNWLoaderContext {
57+
load(readerForString(data), s"the given $extension string",
58+
world, defaultTurtleBreed, defaultLinkBreed, initTurtles, importController.getFileImporter(extension))
59+
}
5160

61+
def load(reader: => Reader, sourceName: String, world: World,
62+
defaultTurtleBreed: AgentSet, defaultLinkBreed: AgentSet,
63+
initTurtles: IterableOnce[Turtle] => Unit,
64+
importer: FileImporter): Unit = GephiUtils.withNWLoaderContext {
5265
val isGdfImport = importer match {
5366
case null =>
54-
throw new ExtensionException("Unable to find importer for " + file)
67+
throw new ExtensionException("Unable to find importer for " + sourceName)
5568

5669
case _: ImporterSpreadsheetCSV =>
5770
throw new ExtensionException("Importing CSV files is not supported.")
@@ -70,7 +83,7 @@ object GephiImport {
7083
val linkBreeds = world.linkBreeds.asScala
7184

7285
val container = try {
73-
using(new FileReader(file))(r => importController.importFile(r, importer))
86+
using(reader)(r => importController.importFile(r, importer))
7487
} catch {
7588
case e: IOException =>
7689
throw new ExtensionException(e)

src/main/org/nlogo/extensions/nw/jung/io/GraphMLImport.scala

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
package org.nlogo.extensions.nw.jung.io
22

3-
import java.io.{BufferedReader, FileReader}
3+
import java.io.{BufferedReader, FileReader, Reader}
44
import java.util.Locale
55

66
import edu.uci.ics.jung
@@ -133,14 +133,18 @@ object GraphMLImport {
133133
elem -> agent
134134
}.toMap
135135

136-
def load(fileName: String, world: World, rng: MersenneTwisterFast): Iterator[Turtle] = {
136+
def load(fileName: String, world: World, rng: MersenneTwisterFast): Iterator[Turtle] =
137+
load(new BufferedReader(new FileReader(fileName)), world, rng, world.turtles, world.links)
138+
139+
def load(reader: => Reader, world: World, rng: MersenneTwisterFast,
140+
defaultTurtleBreed: AgentSet, defaultLinkBreed: AgentSet): Iterator[Turtle] = {
137141
try {
138142
val graphFactory = sparseGraphFactory[Vertex, Edge]
139143
val graphTransformer = transformer { (_: GraphMetadata) => graphFactory.create }
140144

141145
using {
142146
new GraphMLReader2[jung.graph.Graph[Vertex, Edge], Vertex, Edge](
143-
new BufferedReader(new FileReader(fileName)),
147+
reader,
144148
graphTransformer,
145149
transformer(Vertex.apply),
146150
transformer(Edge.apply),
@@ -160,11 +164,11 @@ object GraphMLImport {
160164
val vertices = graph.getVertices.asScala.toList.sortBy(_.id)
161165

162166
val turtles: Map[Vertex, Turtle] =
163-
createAgents(vertices, keyMap(MetadataType.NODE), world.turtles, world.getBreed) {
167+
createAgents(vertices, keyMap(MetadataType.NODE), defaultTurtleBreed, world.getBreed) {
164168
(_, breed) => createTurtle(world, breed, rng)
165169
}
166170

167-
createAgents(graph.getEdges.asScala, keyMap(MetadataType.EDGE), world.links, world.getLinkBreed) {
171+
createAgents(graph.getEdges.asScala, keyMap(MetadataType.EDGE), defaultLinkBreed, world.getLinkBreed) {
168172
(e: Edge, breed) =>
169173
createLink(turtles, graph.getEndpoints(e), graph.getDefaultEdgeType == EdgeType.DIRECTED, breed, world)
170174
}

src/main/org/nlogo/extensions/nw/jung/io/Matrix.scala

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
package org.nlogo.extensions.nw.jung.io
44

5-
import java.io.FileNotFoundException
5+
import java.io.{ BufferedReader, FileNotFoundException, Reader }
66

77
import org.nlogo.agent.AgentSet
88
import org.nlogo.agent.Link
@@ -40,12 +40,24 @@ object Matrix {
4040
}
4141
}
4242

43-
def load(filename: String, turtleBreed: AgentSet, linkBreed: AgentSet, world: World, rng: MersenneTwisterFast) = {
43+
def load(filename: String, turtleBreed: AgentSet, linkBreed: AgentSet, world: World, rng: MersenneTwisterFast): Iterator[Turtle] =
44+
load(matrixFile => matrixFile.load(filename), turtleBreed, linkBreed, world, rng)
45+
46+
def load(reader: Reader, turtleBreed: AgentSet, linkBreed: AgentSet, world: World, rng: MersenneTwisterFast): Iterator[Turtle] = {
47+
val buffered = reader match {
48+
case b: BufferedReader => b
49+
case r => new BufferedReader(r)
50+
}
51+
load(matrixFile => matrixFile.load(buffered), turtleBreed, linkBreed, world, rng)
52+
}
53+
54+
private def load(readGraph: jung.io.MatrixFile[DummyGraph.Vertex, DummyGraph.Edge] => jung.graph.Graph[DummyGraph.Vertex, DummyGraph.Edge],
55+
turtleBreed: AgentSet, linkBreed: AgentSet, world: World, rng: MersenneTwisterFast): Iterator[Turtle] = {
4456
val matrixFile = new jung.io.MatrixFile(
4557
null, // TODO: provide weight key (null means 1) (issue #19)
4658
factoryFor(linkBreed), DummyGraph.vertexFactory, DummyGraph.edgeFactory)
4759
val graph = try {
48-
matrixFile.load(filename)
60+
readGraph(matrixFile)
4961
} catch {
5062
case e: Exception => e.getCause match {
5163
case fileNotFound: FileNotFoundException =>

src/main/org/nlogo/extensions/nw/prim/IO.scala

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
package org.nlogo.extensions.nw.prim
22

33
import java.io.File
4+
import java.util.Locale
45

56
import org.nlogo.agent.World
67
import org.nlogo.api
@@ -10,6 +11,7 @@ import org.nlogo.nvm.ExtensionContext
1011
import org.nlogo.extensions.nw.GraphContextProvider
1112
import org.nlogo.extensions.nw.NetworkExtensionUtil._
1213
import org.nlogo.extensions.nw.gephi.{ GephiExport, GephiImport, GephiUtils }
14+
import org.nlogo.extensions.nw.jung.io.{ GraphMLImport, Matrix }
1315

1416
class Load extends TurtleAskingCommand {
1517
override def getSyntax = commandSyntax(
@@ -25,6 +27,33 @@ class Load extends TurtleAskingCommand {
2527
}
2628
}
2729

30+
class LoadFromString extends TurtleAskingCommand {
31+
override def getSyntax = commandSyntax(
32+
right = List(StringType, StringType, TurtlesetType, LinksetType, CommandBlockType | OptionalType),
33+
blockAgentClassString = Some("-T--"))
34+
override def perform(args: Array[api.Argument], context: api.Context) = GephiUtils.withNWLoaderContext {
35+
implicit val world = context.world.asInstanceOf[World]
36+
val rawFormat = args(0).getString
37+
val format = rawFormat.trim.toLowerCase(Locale.ENGLISH).stripPrefix(".")
38+
val data = args(1).getString
39+
val turtleBreed = args(2).getAgentSet.requireTurtleBreed
40+
val linkBreed = args(3).getAgentSet.requireLinkBreed
41+
val rng = context.getRNG
42+
format match {
43+
case "graphml" =>
44+
askTurtles(context)(GraphMLImport.load(readerForString(data), world, rng, turtleBreed, linkBreed))
45+
case "matrix" =>
46+
askTurtles(context)(Matrix.load(readerForString(data), turtleBreed, linkBreed, world, rng))
47+
case "dl" | "gdf" | "gexf" | "gml" | "vna" =>
48+
GephiImport.loadString(data, "." + format, world, turtleBreed, linkBreed, askTurtles(context))
49+
case _ =>
50+
throw new api.ExtensionException(
51+
s"'$rawFormat' is not a supported network format. Valid formats are: " +
52+
"dl, gdf, gexf, gml, graphml, matrix, and vna.")
53+
}
54+
}
55+
}
56+
2857
class LoadFileType(extension: String) extends TurtleAskingCommand {
2958
override def getSyntax = commandSyntax(right = List(StringType, TurtlesetType, LinksetType, CommandBlockType | OptionalType), blockAgentClassString = Some("-T--"))
3059
override def perform(args: Array[api.Argument], context: api.Context) = GephiUtils.withNWLoaderContext {

0 commit comments

Comments
 (0)