We have a very advanced Iterable<T>.toDataFrame {} API that can be quite difficult to understand. Especially because the docs need improvement: #1479
We support changing properties by name, type, or just in general:
list.toDataFrame {
properties(A::prop1, A::prop2) {
exclude(B::prop3)
/* subgraph settings go here */
}
properties {
preserve(SomeClass::class)
}
preserve(SomeOther::class)
}
now imagine my surprise when
toDataFrame {
preserve(DataFrame::class)
}
returned an empty dataframe, but this does not:
toDataFrame {
preserve(DataFrame::class)
properties()
}
Apparently you can write multiple properties instances and top-level preserve() and exclude() calls apply to all properties {} next to them.
If I remember correctly, the compiler plugin also does not support this notation, so let's change or remove it.
Let's keep the top-level of toDataFrame dedicated to creating columns and move functions related to the TraversePropertiesDsl into properties {}. Isn't it already the case that properties(maxDepth = Int.MAX_VALUE) {} applies to every property at any depth? That already opens up a lot of possibilities. If you really want to apply the same logic to each subgraph, you could always just repeat the statement or create a variable.
toDataFrame {
"myCol" from { it.a }
properties(maxDepth = Int.MAX_VALUE) {
preserve(DataFrame::class)
}
properties(A::prop1, A::prop2) {
exclude(B::prop3)
}
}
We have a very advanced
Iterable<T>.toDataFrame {}API that can be quite difficult to understand. Especially because the docs need improvement: #1479We support changing properties by name, type, or just in general:
list.toDataFrame { properties(A::prop1, A::prop2) { exclude(B::prop3) /* subgraph settings go here */ } properties { preserve(SomeClass::class) } preserve(SomeOther::class) }now imagine my surprise when
toDataFrame { preserve(DataFrame::class) }returned an empty dataframe, but this does not:
toDataFrame { preserve(DataFrame::class) properties() }Apparently you can write multiple
propertiesinstances and top-levelpreserve()andexclude()calls apply to allproperties {}next to them.If I remember correctly, the compiler plugin also does not support this notation, so let's change or remove it.
Let's keep the top-level of
toDataFramededicated to creating columns and move functions related to theTraversePropertiesDslintoproperties {}. Isn't it already the case thatproperties(maxDepth = Int.MAX_VALUE) {}applies to every property at any depth? That already opens up a lot of possibilities. If you really want to apply the same logic to each subgraph, you could always just repeat the statement or create a variable.toDataFrame { "myCol" from { it.a } properties(maxDepth = Int.MAX_VALUE) { preserve(DataFrame::class) } properties(A::prop1, A::prop2) { exclude(B::prop3) } }