[DF] support for fastutil::evalText

README.md

@@ -45,7 +45,7 @@ It offers a wide variety of features, for example:
        ╰ Absolute Paths (absolute-file-paths):
            ╰ certain:
                ╰ Path `/root/x.txt` at 1.1-23
-           ╰ Metadata: totalConsidered: 1, totalUnknown: 0, searchTimeMs: 1, processTimeMs: 0
+           ╰ Metadata: totalConsidered: 1, totalUnknown: 0, searchTimeMs: 0, processTimeMs: 1
        ╰ Unused Definitions (unused-definitions):
            ╰ Metadata: totalConsidered: 0, searchTimeMs: 0, processTimeMs: 0
        ╰ Naming Convention (naming-convention):
@@ -86,7 +86,7 @@ It offers a wide variety of features, for example:
 
     _Results (prettified and summarized):_
 
-    Query: **linter** (2 ms)\
+    Query: **linter** (1 ms)\
        ╰ **Deprecated Functions** (deprecated-functions):\
     &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;╰ _Metadata_: <code>totalCalls: 0, totalFunctionDefinitions: 0, searchTimeMs: 0, processTimeMs: 0</code>\
     &nbsp;&nbsp;&nbsp;╰ **File Path Validity** (file-path-validity):\
@@ -98,7 +98,7 @@ It offers a wide variety of features, for example:
     &nbsp;&nbsp;&nbsp;╰ **Absolute Paths** (absolute-file-paths):\
     &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;╰ certain:\
     &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;╰ Path `/root/x.txt` at 1.1-23\
-    &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;╰ _Metadata_: <code>totalConsidered: 1, totalUnknown: 0, searchTimeMs: 0, processTimeMs: 0</code>\
+    &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;╰ _Metadata_: <code>totalConsidered: 1, totalUnknown: 0, searchTimeMs: 0, processTimeMs: 1</code>\
     &nbsp;&nbsp;&nbsp;╰ **Unused Definitions** (unused-definitions):\
     &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;╰ _Metadata_: <code>totalConsidered: 0, searchTimeMs: 0, processTimeMs: 0</code>\
     &nbsp;&nbsp;&nbsp;╰ **Naming Convention** (naming-convention):\
@@ -116,12 +116,12 @@ It offers a wide variety of features, for example:
     &nbsp;&nbsp;&nbsp;╰ **Stop without call.=False argument** (stop-call):\
     &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;╰ _Metadata_: <code>consideredNodes: 0, searchTimeMs: 0, processTimeMs: 0</code>\
     &nbsp;&nbsp;&nbsp;╰ **Roxygen Arguments** (roxygen-arguments):\
-    &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;╰ _Metadata_: <code>searchTimeMs: 0, processTimeMs: 1</code>\
-    _All queries together required ≈2 ms (1ms accuracy, total 2 ms)_
+    &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;╰ _Metadata_: <code>searchTimeMs: 0, processTimeMs: 0</code>\
+    _All queries together required ≈1 ms (1ms accuracy, total 1 ms)_
 
     <details> <summary style="color:gray">Show Detailed Results as Json</summary>
 
-    The analysis required _2.1 ms_ (including parsing and normalization and the query) within the generation environment.
+    The analysis required _1.4 ms_ (including parsing and normalization and the query) within the generation environment.
 
     In general, the JSON contains the Ids of the nodes in question as they are present in the normalized AST or the dataflow graph of flowR.
     Please consult the [Interface](https://github.com/flowr-analysis/flowr/wiki/Interface) wiki page for more information on how to get those.
@@ -194,7 +194,7 @@ It offers a wide variety of features, for example:
               "totalConsidered": 1,
               "totalUnknown": 0,
               "searchTimeMs": 0,
-              "processTimeMs": 0
+              "processTimeMs": 1
             }
           },
           "unused-definitions": {
@@ -268,16 +268,16 @@ It offers a wide variety of features, for example:
             "results": [],
             ".meta": {
               "searchTimeMs": 0,
-              "processTimeMs": 1
+              "processTimeMs": 0
             }
           }
         },
         ".meta": {
-          "timing": 2
+          "timing": 1
         }
       },
       ".meta": {
-        "timing": 2
+        "timing": 1
       }
     }
     ```
@@ -356,7 +356,7 @@ It offers a wide variety of features, for example:
     N <- 10
     for(i in 1:(N-1)) sum <- sum + i + w
     sum
-    All queries together required ≈2 ms (1ms accuracy, total 2 ms)
+    All queries together required ≈1 ms (1ms accuracy, total 2 ms)
     ```
 
 
@@ -436,6 +436,7 @@ It offers a wide variety of features, for example:
 
     ```text
     https://mermaid.live/view#base64:eyJjb2RlIjoiZmxvd2NoYXJ0IEJUXG4gICAgMChbXCJgIzkxO1JTeW1ib2wjOTM7IHRlc3RcbiAgICAgICgwKVxuICAgICAgKjEuMS00KmBcIl0pXG4gICAlJSBObyBlZGdlcyBmb3VuZCBmb3IgMFxuICAgIDEoW1wiYCM5MTtSU3ltYm9sIzkzOyB0ZXN0ZmlsZXNcbiAgICAgICgxKVxuICAgICAgKjEuNi0xNCpgXCJdKVxuICAgJSUgTm8gZWRnZXMgZm91bmQgZm9yIDFcbiAgICAyW1tcImAjOTE7UkJpbmFyeU9wIzkzOyAvXG4gICAgICAoMilcbiAgICAgICoxLjEtMTQqXG4gICAgKDAsIDEpYFwiXV1cbiAgICBidWlsdC1pbjpfW1wiYEJ1aWx0LUluOlxuL2BcIl1cbiAgICBzdHlsZSBidWlsdC1pbjpfIHN0cm9rZTpncmF5LGZpbGw6Z3JheSxzdHJva2Utd2lkdGg6MnB4LG9wYWNpdHk6Ljg7XG4gICAgMyhbXCJgIzkxO1JTeW1ib2wjOTM7IGV4YW1wbGUuUlxuICAgICAgKDMpXG4gICAgICAqMS4xNi0yNCpgXCJdKVxuICAgJSUgTm8gZWRnZXMgZm91bmQgZm9yIDNcbiAgICA0W1tcImAjOTE7UkJpbmFyeU9wIzkzOyAvXG4gICAgICAoNClcbiAgICAgICoxLjEtMjQqXG4gICAgKDIsIDMpYFwiXV1cbiAgICAyIC0tPnxcInJlYWRzLCBhcmd1bWVudFwifCAwXG4gICAgMiAtLT58XCJyZWFkcywgYXJndW1lbnRcInwgMVxuICAgIDIgLS4tPnxcInJlYWRzLCBjYWxsc1wifCBidWlsdC1pbjpfXG4gICAgbGlua1N0eWxlIDIgc3Ryb2tlOmdyYXk7XG4gICAgNCAtLT58XCJyZWFkcywgYXJndW1lbnRcInwgMlxuICAgIDQgLS0+fFwicmVhZHMsIGFyZ3VtZW50XCJ8IDNcbiAgICA0IC0uLT58XCJyZWFkcywgY2FsbHNcInwgYnVpbHQtaW46X1xuICAgIGxpbmtTdHlsZSA1IHN0cm9rZTpncmF5OyIsIm1lcm1haWQiOnsiYXV0b1N5bmMiOnRydWV9fQ==
+    Copied mermaid url to clipboard (dataflow: 0ms).
     ```
 
 
@@ -734,7 +735,7 @@ It offers a wide variety of features, for example:
     ```
 
 
-    (The analysis required _1.4 ms_ (including parse and normalize, using the [tree-sitter](https://github.com/flowr-analysis/flowr/wiki/Engines) engine) within the generation environment.)
+    (The analysis required _1.0 ms_ (including parse and normalize, using the [tree-sitter](https://github.com/flowr-analysis/flowr/wiki/Engines) engine) within the generation environment.)
 
 
 

src/dataflow/environments/default-builtin-config.ts

@@ -235,7 +235,8 @@ export const DefaultBuiltinConfig = [
 				'.Fortran':  ['.NAME']
 			}
 		},                            assumePrimitive: true },
-	{ type: 'function', names: ['eval'],                                       processor: BuiltInProcName.Eval,                config: { includeFunctionCall: true },                                                 assumePrimitive: true },
+	{ type: 'function', names: ['eval'],                                       processor: BuiltInProcName.Eval,                config: { includeFunctionCall: true, supportFunctionCall: false },                                                 assumePrimitive: true },
+	{ type: 'function', names: ['evalText'],                                   processor: BuiltInProcName.Eval,                config: { includeFunctionCall: true, supportFunctionCall: true },                                                 assumePrimitive: true },
 	{ type: 'function', names: ['cat'],                                        processor: BuiltInProcName.Default,             config: { forceArgs: 'all', hasUnknownSideEffects: { type: 'link-to-last-call', callName: /^sink$/ } },                                                         assumePrimitive: false },
 	{ type: 'function', names: ['switch'],                                     processor: BuiltInProcName.Default,             config: { forceArgs: [true] },                                                         assumePrimitive: false },
 	{ type: 'function', names: ['return'],                                     processor: BuiltInProcName.Default,             config: { returnsNthArgument: 0, cfg: ExitPointType.Return, useAsProcessor: BuiltInProcName.Return }, assumePrimitive: true },

src/dataflow/internal/process/functions/call/built-in/built-in-eval.ts

@@ -45,6 +45,8 @@ export function processEvalCall<OtherInfo>(
 	config: {
 		/** should this produce an explicit source function call in the graph? */
 		includeFunctionCall?: boolean
+		/** if selected processes evalText function call, else processes eval*/
+		supportFunctionCall?: boolean
 	}
 ): DataflowInformation {
 	if(args.length !== 1 || args[0] === EmptyArgument || !args[0].value) {
@@ -71,7 +73,7 @@ export function processEvalCall<OtherInfo>(
 		return information;
 	}
 
-	const code: string[] | undefined = resolveEvalToCode(evalArgument.value as RNode<ParentInformation>, data.environment, data.completeAst.idMap, data.ctx);
+	const code: string[] | undefined = resolveEvalToCode(evalArgument.value as RNode<never>, config, data);
 
 	if(code) {
 		const idGenerator = sourcedDeterministicCountingIdGenerator(name.lexeme + '::' + rootId, name.location);
@@ -109,37 +111,53 @@ export function processEvalCall<OtherInfo>(
 	return information;
 }
 
-function resolveEvalToCode<OtherInfo>(evalArgument: RNode<OtherInfo & ParentInformation>, env: REnvironmentInformation, idMap: AstIdMap, ctx: ReadOnlyFlowrAnalyzerContext): string[] | undefined {
+function resolveEvalToCode<OtherInfo>(evalArgument: RNode<OtherInfo & ParentInformation>, config: { includeFunctionCall?: boolean, supportFunctionCall?: boolean }, data: DataflowProcessorInformation<OtherInfo & ParentInformation>): string[] | undefined {
+	const ctx = data.ctx;
+	const env = data.environment;
+	const idMap = data.completeAst.idMap;
 	const val = evalArgument;
 
-	if(
-		val.type === RType.FunctionCall && val.named && val.functionName.content === 'parse'
-	) {
-		const arg = val.arguments.find(v => v !== EmptyArgument && v.name?.content === 'text');
-		const nArg = val.arguments.find(v => v !== EmptyArgument && v.name?.content === 'n');
-		if(nArg !== undefined || arg === undefined || arg === EmptyArgument) {
-			return undefined;
-		}
-		if(arg.value?.type === RType.String) {
-			return [arg.value.content.str];
-		} else if(arg.value?.type === RType.Symbol) {
-			const resolved = valueSetGuard(resolveIdToValue(arg.value.info.id, { environment: env, idMap: idMap, resolve: ctx.config.solver.variables, ctx }));
+	if(config.supportFunctionCall){
+		if(val.type === RType.String){
+			return [val.content.str];
+		} else if(val.type === RType.Symbol){
+			const resolved = valueSetGuard(resolveIdToValue(val.info.id, { environment: env, idMap: idMap, resolve: ctx.config.solver.variables, ctx }));
 			if(resolved) {
 				return collectStrings(resolved.elements);
 			}
-		} else if(arg.value?.type === RType.FunctionCall && arg.value.named && ['paste', 'paste0'].includes(Identifier.getName(arg.value.functionName.content))) {
-			return handlePaste(ctx.config, arg.value.arguments, env, idMap, arg.value.functionName.content === 'paste' ? [' '] : [''], ctx);
 		}
 		return undefined;
-	} else if(val.type === RType.Symbol) {
-		// const resolved = resolveValueOfVariable(val.content, env);
-		// see https://github.com/flowr-analysis/flowr/pull/1467
-		return undefined;
 	} else {
-		return undefined;
+		if(
+			val.type === RType.FunctionCall && val.named && val.functionName.content === 'parse'
+		) {
+			const arg = val.arguments.find(v => v !== EmptyArgument && v.name?.content === 'text');
+			const nArg = val.arguments.find(v => v !== EmptyArgument && v.name?.content === 'n');
+			if(nArg !== undefined || arg === undefined || arg === EmptyArgument) {
+				return undefined;
+			}
+			if(arg.value?.type === RType.String) {
+				return [arg.value.content.str];
+			} else if(arg.value?.type === RType.Symbol) {
+				const resolved = valueSetGuard(resolveIdToValue(arg.value.info.id, { environment: env, idMap: idMap, resolve: ctx.config.solver.variables, ctx }));
+				if(resolved) {
+					return collectStrings(resolved.elements);
+				}
+			} else if(arg.value?.type === RType.FunctionCall && arg.value.named && ['paste', 'paste0'].includes(Identifier.getName(arg.value.functionName.content))) {
+				return handlePaste(ctx.config, arg.value.arguments, env, idMap, arg.value.functionName.content === 'paste' ? [' '] : [''], ctx);
+			}
+			return undefined;
+		} else if(val.type === RType.Symbol) {
+			// const resolved = resolveValueOfVariable(val.content, env);
+			// see https://github.com/flowr-analysis/flowr/pull/1467
+			return undefined;
+		} else {
+			return undefined;
+		}
 	}
 }
 
+
 function getAsString(config: FlowrConfig, val: RNode<ParentInformation> | undefined, env: REnvironmentInformation, idMap: AstIdMap, ctx: ReadOnlyFlowrAnalyzerContext): string[] | undefined {
 	if(!val) {
 		return undefined;
@@ -184,4 +202,4 @@ function handlePaste(config: FlowrConfig, args: readonly PotentiallyEmptyRArgume
 	}
 
 	return result;
-}
+}

test/functionality/dataflow/functions/eval.test.ts

@@ -0,0 +1,63 @@
+import { describe } from 'vitest';
+import { emptyGraph } from '../../../../src/dataflow/graph/dataflowgraph-builder';
+import { EdgeType } from '../../../../src/dataflow/graph/edge';
+import { OperatorDatabase } from '../../../../src/r-bridge/lang-4.x/ast/model/operators';
+import { label } from '../../_helper/label';
+import { withTreeSitter, assertDataflow } from '../../_helper/shell';
+
+describe.sequential('eval', withTreeSitter(tr => {
+	assertDataflow(label('simple eval use', ['name-normal', ...OperatorDatabase['<-'].capabilities, 'numbers', 'unnamed-arguments', 'strings', 'built-in-evaluation', 'newlines']),
+		tr, 'a <- "1+1"\nx <- "1"\nb <- "3"\nz <- eval(parse(text=x))', emptyGraph()
+			.defineVariable('2@x')
+			.defineVariable('4@z')
+			.definedBy('4@z', '4@eval')
+			.addEdge('4@eval', '4@parse', EdgeType.Argument | EdgeType.Reads | EdgeType.Returns)
+			.addEdge('4@parse', '4@x', EdgeType.Reads)
+			.addEdge('4@x', '2@x', EdgeType.Reads),
+		{
+			expectIsSubgraph:      true,
+			resolveIdsAsCriterion: true,
+			context:               'dataflow'
+		});
+	assertDataflow(label('simple eval use - from 2 variables', ['name-normal', ...OperatorDatabase['<-'].capabilities, 'numbers', 'unnamed-arguments', 'strings', 'built-in-evaluation', 'newlines']),
+		tr, 'x <- 1\ny <- 1\na <- 2\nz <- eval(parse(text="x+y"))', emptyGraph()
+			.definedBy('4@z', '4@eval')
+			.addEdge(17, 'eval::17-4:6-4:9-2', EdgeType.Returns)
+			.addEdge('eval::17-4:6-4:9-2', 'eval::17-4:6-4:9-0', EdgeType.Reads | EdgeType.Argument)
+			.addEdge('eval::17-4:6-4:9-2', 'eval::17-4:6-4:9-1', EdgeType.Reads | EdgeType.Argument)
+			.addEdge('eval::17-4:6-4:9-0', '1@x', EdgeType.Reads)
+			.addEdge('eval::17-4:6-4:9-1', '2@y', EdgeType.Reads),
+		{
+			expectIsSubgraph:      true,
+			resolveIdsAsCriterion: true,
+			context:               'dataflow'
+		});
+}));
+
+describe.sequential('evalText', withTreeSitter(tr => {
+	assertDataflow(label('simple evalText use', ['name-normal', ...OperatorDatabase['<-'].capabilities, 'numbers', 'unnamed-arguments', 'strings', 'built-in-evaluation', 'newlines']),
+		tr, 'a <- "1+1"\nx <- "1"\nb <- "3"\nz <- evalText(x)', emptyGraph()
+			.defineVariable('2@x')
+			.defineVariable('4@z')
+			.definedBy('4@z', '4@evalText')
+			.addEdge('4@evalText', '4@x', EdgeType.Argument | EdgeType.Reads | EdgeType.Returns)
+			.addEdge('4@x', '2@x', EdgeType.Reads),
+		{
+			expectIsSubgraph:      true,
+			resolveIdsAsCriterion: true,
+			context:               'dataflow'
+		});
+	assertDataflow(label('simple evalText use  - from 2 variables', ['name-normal', ...OperatorDatabase['<-'].capabilities, 'numbers', 'unnamed-arguments', 'strings', 'built-in-evaluation', 'newlines']),
+		tr, 'x <- 1\ny <- 1\na <- 2\nz <- evalText("x+y")', emptyGraph()
+			.definedBy('4@z', '4@evalText')
+			.addEdge(13, 'evalText::13-4:6-4:13-2', EdgeType.Returns)
+			.addEdge('evalText::13-4:6-4:13-2', 'evalText::13-4:6-4:13-0', EdgeType.Reads | EdgeType.Argument)
+			.addEdge('evalText::13-4:6-4:13-2', 'evalText::13-4:6-4:13-1', EdgeType.Reads | EdgeType.Argument)
+			.addEdge('evalText::13-4:6-4:13-0', '1@x', EdgeType.Reads)
+			.addEdge('evalText::13-4:6-4:13-1', '2@y', EdgeType.Reads),
+		{
+			expectIsSubgraph:      true,
+			resolveIdsAsCriterion: true,
+			context:               'dataflow'
+		});
+}));

test/functionality/slicing/backward/static-backward-program-slices/eval.test.ts

@@ -31,3 +31,40 @@ print(y)`);
 print(y)`, ['2@print'], `for(v in c("x", "y")) eval(parse(text=paste(k, "<- 2")))
 print(y)`);
 }));
+
+describe.sequential('evalText', withShell(shell => {
+	assertSliced(label('simple evalText use', ['name-normal', ...OperatorDatabase['<-'].capabilities, 'numbers', 'unnamed-arguments', 'strings', 'built-in-evaluation', 'newlines']),
+		shell, 'x <- 1\ny <- 1\nz <- evalText("x+y")', ['3@z'], 'x <- 1\ny <- 1\nz <- evalText("x+y")');
+	assertSliced(label('simple evalText use', ['name-normal', ...OperatorDatabase['<-'].capabilities, 'numbers', 'unnamed-arguments', 'strings', 'built-in-evaluation', 'newlines']),
+		shell, 'foo <- function(x) { return (x+2) }\ny <- evalText("foo(4)*1")', ['2@y'], 'foo <- function(x) return (x+2)\ny <- evalText("foo(4)*1")');
+	assertSliced(label('simple evalText remote string', ['name-normal', ...OperatorDatabase['<-'].capabilities, 'numbers', 'unnamed-arguments', 'strings', 'built-in-evaluation', 'newlines']),
+		shell, 'x <- 2\nt <- "print(x)"\ny <- evalText(t)', ['3@y'], 'x <- 2\nt <- "print(x)"\ny <- evalText(t)');
+	assertSliced(label('indirect evalText call', ['name-normal', ...OperatorDatabase['<-'].capabilities, 'numbers', 'unnamed-arguments', 'strings', 'built-in-evaluation', 'newlines']),
+		shell, 'y <- 1\nfoo <- function(x) { return (evalText("x+y")) }\nz <- foo(4)', ['3@z'], 'y <- 1\nfoo <- function(x) return (evalText("x+y"))\nz <- foo(4)');
+	assertSliced(label('double indirect evalText call', ['name-normal', ...OperatorDatabase['<-'].capabilities, 'numbers', 'unnamed-arguments', 'strings', 'built-in-evaluation', 'newlines']),
+		shell, 'y <- 1\nfoo <- function(x) { return (evalText("x+y")) }\nt <- 3\nz <- evalText("foo(4)*2+t")', ['4@z'], 'y <- 1\nfoo <- function(x) return (evalText("x+y"))\nt <- 3\nz <- evalText("foo(4)*2+t")');
+	assertSliced(label('mixed evalText/eval call', ['name-normal', ...OperatorDatabase['<-'].capabilities, 'numbers', 'unnamed-arguments', 'strings', 'built-in-evaluation', 'newlines']),
+		shell, 'y <- 1\nfoo <- function(x) { return (eval(parse(text ="x+y"))) }\nt <- 3\nz <- evalText("foo(4)*2+t")', ['4@z'], 'y <- 1\nfoo <- function(x) return (eval(parse(text ="x+y")))\nt <- 3\nz <- evalText("foo(4)*2+t")');
+	assertSliced(label('simple evalText conditional string', ['name-normal', ...OperatorDatabase['<-'].capabilities, 'numbers', 'unnamed-arguments', 'strings', 'built-in-evaluation', 'newlines']),
+		shell, `k <- 42
+x <- 2
+if(u) t <- "print(x)" else 
+t <- "print(k)"
+y <- evalText(t)`, ['5@y'], `k <- 42
+x <- 2
+if(u) t <- "print(x)" else
+t <- "print(k)"
+y <- evalText(t)`);
+	assertSliced(label('evalText should resolve back', ['name-normal', ...OperatorDatabase['<-'].capabilities, 'numbers', 'unnamed-arguments', 'strings', 'built-in-evaluation', 'newlines']),
+		shell, `x <- 1
+y <- 0
+for(i in 1:4){
+	y <- 4
+}
+if(u >= 4) y <- 2
+t <- evalText("1+y")
+print(y)`, ['8@print'], `y <- 0
+for(i in 1:4) y <- 4
+if(u >= 4) y <- 2
+print(y)`);
+}));