Skip to content

Commit 4076f57

Browse files
m-kesselm-kessel
committed
added generation services
1 parent 9d36c80 commit 4076f57

88 files changed

Lines changed: 491 additions & 286 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎lasso/docusaurus.config.ts‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,7 @@ const config: Config = {
7272
},
7373
{to: '/hub', label: 'TDSEHub (Pipelines)', position: 'left'},
7474
{to: '/search', label: 'Code Search', position: 'left'},
75+
{to: '/generation', label: 'Code Generation', position: 'left'},
7576
{to: '/labs', label: 'Labs (Playground)', position: 'left'},
7677
{to: '/blog', label: 'Blog', position: 'left'},
7778
{to: '/research/intro', label: 'Research', position: 'left'},
@@ -121,6 +122,10 @@ const config: Config = {
121122
label: 'Code Search',
122123
href: '/search',
123124
},
125+
{
126+
label: 'Code Generation',
127+
href: '/generation',
128+
},
124129
// {
125130
// label: 'Discord',
126131
// href: 'https://discordapp.com/invite/docusaurus',

‎lasso/src/pages/generation.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
# Code Generation Services
2+
3+
## LLM-based Code and Test Generation: Generating Code by Test Specifications with LASSO's Sequence Sheet Notation (SSN)
4+
5+
LASSO's Sequence Sheet Notation (SSN) allows for a unique form of code generation based on test specifications. This *test-driven code generation* approach enables you to pinpoint code that produces the expected behavior defined by your test specifications, represented as stimulus sheets in SSN. This is extremely helpful for verifying functional correctness and ensuring code meets established requirements.
6+
7+
**Learn More & Get Started:**
8+
9+
* [**Try the Test-Driven Code Generation**](lasso/ssn?recommendation=gen) - Begin generating code based on your SSN test specifications.
10+
* [**SSN Documentation**](docs/datastructures/ssn) - Understand the structure and capabilities of SSN for defining test specifications.
11+
12+
### Examples
13+
14+
(under construction)
15+
16+
* [**Base64 encoding**](lasso/ssn?recommendation=gen&example=BASE64)

‎lasso/src/pages/labs.md‎

Lines changed: 12 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -4,15 +4,22 @@
44

55
A public demo LASSO instance (i.e., playground) is available. To try LASSO, head over to [TDSEHub](./hub), select an example LSL pipeline script and press the `Try Now!` button.
66

7-
### Generative AI (LLM) - Code and Test Generation
8-
9-
For example, run your first code/test generation workflow with OpenAI models [here](lasso/submit?exampleId=OPENAI_GEN).
10-
117
## Code Search Services
128

139
The platform offers several code search services, including *interface-driven code search* as well as *test-driven code search*.
1410

15-
Visit the [Search](search) page for details.
11+
**Learn More & Get Started:**
12+
13+
* [**Visit the Code Search Page**](search) for more details
14+
15+
## Code and Test Generation with Generative AI
16+
17+
The platform offers several code and test generation services, supported by the platform's sequence sheet notation and LASSO's query language.
18+
19+
**Learn More & Get Started:**
20+
21+
* [**Visit the Code Generation Page**](generation) for more details
22+
* [**TDSEHub**](lasso/submit?exampleId=OPENAI_GEN) - view example code/test generation LSL pipeline workflows with OpenAI models
1623

1724
## LSLFlow - A Visual Graph Editor for LSL Pipelines
1825

‎lasso/src/pages/lasso/result.tsx‎

Lines changed: 58 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
import { Accordion, AccordionSummary, AccordionDetails, Box, Button, CardActions, CardContent, CircularProgress, Link, Tab, Tabs, Typography, List, ListItem, ListItemIcon, ListItemText, Divider, FormControl, InputLabel, Select, MenuItem, SelectChangeEvent } from '@mui/material';
1+
import { Accordion, AccordionSummary, AccordionDetails, Box, Button, CardActions, CardContent, CircularProgress, Link, Tab, Tabs, Typography, List, ListItem, ListItemIcon, ListItemText, Divider, FormControl, InputLabel, Select, MenuItem, SelectChangeEvent, ListItemButton } from '@mui/material';
22
import Grid from '@mui/material/Grid2';
33
import React, { useEffect, useRef, useState } from 'react';
44

@@ -22,6 +22,8 @@ import AuthService from '@site/src/services/AuthService';
2222
import { Editor } from '@monaco-editor/react';
2323
import GraphComponent from '@site/src/components/Graph/graph';
2424

25+
import ChevronRightIcon from '@mui/icons-material/ChevronRight';
26+
2527
interface TabPanelProps {
2628
children?: React.ReactNode;
2729
index: number;
@@ -337,13 +339,13 @@ DRAFT
337339

338340
<Typography variant="body2" color="text.secondary">
339341
Learn more about <b>this code module</b>: <Link
340-
href={`/web/lasso/search?query=*:*&filter=id:${codeUnit.id}&ds=${codeUnit.dataSource}`}
341-
target="_blank"
342-
rel="noopener"
343-
underline="hover"
344-
>
345-
Details
346-
</Link>
342+
href={`/web/lasso/search?query=*:*&filter=id:${codeUnit.id}&ds=${codeUnit.dataSource}`}
343+
target="_blank"
344+
rel="noopener"
345+
underline="hover"
346+
>
347+
Details
348+
</Link>
347349
</Typography>
348350

349351
</React.Fragment>
@@ -417,8 +419,54 @@ DRAFT
417419
<React.Fragment>
418420
<Typography variant="h6" component="div">Analyze SRM Data in Jupyter Lite (WASM powered Juyper running in the browser!)</Typography>
419421
<Typography component="div">(note: you can also download the Notebook and run it in your local Juypter environment)</Typography>
420-
<br />
421-
<p><Link target="_blank" href={`${LassoService.API_URL}notebooks/lab/index.html?fromURL=${LassoService.API_URL}publicapi/v1/lasso/analytics/srm/${scriptInfo.executionId}.ipynb`}>Open Notebook</Link></p>
422+
{/* <br />
423+
<p><Link target="_blank" href={`${LassoService.API_URL}notebooks/lab/index.html?fromURL=${LassoService.API_URL}publicapi/v1/lasso/analytics/srm/${scriptInfo.executionId}.ipynb`}>Open Notebook</Link></p> */}
424+
425+
426+
<List>
427+
428+
<ListItem disablePadding>
429+
<ListItemButton
430+
component={Link} href={`${LassoService.API_URL}notebooks/lab/index.html?fromURL=${LassoService.API_URL}publicapi/v1/lasso/analytics/srm/value/${scriptInfo.executionId}.ipynb`}
431+
target="_blank"
432+
rel="noopener"
433+
underline="none"
434+
>
435+
<ListItemIcon>
436+
<ChevronRightIcon />
437+
</ListItemIcon>
438+
<ListItemText primary="Open SRM with output observations only" />
439+
</ListItemButton>
440+
</ListItem>
441+
<ListItem disablePadding>
442+
<ListItemButton
443+
component={Link} href={`${LassoService.API_URL}notebooks/lab/index.html?fromURL=${LassoService.API_URL}publicapi/v1/lasso/analytics/srm/${scriptInfo.executionId}.ipynb`}
444+
target="_blank"
445+
rel="noopener"
446+
underline="none"
447+
>
448+
<ListItemIcon>
449+
<ChevronRightIcon />
450+
</ListItemIcon>
451+
<ListItemText primary="Open SRM with all observations" />
452+
</ListItemButton>
453+
</ListItem>
454+
<ListItem disablePadding>
455+
<ListItemButton
456+
component={Link} href={`${LassoService.API_URL}notebooks/lab/index.html?fromURL=${LassoService.API_URL}publicapi/v1/lasso/analytics/raw/srm/${scriptInfo.executionId}.ipynb`}
457+
target="_blank"
458+
rel="noopener"
459+
underline="none"
460+
>
461+
<ListItemIcon>
462+
<ChevronRightIcon />
463+
</ListItemIcon>
464+
<ListItemText primary="Open Raw SRM (each cell in the SRM corresponds to a row)" />
465+
</ListItemButton>
466+
</ListItem>
467+
468+
</List>
469+
422470
</React.Fragment>
423471
</CustomTabPanel>
424472
<CustomTabPanel value={value} index={4}>

lasso/src/pages/lasso/tds.tsx renamed to lasso/src/pages/lasso/ssn.tsx

Lines changed: 113 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -21,19 +21,19 @@ const SEQ_SHEET_DEFAULT: SequenceSheetData = {
2121
invocations: []
2222
};
2323

24-
const LSL_TEMPLATE = `dataSource 'mavenCentral2023'
24+
const TDS_LSL_TEMPLATE = `dataSource 'mavenCentral2023'
2525
study(name: 'TDSGenerated') {
2626
27-
profile('java17Profile') {
28-
scope('class') { type = 'class' }
29-
environment('java17') {
30-
image = 'maven:3.9-eclipse-temurin-17' // docker image (JDK 17)
31-
}
32-
}
27+
profile('java17Profile') {
28+
scope('class') { type = 'class' }
29+
environment('java17') {
30+
image = 'maven:3.9-eclipse-temurin-17' // docker image (JDK 17)
31+
}
32+
}
3333
3434
action(name: 'createStimulusMatrix') {
3535
execute {
36-
stimulusMatrix('Base64', """{{INTERFACE}}""", [/*impls*/], {{TESTS}})
36+
stimulusMatrix('myAb', """{{INTERFACE}}""", [/*impls*/], {{TESTS}})
3737
}
3838
}
3939
@@ -56,7 +56,73 @@ study(name: 'TDSGenerated') {
5656
maxAdaptations = 1 // how many adaptations to try
5757
5858
dependsOn 'select'
59-
include 'Base64'
59+
include '*'
60+
profile('java17Profile')
61+
}
62+
}`
63+
64+
const GEN_LSL_TEMPLATE = `dataSource 'lasso_quickstart'
65+
study(name: 'GenChatGPT') {
66+
67+
// target profile
68+
profile('java17Profile') {
69+
scope('class') { type = 'class' }
70+
environment('java17') {
71+
image = 'maven:3.9-eclipse-temurin-17'
72+
}
73+
}
74+
75+
action(name: 'createStimulusMatrix') {
76+
execute {
77+
stimulusMatrix('myAb', """{{INTERFACE}}""", [/*impls*/], {{TESTS}})
78+
}
79+
}
80+
81+
action(name: 'generateCodeGpt', type: 'GenerateCodeOpenAI') {
82+
// pipeline specific
83+
dependsOn 'createStimulusMatrix'
84+
include '*'
85+
profile('java17Profile')
86+
87+
// action configuration block
88+
apiKey = "demo" // see https://docs.langchain4j.dev/integrations/language-models/open-ai/
89+
model = "gpt-4o-mini"
90+
samples = 1
91+
92+
// custom DSL command offered by the action (for each stimulus matrix, create one prompt to obtain impls)
93+
prompt { stimulusMatrix ->
94+
// can by for any prompts: FA, impls, models etc.
95+
def prompt = [:] // create prompt model
96+
prompt.promptContent = """implement a java class with the following interface specification, but do not inherit a java interface: \`\`\`\${stimulusMatrix.lql}\`\`\`. Only output the java class and nothing else."""
97+
prompt.id = "lql_prompt"
98+
return [prompt] // list of prompts is expected
99+
}
100+
}
101+
102+
action(name: 'generateTestsGpt', type: 'GenerateTestsOpenAI') {
103+
// pipeline specific
104+
dependsOn 'generateCodeGpt'
105+
include '*'
106+
profile('java17Profile')
107+
108+
// action configuration block
109+
apiKey = "demo" // see https://docs.langchain4j.dev/integrations/language-models/open-ai/
110+
model = "gpt-4o-mini"
111+
samples = 1
112+
113+
prompt { stimulusMatrix ->
114+
def prompt = [:] // create prompt model
115+
prompt.promptContent = """generate a junit test class to test the functionality of the following interface specification: \`\`\`\${stimulusMatrix.lql}\`\`\`. Assume that the specification is encapsulated in a class that uses the same naming as in the interface specification. Only output the JUnit test class and nothing else."""
116+
prompt.id = "lql_prompt"
117+
return [prompt] // list of prompts is expected
118+
}
119+
}
120+
121+
action(name: 'execute', type: 'Arena') {
122+
maxAdaptations = 1 // how many adaptations to try
123+
124+
dependsOn 'generateTestsGpt'
125+
include '*'
60126
profile('java17Profile')
61127
}
62128
}`
@@ -68,13 +134,27 @@ function fillTemplate(template: string, values: Record<string, string | number>)
68134
}
69135

70136
export default function SequenceSheetEditorsPage() {
71-
const location = useLocation()
72137
const history = useHistory()
73138

74-
const [currentExampleId, setCurrentExampleId] = useState(location.search.split('=')[1])
139+
// Only run in browser context
140+
let exampleId = ""
141+
let recommendation = "search"
142+
if (typeof window != "undefined") {
143+
const searchParams = new URLSearchParams(window.location.search);
144+
145+
exampleId = searchParams.get("example") ?? "";
146+
recommendation = searchParams.get("recommendation") ?? "search";
147+
148+
console.log(exampleId)
149+
console.log(recommendation)
150+
}
151+
152+
const [currentExampleId, setCurrentExampleId] = useState(exampleId)
153+
const [recommendationType, setRecommendationType] = useState(recommendation)
75154

76155
const [sheets, setSheets] = useState<SequenceSheetData[]>([]);
77156
const [interfaceSpec, setInterfaceSpec] = useState<string>(``);
157+
78158
const [generatedLSL, setGeneratedLSL] = useState<string>("");
79159

80160
const editorRef = useRef<any>(null);
@@ -94,7 +174,9 @@ export default function SequenceSheetEditorsPage() {
94174

95175
const groovyBlock = sequenceSheetsToGroovyDSL(jsonSheets)
96176

97-
const result = fillTemplate(LSL_TEMPLATE, {
177+
const lslTemplate = recommendationType === "search" ? TDS_LSL_TEMPLATE : GEN_LSL_TEMPLATE
178+
179+
const result = fillTemplate(lslTemplate, {
98180
INTERFACE: interfaceSpec,
99181
TESTS: groovyBlock,
100182
ROWS: 10 // FIXME rows
@@ -189,9 +271,9 @@ export default function SequenceSheetEditorsPage() {
189271
}
190272

191273
useEffect(() => {
192-
if(currentExampleId) {
274+
if (currentExampleId) {
193275
console.log("example " + currentExampleId)
194-
276+
195277
const example = TDSExamples.MAP[currentExampleId]
196278
setSheets(example.sheets.map(s => {
197279
const sheetResult: any = jsonlToRows(s.jsonl);
@@ -204,22 +286,26 @@ export default function SequenceSheetEditorsPage() {
204286
rows: sheetResult.rows,
205287
columns: sheetResult.cols,
206288
invocations: s.invocations
207-
}
289+
}
208290
return sheet
209291
}))
210292
setInterfaceSpec(example.lql)
211293
}
212-
}, []);
294+
}, []);
213295

214296
return (
215297
<Layout>
216298
<Head>
217-
<title>Test-driven Code Search</title>
218-
<meta name="description" content="Test-driven Code Search" />
299+
<title>Test-driven Code Recommendation</title>
300+
<meta name="description" content="Test-driven Code Recommendation" />
219301
</Head>
220302

221-
<Typography sx={{ margin: 2 }} variant="h5" component="div">Test-driven Code Search<Typography variant="h6" component="div">Specify your Code Module (Code Index: Snapshot of Maven Central)</Typography></Typography>
303+
{recommendationType === "search" ?
304+
<Typography sx={{ margin: 2 }} variant="h5" component="div">Test-driven Code Search<Typography variant="h6" component="div">Specify your Code Module (Code Index: Snapshot of Maven Central)</Typography></Typography>
305+
306+
: <Typography sx={{ margin: 2 }} variant="h5" component="div">Test-driven Code Generation (using LLMs)<Typography variant="h6" component="div">Specify your Code Module (LSL Template uses ChatGPT and also generates additional Tests)</Typography></Typography>
222307

308+
}
223309

224310
<Stack sx={{ p: 4 }} spacing={3}>
225311
{/* <Typography variant="h4" sx={{ mb: 2 }}>
@@ -301,8 +387,16 @@ export default function SequenceSheetEditorsPage() {
301387
>
302388
Generate LSL Script
303389
</Button>
390+
391+
<Stack direction="row" spacing={1} alignItems="center" sx={{ mt: 1, mb: 3 }}>
392+
<Typography variant="body2" color="text.secondary">
393+
Generate the LSL script and scroll down below to submit your script.
394+
</Typography>
395+
</Stack>
304396
</Box>
305397

398+
399+
306400
{/* Bottom multiline TextField for generated LSL script */}
307401
{/* <TextField
308402
value={generatedLSL}

‎lasso/src/pages/search.md‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,11 +23,11 @@ LASSO's Sequence Sheet Notation (SSN) allows for a unique form of code search ba
2323

2424
**Learn More & Get Started:**
2525

26-
* [**Try the Test-Driven Code Search**](lasso/tds) - Begin exploring code based on your SSN test specifications.
26+
* [**Try the Test-Driven Code Search**](lasso/ssn?recommendation=search) - Begin exploring code based on your SSN test specifications.
2727
* [**SSN Documentation**](docs/datastructures/ssn) - Understand the structure and capabilities of SSN for defining test specifications.
2828

2929
### Examples
3030

3131
(under construction)
3232

33-
* [**Base64 encoding**](lasso/tds?example=BASE64)
33+
* [**Base64 encoding**](lasso/ssn?recommendation=search&example=BASE64)

‎web/404.html‎

Lines changed: 3 additions & 3 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)