Skip to content

Commit 4d1e230

Browse files
authored
feat: Add support for File parameter type
HEXA-1250 Add support for File parameter type
2 parents be0bb44 + d753299 commit 4d1e230

16 files changed

Lines changed: 719 additions & 42 deletions

README.md

Lines changed: 16 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -15,23 +15,23 @@ OpenHEXA Python SDK
1515

1616
OpenHEXA is an open-source data integration platform developed by [Bluesquare](https://bluesquarehub.com).
1717

18-
Its goal is to facilitate data integration and analysis workflows, in particular in the context of public health
18+
Its goal is to facilitate data integration and analysis workflows, in particular in the context of public health
1919
projects.
2020

2121
Please refer to the [OpenHEXA wiki](https://github.com/BLSQ/openhexa/wiki/Home) for more information about OpenHEXA.
2222

23-
This repository contains the code of the OpenHEXA SDK, a library allows you to write code for the OpenHEXA platform.
23+
This repository contains the code of the OpenHEXA SDK, a library allows you to write code for the OpenHEXA platform.
2424
It is particularly useful to write OpenHEXA data pipelines, but can also be used in the OpenHEXA notebooks environment.
2525

26-
The OpenHEXA wiki has a section dedicated to the SDK:
26+
The OpenHEXA wiki has a section dedicated to the SDK:
2727
[Using the OpenHEXA SDK](https://github.com/BLSQ/openhexa/wiki/Using-the-OpenHEXA-SDK).
2828

2929
Requirements
3030
------------
3131

3232
The OpenHEXA SDK requires Python version 3.11, 3.12 or 3.13.
3333

34-
If you want to be able to run pipeline in a containerized environment on your machine, you will need
34+
If you want to be able to run pipeline in a containerized environment on your machine, you will need
3535
[Docker](https://www.docker.com/).
3636

3737
Quickstart
@@ -52,17 +52,17 @@ You can then install the OpenHEXA SDK:
5252
pip install --upgrade openhexa.sdk
5353
```
5454

55-
💡New OpenHEXA SDK versions are released on a regular basis. Don't forget to update your local installations with
55+
💡New OpenHEXA SDK versions are released on a regular basis. Don't forget to update your local installations with
5656
`pip install --upgrade` from times to times!
5757

58-
Now that the SDK is installed withing your virtual environmentYou can now use the `openhexa` CLI utility to create
58+
Now that the SDK is installed withing your virtual environmentYou can now use the `openhexa` CLI utility to create
5959
a new pipeline:
6060

6161
```shell
6262
openhexa pipelines init "My awesome pipeline"
6363
```
6464

65-
Great! As you can see in the console output, the OpenHEXA CLI has created a new directory, which contains the basic
65+
Great! As you can see in the console output, the OpenHEXA CLI has created a new directory, which contains the basic
6666
structure required for an OpenHEXA pipeline. You can now `cd` in the new pipeline directory and run the pipeline:
6767

6868
```shell
@@ -71,7 +71,7 @@ openhexa pipelines run ./my_awesome_pipeline
7171

7272
Congratulations! You have successfully run your first pipeline locally.
7373

74-
If you inspect the actual pipeline code, you will see that it doesn't do a lot of things, but it is still a perfectly
74+
If you inspect the actual pipeline code, you will see that it doesn't do a lot of things, but it is still a perfectly
7575
valid OpenHEXA pipeline.
7676

7777
Let's publish to an actual OpenHEXA workspace so that it can run online.
@@ -89,18 +89,18 @@ You will be prompted for an authentication token, you can find it in the popup a
8989
After adding the workspace using the CLI, you can now push your pipeline:
9090

9191
```shell
92-
openhexa pipelines push
92+
openhexa pipelines push
9393
```
9494

95-
As it is the first time, the CLI will ask you to confirm the creation operation. After confirmation the console will
95+
As it is the first time, the CLI will ask you to confirm the creation operation. After confirmation the console will
9696
output the link to the pipeline screen in the OpenHEXA interface.
9797

9898
You can now open the link and run the pipeline using the OpenHEXA web interface.
9999

100100
Contributing
101101
------------
102102

103-
The following sections explain how you can set up a local development environment if you want to participate to the
103+
The following sections explain how you can set up a local development environment if you want to participate to the
104104
development of the SDK.
105105

106106
### SDK development setup
@@ -114,8 +114,8 @@ pip install -e ".[dev]" # Necessary to be able to run the openhexa CLI
114114
```
115115
### Using a local installation of OpenHEXA to run pipelines
116116

117-
While it is possible to run pipelines locally using only the SDK, if you want to run OpenHEXA in a more realistic
118-
setting you will need to install the OpenHEXA app and frontend components. Please refer to the
117+
While it is possible to run pipelines locally using only the SDK, if you want to run OpenHEXA in a more realistic
118+
setting you will need to install the OpenHEXA app and frontend components. Please refer to the
119119
[installation instructions](https://github.com/BLSQ/openhexa/wiki/Installation-instructions) for more information.
120120

121121
You can then configure the OpenHEXA CLI to connect to your local backend:
@@ -130,7 +130,7 @@ Notes: you can monitor the status of your pipelines using http://localhost:8000/
130130

131131
If you want to use a local version of the SDK to run pipelines, you can build a docker image with the local version of the SDK installed in it :
132132

133-
```shell
133+
```shell
134134
docker build --platform linux/amd64 -t local_image:v1 -f images/Dockerfile .
135135
```
136136

@@ -165,7 +165,7 @@ We use code generation to create Python client code from our GraphQL schema. Thi
165165
The code generation process:
166166

167167
1. The GraphQL schema is manually taken from the [Openhexa Monorepo](https://github.com/BLSQ/openhexa-app/blob/main/frontend/schema.generated.graphql) and saved in [`openhexa/graphql/schema.generated.graphql`](https://github.com/BLSQ/openhexa-sdk-python/blob/main/openhexa/graphql/schema.generated.graphql)
168-
2`ariadne-codegen` uses both the schema and queries to generate typed Python client code
168+
2. `ariadne-codegen` uses both the schema and queries to generate typed Python client code
169169

170170
To run code generation manually:
171171

@@ -191,7 +191,7 @@ OpenHexaClient(server_url="app.demo.openhexa.org", token="supersecuretoken")
191191
```
192192

193193
## Release
194-
194+
195195
This project uses [release-please](https://github.com/googleapis/release-please) to manage releases using conventional commits.
196196

197197
To release a new version:
@@ -200,4 +200,3 @@ To release a new version:
200200
2. release-please will create a new release PR on GitHub.
201201
3. Once the PR is merged, release-please will create a new release on GitHub.
202202
4. A GitHub action will build the package on github release creation and upload it to PyPI and Anaconda.
203-

openhexa/graphql/graphql_client/__init__.py

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -87,6 +87,7 @@
8787
AccessmodFilesetRoleCode,
8888
AccessmodFilesetStatus,
8989
AccessmodProjectOrder,
90+
AddOrganizationMemberError,
9091
AddToFavoritesError,
9192
ApproveAccessmodAccessRequestError,
9293
ArchiveWorkspaceError,
@@ -125,6 +126,8 @@
125126
DeleteDatasetVersionError,
126127
DeleteMembershipError,
127128
DeleteMetadataAttributeError,
129+
DeleteOrganizationInvitationError,
130+
DeleteOrganizationMemberError,
128131
DeletePipelineVersionError,
129132
DeleteTeamError,
130133
DeleteTemplateVersionError,
@@ -147,6 +150,7 @@
147150
GenerateWorkspaceTokenError,
148151
IASOConnectionError,
149152
IASOMetadataType,
153+
InviteOrganizationMemberError,
150154
InviteWorkspaceMembershipError,
151155
JoinWorkspaceError,
152156
LaunchAccessmodAnalysisError,
@@ -156,6 +160,8 @@
156160
MembershipRole,
157161
MessagePriority,
158162
OrderByDirection,
163+
OrganizationInvitationStatus,
164+
OrganizationMembershipRole,
159165
ParameterType,
160166
ParameterWidget,
161167
PermissionMode,
@@ -174,6 +180,7 @@
174180
RegisterError,
175181
RemoveFromFavoritesError,
176182
RequestAccessmodAccessError,
183+
ResendOrganizationInvitationError,
177184
ResendWorkspaceInvitationError,
178185
RunDAGError,
179186
SetDAGRunFavoriteError,
@@ -189,6 +196,7 @@
189196
UpdateDatasetError,
190197
UpdateDatasetVersionError,
191198
UpdateMembershipError,
199+
UpdateOrganizationMemberError,
192200
UpdatePipelineError,
193201
UpdatePipelineVersionError,
194202
UpdateTeamError,
@@ -215,8 +223,10 @@
215223
GetConnectionConnectionBySlug,
216224
GetConnectionConnectionBySlugFields,
217225
)
226+
from .get_file_by_path import GetFileByPath, GetFileByPathGetFileByPath
218227
from .get_users import GetUsers, GetUsersUsers, GetUsersUsersAvatar
219228
from .input_types import (
229+
AddOrganizationMemberInput,
220230
AddPipelineOutputInput,
221231
AddToFavoritesInput,
222232
ApproveAccessmodAccessRequestInput,
@@ -254,6 +264,8 @@
254264
DeleteDatasetVersionInput,
255265
DeleteMembershipInput,
256266
DeleteMetadataAttributeInput,
267+
DeleteOrganizationInvitationInput,
268+
DeleteOrganizationMemberInput,
257269
DeletePipelineInput,
258270
DeletePipelineRecipientInput,
259271
DeletePipelineTemplateInput,
@@ -273,6 +285,7 @@
273285
GeneratePipelineWebhookUrlInput,
274286
GenerateWorkspaceTokenInput,
275287
IASOQueryFilterInput,
288+
InviteOrganizationMemberInput,
276289
InviteWorkspaceMemberInput,
277290
JoinWorkspaceInput,
278291
LaunchAccessmodAnalysisInput,
@@ -294,6 +307,7 @@
294307
RegisterInput,
295308
RemoveFromFavoritesInput,
296309
RequestAccessmodAccessInput,
310+
ResendOrganizationInvitationInput,
297311
ResendWorkspaceInvitationInput,
298312
ResetPasswordInput,
299313
RunDAGInput,
@@ -312,6 +326,7 @@
312326
UpdateDatasetInput,
313327
UpdateDatasetVersionInput,
314328
UpdateMembershipInput,
329+
UpdateOrganizationMemberInput,
315330
UpdatePipelineInput,
316331
UpdatePipelineProgressInput,
317332
UpdatePipelineRecipientInput,
@@ -326,6 +341,8 @@
326341
UpgradePipelineVersionFromTemplateInput,
327342
UploadPipelineInput,
328343
VerifyDeviceInput,
344+
WorkspaceInvitationInput,
345+
WorkspacePermissionInput,
329346
)
330347
from .invite_workspace_member import (
331348
InviteWorkspaceMember,
@@ -405,6 +422,8 @@
405422
"AccessmodFilesetRoleCode",
406423
"AccessmodFilesetStatus",
407424
"AccessmodProjectOrder",
425+
"AddOrganizationMemberError",
426+
"AddOrganizationMemberInput",
408427
"AddPipelineOutputInput",
409428
"AddPipelineRecipient",
410429
"AddPipelineRecipientAddPipelineRecipient",
@@ -531,6 +550,10 @@
531550
"DeleteMembershipInput",
532551
"DeleteMetadataAttributeError",
533552
"DeleteMetadataAttributeInput",
553+
"DeleteOrganizationInvitationError",
554+
"DeleteOrganizationInvitationInput",
555+
"DeleteOrganizationMemberError",
556+
"DeleteOrganizationMemberInput",
534557
"DeletePipeline",
535558
"DeletePipelineDeletePipeline",
536559
"DeletePipelineInput",
@@ -577,6 +600,8 @@
577600
"GetConnection",
578601
"GetConnectionConnectionBySlug",
579602
"GetConnectionConnectionBySlugFields",
603+
"GetFileByPath",
604+
"GetFileByPathGetFileByPath",
580605
"GetUsers",
581606
"GetUsersUsers",
582607
"GetUsersUsersAvatar",
@@ -588,6 +613,8 @@
588613
"IASOConnectionError",
589614
"IASOMetadataType",
590615
"IASOQueryFilterInput",
616+
"InviteOrganizationMemberError",
617+
"InviteOrganizationMemberInput",
591618
"InviteWorkspaceMember",
592619
"InviteWorkspaceMemberInput",
593620
"InviteWorkspaceMemberInviteWorkspaceMember",
@@ -608,6 +635,8 @@
608635
"MessagePriority",
609636
"OrderByDirection",
610637
"OrganizationInput",
638+
"OrganizationInvitationStatus",
639+
"OrganizationMembershipRole",
611640
"ParameterInput",
612641
"ParameterType",
613642
"ParameterWidget",
@@ -659,6 +688,8 @@
659688
"RemoveWebappFromFavoritesRemoveFromFavorites",
660689
"RequestAccessmodAccessError",
661690
"RequestAccessmodAccessInput",
691+
"ResendOrganizationInvitationError",
692+
"ResendOrganizationInvitationInput",
662693
"ResendWorkspaceInvitationError",
663694
"ResendWorkspaceInvitationInput",
664695
"ResetPasswordInput",
@@ -701,6 +732,8 @@
701732
"UpdateDatasetVersionInput",
702733
"UpdateMembershipError",
703734
"UpdateMembershipInput",
735+
"UpdateOrganizationMemberError",
736+
"UpdateOrganizationMemberInput",
704737
"UpdatePipelineError",
705738
"UpdatePipelineInput",
706739
"UpdatePipelineProgressInput",
@@ -738,8 +771,10 @@
738771
"VerifyDeviceError",
739772
"VerifyDeviceInput",
740773
"Workspace",
774+
"WorkspaceInvitationInput",
741775
"WorkspaceInvitationStatus",
742776
"WorkspaceMembershipRole",
777+
"WorkspacePermissionInput",
743778
"WorkspaceWorkspace",
744779
"WorkspaceWorkspaceCountries",
745780
"WorkspaceWorkspacePermissions",

openhexa/graphql/graphql_client/client.py

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@
4242
)
4343
from .delete_webapp import DeleteWebapp, DeleteWebappDeleteWebapp
4444
from .get_connection import GetConnection, GetConnectionConnectionBySlug
45+
from .get_file_by_path import GetFileByPath, GetFileByPathGetFileByPath
4546
from .get_users import GetUsers, GetUsersUsers
4647
from .input_types import (
4748
AddToFavoritesInput,
@@ -1042,3 +1043,26 @@ def get_connection(
10421043
)
10431044
data = self.get_data(response)
10441045
return GetConnection.model_validate(data).connection_by_slug
1046+
1047+
def get_file_by_path(
1048+
self, path: str, workspace_slug: str, **kwargs: Any
1049+
) -> Optional[GetFileByPathGetFileByPath]:
1050+
query = gql(
1051+
"""
1052+
query getFileByPath($path: String!, $workspaceSlug: String!) {
1053+
getFileByPath(workspaceSlug: $workspaceSlug, path: $path) {
1054+
key
1055+
name
1056+
path
1057+
size
1058+
type
1059+
}
1060+
}
1061+
"""
1062+
)
1063+
variables: Dict[str, object] = {"path": path, "workspaceSlug": workspace_slug}
1064+
response = self.execute(
1065+
query=query, operation_name="getFileByPath", variables=variables, **kwargs
1066+
)
1067+
data = self.get_data(response)
1068+
return GetFileByPath.model_validate(data).get_file_by_path

0 commit comments

Comments
 (0)