feat(apps): add build-from-code custom frontend, data connectors, etc - #1391
feat(apps): add build-from-code custom frontend, data connectors, etc#1391wesjdj wants to merge 7 commits into
Conversation
| # A bring-your-own app is unlikely to be aware of the session's URL prefix, | ||
| # so strip it (like RStudio) and serve the app at "/". Apps that *are* | ||
| # prefix-aware can read $RENKU_BASE_URL_PATH instead. | ||
| strip_path_prefix=True, |
There was a problem hiding this comment.
question: All apps are running on a subdomain and the path for each app is /, right?. So we do not need to set this at all.
| @property | ||
| def build_frontend(self) -> str: | ||
| """The frontend token passed to the buildpacks via BP_RENKU_FRONTENDS. | ||
|
|
||
| This is the persisted ``frontend_variant`` for every RenkuLab-provided frontend. | ||
| The "custom" frontend has no built-in UI, so it maps to the buildpack token | ||
| "none": the buildpacks then produce a runnable image without injecting a | ||
| frontend and defer the launch process to the repository Procfile. | ||
| """ | ||
| if self.frontend_variant == FrontendVariant.custom: | ||
| return "none" | ||
| return self.frontend_variant |
There was a problem hiding this comment.
nitpick: It may be nicer to resolve self.frontend_variant into the FrontendVariant enum inside this proprety and just return the enum rather than a string.
| # The k8s limit is 63 characters (the name becomes a hostname label) but we leave | ||
| # some leeway. Must match APP_NAME_MAX_LENGTH in renku_apps/core.py, which is what | ||
| # generate_app_name() truncates to. |
There was a problem hiding this comment.
nitpick: should we add this to the help field rather than just in a comment?
| def generate_app_name(project_slug: str, launcher_id: ULID) -> str: | ||
| """Generate a DNS-1035 label name for an app, bounded to APP_NAME_MAX_LENGTH.""" | ||
| suffix = str(launcher_id)[-_LAUNCHER_ID_SUFFIX_LENGTH:].lower() | ||
| return f"{_slug_label(project_slug)}-{suffix}" |
There was a problem hiding this comment.
Please add a few test cases for this to make sure it works as expected. It should not take long to add them.
| def _slug_label(slug: str) -> str: | ||
| """Coerce a project slug (which may hold dots, underscores or a leading digit) into a DNS-1035 label.""" | ||
| label = re.sub(r"-+", "-", re.sub(r"[^a-z0-9]", "-", slug.lower())) | ||
| if not label[:1].isalpha(): | ||
| label = f"app-{label}" | ||
| return label[:_SLUG_MAX_LENGTH].rstrip("-") |
There was a problem hiding this comment.
question: I understand the outermost re.sub call - that one replaces repeated - in the name with a single -. But the second I am confused about. It replaces the first character or number of slug with a dash? Is this because you are trying to join with dash in f"app-{label}".
Also you have this: re.sub(r"[^a-z0-9]", "-", slug.lower()) but I suspect you want re.sub(r"^[a-z0-9]", "-", slug.lower()), the beggining of string character should be outside of the set.
| app_name = generate_app_name(project.slug, session_launcher.id) | ||
| labels = _app_labels(session_launcher, project) | ||
|
|
||
| work_dir = session_launcher.environment.working_directory or _DEFAULT_WORK_DIR |
There was a problem hiding this comment.
If we follow the same idea as what the session do then we try to get the work dir from the image before we fall back to a hardcoded default.
There was a problem hiding this comment.
See line 892 in components/renku_data_services/notebooks/core_sessions.py to see how we do this for sessions.
| created = await self.__client.create( | ||
| meta.with_manifest(manifest.model_dump(exclude_none=True, mode="json")), refresh=True | ||
| ) | ||
|
|
||
| owner_reference = _service_owner_reference(app_name, str(created.manifest.metadata.uid)) | ||
| for resource in dc_resources: | ||
| await self._create_owned(resource.secret, _SECRET_GVK, resource.name, cluster, owner_reference) | ||
| await self._create_owned(resource.pvc, _PVC_GVK, resource.name, cluster, owner_reference) | ||
|
|
There was a problem hiding this comment.
We need a way to handle partial creation or failed creation. A few things here:
- what happens if the pvc or secret with the same name already exists from a failed create atempt that has not been cleaned up
- how do we clean or recover up if creating the DC resources fail
There was a problem hiding this comment.
You may want to address this in a followup pr if you prefer. Or we can sit and talk about this. You can take a look at how this is handled in the sessions.
| tolerations: list[dict[str, Any]], | ||
| ) -> KnativeService: | ||
| """Build a Knative Service manifest derived from the session launcher.""" | ||
| environment = session_launcher.environment |
There was a problem hiding this comment.
I missed this from a previous review or PR. So I cannot select the code I am actually talking about but it is just below this point. When you create the security context you also need to: drop all capabilities, set privileged to False and also set privilege escallation to false.
| env.extend( | ||
| {"name": var.name, "value": var.value or ""} | ||
| for var in session_launcher.env_variables or [] | ||
| if var.name not in reserved_names | ||
| ) |
There was a problem hiding this comment.
If you set the dictionary first with session_launcher.env_variables and then you set the RENKU_* env variables then you dont have to check for thie reserved names. Because if there are clashes you will overwrite with the RENKU_* env vars which is what we want.
There was a problem hiding this comment.
It should look something like this:
securityContext:
allowPrivilegeEscalation: false
capabilities:
drop:
- ALL
privileged: false
runAsNonRoot: true
runAsUser: <some-id>
runAsGroup: <some-id>
| def _resources_from_resource_class(resource_class: ResourceClass) -> dict[str, Any]: | ||
| """Build a k8s container resources block from a resource class.""" |
There was a problem hiding this comment.
Note for when we merge to main... We added a new feature recently on the resource pool that sets a factor to derive cpu limits based on cpu request. We should take that into account here. The factor is optional and if unset then there are no cpu limits.
No description provided.