Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions docs/contribution-guidelines/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -86,16 +86,15 @@ Avoid including:
```
3. Execute tests:
```bash
cd core
sbt test
```

> For IntelliJ users: ensure the working directory matches the module (`amber` for engine tests, `core` for services).
> For IntelliJ users: ensure the working directory matches the module (`amber` for engine tests, the repo root for services).

### Frontend (Angular)
1. Run unit tests:
```bash
cd core/gui
cd frontend
ng test --watch=false
```
2. Format code:
Expand Down
6 changes: 3 additions & 3 deletions docs/contribution-guidelines/guide-for-developers.md
Original file line number Diff line number Diff line change
Expand Up @@ -263,7 +263,7 @@ This command will optimize the frontend code to make it run faster. This step wi
## 3. Email Notification (Optional)
</summary>

1. Set `smtp` in `config/src/main/resources/user-system.conf`. You need an App password if the account has 2FA.
1. Set `smtp` in `common/config/src/main/resources/user-system.conf`. You need an App password if the account has 2FA.
2. Log in to Texera with an admin account.
3. Open the Gmail dashboard under the admin tab.
5. Send a test email.
Expand All @@ -286,10 +286,10 @@ This part is optional; you only need to do this if you are working on a specific
Note: Jooq creates DAO for simple operations if the requested SQL query is complex, then the developer can use the generated Table classes to implement the operation

### Disable password login
Edit `config/src/main/resources/gui.conf`, change `local-login` to `false`.
Edit `common/config/src/main/resources/gui.conf`, change `local-login` to `false`.

### Enforce invite only
Edit `config/src/main/resources/user-system.conf`, change `invite-only` to `true`.
Edit `common/config/src/main/resources/user-system.conf`, change `invite-only` to `true`.

### Backend endpoints Role Annotation
There are two types of permissions for the backend endpoints:
Expand Down
12 changes: 6 additions & 6 deletions docs/contribution-guidelines/guide-to-implement-java-operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ In this page, we'll explain the basic concepts in Texera and use examples to sho

### Code structure of every operator:

Every operator ideally has three classes that are found in each operator package in `core\workflow-operator\src\main\scala\edu\uci\ics\amber\operator`
Every operator ideally has three classes that are found in each operator package in `common/workflow-operator/src/main/scala/org/apache/texera/amber/operator`
* LogicalOp
* OperatorExecutor
* OperatorExecutorConfig
Expand Down Expand Up @@ -49,7 +49,7 @@ schema.getAttributes().get(1) // Attribute("tweet", AttributeType.String)

A regular expression operator matches a regular expression (regex) on each input tuple. For example, if we search the regex "weather" on the `tweet` attribute, then only tuple 2 will be the result. In other words, the regular expression operator is a kind of `filter()` operation in many programming languages.

To implement a regular expression operator, you will first need to write an `LogicalOp`. The following code is part of class [`RegexOpDesc`](https://github.com/apache/texera/blob/main/core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/regex/RegexOpDesc.scala) .
To implement a regular expression operator, you will first need to write an `LogicalOp`. The following code is part of class [`RegexOpDesc`](https://github.com/apache/texera/blob/main/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/regex/RegexOpDesc.scala) .

```scala
class RegexOpDesc extends FilterOpDesc {
Expand Down Expand Up @@ -127,7 +127,7 @@ abstract class LogicalOp extends PortDescriptor with Serializable {

Now this operator will be automatically available in the frontend. We can now start the system and test this operator.

To add an image for this operator, go to `core/gui/src/assets/operator_images`, then add an image with the _**SAME NAME**_ as what's specified in the operator registration. The image file should be in `png` format, with a transparent background, black and white, and should be square.
To add an image for this operator, go to `frontend/src/assets/operator_images`, then add an image with the _**SAME NAME**_ as what's specified in the operator registration. The image file should be in `png` format, with a transparent background, black and white, and should be square.

For example, for the regex operator, the code `new Type(value = classOf[RegexOpDesc], name = "Regex")` specified a name `Regex`, then the image file name should be `Regex.png`.

Expand All @@ -146,7 +146,7 @@ id tweet sentiment
```


The following code is the implementation of class [`SentimentAnalysisOpDesc`](https://github.com/apache/texera/blob/main/core/workflow-operator/src/main/scala/edu/uci/ics/amber/operator/huggingFace/HuggingFaceSentimentAnalysisOpDesc.scala) in Java.
The following code is the implementation of class [`SentimentAnalysisOpDesc`](https://github.com/apache/texera/blob/main/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceSentimentAnalysisOpDesc.scala) in Java.

```java
public class SentimentAnalysisOpDesc extends MapOpDesc {
Expand Down Expand Up @@ -202,7 +202,7 @@ In Texera, currently we have 4 pre-defined operations you can extend.
- `flatmap()`: for each input tuple, transforms it to a list of output tuples.
- `aggregate()`: performs an aggregation, such as sum, count, average, etc.

To implement an operator, you can first check if your operator can be implemented using the 4 pre-defined operations. You can find these pre-defined operations under [`texera/workflow/common/operators`](https://github.com/Texera/texera/tree/master/core/amber/src/main/scala/edu/uci/ics/texera/workflow/common/operators). Your own operator implementation should be in [`texera/workflow/operators/youroperator`](https://github.com/Texera/texera/tree/master/core/amber/src/main/scala/edu/uci/ics/texera/workflow/operators).
To implement an operator, you can first check if your operator can be implemented using the 4 pre-defined operations. You can find these pre-defined operations under [`common/workflow-operator`](https://github.com/apache/texera/tree/main/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator) (e.g., the `map`, `filter`, `flatmap`, and `aggregate` packages). Your own operator implementation should be in its own package under [the same directory](https://github.com/apache/texera/tree/main/common/workflow-operator/src/main/scala/org/apache/texera/amber/operator).

### Low-level OperatorExecutor API
For more complicated operators, if they cannot be implemented using these operations, then you need to implement `OperatorExecutor` using the following low-level interface.
Expand Down Expand Up @@ -278,6 +278,6 @@ Texera's backend is responsible for determining the UI information to the fronte
```

### Registration and icon
In the file `amber/src/main/scala/edu/uci/ics/texera/workflow/common/operators/LogicalOp.scala`, you will find a list of all registered operators, complete with their descriptor classes and names. After adding an operator's information, you can assign an icon to it. All operator icons are stored in the `/core/new-gui/src/assets/operator_images` directory. It's essential to ensure that the icon filename matches its respective operator descriptor name.
In the file `common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/LogicalOp.scala`, you will find a list of all registered operators, complete with their descriptor classes and names. After adding an operator's information, you can assign an icon to it. All operator icons are stored in the `frontend/src/assets/operator_images` directory. It's essential to ensure that the icon filename matches its respective operator descriptor name.


58 changes: 0 additions & 58 deletions docs/contribution-guidelines/micro-services-local-dev.md

This file was deleted.

2 changes: 1 addition & 1 deletion docs/tutorials/guide-to-use-python-udf.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ class GenerateOperator(UDFSourceOperator):

This `produce()` API returns an iterator of `TupleLike`, `TableLike`, or simply `None`.

See [Generator Operator](https://github.com/Texera/texera/blob/master/core/amber/src/main/python/pytexera/udf/examples/generator_operator.py) for an example of 1-out UDF.
See [Generator Operator](https://github.com/apache/texera/blob/main/amber/src/main/python/pytexera/udf/examples/generator_operator_integer.py) for an example of 1-out UDF.


#### 2-in UDF
Expand Down
Loading