Skip to content

Commit 99ec1bc

Browse files
jeff5jeff5
committed
Begin collection of internal docs in the repo
Material rescued from the Jython Wiki.
1 parent e382428 commit 99ec1bc

3 files changed

Lines changed: 596 additions & 0 deletions

File tree

‎InternalDocs/README.md‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
# Jython Internals Documentation
2+
3+
The documentation in this folder is intended for Jython maintainers
4+
and contributors.
5+
It describes implementation details of Jython, which should not be
6+
assumed to be part of the Python language specification
7+
or API.
8+
9+
These details can change between any two Jython versions.
10+
Some of this material (for Jython 2)
11+
was recovered from the Jython wiki in an archived state,
12+
in which it has lost its contribution history.
13+
Thanks however are due to Jython Developers active before 2013.
14+
15+
<!-- Headings are provisional, from the CPython equivalent. -->
16+
17+
18+
## General Resources
19+
20+
<!-- - [Source Code Structure](structure.md) -->
21+
22+
23+
## Compiling Python Source Code
24+
<!-- - [Guide to the parser](parser.md) -->
25+
<!-- - [Compiler Design](compiler.md) -->
26+
27+
28+
## Runtime Objects
29+
<!-- - [Code Objects](code_objects.md) -->
30+
<!-- - [Generators](generators.md) -->
31+
<!-- - [Frames](frames.md) -->
32+
33+
34+
## Program Execution
35+
<!-- - [The Bytecode Interpreter](interpreter.md) -->
36+
<!-- - [Exception Handling](exception_handling.md) -->
37+
38+
39+
## Built-in Types
40+
41+
- [Defining Python Types in Java](python_types_in_java.md)
42+
- [Python Subclasses of Built-in Types](generated_derived_classes.md)
43+
44+
45+
## Modules
46+
47+

‎InternalDocs/generated_derived_classes.md‎

Lines changed: 286 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,286 @@
1+
# Generating the `*Derived` classes
2+
3+
These are notes on the use of `gderived.py`,
4+
a tool you need when implementing new types in Jython,
5+
so that they may have Python subclasses.
6+
7+
8+
## Background
9+
10+
Many of the Java classes that implement Python types
11+
have a counterpart class with the same name but with "Derived" appended.
12+
For example `PyString` is paired with `PyStringDerived`,
13+
`PyType` with `PyTypeDerived`, and so on.
14+
The `*Derived` classes are each a subclass of their corresponding principal class.
15+
They come into play when you create a subclass (in Python)
16+
and override (in Python) one or more methods
17+
whose base definition is exposed from the Java implementation.
18+
They ensure that this overriding (Python) method is the version invoked,
19+
even when the call is from Java.
20+
21+
There are two parts to this remarkable feature.
22+
One is in the implementation of the principal class itself,
23+
where the static exposed new method checks to see
24+
whether the actual (Python) type of the object being created
25+
is exactly the type that the principal class implements.
26+
If it is, then a new instance of the (Java) class is returned.
27+
If it is not exactly that class,
28+
an instance of the counterpart `*Derived` class is returned.
29+
30+
The second part of the feature is in the `*Derived` class.
31+
There, each exposed method may be overridden in a stylised way:
32+
it will check for the existence of a (Python) method
33+
redefining (the exposed name of) that method.
34+
If it fails to find one,
35+
it calls the version in the principal class (using the super keyword in Java).
36+
If it finds a Python re-definition,
37+
it invokes that using `PyObject.__call__()`.
38+
39+
The `*Derived` counterpart of each principal class is generated
40+
using the script `gderived.py` and a brief specification.
41+
The script is in the `src/templates` directory,
42+
together with several modules it imports,
43+
and it has to be run with that as the current directory.
44+
The specification corresponding to each principal class
45+
is in the same directory.
46+
47+
One of the imported modules is `gexposed.py`.
48+
This used to have a function in its own right,
49+
but it is superseded by the exposer (`org.python.expose.generate.Exposer`)
50+
and the corresponding Ant task.
51+
If you use the new exposer,
52+
even if you prohibit subclassing with `@ExposedType(isBaseType=false)`,
53+
it will generate a reference to the sort of class `gderived.py` creates.
54+
The modern exposer is described in the article
55+
[Python Types in Java](python_types_in_java.md).
56+
57+
58+
### Author's note:
59+
60+
At the time of starting these notes,
61+
there is no user guide to `gderived.py` and what it achieves.
62+
These notes stem from use of the tool and a certain amount of reverse-engineering.
63+
Please improve on them by correcting misunderstandings and omissions.
64+
65+
The work was done on a Windows 7 system,
66+
using Python 2.7 (without trying later versions,
67+
because of the vintage of the code).
68+
The choice of OS shows sometimes in the direction of slashes in pathnames,
69+
but that shouldn't confuse anyone.
70+
Although the motivation was to add a serious Python type (`bytearray`) to Jython,
71+
illustrations will be drawn from a facetiously-named type (`piranha`),
72+
with a Java implementation in `src/org/python/ethel/the/frog/Piranha.java`.
73+
74+
75+
## `gderived.py` as a Command
76+
77+
### The 2-argument Forms
78+
79+
The most transparent form of the command is:
80+
```
81+
python [--lazy] gderived.py <derived-spec> <output-file>
82+
```
83+
When using `gderived.py` in that way one is working with three user files:
84+
85+
`<derived-spec>`, the specification for the contents of the derived Java class.
86+
By convention, this has the extension `.derived`,
87+
and the Python name of the type being defined.
88+
The files for Jython types are in the `src/templates` folder
89+
along with the scripts,
90+
but anywhere seems to work with this form of the command,
91+
so we'll use `src/org/python/ethel/the/frog/piranha.derived`
92+
(note lower case type name `piranha`).
93+
94+
`<output-file>`, the file in which the generated class will be written.
95+
This has to be in the Jython source tree under `src/org/python`.
96+
If your code is not there, `gderived.py` seems to run correctly,
97+
but will get the Java package statement wrong.
98+
You can supply any filename you like,
99+
but the class it writes will be named by adding "Derived"
100+
to the name of the input class.
101+
For our example the output file is
102+
`src/org/python/ethel/the/frog/PiranhaDerived.java`.
103+
104+
And last but not least, the class file that implements your type.
105+
The input file is identified from the directory of the output file
106+
and the class name given in the text of `<derived-spec>` (see below).
107+
This therefore also has to be in the Jython source tree under `src/org/python`.
108+
For our example the input file is `src/org/python/ethel/the/frog/Piranha.java`.
109+
110+
The `--lazy` option causes `gderived.py` only to generate the output file
111+
if the input file is newer.
112+
113+
### The 1-argument and 0-argument Forms
114+
115+
A second form of the command is:
116+
```
117+
python gderived.py [--lazy] [<derived-spec>]
118+
```
119+
When using `gderived.py` in that way
120+
one is working with the same three user files as above,
121+
and a configuration file `src/templates/mappings`.
122+
The entries in that file look like this:
123+
```
124+
int.derived:org.python.core.PyIntegerDerived
125+
object.derived:org.python.core.PyObjectDerived
126+
random.derived:org.python.modules.random.PyRandomDerived
127+
ast_Assert.derived:org.python.antlr.ast.AssertDerived
128+
```
129+
In effect, this file allows `gderived.py` to look up the second argument
130+
given the first,
131+
although this second argument is now given in dotted notation.
132+
In this form, the specification file `<name>.derived`
133+
has to be in `src/templates` and the input and output classes
134+
will be found relative to src.
135+
136+
Finally, the `<derived-spec>` argument is optional.
137+
In the zero-argument form,
138+
`gderived.py` will process all the entries in `src/templates/mappings`.
139+
It is essentially this form,
140+
with the `--lazy` option,
141+
that implements the template Ant target in `build.xml`.
142+
143+
144+
## The Specification file <name>.derived
145+
146+
### Available Directives
147+
148+
`base_class`
149+
Define the name of the input class.
150+
**Do not qualify the class name with the package**:
151+
the script will work it out from the output file path, relative to `src`.
152+
E.g. `base_class: Piranha`
153+
154+
`want_dict` Request creation of a dictionary in the derived class.
155+
If not specified, only a slots array is created.
156+
157+
`require`
158+
159+
160+
`define`
161+
162+
163+
`ctr` arguments to the constructor, after the subtype argument.
164+
For example, in `_json.Scanner.derived` we have `ctr: PyObject context`,
165+
and this leads to the constructor
166+
`ScannerDerived(PyType subtype, PyObject context)`
167+
which calls `super(subtype, context)`.
168+
169+
`incl` include all the methods from this base class,
170+
171+
`unary1`
172+
173+
`binary`
174+
175+
`ibinary`
176+
177+
`no_toString`: `[true|false]`
178+
If the parameter is `false` or the directive is not present,
179+
generated class will be given a custom `toString()` method that invokes
180+
`__repr__`.
181+
If `true`, or the directive is given without a parameter,
182+
the generated class will not be given a custom `toString()` method.
183+
Use this when you already have a satisfactory one in the base.
184+
185+
`rest`: The rest of the file is Java code to insert (pretty much verbatim)
186+
into the derived class.
187+
Use this to provide your own custom overriding methods.
188+
189+
190+
### Related Templates in gderived-defs
191+
192+
TBD
193+
194+
195+
## Examples of Use
196+
197+
### Minimal Case
198+
199+
#### Input `Piranha.java`
200+
201+
Here is an example of a type defined in Java for access as a built-in in Jython.
202+
For information on the annotations and structure see PythonTypesInJava.
203+
```
204+
package org.python.ethel.the.frog;
205+
206+
import org.python.core.PyObject;
207+
import org.python.core.PyString;
208+
import org.python.core.PyType;
209+
import org.python.expose.ExposedMethod;
210+
import org.python.expose.ExposedNew;
211+
import org.python.expose.ExposedType;
212+
213+
@ExposedType(name="piranha")
214+
public class Piranha extends PyObject {
215+
216+
public static final PyType TYPE = PyType.fromClass(Piranha.class);
217+
218+
public Piranha() { this(TYPE); }
219+
220+
public Piranha(PyType subType) { super(subType); }
221+
222+
@ExposedNew
223+
final void newPiranha(PyObject[] args, String[] keywords) {}
224+
225+
@ExposedMethod(names={"theOperation", "theOtherOperation"})
226+
public int operation(int payment) { return payment*2; }
227+
}
228+
```
229+
230+
#### Specification `piranha.derived`
231+
```
232+
base_class: Piranha
233+
```
234+
235+
#### Output `PiranhaDerived.java`
236+
237+
The command
238+
```
239+
python gderived.py piranha.derived ../org/python/ethel/the/frog/PiranhaDerived.java
240+
```
241+
issued with current directory src/templates produces:
242+
```
243+
/* Generated file, do not modify. See jython/src/templates/gderived.py. */
244+
package org.python.ethel.the.frog;
245+
246+
import java.io.Serializable;
247+
import org.python.core.*;
248+
249+
public class PiranhaDerived extends Piranha implements Slotted {
250+
251+
public PyObject getSlot(int index) {
252+
return slots[index];
253+
}
254+
255+
public void setSlot(int index,PyObject value) {
256+
slots[index]=value;
257+
}
258+
259+
private PyObject[]slots;
260+
261+
public String toString() {
262+
PyType self_type=getType();
263+
PyObject impl=self_type.lookup("__repr__");
264+
if (impl!=null) {
265+
PyObject res=impl.__get__(this,self_type).__call__();
266+
if (!(res instanceof PyString))
267+
throw Py.TypeError(
268+
"__repr__ returned non-string (type "+
269+
res.getType().fastGetName()+")");
270+
return((PyString)res).toString();
271+
}
272+
return super.toString();
273+
}
274+
275+
}
276+
```
277+
278+
<!-- There was text under these headings but it was lost
279+
### Additional Features
280+
281+
#### Input `Piranha.java`
282+
283+
#### Specification `piranha.derived`
284+
285+
#### Output `PiranhaDerived.java`
286+
-->

0 commit comments

Comments
 (0)