Skip to content

Commit 02b14b4

Browse files
synap5esynap5e
committed
Initial commit
0 parents  commit 02b14b4

7 files changed

Lines changed: 600 additions & 0 deletions

File tree

‎LICENSE.md‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
Copyright (c) 2021 (12022 Holocene), Simon Pinfold
2+
3+
Permission to use, copy, modify, and/or distribute this software for any purpose with or without fee is hereby granted, provided that the above copyright notice and this permission notice appear in all copies.
4+
5+
THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
6+
7+
Source: http://opensource.org/licenses/ISC

‎README.md‎

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
# onefile_python
2+
3+
Run python from a single exe (without needing to extract anything to disk).
4+
5+
This project uses reflective dll loading and nim's `staticRead` to load the python runtime from the executable itself.
6+
Custom (python) import hooks are installed to support loading modules (both native python (.pyc) and extension modules (.pyd)) from the embedded standard library.
7+
8+
## HOWTO
9+
10+
0. Set up nim
11+
1. Download `python-3.10.1-embed-amd64.zip` to the project directory
12+
2. `nimble build`
13+
3. Run `onefile_python.exe`
14+
15+
## TODO
16+
17+
Currently the exe just drops to an interactive loop. Modification to run a script or embed a file/module should be trivial.
18+
19+
- [ ] Support `onefile_python.exe script.py`
20+
- [ ] Build option for embedding a python file/module and running that on launch
21+
- [ ] Support other versions of python than `3.10.1` (autodetect?)
22+
23+
## Similar projects
24+
25+
- [py2exe](https://www.py2exe.org/index.cgi/FrontPage)
26+
- [PyInstaller](https://pyinstaller.readthedocs.io/en/stable/index.html)
27+
28+
Both projects are better suited for bundling an application (and all its dependencies) to end users. They both support some form of dependency resolution so modules not required by the bundled don't get installed, while this project includes the entire standard library.
29+
30+
PyInstaller supports single exe mode, but this just extracts the runtime to a temporary directory.
31+
py2exe supports a diskless/"bundle" mode.
32+
33+
Both these projects are *much* more complex than this one and support lots of extra features, but sometimes you don't need that...
34+
35+
This project is (maybe) better if you want a single exe that can run any python script [or will be once that is supported], or just want an exe that gives a python REPL.
36+
Being simpler, this project should be easier to hack on or learn from.
37+
38+
39+
## How it works
40+
41+
There's not much to it...
42+
43+
0. Use nim's [staticRead](https://nim-lang.org/docs/system.html#staticRead%2Cstring) to include `python-*-embedded.zip` and `bootstrap.py` inside compiled exe itself.
44+
1. Use [zippy](https://github.com/guzba/zippy) to access the contents of the archive at runtime.
45+
2. Use [memlib](https://github.com/khchen/memlib) to perform reflective dll loading of the embedded `python*.dll`. Reflective dll loading allows for loading the dll from memory rather than from disk. Hook `LdrLoadDll` and `K32EnumProcessModules` so other code using the dll can find it.
46+
3. Call various functions in the (reflectively) loaded dll to partially initialize python. Configure python to not try to load anything from disk (not absolutely required, but prevents conflicts and means the exe doesn't run any code in the current directory)
47+
4. Use [nimpy](https://github.com/yglukhov/nimpy) to initialize a python extension exporting some nim functions that can read data out of the `python*.zip` standard library (contained within the `...-embedded.zip`).
48+
5. Run the embedded `bootsrap.py` code to install an import hook. This import hook uses the functions from (4) to support importing python modules. If a `.pyc` can be found that matches an import, a loader that returns the unmarshalled `.pyc` is provided. If a `.pyd` can be found, the returned loader reflectively loads the `.pyd` and calls the module's initialization routine.
49+
6. Now that python's standard library can be imported, finish initializing python.
50+
6. Reflectively load other `.dlls` inside the `...-embedded.zip`. This is required so extension modules that depend on these dlls work e.g. `ctypes` needs `_ctypes.pyd` which requires `libffi.dll`.
51+
7. Run python code / REPL
52+
53+
## Is this a virus
54+
55+
No.
56+
57+
It uses reflective DLL loading, which is a technique some malware uses so that might upset particularly sensitive AVs.
58+
Like python itself, it could be used to run a malicious script.

‎bootstrap.py‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
import sys
2+
import marshal
3+
import _imp
4+
import _frozen_importlib_external as _bootstrap_external
5+
import _frozen_importlib as _bootstrap
6+
7+
from nimporter import *
8+
9+
_bootstrap._install_external_importers()
10+
_bootstrap_external._set_bootstrap_module(_bootstrap)
11+
12+
13+
DEBUG = globals().get('DEBUG', False)
14+
def debug(s: str):
15+
if DEBUG:
16+
print(s, file=sys.stderr)
17+
18+
19+
class InMemoryExtensionLoader:
20+
def __init__(self, filename: str):
21+
self.filename = filename
22+
23+
def create_module(self, spec):
24+
debug(f'InMemoryExtensionLoader.load_module({self.filename=!r}, {spec=!r})')
25+
module = pyd_load(self.filename, spec.name, spec)
26+
debug(f'\t{module=}')
27+
return module
28+
29+
def exec_module(self, module):
30+
_imp.exec_dynamic(module)
31+
32+
33+
class InMemoryBytecodeLoader(_bootstrap_external._LoaderBasics):
34+
35+
def __init__(self, filename: str):
36+
self.filename = filename
37+
38+
def get_code(self, fullname):
39+
debug(f'InMemoryBytecodeLoader.get_code({self.filename=!r}, {fullname=!r})')
40+
41+
pyc = stdlib_read(self.filename)
42+
code = marshal.loads(pyc[16:])
43+
44+
debug(f'\t{code=}')
45+
return code
46+
47+
48+
class InMemoryFinder:
49+
def find_spec(self, fullname, path, target=None):
50+
debug(f'InMemoryFinder.find_spec(fullname={fullname!r}, path={path!r}, target={target!r})')
51+
52+
loader = None
53+
submodule_search_locations = None
54+
55+
if '.' in fullname:
56+
package_name, module_name = fullname.split('.', 1)
57+
else:
58+
package_name = fullname
59+
module_name = ''
60+
module_path = module_name.replace('.', '/')
61+
62+
pyd = package_name + '.pyd'
63+
if module_path:
64+
init = '/'.join([package_name, module_path, '__init__.pyc'])
65+
else:
66+
init = '/'.join([package_name, '__init__.pyc'])
67+
module = '/'.join([package_name, module_path + '.pyc'])
68+
single_file = package_name + '.pyc'
69+
70+
debug(f'\tChecking {[init, module, single_file]}')
71+
72+
if pyd_has(pyd):
73+
debug(f'\t> found pyd: {pyd!r}')
74+
loader = InMemoryExtensionLoader(pyd)
75+
elif stdlib_has(init):
76+
debug(f'\t> found init: {init!r} (setting submodule_search_locations)')
77+
loader = InMemoryBytecodeLoader(init)
78+
submodule_search_locations = [f'<{fullname}>']
79+
elif stdlib_has(module):
80+
debug(f'\t> found module: {module!r}')
81+
loader = InMemoryBytecodeLoader(module)
82+
elif stdlib_has(single_file):
83+
debug(f'\t> found single file: {single_file!r}')
84+
loader = InMemoryBytecodeLoader(single_file)
85+
86+
if loader:
87+
return _bootstrap_external.spec_from_file_location(
88+
fullname,
89+
loader=loader,
90+
submodule_search_locations=submodule_search_locations
91+
)
92+
else:
93+
debug(f'\t> not found')
94+
95+
sys.meta_path.insert(1, InMemoryFinder())
96+
debug(sys.meta_path)

‎config.nims‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
2+
--gc:orc
3+
--define:windows
4+
when defined(release):
5+
--define:danger
6+
--passL:"-Wl,--gc-sections,-flto,-s" # gc sections, strip, link-time optimization
7+
--passC:"-flto" # link-time optimization
8+
--opt:size
9+

‎cpython_types.nim‎

Lines changed: 204 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,204 @@
1+
#[
2+
typedef struct PyPreConfig {
3+
int _config_init;
4+
int parse_argv;
5+
int isolated;
6+
int use_environment;
7+
int configure_locale;
8+
int coerce_c_locale;
9+
int coerce_c_locale_warn;
10+
#ifdef MS_WINDOWS
11+
int legacy_windows_fs_encoding;
12+
#endif
13+
int utf8_mode;
14+
int dev_mode;
15+
int allocator;
16+
} PyPreConfig;
17+
]#
18+
type PyPreConfig* = object
19+
config_init*: clong
20+
parse_argv*: clong
21+
isolated*: clong
22+
use_environment*: clong
23+
configure_locale*: clong
24+
coerce_c_locale*: clong
25+
coerce_c_locale_warn*: clong
26+
legacy_windows_fs_encoding*: clong
27+
utf8_mode*: clong
28+
dev_mode*: clong
29+
allocator*: clong
30+
31+
32+
#[
33+
typedef struct {
34+
enum {
35+
_PyStatus_TYPE_OK=0,
36+
_PyStatus_TYPE_ERROR=1,
37+
_PyStatus_TYPE_EXIT=2
38+
} _type;
39+
const char *func;
40+
const char *err_msg;
41+
int exitcode;
42+
} PyStatus;
43+
]#
44+
type PyStatus* = object
45+
type_o*: cint
46+
func_o*: cstring
47+
err_msg*: cstring
48+
exitcode*: cint
49+
50+
#[
51+
typedef struct PyConfig {
52+
int _config_init; /* _PyConfigInitEnum value */
53+
54+
int isolated;
55+
int use_environment;
56+
int dev_mode;
57+
int install_signal_handlers;
58+
int use_hash_seed;
59+
unsigned long hash_seed;
60+
int faulthandler;
61+
int tracemalloc;
62+
int import_time;
63+
int show_ref_count;
64+
int dump_refs;
65+
int malloc_stats;
66+
wchar_t *filesystem_encoding;
67+
wchar_t *filesystem_errors;
68+
wchar_t *pycache_prefix;
69+
int parse_argv;
70+
PyWideStringList orig_argv;
71+
PyWideStringList argv;
72+
PyWideStringList xoptions;
73+
PyWideStringList warnoptions;
74+
int site_import;
75+
int bytes_warning;
76+
int warn_default_encoding;
77+
int inspect;
78+
int interactive;
79+
int optimization_level;
80+
int parser_debug;
81+
int write_bytecode;
82+
int verbose;
83+
int quiet;
84+
int user_site_directory;
85+
int configure_c_stdio;
86+
int buffered_stdio;
87+
wchar_t *stdio_encoding;
88+
wchar_t *stdio_errors;
89+
#ifdef MS_WINDOWS
90+
int legacy_windows_stdio;
91+
#endif
92+
wchar_t *check_hash_pycs_mode;
93+
94+
/* --- Path configuration inputs ------------ */
95+
int pathconfig_warnings;
96+
wchar_t *program_name;
97+
wchar_t *pythonpath_env;
98+
wchar_t *home;
99+
wchar_t *platlibdir;
100+
101+
/* --- Path configuration outputs ----------- */
102+
int module_search_paths_set;
103+
PyWideStringList module_search_paths;
104+
wchar_t *executable;
105+
wchar_t *base_executable;
106+
wchar_t *prefix;
107+
wchar_t *base_prefix;
108+
wchar_t *exec_prefix;
109+
wchar_t *base_exec_prefix;
110+
111+
/* --- Parameter only used by Py_Main() ---------- */
112+
int skip_source_first_line;
113+
wchar_t *run_command;
114+
wchar_t *run_module;
115+
wchar_t *run_filename;
116+
117+
/* --- Private fields ---------------------------- */
118+
119+
// Install importlib? If equals to 0, importlib is not initialized at all.
120+
// Needed by freeze_importlib.
121+
int _install_importlib;
122+
123+
// If equal to 0, stop Python initialization before the "main" phase.
124+
int _init_main;
125+
126+
// If non-zero, disallow threads, subprocesses, and fork.
127+
// Default: 0.
128+
int _isolated_interpreter;
129+
} PyConfig;
130+
]#
131+
type PyConfig* = object
132+
config_init*: cint
133+
isolated*: cint
134+
use_environment*: cint
135+
dev_mode*: cint
136+
install_signal_handlers*: cint
137+
use_hash_seed*: cint
138+
hash_seed*: clong
139+
faulthandler*: cint
140+
tracemalloc*: cint
141+
import_time*: cint
142+
show_ref_count*: cint
143+
dump_refs*: cint
144+
malloc_stats*: cint
145+
146+
filesystem_encoding*: pointer
147+
filesystem_errors*: pointer
148+
pycache_prefix*: pointer
149+
150+
parse_argv*: cint
151+
orig_argv_sz*: pointer
152+
orig_argv_data*: pointer
153+
argv_sz*: pointer
154+
argv_data*: pointer
155+
xoptions_sz*: pointer
156+
xoptions_data*: pointer
157+
warnoptions_sz*: pointer
158+
warnoptions_data*: pointer
159+
160+
site_import*: cint
161+
bytes_warning*: cint
162+
warn_default_encoding*: cint
163+
inspect*: cint
164+
interactive*: cint
165+
optimization_level*: cint
166+
parser_debug*: cint
167+
write_bytecode*: cint
168+
verbose*: cint
169+
quiet*: cint
170+
user_site_directory*: cint
171+
configure_c_stdio*: cint
172+
buffered_stdio*: cint
173+
174+
stdio_encoding*: pointer
175+
stdio_errors*: pointer
176+
177+
legacy_windows_stdio*: cint
178+
179+
check_hash_pycs_mode*: pointer
180+
181+
pathconfig_warnings*: cint
182+
program_name*: pointer
183+
pythonpath_env*: pointer
184+
home*: pointer
185+
platlibdir*: pointer
186+
187+
module_search_paths_set*: cint
188+
module_search_paths_sz*: pointer
189+
module_search_paths_data*: pointer
190+
executable*: pointer
191+
base_executable*: pointer
192+
prefix*: pointer
193+
base_prefix*: pointer
194+
exec_prefix*: pointer
195+
base_exec_prefix*: pointer
196+
197+
skip_source_first_line*: cint
198+
run_command*: pointer
199+
run_module*: pointer
200+
run_filename*: pointer
201+
202+
install_importlib*: cint
203+
init_main*: cint
204+
isolated_interpreter*: cint

0 commit comments

Comments
 (0)