Embedded App: do not extract sources

[henderkes]

Describe your feature request

Is your feature request related to a problem? Please describe.
The extraction to /tmp/frankenphp_... brings a few issues with it.

• /tmp must be writable.
• systemd ProtectTmp can lead to unexpected paths
• startup extracts the tar, can be slow
• old files linger in /tmp until restart of the machine
• source code is human readable without any reverse engineering

Describe the solution you'd like
Embedded filesystem inside the binary, rather than extracting the tares?

• possibly encrypted? not a goal to truly make it "secure", but not immediately showing the source code is a plus

Describe your approach

• create named virtual filesystem in go
• redirect caddy to use that vfs for e.g. file_server and matchers
• overwrite php file handlers to redirect to the vfs
• work around possible opcache issues (?)
• find a good solution for folders that need write access (e.g. symfony cache)?
• embed a database...? not sure how this could be possible, but who knows. otherwise create a proof of concept using sqlite.

I'll work on this slowly over the next weeks. Don't expect anything immediately. I think @dunglas mentioned that something like this was planned as a potential commercial offering, so if you don't want me to pick it up, just let me know now before I start.

[henderkes]

Idea for very rudimentary encryption: generate an AES key, encrypt the tar, dump the key into text section somewhere. On startup load it, decrypt the tar, dump it into vfs, zero out the unencrypted tar.

Files are still viewable in plain text in the loaded process memory, but they're not in a contiguous segment ready to be dumped, but scattered across the virtual file system.

[dunglas]

Hi Marc!

I've a working prototype for encryption that I indeed want to provide as a third-party commercial extension (it works with and without FrankenPHP).

To me, it's not the scope of FrankenPHP directly anyway.

I'm all in for all other changes you mentioned!

[henderkes]

Okay! The vfs stuff in general should be simple enough, not sure about the required php hooks yet. I might also change the build system a tiny bit, compress the blob a bit before embedding and put the embedding logic behind -tags=embedapp. The empty, git committed zero-byte files are a little annoying.

[withinboredom]

Isn't this an 'already solved problem' with phars? It's a vfs as well.

The problem is that many distributions (symfony/laravel/wordpress) expect to be able to write to their own filesystem by default -- you have to deliberately opt out of it for production. i.e., compiled DI containers, sqlite files, rendered templates. This breaks entirely in phars (read-only filesystem).

Perhaps a better solution is for frameworks to make it 'just work' when running from a phar? There are already ways to obfuscate and protect phars from modification and casual inspection.

[henderkes]

Isn't this an 'already solved problem' with phars? It's a vfs as well.

The problem is that many distributions (symfony/laravel/wordpress) expect to be able to write to their own filesystem by default -- you have to deliberately opt out of it for production. i.e., compiled DI containers, sqlite files, rendered templates. This breaks entirely in phars (read-only filesystem).

That's exactly what I intend to solve with this.

Perhaps a better solution is for frameworks to make it 'just work' when running from a phar? There are already ways to obfuscate and protect phars from modification and casual inspection.

Hmm, interesting idea, but I'm not sure how we could go about this. Any pointers? It wouldn't be part of FrankenPHP then, though, would it?

[henderkes]

Thinking about it, the phar approach would create a big issue with static assets anyway, as they couldn't be directly served by the Go side anymore.

[withinboredom]

Thinking about it, the phar approach would create a big issue with static assets anyway, as they couldn't be directly served by the Go side anymore.

Phars are like Jars; they're basically just zip files. Go should be able to access them just fine as a vfs.

[withinboredom]

Perhaps a better solution is for frameworks to make it 'just work' when running from a phar? There are already ways to obfuscate and protect phars from modification and casual inspection.

Hmm, interesting idea, but I'm not sure how we could go about this. Any pointers? It wouldn't be part of FrankenPHP then, though, would it?

This isn't solvable by 'just' frankenphp anyway; framework support is required. If you make a read/write vfs to work around it, you still run into the main issues mentioned: you need a place you can write to if you need to.

One interesting solution could be an idea of a frankenphp_register_writable_overlay($dir) or something that attempts to map a host-writable directory to the given directory. Then use stream hooks in php to remap read/writes to the writable directory, but the application is thinking it is writing in it's /database directory or whatever.

[henderkes]

This isn't solvable by 'just' frankenphp anyway; framework support is required. If you make a read/write vfs to work around it, you still run into the main issues mentioned: you need a place you can write to if you need to.

I don't quite follow; if I create a rw vfs, the directory becomes writable for the application. Of course the changes wouldn't persist, but that's not a big deal. People should dump cache, templates and assets before embedding anyway, but in case they forget at least the app is working.

Or do you mean having directories that will actually be read/written from the host system? That's easy to implement with our own vfs, letting users configure it would just need a minimal API.

[withinboredom]

I don't quite follow; if I create a rw vfs, the directory becomes writable for the application. Of course the changes wouldn't persist, but that's not a big deal. People should dump cache, templates and assets before embedding anyway, but in case they forget at least the app is working.

Where the actual vfs block file lives will need to be writable (the core problem of this issue: tmp isn't necessarily writable or even a stable location). You could put the whole application in memory, but some applications are dozens or even hundreds of gbs in assets.

[henderkes]

You could put the whole application in memory, but some applications are dozens or even hundreds of gbs in assets.

Yeah okay, that's actually a real concern. We'd just have to make path mappings configurable then, no way around it. Obviously can't keep that in a virtual filesystem.

For regular apps it should be totally fine to keep everything in memory by default, though. Still a better solution than we have right now (force extraction to an unknown /tmp/ directory).

[henderkes]

Small update: have a virtual filesystem ready. Currently working this way: writes are automatically streamed to /tmp/frankenphp_hash/. When a file in a written folder is requested it looks there.

Will change it to "mount" pwd instead. Still takes a while.