Skip to content

TSRM: make CG, EG, SCNG and AG compile-time offsets #22287

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
henderkes wants to merge 5 commits into from
Closed

TSRM: make CG, EG, SCNG and AG compile-time offsets #22287

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
4b0e0ad
TSRM: make CG, EG, SCNG and AG compile-time offsets
henderkes Jun 12, 2026
eacb878
place hot globals at negative offsets before cache pointer
henderkes Jun 25, 2026
f5fb0e8
clarify comment
henderkes Jun 25, 2026
8cd9791
apply @arnaud-lb's suggestion (ts_allocate_fast_id becomes a bump-all…
henderkes Jun 25, 2026
80a1269
suggestions
henderkes Jun 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions TSRM/TSRM.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,15 @@ struct _tsrm_tls_entry {
tsrm_tls_entry *next;
};

#if defined(__cplusplus) || defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 202311L)
# define TSRM_STATIC_ASSERT(c, m) static_assert((c), m)
#elif defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 201112L) /* C11 */
# define TSRM_STATIC_ASSERT(c, m) _Static_assert((c), m)
#else
# define TSRM_STATIC_ASSERT(c, m)
#endif
TSRM_STATIC_ASSERT(TSRM_ALIGNED_SIZE(sizeof(tsrm_tls_entry)) == TSRM_FAST_RESERVED_BASE,
"TSRM_FAST_RESERVED_BASE must equal the tsrm_tls_entry header size");

@henderkes henderkes Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so this is a two-fold problem:

  • duplicated ZEND_STATIC_ASSERT copy - tsrm can't use zend_portability, because zend_portability depends on tsrm.h
  • tsrm_ls_entry size is only known in this file, not in tsrm.h consumers.
  • AG size can be consumed at runtime, so it gets the last slot, but it's structurally the same problem

In C++ this would be a simple consteval function in the header, but I have no idea how to make it work in C. So I kept the definition private, but force it to the specific size it has.

@arnaud-lb arnaud-lb Jun 25, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly, the cache could be set to the end of tsrm_tls_entry so that code outside of TSRM.c do not have to be aware of the size of tsrm_tls_entry. Alternatively, set _tsrm_ls_cache to the start of tsrm_tls_entry, but allocate the front elements before it and use negative offsets.

@henderkes henderkes Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think moving it to the end makes sense, it would just move the implementation size leak from the fast path to the slow TSRMG_BULK path.

The alternative might work, I'm not sure yet. Let me explore it.

@henderkes henderkes Jun 25, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible to get rid of the size leak, but the concept is harder for me to wrap my head around. I'll push the commit, but I'm not sure it's a good idea.

@arnaud-lb arnaud-lb Jun 25, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think moving it to the end makes sense, it would just move the implementation size leak from the fast path to the slow TSRMG_BULK path.

Indeed

It's possible to get rid of the size leak, but the concept is harder for me to wrap my head around. I'll push the commit, but I'm not sure it's a good idea.

This seems fine to me, but I let you chose what you prefer.

@henderkes henderkes Jun 26, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not my decision to make. I'd probably take a small implementation size leak over the more complicated understanding, but that's subjective to me because I came up with the former x)


typedef struct {
size_t size;
Expand Down Expand Up @@ -319,6 +328,14 @@ TSRM_API void tsrm_reserve(size_t size)
}/*}}}*/


/* Reserve the front of the fast space for fixed-offset fast resources, so the
* bump-allocated ones (ts_allocate_fast_id) are placed after it. */
TSRM_API void tsrm_reserve_fast_front(size_t size)
{/*{{{*/
Comment thread
henderkes marked this conversation as resolved.
Outdated
tsrm_reserved_pos = TSRM_ALIGNED_SIZE(size);
}/*}}}*/


/* allocates a new fast thread-safe-resource id */
TSRM_API ts_rsrc_id ts_allocate_fast_id(ts_rsrc_id *rsrc_id, size_t *offset, size_t size, ts_allocate_ctor ctor, ts_allocate_dtor dtor)
{/*{{{*/
Expand Down Expand Up @@ -368,6 +385,28 @@ TSRM_API ts_rsrc_id ts_allocate_fast_id(ts_rsrc_id *rsrc_id, size_t *offset, siz
return *rsrc_id;
}/*}}}*/


/* Allocate a fast resource id at a fixed, compile-time-constant offset.
* Like tsrm_reserve(), assumes single-threaded startup. */
TSRM_API ts_rsrc_id ts_allocate_fast_id_at(ts_rsrc_id *rsrc_id, size_t *offset, size_t fixed_offset, size_t size, ts_allocate_ctor ctor, ts_allocate_dtor dtor)
{/*{{{*/
size_t saved_pos = tsrm_reserved_pos;

if (fixed_offset - TSRM_FAST_RESERVED_BASE > tsrm_reserved_size) {
TSRM_ERROR((TSRM_ERROR_LEVEL_ERROR, "Unable to allocate space for fixed fast resource"));
*rsrc_id = 0;
*offset = 0;
return 0;
}
tsrm_reserved_pos = fixed_offset - TSRM_FAST_RESERVED_BASE;
ts_allocate_fast_id(rsrc_id, offset, size, ctor, dtor);
/* keep ordinary (bump-allocated) fast resources past the high-water mark */
if (tsrm_reserved_pos < saved_pos) {
tsrm_reserved_pos = saved_pos;
}
return *rsrc_id;
}/*}}}*/

static void set_thread_local_storage_resource_to(tsrm_tls_entry *thread_resource)
{
tsrm_tls_set(thread_resource);
Expand Down
7 changes: 7 additions & 0 deletions TSRM/TSRM.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ typedef void (*ts_allocate_dtor)(void *);

#define THREAD_HASH_OF(thr,ts) (unsigned long)thr%(unsigned long)ts

#define TSRM_FAST_RESERVED_BASE TSRM_ALIGNED_SIZE(4 * sizeof(void *))

#ifdef __cplusplus
extern "C" {
#endif
Expand All @@ -94,6 +96,11 @@ TSRM_API ts_rsrc_id ts_allocate_id(ts_rsrc_id *rsrc_id, size_t size, ts_allocate
TSRM_API void tsrm_reserve(size_t size);
TSRM_API ts_rsrc_id ts_allocate_fast_id(ts_rsrc_id *rsrc_id, size_t *offset, size_t size, ts_allocate_ctor ctor, ts_allocate_dtor dtor);

/* Fast resources at caller-chosen, compile-time-constant offsets. The fixed
* prefix must be reserved after tsrm_reserve() and before any fast id. */
TSRM_API void tsrm_reserve_fast_front(size_t size);
TSRM_API ts_rsrc_id ts_allocate_fast_id_at(ts_rsrc_id *rsrc_id, size_t *offset, size_t fixed_offset, size_t size, ts_allocate_ctor ctor, ts_allocate_dtor dtor);

/* fetches the requested resource for the current thread */
TSRM_API void *ts_resource_ex(ts_rsrc_id id, THREAD_T *th_id);
#define ts_resource(id) ts_resource_ex(id, NULL)
Expand Down
9 changes: 6 additions & 3 deletions Zend/zend.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1019,9 +1019,12 @@ void zend_startup(zend_utility_functions *utility_functions) /* {{{ */
zend_init_rsrc_list_dtors();

#ifdef ZTS
ts_allocate_fast_id(&compiler_globals_id, &compiler_globals_offset, sizeof(zend_compiler_globals), (ts_allocate_ctor) compiler_globals_ctor, (ts_allocate_dtor) compiler_globals_dtor);
ts_allocate_fast_id(&executor_globals_id, &executor_globals_offset, sizeof(zend_executor_globals), (ts_allocate_ctor) executor_globals_ctor, (ts_allocate_dtor) executor_globals_dtor);
ts_allocate_fast_id(&language_scanner_globals_id, &language_scanner_globals_offset, sizeof(zend_php_scanner_globals), (ts_allocate_ctor) php_scanner_globals_ctor, NULL);
ts_allocate_fast_id_at(&compiler_globals_id, &compiler_globals_offset, ZEND_CG_OFFSET, sizeof(zend_compiler_globals), (ts_allocate_ctor) compiler_globals_ctor, (ts_allocate_dtor) compiler_globals_dtor);
ts_allocate_fast_id_at(&executor_globals_id, &executor_globals_offset, ZEND_EG_OFFSET, sizeof(zend_executor_globals), (ts_allocate_ctor) executor_globals_ctor, (ts_allocate_dtor) executor_globals_dtor);
ts_allocate_fast_id_at(&language_scanner_globals_id, &language_scanner_globals_offset, ZEND_SCNG_OFFSET, sizeof(zend_php_scanner_globals), (ts_allocate_ctor) php_scanner_globals_ctor, NULL);
ZEND_ASSERT(compiler_globals_offset == ZEND_CG_OFFSET);
ZEND_ASSERT(executor_globals_offset == ZEND_EG_OFFSET);
ZEND_ASSERT(language_scanner_globals_offset == ZEND_SCNG_OFFSET);
ts_allocate_fast_id(&ini_scanner_globals_id, &ini_scanner_globals_offset, sizeof(zend_ini_scanner_globals), (ts_allocate_ctor) ini_scanner_globals_ctor, NULL);
compiler_globals = ts_resource(compiler_globals_id);
executor_globals = ts_resource(executor_globals_id);
Expand Down
5 changes: 3 additions & 2 deletions Zend/zend_alloc.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2614,7 +2614,7 @@ typedef struct _zend_alloc_globals {
#ifdef ZTS
static int alloc_globals_id;
static size_t alloc_globals_offset;
# define AG(v) ZEND_TSRMG_FAST(alloc_globals_offset, zend_alloc_globals *, v)
# define AG(v) ZEND_TSRMG_FAST(ZEND_AG_OFFSET, zend_alloc_globals *, v)
#else
# define AG(v) (alloc_globals.v)
static zend_alloc_globals alloc_globals;
Expand Down Expand Up @@ -3335,7 +3335,8 @@ ZEND_API void start_memory_manager(void)
# endif
#endif
#ifdef ZTS
ts_allocate_fast_id(&alloc_globals_id, &alloc_globals_offset, sizeof(zend_alloc_globals), (ts_allocate_ctor) alloc_globals_ctor, (ts_allocate_dtor) alloc_globals_dtor);
ts_allocate_fast_id_at(&alloc_globals_id, &alloc_globals_offset, ZEND_AG_OFFSET, sizeof(zend_alloc_globals), (ts_allocate_ctor) alloc_globals_ctor, (ts_allocate_dtor) alloc_globals_dtor);
ZEND_ASSERT(alloc_globals_offset == ZEND_AG_OFFSET);
#else
alloc_globals_ctor(&alloc_globals);
#endif
Expand Down
9 changes: 9 additions & 0 deletions Zend/zend_globals.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,15 @@ struct _zend_executor_globals {
void *reserved[ZEND_MAX_RESERVED_RESOURCES];
};

#ifdef ZTS
/* Fixed compile-time offsets of the hot globals in the TSRM fast space.
* AG is last, sizeof(zend_alloc_globals) isn't header-visible. */
# define ZEND_CG_OFFSET (TSRM_FAST_RESERVED_BASE)
# define ZEND_EG_OFFSET (ZEND_CG_OFFSET + TSRM_ALIGNED_SIZE(sizeof(zend_compiler_globals)))
# define ZEND_SCNG_OFFSET (ZEND_EG_OFFSET + TSRM_ALIGNED_SIZE(sizeof(zend_executor_globals)))
# define ZEND_AG_OFFSET (ZEND_SCNG_OFFSET + TSRM_ALIGNED_SIZE(sizeof(zend_php_scanner_globals)))
#endif

#define EG_FLAGS_INITIAL (0)
#define EG_FLAGS_IN_SHUTDOWN (1<<0)
#define EG_FLAGS_OBJECT_STORE_NO_REUSE (1<<1)
Expand Down
6 changes: 3 additions & 3 deletions Zend/zend_globals_macros.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ BEGIN_EXTERN_C()

/* Compiler */
#ifdef ZTS
# define CG(v) ZEND_TSRMG_FAST(compiler_globals_offset, zend_compiler_globals *, v)
# define CG(v) ZEND_TSRMG_FAST(ZEND_CG_OFFSET, zend_compiler_globals *, v)
#else
# define CG(v) (compiler_globals.v)
extern ZEND_API struct _zend_compiler_globals compiler_globals;
Expand All @@ -40,15 +40,15 @@ ZEND_API int zendparse(void);

/* Executor */
#ifdef ZTS
# define EG(v) ZEND_TSRMG_FAST(executor_globals_offset, zend_executor_globals *, v)
# define EG(v) ZEND_TSRMG_FAST(ZEND_EG_OFFSET, zend_executor_globals *, v)
#else
# define EG(v) (executor_globals.v)
extern ZEND_API zend_executor_globals executor_globals;
#endif

/* Language Scanner */
#ifdef ZTS
# define LANG_SCNG(v) ZEND_TSRMG_FAST(language_scanner_globals_offset, zend_php_scanner_globals *, v)
# define LANG_SCNG(v) ZEND_TSRMG_FAST(ZEND_SCNG_OFFSET, zend_php_scanner_globals *, v)
extern ZEND_API ts_rsrc_id language_scanner_globals_id;
extern ZEND_API size_t language_scanner_globals_offset;
#else
Expand Down
2 changes: 1 addition & 1 deletion Zend/zend_portability.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -862,7 +862,7 @@ extern "C++" {
/** @deprecated */
#define ZEND_CGG_DIAGNOSTIC_IGNORED_END ZEND_DIAGNOSTIC_IGNORED_END

#if defined(__cplusplus)
#if defined(__cplusplus) || defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 202311L)

@henderkes henderkes Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

technically unrelated, but I didn't want to copy an incomplete definition into tsrm.c

@TimWolla TimWolla Jun 12, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see also #21228.

@henderkes henderkes Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd be in favour, getting php-src and extensions to compile in extremely old environments is a pain anyway (I do rhel-7 builds for frankenphp). Should I replace this then?

@TimWolla TimWolla Jun 12, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I replace this then?

Not yet, probably. See: https://news-web.php.net/php.internals/130714

# define ZEND_STATIC_ASSERT(c, m) static_assert((c), m)
#elif defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 201112L) /* C11 */
# define ZEND_STATIC_ASSERT(c, m) _Static_assert((c), m)
Expand Down
6 changes: 6 additions & 0 deletions main/main.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2842,6 +2842,12 @@ PHPAPI bool php_tsrm_startup_ex(int expected_threads)
{
bool ret = tsrm_startup(expected_threads, 1, 0, NULL);
php_reserve_tsrm_memory();
/* Must reserve exactly the prefix laid out by ZEND_*_OFFSET (zend_globals.h). */
Comment thread
henderkes marked this conversation as resolved.
Outdated
tsrm_reserve_fast_front(
TSRM_ALIGNED_SIZE(sizeof(zend_compiler_globals)) +
TSRM_ALIGNED_SIZE(sizeof(zend_executor_globals)) +
TSRM_ALIGNED_SIZE(sizeof(zend_php_scanner_globals)) +
TSRM_ALIGNED_SIZE(zend_mm_globals_size())); // AG size, exposed through function call

@henderkes henderkes Jun 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

really unhappy about this, same issue as below:

(void)ts_resource(0);
return ret;
}
Expand Down
Loading