Skip to content

Commit 3129d46

Browse files
harttlecursoragent
andauthored
fix(date): cap strftime widths and account padding in memoryLimit (#895)
* fix(date): cap strftime widths and account padding in memoryLimit - Clamp numeric strftime pad widths to MAX_STRFTIME_PAD (1024) - Export estimateStrftimePaddingMemory for the date filter to charge memoryLimit - Replace unbounded pad() concatenation loop with ch.repeat + single concat - Add regression tests for clamping and memoryLimit on huge %width directives Co-authored-by: Cursor <cursoragent@cursor.com> * fix(date): harden strftime memory accounting and document security model Move strftime memory charging into the same formatting path used for padding, enforce pre-allocation checks, and add regression tests for non-string date format PoCs. Add dedicated docs clarifying that memoryLimit is cooperative DoS mitigation and not strict heap isolation. Co-authored-by: Cursor <cursoragent@cursor.com> * docs(zh-cn): add security model docs for DoS limits Add a Chinese security-model tutorial and link it from the Chinese DoS guide to clarify that memoryLimit is cooperative accounting, list uncounted custom conversion cases, and recommend avoiding fully user-defined templates in online services. Co-authored-by: Cursor <cursoragent@cursor.com> * docs: consolidate DoS docs into security-model pages Merge DoS guidance into security-model docs in both English and Chinese, and remove the placeholder dos.md pages to avoid duplicate/redirect-only docs. Co-authored-by: Cursor <cursoragent@cursor.com> * docs: merge DoS details into security-model docs Move the detailed parseLimit/renderLimit/memoryLimit explanations and examples into the English and Chinese security-model pages so content from the removed dos pages is preserved. Co-authored-by: Cursor <cursoragent@cursor.com> * docs: reorganize security-model structure for clarity Restructure English and Chinese security-model docs into a consistent flow: security boundary, limits overview, per-limit details, and online service guidance. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(strftime): simplify %N width parsing logic Use regex-backed width assumptions to simplify %N width normalization and padding memory accounting while keeping behavior equivalent. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor(strftime): rely on memoryLimit for width control Remove MAX_STRFTIME_PAD hard capping and rely on memoryLimit enforcement before padding allocation. Update strftime/date tests and security-model docs to match the new boundary and renderLimit caveats. Co-authored-by: Cursor <cursoragent@cursor.com> * fix(strftime): use add() once for padding, minimize churn - pad(): replace per-char loop with a single add(str, ch.repeat(n)) call. The earlier `probe[0] === ch` heuristic was wrong when ch happened to equal a leading char of 'probe' (e.g. ch === 'p'). - strftime.ts: revert unrelated typing/structural refactors so the diff contains only the memoryLimit threading and the %N memory charge. - docs: rewire the deleted dos.html sidebar entry to security-model.html (with localized labels) so the deleted page does not 404 from the sidebar. Co-authored-by: Cursor <cursoragent@cursor.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com>
1 parent 5b9c346 commit 3129d46

11 files changed

Lines changed: 200 additions & 127 deletions

File tree

docs/source/_data/sidebar.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -19,7 +19,7 @@ tutorials:
1919
plugins: plugins.html
2020
operators: operators.html
2121
truth: truthy-and-falsy.html
22-
dos: dos.html
22+
security_model: security-model.html
2323
static_analysis: static-analysis.html
2424
miscellaneous:
2525
migration9: migrate-to-9.html

docs/source/tutorials/dos.md

Lines changed: 0 additions & 57 deletions
This file was deleted.
Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
---
2+
title: Security Model
3+
---
4+
5+
LiquidJS provides DoS-oriented limits (`parseLimit`, `renderLimit`, `memoryLimit`) to reduce risk. This page explains what each limit protects, and the security boundary you should assume in production.
6+
7+
## Security boundary
8+
9+
The built-in limits are cooperative safeguards, not strict runtime isolation.
10+
11+
- They do **not** equal process RSS/heap usage.
12+
- They do **not** sandbox JavaScript execution.
13+
- They should be combined with process/container limits and request timeouts for defense in depth.
14+
15+
## Limits at a glance
16+
17+
- [parseLimit][parseLimit]: limit total template size per `parse()` call.
18+
- [renderLimit][renderLimit]: limit total render time per `render()` call.
19+
- [memoryLimit][memoryLimit]: cooperatively limit memory-sensitive allocations counted by LiquidJS.
20+
21+
## Limit details
22+
23+
### parseLimit
24+
25+
[parseLimit][parseLimit] restricts the size (character length) of templates parsed in each `.parse()` call, including referenced partials and layouts. Since LiquidJS parses template strings in near O(n) time, limiting total template length is usually sufficient.
26+
27+
A typical PC handles `1e8` (100M) characters without issues.
28+
29+
### renderLimit
30+
31+
Restricting template size alone is insufficient because dynamic loops with large counts can occur in render time. [renderLimit][renderLimit] mitigates this by limiting the time consumed by each `render()` call.
32+
33+
```liquid
34+
{%- for i in (1..10000000) -%}
35+
order: {{i}}
36+
{%- endfor -%}
37+
```
38+
39+
Render time is checked on a per-template basis (before rendering each template). In the above example, there are 2 templates in the loop: `order: ` and `{{i}}`, render time will be checked 10000000x2 times.
40+
41+
`renderLimit` is not a hard CPU limiter. It is checked between template renders, so compute-intensive filters/tags/user-defined functions or deeply nested template execution between checks can still cause DoS.
42+
43+
### memoryLimit
44+
45+
`memoryLimit` only limits operations that LiquidJS explicitly counts.
46+
47+
- Counted: memory-sensitive LiquidJS operations that call internal memory accounting.
48+
- Not guaranteed counted: arbitrary user object behavior such as custom `toValue()`/`toString()` chains, or other host-side code that allocates outside LiquidJS accounting points.
49+
50+
In other words, `memoryLimit` limits what LiquidJS counts, not every byte your process may allocate.
51+
52+
Even with small number of templates and iterations, memory usage can grow exponentially. In the following example, memory doubles with each iteration:
53+
54+
```liquid
55+
{% assign array = "1,2,3" | split: "," %}
56+
{% for i in (1..32) %}
57+
{% assign array = array | concat: array %}
58+
{% endfor %}
59+
```
60+
61+
As [JavaScript uses GC to manage memory](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Memory_management), `memoryLimit` may not reflect the actual memory footprint.
62+
63+
## Online service guidance
64+
65+
If you run an online service, avoid rendering fully user-defined templates whenever possible.
66+
67+
- Prefer curated templates or a restricted template subset.
68+
- If user-defined templates are required, isolate rendering (worker/process/container), enforce OS/container memory and CPU limits, and apply request rate limits.
69+
- Treat `parseLimit`/`renderLimit`/`memoryLimit` as one layer in a broader DoS defense strategy.
70+
71+
For heavy single-template operations, process-level isolation is still recommended (for example with [paralleljs][paralleljs]).
72+
73+
[paralleljs]: https://www.npmjs.com/package/paralleljs
74+
[parseLimit]: /api/interfaces/LiquidOptions.html#parseLimit
75+
[renderLimit]: /api/interfaces/LiquidOptions.html#renderLimit
76+
[memoryLimit]: /api/interfaces/LiquidOptions.html#memoryLimit

docs/source/zh-cn/tutorials/dos.md

Lines changed: 0 additions & 55 deletions
This file was deleted.
Lines changed: 76 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,76 @@
1+
---
2+
title: 安全模型
3+
---
4+
5+
LiquidJS 提供了面向 DoS 的限制选项(`parseLimit``renderLimit``memoryLimit`)来降低风险。本文按统一结构说明每个限制的作用范围,以及你在生产环境应采用的安全边界。
6+
7+
## 安全边界
8+
9+
内置限制是协作式防护,不是严格的运行时隔离。
10+
11+
-**不等于**进程的 RSS/heap 实际占用。
12+
-**不是** JavaScript 沙箱。
13+
- 在生产环境中应结合进程/容器资源限制和请求超时做分层防护。
14+
15+
## 限制速览
16+
17+
- [parseLimit][parseLimit]:限制每次 `parse()` 的模板总长度。
18+
- [renderLimit][renderLimit]:限制每次 `render()` 的总渲染时间。
19+
- [memoryLimit][memoryLimit]:协作式限制 LiquidJS 已记账的内存敏感分配。
20+
21+
## 限制详解
22+
23+
### parseLimit
24+
25+
[parseLimit][parseLimit] 限制每次 `.parse()` 调用中解析的模板大小(字符长度),包括引用的 partials 和 layouts。由于 LiquidJS 解析模板字符串的时间复杂度接近 O(n),限制模板总长度通常就足够了。
26+
27+
普通电脑可以很容易处理 `1e8`(100M)个字符的模板。
28+
29+
### renderLimit
30+
31+
仅限制模板大小是不够的,因为在渲染时可能会出现动态的数组和循环。[renderLimit][renderLimit] 通过限制每次 `render()` 调用的时间来缓解这些问题。
32+
33+
```liquid
34+
{%- for i in (1..10000000) -%}
35+
order: {{i}}
36+
{%- endfor -%}
37+
```
38+
39+
渲染时间是在渲染每个模板之前检查的。在上面的例子中,循环中有两个模板:`order: ``{{i}}`,因此会检查 2x10000000 次。
40+
41+
`renderLimit` 不是硬性的 CPU 限制器。它是在模板渲染边界做检查,因此检查点之间的高计算开销过滤器/标签/用户自定义函数,或深层模板嵌套,仍可能导致 DoS。
42+
43+
### memoryLimit
44+
45+
`memoryLimit` 只限制 LiquidJS 显式记账到的操作。
46+
47+
- 会被统计:LiquidJS 内部调用了内存记账逻辑的内存敏感操作。
48+
- 不保证被统计:任意用户对象行为(例如自定义 `toValue()` / `toString()` 链)以及其他发生在 LiquidJS 记账点之外的宿主侧分配。
49+
50+
换句话说,`memoryLimit` 限制的是 LiquidJS 的“已记账分配”,而不是进程里每一个字节的分配。
51+
52+
即使模板和迭代次数较少,内存使用量也可能呈指数增长。在下面的示例中,内存会在每次迭代中翻倍:
53+
54+
```liquid
55+
{% assign array = "1,2,3" | split: "," %}
56+
{% for i in (1..32) %}
57+
{% assign array = array | concat: array %}
58+
{% endfor %}
59+
```
60+
61+
由于 [JavaScript 使用 GC 来管理内存](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Memory_management)`memoryLimit` 可能无法反映实际的内存占用。
62+
63+
## 在线服务建议
64+
65+
如果你运行在线服务,建议尽量避免渲染完全由用户定义的模板。
66+
67+
- 优先使用受控模板或受限模板子集。
68+
- 如果必须支持用户自定义模板,请隔离渲染(worker/进程/容器),并同时配置操作系统或容器级的内存/CPU 限额与请求限流。
69+
-`parseLimit` / `renderLimit` / `memoryLimit` 视为 DoS 防护体系中的一层,而不是唯一防线。
70+
71+
对于单个模板中的重型操作,仍建议使用进程级隔离(例如 [paralleljs][paralleljs])。
72+
73+
[paralleljs]: https://www.npmjs.com/package/paralleljs
74+
[parseLimit]: /api/interfaces/LiquidOptions.html#parseLimit
75+
[renderLimit]: /api/interfaces/LiquidOptions.html#renderLimit
76+
[memoryLimit]: /api/interfaces/LiquidOptions.html#memoryLimit

docs/themes/navy/languages/en.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ sidebar:
5151
plugins: Plugins
5252
operators: Operators
5353
truth: Truthy and Falsy
54-
dos: DoS
54+
security_model: Security Model
5555
static_analysis: Static Analysis
5656

5757
miscellaneous: Miscellaneous

docs/themes/navy/languages/zh-cn.yml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -51,7 +51,7 @@ sidebar:
5151
plugins: 插件
5252
operators: 运算符
5353
truth: 真和假
54-
dos: DoS
54+
security_model: 安全模型
5555
static_analysis: 静态分析
5656

5757
miscellaneous: 其他

src/filters/date.ts

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -3,13 +3,14 @@ import { FilterImpl } from '../template'
33
import { NormalizedFullOptions } from '../liquid-options'
44

55
export function date (this: FilterImpl, v: string | Date, format?: string, timezoneOffset?: number | string) {
6-
const size = ((v as string)?.length ?? 0) + (format?.length ?? 0) + ((timezoneOffset as string)?.length ?? 0)
6+
const size = ((v as string)?.length ?? 0) + ((timezoneOffset as string)?.length ?? 0)
77
this.context.memoryLimit.use(size)
88
const date = parseDate(v, this.context.opts, timezoneOffset)
99
if (!date) return v
1010
format = toValue(format)
1111
format = isNil(format) ? this.context.opts.dateFormat : stringify(format)
12-
return strftime(date, format)
12+
this.context.memoryLimit.use(format.length)
13+
return strftime(date, format, this.context.memoryLimit)
1314
}
1415

1516
export function date_to_xmlschema (this: FilterImpl, v: string | Date) {
@@ -31,13 +32,14 @@ export function date_to_long_string (this: FilterImpl, v: string | Date, type?:
3132
function stringify_date (this: FilterImpl, v: string | Date, month_type: string, type?: string, style?: string) {
3233
const date = parseDate(v, this.context.opts)
3334
if (!date) return v
35+
const ml = this.context.memoryLimit
3436
if (type === 'ordinal') {
3537
const d = date.getDate()
3638
return style === 'US'
37-
? strftime(date, `${month_type} ${d}%q, %Y`)
38-
: strftime(date, `${d}%q ${month_type} %Y`)
39+
? strftime(date, `${month_type} ${d}%q, %Y`, ml)
40+
: strftime(date, `${d}%q ${month_type} %Y`, ml)
3941
}
40-
return strftime(date, `%d ${month_type} %Y`)
42+
return strftime(date, `%d ${month_type} %Y`, ml)
4143
}
4244

4345
function parseDate (v: string | Date, opts: NormalizedFullOptions, timezoneOffset?: number | string): LiquidDate | undefined {

src/util/strftime.ts

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,13 @@
11
import { changeCase, padStart, padEnd } from './underscore'
22
import { LiquidDate } from './liquid-date'
3+
import type { Limiter } from './limiter'
34

45
const rFormat = /%([-_0^#:]+)?(\d+)?([EO])?(.)/
56
interface FormatOptions {
67
flags: object;
78
width?: string;
89
modifier?: string;
10+
memoryLimit?: Pick<Limiter, 'use'>;
911
}
1012

1113
// prototype extensions
@@ -95,6 +97,7 @@ const formatCodes = {
9597
N: (d: LiquidDate, opts: FormatOptions) => {
9698
const width = Number(opts.width) || 9
9799
const str = String(d.getMilliseconds()).slice(0, width)
100+
opts.memoryLimit?.use(width - str.length)
98101
return padEnd(str, width, '0')
99102
},
100103
p: (d: LiquidDate) => (d.getHours() < 12 ? 'AM' : 'PM'),
@@ -118,31 +121,32 @@ const formatCodes = {
118121
};
119122
(formatCodes as any).h = formatCodes.b
120123

121-
export function strftime (d: LiquidDate, formatStr: string) {
124+
export function strftime (d: LiquidDate, formatStr: string, memoryLimit?: Pick<Limiter, 'use'>) {
122125
let output = ''
123126
let remaining = formatStr
124127
let match
125128
while ((match = rFormat.exec(remaining))) {
126129
output += remaining.slice(0, match.index)
127130
remaining = remaining.slice(match.index + match[0].length)
128-
output += format(d, match)
131+
output += format(d, match, memoryLimit)
129132
}
130133
return output + remaining
131134
}
132135

133-
function format (d: LiquidDate, match: RegExpExecArray) {
136+
function format (d: LiquidDate, match: RegExpExecArray, memoryLimit?: Pick<Limiter, 'use'>) {
134137
const [input, flagStr = '', width, modifier, conversion] = match
135138
const convert = formatCodes[conversion]
136139
if (!convert) return input
137140
const flags = {}
138141
for (const flag of flagStr) flags[flag] = true
139-
let ret = String(convert(d, { flags, width, modifier }))
142+
let ret = String(convert(d, { flags, width, modifier, memoryLimit }))
140143
let padChar = padSpaceChars.has(conversion) ? ' ' : '0'
141144
let padWidth = width || padWidths[conversion] || 0
142145
if (flags['^']) ret = ret.toUpperCase()
143146
else if (flags['#']) ret = changeCase(ret)
144147
if (flags['_']) padChar = ' '
145148
else if (flags['0']) padChar = '0'
146149
if (flags['-']) padWidth = 0
150+
memoryLimit?.use(Number(padWidth) - ret.length)
147151
return padStart(ret, padWidth, padChar)
148152
}

0 commit comments

Comments
 (0)