Skip to content

removed usage of static buffers#7002

Closed
badasahog wants to merge 2 commits into
Rdatatable:masterfrom
badasahog:devBranch18
Closed

removed usage of static buffers#7002
badasahog wants to merge 2 commits into
Rdatatable:masterfrom
badasahog:devBranch18

Conversation

@badasahog
Copy link
Copy Markdown
Contributor

Using a statically allocated buffer to avoid breaking lifetime limitations is unsafe. It's always better to pass the buffer in as a parameter.

It's also faster to do it this way, static storage duration is slow and can't be aliased with other memory nearly as easily.

@badasahog badasahog requested a review from aitap as a code owner May 21, 2025 09:31
@codecov
Copy link
Copy Markdown

codecov Bot commented May 21, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.69%. Comparing base (6f05e4a) to head (808f4b4).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7002      +/-   ##
==========================================
- Coverage   98.69%   98.69%   -0.01%     
==========================================
  Files          79       79              
  Lines       14677    14675       -2     
==========================================
- Hits        14486    14484       -2     
  Misses        191      191              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@badasahog badasahog requested review from MichaelChirico and ben-schwen and removed request for MichaelChirico and aitap May 21, 2025 09:51
Comment thread src/fread.c Outdated
Comment thread src/fread.c
if (ndigits == 0 || (fsize == (fsize >> shift << shift))) {
if (i < sizeof(suffixes)) {
snprintf(output, sizeof(output), "%"PRIu64"%cB (%"PRIu64" bytes)", // # notranslate
snprintf(output, 100, "%"PRIu64"%cB (%"PRIu64" bytes)", // # notranslate
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a lot of magic numbers to introduce into the code. Isn't there a better solution?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sizeof(output) is just sizeof(char) in this context

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, sizeof(char*) to be precise, but I see what you mean. And yet now the code, all of the callers and multiple places in the callee, has to remember the size of the buffer. Isn't the cure worse than the disease?

@badasahog badasahog requested a review from aitap May 21, 2025 11:58
Comment thread src/fread.c
char *ch2 = ptr;
limit = imin(limit, 500);
static const char* strlim(const char *ch, char buf[static 500], size_t limit) {
char *ch2 = buf;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of making the caller always promise a 500-byte buffer, sometimes to fit only 30 characters, why not make strlim accept size_t limit, char buf[limit]? Just make sure there's no off-by-one error when limit is equal to the size of the buffer.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that would involve using a VLA, which I believe is against the rules of this project?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can change buf to be a standard pointer, which would allow passing in a smaller buffer, but I'm not sure that would save memory because it's a temporary stack allocation.

Plus, we would lose compile time checks.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It wouldn't create a VLA if such a declaration shows up in the function arguments. The point about the compile time checks is taken, thanks.

Comment thread src/fread.c
if (ndigits == 0 || (fsize == (fsize >> shift << shift))) {
if (i < sizeof(suffixes)) {
snprintf(output, sizeof(output), "%"PRIu64"%cB (%"PRIu64" bytes)", // # notranslate
snprintf(output, 100, "%"PRIu64"%cB (%"PRIu64" bytes)", // # notranslate
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, sizeof(char*) to be precise, but I see what you mean. And yet now the code, all of the callers and multiple places in the callee, has to remember the size of the buffer. Isn't the cure worse than the disease?

@badasahog
Copy link
Copy Markdown
Contributor Author

@aitap the callers already had to give a size limit, that hasn't changed. Only the callee has changed.

@MichaelChirico
Copy link
Copy Markdown
Member

Please reopen this PR on a branch on this repo. It is required for any PR purporting performance effects to go through the atime suite.

@badasahog
Copy link
Copy Markdown
Contributor Author

badasahog commented May 21, 2025

@MichaelChirico I don't believe I have permission to create branches?

@MichaelChirico
Copy link
Copy Markdown
Member

MichaelChirico commented May 21, 2025

@MichaelChirico I don't believe I have permission to create branches?

You should -- all Members should have branch write permission:

https://github.com/orgs/Rdatatable/teams/project-members/members

image

@badasahog
Copy link
Copy Markdown
Contributor Author

@MichaelChirico done #7005

@badasahog badasahog closed this May 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants