Skip to content

Commit a19a982

Browse files
authored
Merge pull request #13 from Alokzh/improve-blogpost
Docs: Update memory units and update blog with DataFrame optimization Note
2 parents 43b957a + 95f4ef7 commit a19a982

1 file changed

Lines changed: 4 additions & 2 deletions

File tree

docs/DataFrame vs Buffer.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -198,7 +198,7 @@ Here are the results from running this benchmark on a 8,000,000 row dataset (app
198198
| Metric | DataFrame | Circular Buffer | Improvement |
199199
|--------|-----------|-----------------|-------------|
200200
| **Execution Time** | ~968,804 ms | ~46,258 ms | **21x faster** |
201-
| **Memory Usage** | ~222,730 MB | ~14,148 MB | **16x less memory** |
201+
| **Memory Usage** | ~223 GB | ~14 GB | **~16x smaller growth** |
202202
| **GC Events** | ~14,943 | ~944 | **94% fewer** |
203203
| **GC Time** | ~826,382 ms | ~405 ms | **2,040x less GC overhead** |
204204
| **Results Generated** | 7,999,901 | 7,999,901 | Identical accuracy |
@@ -207,7 +207,7 @@ Here are the results from running this benchmark on a 8,000,000 row dataset (app
207207

208208
**Key Insights:**
209209
- Circular buffers processed the same data **21x faster**
210-
- Used **10.6x less memory** despite processing the same amount of data
210+
- Used **16x less memory** despite processing the same amount of data
211211
- Had **94% fewer garbage collection events**, leading to smoother performance
212212
- Spent virtually no time on memory cleanup (405ms vs 826+ seconds)
213213
- Produced identical results, proving accuracy isn't compromised
@@ -237,4 +237,6 @@ DataFrames are, and will remain, an essential, powerful, and correct choice for
237237

238238
However, when dataset is massive & we need only simple manipulations like calculating moving averages which we could do with a sliding window without loading everything into memory, a circular buffer can provide significant performance benefits. The key takeaway is to understand your data processing needs and choose the right tool for the job.
239239

240+
**Note:** It is also fair to mention that other DataFrame libraries vary significantly in optimization and performance. Libraries like [Pandas in Python](https://pandas.pydata.org/) are heavily optimized with specialized type systems, C extensions, and native computational libraries. Pharo's DataFrame implementation, while powerful and growing, has not yet reached that level of optimization. For this reason, if one were to repeat this benchmark with a more optimized DataFrame library, the performance difference would likely be smaller.
241+
240242
Want to explore more? Check out the [Containers-Buffer repository](https://github.com/pharo-containers/Containers-Buffer) to see the complete implementation and examples.

0 commit comments

Comments
 (0)