fix(ts-empty): drop unused @crawlee/cheerio dependency#832
Draft
DaveHanns wants to merge 1 commit into
Draft
Conversation
The ts-empty template ships `@crawlee/cheerio` in `dependencies` even
though the template's `src/main.ts` imports nothing from Crawlee — the
sole reference is a commented-out `// import { CheerioCrawler } ...`
line. This has a few unwanted side effects for users of the template
and for tooling that inspects the generated project:
- ~190 packages get installed for a stub Actor that should install just
`apify` and its transitive deps.
- `apify run` classifies the project as a Crawlee project based on the
presence of the dep and applies the default `--purge` behavior, which
is surprising for an empty template that has no crawler state to
purge.
- Code assistants / LLMs reading the manifest see the dep and reach
for Crawlee APIs (e.g. `new CheerioCrawler({ router })`), which then
fail against the currently installed Crawlee major version and
produce confusing errors from an "empty" starter.
The parallel `ts-crawlee-cheerio` template already exists as the
canonical starting point for users who want a Crawlee-based scraper,
so this change simply drops the unused dep from `ts-empty`, cleans up
the commented Crawlee/`routes.js` imports in `src/main.ts`, and points
the README at `ts-crawlee-cheerio` for the scraping use case.
Surfaced during an evaluation of Apify surfaces for agent-driven Actor development.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
ts-emptytemplate lists@crawlee/cheerioindependencieseven though the template'ssrc/main.tsdoesn't actually import anything from Crawlee — the only reference is a commented-out line:// import { CheerioCrawler } from '@crawlee/cheerio';This PR drops the unused dep, removes the commented-out Crawlee/
routes.jsimports fromsrc/main.ts, and rewords the README so it doesn't advertise Crawlee as an included feature. Users who actually want a Crawlee-based starter already have the parallelts-crawlee-cheeriotemplate — the README now points there.Why
Carrying the unused dep has a few concrete downsides:
ts-emptycurrently pulls in Crawlee + its transitive deps (~190 packages) for a project that only doesActor.init()/log.info/Actor.exit().apify runbehavior. The CLI treats the project as a Crawlee project based on the presence of@crawlee/*independenciesand applies the default--purgebehavior, which is surprising for an "empty" template with no crawler state to purge.package.jsonto decide "does this project use Crawlee?" answers "yes" and then generatesnew CheerioCrawler({ router })snippets against an API that no longer matches the version pin — producing errors from a supposedly empty starter. The commented// import { CheerioCrawler }line insrc/main.tsreinforces the same wrong signal.The two-line "if you'd like to use Crawlee, uncomment this import" instruction in the current README hasn't kept up with Crawlee's evolution (
@crawlee/cheerioalone isn't enough for a working crawler anymore) and duplicates whatts-crawlee-cheerioalready provides in a first-class way.Evidence
Current
templates/ts-empty/package.json(before this PR):Current
templates/ts-empty/src/main.ts(before this PR):Nothing in the file references either
CheerioCrawlerorrouter.Changes
templates/ts-empty/package.json— remove@crawlee/cheeriofromdependencies(leaves onlyapify).templates/ts-empty/src/main.ts— remove the commented Crawlee import and the commentedroutes.jsimport.templates/ts-empty/README.md— drop the Crawlee "included feature" bullet, remove the "uncomment this import" hint, and add a one-liner pointing atts-crawlee-cheeriofor the scraping use case.Notes
js-emptyhas the same shape (unused@crawlee/cheerio, same commented import insrc/main.js). Not touched here so this PR stays atomic and reviewable; happy to open a follow-up if the same fix is wanted there.templates/manifest.jsonalready describests-emptyas"Empty template with basic structure for the Actor with Apify SDK ..."withtechnologies: ["nodejs"]and no mention of Crawlee, so no manifest change is needed.Test plan
apify create foo -t ts-empty— verify the resulting project installs onlyapify+ transitive deps (no@crawlee/*innode_modules).apify runin the scaffoldedts-emptyproject does not treat it as a Crawlee project.apify create bar -t ts-crawlee-cheeriostill installs Crawlee and runs the sample crawler unchanged.npm test) still pass.Surfaced during an evaluation of Apify surfaces for agent-driven Actor development.