[feature-request] Support for ignoring files for `apify push`

[JJetmar]

The documentation says:

One important thing to note is that you can use a .gitignore file to exclude files from being pushed. When you use apify push without a .gitignore, the full folder contents will be pushed, meaning that even the storage and node_modules will be pushed. These files are unnecessary to push, as they are both generated on the platform.

But there might be completely valid use-cases when I want to keep my whole project (code + documentation + assets/images) in GH repository but I am not interested in pushing these via apify push as it would be extra bandwidth, time during building Dockerimage etc.

Would it be possible to introduce support for ignored files, that would be considered just by apify push (additionally to .gitignore)?

[B4nan]

Can you give us an example when you would want to do this?

[JJetmar]

@B4nan Mostly during development as the docs mentions claims:

The apify push command should only really be used for quickly pushing and testing Actors on the platform during development

I mentioned one use-case when the working directory includes the some additional docs and assets (images) - I do not need these to be pushed to the Console as these are not visible anywhere and do not affect the functionality of the Actor.

Also we currently have a use-case of "general" Actor that wrap few hundreds of implementations (js files) and the build is taking long time (>4 minutes) <- This is probably bug with creating zip archive and we may address it in different PR/issue once we investigate more. cc @matyascimbulka

[JJetmar]

Related to #170 and #172

[l2ysho]

Related to #170 and #172

In the meantime underlying library for file-matching was changed so #172 is not so easy to merge. I can update current flow but I cannot say If I manage to do it for v1.2.2. Of course if we are ok with idea to introduce .apifyignore cc @B4nan

[BowTiedRaccoon]

Just upgraded to 1.3.0 and am facing similar but opposite problem.
We have a bunch of artifacts that get copied into the actor directory and then should get included in the apify push cmd.

New behavior uses .gitignore files to exclude those artifacts effectively breaking our deploys.
I would like some mechanism to tell Apify to include them. Something like a .apifyignore file would work.

[l2ysho]

Working on this right now. .apifyignore is a doable approach, but I'm not sure if it's the right thing to do as it opens the door to a lot of corner cases we can face (especially if !negation patterns are used), which brings me to 2 ideas.

1. Use .apifyignore in some simple form, but we have to let users know it does not have the same capabilities as .gitignore (especially nested ignore patterns)
2. Expand hardcoded ignore patterns and introduce a new folder type (private — ignored always), also introduce one to always push (metadata/deploy etc).

@vladfrangu @B4nan

[vladfrangu]

I personally think one top-level actorignore with same git pattern support is enough, no? No need to overcomplicate this, just make sure .actorignore takes precedence over gitignore

[BowTiedRaccoon]

Thanks for the feedback.
I like the root .actorignore concept. One less file to maintain in every actor directory, and harder to abuse in non integration related cases.

[l2ysho]

@BowTiedRaccoon @JJetmar .actorignore available in v1.3.1-beta.3 should cover both cases

[BowTiedRaccoon]

Thanks for the fix folks. Just confirmed its all working now.