Include file inside excluded path will include all files in excluded path

[Reefkeeper]

Also see follow-up issue #2355

---

BackInTime 1.1.12 on Linux Mint 17.3
I want to backup my /home/myname directory but want to exclude the Dropbox directory.
Although serveral directories as "Downloads", "Encfs" are correctly excluded as they are marked so, BIT does backup the directory /home/myname/Dropbox which is also in the list of exclusions.
See screenshot below!
What makes this directory different that it cannot be excluded?
Thanks!
Friedhelm

[Germar]

AFAIK there is nothing special with Dropbox folder. Did you add it through Ornder hinzufügen? If not, please double-check that it is spelled correct. It is case sensitive, maybe your original folder is camel-case DropBox?

[Reefkeeper]

Hi Germar,
thank you for reassuring that there is nothing generally wrong with using the Dropbox folder.
Yes, I added it via "Ordner hinzufügen" (add folder), but I also tried manual adding, with and without trailing '/' and '*' with no change. Consulting the ~/.config/backintime file does not reveal anything wrong (excerpt):

profile2.snapshots.exclude.2.value=/home/friedhelm/Daten/_fotos/Lightroom/Backups
profile2.snapshots.exclude.20.value=.Private
profile2.snapshots.exclude.21.value=/home/friedhelm/Dropbox
profile2.snapshots.exclude.22.value=/home/friedhelm/Encfs
profile2.snapshots.exclude.23.value=/home/friedhelm/Downloads

As said, /home/friedhelm/Encfs and /home/friedhelm/Downloads are excluded, but Dropbox (spelled correctly) is always backed up.

[Germar]

How comes that you've profile2.snapshots.exclude.20 up to 23? This should normaly be profile2.snapshots.exclude.1 to 5 and you should have profile2.snapshots.exclude.size=5. Please make a backup of your config and manually fix this in an editor.

[Reefkeeper]

Sorry, I showed only an excerpt from the config file. I do not exclude only those 5 directories, but 23.
BIT automatically makes 2 digit numbers when it wants to keep a sort order, and sorts them according to their character values.

But to eliminate even this possible cause, I reduced the number of exclusions to 2 items, Dropbox and the Downloads folder:
Here is the config file, I tested with this and - Dropbox is still being backed up, while Downloads folder is not. I do not understand it:

profile2.name=2 TB Seagate
profile2.qt4.last_path=/home/friedhelm
profile2.qt4.places.SortColumn=1
profile2.qt4.places.SortOrder=0
profile2.qt4.settingsdialog.exclude.SortColumn=1
profile2.qt4.settingsdialog.exclude.SortOrder=0
profile2.qt4.settingsdialog.include.SortColumn=1
profile2.qt4.settingsdialog.include.SortOrder=0
profile2.snapshots.automatic_backup_anacron_period=1
profile2.snapshots.automatic_backup_anacron_unit=20
profile2.snapshots.automatic_backup_day=1
profile2.snapshots.automatic_backup_mode=0
profile2.snapshots.automatic_backup_time=0
profile2.snapshots.automatic_backup_weekday=7
profile2.snapshots.backup_on_restore.enabled=true
profile2.snapshots.bwlimit.enabled=false
profile2.snapshots.bwlimit.value=3000
profile2.snapshots.check_for_changes=true
profile2.snapshots.continue_on_errors=true
profile2.snapshots.copy_links=false
profile2.snapshots.copy_unsafe_links=false
profile2.snapshots.cron.ionice=true
profile2.snapshots.cron.nice=true
profile2.snapshots.cron.redirect_stderr=false
profile2.snapshots.cron.redirect_stdout=true
profile2.snapshots.custom_backup_time=8,12,18,23
profile2.snapshots.dont_remove_named_snapshots=true
profile2.snapshots.exclude.1.value=/home/friedhelm/Dropbox
profile2.snapshots.exclude.2.value=/home/friedhelm/Downloads
profile2.snapshots.exclude.bysize.enabled=false
profile2.snapshots.exclude.bysize.value=500
profile2.snapshots.exclude.size=2
profile2.snapshots.full_rsync=false
profile2.snapshots.full_rsync.take_snapshot_regardless_of_changes=false
profile2.snapshots.include.1.type=0
profile2.snapshots.include.1.value=/home/friedhelm
profile2.snapshots.include.size=1
profile2.snapshots.local.nocache=false
profile2.snapshots.local.password.save=false
profile2.snapshots.local.password.use_cache=true
profile2.snapshots.local_encfs.path=/media/friedhelm/Externe_HD3
profile2.snapshots.log_level=3
profile2.snapshots.min_free_inodes.enabled=true
profile2.snapshots.min_free_inodes.value=2
profile2.snapshots.min_free_space.enabled=true
profile2.snapshots.min_free_space.unit=20
profile2.snapshots.min_free_space.value=10
profile2.snapshots.mode=local
profile2.snapshots.no_on_battery=false
profile2.snapshots.notify.enabled=true
profile2.snapshots.path=/media/friedhelm/Externe_HD3
profile2.snapshots.path.host=linux
profile2.snapshots.path.profile=2
profile2.snapshots.path.user=friedhelm
profile2.snapshots.preserve_acl=false
profile2.snapshots.preserve_xattr=false
profile2.snapshots.remove_old_snapshots.enabled=true
profile2.snapshots.remove_old_snapshots.unit=80
profile2.snapshots.remove_old_snapshots.value=10
profile2.snapshots.rsync_options.enabled=false
profile2.snapshots.rsync_options.value=
profile2.snapshots.smart_remove=true
profile2.snapshots.smart_remove.keep_all=2
profile2.snapshots.smart_remove.keep_one_per_day=7
profile2.snapshots.smart_remove.keep_one_per_month=24
profile2.snapshots.smart_remove.keep_one_per_week=4
profile2.snapshots.smart_remove.run_remote_in_background=false
profile2.snapshots.ssh.cipher=default
profile2.snapshots.ssh.host=
profile2.snapshots.ssh.ionice=false
profile2.snapshots.ssh.nice=false
profile2.snapshots.ssh.nocache=false
profile2.snapshots.ssh.path=
profile2.snapshots.ssh.port=22
profile2.snapshots.ssh.prefix.enabled=false
profile2.snapshots.ssh.prefix.value=PATH=/opt/bin:/opt/sbin:\$PATH
profile2.snapshots.ssh.private_key_file=
profile2.snapshots.ssh.user=friedhelm
profile2.snapshots.use_checksum=false
profile2.snapshots.user_backup.ionice=false

As you can see, the snapshots.exclude.size=2 (/home/friedhelm/Dropbox and /home/friedhelm/Downloads) and the snapshots.include.size=1 (/home/friedhelm).

Dropbox is still backed up!!!! (Grrrrr..)

[Germar]

Hmm, that's strange. What's the output of ls -la /home/friedhelm/Dropbox/?

[Reefkeeper]

ls -la /home/friedhelm/Dropbox/
insgesamt 280
drwx------ 12 friedhelm friedhelm  4096 Mär 26 16:23 .
drwxr-xr-x 63 friedhelm friedhelm  4096 Mär 26 16:23 ..
drwxr-xr-x  3 friedhelm friedhelm  4096 Dez  7  2014 Apps
drwxr-xr-x  4 friedhelm friedhelm  4096 Feb 18 17:49 boxcryptor.bc
drwxr-xr-x  2 friedhelm friedhelm  4096 Sep  8  2014 .comments
-rwxr-xr-x  1 friedhelm friedhelm    35 Jul 30  2015 .dropbox
drwxr-xr-x  5 friedhelm friedhelm 81920 Mär 26 20:03 .dropbox.cache
drwxr-xr-x  2 friedhelm friedhelm  4096 Mär 26 17:46 ForMyEyesOnly
-rw-r--r--  1 friedhelm friedhelm 23294 Jan 10  2014 hangouts_message.ogg
drwxr-xr-x  3 friedhelm friedhelm 12288 Mär 25 18:59 Kamera-Uploads
drwxr-xr-x  2 friedhelm friedhelm  4096 Jun 22  2013 Kameravideos
drwxr-xr-x  4 friedhelm friedhelm  4096 Dez 25 12:28 PDF
drwxr-xr-x  3 friedhelm friedhelm  4096 Mär 25 13:09 Photos
drwxr-xr-x  2 friedhelm friedhelm  4096 Mär 28  2015 Public
-rw-r--r--  1 friedhelm friedhelm  2837 Dez 28  2014 schwarzer_hintergrund.png
-rw-r--r--  1 friedhelm friedhelm 62266 Dez  6  2014 Tux.png

[Germar]

Looks okay to me. Sorry, I'm out of ideas.

[colinl]

Could this happen if there were something like a link to the Dropbox folder from somewhere else in the heirarchy? Though I suppose then it would appear in the snapshot in a different place.

[Germar]

@colinl As long as you did not activate Copy links or Copy unsave links symlinks will be backed up as symlinks. So the linked folder should not be included.

[Reefkeeper]

Thank you Germar and Colinl for your ideas and Happy Easter!
Those options (copy links, copy unsafe links, symlinks) are not activated with me.
Seems like I have to live with it.
Anyway if someone should have spare time he could try out backing up his home directory without his Dropbox directory ;-)

Nevertheless let me say a big THANK YOU for your software!
Friedhelm

[colinl]

A couple of further ideas. If in the BIT GUI you select the latest snapshot does Dropbox appear in the folder list under Backup folders? If not then how do you browse to it in the snapshot? Also have you explicitly included Dropbox in the Include folders?

[Reefkeeper]

If in the BIT GUI you select the latest snapshot does Dropbox appear in the folder list under Backup folders?
Yes, it does! See screenshot, additionally I check the directories on the ext. HD.

As you can see "Downloads" for example, does not appear, which is correct.

Also have you explicitly included Dropbox in the Include folders?
No Dropbox is not included explicitly. Only /home/friedhelm is indicated (which includes Dropbox implicitly).
This is what I include:

[Germar]

On LinuxMintUsers I've seen you actually tried to include parts of the Dropbox folder before. Which is totally fine and SHOULD work. But maybe this previous attempts causing your troubles now.

So just to double-check, could you please rename your ~/.config/backintime/config into config.sic and create a new profile pointing to a new, clean directory?

I tried your config above yesterday and it worked as expected. My dummy Dropbox folder was excluded. So I guess this has something to do with your previous snapshots. BTW do you use Full rsync mode or not?

[Reefkeeper]

Hi Germar,
I made the test you proposed, and it shows the problem! I do not use full rsync mode. Do you suggest I should? The target file system type is EXT4.

Step 1: Completely new config file
included: /home/friedhelm
excluded: /home/friedhelm/Dropbox and several directories as proposed by BIT.
The config looks like this:

profile1.snapshots.exclude.1.value=.gvfs
profile1.snapshots.exclude.10.value=/dev/*
profile1.snapshots.exclude.11.value=/run/*
profile1.snapshots.exclude.12.value=/etc/mtab
profile1.snapshots.exclude.13.value=/var/cache/apt/archives/*.deb
profile1.snapshots.exclude.14.value=lost+found/*
profile1.snapshots.exclude.15.value=/tmp/*
profile1.snapshots.exclude.16.value=/var/tmp/*
profile1.snapshots.exclude.17.value=/var/backups/*
profile1.snapshots.exclude.18.value=.Private
profile1.snapshots.exclude.19.value=/home/friedhelm/_fotos
profile1.snapshots.exclude.2.value=.cache/*
profile1.snapshots.exclude.20.value=/home/friedhelm/Downloads
profile1.snapshots.exclude.21.value=/home/friedhelm/Encfs
profile1.snapshots.exclude.22.value=/home/friedhelm/VirtualBox VMs
profile1.snapshots.exclude.23.value=/home/friedhelm/Zeitung
profile1.snapshots.exclude.24.value=/home/friedhelm/Dropbox
profile1.snapshots.exclude.25.value=/home/friedhelm/Daten
profile1.snapshots.exclude.3.value=.thumbnails*
profile1.snapshots.exclude.4.value=[Tt]rash*
profile1.snapshots.exclude.5.value=*.backup*
profile1.snapshots.exclude.6.value=*~
profile1.snapshots.exclude.7.value=.dropbox*
profile1.snapshots.exclude.8.value=/proc/*
profile1.snapshots.exclude.9.value=/sys/*
profile1.snapshots.exclude.bysize.enabled=false
profile1.snapshots.exclude.bysize.value=500
profile1.snapshots.exclude.size=25
profile1.snapshots.full_rsync=false
profile1.snapshots.full_rsync.take_snapshot_regardless_of_changes=false
profile1.snapshots.include.1.type=0
profile1.snapshots.include.1.value=/home/friedhelm
profile1.snapshots.include.size=1

Result: Dropbox is not backed up, which is correct!

Step 2:
Now include 1 file within Dropbox folder: ../ForMyEyesOnly/test.txt.
The config now looks like this (these are the only differences to #1)

profile1.snapshots.include.2.type=1
profile1.snapshots.include.2.value=/home/friedhelm/Dropbox/ForMyEyesOnly/test.txt
profile1.snapshots.include.size=2

Now the complete Dropbox folder with everything in it is backed up!!! This is wrong.

Step 3:
Remove the include in step 2 within Dropbox folder: ../ForMyEyesOnly/test.txt.
Result:
The config file looks exactly as in Step 1 AND the Dropbox folder is NOT backed up, which is correct.
Although this is a contradiction to the previous runs where removing the inclusion of a specific Dropbox file did not change the behaviour and the Dropbox folder was backed up although it shouldn't. This may as you said have something to do with previous snapshots.

In total:
I do not manage to exclude a subdirectory within a folder which is to be backed up.
If at least one file is indicated for backup, all the folder contents is backed up and the exclusion is ignored. In my opinion this has to do with the way rsync treats the exclusions?

Thank you for your patience!

[Germar]

Okay, I was able to reproduce this here. I'll have a look why the whole excluded path is backed up when only a single file inside the excluded path is included.