-
Notifications
You must be signed in to change notification settings - Fork 1
Expand file tree
/
Copy pathdaylight-savings.html
More file actions
183 lines (151 loc) · 6.88 KB
/
daylight-savings.html
File metadata and controls
183 lines (151 loc) · 6.88 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
<HTML><HEAD>
<TITLE>DST change and date comparisons</TITLE>
</HEAD><BODY>
<p><small><i>J.W. Schultz wrote the following text in a
<a href="http://www.cygwin.com/ml/cygwin/2003-10/msg00995.html">post</a>
to the cygwin mailing list. It has been slightly edited and beautified
for inclusion here.</i></small>
<h1>How the DST Change can adversely affect FAT filesystems</h1>
<p>
The vernal and autumnal transitions to and from daylight-savings
time have important implications
for those with Microsoft systems and use utilities that
compare file timestamps on different filesystem types or
with filesystems on other operating systems that can lead to
a problem in how the file's date is handled.
This problem lies in the way FAT filesystems stores
timestamps and how Windows converts between local time and
UTC.
<h2>Background</h2>
<p>
In UNIX and UNIX-like systems (such as Linux) file timestamps
are stored in UTC (universal time) and are only converted to
local-time by user-space programs for display purposes. At
the system call level all time values are in UTC and
utilities that compare timestamps do so in UTC. Also, the
standard UTC->local and local->UTC conversion functions are
aware of DST and conversions reflect this so that if a
timestamp was recorded during ST it will be converted using
the ST offset even when the current system time is DST.
<p>
In Windows things are not so simple. Windows operates in
local-time. Timestamps in the various FAT derived
filesystems are stored in local-time. Timestamps in NTFS
filesystems are stored in UTC. This inconsistency is
further complicated by the fact that the conversion routines
used are not DST aware. Instead of being DST aware the
system has a fixed offset to convert between local-time and
UTC regardless of the date in the timestamp. This fixed
offset is calculated at boot time and only changed when
systems transition to or from DST. As a result the apparent
modification time of a file on NTFS as reported in a windows
utility will change by one hour when reported in local-time
and FAT based files when reported in UTC.
<p>
The difficulty that this produces is that any utilities that
compare timestamps between FAT and NTFS filesystems or
between Windows and other platforms will view files that
have not changed as having a different modified time. Among other things
this will affect rsync, rdiff, unison, wget, and make. However,
for the purposes of this document, we will only discuss rsync.
<p>
With the reduced cost of hard disks many newer backup
systems are using hard disk based storage and take advantage
of timestamp comparison to detect file changes for the sake
of efficiency. Rsync is probably premier in this role and
is used by a fair number of free and even commercial backup
systems as well as being the basis for many home-brew backup
solutions.
<p>
With rsync and similar systems the effect of this is that
every file will appear to have been changed. The result is
any space savings associated with linking (--link-dest) or
with decremental backup approaches (--compare-dest and
--backup-dir) will be defeated. Perhaps worse, because
every file will appear to have changed the time required to
do a backup or a non-backup rsync will be much longer than
normal. In some cases backups that normally complete in
less than one hour can take several days.
<p>
So what can be done about it? Several things, there are
ways to merely mitigate the problem, to correct it and finally
to prevent the problem entirely.
<h2>Mitigation</h2>
<p>
Rsync has a --modify-window option. Many of you already use
--modify-window=1 to cope with the fact that windows often
stores timestamps with a two second resolution. Using a
--modify-window=3601 will cause rsync to ignore timestamp
differences of up to one hour.
<p>
This if often not particularly dangerous because a file would have to
be changed, synced and changed again without changing size
within a single hour and have no subsequent changes for this copy
to miss a file change. However, for some systems any such risk is
unacceptable, so other solutions are needed.
<h2>Correcting the Timestamps</h2>
<p>
There are two ways to correct.
<p>For the first run after the time change, you can run rsync with the
--checksum option in order to ensure that only files that have a changed
checksum get transferred, and update the modified time on all unchanged
files. This option has the drawback that it increases disk I/O by
a large amount on both the sending and receiving side, slowing down the
copy.
<p>
The other way to correct things is to change the timestamps
on the files on the backup server before doing a copy after a
time change has occurred.
<p>
Included here is an example perl script that will change the
timestamps of files in a list on standard-input. Whether
you use a positive or negative shift will depend on which
end you decide to adjust.
<p>
This is an example of how to use the script:
<blockquote><pre>
touch -d '01:00 13-apr-03' /tmp/cmpfile
find . -type f ! -newer /tmp/cmpfile | shifttime.pl 3600
</pre></blockquote>
<hr>
<pre>
#!/usr/bin/perl
use strict;
my $offset = shift() + 0;
die "Usage: $0 OFFSET_SECONDS\n" unless $offset;
while (<>) {
chomp;
my $mtime = (stat $_)[9];
next unless $mtime;
$mtime += $offset;
utime $mtime, $mtime, $_;
}
</pre>
<hr>
<h2>Prevention</h2>
<p>
To prevent the problem in the first place you need to
prevent changing to DST. This can be done by either running
the windows system in UTC, by disabling DST and changing
the system time manually twice each year, or by avoiding the use
of the FAT filesystem (perhaps by switching to a different OS).
<h2>Notes and References</h2>
<p>
Here are some references that Wayne Piekarski collected
while researching this problem. They contain a lot of
information about the ways that Windows deals with
timestamps on NTFS and FAT filesystems.
<p>
<a href="http://optics.ph.unimelb.edu.au/help/rsync/rsync_pc1.html#gotchas">http://optics.ph.unimelb.edu.au/help/rsync/rsync_pc1.html#gotchas</a></br>
<a href="http://list-archive.xemacs.org/xemacs-nt/199911/msg00130.html">http://list-archive.xemacs.org/xemacs-nt/199911/msg00130.html</a></br>
<a href="http://p2p.wrox.com/archive/c_plus_plus_programming/2001-06/53.asp">http://p2p.wrox.com/archive/c_plus_plus_programming/2001-06/53.asp</a></br>
<a href="http://www.codeproject.com/datetime/dstbugs.asp">http://www.codeproject.com/datetime/dstbugs.asp</a></br>
<a href="http://support.microsoft.com/default.aspx?scid=kb;[LN];158588">http://support.microsoft.com/default.aspx?scid=kb;[LN];158588</a>
<p>
I wish to thank Wayne Piekarski for having copiled the
references and also supplying some additional insights.
<p>
Permission is granted without reservation reprint and
distribute this in whole and in part to any interested
parties.
<p>—J.W. Schultz