Radicale search is returning multiple events instead of one

[lbt]

Using Radicale after 3.1.9 returns multiple events for a search

python-caldav 1.6.0 fails and I've created a minimal test case
https://gist.github.com/lbt/01b5932d80bc1c87eccd0695fe4189f8

This currently returns 11 events for a recurring event.

git bisect shows the problem appears to have been introduced in 513e04e

commit by @metallerok

python-caldav now has a workaround so 2.0.1 works again:
python-caldav/caldav#439
python-caldav/caldav@6f0bd41

[pbiering]

this was introduced by following PR: #1293

@lbt : is python-caldav specifying the expand property during REPORT request or @metallerok is this a bug on server side?

[metallerok]

The provided test example does not specify what event it was called for. Could you specify the event so that the error can be reproduced more accurately?

[lbt]

@pbiering caldav does specify expand in 1.6.0 but not in 2.0.1 (due to their workaround)

Running against 3.5.4

I added this to the test case:

    import logging
    import http.client
    LOGGER = logging.getLogger(__name__)
    logging.basicConfig(level=logging.DEBUG)
    http.client.HTTPConnection.debuglevel = 5

1.6.0 caldav:

DEBUG:urllib3.connectionpool:Resetting dropped connection: group.dgreaves.com
send: b'REPORT /home/df8ffba2-9c13-11ed-b6e9-1912c7a834fa/ HTTP/1.1\r\nHost: group.dgreaves.com:5232\r\nUser-Agent: python-caldav/1.6.0\r\nAccept-Encoding: gzip, deflate\r\nAccept: text/xml, text/calendar\r\nConnection: keep-alive\r\nContent-Type: application/xml; charset="utf-8"\r\nDepth: 1\r\nContent-Length: 420\r\nAuthorization: XXXXX==\r\n\r\n'
send: b'<?xml version=\'1.0\' encoding=\'utf-8\'?>\r\n<C:calendar-query xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav"><D:prop><C:calendar-data><C:expand start="20250627T183000Z" end="20250627T183001Z"/></C:calendar-data></D:prop><C:filter><C:comp-filter name="VCALENDAR"><C:comp-filter name="VEVENT"><C:time-range start="20250627T183000Z" end="20250627T183001Z"/></C:comp-filter></C:comp-filter></C:filter></C:calendar-query>'
reply: 'HTTP/1.0 207 Multi-Status\r\n'
header: Date: Sun, 29 Jun 2025 11:06:48 GMT
header: Server: WSGIServer/0.2 CPython/3.11.2
header: Content-Type: text/xml; charset=utf-8
header: Content-Encoding: gzip
header: Content-Length: 698

This returns 11 events

event: Event: https://group.dgreaves.com:5232/home/df8ffba2-9c13-11ed-b6e9-1912c7a834fa/272413CE-A0B6-4516-95F7-77C27FCAB7A4.ics
BEGIN:VCALENDAR
VERSION:2.0
PRODID:DAVx5/4.4.6-beta.1-ose ical4j/3.2.19
BEGIN:VEVENT
SUMMARY:TV Room
DTSTART:20230101T180000Z
DTEND:20230101T233000Z
DTSTAMP:20250515T182647Z
UID:272413CE-A0B6-4516-95F7-77C27FCAB7A4
SEQUENCE:11
CREATED:20230130T181142Z
RELATED-TO;RELTYPE=X-CALENDARSERVER-RECURRENCE-SET:3EF0E463-40EB-47FF-B825
 -D474CE894708
X-MOZ-GENERATION:23
END:VEVENT
END:VCALENDAR

event: Event: https://group.dgreaves.com:5232/home/df8ffba2-9c13-11ed-b6e9-1912c7a834fa/272413CE-A0B6-4516-95F7-77C27FCAB7A4.ics
BEGIN:VCALENDAR
VERSION:2.0
PRODID:DAVx5/4.4.6-beta.1-ose ical4j/3.2.19
BEGIN:VEVENT
SUMMARY:TV Room
DTSTART:20240113T183000Z
DTEND:20240113T230000Z
DTSTAMP:20250515T182647Z
UID:272413CE-A0B6-4516-95F7-77C27FCAB7A4
RECURRENCE-ID:20240113T180000Z
SEQUENCE:5
END:VEVENT
END:VCALENDAR

event: Event: https://group.dgreaves.com:5232/home/df8ffba2-9c13-11ed-b6e9-1912c7a834fa/272413CE-A0B6-4516-95F7-77C27FCAB7A4.ics
BEGIN:VCALENDAR
VERSION:2.0
PRODID:DAVx5/4.4.6-beta.1-ose ical4j/3.2.19
BEGIN:VEVENT
SUMMARY:TV Room
DTSTART:20231227T203000Z
DTEND:20231227T233000Z
DTSTAMP:20250515T182647Z
UID:272413CE-A0B6-4516-95F7-77C27FCAB7A4
RECURRENCE-ID:20231227T180000Z
SEQUENCE:3
END:VEVENT
END:VCALENDAR

event: Event: https://group.dgreaves.com:5232/home/df8ffba2-9c13-11ed-b6e9-1912c7a834fa/272413CE-A0B6-4516-95F7-77C27FCAB7A4.ics
BEGIN:VCALENDAR
VERSION:2.0
PRODID:DAVx5/4.4.6-beta.1-ose ical4j/3.2.19
BEGIN:VEVENT
SUMMARY:TV Room
DTSTART:20231126T180000Z
DTEND:20231126T223000Z
DTSTAMP:20250515T182647Z
UID:272413CE-A0B6-4516-95F7-77C27FCAB7A4
RECURRENCE-ID:20231126T180000Z
SEQUENCE:3
END:VEVENT
END:VCALENDAR

event: Event: https://group.dgreaves.com:5232/home/df8ffba2-9c13-11ed-b6e9-1912c7a834fa/272413CE-A0B6-4516-95F7-77C27FCAB7A4.ics
BEGIN:VCALENDAR
VERSION:2.0
PRODID:DAVx5/4.4.6-beta.1-ose ical4j/3.2.19
BEGIN:VEVENT
SUMMARY:TV Room
DTSTART:20231225T211500Z
DTEND:20231225T233000Z
DTSTAMP:20250515T182647Z
UID:272413CE-A0B6-4516-95F7-77C27FCAB7A4
RECURRENCE-ID:20231225T180000Z
SEQUENCE:3
END:VEVENT
END:VCALENDAR

event: Event: https://group.dgreaves.com:5232/home/df8ffba2-9c13-11ed-b6e9-1912c7a834fa/272413CE-A0B6-4516-95F7-77C27FCAB7A4.ics
BEGIN:VCALENDAR
VERSION:2.0
PRODID:DAVx5/4.4.6-beta.1-ose ical4j/3.2.19
BEGIN:VEVENT
SUMMARY:TV Room
DTSTART:20231129T173000Z
DTEND:20231129T233000Z
DTSTAMP:20250515T182647Z
UID:272413CE-A0B6-4516-95F7-77C27FCAB7A4
RECURRENCE-ID:20231129T180000Z
SEQUENCE:2
END:VEVENT
END:VCALENDAR

event: Event: https://group.dgreaves.com:5232/home/df8ffba2-9c13-11ed-b6e9-1912c7a834fa/272413CE-A0B6-4516-95F7-77C27FCAB7A4.ics
BEGIN:VCALENDAR
VERSION:2.0
PRODID:DAVx5/4.4.6-beta.1-ose ical4j/3.2.19
BEGIN:VEVENT
SUMMARY:TV Room
DTSTART:20240220T173000Z
DTEND:20240220T233000Z
DTSTAMP:20250515T182647Z
UID:272413CE-A0B6-4516-95F7-77C27FCAB7A4
RECURRENCE-ID:20240220T180000Z
SEQUENCE:5
END:VEVENT
END:VCALENDAR

event: Event: https://group.dgreaves.com:5232/home/df8ffba2-9c13-11ed-b6e9-1912c7a834fa/272413CE-A0B6-4516-95F7-77C27FCAB7A4.ics
BEGIN:VCALENDAR
VERSION:2.0
PRODID:DAVx5/4.4.6-beta.1-ose ical4j/3.2.19
BEGIN:VEVENT
SUMMARY:TV Room
DTSTART:20240310T174500Z
DTEND:20240310T233000Z
DTSTAMP:20250515T182647Z
UID:272413CE-A0B6-4516-95F7-77C27FCAB7A4
RECURRENCE-ID:20240310T180000Z
SEQUENCE:5
END:VEVENT
END:VCALENDAR

event: Event: https://group.dgreaves.com:5232/home/df8ffba2-9c13-11ed-b6e9-1912c7a834fa/272413CE-A0B6-4516-95F7-77C27FCAB7A4.ics
BEGIN:VCALENDAR
VERSION:2.0
PRODID:DAVx5/4.4.6-beta.1-ose ical4j/3.2.19
BEGIN:VEVENT
SUMMARY:TV Room
DTSTART:20240324T183000Z
DTEND:20240324T233000Z
DTSTAMP:20250515T182648Z
UID:272413CE-A0B6-4516-95F7-77C27FCAB7A4
RECURRENCE-ID:20240324T180000Z
SEQUENCE:6
END:VEVENT
END:VCALENDAR

event: Event: https://group.dgreaves.com:5232/home/df8ffba2-9c13-11ed-b6e9-1912c7a834fa/272413CE-A0B6-4516-95F7-77C27FCAB7A4.ics
BEGIN:VCALENDAR
VERSION:2.0
PRODID:DAVx5/4.4.6-beta.1-ose ical4j/3.2.19
BEGIN:VEVENT
SUMMARY:TV Room
DTSTART:20241027T173000Z
DTEND:20241027T233000Z
DTSTAMP:20250515T182648Z
UID:272413CE-A0B6-4516-95F7-77C27FCAB7A4
RECURRENCE-ID:20241027T180000Z
SEQUENCE:7
END:VEVENT
END:VCALENDAR

event: Event: https://group.dgreaves.com:5232/home/df8ffba2-9c13-11ed-b6e9-1912c7a834fa/272413CE-A0B6-4516-95F7-77C27FCAB7A4.ics
BEGIN:VCALENDAR
VERSION:2.0
PRODID:DAVx5/4.4.6-beta.1-ose ical4j/3.2.19
BEGIN:VEVENT
SUMMARY:TV Room
DTSTART:20241226T193000Z
DTEND:20241227T003000Z
DTSTAMP:20250515T182648Z
UID:272413CE-A0B6-4516-95F7-77C27FCAB7A4
RECURRENCE-ID:20241226T180000Z
SEQUENCE:10
END:VEVENT
END:VCALENDAR

2.0.1 caldav:

DEBUG:urllib3.connectionpool:Resetting dropped connection: group.dgreaves.com
send: b'REPORT /home/df8ffba2-9c13-11ed-b6e9-1912c7a834fa/ HTTP/1.1\r\nHost: group.dgreaves.com:5232\r\nUser-Agent: python-caldav/2.0.1\r\nAccept-Encoding: gzip, deflate\r\nAccept: text/xml, text/calendar\r\nConnection: keep-alive\r\nContent-Type: application/xml; charset="utf-8"\r\nDepth: 1\r\nContent-Length: 344\r\nAuthorization: XXXXX==\r\n\r\n'
send: b'<?xml version=\'1.0\' encoding=\'utf-8\'?>\r\n<C:calendar-query xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav"><D:prop><C:calendar-data/></D:prop><C:filter><C:comp-filter name="VCALENDAR"><C:comp-filter name="VEVENT"><C:time-range start="20250627T183000Z" end="20250627T183001Z"/></C:comp-filter></C:comp-filter></C:filter></C:calendar-query>'
reply: 'HTTP/1.0 207 Multi-Status\r\n'
header: Date: Sun, 29 Jun 2025 11:12:15 GMT
header: Server: WSGIServer/0.2 CPython/3.11.2
header: Content-Type: text/xml; charset=utf-8
header: Content-Encoding: gzip
header: Content-Length: 932

This returns 1 event:

event: Event: https://group.dgreaves.com:5232/home/df8ffba2-9c13-11ed-b6e9-1912c7a834fa/272413CE-A0B6-4516-95F7-77C27FCAB7A4.ics
BEGIN:VCALENDAR
VERSION:2.0
PRODID:DAVx5/4.4.6-beta.1-ose ical4j/3.2.19
BEGIN:VEVENT
SUMMARY:TV Room
DTSTART;TZID=Europe/London:20250627T180000
DTEND;TZID=Europe/London:20250627T233000
DTSTAMP:20250515T182647Z
UID:272413CE-A0B6-4516-95F7-77C27FCAB7A4
RECURRENCE-ID;TZID=Europe/London:20250627T180000
SEQUENCE:11
CREATED:20230130T181142Z
RELATED-TO;RELTYPE=X-CALENDARSERVER-RECURRENCE-SET:3EF0E463-40EB-47FF-B825
 -D474CE894708
X-MOZ-GENERATION:23
END:VEVENT
END:VCALENDAR

@metallerok Here's the event file taken from the collection.
TV_Room_event.ics.gz

[pbiering]

imho the behavior from Radicale is ok, so what is now the problem and the expectation and what means here "workaround"?

[lbt]

The bug is that Radicale seems to return 11 events for a search with 'expand' for a daily-recurring event during a 1 second time period when I would expect a single event for that days recurrence.

Is this expected?

The 'workaround' is that the python-caldav library in 2.0.1 has recognised this bug and does client-side expand which is why it works for caldav 2.0.1 and returns 1 event as expected.

[pbiering]

@metallerok - there is something very strange ongoing in current code, I (by accident from a former test) had an expand range in the future defined and then Radicale is generating events during the expand range even that the event already expired. Also the time filter looks like not working.

Tried following filters:

<?xml version="1.0" encoding="utf-8" ?>
            <C:calendar-query xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
                <D:prop>
                    <C:calendar-data>
                        <C:expand start="20250103T000000Z" end="20280105T000000Z"/>
                    </C:calendar-data>
                </D:prop>
            </C:calendar-query>

<?xml version="1.0" encoding="utf-8" ?>
            <C:calendar-query xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
                <D:prop>
                    <C:calendar-data>
                        <C:expand start="20250103T000000Z" end="20280105T000000Z"/>
                    </C:calendar-data>
                </D:prop>
                <C:filter>
                    <C:comp-filter name="VCALENDAR">
                        <C:comp-filter name="VEVENT">
                            <C:time-range start="20250103T000000Z" end="20280105T000000Z"/>
                        </C:comp-filter>
                    </C:comp-filter>
                </C:filter>
            </C:calendar-query>

[metallerok]

I haven't had time to look at it fully yet, but as far as I understand, the error is that the redefined recurrence events are considered separate, and not an extension of the main one, and therefore can be included in the result

[metallerok]

Events are created in the entire range because RRULE inside test event does not have an UNTIL parameter.

[pbiering]

Events are created in the entire range because RRULE inside test event does not have an UNTIL parameter.

this makes sense, even "InfCloud" client is rendering this original event without any end.

@lbt - are you sure that "Radicale" has an internal issue with expansion of this particular event?

[lbt]

I'm not sure I follow - Radicale should return events that match the filter:

<C:comp-filter name="VEVENT"><C:time-range
start="20250627T183000Z"
end="20250627T183001Z"/>
</C:comp-filter>

The filter asks for events in a 1 second window. There is a single event in that window.

Radicale >1.6.0 returns multiple events outside this window with DTSTART/DTEND in a different year.

Is that expected? Radicale <= 1.6.0 and returned 1 event.

(In addition the python-caldav client considers this to be a bug too and now considers Radicale's 'expand' to be broken)