Skip to content

Commit 9cd68c0

Browse files
m-filahegnertmadlener
authored
Document 'Collection' comparison with 'Container' named requirement (#598)
* Add document comparing collection with standard 'container' named requirement * Add to collection: `cbegin`, `cend`, `max_size`, `size_type`, `difference_type` required by container * Add basic tests for collection compliance with container named requirement * add missing `max_size` implementation, add collection methods and aliases to `UserDataCollection` * update documentation with container like methods and aliases * check Erasable, fix consistency, add links to reference * fix consistency, add macro to indicate checks that may need updating docs, add more checks * add short note on AllocatorAwareContainer, simplify comments * add singularity iterators check * rename pre-increment and post-increment * add comment on expression and statements in AllocatorAwareContainer * fix contextually convertible, add multipass guarantee * add placeholders for LegacyOutputIterator * add adaptors table, algorithms table * add comment on algorithms and ranges * add iterator concepts table, move iterator concepts to separate test * add tests for dereference assignment (and increment) * Return type aliases in UserDataContainer begin and end methods * add mention the rest of legacy iterators * add check iterator_category * add clarification *mutable* in std iterators vs *mutable* in podio * add checks for adaptors * add checks for move iterator adaptor * add collection vs container docs to index * add paragraph on internals * clarify container value_type --------- Co-authored-by: hegner <benedikt.hegner@cern.ch> Co-authored-by: tmadlener <thomas.madlener@desy.de>
1 parent d275460 commit 9cd68c0

8 files changed

Lines changed: 1153 additions & 5 deletions

File tree

‎doc/collections_as_container.md‎

Lines changed: 183 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,183 @@
1+
# PODIO Collection as a *Container*
2+
3+
Comparison of the PODIO `Collection`s with a C++ named requirement [*Container*](https://en.cppreference.com/w/cpp/named_req/Container).
4+
5+
The PODIO `Collection`s interface was designed to mimic the standard *Container* interface, in particular `std::vector`. Perfect compliance with the *Container* is not achieved as the `Collection`s are concerned with additional semantics such as mutable/immutable element access, associations and relations, and IO which that are not part of *Container*.
6+
7+
On the implementation level most of the differences with respect to the *Container* comes from the fact that in order to satisfy the additional semantics a `Collection` doesn't directly store [user layer objects](design.md#the-user-layer). Instead, [data layer objects](design.md#the-internal-data-layer) are stored and user layer objects are constructed and returned when needed. Similarly, the `Collection` iterators operate on the user layer objects but don't expose `Collection`'s storage directly to the users. Instead, they construct and return user layer objects when needed.
8+
In other words, a `Collection` utilizes the user layer type as a reference type instead of using plain references (`&` or `&&`) to stored data layer types.
9+
10+
As a consequence some of the **standard algorithms may not work** with PODIO `Collection` iterators. See [standard algorithm documentation](#collection-and-standard-algorithms) below.
11+
12+
The following tables list the compliance of a PODIO generated collection with the *Container* named requirement, stating which member types, interfaces, or concepts are fulfilled and which are not. Additionally, there are some comments explaining missing parts or pointing out differences in behaviour.
13+
14+
### Container Types
15+
16+
| Name | Type | Requirements | Fulfilled by Collection? | Comment |
17+
|------|------|--------------|--------------------------|---------|
18+
| `value_type` | `T` | *[Erasable](https://en.cppreference.com/w/cpp/named_req/Erasable)* | ✔️ yes | Defined as an immutable user layer object type |
19+
| `reference` | `T&` | | ❌ no | Not defined |
20+
| `const_reference` | `const T&` | | ❌ no | Not defined |
21+
| `iterator` | Iterator whose `value_type` is `T` | [*LegacyForwardIterator*](https://en.cppreference.com/w/cpp/named_req/ForwardIterator) convertible to `const_iterator` | ❌ no | Defined as podio `MutableCollectionIterator`. `iterator::value_type` not defined, not [*LegacyForwardIterator*](https://en.cppreference.com/w/cpp/named_req/ForwardIterator) ([see below](#legacyforwarditerator)), not convertible to `const_iterator`|
22+
| `const_iterator` | Constant iterator whose `value_type` is `T` | [*LegacyForwardIterator*](https://en.cppreference.com/w/cpp/named_req/ForwardIterator) | ❌ no | Defined as podio `CollectionIterator`. `const_iterator::value_type` not defined, not [*LegacyForwardIterator*](https://en.cppreference.com/w/cpp/named_req/ForwardIterator) ([see below](#legacyforwarditerator))
23+
| `difference_type`| Signed integer | Must be the same as `std::iterator_traits::difference_type` for `iterator` and `const_iterator` | ❌ no | `std::iterator_traits::difference_type` not defined |
24+
| `size_type` | Unsigned integer | Large enough to represent all positive values of `difference_type` | ✔️ yes | |
25+
26+
### Container member functions and operators
27+
28+
| Expression | Return type | Semantics | Fulfilled by Collection? | Comment |
29+
|------------|-------------|-----------|--------------------------|---------|
30+
| `C()` | `C` | Creates an empty container | ✔️ yes | |
31+
| `C(a)` | `C` | Creates a copy of `a` | ❌ no | Not defined, non-copyable by design |
32+
| `C(rv)` | `C` | Moves `rv` | ✔️ yes | |
33+
| `a = b` | `C&` | Destroys or copy-assigns all elements of `a` from elements of `b` | ❌ no | Not defined, non-copyable by design |
34+
| `a = rv` | `C&` | Destroys or move-assigns all elements of `a` from elements of `rv` | ✔️ yes | |
35+
| `a.~C()` | `void` | Destroys all elements of `a` and frees all memory| ✔️ yes | Invalidates all handles retrieved from this collection |
36+
| `a.begin()` | `(const_)iterator` | Iterator to the first element of `a` | ✔️ yes | |
37+
| `a.end()` | `(const_)iterator` | Iterator to one past the last element of `a` | ✔️ yes | |
38+
| `a.cbegin()` | `const_iterator` | Same as `const_cast<const C&>(a).begin()` | ✔️ yes | |
39+
| `a.cend()` | `const_iterator` | Same as `const_cast<const C&>(a).end()`| ✔️ yes | |
40+
| `a == b` | Convertible to `bool` | Same as `std::equal(a.begin(), a.end(), b.begin(), b.end())`| ❌ no | Not defined |
41+
| `a != b` | Convertible to `bool` | Same as `!(a == b)` | ❌ no | Not defined |
42+
| `a.swap(b)` | `void` | Exchanges the values of `a` and `b` | ❌ no | Not defined |
43+
| `swap(a,b)` | `void` | Same as `a.swap(b)` | ❌ no | `a.swap(b)` not defined |
44+
| `a.size()` | `size_type` | Same as `std::distance(a.begin(), a.end())` | ✔️ yes | |
45+
| `a.max_size()` | `size_type` | `b.size()` where b is the largest possible container | ✔️ yes | |
46+
| `a.empty()` | Convertible to `bool` | Same as `a.begin() == a.end()` | ✔️ yes | |
47+
48+
## Collection as an *AllocatorAwareContainer*
49+
50+
The C++ standard specifies [AllocatorAwareContainer](https://en.cppreference.com/w/cpp/named_req/AllocatorAwareContainer) for containers that can use other allocators beside the default allocator.
51+
52+
PODIO collections don't provide a customization point for allocators and use only the default allocator. Therefore they are not *AllocatorAwareContainers*.
53+
54+
### AllocatorAwareContainer types
55+
56+
| Name | Requirements | Fulfilled by Collection? | Comment |
57+
|------|--------------|--------------------------|---------|
58+
| `allocator_type` | `allocator_type::value_type` same as `value_type` | ❌ no | `allocator_type` not defined |
59+
60+
### *AllocatorAwareContainer* expression and statements
61+
62+
The PODIO Collections currently are not checked against expression and statements requirements for *AllocatorAwareContainer*.
63+
64+
## Collection iterators as an *Iterator*
65+
66+
The C++ specifies a set of named requirements for iterators. Starting with C++20 the standard specifies also iterator concepts. The requirements imposed by the concepts and named requirements are similar but not identical.
67+
68+
In the following tables a convention from `Collection` is used: `iterator` stands for PODIO `MutableCollectionIterator` and `const_iterator` stands for PODIO `CollectionIterator`.
69+
### Iterator summary
70+
71+
| Named requirement | `iterator` | `const_iterator` |
72+
|-------------------|-----------------------|-----------------------------|
73+
| [LegacyIterator](https://en.cppreference.com/w/cpp/named_req/Iterator) | ❌ no ([see below](#legacyiterator)) | ❌ no ([see below](#legacyiterator)) |
74+
| [LegacyInputIterator](https://en.cppreference.com/w/cpp/named_req/InputIterator) | ❌ no ([see below](#legacyinputiterator)) | ❌ no ([see below](#legacyinputiterator)) |
75+
| [LegacyForwardIterator](https://en.cppreference.com/w/cpp/named_req/ForwardIterator) | ❌ no ([see below](#legacyforwarditerator)) | ❌ no ([see below](#legacyforwarditerator)) |
76+
| [LegacyOutputIterator](https://en.cppreference.com/w/cpp/named_req/OutputIterator) | ❌ no ([see below](#legacyoutputiterator)) | ❌ no ([see below](#legacyoutputiterator)) |
77+
| [LegacyBidirectionalIterator](https://en.cppreference.com/w/cpp/named_req/BidirectionalIterator) | ❌ no | ❌ no |
78+
| [LegacyRandomAccessIterator](https://en.cppreference.com/w/cpp/named_req/RandomAccessIterator) | ❌ no | ❌ no |
79+
| [LegacyContiguousIterator](https://en.cppreference.com/w/cpp/named_req/ContiguousIterator) | ❌ no | ❌ no |
80+
81+
| Concept | `iterator` | `const_iterator` |
82+
|---------|------------------------|------------------------------|
83+
| `std::indirectly_readable` | ❌ no | ❌ no |
84+
| `std::indirectly_writable` | ❌ no | ❌ no |
85+
| `std::weakly_incrementable` | ❌ no | ❌ no |
86+
| `std::incrementable` | ❌ no | ❌ no |
87+
| `std::input_or_output_iterator` | ❌ no | ❌ no |
88+
| `std::input_iterator` | ❌ no | ❌ no |
89+
| `std::output_iterator` | ❌ no | ❌ no |
90+
| `std::forward_iterator` | ❌ no | ❌ no |
91+
| `std::bidirectional_iterator` | ❌ no | ❌ no |
92+
| `std::random_access_iterator` | ❌ no | ❌ no |
93+
| `std::contiguous_iterator` | ❌ no | ❌ no |
94+
95+
### LegacyIterator
96+
97+
| Requirement | Fulfilled by `iterator`/`const_iterator`? | Comment |
98+
|-------------|-------------------------------------------|---------|
99+
| [*CopyConstructible*](https://en.cppreference.com/w/cpp/named_req/CopyConstructible) | ❌ no / ❌ no | Move constructor and copy constructor not defined |
100+
| [*CopyAssignable*](https://en.cppreference.com/w/cpp/named_req/CopyAssignable) | ❌ no / ❌ no | Move assignment and copy assignment not defined |
101+
| [*Destructible*](https://en.cppreference.com/w/cpp/named_req/Destructible) | ✔️ yes / ✔️ yes | |
102+
| [*Swappable*](https://en.cppreference.com/w/cpp/named_req/Swappable) | ❌ no / ❌ no | |
103+
| `std::iterator_traits::value_type` (Until C++20 ) | ❌ no / ❌ no | Not defined |
104+
| `std::iterator_traits::difference_type` | ❌ no / ❌ no | Not defined |
105+
| `std::iterator_traits::reference` | ❌ no / ❌ no | Not defined |
106+
| `std::iterator_traits::pointer` | ❌ no / ❌ no | Not defined |
107+
| `std::iterator_traits::iterator_category` | ❌ no / ❌ no | Not defined |
108+
109+
| Expression | Return type | Semantics | Fulfilled by `iterator`/`const_iterator`? | Comment |
110+
|------------|-------------|-----------|-------------------------------------------|---------|
111+
| `*r` | Unspecified | | ✔️ yes / ✔️ yes | |
112+
| `++r` | `It&` | | ✔️ yes / ✔️ yes | |
113+
114+
### LegacyInputIterator
115+
116+
| Requirement | Fulfilled by `iterator`/`const_iterator`? | Comment |
117+
|-------------|-------------------------------------------|---------|
118+
| [*LegacyIterator*](https://en.cppreference.com/w/cpp/named_req/Iterator) | ❌ no / ❌ no | [See above](#legacyiterator) |
119+
| [*EqualityComparable*](https://en.cppreference.com/w/cpp/named_req/EqualityComparable) | ✔️ yes / ✔️ yes | |
120+
121+
| Expression | Return type | Semantics | Fulfilled by `iterator`/`const_iterator`? | Comment |
122+
|------------|-------------|-----------|-------------------------------------------|---------|
123+
| `i != j` | Contextually convertible to `bool` | Same as `!(i==j)` | ✔️ yes / ✔️ yes | |
124+
| `*i` | `reference`, convertible to `value_type` | | ❌ no / ❌ no | `reference` and `value_type` not defined |
125+
| `i->m` | | Same as `(*i).m` | ✔️ yes / ✔️ yes | |
126+
| `++r` | `It&` | | ✔️ yes / ✔️ yes | |
127+
| `(void)r++` | | Same as `(void)++r` | ❌ no / ❌ no | Post-increment not defined |
128+
| `*r++` | Convertible to `value_type` | Same as `value_type x = *r; ++r; return x;` | ❌ no / ❌ no | Post-increment and `value_type` not defined |
129+
130+
### LegacyForwardIterator
131+
132+
In addition to the *LegacyForwardIterator* the C++ standard specifies also the *mutable LegacyForwardIterator*, which is both *LegacyForwardIterator* and *LegacyOutputIterator*. The term **mutable** used in this context doesn't imply mutability in the sense used in the PODIO.
133+
134+
135+
| Requirement | Fulfilled by `iterator`/`const_iterator`? | Comment |
136+
|-------------|-------------------------------------------|---------|
137+
| [*LegacyInputIterator*](https://en.cppreference.com/w/cpp/named_req/InputIterator) | ❌ no / ❌ no | [See above](#legacyinputiterator)|
138+
| [*DefaultConstructible*](https://en.cppreference.com/w/cpp/named_req/DefaultConstructible) | ❌ no / ❌ no | Value initialization not defined |
139+
| If *mutable* iterator then `reference` same as `value_type&` or `value_type&&`, otherwise same as `const value_type&` or `const value_type&&` | ❌ no / ❌ no | `reference` and `value_type` not defined |
140+
| [Multipass guarantee](https://en.cppreference.com/w/cpp/named_req/ForwardIterator) | ❌ no / ❌ no | Copy constructor not defined |
141+
| [Singular iterators](https://en.cppreference.com/w/cpp/named_req/ForwardIterator) | ❌ no / ❌ no | Value initialization not defined |
142+
143+
| Expression | Return type | Semantics | Fulfilled by `iterator`/`const_iterator`? | Comment |
144+
|------------|-------------|-----------|-------------------------------------------|---------|
145+
| `i++` | `It` | Same as `It ip = i; ++i; return ip;` | ❌ no / ❌ no | Post-increment not defined |
146+
| `*i++` | `reference` | | ❌ no / ❌ no | Post-increment and `reference` not defined |
147+
148+
### LegacyOutputIterator
149+
150+
| Requirement | Fulfilled by `iterator`/`const_iterator`? | Comment |
151+
|-------------|-------------------------------------------|---------|
152+
| [*LegacyIterator*](https://en.cppreference.com/w/cpp/named_req/Iterator) | ❌ no / ❌ no | [See above](#legacyiterator) |
153+
| Is pointer type or class type | ✔️ yes / ✔️ yes | |
154+
155+
| Expression | Return type | Semantics | Fulfilled by `iterator`/`const_iterator`? | Comment |
156+
|------------|-------------|-----------|-------------------------------------------|---------|
157+
| `*r = o` | | | ❗ attention / ❗ attention | Defined but an assignment doesn't modify objects inside collection |
158+
| `++r` | `It&` | | ✔️ yes / ✔️ yes | |
159+
| `r++` | Convertible to `const It&` | Same as `It temp = r; ++r; return temp;` | ❌ no / ❌ no | Post-increment not defined |
160+
| `*r++ = o` | | Same as `*r = o; ++r;`| ❌ no / ❌ no | Post-increment not defined |
161+
162+
## Collection iterators and standard iterator adaptors
163+
164+
| Adaptor | Compatible with Collection? | Comment |
165+
|---------|-----------------------------|---------|
166+
| `std::reverse_iterator` | ❌ no | `iterator` and `const_iterator` not *LegacyBidirectionalIterator* or `std::bidirectional_iterator` |
167+
| `std::back_insert_iterator` | ❗ attention | Compatible only with SubsetCollections, otherwise throws `std::invalid_argument` |
168+
| `std::front_insert_iterator` | ❌ no | `push_front` not defined |
169+
| `std::insert_iterator` | ❌ no | `insert` not defined |
170+
| `std::const_iterator` | ❌ no | `iterator` and `const_iterator` not *LegacyInputIterator* or `std::input_iterator` |
171+
| `std::move_iterator` | ❌ no | `iterator` and `const_iterator` not *LegacyInputIterator* or `std::input_iterator` |
172+
| `std::counted_iterator` | ❌ no | `iterator` and `const_iterator` not `std::input_or_output_iterator` |
173+
174+
175+
## Collection and standard algorithms
176+
177+
Most of the standard algorithms require the iterators to be at least *InputIterator*. The iterators of PODIO collection don't fulfil this requirement, therefore they are not compatible with standard algorithms according to the specification.
178+
179+
In practice, some algorithms may still compile with the collections depending on the implementation of a given algorithm. In general, the standard **algorithms mutating a collection will give wrong results**, while the standard algorithms not mutating a collection in principle should give correct results if they compile.
180+
181+
## Standard range algorithms
182+
183+
The standard range algorithm use constrains to operate at least on `std::input_iterator`s and `std::ranges::input_range`s. The iterators of PODIO collection don't model these concepts, therefore can't be used with standard range algorithms. The range algorithms won't compile with PODIO `Collection` iterators.

‎doc/index.rst‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -19,5 +19,6 @@ Welcome to PODIO's documentation!
1919
advanced_topics.md
2020
templates.md
2121
python.md
22+
collections_as_container.md
2223
cpp_api/api
2324
py_api/modules

‎include/podio/CollectionBase.h‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,9 @@ class CollectionBase {
5555
/// number of elements in the collection
5656
virtual size_t size() const = 0;
5757

58+
/// maximal number of elements in the collection
59+
virtual std::size_t max_size() const = 0;
60+
5861
/// Is the collection empty
5962
virtual bool empty() const = 0;
6063

‎include/podio/UserDataCollection.h‎

Lines changed: 21 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,12 @@ class UserDataCollection : public CollectionBase {
7777
VectorMembersInfo m_vecmem_info{};
7878

7979
public:
80+
using value_type = typename std::vector<BasicType>::value_type;
81+
using const_iterator = typename std::vector<BasicType>::const_iterator;
82+
using iterator = typename std::vector<BasicType>::iterator;
83+
using difference_type = typename std::vector<BasicType>::difference_type;
84+
using size_type = typename std::vector<BasicType>::size_type;
85+
8086
UserDataCollection() = default;
8187
/// Constructor from an existing vector (which will be moved from!)
8288
UserDataCollection(std::vector<BasicType>&& vec) : _vec(std::move(vec)) {
@@ -133,6 +139,11 @@ class UserDataCollection : public CollectionBase {
133139
return _vec.size();
134140
}
135141

142+
/// maximal number of elements in the collection
143+
size_t max_size() const override {
144+
return _vec.max_size();
145+
}
146+
136147
/// Is the collection empty
137148
bool empty() const override {
138149
return _vec.empty();
@@ -194,18 +205,24 @@ class UserDataCollection : public CollectionBase {
194205

195206
// ----- some wrappers for std::vector and access to the complete std::vector (if really needed)
196207

197-
typename std::vector<BasicType>::iterator begin() {
208+
iterator begin() {
198209
return _vec.begin();
199210
}
200-
typename std::vector<BasicType>::iterator end() {
211+
iterator end() {
201212
return _vec.end();
202213
}
203-
typename std::vector<BasicType>::const_iterator begin() const {
214+
const_iterator begin() const {
204215
return _vec.begin();
205216
}
206-
typename std::vector<BasicType>::const_iterator end() const {
217+
const_iterator end() const {
207218
return _vec.end();
208219
}
220+
const_iterator cbegin() const {
221+
return _vec.cbegin();
222+
}
223+
const_iterator cend() const {
224+
return _vec.cend();
225+
}
209226

210227
typename std::vector<BasicType>::reference operator[](size_t idx) {
211228
return _vec[idx];

‎python/templates/Collection.cc.jinja2‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,10 @@ std::size_t {{ collection_type }}::size() const {
5959
return m_storage.entries.size();
6060
}
6161

62+
std::size_t {{ collection_type }}::max_size() const {
63+
return m_storage.entries.max_size();
64+
}
65+
6266
bool {{ collection_type }}::empty() const {
6367
return m_storage.entries.empty();
6468
}

‎python/templates/Collection.h.jinja2‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -51,6 +51,8 @@ public:
5151
using value_type = {{ class.bare_type }};
5252
using const_iterator = {{ class.bare_type }}CollectionIterator;
5353
using iterator = {{ class.bare_type }}MutableCollectionIterator;
54+
using difference_type = ptrdiff_t;
55+
using size_type = size_t;
5456

5557
{{ class.bare_type }}Collection();
5658
{{ class.bare_type }}Collection({{ class.bare_type }}CollectionData&& data, bool isSubsetColl);
@@ -86,6 +88,9 @@ public:
8688
/// number of elements in the collection
8789
std::size_t size() const final;
8890

91+
/// maximal number of elements in the collection
92+
std::size_t max_size() const final;
93+
8994
/// Is the collection empty
9095
bool empty() const final;
9196

@@ -153,12 +158,18 @@ public:
153158
const_iterator begin() const {
154159
return const_iterator(0, &m_storage.entries);
155160
}
161+
const_iterator cbegin() const {
162+
return begin();
163+
}
156164
iterator end() {
157165
return iterator(m_storage.entries.size(), &m_storage.entries);
158166
}
159167
const_iterator end() const {
160168
return const_iterator(m_storage.entries.size(), &m_storage.entries);
161169
}
170+
const_iterator cend() const {
171+
return end();
172+
}
162173

163174
{% for member in Members %}
164175
std::vector<{{ member.full_type }}> {{ member.name }}(const size_t nElem = 0) const;

‎tests/unittests/CMakeLists.txt‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ if(NOT Catch2_FOUND)
4040
endif()
4141

4242
find_package(Threads REQUIRED)
43-
add_executable(unittest_podio unittest.cpp frame.cpp buffer_factory.cpp interface_types.cpp)
43+
add_executable(unittest_podio unittest.cpp frame.cpp buffer_factory.cpp interface_types.cpp std_interoperability.cpp)
4444
target_link_libraries(unittest_podio PUBLIC TestDataModel PRIVATE Catch2::Catch2WithMain Threads::Threads podio::podioRootIO)
4545
if (ENABLE_SIO)
4646
target_link_libraries(unittest_podio PRIVATE podio::podioSioIO)

0 commit comments

Comments
 (0)