Commit 510d73d
Fix feature handling in Radon and NaN checks in LAION400M.
In radon_dataset_builder.py, the feature extraction now iterates through the defined features and includes error handling for conversion failures.
In laion400m.py, NaN values for similarity and license are now checked using pd.isna() for more robust handling.
PiperOrigin-RevId: 9118600781 parent 1dac9e9 commit 510d73d
2 files changed
Lines changed: 21 additions & 11 deletions
File tree
- tensorflow_datasets
- datasets/radon
- vision_language/laion400m
Lines changed: 9 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
133 | 133 | | |
134 | 134 | | |
135 | 135 | | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
136 | 144 | | |
137 | 145 | | |
138 | | - | |
139 | | - | |
140 | | - | |
| 146 | + | |
141 | 147 | | |
Lines changed: 12 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| 17 | + | |
17 | 18 | | |
18 | 19 | | |
19 | 20 | | |
20 | 21 | | |
21 | 22 | | |
22 | | - | |
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| |||
78 | 78 | | |
79 | 79 | | |
80 | 80 | | |
| 81 | + | |
81 | 82 | | |
82 | 83 | | |
83 | 84 | | |
84 | 85 | | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
85 | 89 | | |
86 | 90 | | |
87 | 91 | | |
88 | | - | |
89 | | - | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
90 | 96 | | |
91 | 97 | | |
92 | 98 | | |
| |||
174 | 180 | | |
175 | 181 | | |
176 | 182 | | |
177 | | - | |
178 | | - | |
179 | | - | |
180 | | - | |
181 | | - | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
182 | 186 | | |
183 | 187 | | |
184 | 188 | | |
| |||
0 commit comments