Commit bc798e1
Fix feature handling in Radon and NaN checks in LAION400M.
In radon_dataset_builder.py, the feature extraction now iterates through the defined features and includes error handling for conversion failures.
In laion400m.py, NaN values for similarity and license are now checked using pd.isna() for more robust handling.
PiperOrigin-RevId: 9117445801 parent 1dac9e9 commit bc798e1
2 files changed
Lines changed: 21 additions & 11 deletions
File tree
- tensorflow_datasets
- datasets/radon
- vision_language/laion400m
Lines changed: 9 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
133 | 133 | | |
134 | 134 | | |
135 | 135 | | |
| 136 | + | |
| 137 | + | |
| 138 | + | |
| 139 | + | |
| 140 | + | |
| 141 | + | |
| 142 | + | |
| 143 | + | |
136 | 144 | | |
137 | 145 | | |
138 | | - | |
139 | | - | |
140 | | - | |
| 146 | + | |
141 | 147 | | |
Lines changed: 12 additions & 8 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
| 17 | + | |
17 | 18 | | |
18 | 19 | | |
19 | 20 | | |
20 | 21 | | |
21 | 22 | | |
22 | | - | |
23 | 23 | | |
24 | 24 | | |
25 | 25 | | |
| |||
78 | 78 | | |
79 | 79 | | |
80 | 80 | | |
| 81 | + | |
81 | 82 | | |
82 | 83 | | |
83 | 84 | | |
84 | 85 | | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
85 | 89 | | |
86 | 90 | | |
87 | 91 | | |
88 | | - | |
89 | | - | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
90 | 96 | | |
91 | 97 | | |
92 | 98 | | |
| |||
174 | 180 | | |
175 | 181 | | |
176 | 182 | | |
177 | | - | |
178 | | - | |
179 | | - | |
180 | | - | |
181 | | - | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
182 | 186 | | |
183 | 187 | | |
184 | 188 | | |
| |||
0 commit comments