Skip to content

Commit 8f3cf74

Browse files
committed
Ignore separator columns in Data, Data2, Data3 worksheets - see HEA-688
The Data, Data2 and Data3 worksheets that contain the Livelihood Actitivies contain blank columns separating the Wealth Categories (VP, P, etc.). Sometimes these blank columns contain data where a 0 value or a formula has been copied across the row. We can detect the separator column because it won't have a Wealth Category, and then ignore any Livelihood Activities we find in it.
1 parent 2766db9 commit 8f3cf74

1 file changed

Lines changed: 5 additions & 1 deletion

File tree

pipelines/assets/livelihood_activity.py

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -420,10 +420,14 @@ def get_instances_from_dataframe(
420420
# to the list, provided that it has at least one Livelihood Activity where there is some income,
421421
# expediture or consumption. This excludes empty activities that only contain attributes for,
422422
# for example, 'type_of_milk_sold_or_other_uses'.
423+
# Also ignore any livelihood activities that don't have a Wealth Category component to the Wealth Group
424+
# natural key. These are from blank columns between Wealth Category groups in the BSS, which sometimes
425+
# contain data where values or formulae have been copied across all the columns in a row.
423426
non_empty_livelihood_activities = [
424427
livelihood_activity
425428
for livelihood_activity in livelihood_activities_for_strategy
426-
if any(
429+
if livelihood_activity["wealth_group"][2] # Make sure there is a Wealth Category
430+
and any(
427431
(
428432
field in livelihood_activity
429433
and (livelihood_activity[field] or livelihood_activity[field] == 0)

0 commit comments

Comments
 (0)