fix: handle hubspot sync edge cases (409 conflicts, empty line items, duplicate hubspot_id mappings)

[Anas12091101]

What are the relevant tickets?

https://github.com/mitodl/hq/issues/11149

Description

While running the sync_db_to_hubspot management command, three edge cases were causing crashes that prevented the full sync from completing.

1. batch_create_hubspot_objects_chunked — ApiException on 400/409 conflicts

When objects (products/contacts) already existed in HubSpot but the local HubspotObject mapping was missing (e.g. after a DB restore), the batch create API returned 400/409. The task re-raised these as ApiException, crashing the entire Celery chord. Non-retryable errors (400, 409, 500) are now logged and skipped. 429 (rate limit) errors still raise so Celery autoretry continues to work.

2. sync_contact_hubspot_ids_to_db — IntegrityError on duplicate hubspot_id

When HubSpot returned a contact whose hubspot_id was already mapped to a different local user (e.g. stale mapping from a user email change), update_or_create violated the unique constraint (hubspot_id, content_type_id). The function now checks for existing conflicting mappings and skips them, leaving existing data unchanged.

3. sync_deal_line_hubspot_ids_to_db — IndexError on empty line items

When a deal in HubSpot had no associated line items (B2B orders enter the is_b2b branch unconditionally), accessing line_items[0] caused an IndexError. The function now returns False early when line_items is empty.

How can this change be tested?

Each scenario can be verified end-to-end against a real HubSpot account. Ensure MITOL_HUBSPOT_API_PRIVATE_TOKEN is set in your .env. Run each script via:

docker-compose run --rm web python manage.py shell < script_name.py

On master all 3 scripts CRASH. On this branch all 3 PASS.

Scenario 1: Product batch create with 400/409 conflict

Save as test_scenario1.py:

"""
On master: CRASHES with ApiException(status=400)
On fix branch: PASSES - returns []
"""
import json
import uuid
import sys
from hubspot.crm.objects import ApiException
from mitol.hubspot_api.api import HubspotApi, HubspotObjectType
from mitol.hubspot_api.models import HubspotObject
from django.contrib.contenttypes.models import ContentType
from ecommerce.models import Product
from hubspot_xpro import tasks
from hubspot_xpro.api import make_product_sync_message

client = HubspotApi()
ct = ContentType.objects.get_for_model(Product)
product = Product.objects.first()
assert product, "No products in DB"

hs_id = None
try:
    body = make_product_sync_message(product.id)
    try:
        resp = client.crm.objects.basic_api.create(
            object_type=HubspotObjectType.PRODUCTS.value,
            simple_public_object_input_for_create=body,
        )
        hs_id = resp.id
    except ApiException as e:
        if e.status == 409:
            hs_id = json.loads(e.body)["message"].split("ID: ")[-1]
        elif "already has that value" in str(e.body):
            hs_id = json.loads(e.body)["message"].split(". ")[0].split(" on ")[-1]
        else:
            raise

    print(f"[SETUP] Product {product.id} exists in HubSpot as {hs_id}")
    HubspotObject.objects.filter(content_type=ct, object_id=product.id).delete()
    print("[SETUP] Deleted local HubspotObject mapping")

    try:
        result = tasks.batch_create_hubspot_objects_chunked(
            HubspotObjectType.PRODUCTS.value, "product", [product.id]
        )
        print(f"PASSED - batch create returned: {result}")
    except ApiException as e:
        print(f"CRASHED - ApiException(status={e.status})")
    except Exception as e:
        print(f"CRASHED - {type(e).__name__}: {e}")
finally:
    if hs_id:
        HubspotObject.objects.update_or_create(
            content_type=ct, object_id=product.id, defaults={"hubspot_id": hs_id},
        )
        print(f"[CLEANUP] Restored mapping product {product.id} -> {hs_id}")

Scenario 2: Contact sync with duplicate hubspot_id

Save as test_scenario2.py (note: runs full contact sync, may take a minute):

"""
On master: CRASHES with IntegrityError (duplicate key)
On fix branch: PASSES - sync completes
"""
import json
from django.contrib.contenttypes.models import ContentType
from django.db.utils import IntegrityError
from hubspot.crm.objects import ApiException
from mitol.hubspot_api.api import HubspotApi, HubspotObjectType
from mitol.hubspot_api.models import HubspotObject
from hubspot_xpro.api import sync_contact_hubspot_ids_to_db
from users.models import User

client = HubspotApi()
ct = ContentType.objects.get_for_model(User)

user_a = User.objects.filter(is_active=True).order_by("id")[0]
user_b = User.objects.filter(is_active=True).exclude(id=user_a.id).order_by("id")[0]
print(f"[SETUP] user_a={user_a.id} ({user_a.email}), user_b={user_b.id} ({user_b.email})")

created_contact = False
try:
    resp = client.crm.objects.basic_api.create(
        object_type=HubspotObjectType.CONTACTS.value,
        simple_public_object_input_for_create={
            "properties": {"email": user_b.email, "firstname": "E2E", "lastname": "Test"}
        },
    )
    hs_id = resp.id
    created_contact = True
except ApiException as e:
    if e.status == 409:
        hs_id = json.loads(e.body)["message"].split("Existing ID: ")[-1]
    else:
        raise
print(f"[SETUP] HubSpot contact {hs_id} has email={user_b.email}")

orig_a = HubspotObject.objects.filter(content_type=ct, object_id=user_a.id).first()
orig_a_hs_id = orig_a.hubspot_id if orig_a else None
orig_target = HubspotObject.objects.filter(content_type=ct, hubspot_id=hs_id).first()
orig_target_user_id = orig_target.object_id if orig_target else None

HubspotObject.objects.update_or_create(
    content_type=ct, object_id=user_a.id, defaults={"hubspot_id": hs_id},
)
print(f"[SETUP] Mapped hubspot_id={hs_id} to user_a={user_a.id} (conflict: email belongs to user_b)")

try:
    try:
        sync_contact_hubspot_ids_to_db()
        print("PASSED - sync completed without IntegrityError")
    except IntegrityError as e:
        print(f"CRASHED - IntegrityError: {e}")
    except Exception as e:
        print(f"CRASHED - {type(e).__name__}: {e}")
finally:
    HubspotObject.objects.filter(content_type=ct, object_id=user_a.id).delete()
    if orig_a_hs_id:
        HubspotObject.objects.update_or_create(
            content_type=ct, object_id=user_a.id, defaults={"hubspot_id": orig_a_hs_id}
        )
    if orig_target_user_id and orig_target_user_id != user_a.id:
        HubspotObject.objects.update_or_create(
            content_type=ct, object_id=orig_target_user_id, defaults={"hubspot_id": hs_id}
        )
    if created_contact:
        client.crm.objects.basic_api.archive(
            object_type=HubspotObjectType.CONTACTS.value, object_id=hs_id
        )
    print("[CLEANUP] Restored original mappings")

Scenario 3: Deal with no line items

Save as test_scenario3.py:

"""
On master: CRASHES with IndexError (list index out of range)
On fix branch: PASSES - returns False
"""
import uuid
from mitol.hubspot_api.api import HubspotApi, HubspotObjectType
from b2b_ecommerce.models import B2BOrder
from hubspot_xpro.api import sync_deal_line_hubspot_ids_to_db

client = HubspotApi()

order = B2BOrder.objects.first()
assert order, "No B2BOrder in DB"
print(f"[SETUP] Using B2BOrder id={order.id}")

resp = client.crm.objects.basic_api.create(
    object_type=HubspotObjectType.DEALS.value,
    simple_public_object_input_for_create={
        "properties": {
            "dealname": f"e2e-empty-deal-{uuid.uuid4().hex[:8]}",
            "amount": "0.00",
        }
    },
)
hs_deal_id = resp.id
print(f"[SETUP] Created HubSpot deal {hs_deal_id} with no line items")

try:
    try:
        result = sync_deal_line_hubspot_ids_to_db(order, hs_deal_id)
        print(f"PASSED - returned {result}")
    except IndexError as e:
        print(f"CRASHED - IndexError: {e}")
    except Exception as e:
        print(f"CRASHED - {type(e).__name__}: {e}")
finally:
    client.crm.objects.basic_api.archive(
        object_type=HubspotObjectType.DEALS.value, object_id=hs_deal_id
    )
    print(f"[CLEANUP] Deleted HubSpot deal {hs_deal_id}")

Verified results

Scenario	master	This branch
1. Product batch create conflict	CRASHED - ApiException(status=400)	PASSED - returned []
2. Contact duplicate hubspot_id	CRASHED - IntegrityError: duplicate key	PASSED - sync completed
3. Deal with no line items	CRASHED - IndexError: list index out of range	PASSED - returned False

Additional Context

[arslanashraf7]

@Anas12091101 is this ready for review?