Debug a failed feed¶

Use this guide to investigate and fix a failed product feed during validation or ETL processing. Use this workflow during partner onboarding or when troubleshooting production ingestion issues.

Overview¶

In this guide, you will:

Identify the failed feed or job
Retrieve validation and ETL processing details
Interpret error messages
Fix data issues
Re-run processing and verify results

Prerequisites¶

Base URL:

http://localhost:8000

or

http://partner-catalog-alb-1398338240.us-east-2.elb.amazonaws.com

API key:

x-api-key: demo-secret-key

A previously uploaded feed (feed_id) or job (job_id)

1. Identify the failed feed¶

Retrieve the feed to confirm its status.

Request¶

curl -X GET "http://localhost:8000/feeds/FD00009" \
  -H "x-api-key: demo-secret-key"

Response¶

{
  "feed_id": "FD00009",
  "partner_name": "Tronics",
  "file_name": "electronics_catalog.csv",
  "status": "failed",
  "validation_job_id": "JV00009",
  "validation_status": "failed",
  "validation_message": "Missing required field: sku"
}

What to look for¶

status: failed
validation_status: failed
validation_message describing the issue

2. Retrieve job details¶

Get ETL processing results for the associated job.

Request¶

curl -X GET "http://localhost:8000/jobs/JV00009" \
  -H "x-api-key: demo-secret-key"

Response¶

{
  "job_id": "JV00009",
  "status": "failed",
  "result": "Products processed: 10. Inserted: 0. Updated: 0. Unchanged: 0. Skipped: 10."
}

Interpret the result¶

All rows were skipped
No records were inserted or updated
The issue likely affects the entire file (schema or required fields)

3. Diagnose the failure¶

Use the patterns below to identify common issues.

Missing required fields¶

Symptoms

High Skipped count
Validation message references missing fields

Example

Missing required field: sku

Fix

Ensure every row includes:

sku
product_name

Invalid data format¶

Symptoms

Rows skipped or partially processed
Unexpected values in results

Examples

Non-numeric price
Invalid availability values

Fix

Validate data types and normalize values before upload.

Duplicate records¶

Symptoms

No new inserts
High Updated or Unchanged counts

Cause

Deduplication uses:

(partner_name, sku)

Fix

Use unique SKUs for new products
Confirm whether updates are expected

Partial success¶

Symptoms

Mix of Inserted, Updated, and Skipped

Interpretation

Some rows are valid
Some rows failed validation

Fix

Review skipped rows
Correct only invalid records

4. Fix the source file¶

Update the CSV file based on the identified issue.

Invalid example¶

sku,product_name,price
,4K Smart TV,799.99
TV-002,,599.99

Corrected example¶

sku,product_name,price
TV-001,4K Smart TV,799.99
TV-002,LED TV,599.99

5. Re-run ETL processing¶

After fixing the data, reprocess the feed.

Re-run the existing job¶

curl -X POST "http://localhost:8000/jobs/JV00009/run" \
  -H "x-api-key: demo-secret-key"

Upload a new feed¶

curl -X POST "http://localhost:8000/feeds/upload" \
  -H "x-api-key: demo-secret-key" \
  -F "file=@corrected_catalog.csv" \
  -F "partner_name=Tronics"

6. Verify success¶

Retrieve the job again to confirm successful ETL processing.

curl -X GET "http://localhost:8000/jobs/JV00009" \
  -H "x-api-key: demo-secret-key"

Successful response¶

{
  "job_id": "JV00009",
  "status": "completed",
  "result": "Products processed: 10. Inserted: 10. Updated: 0. Unchanged: 0. Skipped: 0."
}

Debugging checklist¶

Use this checklist when troubleshooting:

Confirm required fields (sku, product_name)
Validate data types (price, availability)
Check for duplicate SKUs
Review validation_message
Compare processed vs skipped counts
Re-run the job after fixes

How the system processes a feed¶

When a feed is processed:

Validation
Checks CSV structure and required fields
Generates validation errors
ETL processing
Transforms valid rows
Skips invalid rows
Result aggregation
Counts inserted, updated, unchanged, and skipped records
Status update
Sets job and feed status to failed or completed

Summary¶

In this workflow, you:

Identified the failed feed
Retrieved validation and job details
Diagnosed the issue
Fixed the data
Reprocessed and verified the results

This workflow reflects a typical troubleshooting process for feed ingestion failures.