Why Your Shopify Bulk Operation Keeps Timing Out (And How to Fix It)
The Problem
You're running a Shopify bulk operation to export your product catalog. Maybe you're building a Google Shopping feed, migrating to another platform, or pulling data into a warehouse for analysis. You write a reasonable GraphQL query, kick off the bulk operation, and it works perfectly on your dev store with 500 products.
Then you point it at your production store with 10,000+ products. The bulk operation
runs for 10 minutes and times out. Or it completes but the JSONL file is suspiciously
small, missing half your catalog. You check the status field and it says
COMPLETED, but the objectCount doesn't match what you expected.
This is one of the most frustrating issues with the Shopify Bulk Operations API because it doesn't fail loudly. It just quietly returns partial data, or stalls until the operation expires. And the fix isn't obvious from the documentation.
Why It Happens
Shopify's Bulk Operations API works by taking your GraphQL query and internally paginating
through the results. You don't pass first or after cursor arguments
on the top-level connection; Shopify handles that for you. But you do control
the nested connections inside your query, and that's where the problem lives.
The most common culprit is presentmentPrices. Every product variant in
Shopify has presentment prices for each market and currency combination you've configured.
If you're selling internationally, you might have 30–50 markets set up. Now do the math:
50 markets × 10,000 products × 3 variants each = 1,500,000 price recordsThat's 1.5 million additional rows of data that your bulk operation has to traverse and serialize into the JSONL output, on top of the product and variant data itself. The bulk operation doesn't just time out from the volume. The internal query planner has to resolve each of those nested connections, and Shopify imposes an execution time limit on the entire operation.
The same problem applies to other deeply nested connections: metafields
without a limit, images on products with hundreds of media items, or
inventoryLevels across dozens of locations. Any connection that fans out
multiplicatively will blow up your execution time.
The Fix
Here's a query that looks reasonable but will time out on large catalogs:
mutation {
bulkOperationRunQuery(
query: """
{
products {
edges {
node {
id
title
handle
vendor
productType
variants {
edges {
node {
id
sku
price
inventoryQuantity
presentmentPrices {
edges {
node {
price {
amount
currencyCode
}
compareAtPrice {
amount
currencyCode
}
}
}
}
}
}
}
}
}
}
}
"""
) {
bulkOperation {
id
status
}
userErrors {
field
message
}
}
}
The presentmentPrices connection has no limit, so Shopify will try to
return every price for every market for every variant. Here's the fixed version:
mutation {
bulkOperationRunQuery(
query: """
{
products {
edges {
node {
id
title
handle
vendor
productType
variants {
edges {
node {
id
sku
price
inventoryQuantity
presentmentPrices(first: 5) {
edges {
node {
price {
amount
currencyCode
}
compareAtPrice {
amount
currencyCode
}
}
}
}
}
}
}
}
}
}
}
"""
) {
bulkOperation {
id
status
}
userErrors {
field
message
}
}
}The key changes:
1. Limit nested connections
Add first: N to any nested connection that can fan out.
presentmentPrices(first: 5) will return prices for your top 5 markets
instead of all 50. If you only sell in USD and EUR, first: 2 is plenty.
2. Only request fields you need
Don't pull descriptionHtml if you're building a price feed. Don't pull
images if you only need SKUs. Every field adds serialization time. Be
surgical with your selection set.
3. Split large catalogs into date ranges
If you genuinely need all presentment prices for all products, break the operation into smaller chunks using a query filter:
# First batch: products created before 2025
products(query: "created_at:<2025-01-01") {
...
}
# Second batch: products created in 2025
products(query: "created_at:>=2025-01-01 AND created_at:<2026-01-01") {
...
}
# Third batch: products created in 2026
products(query: "created_at:>=2026-01-01") {
...
}Run each as a separate bulk operation (one at a time, since Shopify allows only one bulk operation per app in API versions before 2026-01), then merge the results.
4. Poll with exponential backoff
Don't poll for completion every second. Bulk operations on large stores can take 5–15 minutes. Polling aggressively doesn't make them faster and it eats into your API rate limit. Use exponential backoff:
async function pollBulkOperation(client, operationId) {
let delay = 2000; // Start at 2 seconds
const maxDelay = 30000; // Cap at 30 seconds
while (true) {
const result = await client.query({
data: POLL_QUERY,
variables: { id: operationId }
});
const status = result.body.data.node.status;
if (status === 'COMPLETED') {
return result.body.data.node.url;
}
if (status === 'FAILED' || status === 'CANCELED') {
throw new Error(`Bulk operation ${status}`);
}
await sleep(delay);
delay = Math.min(delay * 1.5, maxDelay);
}
}
Alternatively, use webhooks. Register for the bulk_operations/finish
webhook topic and let Shopify tell you when it's done instead of asking repeatedly.
Going Further
If you just need to run a one-off export or build it into a script, I open-sourced a Python CLI tool that handles the full lifecycle: trigger the bulk operation, poll for completion, download the result, parse the nested JSONL, and flatten it into clean CSV or JSON. It handles all the edge cases described in this post (streaming, parent/child assembly, inventory aggregation by location, presentment price limiting).
pip install shopify-bulk
# Fetch your catalog from Shopify
shopify-bulk fetch --shop mystore.myshopify.com --token shpat_xxxxx -o export.jsonl
# Flatten to CSV
shopify-bulk process export.jsonl -c products -o catalog.csvSource and docs on GitHub: snowthen-o7/shopify-bulk
If you're doing this kind of catalog export regularly (daily feeds, ongoing syncs, warehouse ETL), you'll eventually want scheduling, transforms, and multi-destination pushes on top of the raw export. That's what SnowPipe is for: a pipeline tool that handles Shopify catalog syncs with the batching and backoff strategies baked in, so you don't have to re-discover these edge cases every time you scale to a new store.