How to break ForEach loop in ADF

Fasd 0 Reputation points

Hi everyone,

I have a ForEach activity containing a Copy Data task.

The Problem:

By default, if one iteration fails, the ForEach continues to trigger all other iterations. In my case, if the first one fails, the following ones are guaranteed to fail as well. I want to "exit" or "break" the loop immediately upon the first error to avoid unnecessary activity runs.

My current workaround:

To solve this, I'm using an If Condition inside the ForEach:

I check a variable status.

If status == 'Success', I run the Copy Data.

If the copy fails (red path), I set the status to 'Failed'.

This prevents the actual copy from running in subsequent iterations, but the ForEach still technically "runs" every item in the list.

My Question:

Is there a more "native" or "clean" way to stop a ForEach loop immediately when an error occurs? I'm looking to:

Avoid wrapping everything in an If Condition.

Physically stop the loop so it doesn't even "check" the remaining items.

Is there a setting I missed, or is switching to an Until activity the only true solution for a "hard break"?

Thanks!

  1. SAI JAGADEESH KUDIPUDI 3,470 Reputation points • Microsoft External Staff • Moderator

    Hi Fasd,
    in ADF today there isn’t a built-in “break” on the ForEach the way you’d have in a programming language, so you’ll need to simulate one. Here are a couple of patterns folks use:

    1. Sequential + Fail activity • Set your ForEach to isSequential=true (batchCount=1). • On your Copy’s “On Failure” path, add a Fail activity. • As soon as one iteration fails, the Fail activity aborts the pipeline and no further loop items run.
    2. Until loop pattern • Replace the ForEach with an Until activity driven by two variables: – errorFlag (boolean) – index (int) • Inside Until, check errorFlag == true or index >= arrayLength; if neither, execute your Copy/Data activity on item[index]. • On success: increment index; on failure: set errorFlag = true (and optionally Fail). • The Until will exit as soon as errorFlag flips, so you get a hard stop.
    3. ExecutePipeline + Fail • Wrap your logic in a child pipeline that takes one item as a parameter. • In the parent pipeline’s ForEach, call that child via Execute Pipeline. • In the child, on error route, use a Fail activity—this bubbles up and halts the parent run.

    If you don’t mind running everything sequentially and want a quick win, option #1 is usually the simplest. If you need more control or more complex branching, the Until-based approach gives you a true “stop on error” loop.

    Hope that gives you some options!

    Reference list

    • ForEach activity limitations & workarounds: https://learn.microsoft.com/azure/data-factory/control-flow-for-each-activity#limitations-and-workarounds

    • ForEach degree of parallelism: https://learn.microsoft.com/azure/data-factory/control-flow-for-each-activity#parallel-execution

    • Until activity guide: https://learn.microsoft.com/azure/data-factory/control-flow-until-activity

    • Fail activity: https://learn.microsoft.com/azure/data-factory/control-flow-fail-activity
    Hope this helps. If you have any follow-up questions, please let me know. I would be happy to help.

  2. Fasd 0 Reputation points

    Hello SAI JAGADEESH KUDIPUDI,

    Thanks for taking the time to answer me!

    I wanted to share my feedback:

    Option #1 (Sequential + Fail) didn't actually stop the loop in my tests; the ForEach just kept going to the next item regardless of the failure.

    More importantly, all the solutions that provide a 'clean' stop seem to force the pipeline into sequential mode. Since I am working on a real-time project, I cannot afford to lose the speed of parallelism. At this stage, I do not see a native solution for my case that combines parallelism, an immediate stop, and an accurate 'Failed' status for monitoring.

    Let me know if you think of another solution, I’m definitely interested!

    Thanks again!

  3. SAI JAGADEESH KUDIPUDI 3,470 Reputation points • Microsoft External Staff • Moderator

    Hi @Fasd ,
    I hope you had a chance to review the information shared earlier, and I hope this information has been helpful! If you still have questions, please let us know what is needed in the comments so the question can be answered.

  4. Fasd 0 Reputation points

    Hi @SAI JAGADEESH KUDIPUDI ,

    Thank you so much for your follow-up and for all the detailed explanations you provided!

    After carefully weighing the trade-offs between performance and control, I finally decided to switch to the Until activity pattern as you suggested. It turned out to be the most reliable way for me to ensure a true 'hard break' and keep an accurate monitoring status when a failure occurs.

    Thank you again for your time and for guiding me to the right solution.

Sign in to comment

2 answers

  1. SAI JAGADEESH KUDIPUDI 3,470 Reputation points • Microsoft External Staff • Moderator

    Hi @Fasd ,

    Thanks again for your detailed feedback

    You are absolutely right in what you’re seeing, and your conclusion aligns with the current design of Azure Data Factory.

    In ADF, the ForEach activity does not support a native “break” or fail-fast behavior, especially when running in parallel mode. Each iteration is treated as an independent execution, and once iterations are scheduled, they continue regardless of failures in other iterations.

    This explains both points you raised:

    • The Sequential + Fail approach may not always stop execution as expected in your scenario.
    • More importantly, parallel execution cannot be interrupted mid-run, which prevents a true immediate stop.

    Why your requirement is currently not achievable together

    At present, ADF does not provide a built-in way to combine all three of the following:

    • Parallel execution (for performance)
    • Immediate stop on first failure
    • Accurate pipeline failure state for monitoring

    This is because in parallel mode, multiple activities are already triggered asynchronously, and ADF does not support cancelling already-running iterations.

    If a strict fail-fast behavior is required, the only supported pattern today is to:

    Use an Until activity instead of ForEach

    • Control iteration using variables (index + error flag)
    • Exit immediately when a failure is detected
    • Ensure accurate pipeline failure handling

    This is the closest approach to a true “break” in ADF.

    If parallelism is a hard requirement, then your current design (status variable + If condition) is still the best practical workaround — with the understanding that it prevents further processing but does not stop iteration scheduling

    I completely understand your real-time performance concern — this is a known gap in the current ADF behavior. At the moment, choosing between parallel performance and fail-fast control is a design trade-off.

    Microsoft Reference Link :
    ForEach activity – limitations and behavior

    Hope this helps. If you have any follow-up questions, please let me know. I would be happy to help.

    1. SAI JAGADEESH KUDIPUDI 3,470 Reputation points • Microsoft External Staff • Moderator

      Hi @Fasd ,
      I hope you had a chance to review the information shared earlier, and I hope this information has been helpful! If you still have questions, please let us know what is needed in the comments so the question can be answered.

    Sign in to comment
  2. Vinodh247-1375 43,181 Reputation points • Volunteer Moderator

    Hi ,

    Thanks for reaching out to Microsoft Q&A.

    there is no native “break” or “fail-fast” option in a ForEach activity in Azure Data Factory or Fabric pipelines.

    ForEach is designed to be resilient and independent per iteration, so even on failure it will continue scheduling remaining items (especially in parallel mode). What you are doing with a status variable + If Condition is the standard workaround, but as you observed, it does not truly stop execution, it only skips logic.

    If you need a true “hard stop” (stop further iterations from even triggering), the only clean pattern today is to replace ForEach with an Until loop where you control the index and exit condition explicitly. In that design, you can fail fast and break the loop immediately when an error occurs. Another partial option is to set ForEach to sequential (batch count = 1) and let the pipeline fail on first error, but even then it does not behave like a strict break control.

    Bottom line: no native break; Until is the only proper fail-fast pattern.

    Please 'Upvote'(Thumbs-up) and 'Accept' as answer if the reply was helpful. This will be benefitting other community members who face the same issue.

    0 comments No comments
    Sign in to comment
Sign in to answer

Your answer