Note
Access to this page requires authorization. You can try signing in or .
Access to this page requires authorization. You can try .
Microsoft.IoTOperations instances/dataflowProfiles/dataflows
Bicep resource definition
The instances/dataflowProfiles/dataflows resource type can be deployed with operations that target:
- Resource groups - See resource group deployment commands
For a list of changed properties in each API version, see change log.
Resource format
To create a Microsoft.IoTOperations/instances/dataflowProfiles/dataflows resource, add the following Bicep to your template.
resource symbolicname 'Microsoft.IoTOperations/instances/dataflowProfiles/dataflows@2026-03-01' = {
parent: resourceSymbolicName
extendedLocation: {
name: 'string'
type: 'string'
}
name: 'string'
properties: {
mode: 'string'
operations: [
{
builtInTransformationSettings: {
datasets: [
{
description: 'string'
expression: 'string'
inputs: [
'string'
]
key: 'string'
schemaRef: 'string'
}
]
filter: [
{
description: 'string'
expression: 'string'
inputs: [
'string'
]
type: 'string'
}
]
map: [
{
description: 'string'
expression: 'string'
inputs: [
'string'
]
output: 'string'
type: 'string'
}
]
schemaRef: 'string'
serializationFormat: 'string'
}
destinationSettings: {
dataDestination: 'string'
endpointRef: 'string'
headers: [
{
actionType: 'string'
// For remaining properties, see DataflowDestinationHeaderAction objects
}
]
}
name: 'string'
operationType: 'string'
sourceSettings: {
assetRef: 'string'
dataSources: [
'string'
]
endpointRef: 'string'
schemaRef: 'string'
serializationFormat: 'string'
}
}
]
requestDiskPersistence: 'string'
}
}
DataflowDestinationHeaderAction objects
Set the actionType property to specify the type of object.
For AddIfNotPresent, use:
{
actionType: 'AddIfNotPresent'
key: 'string'
value: 'string'
}
For AddOrReplace, use:
{
actionType: 'AddOrReplace'
key: 'string'
value: 'string'
}
For Remove, use:
{
actionType: 'Remove'
key: 'string'
}
Property Values
Microsoft.IoTOperations/instances/dataflowProfiles/dataflows
| Name | Description | Value |
|---|---|---|
| extendedLocation | Edge location of the resource. | ExtendedLocation |
| name | The resource name | string Constraints: Min length = 3 Max length = 63 Pattern = ^[a-z0-9][a-z0-9-]*[a-z0-9]$ (required) |
| parent | In Bicep, you can specify the parent resource for a child resource. You only need to add this property when the child resource is declared outside of the parent resource. For more information, see Child resource outside parent resource. |
Symbolic name for resource of type: instances/dataflowProfiles |
| properties | The resource-specific properties for this resource. | DataflowProperties |
DataflowBuiltInTransformationDataset
| Name | Description | Value |
|---|---|---|
| description | A user provided optional description of the dataset. | string Constraints: Max length = 256 |
| expression | Condition to enrich data from Broker State Store. Example: $1 < 0 || $1 > $2 (Assuming inputs section $1 and $2 are provided) | string |
| inputs | List of fields for enriching from the Broker State Store. | string[] (required) |
| key | The key of the dataset. | string Constraints: Max length = 64 (required) |
| schemaRef | The reference to the schema that describes the dataset. Allowed: JSON Schema/draft-7. | string |
DataflowBuiltInTransformationFilter
| Name | Description | Value |
|---|---|---|
| description | A user provided optional description of the filter. | string Constraints: Max length = 256 |
| expression | Condition to filter data. Can reference input fields with {n} where n is the index of the input field starting from 1. Example: $1 < 0 || $1 > $2 (Assuming inputs section $1 and $2 are provided) | string (required) |
| inputs | List of fields for filtering in JSON path expression. | string[] (required) |
| type | The type of dataflow operation. | 'Filter' |
DataflowBuiltInTransformationMap
| Name | Description | Value |
|---|---|---|
| description | A user provided optional description of the mapping function. | string Constraints: Max length = 256 |
| expression | Modify the inputs field(s) to the final output field. Example: $1 * 2.2 (Assuming inputs section $1 is provided) | string |
| inputs | List of fields for mapping in JSON path expression. | string[] (required) |
| output | Where and how the input fields to be organized in the output record. | string (required) |
| type | Type of transformation. | 'BuiltInFunction' 'Compute' 'NewProperties' 'PassThrough' 'Rename' |
DataflowBuiltInTransformationSettings
| Name | Description | Value |
|---|---|---|
| datasets | Enrich data from Broker State Store. Dataset references a key in Broker State Store. | DataflowBuiltInTransformationDataset[] |
| filter | Filters input record or datapoints based on condition. | DataflowBuiltInTransformationFilter[] |
| map | Maps input to output message. | DataflowBuiltInTransformationMap[] |
| schemaRef | Reference to the schema that describes the output of the transformation. | string |
| serializationFormat | Serialization format. Optional; defaults to JSON. Allowed value JSON Schema/draft-7, Parquet. Default: Json | 'Delta' 'Json' 'Parquet' |
DataflowDestinationAddIfNotPresentHeaderAction
| Name | Description | Value |
|---|---|---|
| actionType | The type of header operation to perform. | 'AddIfNotPresent' (required) |
| key | The name of the header to add. | string (required) |
| value | The value of the header to add. | string (required) |
DataflowDestinationAddOrReplaceHeaderAction
| Name | Description | Value |
|---|---|---|
| actionType | The type of header operation to perform. | 'AddOrReplace' (required) |
| key | The name of the header to add or replace. | string (required) |
| value | The value of the header to add or replace. | string (required) |
DataflowDestinationHeaderAction
| Name | Description | Value |
|---|---|---|
| actionType | Set to 'AddIfNotPresent' for type DataflowDestinationAddIfNotPresentHeaderAction. Set to 'AddOrReplace' for type DataflowDestinationAddOrReplaceHeaderAction. Set to 'Remove' for type DataflowDestinationRemoveHeaderAction. | 'AddIfNotPresent' 'AddOrReplace' 'Remove' (required) |
DataflowDestinationOperationSettings
| Name | Description | Value |
|---|---|---|
| dataDestination | Destination location, can be a topic or table name. Supports dynamic values with $topic, $systemProperties, $userProperties, $payload, $context, and $subscription. | string Constraints: Min length = 1 (required) |
| endpointRef | Reference to the Endpoint CR. Can be of Broker, Kafka, Fabric, ADLS, ADX type. | string (required) |
| headers | Headers for the output data. | DataflowDestinationHeaderAction[] |
DataflowDestinationRemoveHeaderAction
| Name | Description | Value |
|---|---|---|
| actionType | The type of header operation to perform. | 'Remove' (required) |
| key | The name of the header to remove. | string (required) |
DataflowOperation
| Name | Description | Value |
|---|---|---|
| builtInTransformationSettings | Built In Transformation configuration. | DataflowBuiltInTransformationSettings |
| destinationSettings | Destination configuration. | DataflowDestinationOperationSettings |
| name | Optional user provided name of the transformation. | string Constraints: Max length = 64 |
| operationType | Type of operation. | 'BuiltInTransformation' 'Destination' 'Source' (required) |
| sourceSettings | Source configuration. | DataflowSourceOperationSettings |
DataflowProperties
| Name | Description | Value |
|---|---|---|
| mode | Mode for Dataflow. Optional; defaults to Enabled. | 'Disabled' 'Enabled' |
| operations | List of operations including source and destination references as well as transformation. | DataflowOperation[] (required) |
| requestDiskPersistence | Disk persistence mode. | 'Disabled' 'Enabled' |
DataflowSourceOperationSettings
| Name | Description | Value |
|---|---|---|
| assetRef | Reference to the resource in Azure Device Registry where the data in the endpoint originates from. | string |
| dataSources | List of source locations. Can be Broker or Kafka topics. Supports wildcards # and +. | string Constraints: Min length = 1[] (required) |
| endpointRef | Reference to the Dataflow Endpoint resource. Can only be of Broker and Kafka type. | string (required) |
| schemaRef | Schema CR reference. Data will be deserialized according to the schema, and dropped if it doesn't match. | string |
| serializationFormat | Content is a JSON Schema. Allowed: JSON Schema/draft-7. | 'Json' |
ExtendedLocation
| Name | Description | Value |
|---|---|---|
| name | The name of the extended location. | string (required) |
| type | Type of ExtendedLocation. | 'CustomLocation' (required) |
ARM template resource definition
The instances/dataflowProfiles/dataflows resource type can be deployed with operations that target:
- Resource groups - See resource group deployment commands
For a list of changed properties in each API version, see change log.
Resource format
To create a Microsoft.IoTOperations/instances/dataflowProfiles/dataflows resource, add the following JSON to your template.
{
"type": "Microsoft.IoTOperations/instances/dataflowProfiles/dataflows",
"apiVersion": "2026-03-01",
"name": "string",
"extendedLocation": {
"name": "string",
"type": "string"
},
"properties": {
"mode": "string",
"operations": [
{
"builtInTransformationSettings": {
"datasets": [
{
"description": "string",
"expression": "string",
"inputs": [ "string" ],
"key": "string",
"schemaRef": "string"
}
],
"filter": [
{
"description": "string",
"expression": "string",
"inputs": [ "string" ],
"type": "string"
}
],
"map": [
{
"description": "string",
"expression": "string",
"inputs": [ "string" ],
"output": "string",
"type": "string"
}
],
"schemaRef": "string",
"serializationFormat": "string"
},
"destinationSettings": {
"dataDestination": "string",
"endpointRef": "string",
"headers": [ {
"actionType": "string"
// For remaining properties, see DataflowDestinationHeaderAction objects
} ]
},
"name": "string",
"operationType": "string",
"sourceSettings": {
"assetRef": "string",
"dataSources": [ "string" ],
"endpointRef": "string",
"schemaRef": "string",
"serializationFormat": "string"
}
}
],
"requestDiskPersistence": "string"
}
}
DataflowDestinationHeaderAction objects
Set the actionType property to specify the type of object.
For AddIfNotPresent, use:
{
"actionType": "AddIfNotPresent",
"key": "string",
"value": "string"
}
For AddOrReplace, use:
{
"actionType": "AddOrReplace",
"key": "string",
"value": "string"
}
For Remove, use:
{
"actionType": "Remove",
"key": "string"
}
Property Values
Microsoft.IoTOperations/instances/dataflowProfiles/dataflows
| Name | Description | Value |
|---|---|---|
| apiVersion | The api version | '2026-03-01' |
| extendedLocation | Edge location of the resource. | ExtendedLocation |
| name | The resource name | string Constraints: Min length = 3 Max length = 63 Pattern = ^[a-z0-9][a-z0-9-]*[a-z0-9]$ (required) |
| properties | The resource-specific properties for this resource. | DataflowProperties |
| type | The resource type | 'Microsoft.IoTOperations/instances/dataflowProfiles/dataflows' |
DataflowBuiltInTransformationDataset
| Name | Description | Value |
|---|---|---|
| description | A user provided optional description of the dataset. | string Constraints: Max length = 256 |
| expression | Condition to enrich data from Broker State Store. Example: $1 < 0 || $1 > $2 (Assuming inputs section $1 and $2 are provided) | string |
| inputs | List of fields for enriching from the Broker State Store. | string[] (required) |
| key | The key of the dataset. | string Constraints: Max length = 64 (required) |
| schemaRef | The reference to the schema that describes the dataset. Allowed: JSON Schema/draft-7. | string |
DataflowBuiltInTransformationFilter
| Name | Description | Value |
|---|---|---|
| description | A user provided optional description of the filter. | string Constraints: Max length = 256 |
| expression | Condition to filter data. Can reference input fields with {n} where n is the index of the input field starting from 1. Example: $1 < 0 || $1 > $2 (Assuming inputs section $1 and $2 are provided) | string (required) |
| inputs | List of fields for filtering in JSON path expression. | string[] (required) |
| type | The type of dataflow operation. | 'Filter' |
DataflowBuiltInTransformationMap
| Name | Description | Value |
|---|---|---|
| description | A user provided optional description of the mapping function. | string Constraints: Max length = 256 |
| expression | Modify the inputs field(s) to the final output field. Example: $1 * 2.2 (Assuming inputs section $1 is provided) | string |
| inputs | List of fields for mapping in JSON path expression. | string[] (required) |
| output | Where and how the input fields to be organized in the output record. | string (required) |
| type | Type of transformation. | 'BuiltInFunction' 'Compute' 'NewProperties' 'PassThrough' 'Rename' |
DataflowBuiltInTransformationSettings
| Name | Description | Value |
|---|---|---|
| datasets | Enrich data from Broker State Store. Dataset references a key in Broker State Store. | DataflowBuiltInTransformationDataset[] |
| filter | Filters input record or datapoints based on condition. | DataflowBuiltInTransformationFilter[] |
| map | Maps input to output message. | DataflowBuiltInTransformationMap[] |
| schemaRef | Reference to the schema that describes the output of the transformation. | string |
| serializationFormat | Serialization format. Optional; defaults to JSON. Allowed value JSON Schema/draft-7, Parquet. Default: Json | 'Delta' 'Json' 'Parquet' |
DataflowDestinationAddIfNotPresentHeaderAction
| Name | Description | Value |
|---|---|---|
| actionType | The type of header operation to perform. | 'AddIfNotPresent' (required) |
| key | The name of the header to add. | string (required) |
| value | The value of the header to add. | string (required) |
DataflowDestinationAddOrReplaceHeaderAction
| Name | Description | Value |
|---|---|---|
| actionType | The type of header operation to perform. | 'AddOrReplace' (required) |
| key | The name of the header to add or replace. | string (required) |
| value | The value of the header to add or replace. | string (required) |
DataflowDestinationHeaderAction
| Name | Description | Value |
|---|---|---|
| actionType | Set to 'AddIfNotPresent' for type DataflowDestinationAddIfNotPresentHeaderAction. Set to 'AddOrReplace' for type DataflowDestinationAddOrReplaceHeaderAction. Set to 'Remove' for type DataflowDestinationRemoveHeaderAction. | 'AddIfNotPresent' 'AddOrReplace' 'Remove' (required) |
DataflowDestinationOperationSettings
| Name | Description | Value |
|---|---|---|
| dataDestination | Destination location, can be a topic or table name. Supports dynamic values with $topic, $systemProperties, $userProperties, $payload, $context, and $subscription. | string Constraints: Min length = 1 (required) |
| endpointRef | Reference to the Endpoint CR. Can be of Broker, Kafka, Fabric, ADLS, ADX type. | string (required) |
| headers | Headers for the output data. | DataflowDestinationHeaderAction[] |
DataflowDestinationRemoveHeaderAction
| Name | Description | Value |
|---|---|---|
| actionType | The type of header operation to perform. | 'Remove' (required) |
| key | The name of the header to remove. | string (required) |
DataflowOperation
| Name | Description | Value |
|---|---|---|
| builtInTransformationSettings | Built In Transformation configuration. | DataflowBuiltInTransformationSettings |
| destinationSettings | Destination configuration. | DataflowDestinationOperationSettings |
| name | Optional user provided name of the transformation. | string Constraints: Max length = 64 |
| operationType | Type of operation. | 'BuiltInTransformation' 'Destination' 'Source' (required) |
| sourceSettings | Source configuration. | DataflowSourceOperationSettings |
DataflowProperties
| Name | Description | Value |
|---|---|---|
| mode | Mode for Dataflow. Optional; defaults to Enabled. | 'Disabled' 'Enabled' |
| operations | List of operations including source and destination references as well as transformation. | DataflowOperation[] (required) |
| requestDiskPersistence | Disk persistence mode. | 'Disabled' 'Enabled' |
DataflowSourceOperationSettings
| Name | Description | Value |
|---|---|---|
| assetRef | Reference to the resource in Azure Device Registry where the data in the endpoint originates from. | string |
| dataSources | List of source locations. Can be Broker or Kafka topics. Supports wildcards # and +. | string Constraints: Min length = 1[] (required) |
| endpointRef | Reference to the Dataflow Endpoint resource. Can only be of Broker and Kafka type. | string (required) |
| schemaRef | Schema CR reference. Data will be deserialized according to the schema, and dropped if it doesn't match. | string |
| serializationFormat | Content is a JSON Schema. Allowed: JSON Schema/draft-7. | 'Json' |
ExtendedLocation
| Name | Description | Value |
|---|---|---|
| name | The name of the extended location. | string (required) |
| type | Type of ExtendedLocation. | 'CustomLocation' (required) |
Usage Examples
Terraform (AzAPI provider) resource definition
The instances/dataflowProfiles/dataflows resource type can be deployed with operations that target:
- Resource groups
For a list of changed properties in each API version, see change log.
Resource format
To create a Microsoft.IoTOperations/instances/dataflowProfiles/dataflows resource, add the following Terraform to your template.
resource "azapi_resource" "symbolicname" {
type = "Microsoft.IoTOperations/instances/dataflowProfiles/dataflows@2026-03-01"
name = "string"
parent_id = "string"
body = {
extendedLocation = {
name = "string"
type = "string"
}
properties = {
mode = "string"
operations = [
{
builtInTransformationSettings = {
datasets = [
{
description = "string"
expression = "string"
inputs = [
"string"
]
key = "string"
schemaRef = "string"
}
]
filter = [
{
description = "string"
expression = "string"
inputs = [
"string"
]
type = "string"
}
]
map = [
{
description = "string"
expression = "string"
inputs = [
"string"
]
output = "string"
type = "string"
}
]
schemaRef = "string"
serializationFormat = "string"
}
destinationSettings = {
dataDestination = "string"
endpointRef = "string"
headers = [
{
actionType = "string"
// For remaining properties, see DataflowDestinationHeaderAction objects
}
]
}
name = "string"
operationType = "string"
sourceSettings = {
assetRef = "string"
dataSources = [
"string"
]
endpointRef = "string"
schemaRef = "string"
serializationFormat = "string"
}
}
]
requestDiskPersistence = "string"
}
}
}
DataflowDestinationHeaderAction objects
Set the actionType property to specify the type of object.
For AddIfNotPresent, use:
{
actionType = "AddIfNotPresent"
key = "string"
value = "string"
}
For AddOrReplace, use:
{
actionType = "AddOrReplace"
key = "string"
value = "string"
}
For Remove, use:
{
actionType = "Remove"
key = "string"
}
Property Values
Microsoft.IoTOperations/instances/dataflowProfiles/dataflows
| Name | Description | Value |
|---|---|---|
| extendedLocation | Edge location of the resource. | ExtendedLocation |
| name | The resource name | string Constraints: Min length = 3 Max length = 63 Pattern = ^[a-z0-9][a-z0-9-]*[a-z0-9]$ (required) |
| parent_id | The ID of the resource that is the parent for this resource. | ID for resource of type: instances/dataflowProfiles |
| properties | The resource-specific properties for this resource. | DataflowProperties |
| type | The resource type | "Microsoft.IoTOperations/instances/dataflowProfiles/dataflows@2026-03-01" |
DataflowBuiltInTransformationDataset
| Name | Description | Value |
|---|---|---|
| description | A user provided optional description of the dataset. | string Constraints: Max length = 256 |
| expression | Condition to enrich data from Broker State Store. Example: $1 < 0 || $1 > $2 (Assuming inputs section $1 and $2 are provided) | string |
| inputs | List of fields for enriching from the Broker State Store. | string[] (required) |
| key | The key of the dataset. | string Constraints: Max length = 64 (required) |
| schemaRef | The reference to the schema that describes the dataset. Allowed: JSON Schema/draft-7. | string |
DataflowBuiltInTransformationFilter
| Name | Description | Value |
|---|---|---|
| description | A user provided optional description of the filter. | string Constraints: Max length = 256 |
| expression | Condition to filter data. Can reference input fields with {n} where n is the index of the input field starting from 1. Example: $1 < 0 || $1 > $2 (Assuming inputs section $1 and $2 are provided) | string (required) |
| inputs | List of fields for filtering in JSON path expression. | string[] (required) |
| type | The type of dataflow operation. | 'Filter' |
DataflowBuiltInTransformationMap
| Name | Description | Value |
|---|---|---|
| description | A user provided optional description of the mapping function. | string Constraints: Max length = 256 |
| expression | Modify the inputs field(s) to the final output field. Example: $1 * 2.2 (Assuming inputs section $1 is provided) | string |
| inputs | List of fields for mapping in JSON path expression. | string[] (required) |
| output | Where and how the input fields to be organized in the output record. | string (required) |
| type | Type of transformation. | 'BuiltInFunction' 'Compute' 'NewProperties' 'PassThrough' 'Rename' |
DataflowBuiltInTransformationSettings
| Name | Description | Value |
|---|---|---|
| datasets | Enrich data from Broker State Store. Dataset references a key in Broker State Store. | DataflowBuiltInTransformationDataset[] |
| filter | Filters input record or datapoints based on condition. | DataflowBuiltInTransformationFilter[] |
| map | Maps input to output message. | DataflowBuiltInTransformationMap[] |
| schemaRef | Reference to the schema that describes the output of the transformation. | string |
| serializationFormat | Serialization format. Optional; defaults to JSON. Allowed value JSON Schema/draft-7, Parquet. Default: Json | 'Delta' 'Json' 'Parquet' |
DataflowDestinationAddIfNotPresentHeaderAction
| Name | Description | Value |
|---|---|---|
| actionType | The type of header operation to perform. | 'AddIfNotPresent' (required) |
| key | The name of the header to add. | string (required) |
| value | The value of the header to add. | string (required) |
DataflowDestinationAddOrReplaceHeaderAction
| Name | Description | Value |
|---|---|---|
| actionType | The type of header operation to perform. | 'AddOrReplace' (required) |
| key | The name of the header to add or replace. | string (required) |
| value | The value of the header to add or replace. | string (required) |
DataflowDestinationHeaderAction
| Name | Description | Value |
|---|---|---|
| actionType | Set to 'AddIfNotPresent' for type DataflowDestinationAddIfNotPresentHeaderAction. Set to 'AddOrReplace' for type DataflowDestinationAddOrReplaceHeaderAction. Set to 'Remove' for type DataflowDestinationRemoveHeaderAction. | 'AddIfNotPresent' 'AddOrReplace' 'Remove' (required) |
DataflowDestinationOperationSettings
| Name | Description | Value |
|---|---|---|
| dataDestination | Destination location, can be a topic or table name. Supports dynamic values with $topic, $systemProperties, $userProperties, $payload, $context, and $subscription. | string Constraints: Min length = 1 (required) |
| endpointRef | Reference to the Endpoint CR. Can be of Broker, Kafka, Fabric, ADLS, ADX type. | string (required) |
| headers | Headers for the output data. | DataflowDestinationHeaderAction[] |
DataflowDestinationRemoveHeaderAction
| Name | Description | Value |
|---|---|---|
| actionType | The type of header operation to perform. | 'Remove' (required) |
| key | The name of the header to remove. | string (required) |
DataflowOperation
| Name | Description | Value |
|---|---|---|
| builtInTransformationSettings | Built In Transformation configuration. | DataflowBuiltInTransformationSettings |
| destinationSettings | Destination configuration. | DataflowDestinationOperationSettings |
| name | Optional user provided name of the transformation. | string Constraints: Max length = 64 |
| operationType | Type of operation. | 'BuiltInTransformation' 'Destination' 'Source' (required) |
| sourceSettings | Source configuration. | DataflowSourceOperationSettings |
DataflowProperties
| Name | Description | Value |
|---|---|---|
| mode | Mode for Dataflow. Optional; defaults to Enabled. | 'Disabled' 'Enabled' |
| operations | List of operations including source and destination references as well as transformation. | DataflowOperation[] (required) |
| requestDiskPersistence | Disk persistence mode. | 'Disabled' 'Enabled' |
DataflowSourceOperationSettings
| Name | Description | Value |
|---|---|---|
| assetRef | Reference to the resource in Azure Device Registry where the data in the endpoint originates from. | string |
| dataSources | List of source locations. Can be Broker or Kafka topics. Supports wildcards # and +. | string Constraints: Min length = 1[] (required) |
| endpointRef | Reference to the Dataflow Endpoint resource. Can only be of Broker and Kafka type. | string (required) |
| schemaRef | Schema CR reference. Data will be deserialized according to the schema, and dropped if it doesn't match. | string |
| serializationFormat | Content is a JSON Schema. Allowed: JSON Schema/draft-7. | 'Json' |
ExtendedLocation
| Name | Description | Value |
|---|---|---|
| name | The name of the extended location. | string (required) |
| type | Type of ExtendedLocation. | 'CustomLocation' (required) |
Feedback
Was this page helpful?