Data Factory – How to use a Group By Node

In Azure Data Factory, the Group By node is a powerful tool that allows you to group data based on specified columns or expressions. This can be useful for performing aggregate functions or summaries on data sets. Here’s a guide on how to use the Group By node in Data Factory:

Step 1: Add a Group By activity to your pipeline

First, you need to add a Group By activity to your pipeline in Data Factory. This activity can be found in the Data Flow section of the authoring tool.

Step 2: Configure the Group By activity

Once you have added the Group By activity to your pipeline, you need to configure it by specifying the inputs, outputs, group by columns, and aggregate functions. You can select the columns that you want to group by and define the aggregate functions that you want to perform on other columns.

Step 3: Run the pipeline

After configuring the Group By activity, you can run the pipeline in Data Factory to start processing your data. The Group By node will group the data based on the specified columns and perform the aggregate functions that you defined.

Example:

For example, let’s say you have a dataset with customer information including their names, cities, and total purchase amounts. You can use the Group By node to group the data by city and calculate the total purchase amount for each city.

Group By Configuration:

• Group By Column: City
• Aggregate Function: Sum(Total Purchase Amount)

Conclusion

The Group By node in Data Factory is a powerful tool for grouping and summarizing data sets. By following the steps outlined in this guide, you can effectively use the Group By activity in your pipelines to perform aggregate functions on your data. Happy data processing!