Power Query Workout 04 - Allocate the Employee’s Extra Points

Difficulty Rating:


or

Introduction:

Every day the employees of our store can earn bonus points, which do have an influence on their monthly pay. So it is extremely important that you summarize the bonus points correctly.
In addition the employees can earn each day 20 Extra Points if they have the max points of the day.
And to keep the employees happy they can also earn 10 Extra Points if they have the max points in the category with the most points of the day.
Be aware that more than one employee can have the max points per day or per category.

Your task:
Build a dynamic solution which is independent from the number of categories and the number of employees in the data.
This is an intermediate workout for the Max Points and an advanced workout including the Cat Points => You can choose which level you want to solve.
[And if you are a beginner, just summarize the bonus points.]

(NOTE: For more advanced users, attempt to minimize the number of manual changes using relevant M code functions)
Power Query Workout4 Allocate Employee's Extra Points
4 Add Extra Points.xlsx (18.6 KB)

This workout was posted on Monday April 17, 2023, and the author’s solution will be posted on Sunday April 23, 2023.

2 Likes
let
    Fonte = Bonus,
    gp1 = Table.Group(Fonte, {"Date", "Employee"}, {{"Points", each List.Sum(_[Points])}}),
    gp2 = Table.Group(gp1, {"Date"}, {{"Contagem", each List.Max(_[Points])}}),
    mesc = Table.NestedJoin(gp2, {"Date", "Contagem"}, gp1, {"Date", "Points"}, "gp2", JoinKind.LeftOuter),
    gp2_exp = Table.ExpandTableColumn(mesc, "gp2", {"Employee"}, {"Employee"}),
    res2 = Table.AddColumn(gp2_exp, "Max Points", each 20),
    p = Table.Group(Fonte, {"Date","Category"}, {{"Soma", each List.Sum(_[Points])},{"Max", each List.Max(_[Points])}}),
    g = Table.Group(p, {"Date"}, {{"Soma", each List.Max([Soma])}}),
    mes = Table.NestedJoin(g, {"Soma"}, p, {"Soma"}, "g", JoinKind.LeftOuter),
    gp3 = Table.ExpandTableColumn(mes, "g", {"Max"})[[Date],[Max]],
    mesc1 = Table.NestedJoin(gp3, {"Date", "Max"}, Fonte, {"Date", "Points"}, "gp3", JoinKind.LeftOuter),
    res3 = Table.ExpandTableColumn(mesc1, "gp3", {"Employee"}, {"Employee"}),
    res_3 = Table.AddColumn(res3, "Cat Points", each 10),
    res = gp1,
    mesc2 = Table.NestedJoin(res, {"Date", "Employee", "Points"}, res2, {"Date", "Employee", "Contagem"}, "res", JoinKind.LeftOuter),
    exp = Table.ExpandTableColumn(mesc2, "res", {"Max Points"}, {"Max Points"}),
    m = Table.NestedJoin(exp, {"Date", "Employee"}, res_3, {"Date", "Employee"}, "res Expandido", JoinKind.LeftOuter),
    fim = Table.ExpandTableColumn(m, "res Expandido", {"Cat Points"}, {"Cat Points"})
in
    fim
1 Like

@Matthias ,

REALLY like the “choose your own difficulty” of this challenge. Clearly a lot of thought went into the development of this one. Well done!

Here’s my full solution - definitely had to break this down into discrete pieces and then join together at the end.


let
    Source = Table.TransformColumnTypes( Excel.CurrentWorkbook(){[Name="Bonus"]}[Content], {"Date", Date.Type}),
    PointsPerDay = Table.Group(Source, {"Date", "Employee"}, {{"PoinPerDay", each List.Sum([Points]), type number}}),
    MaxPoiintsPerDay = Table.Group(PointsPerDay, {"Date"}, {{"All", each _, type table [Date=nullable date, Employee=text, PoinPerDay=number]}, {"MaxPointsPerDay", each List.Max([PoinPerDay]), type number}}),
    Expand = Table.ExpandTableColumn(MaxPoiintsPerDay, "All", {"Employee", "PoinPerDay"}, {"Employee", "PoinPerDay"}),
    AddMaxDailyBonus = Table.AddColumn(Expand, "Max Points", each if [PoinPerDay] = [MaxPointsPerDay] then 20 else null),
    AddDailyPointsByCategory = Table.Group(Source, {"Date", "Category"}, {{"DailyPointsByCategory", each List.Sum([Points]), type number}}),
    AddMaDailyPointsByCategory = Table.Group(AddDailyPointsByCategory, {"Date"}, {{"All", each _, type table [Date=nullable date, Category=text, DailyPointsByCategory=number]}, {"MaxDailyPointsByCategory", each List.Max([DailyPointsByCategory]), type number}}),
    Expand2 = Table.ExpandTableColumn(AddMaDailyPointsByCategory, "All", {"Category", "DailyPointsByCategory"}, {"Category", "DailyPointsByCategory"}),
    AddTopSellingCategoryPerDay = Table.AddColumn(Expand2, "TopSellingCategoryPerDay", each if [DailyPointsByCategory] = [MaxDailyPointsByCategory] then [Category] else null),
    CleanTopSellingCategoryByDay = Table.RemoveColumns( Table.SelectRows(AddTopSellingCategoryPerDay, each ([TopSellingCategoryPerDay] <> null)), {"Category", "DailyPointsByCategory", "MaxDailyPointsByCategory"}),
    AddTotalPointsbyEmpCatDay = Table.Group(Source, {"Date", "Employee", "Category"}, {{"TotalPointsByDayCategoryEmployee", each List.Sum([Points]), type number}}),
    JoinWinCat = Table.Join( AddTotalPointsbyEmpCatDay, "Date", CleanTopSellingCategoryByDay, "Date"),
    FilteTopSellingCatPerDay = Table.SelectRows( JoinWinCat, each [Category] = [TopSellingCategoryPerDay] ),
    #"Grouped Rows" = Table.Group(FilteTopSellingCatPerDay, {"Date"}, {{"All", each _, type table [Date=nullable date, Employee=text, Category=text, TotalPointsByDayCategoryEmployee=number, TopSellingCategoryPerDay=text]}, {"MaxCatPointsPerDay", each List.Max([TotalPointsByDayCategoryEmployee]), type number}}),
    #"Expanded All" = Table.ExpandTableColumn(#"Grouped Rows", "All", {"Employee", "Category", "TotalPointsByDayCategoryEmployee", "TopSellingCategoryPerDay"}, {"Employee", "Category", "TotalPointsByDayCategoryEmployee", "TopSellingCategoryPerDay"}),
    AddCatPoints = Table.AddColumn(#"Expanded All", "Cat Points", each if [TotalPointsByDayCategoryEmployee] = [MaxCatPointsPerDay] then 10 else null),
    FinalCatPointsPerDay = Table.RemoveColumns(AddCatPoints,{"Category", "TotalPointsByDayCategoryEmployee", "TopSellingCategoryPerDay", "MaxCatPointsPerDay"}),
    JoinFinal  = Table.Join( AddMaxDailyBonus, {"Date", "Employee"}, FinalCatPointsPerDay, {"Date", "Employee"}),
    CleanFinal = Table.RenameColumns( Table.RemoveColumns(JoinFinal,{"MaxPointsPerDay"}), {"PoinPerDay", "Points"})
in
    CleanFinal

4 Add Extra Points - Brian Julius Solution.xlsx (24.0 KB)

  • Brian
1 Like

This is the sort of situation I wish I had the dataflow UI rather than the more primitive one in Excel.

Dataflow UI Snip

Anyway, this workout can be solved mostly via a bunch of Group By and Merge steps.

let
    Source = Table.TransformColumnTypes(Excel.CurrentWorkbook(){[Name="Bonus"]}[Content], {{"Date", type date}}),
    GroupEmployee = Table.Group(Source, {"Date", "Employee"}, {{"Points", each List.Sum([Points]), Int64.Type}}),
    GroupCategory = Table.Group(Source, {"Date", "Category"}, {{"Points", each List.Sum([Points]), Int64.Type}}),
    MaxPoints     = Table.Group(GroupEmployee, {"Date"}, {{"Max Points", each List.Max([Points]), Int64.Type}}),
    EmpMaxPoints  = Table.NestedJoin(GroupEmployee, {"Date", "Points"}, MaxPoints, {"Date", "Max Points"}, "Max Points", JoinKind.LeftOuter),
    MaxCategories = Table.Group(GroupCategory, {"Date"}, {{"Category", each Table.Max(_, "Points")[Category]}}),
    EmpCatPoints  = Table.NestedJoin(MaxCategories, {"Date", "Category"}, Source, {"Date", "Category"}, "Employee", JoinKind.LeftOuter),
    TopCatEmp     = Table.TransformColumns(EmpCatPoints, {{"Employee", each Table.Max(_, "Points")[Employee]}}),
    MergeSteps    = Table.NestedJoin(EmpMaxPoints, {"Date", "Employee"}, TopCatEmp, {"Date", "Employee"}, "Cat Points", JoinKind.LeftOuter),
    AssignPoints  = Table.TransformColumns(MergeSteps, {{"Max Points", each if Table.IsEmpty(_) then null else 20, Int64.Type}, {"Cat Points", each if Table.IsEmpty(_) then null else 10, Int64.Type}})
in
    AssignPoints
2 Likes
let
    Source = Excel.CurrentWorkbook(){[Name="Bonus"]}[Content],
    Cat = Table.Group(Table.Group(Source, {"Date","Category"}, {"Pt", each List.Sum([Points])}),"Date", {"Cat",each Table.Sort(_,{"Pt",1}){0}[Category] }),
    Point = Table.Group(Source, {"Date", "Employee"}, {{"Points", each List.Sum([Points])} ,{"Cat",each Table.Join(_,{"Date","Category"},Cat,{"Date","Cat"})[Points]{0} }} ),
    Max = Table.Combine(Table.Group(Point, {"Date"}, {{"T", each Table.AddColumn(Table.AddColumn(_,
        "Max Points",(t)=> if t[Points]= List.Max([Points]) then 20 else null),
        "Cat Points",(t)=> if t[Cat]= List.Max([Cat]) then 10 else null ) }})[T]),
    Fin = Table.RemoveColumns(Max,"Cat")
in
    Fin
4 Likes

Adjunto mi query …

let
Source = Excel.CurrentWorkbook(){[Name=“Bonus”]}[Content],
Group = Table.Group(Source, {“Date”}, {{“All”, each
let
a = ,
b = Table.Group(a, {“Employee”}, {{“Points”, each List.Sum([Points])}}),
c = Table.AddColumn(b, “Max Points”, each if List.Max(b[Points]) = [Points] then 20 else null)
in c}}),
Expand = Table.ExpandTableColumn(Group, “All”, {“Employee”, “Points”, “Max Points”}),
Group2 = Table.Group(Source, {“Date”, “Category”}, {{“Count”, each List.Sum([Points])}, {“Name”, each Table.Max(
, “Points”)[Employee]}}),
Group3 = Table.ToRows(Table.Group(Group2, {“Date”}, {{“Count”, each List.Max([Count])}, {“Cat”, each Group2{[Count = List.Max([Count])]}[Category]}, {“Name”, each Group2{[Count = List.Max([Count])]}[Name]}})[[Date],[Name]]),
Sol = Table.AddColumn(Expand, “Cat Points”, each if List.Contains(Group3, {[Date]} & {[Employee]}) then 10 else null)
in
Sol

3 Likes

@AlexisOlson,

Nice clean, compact solution.

I think I recall reading in one of the wave plan docs that the online view was coming to Desktop sometime this Spring.

  • Brian

Hello, thanks for the nice workout!
I tired to solve it using Melissa de Korte’s nested structures course here on EDNA - I found it super interesting and wanted to practice :slight_smile:
Have a good day!

Code:

let
Source = Excel.Workbook(File.Contents(mypath\4 Add Extra Points.xlsx"), null, false),
Bonus_Table = Source{[Item=“Bonus”,Kind=“Table”]}[Data],
#“Inserted Merged Column” = Table.AddColumn(Bonus_Table, “CatPoints”, each Text.Combine({[Category], Text.From([Points], “en-US”)}, “,”), type text),
#“Grouped Rows” = Table.Group(#“Inserted Merged Column”, {“Date”}, {{“All”, each , type table [Date=nullable date, Employee=nullable text, Category=nullable text, Points=nullable number,CatPoints=nullable number]}, {“MyTable”, each , type table [Date=nullable date, Employee=nullable text, Category=nullable text, Points=nullable number,CatPoints=nullable number]}}),
#“Remove Category from Nested Table” = Table.TransformColumns( #“Grouped Rows”,{{“All”, each Table.RemoveColumns(
,“Category”)}}),
#“Group NT by Employee”=
Table.TransformColumns( #“Remove Category from Nested Table”,
{{“All”,
each Table.Group(
, {“Employee”}, {{“SumPoints”, each List.Sum([Points]), type nullable number}})
}}
),
#“Add Ranking” = Table.TransformColumns( #“Group NT by Employee”,{{“All”, each Table.AddRankColumn(,“Rank”,{“SumPoints”,Order.Descending})}}),
#“Add Max points to NT” = Table.TransformColumns( #“Add Ranking”,{{“All”, each Table.AddColumn(
,“Max Points”,each if [Rank]=1 then 20 else null)}}),
#“Remove Rank Column” = Table.TransformColumns(#“Add Max points to NT”,{{“All”, each Table.RemoveColumns(,“Rank”)}}),
#“Group by Category in NT” = Table.TransformColumns( #“Remove Rank Column”,
{{“MyTable”,
each Table.Group(
, {“Category”}, {{“SumPoints”, each List.Sum([Points]), type nullable number},{“MaxPoints”, each List.Max([Points]), type nullable number}})
}}
),
#“MaxPoints by Category” = Table.TransformColumns(#“Group by Category in NT”,{{“MyTable”, each Table.SelectRows(Table.AddRankColumn(,“Rank”,{“SumPoints”,Order.Descending}),each [Rank]=1)}}),
#“Remove Rank Column 2” = Table.TransformColumns(#“MaxPoints by Category”,{{“MyTable”, each Table.RemoveColumns(
,“Rank”)}}),
#“Add Combine CatPoints” = Table.TransformColumns(#“Remove Rank Column 2”,{{“MyTable”, each Table.AddColumn(,“CatPoints”,each Text.Combine({[Category], Text.From([MaxPoints], “en-US”)}, “,”))}}),
#“Add Max CatPoints Employee” = Table.TransformColumns(#“Add Combine CatPoints”,{{“MyTable”, each Table.AddColumn(
,“Employee”,each Table.SelectRows(#“Inserted Merged Column”, (x)=> x[CatPoints]=[CatPoints])[Employee]{0})}}),
#“Expanded All” = Table.ExpandTableColumn(#“Add Max CatPoints Employee”, “All”, {“Employee”, “SumPoints”, “Max Points”}, {“Employee”, “Points”, “Max Points”}),
#“Expanded MyTable” = Table.ReplaceValue(Table.ExpandTableColumn(#“Expanded All”, “MyTable”, {“Employee”}, {“Cat Points”}),
each [Cat Points],each if [Employee]=[Cat Points] then 10 else null,Replacer.ReplaceValue,{“Cat Points”})
in
#“Expanded MyTable”

2 Likes

I realized I had left out the potential to have several employees for the same Cat Points.
I re-edited that step in my code to take that into account:
#“Add Max CatPoints Employee” = Table.TransformColumns(#“Add Combine CatPoints”,{{“MyTable”, each Table.ExpandListColumn(Table.AddColumn(_,“Employee”,each Table.SelectRows(#“Inserted Merged Column”, (x)=> x[CatPoints]=[CatPoints])[Employee]), “Employee”)}}),

2 Likes

Hello,
thanks to everyone who participated! There was a difficulty in the workout which I didn’t hide in the small prints: “Be aware that more than one employee can have the max points per day or per category.” I also repeated the warning on LinkedIn, but probably I should have shouted it louder …

I’ll present both stages (intermediate and advanced) based on step-by-step grouping and merging, which will make sure that we find all max points per day and per category.

Both stages start off with two groupings: Mark the Date and Employee columns and choose on the Home tab Group By. Change New Column Name to Points and Operation to Sum and select as Column Points:

To get the Max Points per Date select the Date column and choose on the Home tab Group By. Change New Column Name to Max Points and Operation to Max and select as Column Points:

Join then the query against itself by pressing on the Home tab Merge Queries. For this you select the … (current) query and select both columns while pressing the CTRL key:


=> you need to change in the formula bar the first #“Grouped Max Date” to #“Grouped Employees” and the first Max Points to Points:
= Table.NestedJoin(#“Grouped Employees”, {“Date”, “Points”}, #“Grouped Max Date”, {“Date”, “Max Points”}, “Grouped Max Date”, JoinKind.LeftOuter)

Expand Max Points from the column “Grouped Max Date”:

Now you “simply” need to replace the values in Max Points with 20. You can do this with replacing any value on the Max Points column with 20 and then adjusting the code in the formula bar like this:
= Table.ReplaceValue(#“Expanded Grouped Max Date”, each if [Max Points] <> null then [Max Points] else “xyz”, 20,Replacer.ReplaceValue,{“Max Points”})

Or for an alternative technique mark the Max Points column and click on the Transform tab the Format button and select on the dropdown Add Prefix and then press any letter. That will create this basic syntax which we are going to change in the formula bar:
= Table.TransformColumns(#“Expanded Grouped Max Date”, {{“Max Points”, each “k” & Text.From(_, “de-DE”), type text}})
= Table.TransformColumns(#“Expanded Grouped Max Date”, {{“Max Points”, each if _ <> null then 20 else _}})

Change the step name to Transform and you’re done with the intermediate stage. Not too many steps and each step hopefully is not too difficult to understand and reproduce for you.

let
    Source = Excel.CurrentWorkbook(){[Name="Bonus"]}[Content],
    #"Grouped Employees" = Table.Group(Source, {"Date", "Employee"}, {{"Points", each List.Sum([Points]), type number}}),
    #"Grouped Max Date" = Table.Group(#"Grouped Employees", {"Date"}, {{"Max Points", each List.Max([Points]), type number}}),
    #"Merged Queries" = Table.NestedJoin(#"Grouped Employees", {"Date", "Points"}, #"Grouped Max Date", {"Date", "Max Points"}, "Grouped Max Date", JoinKind.LeftOuter),
    #"Expanded Grouped Max Date" = Table.ExpandTableColumn(#"Merged Queries", "Grouped Max Date", {"Max Points"}, {"Max Points"}),
    Transformation = Table.TransformColumns(#"Expanded Grouped Max Date", {{"Max Points", each if _ <> null then 20 else _}})
in
    Transformation

Second Stage incl. Cat Points:
After the identical first two steps as above, we want to build something similar for the max points of the category with the most points of the day. We use the same technique as before to be able to grab muliple first places, just this time we have to cover one level more.

But how can we refer to Source after the first two steps? Well, we can click on Source itself and then mark the Date and Employee columns and choose on the Home tab Group By. Change New Column Name to Points and Operation to Sum and select as Column Points like before but this time add two aggregations for Max and All Rows:


Now pull this newly created step to the last position of the steps and change the referenced step name to Source:
= Table.Group(Source, {“Date”, “Category”}, {{“Points”, each List.Sum([Points]), type number}, {“Max”, each List.Max([Points]), type number}, {“All”, each _, type table [Date=datetime, Category=text, Employee=text, Points=number]}})

To get the max points of the categories per Date select the Date column and choose on the Home tab Group By. Change New Column Name to Max Points and Operation to Max and select as Column Points. In addition add one aggregation for All Rows to not lose any data:

Now we can twice expand the All column. The second time it will be automatically called All.1.
First time all columns except Date:


Second time all columns except Category and Date:

That gives us a lot of rows, but we only need the rows where Max Points correspond to Points and Max Points.1 equals Points.1. So let’s select any of the columns and select any of the values and then bring this to condition to the formula bar: each ([Max] = [Points] and [Max.1] = [Points.1])

Now comes the merging phase. Join the query against itself by pressing on the Home tab Merge Queries. For this you select the … (current) query and then bring in the same code as above:
= Table.NestedJoin(#“Grouped Employees”, {“Date”, “Points”}, #“Grouped Max Date”, {“Date”, “Max Points”}, “Max Points”, JoinKind.LeftOuter)

Join once more the query against itself by pressing on the Home tab Merge Queries. For this you select the … (current) query and select the Date and Employee columns while pressing the CTRL key. Then change in the formula bar the second step name and the column name:
= Table.NestedJoin(#“Merged Queries”, {“Date”, “Employee”}, #“Filtered Rows”, {“Date”, “Employee”}, “Cat Points”, JoinKind.LeftOuter)

We are almost there, if you want you can go ahead and expand both columns and replace / transform in several steps as above. Simply do this step-by-step if you want to practice the above techniques.

Or let’s enhance the transformation step, to directly check if the tables in the two columns are empty. In that case we write null and if they are not empty it means they contain the max values and we can allocate the Max Points respectively the Cat Points. Mark the Max Points column and click on the Transform tab the Format button and select on the dropdown Add Prefix and then press any letter. That will create this basic syntax which we are going to change in the formula bar:
= Table.TransformColumns(#“Merged Queries1”, {{“Max Points”, each “k” & Text.From(_, “de-DE”), type text}})
[Yes, that was right, mark only one column. The text for the second column you can copy and adapt.]
= Table.TransformColumns(#“Merged Queries1”, {{“Max Points”, each if Table.IsEmpty(_) then null else 20}, {“Cat Points”, each if Table.IsEmpty(_) then null else 10}})

Done! Basically the same technique, just a bit more complex. Means if you understood the first stage you can follow along also for the second stage.

let
    Source = Excel.CurrentWorkbook(){[Name="Bonus"]}[Content],
    #"Grouped Employees" = Table.Group(Source, {"Date", "Employee"}, {{"Points", each List.Sum([Points]), type number}}),
    #"Grouped Max Date" = Table.Group(#"Grouped Employees", {"Date"}, {{"Max Points", each List.Max([Points]), type number}}),
    #"Grouped Categories" = Table.Group(Source, {"Date", "Category"}, {{"Points", each List.Sum([Points]), type number}, {"Max", each List.Max([Points]), type number}, {"All", each _, type table [Date=datetime, Category=text, Employee=text, Points=number]}}),
    #"Grouped Rows" = Table.Group(#"Grouped Categories", {"Date"}, {{"Max", each List.Max([Points]), type number}, {"All", each _, type table [Date=datetime, Category=text, Points=number, Max=number, All=table]}}),
    #"Expanded All" = Table.ExpandTableColumn(#"Grouped Rows", "All", {"Category", "Points", "Max", "All"}, {"Category", "Points", "Max.1", "All.1"}),
    #"Expanded All.1" = Table.ExpandTableColumn(#"Expanded All", "All.1", {"Employee", "Points"}, {"Employee", "Points.1"}),
    #"Filtered Rows" = Table.SelectRows(#"Expanded All.1", each ([Max] = [Points] and [Max.1] = [Points.1])),
    #"Merged Queries" = Table.NestedJoin(#"Grouped Employees", {"Date", "Points"}, #"Grouped Max Date", {"Date", "Max Points"}, "Max Points", JoinKind.LeftOuter),
    #"Merged Queries1" = Table.NestedJoin(#"Merged Queries", {"Date", "Employee"}, #"Filtered Rows", {"Date", "Employee"}, "Cat Points", JoinKind.LeftOuter),
    Transformation = Table.TransformColumns(#"Merged Queries1", {{"Max Points", each if Table.IsEmpty(_) then null else 20}, {"Cat Points", each if Table.IsEmpty(_) then null else 10}})
in
    Transformation

Try both stages, but make sure to check also some of the neat and compact approaches above! => Hope you learn something!

Big thanks to everyone who has blured or hidden the details! I have taken this now out so that it’s easier to compare solutions.

3 Likes