Power Query Workout 11 - Define Segments

Difficulty Rating:

Introduction:

You have two columns, each with the lower and upper bound of a range.
What you need to get is every interval of 10 from the lower until the upper bound, e.g. for the first row 40, 50, 60 (s. picture).
You present your result to your boss, and your boss asks you if you can build the same also for intervals of 5.

Your task:
Build a table with segment steps of 10. If possible, do the same for segment steps of 5. Feel free to use the same or a different method.

(NOTE: For more advanced users, attempt to minimize the number of manual changes using relevant M code functions)

Power Query Workout11 Define Segments

11 Define Segments.xlsx (31.2 KB)

This workout was posted on Monday June 05, 2023, and the author’s solution will be posted on Sunday June 11, 2023.

4 Likes

@Matthias ,

Excellent workout – if you are just starting out in Power Query, building lists is an absolutely fundamental skill to learn. If you are experienced, it’s always still a good one to practice.

Click for M Code

Here’s the solution for the ten-step intervals:

let
    Source = Excel.CurrentWorkbook(){[Name="Segments"]}[Content],
    ListCreate = Table.AddColumn(Source, "Intervals", each List.Select({[SEGMENT CRITERIA1]..[SEGMENT CRITERIA2]}, each Number.Mod(_, 10) = 0)),
    Expand = Table.SelectColumns( Table.ExpandListColumn(ListCreate, "Intervals"), {"Segment", "Intervals"})
in
    Expand

Solution for the five-step intervals reuses exactly the same code with just the second parameter of the Number.Mod function changed from 10 to 5.

let
    Source = Excel.CurrentWorkbook(){[Name="Segments"]}[Content],
    ListCreate = Table.AddColumn(Source, "Intervals", each List.Select({[SEGMENT CRITERIA1]..[SEGMENT CRITERIA2]}, each Number.Mod(_, 10) = 0)),
    Expand = Table.SelectColumns( Table.ExpandListColumn(ListCreate, "Intervals"), {"Segment", "Intervals"})
in
    Expand

– Brian

3 Likes

Adjunto mi query…

Utilizando funciones de texto

M Code Solutions

Intervals of 5


let
    Source = Excel.CurrentWorkbook(){[Name="Segments"]}[Content],
    Intervals = Table.AddColumn(Source, "Intervals", each 
        let
        a = List.Transform({[SEGMENT CRITERIA1]..[SEGMENT CRITERIA2]}, Text.From),
        b = List.Select(a,  each Text.EndsWith(_, "0") or Text.EndsWith(_, "5")),
        c = List.Transform(b, Number.From)
        in c)[[Segment], [Intervals]],
    Sol = Table.ExpandListColumn(Intervals, "Intervals")
in
    Sol

Intervals of 10

let
    Source = Excel.CurrentWorkbook(){[Name="Segments"]}[Content],
    Intervals = Table.AddColumn(Source, "Intervals", each 
        let
        a = List.Transform({[SEGMENT CRITERIA1]..[SEGMENT CRITERIA2]}, Text.From),
        b = List.Select(a,  each Text.EndsWith(_, "0")),
        c = List.Transform(b, Number.From)
        in c)[[Segment], [Intervals]],
    Sol = Table.ExpandListColumn(Intervals, "Intervals")
in
    Sol
2 Likes

@AlejandroSimon ,

Clever. For filtering the lists, my thoughts jumped immediately to modulo - never even considered using Text.EndsWith function. Always fascinating how even on simple workouts, people take very different paths…

  • Brian
2 Likes

Hello,
I learned how to use List.Generate for this workout!! (well, starting to :wink: ))
Here is my attempt… just change the Step value to have what is needed.
Thanks!

Summary

let
Source = #“Excel W11”,
Step=5,
Expanded = Table.ExpandListColumn(Table.AddColumn(Source, “Intervals”, (x)=> List.Generate(() => x[SEGMENT CRITERIA1], each _ <= x[SEGMENT CRITERIA2] , each _ + Step))[[Segment],[Intervals]],“Intervals”)
in
Expanded

1 Like
Summary
Intervalo de 10

let
    Fonte = Segments,
    ad = Table.AddColumn(Fonte, "Personalizar", each 
    [
    a = List.Zip({
    {[SEGMENT CRITERIA1]..[SEGMENT CRITERIA2]},
    List.Transform({[SEGMENT CRITERIA1]..[SEGMENT CRITERIA2]}, each Number.Mod(Number.From(_),10))}),
    b = List.Select(a, each _{1} = 0),
    c = List.Transform(b, each _{0})
    ][c]),
        rs = Table.ExpandListColumn(ad, "Personalizar")[[Segment],[Personalizar]]
in
    rs

/-------------------------------------------------------------------------------\
Intervalo de 5

let
    Fonte = Segments,
    ad = Table.AddColumn(Fonte, "Personalizar", each 
[
a = List.Zip({
{[SEGMENT CRITERIA1]..[SEGMENT CRITERIA2]},
List.Transform({[SEGMENT CRITERIA1]..[SEGMENT CRITERIA2]}, each Number.Mod(Number.From(_),5))}),
b = List.Select(a, each _{1} = 0),
c = List.Transform(b, each _{0})
][c]),
    rs = Table.ExpandListColumn(ad, "Personalizar")[[Segment],[Personalizar]]
in
    rs
2 Likes
Summary
let
    Source = Excel.CurrentWorkbook(){[Name="Segments"]}[Content],
    n = 10,
    Ans = Table.ExpandListColumn(Table.Group(Source, {"Segment"}, {"Intervals", each 
      let  r= Record.ToList(_{0}) in List.Numbers(r{1}, Number.IntegerDivide(r{2}-r{1},n)+1,n)}),"Intervals")
in
    Ans
1 Like

let
Source = Excel.CurrentWorkbook(){[Name=“Segments3”]}[Content],
Rept = Table.AddColumn(Source, “Repeat”, each List.Repeat({“”},([SEGMENT CRITERIA2]-[SEGMENT CRITERIA1])/Inter()+1)),
Expanded = Table.ExpandListColumn(Rept, “Repeat”),
Grouped = Table.Group(Expanded, {“Segment”, “SEGMENT CRITERIA1”}, {{“Count”, each _, type table [Segment=text, SEGMENT CRITERIA1=number, SEGMENT CRITERIA2=number, Repeat=text]}}),
Interval = Table.AddColumn(Grouped, “Indx”, each Table.AddIndexColumn([Count],“Inter”,[SEGMENT CRITERIA1],Inter())),
Expand = Table.ExpandTableColumn(Interval, “Indx”, {“Inter”}, {“Intervals”}),
Removed = Table.RemoveColumns(Expand,{“SEGMENT CRITERIA1”, “Count”})
in
Removed

Interval is an invoked function

I prefer to avoid a list all then filter down approach when possible, especially if the filtered list is a small fraction of the unfiltered list. It doesn’t matter much for this simple example but can save huge amounts of time and memory for more complex tasks.

I propose this as an alternative:

M Code
let
  Source = Excel.CurrentWorkbook(){[Name = "Segments"]}[Content], 
  #"Added Custom" = Table.AddColumn(
    Source, 
    "Invervals", 
    each List.Transform(
      {0 .. ([SEGMENT CRITERIA2] - [SEGMENT CRITERIA1]) / 5}, 
      (x) => [SEGMENT CRITERIA1] + 5 * x
    )
  ), 
  #"Expanded Invervals" = Table.ExpandListColumn(#"Added Custom", "Invervals")
in
  #"Expanded Invervals"
2 Likes

@AlexisOlson ,

Interesting approach, and like you I expected it to outperform the list all and filter down approach. The actual results are mixed.

I ran all segments out to 500,000, and ran 10 trials at each privacy settting. Your approach ran slightly faster with privacy ignored, while mine ran slightly faster with privacy active

2 Likes

Ooh, empiricism. I like it!

At a ratio of 1/5, this isn’t too surprising. It’s hard to anticipate exactly how things are optimized and implemented under the hood.

For this problem, the difference is mostly aesthetic. For generating a list of primes, a list-and-filter approach is the best option available. For generating a list of powers of 10, list-and-filter is much worse beyond about the first half dozen terms.

1 Like

@AlexisOlson ,

Generally, I find Power Query/M very straightforward to work in, because you can see the workings of each step. The one exception is with regard to optimization.

While DAX is somewhat predictable in terms of how different structures and functions will perform (and thus how you can optimize them), I find Power Query totally opaque and unpredicatable in this regard.

The only way I’ve found to optimize my queries is with this sort of empirical testing, across a wide range of different constructs.

-Brian

1 Like

Hello,
thanks to everyone who participated! I did expect various approaches and wasn’t disapointed. :blush:

I’ll add three more solutions of which I think that they are on the easy side.

These workouts are very often based on real world tasks. This one too and in the real world it was just about the easier task with 10.

Let’s start off on the Add Column tab with a Custom Column:
image
We call it Intervals and use this simple formula to build a list:
{[SEGMENT CRITERIA1]/10…[SEGMENT CRITERIA2]/10}


The “trick” here is to divide both the interval start and the interval end by 10.

We can now get rid already of SEGMENT CRITERIA1 and EGMENT CRITERIA1 by clicking on them and pressing Delete (no picture for that).

Then just expand column Intervals to new rows.
image

Mark the Intervals column and on the Transform tab select Standard > Multiply
image

Enter 10 in the Multiply menu:

Done!

let
    Source = Excel.CurrentWorkbook(){[Name="Segments"]}[Content],
    #"Added Custom" = Table.AddColumn(Source, "Intervals", each {[SEGMENT CRITERIA1]/10..[SEGMENT CRITERIA2]/10}),
    #"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"SEGMENT CRITERIA1", "SEGMENT CRITERIA2"}),
    #"Expanded Custom" = Table.ExpandListColumn(#"Removed Columns", "Intervals"),
    #"Multiplied Column" = Table.TransformColumns(#"Expanded Custom", {{"Intervals", each _ * 10, type number}})
in
    #"Multiplied Column"

This approach is UI supported, super easy and was good enough for the real world! And just change the 10 with 5 and it will provide you also intervals of 5.

Here is another UI supported solution - basically @AlejandroSimon’s solution just built with the UI.

Let’s start off again on the Add Column tab with a Custom Column:
image
We call it Intervals and use this simple formula - without any division - to build a list:
{[SEGMENT CRITERIA1]…[SEGMENT CRITERIA2]}

We can now get rid of SEGMENT CRITERIA1 and EGMENT CRITERIA1 by clicking on them and pressing Delete (again no picture for that).

Then just expand column Intervals to new rows.
image

Now right mouse on the Intervals column and select Change Type > Text:

Again on Intervals click on the top right triangle and select Text Filters > Ends With…
image
We want to keep rows which end with 0 Or 5

Finally, we change back the data type to Whole Number and we are done:
image

let
    Source = Excel.CurrentWorkbook(){[Name="Segments"]}[Content],
    #"Added Custom" = Table.AddColumn(Source, "Intervals", each {[SEGMENT CRITERIA1]..[SEGMENT CRITERIA2]}),
    #"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"SEGMENT CRITERIA1", "SEGMENT CRITERIA2"}),
    #"Expanded Custom" = Table.ExpandListColumn(#"Removed Columns", "Intervals"),
    #"Changed Type" = Table.TransformColumnTypes(#"Expanded Custom",{{"Intervals", type text}}),
    #"Filtered Rows" = Table.SelectRows(#"Changed Type", each Text.EndsWith([Intervals], "0") or Text.EndsWith([Intervals], "5")),
    #"Changed Type1" = Table.TransformColumnTypes(#"Filtered Rows",{{"Intervals", Int64.Type}})
in
    #"Changed Type1"

Let no one complain that it has too many steps or takes up too much memory. It’s a valid solution and still super easy!

For the final solution I won’t go step by step because there are only 3 simple steps. Add column > remove columns > expand column. The “secret” is in using List.Numbers:
List.Numbers(start as number, count as number, optional increment as nullable number)
start is obviously [SEGMENT CRITERIA1]
count is the number of values to create and needs a bit more attention, we calculate it by dividing the Intervals span plus 5 by 5
increment is of course 5

List.Numbers([SEGMENT CRITERIA1],([SEGMENT CRITERIA2]-[SEGMENT CRITERIA1]+5)/5, 5)

let
    Source = Excel.CurrentWorkbook(){[Name="Segments"]}[Content],
    #"Added Custom" = Table.AddColumn(Source, "Intervals", each List.Numbers([SEGMENT CRITERIA1],([SEGMENT CRITERIA2]-[SEGMENT CRITERIA1]+5)/5, 5)),
    #"Removed Columns" = Table.RemoveColumns(#"Added Custom",{"SEGMENT CRITERIA1", "SEGMENT CRITERIA2"}),
    #"Expanded Custom" = Table.ExpandListColumn(#"Removed Columns", "Intervals")
in
    #"Expanded Custom"

I think this is a very good use of List.Numbers. Probably not a formula known to beginners, but if you can get here your head around count this solution is probably the easiest of the three solutions.

Try them out and make sure to check out the other approaches above! => Hope you learn something!

Big thanks to everyone who has blurred or hidden the details! It allows readers to develop their own ideas without being influenced - and then it’s easy enough to check other posts.

2 Likes

Thank you @Matthias, another fun workout with a lot of use cases!

I used List.Numbers myself, but I had a question on your remark that your first solution isn’t working on the interval of 5. I wondered why not? Replace all 10’s by 5’s and it’s working fine by me. Am I missing something?

Thank you, I’m looking forward to your reply and next workouts!

1 Like

Hey @MartijnPanjer, you are absolutely correct and I changed it. Great that you like the workouts. Bedankt!