Power Query Workout 12 - Supplier Invoices

Difficulty Rating:

Introduction:

You have received a csv with, with information about supplier invoices and payments.
It looks quite regular so you are confident, that you can transform it to the required structure.
You have no idea what language Utbetaining could be, but as you know what to bring it to, you have very little doubt what it is about.

Your task:
Transform the csv data into the required structure. You can compare your results with the attached Excel file.
(NOTE: For more advanced users, attempt to minimize the number of manual changes using relevant M code functions)

Power Query Workout12 Supplier Invoices

12 Supplier Invoices.csv (721 Bytes)
12 Supplier Invoices.xlsx (31.4 KB)

This workout was posted on Monday June 12, 2023, and the author’s solution will be posted on Sunday June 18, 2023.

5 Likes

Adjunto mi query…

Click for Solution

let
    Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
    SupNo = Table.AddColumn(Source, "Supplyer #", each if [Column3] = null then [Column1] else null),
    SupName = Table.AddColumn(SupNo, "Supplyer Name", each if [Column3] = null then [Column2] else null),
    Invoice = Table.AddColumn(SupName, "Invoice", each if List.ContainsAny(Text.ToList(Text.From([Column2])), {"0".."9"}) then Text.From([Column2]) else null),
    Fill = Table.FillDown(Invoice,{"Column1", "Supplyer #", "Supplyer Name", "Invoice"}),
    Filter = Table.SelectRows(Fill, each ([Column3] <> null)),
    Type = Table.TransformColumns(Filter, {"Column2", each if List.ContainsAny({"0".."9"},  Text.ToList(Text.From(_)))  then "Invoice" else "Payment" }),
    Date = Table.TransformColumns(Type, {"Column3",  each 
        let
        a = List.Transform(Text.Split(_, "."), Number.From),
        b = #date(a{2}, a{1}, a{0})
        in b}),
    Sol = Table.RenameColumns(Date,{{"Column1", "System #"}, {"Column2", "Type"}, {"Column3", "Date"}})
in
    Sol
1 Like

@Matthias ,

Thanks - excellent “bread and butter” data cleaning exercise that closely mirrors the type of cleaning done in real-life (I would not at all be surprised if this was masked real data).

Click for Solution Code
let
    Source = Excel.CurrentWorkbook(){[Name="Table1"]}[Content],
    SeparateSupplier = Table.SelectRows( Table.FillDown( Table.AddColumn(Source, "Temp", each if Text.Contains( [Column1], "AB" ) then [Column1] else null), {"Temp"}), each [Column1] <> [Temp]),
    AddType = Table.AddColumn(SeparateSupplier, "Type", each if Text.StartsWith( [Column1], ";Utbe" ) then "Payment" else "Invoice"),
    AddDate = Table.AddColumn(AddType, "Date", each Text.End( [Column1], 10 )),
    #"AddSystem#" = Table.FillDown( Table.AddColumn(AddDate, "System #", each if Text.Contains( [Column1], "Utbe" ) then null else Text.Start( [Column1], 5 )), {"System #"}),
    AddInvoice = Table.RemoveColumns( Table.FillDown( Table.AddColumn(#"AddSystem#", "Invoice", each if Text.Contains( [Column1], "Utbe" ) then null else Text.Middle( [Column1], 6, 7)), {"Invoice"}), "Column1"),
    Split = Table.SplitColumn(AddInvoice, "Temp", Splitter.SplitTextByDelimiter(";", QuoteStyle.Csv), {"Supplier #", "Temp.2", "Temp.3"}),
    Clean = Table.SplitColumn(Split, "Temp.2", Splitter.SplitTextByEachDelimiter({"AB"}, QuoteStyle.Csv, true), {"Supplier Name", "Temp.2.2"})
in
    Clean

  • Brian
1 Like
This workout deserves more than 1 star 😉
let
    Source = Csv.Document(File.Contents("D:\Bo\Desktop\12 Supplier Invoices.csv"),[Delimiter=";"]),
    Sup = Table.AddColumn(Source,"T", each (if [Column3] ="" then [#"Supplier #"=[Column1],Supplier Name =[Column2]] else [I=null])&
        [Inovie= if [Column1]="" then null else [Column2]]),
    Expand = Table.ExpandRecordColumn(Sup, "T", Record.FieldNames(Sup[T]{0})),
    Trans = Table.TransformColumns(Expand,{{"Column1",each if _ = "" then null else _},{"Column2",each if _="Utbetaining" then "Payment" else "Invoice"},{"Column3", each Date.From(_,"en-GB")}}),
    Fill = Table.SelectRows(Table.FillDown(Trans,Table.ColumnNames(Trans)), each ([Column3] <> null)),
    Rename = Table.RenameColumns(Fill, List.Zip({List.FirstN(Table.ColumnNames(Fill),3),{"System #","Type","Date"}}))
in
    Rename
1 Like
Summary
let
    Fonte = Tabela1,
    add = Table.AddColumn(Fonte, "Personalizar", each if [Column3] ="" then 
[#"Supplier #"=[Column1],Supplier Name =[Column2]] else null),
    exp = Table.ExpandRecordColumn(add, "Personalizar", {"Supplier #", "Supplier Name"}),
    sub = Table.ReplaceValue(exp,each [Column1],null,(a,b,c)=> if b = "" then null else a, {"Column1","Column2"}),
    pb = Table.SelectRows(Table.FillDown(sub,Table.ColumnNames(sub) ), each ([Column3] <> "")),
    gp = Table.Group(pb, {"Column1"}, {{"Contagem", each 
Table.AddColumn(Table.AddIndexColumn(_, "Ind", 1 ,1),"Type", each if [Ind] = 1 then "Invoicent" else "Payment")}})
[[Contagem]],
    res = Table.ExpandTableColumn(gp, "Contagem", 
    {"Column1","Type", "Column3", "Supplier #", "Supplier Name","Column2" }, 
    {"Sytem #", "Type", "Date", "Supplier #", "Supplier Name","Invoice"})
in
    res
1 Like

@Matthias Here is my solution

Summary
let
    Source = Data, 
    Custom = Table.AddColumn(Source, "Custom", each 
                     [#"System #"   = [Column1],
                      Type          = if [Column1] = null then "Payment" else "Invoice",
                      Date          = Date.From([Column3], "en-GB"),
                      #"Supplier #" = if [Column3] = "" then [Column1] else null,
                      Supplier Name = if [Column3] = "" then [Column2] else null,
                      Invoice = if [Column2] = "Utbetaining" then null else [Column2]])[[Custom]],
    #"Expanded" = Table.ExpandRecordColumn(Custom, "Custom", Record.FieldNames(Custom[Custom]{0})),
    #"Filled Down" = Table.FillDown(#"Expanded",{"System #", "Supplier #", "Supplier Name", "Invoice"}),
    #"Filtered Rows" = Table.SelectRows(#"Filled Down", each ([Date] <> null))
in
    #"Filtered Rows"

1 Like
Summary

let
Source = Csv.Document(File.Contents(“C:\Users\hp\Downloads\12 Supplier Invoices.csv”),[Delimiter=“;”, Columns=3, Encoding=65001, QuoteStyle=QuoteStyle.None]),
Fill = Table.FillDown(Source,{“Column1”}),
Change = Table.TransformColumnTypes(Fill,{{“Column3”, type date}, {“Column1”, Int64.Type}}),
FillDown = Table.FillDown(Change,{“Column1”}),
SupplierName = Table.AddColumn(FillDown, “Supplier Name”, each if [Column3] = null then [Column2] else null),
FillDown1 = Table.FillDown(SupplierName,{“Supplier Name”}),
Filter = Table.SelectRows(FillDown1, each ([Column3] <> null)),
Type = Table.AddColumn(Filter, “Type”, each if [Column2] = “Utbetaining” then “Payment” else “Invoice”),
Invoice = Table.AddColumn(Type, “Invoice”, each if [Type] = “Invoice” then [Column2] else null),
FillDown2 = Table.FillDown(Invoice,{“Invoice”, “Column1”}),
Group = Table.Group(FillDown2, {“Supplier Name”}, {{“Grp”, each _, type table [#“System #”=text, Type=text, Date=nullable date, Supplier Name=text, Invoice=text]}}),
Index = Table.AddIndexColumn(Group, “Index”, 1, 1, Int64.Type),
Expand = Table.ExpandTableColumn(Index, “Grp”, {“Column1”, “Type”, “Column3”, “Invoice”}, {“System #”, “Type”, “Date”, “Invoice”}),
SupplierNo = Table.AddColumn(Expand, “Supplier #”, each “1000”& Text.From([Index])),
Removed = Table.RemoveColumns(SupplierNo,{“Index”}),
Reorder = Table.ReorderColumns(Removed,{“System #”, “Type”, “Date”, “Supplier #”, “Supplier Name”, “Invoice”})
in
Reorder

1 Like

@Matthias Here is my solution

Summary

let
Source = Csv.Document(File.Contents(“C:\Users\mccumisg\Desktop\EDNA\PQ Workout 12\12 Supplier Invoices.csv”),[Delimiter=“;”, Columns=3, Encoding=65001, QuoteStyle=QuoteStyle.None]),
ExtractSupplierCode = Table.AddColumn(Source, “Supplier #”, each if Text.Contains([Column2], “AB”) then [Column1] else null, type text),
ExtractSupplierName = Table.AddColumn(ExtractSupplierCode, “Supplier Name”, each if Text.Contains([Column2], “AB”) then [Column2] else null, type text),
#“Added Conditional Column” = Table.AddColumn(ExtractSupplierName, “Invoice”, each if Text.StartsWith([Column2], “Utbe”) then null else [Column2], type text),
ExtractSystemData = Table.AddColumn(#“Added Conditional Column”, “System #”, each if [Invoice] = null then null else [Column1], type text),
DefineInvoiceVsPayment = Table.AddColumn(ExtractSystemData, “Type”, each if [Invoice] = null then “Invoice” else “Payment”, type text),
ConvertToDate = Table.AddColumn(DefineInvoiceVsPayment, “Date”, each Date.FromText([Column3], “en-GB”), type date),
FillNullData = Table.FillDown(ConvertToDate,{“Supplier #”, “Supplier Name”, “System #”, “Invoice”}),
RemoveSupplierNull = Table.SelectRows(FillNullData, each ([Date] <> null)),
CleanUpAndReorderCols = Table.SelectColumns(RemoveSupplierNull,{“System #”, “Type”, “Date”, “Supplier #”, “Supplier Name”, “Invoice”})
in
CleanUpAndReorderCols

1 Like

Hello, here is my take at it:

Summary

let
Source = #“12 Supplier Invoices”,
#“Added Custom” = Table.AddColumn(Source, “Supplier Name”, each if [Column3]=“” then [Column2] else null,type text),
#“Filled Down” = Table.FillDown(#“Added Custom”,{“Supplier Name”}),
#“Added Custom1” = Table.AddColumn(#“Filled Down”, “Supplier #”, each if [Column3]= “” then [Column1] else null, type text),
#“Filled Down1” = Table.FillDown(#“Added Custom1”,{“Supplier #”}),
#“Filtered Rows” = Table.SelectRows(#“Filled Down1”, each [Column3] <> null and [Column3] <> “”),
#“Added Custom2” = Table.AddColumn(#“Filtered Rows”, “Type”, each if [Column2] = “Utbetaining” then “Payment” else “Invoice”,type text),
#“Replaced Value” = Table.ReplaceValue(#“Added Custom2”,“Utbetaining”,null,Replacer.ReplaceValue,{“Column2”}),
#“Filled Down2” = Table.FillDown(#“Replaced Value”,{“Column1”, “Column2”}),
#“Renamed Columns” = Table.RenameColumns(#“Filled Down2”,{{“Column1”, “System #”}, {“Column3”, “Date”}, {“Column2”, “Invoice”}}),
#“Added Custom3” = Table.RemoveColumns(Table.AddColumn(#“Renamed Columns”, “Custom”, each Date.FromText(Text.Range([Date],3,2)&“/”&Text.Range([Date],0,2)&“/”&Text.Range([Date],6,4)),type date),“Date”),
#“Renamed Columns1” = Table.RenameColumns(#“Added Custom3”,{{“Custom”, “Date”}}),
#“Reordered Columns” = Table.ReorderColumns(#“Renamed Columns1”,{“System #”, “Type”, “Date”, “Supplier #”, “Supplier Name”, “Invoice”})
in
#“Reordered Columns”

1 Like

This is my solution using UI.

Summary
let
    Source = Csv.Document(File.Contents("C:\Users\m.baniasadi\Downloads\12 Supplier Invoices.csv"),[Delimiter=";", Columns=3, Encoding=65001, QuoteStyle=QuoteStyle.None]),
    AddedConditional1 = Table.AddColumn(Source, "Supplier #", each if [Column3] = "" then [Column1] else null),
    AddedConditional2 = Table.AddColumn(AddedConditional1, "Supplier Name", each if [Column3] = "" then [Column2] else null),
    FilledDown = Table.FillDown(AddedConditional2,{"Supplier Name","Supplier #"}),
    FilteredRows = Table.SelectRows(FilledDown, each ([Column3] <> "")),
    AddedConditional3 = Table.AddColumn(FilteredRows, "Type", each if [Column1] <> "" then "Invoice" else "Payment"),
    ReplacedValue = Table.ReplaceValue(AddedConditional3,"Utbetaining",null,Replacer.ReplaceValue,{"Column2"}),
    ChangedType = Table.TransformColumnTypes(ReplacedValue,{{"Column1", Int64.Type}, {"Column3", type date}}),
    FilledDown1 = Table.FillDown(ChangedType,{"Column1", "Column2"}),
    RenamedColumns = Table.RenameColumns(FilledDown1,{{"Column1", "System #"}, {"Column2", "Invoice"}, {"Column3", "Date"}}),
    ReorderedColumns = Table.ReorderColumns(RenamedColumns,{"System #", "Type", "Date", "Supplier #", "Supplier Name", "Invoice"})
in
    ReorderedColumns

1 Like

Thanks Matthias for this new WORKOUT proposal : For me this wasn’t LEVEL1 :
Had to search a little bit more than initially expected for every transformations to be done.
That was FUN : :thinking: :smiley:

M-Code
let
    Source = Csv.Document(File.Contents(pINPUTFile),[Delimiter=";", Columns=3, Encoding=65001, QuoteStyle=QuoteStyle.None]),
    Add_SData = Table.AddColumn(Source, "S", 
        each if ([Column3]="") then [S=[Column1], SN=[Column2], I=null] else (if ([Column1]="") then null else [I=[Column2]])),
    Expand_SData = Table.ExpandRecordColumn(Table.FillDown(Add_SData,{"S"}), "S", {"S", "SN", "I"}, {"Supplier #", "Supplier Name", "Invoice"} ),
    Transform = Table.TransformColumns(Expand_SData, {
        {"Column1", each if _<>"" then _ else null},
        {"Column2", each if _<>"Utbetaining" then "Invoice" else "Payment"},
        {"Column3", each Date.From(_, "fr-FR")}
        }),
    FillDown = Table.FillDown(Transform,{"Column1", "Supplier #", "Supplier Name"}),
    FilterRows = Table.SelectRows(FillDown, each ([Invoice] <> null)),
    Rename = Table.RenameColumns(FilterRows,{{"Column1", "System #"}, {"Column2", "Type"}, {"Column3", "Date"}}),
    ChangeType = Table.TransformColumnTypes(Rename,{{"System #", Int64.Type}, {"Date", type date}, {"Supplier #", Int64.Type}})
in
    ChangeType
1 Like

Hello,
thanks to everyone who participated! I loved to see so many variants of solutions with records from @borydobon, @Luan, @Rajesh, @elaf70. Many solutions were with multiple Table.AddColumn - and I am going to add 2 more to them to show that it is indeed not an advanced problem.

We started off normally in the Power Query Editor with an existing table. This time we are going for New Source on the Home tab and we select File > Text/CSV
image

Bring in the path to the CSV file and press open:


In this case we can trust what Power Query suggests and simply press OK:

All three columns have a data type text, which is fine for the first two, but we are going to change Column3. Right mouse click on it and select Change Type > Date:

Now we select on the Add Column tab Custom Column and call it Supplier # and bring in this if formula:
if [Column3]=null then [Column1] else null

We do the same again for Supplier Name and this time we use this formula:
if [Column3]=null then [Column2] else null


You have seen it’s twice the same simple condition which allows us to identify the info we need.

Three times is a charm so we do it once more for Invoice with a different but still simple condition:
if [Column2] <> “Utbetaining” then [Column2] else null

You probably understand that we want to fill down these three columns, but we also want to fill down Column1. To do that we right mouse click Column1 and select Replace Values…
image
Leave the first field empty and write in Replace With null:

Press CTRL and click on Column1, Supplier #, Supplier Name and Invoice and then right mouse click on one of them to select Fill > Down:

Click on the top right triangle on Column3 and deselect null to get rid of the rows with null in Column3:
image

That looks already good, but we need to replace the content in Column2. Right mouse click on it and select once more Replace Values…
image
Leave the first field empty again and write in Replace With Invoice:


You will see in the formula bar
= Table.ReplaceValue(#“Filtered Rows”,“”,“Invoice”,Replacer.ReplaceText,{“Column2”})
and you replace “” with each [Invoice]
= Table.ReplaceValue(#“Filtered Rows”, each [Invoice],“Invoice”,Replacer.ReplaceText,{“Column2”})

Every time the content in Column2 equals the content in column Invoice it will be replaced by the word “Invoice”.

Make a double click on the Column1and change the name to System #
image
image

Repeat for Column2 to change it to Type and then once mor for Column3 to name it Date. Done!

With adapted name steps the code would look like this:

let
    Source = Csv.Document(File.Contents("C:\YourPath\12 Supplier Invoices.csv"),[Delimiter=";", Columns=3, Encoding=65001, QuoteStyle=QuoteStyle.None]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"Column3", type date}}),
    #"Added Supplier #" = Table.AddColumn(#"Changed Type", "Supplier #", each if [Column3]=null then [Column1] else null),
    #"Added Supplier" = Table.AddColumn(#"Added Supplier #", "Supplier Name", each if [Column3]=null then [Column2] else null),
    #"Added Invoice" = Table.AddColumn(#"Added Supplier", "Invoice", each if [Column2] <> "Utbetaining" then [Column2] else null),
    #"Replaced Value" = Table.ReplaceValue(#"Added Invoice","",null,Replacer.ReplaceValue,{"Column1"}),
    #"Filled Down" = Table.FillDown(#"Replaced Value",{"Column1", "Supplier #", "Supplier Name", "Invoice"}),
    #"Filtered Rows" = Table.SelectRows(#"Filled Down", each ([Column3] <> null)),
    #"Replaced Value1" = Table.ReplaceValue(#"Filtered Rows",each [Invoice],"Invoice",Replacer.ReplaceText,{"Column2"}),
    #"Replaced Value2" = Table.ReplaceValue(#"Replaced Value1","Utbetaining","Payment",Replacer.ReplaceText,{"Column2"}),
    #"Renamed Columns" = Table.RenameColumns(#"Replaced Value2",{{"Column1", "System #"}, {"Column2", "Type"}, {"Column3", "Date"}})
in
    #"Renamed Columns"

That’s not at all complicated.

If you are open to another replacing logic for Column2 we can get rid of one replacing step. Right mouse click on Column2 and select Replace Values…
image
This time we leave both fields empty and just press OK.
We replace this formula
= Table.ReplaceValue(#“Filtered Rows”,“”,“”,Replacer.ReplaceValue,{“Column2”})
with
= Table.ReplaceValue(#“Filtered Rows”,each [Column2],each if [Column2] = “Utbetaining” then “Payment” else “Invoice”,Replacer.ReplaceText,{“Column2”})

This will replace each Utbetaining in Column2 with the word Payment and all other Column2 values with the word Invoice.

Then rename and you are done. Also not too complicated.

let
    Source = Csv.Document(File.Contents("C:\YourPath\12 Supplier Invoices.csv"),[Delimiter=";", Columns=3, Encoding=65001, QuoteStyle=QuoteStyle.None]),
    #"Changed Type" = Table.TransformColumnTypes(Source,{{"Column3", type date}}),
    #"Added Supplier #" = Table.AddColumn(#"Changed Type", "Supplier #", each if [Column3]=null then [Column1] else null),
    #"Added Supplier" = Table.AddColumn(#"Added Supplier #", "Supplier Name", each if [Column3]=null then [Column2] else null),
    #"Added Invoice" = Table.AddColumn(#"Added Supplier", "Invoice", each if [Column2] <> "Utbetaining" then [Column2] else null),
    #"Replaced Value" = Table.ReplaceValue(#"Added Invoice","",null,Replacer.ReplaceValue,{"Column1"}),
    #"Filled Down" = Table.FillDown(#"Replaced Value",{"Column1", "Supplier #", "Supplier Name", "Invoice"}),
    #"Filtered Rows" = Table.SelectRows(#"Filled Down", each ([Column3] <> null)),
    #"Replaced Value1" = Table.ReplaceValue(#"Filtered Rows",each [Column2],each if [Column2] = "Utbetaining" then "Payment" else "Invoice",Replacer.ReplaceText,{"Column2"}),
    #"Renamed Columns" = Table.RenameColumns(#"Replaced Value1",{{"Column1", "System #"}, {"Column2", "Type"}, {"Column3", "Date"}})
in
    #"Renamed Columns"

Try them both and make sure to check out the other approaches above! => Hope you learn something!

Big thanks to everyone who has blurred or hidden the details! It allows readers to develop their own ideas without being influenced - and it’s easy enough to check them afterwards.

3 Likes