The problem you are having is that a Matrix table is just like a Pivot table in Excel - and you would not be able to easily achieve what you are looking for with an Excel pivot table.
if you must have this type of layout (although I recommend rethinking your solution - or consider moving your report into Excel Power Query if you must have an Excel layout)
My suggestion for a potential solution is two Matrix tables: