New dataflows docs

Power BI Online Service
1

Hi everyone,

Matthew Roche shared this on his blog

"The Power Query team has released a treasure trove of new documentation for dataflows in Power BI and Power Apps. You can find the new docs here:

There are over 20 articles in this new resource, and the content they contain answers many of the questions I hear most frequently."

.

Think anyone exploring/considering dataflows and where they fit in will find this a useful resource.

3 Likes
2

Just marking the thread as Solved

3

What’s your take on dataflows and is general use within organisations. Is it ready for prime time and does it make that big a difference?

4

Hi @sam.mckay,

In my view dataflows definitely have added value…

  • Dataflows leverage Power Query, so you can start using them quickly.
  • If your company doesn’t have Data Architects and/or a Data Warehouse for example, you could use dataflows as a central location to prep and have your data ready for consumption.
    ** develop and design just once
    ** schedule automatic refresh
    ** becomes the single source of truth
    ** with one artifact to maintain (should that need ever arise)
  • In report development you’ll never have to query directly against your database (performance).
  • Speeds up Model development because all building blocks (Tables) are ready for use.
  • Dataflows can also be used to store data from legacy systems:
    ** setup once
    ** refresh once
    ** enjoy forever :wink:
  • Dataflows can be used in other tools and services, they’re not exclusive to Power BI.
  • Dataflows can be mapped to CDS, the Common Data Service wich is a set of a standardized data schemas and a metadata system that allows consistency of data and its meaning across applications and business processes.
  • Dataflows run in the cloud. No additional infrastructure required.
  • Incremental refresh is available for dataflows

Just to name a few :upside_down_face:
.
New article and more food for thought…

.
and just with everything else in the ‘power’ stack it’s in constant development. :+1:

3 Likes
5

Yeah, all the above, but: …

Regarding the mapping to CDS, it has some value but it would be better that the powerplatform creates an accessible solution to hook up to display names used in solutions. All the mapping does is convert logical names to display names but those are generally also altered in solutions.

Another weird thing about dataflows. You basically have to create them in the powerbi service, but you can only create reports from them in the desktop? You can’t really make them in power bi desktop because you will need to copy all the queries still manually to the service. Makes absolutely zero sense.

And one more annoyance … it’s quite slow to setup, so would still recommend to still create everything in desktop and then copy/paste into the dataflow in the service. Just done a few larger ones (48 entities) on a premium capacity tennant … get used to wait for each change of a query step. Also the validations of the query steps could push the limits of your patience as sometimes, they only pop-up much later (or sometimes they are already resolved but they just stay there as they weren’t picked up on change).

But the concept is really good to enable self-service reporting at enterprise level.

PS: also … don’t use hashtags in query names. Unlike in desktop, it breaks the solution.

1 Like
6

Why is that weird?
If you had a Datawarehouse you’d design and store your data there. Now through Power Query Online in the Power BI Service you are storing queries in Azure Data Lake Storage Gen2. Makes sense to me.

.

Well that depends… if you have a datamodel based on dataflows ready for use in the Power BI Service you can create a report there. However if that is not the case and you have to design a datamodel then you’ll have to use Desktop, the actual restriction here is datamodel design not the data(flow) source.

.

I do agree it can be slow to set up. Like you most of the time I set up my queries in desktop and copy them to the Power BI Service.

1 Like
7

Weird because:
For report creation based on dataflow: you need desktop (you would assume you can just a select a dataflow as source).
For dataflow creation, you need online (you would assume you can just publish as dataflow).
=> one needs the other yet you cannot do it on the same “platform”.

So regarding the that depends (not sure how you quote):
When you have set up a dataflow in the service, you still can’t use it for report creation in the service. You first need to setup a report in desktop using that dataflow and then publish it to the service. After which you have a dataset which can only then use further in the service.

True self-service would be: the dataflow has been created in the service, you can now start creating dataset/reports from that in the service.

8

Think that’s a misunderstanding for report creation you need a datamodel or a shared dataset. However one of the data sources for that model can be a dataflow…

No argument there!

.

RADACAD did an article on dataflow vs dataset, you can find that here.

.
To quote, just highlight that section and the option to “Quote” will become visible. The quote will be entered in the edit widow at the position of your cursor.

1 Like
9

dataflow is awesome. I can run multiple thread with different refresh interval. Product refresh maybe good enough for once a day, but onhand/sales transaction may be refreshed every several hours/day

1 Like
10

Great points above everyone.

Appreciate your input Melissa.

Sam

11

Hi all,

Just wanted to give you all a heads up on this post by Matthew Roche, see key points below.

There’s a new Power Automate (Flow) connector that will enable any authorized user to take action based on dataflow refresh completion, including:

  • Sending a notification – so you can keep additional users and groups informed
  • Trigger a dataflow or dataset refresh – so you don’t need to maintain separate refresh schedules or live with unnecessary processing delays
  • Trigger a Power Automate flow – so you can do pretty much whatever you want, because Power Automate has a ton of capabilities this blog post won’t even attempt to mention

You can also use the connector to refresh dataflows and monitor dataflow refresh status.

.

Read the full post by Matthew Roche here on his blog.

or check out the official Power BI introduction post here.

1 Like