Has anyone implemented some kind of test automation for Power BI reports and if so, what technique was used?
I’d like to automate that
The report is accessible, e.g. it loads and displays. This could be done by just comparing the report in the service image with a predefined stored image.
Performance like load time. i.e time from when report is requested until it has finished rendering.
This could be used for regression testing as well as to alert when reports are not accessible for whatever reason.
I am thinking that Selenium or similar tool would be what is closest to solving this but would be interested to know if anyone has tried somethin like this. This is not a method to test the actual data, that is another topic, just that the reports are accessible and loads as expected.
I’m a bit confused with this one. Are you just looking to have the page load automatically with new data?
You know this can be done utilising the online service and the gateway?
Maybe I’m just not truly understanding your requirements here, it’s likely why I am a bit confused.
It’s also probably because I’m not that familiar with what you were discussing or have placed into the forum post. Maybe if you clear it up and expand on what you’re talking about that would be more helpful.
Basically I want to make sure that reports in the service are accessible. With accessible I mean that it loads in the service and that the visuals render OK. There could be all sorts of reasons for the report NOT to display OK like:
A) Loading from DW was unsuccessful (which in it’s turn can have a gazillion different reasons)
B) Gateway is down
C) Power BI service is down
D) Someone tampered with the report
I want to avoid that users that try to access the reports in the morning are the first that discovers that they are unavailable and therefore I want to setup some kind of automatic testing that accesses the report (Selenium perhaps) and makes sure that it’s accessible. If not, a notification could be sent that the report is inaccessible and you can start troubleshooting.
This could be expanded to do regression testing to ensure that the report gives the same result as before when you for example have done some redeisgn to the underlying DW. Example. Open the report and set a date filter for a specific date and then compare the result with what you had before redeisgn.
I hope that I have been able to clarify what I am looking for.
Thanks
Fredrik
Really I’m not sure if this exists. I haven’t seen of heard of anything that would solve this specfically.
Regarding the regression example, I think you just want to look at building your models and reports so that dynamically you can test things and see results, rather than having to use another technology and overlay that to what you are doing in Power BI. Maybe it’s not totally logical to your example but I believe there would be a way to design the model so you could achieve it.