Here’s the path I went down the other night on this:
The original article that get cited a lot is this one:
https://www.mattmasson.com/2014/11/iterating-over-multiple-pages-of-web-data-using-power-query/
However, this takes a static, brute force approach to the multiple page problem.
These two sources improve on that approach by iterating until an empty table is hit
And Gil Raviv provides an approach for dealing with cursor-based pagination:
From the Datachant post, I then located two Chris Webb blog posts that address the problem of doing this in scheduled refresh:
I’m still very much in learning mode about some of this, but I hope at least a portion of what I’ve found is helpful to you.
- Brian