Built a report tracking the Flow Metrics using Azure DevOps (ADO) work items. The Flow Velocity metric reports the number of work items considered “Done”. The customer’s definition of Done is when the work item is associated with the build AND deployed to the production environments through a YAML pipeline. The report’s pipeline data is retrieved from ADO through a combination of OData queries for pipeline data and API calls to obtain the work items related to the YAML builds and deployments. This part of the report is completed.
The problem is the build retention policy removes the pipelines after 30 days. This process removes the records from the analytics data linking the pipeline to the work items as well as other pipeline artifacts. When refreshing the report, the number of work items considered Done (deployed to an environment) is reduced as the pipelines are removed and there is no historical data.
On a refresh, is there a method to append the data to the existing dataset in Power BI and preserve the existing data? The report needs the pipeline and work item links to remain after the pipeline is removed from ADO.
If not, then would the recommendation be to store the pipeline data into a database to manage the appending of pipeline data and point the report to the database?