Consistency in datasets from data consumption Exports

[erincon01]

this capture shows the column names from data consumption export from two clients: one is EA, and the other CSP Azure Plan.

see that the column names for EA are CamelCase (first letter uppercase), and for CSP camelCase (first letter lowercase).
I know that CSP is not supported at this time, but you should take into account for next milestones.

in the EA dataset, billing period dates are:

• BillingPeriodStartDate
• BillingPeriodEndDate

However, in the function ftk_CostDetails from CostSummary.pbix, you expect:

• BillingPeriodStart
• BillingPeriodEnd

Other error is the x_SkuDetails, that seems is not created defined anywhere in Power Query.

It seems that the files you are using are different to the files we get from the Export in the Cost Management Portal.
For example, in your code you expect 60 columns:

while my dataset, has only 55 columns (EA client).

For debugging purposes, @flanakin would you share the .parquet files you use for developing.

Thanks.

[flanakin]

This is an awesome level of detail, @erincon01! Thank you!

The issue you're running into here is that you're using amortized cost exports instead of FOCUS exports. This was added to the docs, but it seems that wasn't clear enough 🙁

I also want to add that we do support CSP. CSP support was added with FinOps toolkit 0.1 as part of MCA support. In general, "MCA" is inclusive of CSP unless otherwise noted. The casing was normalized in 0.1.x Power BI reports. We removed that in 0.2 when we moved to FOCUS, which is already normalized across EA, MCA, MPA, and CSP with an Azure plan. (Trying to be very explicit 😉)

I'm doing a few things to help with this:

1. I'm adding an upgrade guide and will be more explicit about the upgrade steps when moving from 0.1.x to 0.2 and beyond.
2. I'll update docs to be more explicit about CSP support.
3. I'm working on a Power BI report that will help manage FinOps hubs.

Let me know if there's anything you'd like to see in the Power BI report for managing the hub. I'll probably keep it simple at first to get something out quickly, but it'll be good to know what people want to see.

[erincon01]

Thank you @flanakin for your answers.
How are the FOCUS exports created? AFAIK there is no option in the Cost Management Portal.
Sample csv or parquet files would help to match export files with the expected data for the Power BI Reports.

Thanks.

[BMichaelJ]

Hi @erincon01
This enables you to create the FOCUS export: https://aka.ms/exportsv2
Per documentation here: https://microsoft.github.io/finops-toolkit/hubs#-create-a-new-hub

[erincon01]

thanks @BMichaelJ .

my fault... the FOCUS option is not in the Azure Plan subscription:

however, in EA subsctiptions, the option is there:

thank you!

[erincon01]

@flanakin , a little OT, but...
I have a few tenants that dont have the FOCUS Exports enabled. How can I onboard the tenants on the Preview?

[flanakin]

@erincon01 FOCUS exports are supposed to be there for CSP subscriptions. I remember hearing about a bug but I thought that was fixed. Email [email protected] and share the screenshots above. They don't have any context on FinOps hubs, but as long as you tell them you don't see FOCUS exports for CSP subscriptions, that should be good. I'll also nudge internally.