AWS and Azure cost ingestion, reconciliation, normalization, and stored cost fields.

Cost

The Cost area of FinOps collects per-resource cost data for supported AWS and Azure accounts. Daily jobs load provisional usage for the previous day, and the monthly reconciliation job refreshes the previous month once provider billing data is finalized.

For canonical provider, resource type, and category terminology, see the Provider and Resource Reference.

Collection Schedule

Process	Timing	Purpose
Daily collection	Once per day	Ingest yesterday's cost data at per-resource granularity.
Monthly reconciliation	9th of each month	Re-fetch the full previous month and mark finalized entries as reconciled.

Provider billing data is finalized at different times:

Provider	Data Finalization Date	Reconciliation Behavior
AWS	7th of the following month	Re-fetch the full previous month from AWS CUR on the 9th.
Azure	5th of the following month	Re-fetch the full previous month from Azure Cost Details on the 9th.

Provider modifications made after the reconciliation date are not captured by the monthly reconciliation run.

AWS Cost Collection

AWS cost data is collected from AWS Cost and Usage Report data exported to an MCM-managed S3 bucket.

Required IAM Permissions

Permission	Purpose
`s3:CreateBucket`	Create the CUR export bucket during setup.
`s3:ListBucket`	List available CUR export objects.
`s3:GetObject`	Download CUR CSV files.
`s3:DeleteObject`	Clean up old export objects.
`bcm-data-exports:ListExport`	Inspect existing billing data exports.
`bcm-data-exports:CreateExport`	Create the CUR 2.0 export.
`bcm-data-exports:GetExport`	Read export configuration and status.
`bcm-data-exports:DeleteExport`	Remove the export when the account is deleted or FinOps expires.
`cur:putReportDefinition`	Create or update CUR report definitions where required.
`ce:ListCostAllocationTags`	Read cost allocation tags.
`ce:UpdateCostAllocationTagsStatus`	Enable required cost allocation tags.
`organizations:DescribeOrganization`	Read AWS organization context for billing setup.

CUR Setup

Step	Detail
Create S3 bucket	Bucket name `MCM-cur-{AccountID}` in `us-east-1`.

CUR Export Settings

Setting	Value
Export type	Standard data export
Export name	`MCM-cur-report` or preferred configured name
Data table	CUR 2.0
Include resource IDs	Yes
Time granularity	Daily
Compression	gzip text/csv
File versioning	Overwrite existing
S3 bucket	`MCM-cur-{AccountID}`
S3 path prefix	`MCM` or preferred configured prefix
S3 region	`us-east-1`

Reference: AWS CreateExport API

AWS Daily Flow

Download the CUR CSV file from the configured S3 path.
Filter data for yesterday, such as July 3 00:00:00-23:59
when the job runs on July 4.
Normalize the data to the stored cost schema.
Store cost records with reconciled=false.
Maintain a maximum of three monthly folders in the CUR bucket.
Delete the oldest monthly folder on the 9th of each month.

AWS Failure Scenarios

Scenario	Action
Cannot create CUR	Retry on every Job until CUR is set up.
Cannot connect to S3	Retry up to 3 times with a 10-second delay; log and skip if still failing.
CSV file not found in S3	Retry up to 3 times with a 10-second delay; log and skip if still failing.
Invalid data in CSV	Log the problematic record and skip it.
Missing S3 permissions	Store the error with Job details.

Azure Cost Collection

Azure cost data is collected from the Azure Cost Details Report API and normalized into the same stored schema as AWS cost data.

Required Permission

Permission	Purpose
`Microsoft.CostManagement/generateCostDetailsReport/read`	Generate and download Azure cost details reports.

Azure Daily Flow

Generate cost data with the Azure Cost Details Report API, filtered for yesterday.
Download the report CSV from the Blob link returned by the API response.
Normalize the data to the stored cost schema.
Store cost records with reconciled=false.

Reference: Azure Cost Details Report API

Azure Failure Scenarios

Scenario	Action
Azure Cost Details API failure	Retry up to 3 times with a 10-second delay.
Invalid CSV data	Log and skip invalid records.
Missing API permission	Store error in Job Detail.
API rate limit hit	Retry based on API response headers.

Stored Cost Fields

Cost records are stored in the FinOps data model used by the API and dashboards. Labels and provider tags are stored separately from the normalized cost row so they can be searched and joined by account and resource.

Field	Type	Description	Required Behavior
Provider	string	Canonical provider name, such as AWS or Azure.	Stored for every record.
Account ID	string	Account identifier from provider onboarding.	Stored for every record.
Account Name	string	Cloud account display name.	Stored when available.
Region	string	Resource region.	Skip the record if missing.
Category	string	Category such as Compute, Network, or Storage.	Default to Other when unknown.
Resource Type	string	Canonical uppercase snake-case resource type such as `AWS_EC2` or `AZURE_VM`.	Default to Other when unknown.
Resource Name	string	Resource display name.	Default to Resource ID when missing.
Resource ID	string	Unique provider resource identifier.	Skip the record if missing.
Cost Type	string	`DIRECT_COST` or `TRANSFER_COST`.	Default to `DIRECT_COST`.
Cost Sub-type	string	Provider usage metric or charge subtype.	Default to Other when unknown.
Quantity	number	Amount of resource usage.	Skip the record if missing.
Quantity Unit	string	Usage unit such as Hrs or GB.	Skip the record if missing.
Cloud Cost	number	Cost in the provider billing currency.	Skip the record if missing.
Cloud Cost Unit	string	Billing currency code such as USD.	Skip the record if missing.
Cost	number	Cost normalized to USD for cross-provider dashboards.	Stored when conversion is available.
Start Date	date	Start of the cost interval.	Skip the record if missing.
End Date	date	End of the cost interval.	Skip the record if missing.
Reconciled	boolean	`false` for daily records, `true` after monthly reconciliation.	Stored for every record.
Tags	list of `{key, value}`	Raw provider tags.

AWS CUR Field Mapping

MCM Field	Mapping Logic	AWS CUR Field
Provider	Static value: AWS.	—
Region	Extract from product data where key is region.	`product`
Resource Type	Regex match on Resource ID.	`line_item_resource_id`
Category	Derived from the canonical resource mapping in the Provider and Resource Reference.	—
Resource Name	Extract from labels where key is `user_name`.	`resource_tags`
Resource ID	Direct map.	`line_item_resource_id`
Cost Type	`TRANSFER_COST` if value contains transfer or data transfer, else `DIRECT_COST`.	`product_product_family`
Cost Sub-type	Direct map.	`line_item_usage_type`
Quantity	Direct map.	`line_item_usage_amount`
Quantity Unit	Direct map.	`pricing_unit`
Cloud Cost	Direct map.	`line_item_unblended_cost`
Cloud Cost Unit	Direct map.	`pricing_currency`
Start Date	Extract start from interval.	`identity_time_interval`
End Date	Extract end from interval.	`identity_time_interval`
Reconciled	`true` for monthly reconciliation; `false` for daily collection.	—

Azure Field Mapping

MCM Field	Mapping Logic	Azure Field
Provider	Static value: Azure.	—
Region	Direct map.	`ResourceLocation`
Resource Type	Regex match on Resource ID.	`ResourceId`
Category	Derived from the canonical resource mapping in the Provider and Resource Reference.	—
Resource Name	Direct map.	`ResourceName`
Resource ID	Direct map.	`ResourceId`
Cost Type	`TRANSFER_COST` if MeterName contains Data Transfer, else `DIRECT_COST`.	`MeterName`
Cost Sub-type	Concatenate MeterCategory and MeterSubCategory.	`MeterCategory`, `MeterSubCategory`
Quantity	Direct map.	`Quantity`
Quantity Unit	Direct map.	`UnitOfMeasure`
Cloud Cost	Direct map.	`CostInBillingCurrency`
Cloud Cost Unit	Direct map.	`BillingCurrencyCode`
Start Date	Direct map.	`BillingPeriodStartDate`
End Date	Direct map.	`BillingPeriodEndDate`
Reconciled	`true` for monthly reconciliation; `false` for daily collection.	—

Cost

On this page