-
Notifications
You must be signed in to change notification settings - Fork 1.2k
[DOCS-11489] CCM new data source #30521
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: master
Are you sure you want to change the base?
Conversation
📝 Documentation Team Review RequiredThis pull request requires approval from the @DataDog/documentation team before it can be merged. Please ensure your changes follow our documentation guidelines and wait for a team member to review and approve your changes. |
Preview links (active after the
|
- Create suballocations by [partitioning](#step-5---optional-apply-a-partition) the allocation rule: **environment** (`env`). | ||
- Choose the data source: **Database Queries**. Tip: Review available metrics and tags in the [Metrics Summary][2]. | ||
- Choose the [destination tag](#step-3---define-the-destination) to split your costs by: **Service** (`trace.caller.service`). | ||
- Create suballocations by [partitioning](#step-4---optional-apply-a-partition) the allocation rule: **environment** (`env`). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Create suballocations by [partitioning](#step-4---optional-apply-a-partition) the allocation rule: **environment** (`env`). | |
- Create suballocations by [partitioning](#step-4---optional-apply-a-partition) the allocation rule. To do this, first apply another group by to the metric. In Network, do this by filling out the `View servers as` section: **Gateway ID** (`gateway_id`). This groups the metric by gateway_id. Then, select `server_gateway_id` in the **Partition source costs by** section. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Does this make sense? Should we add another sentence "Note: if you do not define a partition, the allocation rule will consider all group by tags as your destination tags, and split source costs across all destinations."
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This part is a little confusing - is the filter under Define the source or Choose split method?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Specific to the Dynamic by metric, the filter, partition, and destination sections are all selected in the query editor (i'll send a screenshot to yu on slack!)
Co-authored-by: helenmtang <[email protected]>
Co-authored-by: helenmtang <[email protected]>
Co-authored-by: Rosa Trieu <[email protected]>
- Create suballocations by [partitioning](#step-5---optional-apply-a-partition) the allocation rule: **environment** (`env`). | ||
- Choose the data source: **Database Queries**. Tip: Review available metrics and tags in the [Metrics Summary][2]. | ||
- Choose the [destination tag](#step-3---define-the-destination) to split your costs by: **Service** (`trace.caller.service`). | ||
- Create suballocations by [partitioning](#step-4---optional-apply-a-partition) the allocation rule: **environment** (`env`). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Specific to the Dynamic by metric, the filter, partition, and destination sections are all selected in the query editor (i'll send a screenshot to yu on slack!)
|
||
### Step 4 - (optional) Apply filter(s) | ||
|
||
Apply a filter across the entire allocation rule. Filters help you target the allocation rule to the relevant subset of your cloud spend. | ||
|
||
_Example: Only apply cost allocation where environment is production._ | ||
- **Proportional by spend**: You can add filters to narrow the scope of your allocation. For example, filter by `aws_product:ec2` to create an allocation that only applies to EC2 costs, then proportionally distribute those costs based on each team's EC2 spend. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- **Proportional by spend**: You can add filters to narrow the scope of your allocation. For example, filter by `aws_product:ec2` to create an allocation that only applies to EC2 costs, then proportionally distribute those costs based on each team's EC2 spend. | |
For example, let's take a **proportional by spend** allocation rule. Say you're allocating shared costs to the team tag. Filter by `aws_product:ec2` to create an allocation that proportionally distributes shared costs costs based on each team's EC2 spend. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what do you think about making this a "general" example? Since filters apply to multiple allocation rule types.
|
||
Decide which dimensions, such as `team`, `department`, or `service`, receive the allocated costs. For example: | ||
1. Select the destinations you want to allocate costs to, such as `team`, `department`, or `service`, that receive the allocated costs. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
1. Select the destinations you want to allocate costs to, such as `team`, `department`, or `service`, that receive the allocated costs. | |
1. Select the destinations you want to allocate costs to, such as `team`, `department`, or `service`. |
- Choose the allocation method: **Proportional by spend**. | ||
- Choose the [destination tag](#step-3---define-the-destination) to split your costs by: **User** (`User A`, `User B`, `User C`). | ||
- Refine the allocation by applying [filters](#step-4---optional-apply-filters): **EC2** (`aws_product:ec2`). | ||
- Create suballocations by [partitioning](#step-5---optional-apply-a-partition) the allocation rule: **environment** (`env`). | ||
- Create suballocations by [partitioning](#step-4---optional-apply-a-partition) the allocation rule: **environment** (`env`). | ||
|
||
You can also specify how cost proportions should be partitioned to ensure segment-specific allocations. For example, if you partition your costs by environment using tags like `staging` and `production`, the proportions are calculated separately for each environment. This ensures allocations are based on the specific proportions within each partition. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You can also specify how cost proportions should be partitioned to ensure segment-specific allocations. For example, if you partition your costs by environment using tags like `staging` and `production`, the proportions are calculated separately for each environment. This ensures allocations are based on the specific proportions within each partition. | |
You can also specify how the cost allocation rule should be partitioned to create multiple suballocations. For example, if you partition your costs by the `environment` tag, the allocation rule is calculated separately for each environment. |
- Create suballocations by [partitioning](#step-5---optional-apply-a-partition) the allocation rule: **environment** (`env`). | ||
|
||
{{< img src="cloud_cost/custom_allocation_rules/ui-dynamic-by-metric.png" alt="The dynamic by metric split strategy as seen in Datadog" style="width:90%;" >}} | ||
For example, the Network query `sum:network.bytes_written[server_gateway_id:nat-*] by client_service and server_gateway_id` (shown below) tracks the total traffic volume through NAT gateways by service. The relative values are then used to determine what proportion of total NAT gateway costs should be allocated to each service. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
For example, the Network query `sum:network.bytes_written[server_gateway_id:nat-*] by client_service and server_gateway_id` (shown below) tracks the total traffic volume through NAT gateways by service. The relative values are then used to determine what proportion of total NAT gateway costs should be allocated to each service. | |
For example, the Network query `sum:network.bytes_written[server_gateway_id:nat-*] by client_service and server_gateway_id` (shown below) tracks the total traffic volume through NAT gateways by service. The relative traffic volume per `client_service` and `server_gateway_id` is then used to determine what proportion of total NAT gateway costs should be allocated to each destination. |
|
||
### Data source limitations | ||
|
||
Before creating this type of rule, be aware: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Before creating this type of rule, be aware: | |
Before creating this type of rule, please note: |
|
||
Before creating this type of rule, be aware: | ||
|
||
- **Data sources support** - The dynamic by metric allocation method only supports specific data sources, listed in the dropdown below. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- **Data sources support** - The dynamic by metric allocation method only supports specific data sources, listed in the dropdown below. |
- In the **"Define the metric to split the source costs"** section (which uses the same interface as the Metrics Explorer query editor): | ||
- **Filters**: Apply filters to refine your metric query. For example, add `server_gateway_id:nat-*` to filter the metric to only return data for your NAT Gateway usage. These filters become the filter for your allocation. | ||
- **Group bys**: Select group bys to define your destination and partition: | ||
- **Destination**: The primary group by becomes your destination tag. For example, group by `client_service` to split costs by service. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- **Destination**: The primary group by becomes your destination tag. For example, group by `client_service` to split costs by service. | |
- **Destination**: By default, the tag(s) you group by become the destination tag(s) for your allocation. For example, group by `client_service` to split costs by service. |
- **Filters**: Apply filters to refine your metric query. For example, add `server_gateway_id:nat-*` to filter the metric to only return data for your NAT Gateway usage. These filters become the filter for your allocation. | ||
- **Group bys**: Select group bys to define your destination and partition: | ||
- **Destination**: The primary group by becomes your destination tag. For example, group by `client_service` to split costs by service. | ||
- **Partition**: Additional group bys become your partition. For example, add another group by for `gateway_id` to create suballocations by Gateway ID. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- **Partition**: Additional group bys become your partition. For example, add another group by for `gateway_id` to create suballocations by Gateway ID. | |
- **Partition**: To define a partition, first group by the tag you wish to partition on. For example, add another group by for `gateway_id` to create suballocations by Gateway ID. |
- **Group bys**: Select group bys to define your destination and partition: | ||
- **Destination**: The primary group by becomes your destination tag. For example, group by `client_service` to split costs by service. | ||
- **Partition**: Additional group bys become your partition. For example, add another group by for `gateway_id` to create suballocations by Gateway ID. | ||
- In section 3, **"Choose the destination(s) to split costs across"**, specify which group by tag is your destination and which will be your partition. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- In section 3, **"Choose the destination(s) to split costs across"**, specify which group by tag is your destination and which will be your partition. | |
- In section 3, **"Choose the destination(s) to split costs across"**, specify which group by tag is your partition tag. All group by tags are destination tags by default. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
let me know if these changes make sense!
|
||
- **Proportional by spend**: Let's say you allocate shared costs to the team tag, proportional to how much each team spends. You can add a filter, creating a cost allocation that is proportional to how much team spends on `aws_product` is `ec2`. | ||
- **Dynamic by metric**: Let's say you allocate shared PostgreSQL costs to the service tag, proportional to the query execution time of each service. You can add a filter, creating a cost allocation that only applies where `environment` is `production`. | ||
- **Dynamic by metric**: You can filter your metric query to focus on specific data. For example, add `environment:production` to your metric query so the allocation is based only on production environment usage data. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- **Dynamic by metric**: You can filter your metric query to focus on specific data. For example, add `environment:production` to your metric query so the allocation is based only on production environment usage data. |
|
||
### Step 4 - (optional) Apply filter(s) | ||
|
||
Apply a filter across the entire allocation rule. Filters help you target the allocation rule to the relevant subset of your cloud spend. | ||
|
||
_Example: Only apply cost allocation where environment is production._ | ||
- **Proportional by spend**: You can add filters to narrow the scope of your allocation. For example, filter by `aws_product:ec2` to create an allocation that only applies to EC2 costs, then proportionally distribute those costs based on each team's EC2 spend. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what do you think about making this a "general" example? Since filters apply to multiple allocation rule types.
What does this PR do? What is the motivation?
Merge instructions
Merge readiness:
For Datadog employees:
Your branch name MUST follow the
<name>/<description>
convention and include the forward slash (/
). Without this format, your pull request will not pass CI, the GitLab pipeline will not run, and you won't get a branch preview. Getting a branch preview makes it easier for us to check any issues with your PR, such as broken links.If your branch doesn't follow this format, rename it or create a new branch and PR.
[6/5/2025] Merge queue has been disabled on the documentation repo. If you have write access to the repo, the PR has been reviewed by a Documentation team member, and all of the required checks have passed, you can use the Squash and Merge button to merge the PR. If you don't have write access, or you need help, reach out in the #documentation channel in Slack.
Additional notes