Skip to content

[DOCS-11489] CCM new data source #30521

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: master
Choose a base branch
from

Conversation

rtrieu
Copy link
Contributor

@rtrieu rtrieu commented Jul 16, 2025

What does this PR do? What is the motivation?

  • Reorders a few steps based on updated UI
  • Updated screenshots
  • Adds info about data source limitations

Merge instructions

Merge readiness:

  • Ready for merge

For Datadog employees:

Your branch name MUST follow the <name>/<description> convention and include the forward slash (/). Without this format, your pull request will not pass CI, the GitLab pipeline will not run, and you won't get a branch preview. Getting a branch preview makes it easier for us to check any issues with your PR, such as broken links.

If your branch doesn't follow this format, rename it or create a new branch and PR.

[6/5/2025] Merge queue has been disabled on the documentation repo. If you have write access to the repo, the PR has been reviewed by a Documentation team member, and all of the required checks have passed, you can use the Squash and Merge button to merge the PR. If you don't have write access, or you need help, reach out in the #documentation channel in Slack.

Additional notes

@rtrieu rtrieu requested a review from a team as a code owner July 16, 2025 20:08
@rtrieu rtrieu added the WORK IN PROGRESS No review needed, it's a wip ;) label Jul 16, 2025
@github-actions github-actions bot added the Images Images are added/removed with this PR label Jul 16, 2025
Copy link
Contributor

github-actions bot commented Jul 16, 2025

📝 Documentation Team Review Required

This pull request requires approval from the @DataDog/documentation team before it can be merged.

Please ensure your changes follow our documentation guidelines and wait for a team member to review and approve your changes.

Copy link
Contributor

Preview links (active after the build_preview check completes)

Modified Files

@rtrieu rtrieu requested a review from helenmtang July 17, 2025 17:13
- Create suballocations by [partitioning](#step-5---optional-apply-a-partition) the allocation rule: **environment** (`env`).
- Choose the data source: **Database Queries**. Tip: Review available metrics and tags in the [Metrics Summary][2].
- Choose the [destination tag](#step-3---define-the-destination) to split your costs by: **Service** (`trace.caller.service`).
- Create suballocations by [partitioning](#step-4---optional-apply-a-partition) the allocation rule: **environment** (`env`).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Create suballocations by [partitioning](#step-4---optional-apply-a-partition) the allocation rule: **environment** (`env`).
- Create suballocations by [partitioning](#step-4---optional-apply-a-partition) the allocation rule. To do this, first apply another group by to the metric. In Network, do this by filling out the `View servers as` section: **Gateway ID** (`gateway_id`). This groups the metric by gateway_id. Then, select `server_gateway_id` in the **Partition source costs by** section.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this make sense? Should we add another sentence "Note: if you do not define a partition, the allocation rule will consider all group by tags as your destination tags, and split source costs across all destinations."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part is a little confusing - is the filter under Define the source or Choose split method?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specific to the Dynamic by metric, the filter, partition, and destination sections are all selected in the query editor (i'll send a screenshot to yu on slack!)

- Create suballocations by [partitioning](#step-5---optional-apply-a-partition) the allocation rule: **environment** (`env`).
- Choose the data source: **Database Queries**. Tip: Review available metrics and tags in the [Metrics Summary][2].
- Choose the [destination tag](#step-3---define-the-destination) to split your costs by: **Service** (`trace.caller.service`).
- Create suballocations by [partitioning](#step-4---optional-apply-a-partition) the allocation rule: **environment** (`env`).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Specific to the Dynamic by metric, the filter, partition, and destination sections are all selected in the query editor (i'll send a screenshot to yu on slack!)


### Step 4 - (optional) Apply filter(s)

Apply a filter across the entire allocation rule. Filters help you target the allocation rule to the relevant subset of your cloud spend.

_Example: Only apply cost allocation where environment is production._
- **Proportional by spend**: You can add filters to narrow the scope of your allocation. For example, filter by `aws_product:ec2` to create an allocation that only applies to EC2 costs, then proportionally distribute those costs based on each team's EC2 spend.
Copy link
Contributor

@helenmtang helenmtang Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Proportional by spend**: You can add filters to narrow the scope of your allocation. For example, filter by `aws_product:ec2` to create an allocation that only applies to EC2 costs, then proportionally distribute those costs based on each team's EC2 spend.
For example, let's take a **proportional by spend** allocation rule. Say you're allocating shared costs to the team tag. Filter by `aws_product:ec2` to create an allocation that proportionally distributes shared costs costs based on each team's EC2 spend.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you think about making this a "general" example? Since filters apply to multiple allocation rule types.


Decide which dimensions, such as `team`, `department`, or `service`, receive the allocated costs. For example:
1. Select the destinations you want to allocate costs to, such as `team`, `department`, or `service`, that receive the allocated costs.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
1. Select the destinations you want to allocate costs to, such as `team`, `department`, or `service`, that receive the allocated costs.
1. Select the destinations you want to allocate costs to, such as `team`, `department`, or `service`.

- Choose the allocation method: **Proportional by spend**.
- Choose the [destination tag](#step-3---define-the-destination) to split your costs by: **User** (`User A`, `User B`, `User C`).
- Refine the allocation by applying [filters](#step-4---optional-apply-filters): **EC2** (`aws_product:ec2`).
- Create suballocations by [partitioning](#step-5---optional-apply-a-partition) the allocation rule: **environment** (`env`).
- Create suballocations by [partitioning](#step-4---optional-apply-a-partition) the allocation rule: **environment** (`env`).

You can also specify how cost proportions should be partitioned to ensure segment-specific allocations. For example, if you partition your costs by environment using tags like `staging` and `production`, the proportions are calculated separately for each environment. This ensures allocations are based on the specific proportions within each partition.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
You can also specify how cost proportions should be partitioned to ensure segment-specific allocations. For example, if you partition your costs by environment using tags like `staging` and `production`, the proportions are calculated separately for each environment. This ensures allocations are based on the specific proportions within each partition.
You can also specify how the cost allocation rule should be partitioned to create multiple suballocations. For example, if you partition your costs by the `environment` tag, the allocation rule is calculated separately for each environment.

- Create suballocations by [partitioning](#step-5---optional-apply-a-partition) the allocation rule: **environment** (`env`).

{{< img src="cloud_cost/custom_allocation_rules/ui-dynamic-by-metric.png" alt="The dynamic by metric split strategy as seen in Datadog" style="width:90%;" >}}
For example, the Network query `sum:network.bytes_written[server_gateway_id:nat-*] by client_service and server_gateway_id` (shown below) tracks the total traffic volume through NAT gateways by service. The relative values are then used to determine what proportion of total NAT gateway costs should be allocated to each service.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
For example, the Network query `sum:network.bytes_written[server_gateway_id:nat-*] by client_service and server_gateway_id` (shown below) tracks the total traffic volume through NAT gateways by service. The relative values are then used to determine what proportion of total NAT gateway costs should be allocated to each service.
For example, the Network query `sum:network.bytes_written[server_gateway_id:nat-*] by client_service and server_gateway_id` (shown below) tracks the total traffic volume through NAT gateways by service. The relative traffic volume per `client_service` and `server_gateway_id` is then used to determine what proportion of total NAT gateway costs should be allocated to each destination.


### Data source limitations

Before creating this type of rule, be aware:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Before creating this type of rule, be aware:
Before creating this type of rule, please note:


Before creating this type of rule, be aware:

- **Data sources support** - The dynamic by metric allocation method only supports specific data sources, listed in the dropdown below.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Data sources support** - The dynamic by metric allocation method only supports specific data sources, listed in the dropdown below.

- In the **"Define the metric to split the source costs"** section (which uses the same interface as the Metrics Explorer query editor):
- **Filters**: Apply filters to refine your metric query. For example, add `server_gateway_id:nat-*` to filter the metric to only return data for your NAT Gateway usage. These filters become the filter for your allocation.
- **Group bys**: Select group bys to define your destination and partition:
- **Destination**: The primary group by becomes your destination tag. For example, group by `client_service` to split costs by service.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Destination**: The primary group by becomes your destination tag. For example, group by `client_service` to split costs by service.
- **Destination**: By default, the tag(s) you group by become the destination tag(s) for your allocation. For example, group by `client_service` to split costs by service.

- **Filters**: Apply filters to refine your metric query. For example, add `server_gateway_id:nat-*` to filter the metric to only return data for your NAT Gateway usage. These filters become the filter for your allocation.
- **Group bys**: Select group bys to define your destination and partition:
- **Destination**: The primary group by becomes your destination tag. For example, group by `client_service` to split costs by service.
- **Partition**: Additional group bys become your partition. For example, add another group by for `gateway_id` to create suballocations by Gateway ID.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Partition**: Additional group bys become your partition. For example, add another group by for `gateway_id` to create suballocations by Gateway ID.
- **Partition**: To define a partition, first group by the tag you wish to partition on. For example, add another group by for `gateway_id` to create suballocations by Gateway ID.

- **Group bys**: Select group bys to define your destination and partition:
- **Destination**: The primary group by becomes your destination tag. For example, group by `client_service` to split costs by service.
- **Partition**: Additional group bys become your partition. For example, add another group by for `gateway_id` to create suballocations by Gateway ID.
- In section 3, **"Choose the destination(s) to split costs across"**, specify which group by tag is your destination and which will be your partition.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- In section 3, **"Choose the destination(s) to split costs across"**, specify which group by tag is your destination and which will be your partition.
- In section 3, **"Choose the destination(s) to split costs across"**, specify which group by tag is your partition tag. All group by tags are destination tags by default.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let me know if these changes make sense!


- **Proportional by spend**: Let's say you allocate shared costs to the team tag, proportional to how much each team spends. You can add a filter, creating a cost allocation that is proportional to how much team spends on `aws_product` is `ec2`.
- **Dynamic by metric**: Let's say you allocate shared PostgreSQL costs to the service tag, proportional to the query execution time of each service. You can add a filter, creating a cost allocation that only applies where `environment` is `production`.
- **Dynamic by metric**: You can filter your metric query to focus on specific data. For example, add `environment:production` to your metric query so the allocation is based only on production environment usage data.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Dynamic by metric**: You can filter your metric query to focus on specific data. For example, add `environment:production` to your metric query so the allocation is based only on production environment usage data.


### Step 4 - (optional) Apply filter(s)

Apply a filter across the entire allocation rule. Filters help you target the allocation rule to the relevant subset of your cloud spend.

_Example: Only apply cost allocation where environment is production._
- **Proportional by spend**: You can add filters to narrow the scope of your allocation. For example, filter by `aws_product:ec2` to create an allocation that only applies to EC2 costs, then proportionally distribute those costs based on each team's EC2 spend.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you think about making this a "general" example? Since filters apply to multiple allocation rule types.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Images Images are added/removed with this PR WORK IN PROGRESS No review needed, it's a wip ;)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants