The GenAI Impact report helps you measure and report on the impact of AI code assistants.

Setup instructions

To set up the report, you need a completed Snapshot that includes:

The AI Code Assistant workflow item
User attribute for AI code assistant usage

The User attribute can be populated through self-report in a Snapshot, or uploaded into DX using external data. If you have GitHub Copilot Usage data connector enabled, you can run the SQL query below in Data Studio to generate a CSV of usage data which can then be uploaded.

WITH recent_usage AS (
  SELECT 
    gcd.login,
    COUNT(CASE WHEN gcd.active = true THEN 1 END) as active_days
  FROM github_copilot_daily_usages gcd
  WHERE gcd.date >= CURRENT_DATE - INTERVAL '30 days'
  GROUP BY gcd.login
)
SELECT 
  du.email,
  CASE 
    WHEN ru.active_days >= 20 THEN 'Daily'
    WHEN ru.active_days >= 3 THEN 'Weekly'
    WHEN ru.active_days >= 1 THEN 'Monthly'
    ELSE 'Never'
  END as "attr:Copilot usage"
FROM dx_users du
LEFT JOIN identities i ON i.dx_user_id = du.id AND i.source = 'github'
LEFT JOIN github_users gu ON gu.id = i.source_id
LEFT JOIN recent_usage ru ON LOWER(ru.login) = LOWER(gu.login)
WHERE du.deleted_at IS NULL
ORDER BY COALESCE(ru.active_days, 0) DESC

ROI calculations

The ROI tab provides data on cost savings, alongside industry benchmarks and potential unrealized cost savings.

Current cost savings per developer are calculated as follows:

DX computes the average per-developer weekly time savings for each cohort (e.g., “Daily Active User”, “Weekly Active User”).
For each cohort, DX multiplies weekly time savings by the number of developers in the cohort, then multiples that number by 50 [weeks] to compute the annualized time savings per cohort.
DX takes the sum of all cohorts’ annualized time savings to calculate total annualized time savings for the organization.
DX divides the annualized time savings for the organization by the number of developers, calculating the average annualized time savings per developer.
DX multiplies annualized time savings for the organization by fully loaded FTE cost, divided by the number of work hours per year (2,080 hours), to calculate total annualized dollar savings for the organization.

Potential cost savings are calculated by comparing the current cost savings against the hypothetical state in which all developers are attaining the same amount of time savings as the 75th percentile of the most active cohort.

Correlations

DX correlates various developer metrics against GenAI usage to provide additional signal into how GenAI usage is affecting developer productivity.

Below are a few tips:

To control how your developer population is segmented, click the gear icon to access report settings where you can configure which cohorts should be treated as “Active users”.

To get more information about the metrics being correlated, read the tooltips displayed in the titles of each chart.

Don’t be alarmed if correlations don’t tell the story you were expecting—correlation is not causation, and many factors can influence metrics.