Track AI adoption and impact

Implement with AI

Send this guide to your coding agent.

Prerequisite: Install Port MCP

Open in ClaudeOpen in CodexOpen in Cursor

Open plan mode if your tool supports it; otherwise present the plan below filled in and wait for my approval. Implement this Port guide in my org via MCP:

https://docs.port.io/guides/all/track-ai-adoption-and-impact

Read the raw markdown version at https://docs.port.io/guides/all/track-ai-adoption-and-impact.md - it contains every tab and code block without page markup.

Goal: get the guide's core flow working end-to-end in my org; adapting it to fit my existing setup takes priority over matching the guide 1:1.

Plan:
1. Confirm MCP is connected, in the right org, with sufficient permissions.
2. If the guide offers alternative implementation paths (tabs), pick the one matching my installed integrations and tools, confirm it with me, and implement only that path.
3. Diff the guide's data model (blueprints, properties, relations, workflows, actions, agents, automations, integrations, webhook data sources, secrets) against mine.
4. Propose adaptations for gaps, reusing existing blueprints/relations over guide-named duplicates.
5. Flag what needs a UI click, credential, or secret from me, testing MCP capability empirically before ruling anything out. If the guide has a "Set up via API" section, use it for anything MCP can't do before treating a step as UI-only.
6. Stop on any blocker and give me options. Approving this plan authorizes the writes it lists; pause only for writes beyond what's listed.

Build:
- Extend blueprint schema additively when upserting; don't remove or overwrite existing properties, and treat type conflicts as a blocker, not an auto-fix.
- Never print secret values into the chat or logs; ask me to set them in Port, or write them via the secrets API without echoing them back.
- List any mock data in the plan, minimal and labeled mock; once approved, seed it without re-asking, and tell me what you seeded.
- For anything the guide writes downstream (e.g. a webhook target), use a real entity, not a mock.
- For pages/widgets, use the real page identifier from the app URL, not a guessed slug.
- When you hit a UI step confirmed (not assumed) unsupported via MCP and not covered by the guide's API sections, pause, give exact clicks, then resume via MCP.
- Validate and give links after each meaningful step (only a tool-returned URL, no guessed paths); don't proceed if the last run wasn't a success.

Done:
- Run the guide's "Let's test it" steps where possible (e.g. execute a workflow test run) and confirm the expected output exists in Port.
- Summarize adaptations, seeded data, what was mocked or skipped, remaining UI steps, and how to verify.

Tracking AI adoption provides visibility into how engineering teams are leveraging AI coding agents and the impact these tools have on delivery outcomes. Without visibility into AI usage patterns, teams struggle to understand adoption rates, measure productivity gains, and identify which teams or services benefit most from AI assistance.

This guide helps engineering managers, platform engineers, and product leaders answer critical questions about AI adoption:

• Adoption: Which teams and services are using AI coding tools?
• Impact: How does AI assistance affect PR throughput and cycle time?
• Effectiveness: Are AI-assisted PRs delivering faster or higher quality outcomes?
• Cost optimization: Are AI tool licenses being fully utilized?

By the end of this guide, we will have dashboards that track AI adoption metrics, enabling you to understand usage patterns, measure productivity impact, and make informed decisions about AI tool investments across your organization.

Common use cases

• Compare PR throughput and cycle time between AI-assisted and traditional workflows.
• Identify teams and services with the highest AI adoption rates.
• Track AI usage trends over time to understand adoption patterns.
• Monitor AI tool license utilization and revoke unused licenses to optimize costs.
• Correlate AI adoption with operational metrics like bug rates and incident resolution time.

Prerequisites

This guide assumes the following:

• You have a Port account and have completed the onboarding process.
• Port's GitHub Ocean integration is installed in your account.
• The githubPullRequest and githubRepository blueprints are already created (these are created when you install GitHub Ocean).
• For Cursor tracking: Port's Cursor integration is installed in your account, with the cursor_usage_record, cursor_user_usage_record, and cursor_team_usage_record blueprints created as described in the Cursor integration setup.
• For Copilot tracking: Port's GitHub Copilot integration is installed in your account.
• For AI impact on operations: Port's Jira integration is installed in your account (optional, for tracking bugs, security issues, and incidents).
• For the AI Impact Insights widget: Port AI is enabled in your account.

Set up data model

We will create and update blueprints to support AI adoption tracking.

Update the pull request blueprint

Regardless of which AI tool you're tracking, you need to add a property to identify AI-assisted PRs.

1. Go to the Builder page in Port.

2. Find the githubPullRequest blueprint and click on it.

3. Click on the {...} button in the top right corner, and choose Edit JSON.

4. Add the following property to the properties section:

   AI agent property for Pull Request blueprint (click to expand)

   "created_by_agent": {
     "type": "boolean",
     "title": "Created By AI Agent",
     "description": "Determines whether or not the PR was created by an AI agent such as Copilot, Claude, or Devin"
   }

5. Click Save to update the blueprint.

Update the Jira issue blueprint

To correlate AI adoption with operational metrics like bug resolution and incident response time, we need to add properties to track issue categories and lead time.

1. Go to the Builder page in Port.

2. Find the jiraIssue blueprint and click on it.

3. Click on the {...} button in the top right corner, and choose Edit JSON.

4. Add the following properties to the properties section:

   Issue category and lead time properties (click to expand)

   "issue_category": {
     "title": "Issue Category",
     "type": "string",
     "enum": ["Bug", "Security Issues", "Incidents", "Other"],
     "enumColors": {
       "Bug": "red",
       "Security Issues": "orange",
       "Incidents": "pink",
       "Other": "lightGray"
     },
     "description": "Category of the issue for tracking purposes"
   },
   "lead_time_hours": {
     "title": "Lead Time (Hours)",
     "type": "number",
     "description": "Time from issue creation to resolution in hours"
   }

5. Click Save to update the blueprint.

Update integration mapping

Now we'll configure the GitHub integration to detect AI-assisted PRs and Jira issues to populate the issue category and lead time properties.

GitHub integration mapping

1. Go to your Data Source page.

2. Select the GitHub integration.

3. Add or update the following YAML block in the editor. Choose your integration type:

   GitHub Ocean integration configuration (click to expand)

   resources:
     - kind: repository
       selector:
         query: 'true'
         include: ["teams"]
         includedFiles:
           - README.md
       port:
         entity:
           mappings:
             identifier: .full_name
             title: .name
             blueprint: '"githubRepository"'
             properties:
               readme: .__includedFiles["README.md"]
               url: .html_url
               defaultBranch: .default_branch
               last_push: .pushed_at
               visibility: .visibility
               language: .language
     - kind: pull-request
       selector:
         query: 'true'
         states: ["open", "closed"]
         maxResults: 100
         since: 90
       port:
         entity:
           mappings:
             identifier: .id|tostring
             title: .title
             blueprint: '"githubPullRequest"'
             properties:
               status: .state
               closedAt: .closed_at
               updatedAt: .updated_at
               mergedAt: .merged_at
               createdAt: .created_at
               link: .html_url
               created_by_agent: .user.login | ascii_downcase | test("(copilot|claude|devin)")
               leadTimeHours: >-
                 (.created_at as $createdAt | .merged_at as $mergedAt | ($createdAt
                 | sub("\\..*Z$"; "Z") | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime)
                 as $createdTimestamp | ($mergedAt | if . == null then null else
                 sub("\\..*Z$"; "Z") | strptime("%Y-%m-%dT%H:%M:%SZ") | mktime end)
                 as $mergedTimestamp | if $mergedTimestamp == null then null else
                 (((($mergedTimestamp - $createdTimestamp) / 3600) * 100 | floor) /
                 100) end)
               pr_age: >-
                 ((now - (.created_at | sub("\\.[0-9]+Z$"; "Z") | fromdateiso8601))
                 / 86400) | round
               cycle_time: >-
                 if .merged_at then (((.merged_at   | sub("\\.[0-9]+Z$"; "Z") |
                 fromdateiso8601) - (.created_at | sub("\\.[0-9]+Z$"; "Z") |
                 fromdateiso8601)) / 86400 | round) else null end
             relations:
               repository: .__repository

   Ocean property mapping

   For GitHub Ocean, map status from .state in your entity mappings, and use .__repository for the repository relation. See the GitHub Ocean migration guide for details.

AI agent detection

The mapping uses a regex pattern to detect AI agents by checking if the PR creator's username contains "copilot", "claude", or "devin" (case-insensitive). You can customize this pattern to match your organization's AI agent naming conventions.

4. Click Save & Resync to apply the mapping.

Configure Jira integration mapping

1. Go to your Data Source page.

2. Select the Jira integration.

3. Update the jiraIssue mapping to include the new properties:

   Jira integration mapping for issue properties (click to expand)

   - kind: issue
     selector:
       query: 'true'
     port:
       entity:
         mappings:
           identifier: .key
           title: .fields.summary
           blueprint: '"jiraIssue"'
           properties:
             # ... existing properties ...
             issue_category: |
               if .fields.labels | any(. == "security") then "Security Issues"
               elif .fields.issuetype.name == "Bug" then "Bug"
               elif .fields.issuetype.name == "Incident" then "Incidents"
               else "Other" end
             lead_time_hours: |
               if .fields.resolutiondate != null then
                 ((.fields.resolutiondate | fromdateiso8601) - (.fields.created | fromdateiso8601)) / 3600 | round
               else
                 null
               end

   Issue category logic

   The mapping categorizes issues based on labels and issue type. Issues with a "security" label are classified as Security Issues, Bug types as Bug, and Incident types as Incidents. You can customize this logic to match your organization's issue classification.

4. Click Save & Resync to apply the mapping.

Visualize metrics

Once the data is synced, we can create a dedicated dashboard in Port to monitor and analyze AI adoption and impact metrics.

Create a dashboard

1. Navigate to your context lake.
2. Click on the + button in the left sidebar.
3. Select New dashboard.
4. Name the dashboard AI Adoption and Impact.
5. Click Create.

We now have a blank dashboard where we can start adding widgets to visualize AI adoption metrics.

Add widgets

In the new dashboard, create the following widgets to match the demo dashboard. Some widgets require the Cursor integration, while others work with any AI tool.

Cursor integration widgets

Widgets that use the cursor_usage_record, cursor_user_usage_record, or cursor_team_usage_record blueprints require Port's Cursor integration to be installed and syncing data.

Cursor adoption across users pie chart (click to expand)

This chart shows the distribution of active vs inactive Cursor users, giving a quick view of adoption across your organization.

1. Click + Widget and select Pie chart.
2. Title: Cursor Adoption Across Users.
3. Description: Based on active usage.
4. Choose the User blueprint.
5. Under Breakdown by property, select the Cursor Active property.
6. Click Save.

Total number of active Cursor users (monthly) line chart (click to expand)

1. Click + Widget and select Line Chart.
2. Title: Total Number of Active Cursor Users (Monthly).
3. Description: Active Cursor Users during the month.
4. Set X axis:
   • Title: Date.
   • Time interval: Month.
   • Time range: In the past 6 months.
5. Set Y axis title: Number of Users.
6. Click + Line and configure:
   • Title: Total Active Users.
   • Chart type: Aggregate by property.
   • Blueprint: Cursor Usage Record.
   • Property: total_active_users.
   • Function: Sum.
   • Measure time by: record_date.
7. Click Save.

AI Cursor usage trend over time line chart (click to expand)

This chart tracks AI coding activity over time, showing the trend of rejected suggestions and lines of code added or deleted.

1. Click + Widget and select Line Chart.
2. Title: AI Cursor Usage Trend Over Time.
3. Description: Track AI coding activity over time.
4. Set X axis:
   • Title: Date.
   • Time interval: Week.
   • Time range: In the past 30 days.
5. Set Y axis title: Count.
6. Click + Line and configure:
   • Title: Total Rejects.
   • Chart type: Aggregate by property.
   • Blueprint: Cursor Usage Record.
   • Property: Total Rejects.
   • Function: Average.
   • Measure time by: record_date.
7. Click + Line and configure:
   • Title: Total Lines Added.
   • Chart type: Aggregate by property.
   • Blueprint: Cursor Usage Record.
   • Property: Total Lines Added.
   • Function: Average.
   • Measure time by: record_date.
8. Click + Line and configure:
   • Title: Total Lines Deleted.
   • Chart type: Aggregate by property.
   • Blueprint: Cursor Usage Record.
   • Property: Total Lines Deleted.
   • Function: Average.
   • Measure time by: record_date.
9. Click Save.

Daily cost trends line chart (click to expand)

1. Click + Widget and select Line Chart.
2. Title: Daily Cost Trends.
3. Description: Track AI usage costs over time.
4. Set X axis:
   • Title: Date.
   • Time interval: Week.
   • Time range: In the past 30 days.
5. Set Y axis title: Cost.
6. Click + Line and configure:
   • Title: Total Cost.
   • Chart type: Aggregate by property.
   • Blueprint: Cursor Usage Record.
   • Property: total_cents.
   • Function: Sum.
   • Measure time by: record_date.
7. Click Save.

Average Lead Time (Monthly) line chart (click to expand)

1. Click + Widget and select Line Chart.

2. Title: Average Lead Time (Monthly).

3. Description: Shows the average number of days from issue creation to resolution.

4. Set X axis:

   • Title: Date.
   • Time interval: Month.
   • Time range: In the past 6 months.

5. Set Y axis title: Lead Time (Days).

6. Click + Line and configure:

   • Title: Lead Time.

   • Chart type: Aggregate by property.

   • Blueprint: Pull Request.

   • Property: leadTimeHours.

   • Function: Average.

   • Measure time by: createdAt.

   • Additional filters:

     {
       "combinator": "and",
       "rules": [
         {
           "value": "merged",
           "property": "status",
           "operator": "="
         }
       ]
     }

7. Click Save.

PR Throughput (Monthly) line chart (click to expand)

1. Click + Widget and select Line Chart.

2. Title: PR Throughput (Monthly).

3. Description: Monthly PRs merged.

4. Set X axis:

   • Title: Date.
   • Time interval: Month.
   • Time range: In the past 6 months.

5. Set Y axis title: Pull Requests.

6. Click + Line and configure:

   • Title: PR Count.

   • Chart type: Count entities.

   • Blueprint: Pull Request.

   • Function: Count.

   • Measure time by: createdAt.

   • Additional filters:

     {
       "combinator": "and",
       "rules": [
         {
           "value": "merged",
           "property": "status",
           "operator": "="
         }
       ]
     }

7. Click Save.

PR Cycle Time (Monthly) line chart (click to expand)

1. Click + Widget and select Line Chart.

2. Title: PR Cycle Time (Monthly).

3. Description: Monthly average PR time to merge (hours).

4. Set X axis:

   • Title: Date.
   • Time interval: Month.
   • Time range: In the past 6 months.

5. Set Y axis title: Lead time (Hours).

6. Click + Line and configure:

   • Title: Cycle Time.

   • Chart type: Aggregate by property.

   • Blueprint: Pull Request.

   • Property: cycle_time.

   • Function: Average.

   • Measure time by: createdAt.

   • Additional filters:

     {
       "combinator": "and",
       "rules": [
         {
           "value": "merged",
           "property": "status",
           "operator": "="
         }
       ]
     }

7. Click Save.

AI adoption impact on operations line chart (click to expand)

This chart shows the monthly trend of bugs, security issues, and incidents to help correlate AI adoption with operational quality.

1. Click + Widget and select Line Chart.

2. Title: AI Adoption Impact On Operations.

3. Description: Issues created each month.

4. Set X axis:

   • Title: Date.
   • Time interval: Month.
   • Time range: In the past 6 months.

5. Set Y axis title: Issues.

6. Click + Line and configure:

   • Title: Issue Count.

   • Chart type: Count entities.

   • Blueprint: Jira Issue.

   • Breakdown by property: Issue Category.

   • Measure time by: createdAt.

   • Additional filters (to only show relevant categories):

     {
       "combinator": "and",
       "rules": [
         {
           "property": "issue_category",
           "operator": "in",
           "value": ["Bug", "Security Issues", "Incidents"]
         }
       ]
     }

7. Click Save.

Teams with the most AI-created PRs bar chart (click to expand)

1. Click + Widget and select Bar Chart.

2. Title: Teams with the most AI-created PRs.

3. Choose the Pull Request blueprint.

4. Under Breakdown by property, select the Owning Team property (via repository relation).

5. Add this JSON to the Additional filters editor:

   {
     "combinator": "and",
     "rules": [
       {
         "value": true,
         "property": "created_by_agent",
         "operator": "="
       }
     ]
   }

6. Click Save.

PRs created with AI pie chart (click to expand)

1. Click + Widget and select Pie chart.
2. Title: PRs Created with AI.
3. Choose the Pull Request blueprint.
4. Under Breakdown by property, select the Created By AI Agent property.
5. Click Save.

AI model usage distribution pie chart (click to expand)

This chart shows which AI models are being used most frequently across your organization.

1. Click + Widget and select Pie chart.
2. Title: AI Model Usage Distribution.
3. Description: Which AI models are being used most frequently.
4. Choose the Cursor Usage Record blueprint.
5. Under Breakdown by property, select the Most Used Model property.
6. Click Save.

Team activity breakdown table (click to expand)

This table provides a per-team view of Cursor AI usage, showing how each team is adopting AI tools and their acceptance rates.

1. Click + Widget and select Table.
2. Title: Team Activity Breakdown.
3. Description: Detailed view of team productivity and AI usage patterns.
4. Choose the Cursor Team Usage Record blueprint.
5. Click Save to add the widget to the dashboard.
6. Click on the ... button in the top right corner of the table and select Customize table.
7. In the top right corner of the table, click on Manage Properties and add the following columns:
   • Team: The team name.
   • Total Lines Added: Total lines of code added with AI assistance.
   • Agent Requests: Number of AI agent requests made.
   • Input Tokens: Total input tokens consumed.
   • Output Tokens: Total output tokens generated.
   • Total Rejects: Number of AI suggestions rejected.
   • Total Accepts: Number of AI suggestions accepted.
   • Acceptance Rate: Percentage of AI suggestions accepted.
8. Click on the save icon in the top right corner of the widget to save the customized table.

Total cost number chart (click to expand)

1. Click + Widget and select Number Chart.
2. Title: Total Cost.
3. Description: Total cost of Cursor AI usage.
4. Choose the Cursor Usage Record blueprint.
5. Select Aggregate by property Chart type.
6. Select total_cents as the Property.
7. Select sum for the Function.
8. Select $ as the Unit.
9. Click Save.

Related guides

• Visualize Cursor metrics
• Visualize GitHub Copilot metrics
• Track AI-driven pull requests
• Measure reliability and stability of delivery pipeline