Learn how Worklytics can boost AI adoption in your organization

Learn more

Track Databricks Usage Across Your Organization

Track Databricks usage across your organization: active users, compute and DBU cost by team, Genie Code adoption, and the system tables that hold the data.

To track Databricks usage across your organization, query the system tables in Unity Catalog: audit logs show who did what, billable usage shows what it cost, query history shows analytics activity, and the assistant events table shows Genie Code adoption. Join those tables to your HR department or team data and you can see adoption, cost and AI assistant use by team rather than one account-wide total.

This guide covers why that visibility matters, which metrics to track, where each one lives in Databricks, and how to turn the numbers into decisions about training, cost and investment.

Why Visibility into Databricks Usage Matters

Databricks is widely adopted in the enterprise: the company says more than 20,000 organizations worldwide use the platform (Databricks press kit). Within a single company, however, adoption can be uneven. Often, technical teams embrace Databricks quickly while other departments lag behind. This matters: if only a fraction of the people who have access use the platform, you are not getting the full return on your investment. Tracking usage shows these gaps so you can address them. Here are the specific reasons visibility into usage is important:

  • Optimize ROI: Databricks isn’t cheap, so executives want to ensure it’s delivering value. Usage data helps quantify that value by showing whether capacity is underutilized or driving faster delivery and new insights. If usage aligns with productivity gains, that success can justify further investment.
  • Identify Adoption Gaps: Usage tracking shows which teams have embraced Databricks and which haven’t. You might discover the data engineering group is running hundreds of notebooks a week while the marketing team hasn’t run a single query. These insights highlight where enablement or training is needed to spread usage more evenly.
  • Prevent Shadow IT and Data Silos: If certain departments aren’t using the sanctioned Databricks platform, they may turn to unsanctioned tools or manual work. Tracking usage reveals these gaps, giving IT the opportunity to intervene, align workflows, and establish a single source of truth.
  • Resource and Cost Management: Tracking usage is also a matter of cost control and performance. Databricks usage translates into compute costs, so visibility into who is spinning up clusters and for how long helps pinpoint inefficiencies. Observability enables accountability: when teams see how their usage affects cost, they are more likely to adopt efficient practices and right-size workloads.
  • Governance and Compliance: In regulated industries, tracking who is accessing what data and running which workloads is critical. Usage logs provide an audit trail that ensures teams adhere to security policies and flag suspicious activity early. Even for general governance, usage data helps set smart policies, such as limiting access to powerful compute resources or scheduling heavy jobs during off-hours.

Tracking usage is about insight, not oversight. Report it by team, department or role, not by individual, and use it to decide where to invest in training, how to allocate compute, and how to improve your data and AI programs. With the reasons established, here is what to measure.

What to Measure in Databricks Usage

“Databricks usage” can have different meanings for different stakeholders. It’s helpful to define the key metrics and usage aspects you might want to track:

  • Active Users: How many people are actually using Databricks on a regular basis? You might track daily active users (DAU) or monthly active users (MAU) of the platform. Breaking this down by department or role is even more insightful:
    • For example, 50% of engineers use Databricks on a weekly basis, but only 5% of analysts do.
  • Frequency and Duration of Use: Beyond logins, consider how often users run workloads and for how long they run. Some may run jobs daily while others only once a month. Measuring session duration or commands executed per session helps gauge engagement depth.
  • Workloads Run: Track notebooks executed, jobs run, and queries executed to see the volume of work. An increase over time suggests growing reliance on Databricks. Breaking it down by workload type (ETL pipelines, ML training, ad-hoc analytics) or by scheduled pipelines provides sharper insight into how the platform supports daily operations.
  • Resource Consumption: This is crucial for cost and capacity planning. Monitor compute usage, including cluster hours, CPU/memory utilization, and Databricks Units (DBUs). Tagging clusters and jobs enables you to attribute usage to teams and identify imbalances. Tracking trends over time also aids in forecasting and budget planning.
  • Feature Adoption: Databricks is a comprehensive platform with numerous capabilities – ranging from basic Spark notebooks to MLflow, Delta Lake, Databricks SQL, and new features like Databricks Assistant (an AI-powered code assistant). It’s valuable to track which features are being used. Feature-specific usage can reveal if certain tools are underutilized. If nobody in your org is using MLflow for model tracking, you might investigate why – perhaps data scientists are using a different tool, or they need training on this feature.
  • User Satisfaction or Productivity: Though harder to quantify, correlating usage with outcomes is key. Do high-usage teams deliver projects faster or produce better results? Surveys can also reveal whether Databricks improves productivity. High usage with positive feedback indicates healthy adoption, while heavy use accompanied by frustration suggests areas that need attention.

In essence, think of Databricks as a product and your employees as the users of that product. The same metrics product managers track for software adoption apply here – active users, frequency, feature usage, retention, etc.. By measuring these, you get a 360° view of how Databricks is (or isn’t) becoming ingrained in daily work.

How to Track Databricks Usage: Tools and Techniques

Knowing what to track is half the battle – the next step is figuring out how to collect this data. The good news is that Databricks and its ecosystem offer several methods for gathering usage metrics. Below are the primary methods to monitor and measure usage across your Databricks workspaces:

  • Databricks Audit Logs: Available in Premium and higher tiers, audit logs capture nearly every user action, including logins, notebook runs, job starts, and cluster activity. They’re delivered in JSON to cloud storage or exposed via system tables. By querying them, admins can see how many commands each user has run or how often clusters have been created. They provide rich raw data, although it often needs processing to be human-readable.
  • System Tables for Usage (Unity Catalog): The most complete option is Databricks system tables. With Unity Catalog enabled, Databricks provides read-only tables in the system catalog that capture usage and operational data across your account (Databricks docs). Some of them, including the audit log and the MLflow tables, are still in Public Preview, so check the status page before you build reporting on them. Relevant tables include:
    • Audit (system.access.audit as mentioned above).
    • Query History (system.query.history), which captures a record of every SQL query run on a SQL warehouse or Databricks notebook. This is incredibly useful, for example, to see how many queries a particular analyst ran over the past month or to identify the most commonly queried tables.
    • Clusters and Warehouse Events (system.compute.clusters, system.compute.warehouse_events), which log changes to clusters and SQL endpoints (e.g., created, started, terminated). These can help track how often teams spin up clusters or how long warehouses run.
    • Billable Usage (system.billing.usage), which logs all billable usage events (compute, storage, etc.) across the account. This ties usage to cost, down to a very granular level. By joining this with tags or workspace info, you can get cost/usage per team or project for chargeback or optimization purposes.
    • Job Runs (system.lakeflow.job_run_timeline and related tables), which track pipeline/job execution history.
    • MLflow Usage (system.mlflow.* tables for experiment and run metadata) to see how heavily the ML tracking features are used.
  • UI and Built-in Dashboards: For quicker access, admins can use the Account Console’s usage page or import pre-built dashboards. These provide visual breakdowns of DBU consumption by workspace, tags, or product SKU, and highlight top operations. There’s also a Query History panel in Databricks SQL, which allows you to see who ran which queries, making it useful for tracking analytics adoption without writing code.
  • REST APIs: Databricks provides APIs that allow you to fetch usage data programmatically. The Query History API returns metadata like execution time, user, and query text, while other APIs expose job runs and cluster events. These are handy for automation or consolidating usage data with other tools, especially if you want daily ingestion into your own reports.
  • Cloud Tracking & Third-Party Tools: Cloud-native services, such as AWS CloudWatch or Azure Log Analytics, can capture Databricks events and costs. Tools like Datadog can also monitor Spark metrics and cluster stats, offering real-time alerts. These work well for live Tracking but should complement, not replace, Databricks system tables as the main source of record.
  • Manual Surveys and Feedback: Beyond system data, direct feedback from teams helps explain usage patterns. For example, low adoption in logs may reflect usability issues or missing features. Adding lightweight surveys can surface these insights, though quantitative logs remain the most reliable usage data.

In practice, you will likely use a combination of these methods. Many organizations start with the basics – downloading audit logs or billable usage reports – and later evolve to more real-time dashboards via system tables or integrate data into a BI tool. The key is that the data exists. Databricks has ensured that nearly every aspect of usage can be captured, whether through logs, tables, or APIs. It’s up to you to retrieve and analyze it.

How to Track Genie Code (Databricks Assistant) Adoption

Genie Code is the AI assistant built into Databricks notebooks and the SQL editor. If you are measuring AI adoption, it is the Databricks usage most leaders ask about first.

Databricks records it in the system.access.assistant_events table (Public Preview). Each row is one message a user sent in the Genie Code window or the in-cell Assistant, with the user's email (initiated_by), the workspace, and the event time (Databricks docs). That is enough to count weekly active users and messages per active user:

SELECT
  date_trunc('week', event_date) AS week,
  workspace_id,
  count(DISTINCT initiated_by) AS active_users,
  count(*) AS messages,
  round(count(*) / count(DISTINCT initiated_by), 1) AS messages_per_active_user
FROM system.access.assistant_events
GROUP BY 1, 2
ORDER BY 1 DESC;

To see adoption by team, join initiated_by to your HR or directory data on email and group by department instead of workspace. Report groups, not individuals, and set a minimum group size before you share the results. The table records that a message was sent, not what it said, so it measures adoption and frequency, not the quality of the help.

Seeing Databricks in the Context of Your Other AI Tools

Databricks system tables tell you how the platform is used. Most organizations also want to know how that compares with the other AI tools their people use, and whether adoption is spreading across teams or staying in a few.

Worklytics measures AI adoption across tools such as ChatGPT Enterprise, Claude, Claude Code, Cursor, Gemini in Workspace, GitHub Copilot, Microsoft Copilot, Glean, Gong, Slack AI and Windsurf in one AI adoption dashboard. It is privacy-first: personal identifiers are pseudonymized before analysis, and results are only shown for groups above a minimum size, so leaders see adoption by team, department and role without exposing individuals or any message content.

Databricks and Genie Code are on the Worklytics roadmap. If you want Databricks adoption next to your other AI tools in the same view, tell us and we will prioritize it with the teams that ask.

Diagram of the Worklytics privacy design: identifiers are pseudonymized before analysis
Privacy Design of Worklytics

Conclusion

Databricks already records almost everything you need to track usage: who is active, what they run, what it costs, and how often they use Genie Code. The work is joining those tables to team data and reviewing the results on a regular schedule. Start with active users and DBU cost by team, add Genie Code adoption, and use the gaps you find to decide where training, enablement or cost controls will pay off.

FAQ

How do I see who is using Databricks?

Query the system.access.audit table, which records user actions such as logins, notebook runs and job starts, and count distinct users per day, week or month. Join the user email to your HR data to break active users down by department. The audit table requires Unity Catalog and is in Public Preview.

How do I track Databricks cost by team?

Use the system.billing.usage table, which records billable usage across the account. Tag clusters, jobs and SQL warehouses with a team or cost center, then group usage by that tag. Without consistent tags, cost can only be split by workspace.

How can I measure Genie Code or Databricks Assistant adoption?

The system.access.assistant_events table records each message a user sends in the Genie Code window or the in-cell Assistant. Count distinct users and messages per week to get active users and messages per active user, then join to HR data to compare teams.

Do I need Unity Catalog to use Databricks system tables?

Yes. System tables require a Unity Catalog enabled workspace, and users need permission to use the system catalog and schemas and to select from the tables. Without Unity Catalog you can still use audit log delivery to cloud storage and the REST APIs.

Should Databricks usage be reported per person?

For adoption and cost decisions, report by team, department or role. Group-level reporting answers the investment and enablement questions leaders have, and a minimum group size protects individual privacy.

Request a demo

Schedule a demo with our team to learn how Worklytics can help your organization.

Book a Demoor take the product tour →