# Learning Data Modeling Is Essential for New Data Engineers  

## Deciding how data fits together

Learning to build pipelines is one thing.

Deciding how the data should fit together is another.

I’m writing this post using a scenario where the data team is using Microsoft Azure tools like Azure Data Factory, Microsoft Fabric, and Power BI. However, these principles can be applied using other tools.

Imagine your IT manager asks a simple question:

> “How much are we spending on virtual machines, and are we using what we pay for?”

You may have a list of VMs, CPU and memory metrics. You may also have cloud billing exports.

But those data files may not fit together neatly.

The VM inventory describes what is deployed. Monitoring data shows how resources are used. Billing data shows what those resources cost.

Before you build a useful report, you need to **organize that data**.

That is where **data modeling** comes in.

## Moving data is the start

When you start learning cloud or data engineering, it makes sense to focus on tools.

Maybe you learn how to move data with Azure Data Factory. You practice SQL and Python. You build pipelines and explore Microsoft Fabric.

### But what if you are tasked to turn that data into a report?

That is when a different set of questions appears:

*   Which tables should we create?
    
*   How should they connect?
    
*   What does each row represent?
    
*   How do we avoid counting the same cost twice?
    

Data modeling helps you answer those questions. It is the critical bridge between collecting data and making it useful. This allows you to turn raw data into a structure that supports meaningful analysis.

Microsoft Fabric makes this connection easier to see. Its lakehouse supports data engineering and analytics through the use of Delta tables, Spark, and SQL. Those tables then serve as the foundation of Power BI analysis.

Once you know how to ingest and transform data, the next essential skill is learning how to model it well.

How you structure your data determines how effectively you can analyze it, uncover insights, and build meaningful reports. Data modeling is what makes your data truly useful.

## A familiar problem

Imagine an IT department that manages VMs for several teams.

Some VMs run business applications. Others support development, databases, or GPU workloads.

The department wants to answer questions such as:

*   How many VMs does each team own?
    
*   How much CPU, RAM, and storage have we provisioned?
    
*   Which VMs have low CPU utilization?
    
*   How much does each environment cost?
    
*   Which resources are driving this month’s spending?
    

For this example, we will use three groups of data:

| Data source | What it tells us |
| --- | --- |
| VM inventory | VM name, resource ID, size, operating system, owner, and environment |
| Monitoring data | CPU utilization, memory usage, and disk usage |
| Cloud billing data | Resource charges, service categories, dates, and costs |

These sources answer different questions. A useful model connects them without mixing them up.

## What is data modeling?

Data modeling means deciding how to organize data and how the tables relate.

For our IT department, that means separating:

*   The resources we manage
    
*   The measurements we collect
    
*   The costs we incur
    
*   The teams responsible for them
    

A good model lets someone ask, “What did the development team spend last month?” without rebuilding the data for that question.

For Power BI reporting, a useful starting point is a star schema. Microsoft recommends separating tables into dimensions for filtering and grouping, and facts for summarizing observations or events.

## What is a star schema?

A **star schema** is a well-established data modeling approach widely used in relational data warehouses.

It organizes tables into two types:

*   **Fact tables:** store measurements or events
    
*   **Dimension tables:** describe the things involved
    

![Diagram showing sample Star Schema with one Fact table and multiple Dimension tables](https://cdn.hashnode.com/uploads/covers/609ec78747373d4cb056441b/c4e4e829-57f8-4029-a449-4e1940405f11.png align="center")

For an IT environment, dimensions might describe VMs, teams, dates, and cloud services.

Facts might contain daily resource usage or cloud charges.

> You can read more about the Star Schema:  
> [Understanding Star Schema & the Importance For PowerBI](https://learn.microsoft.com/en-us/power-bi/guidance/star-schema)

## Dimension tables

These tables describe the resource.

A VM dimension could look like this:

| VMKey | VMName | Environment | OperatingSystem |
| --- | --- | --- | --- |
| 101 | app-prod-01 | Production | Linux |
| 102 | dev-tools-01 | Development | Windows |
| 103 | ml-worker-01 | Development | Linux |

The key of this table is an internal identifier that connects it to other tables. In a real project, we would also retain the cloud provider’s resource ID.

Other **dimensions** could include:

| Dimension | Example attributes |
| --- | --- |
| Date | Date, month, quarter, year |
| Team | Team name, department, cost center |
| Cloud service | Compute, storage, networking |
| Resource | Resource ID, resource name, resource type |

These tables give people understandable fields to filter by.

## Fact tables

They record measurements and costs.

A daily VM snapshot could look like this:

| DateKey | VMKey | ProvisionedVCPU | ProvisionedRAMGiB | GPUCount | ProvisionedDiskGiB | AvgCPUPercent |
| --- | --- | --- | --- | --- | --- | --- |
| 20261001 | 101 | 4 | 16 | 0 | 256 | 24 |
| 20261001 | 102 | 2 | 8 | 0 | 128 | 9 |
| 20261001 | 103 | 8 | 56 | 1 | 512 | 67 |

Keep in mind that these numbers are just **examples** to explain the idea.

It's important to notice the distinction between capacity and usage.

A VM with 16 GiB of provisioned RAM is not necessarily using 16 GiB. A VM with 256 GiB of attached disk capacity is not necessarily storing 256 GiB of files.

You should name columns clearly so readers know what each number means. I have made the mistake of creating columns such as `ProvisionedDisk` and `ProvisionedRAM` without explicitly showing what the unit of measure is. This could easily create confusion and mistakes when calculating consumption and costs.

## What does one row represent?

Before building a fact table, finish this sentence: “One row represents…”

This is called the table’s **grain**. Microsoft recommends keeping a consistent grain within each fact table.

For our VM snapshot: One row represents one VM on one day.

For a cloud cost table: One row represents one resource, on one day, for one service category, in one billing currency, using one cost basis.

That second table might look like this:

| DateKey | ResourceKey | ServiceCategory | CostUSD |
| --- | --- | --- | --- |
| 20261001 | 501 | Compute | 7.20 |
| 20261001 | 502 | Storage | 0.85 |
| 20261001 | 503 | Networking | 0.30 |

Again, these are **example** amounts, not actual Azure price estimates.

In this project, we would use one billing currency and choose either actual or amortized costs. Actual costs help reconcile spending with billing. Amortized costs spread reservation or savings plan costs across the benefit period and allocate them to usage. Do not add the two together as if they were separate charges.

## Why not put everything in one big table?

One big table is the easiest way to organize data. Of course, convenience doesn't always equal efficiency.

You could put the VM name, team, RAM, CPU, storage, utilization, and billing charges in every row. But the sources may have different grains.

Imagine that one VM has:

*   One daily inventory snapshot
    
*   Many monitoring samples that day
    
*   Several billing records that day
    

If you join those records without care, you can repeat a billing charge across multiple monitoring rows. Your report may then overstate the cost.

The same problem works in reverse. You might repeat a VM’s RAM and CPU capacity across every billing record.

Keeping facts at a consistent grain helps prevent this. Shared dimensions can filter multiple fact tables without requiring you to join the facts directly. Microsoft recommends this approach instead of directly relating fact tables.

A giant table can also introduce:

*   Repeated resource and team details
    
*   A larger model
    
*   More difficult maintenance
    
*   Complicated [DAX](https://learn.microsoft.com/en-us/dax/dax-overview) to correct duplicated values
    
*   Slower reports, depending on the model and queries
    

A flat table is not automatically bad. It can work for a small, focused analysis. But a star schema is a stronger starting point when you need several types of measurements and reusable reporting.

## How the model fits together

For this project, we would build two related stars.

The VM snapshot model:

```plaintext
DimDate ──────┐
DimVM ────────┼── FactVMDailySnapshot
DimTeam ──────┘
```

The cloud cost model:

```plaintext
DimDate ──────────┐
DimResource ──────┤
DimTeam ──────────┼── FactCloudCost
DimCloudService  ─┘
```

The shared date and team dimensions let us compare capacity and cost by day or department.

The resource dimension also gives us room to track costs beyond VMs, such as disks, storage accounts, and networking resources.

For the first version, keep each resource assigned to one team. Shared resources and historical ownership changes can be separate improvements.

## Medallion Architecture

Medallion architecture organizes data into Bronze, Silver, and Gold layers. In Fabric, Bronze holds raw data, Silver improves its quality, and Gold prepares it for reporting and analytics.

Here is how that could look for our IT project:

| Layer | What we do |
| --- | --- |
| Bronze | Store inventory exports, monitoring records, and billing files |
| Silver | Standardize resource IDs, clean dates, remove duplicates, and validate units |
| Gold | Build dimensions, daily VM snapshots, and cloud cost facts |

Think of the two ideas this way:

*   A star schema describes how reporting tables fit together.
    
*   Medallion architecture describes how data becomes more useful.
    

The Gold layer is a natural place for our reporting model. But it does not mean every Gold table must be a star schema.

## Semantic models, Power BI, and Direct Lake

Once the reporting tables are ready, we build a Power BI semantic model.

The semantic model defines relationships, reusable calculations, business-friendly names, and security rules. It gives report builders a consistent way to work with the data.

For example, our cost measure could be:

```plaintext
// Adds cloud charges within the current report filters.
// This example uses USD and one selected cost basis.

Total Cloud Cost =
SUM ( FactCloudCost[CostUSD] )
```

A user can then filter that measure by month, team, or service category.

If the tables are stored as Delta tables in Fabric, **Direct Lake** is another option. It loads data from OneLake Delta tables into memory for interactive analysis. It avoids a separate traditional import copy, but it does not remove the need for a well-designed model.

## Modeling habits to learn early

Six modeling habits to learn early

1.  Define the grain first
    
    Write down what one row represents before creating the table.
    
    If you cannot explain the grain, it will be difficult to know whether your totals are correct. I've personally struggled with this step sometimes. But it is very worth it to go through this thinking process to figure out how to best organize your data.
    
    Consistent fact-table grain is a core principle here.
    
2.  Use dimensions for filtering
    
    For our sample project, you could use team, environment, service, and date dimensions to filter reports.
    
    Keep measurements and costs in fact tables. This gives each table a clear job.
    
3.  Do not sum every number
    
    Cloud costs can usually be added across days when the currency and cost basis match.
    
    VM capacity is different. A VM with 16 GiB of RAM on Monday and 16 GiB on Tuesday does not have 32 GiB of RAM.
    
    For capacity reporting, choose a specific snapshot date or define a meaningful average.
    
    The table’s grain should guide how you summarize it.
    
4.  Use measures for report calculations
    
    Use **measures** for calculations that change with report filters, such as cloud cost by month or team.
    
    Use calculated columns when you need a value for each row, such as a resource category.
    
    Both have a purpose. Choose the option that keeps your model simple and your calculations easy to reuse.
    
5.  Create a shared date dimension
    
    Use the same date dimension to filter daily costs and daily VM snapshots.
    
    Shared dimensions help you analyze multiple fact tables together without joining those facts directly.
    
6.  Plan security with the model
    
    If team managers should only see their own team’s resources, include that requirement in the design.
    
    Row-level security (RLS) belongs in the semantic model. A clean team dimension gives you a clear starting point for that work.
    

## Why learn this now?

Fabric connects data engineering work with reporting and analytics. Direct Lake makes it easier for reporting and analytics models to work directly with the tables engineers build. That means data modeling is an important skill to learn alongside data pipelines, SQL, and notebooks.

Moving data gets it into the platform. Modeling turns that data into something people can understand and use.