Glossary

Glossary

A comprehensive reference of key terms and concepts in Daana CLI.

Core Concepts

DMDL (Daana Model Description Language)

The YAML-based declarative modeling system used to define business data models. DMDL allows you to describe entities, attributes, and relationships in a human-readable format that compiles to executable JSON.

Example:

model:
  id: "MY_MODEL"
  name: "MyModel"
  entities:
    - id: "CUSTOMER"
      name: "CUSTOMER"
      attributes:
        - id: "CUSTOMER_ID"
          name: "CUSTOMER_ID"
          type: "STRING"

Model

The top-level container for your data definition. Every DMDL file starts with a model declaration that includes metadata about your business data model, entities, attributes, and relationships.

Entity

A business object representing a real-world concept like "CUSTOMER", "ORDER", or "DEVICE". Entities are the primary building blocks of your data model and contain attributes that describe their properties.

Key characteristics:

  • Must have both id and name fields
  • Contains one or more attributes
  • Can participate in relationships with other entities

Attribute

A property of an entity with a specific data type (STRING, NUMBER, START_TIMESTAMP, END_TIMESTAMP, etc.). Each attribute represents a piece of information about an entity.

Examples:

  • CUSTOMER_ID (STRING)
  • ORDER_VALUE (NUMBER)
  • ORDER_PURCHASE_TS (START_TIMESTAMP)

Relationship

A connection between two entities that defines how they relate to each other. Relationships describe the associations in your business model.

Example: An ORDER "IS_PLACED_BY" a CUSTOMER

Required fields:

  • name: Display name for the relationship
  • definition: What this relationship represents
  • source_entity: The entity where the relationship originates
  • target_entity: The entity where the relationship points

Optional fields:

  • description: Detailed explanation of the relationship

Database Architecture

internaldb

Port: 5434 (development)

Daana's internal database that stores your model definitions, mappings, and workflow configurations. Think of this as the "blueprint storage" - it contains the metadata that describes how your transformations should work.

What's stored here:

  • Compiled model definitions
  • Entity and attribute metadata
  • Mapping configurations
  • Workflow definitions
  • Focal framework schemas and stored procedures

Configuration: Set up once in ~/.daana/config.yaml

customerdb

Port: 5432 (development)

Your data warehouse database that contains both source data and transformed business entities. This is where the actual data lives and where your BI tools will connect.

What's stored here:

  • Source data (e.g., olist_orders, olist_customers tables)
  • Transformed data in the daana_dw schema
  • Business-ready tables generated by Daana transformations

Configuration: Managed via connection profiles

Two-Database Architecture

The separation of concerns between internaldb (metadata/blueprints) and customerdb (actual data). This architecture keeps your transformation logic separate from your business data.

Benefits:

  • Clean separation of metadata and data
  • Easier to manage and backup
  • Can connect to multiple data warehouses using one internaldb

Model Elements

id vs name

Two required fields for all model elements (entities, attributes, relationships):

  • id: Technical identifier used internally by Daana. Must be unique and typically uppercase (e.g., "CUSTOMER_ID")
  • name: Display name for human readability. Often the same as id for clarity

Example:

- id: "CUSTOMER_EMAIL"
  name: "CUSTOMER_EMAIL"
  definition: "Customer's email address"
  type: "STRING"

effective_timestamp

A boolean flag (true/false) that marks an attribute as tracking changes over time. When set to true, Daana tracks the historical values of this attribute.

Use cases:

  • Customer addresses (they move)
  • Order statuses (they change from pending → shipped → delivered)
  • Product prices (they fluctuate)

Enables questions like:

  • "What was this customer's city in 2017?"
  • "How did this order's status change over time?"

Grouped Attributes

Related fields bundled together to maintain context. Used when multiple attributes represent different aspects of the same concept.

Example: ORDER_VALUE as a grouped attribute

- id: "ORDER_VALUE"
  name: "ORDER_VALUE"
  effective_timestamp: true
  group:
    - id: "ORDER_VALUE_AMOUNT"
      name: "ORDER_VALUE_AMOUNT"
      definition: "Monetary value of the order"
      type: "NUMBER"
    - id: "ORDER_VALUE_CURRENCY"
      name: "ORDER_VALUE_CURRENCY"
      definition: "Currency of the order value"
      type: "STRING"

Common use cases:

  • Amount + Currency
  • Address components (street, city, state, zip)
  • Name components (first name, last name, middle initial)

Data Types

STRING

Use STRING for any text-based information in your model, including names, descriptions, addresses, email addresses, and even text-based identifiers.

Examples:

  • CUSTOMER_NAME (Full name of the customer)
  • EMAIL (Customer email address)
  • ORDER_NUMBER (Human-readable order identifier)
  • PRODUCT_DESCRIPTION (Detailed product information)

NUMBER

Use NUMBER for any numeric values in your model, including quantities, amounts, measurements, counts, and percentages. This type handles both integers and decimal numbers.

Examples:

  • ORDER_AMOUNT (Total order value)
  • QUANTITY (Number of items)
  • PRICE (Unit price)
  • DISCOUNT_PERCENTAGE (Discount rate applied)

⚠️ Important: For monetary amounts with currency, consider using UNIT type or grouped attributes to keep amount and currency together.

UNIT

Use UNIT when you need to track both a numeric value and its unit of measurement together. This is essential for monetary amounts with currency, measurements with units, or any quantity that needs a denominator. UNIT types are always defined as grouped attributes.

Examples:

  • ORDER_AMOUNT (with currency) (Order value with currency code)
  • PRODUCT_WEIGHT (Product weight with unit (kg, lbs))
  • DISTANCE (Distance traveled with unit (km, miles))

⚠️ Important: UNIT types must always be defined using the 'group' field with at least two attributes: the numeric value and the unit identifier.

START_TIMESTAMP

START_TIMESTAMP is your default choice for any timestamp attribute. Use it for single timestamps, when events occur, or when something begins. This type tells Daana's transformation engine how to handle temporal data correctly.

Examples:

  • ORDER_PURCHASE_TS (When customer completed the purchase)
  • ACCOUNT_CREATED_AT (When customer account was created)
  • EVENT_START_TIME (When event began)
  • SUBSCRIPTION_START_DATE (When subscription activated)

⚠️ Important: This is the generic timestamp type - use it whenever you have a single point in time. For tracking periods or durations, use START_TIMESTAMP together with END_TIMESTAMP.

END_TIMESTAMP

Use END_TIMESTAMP to mark the end of a time period or the completion of an event. This type is always used together with START_TIMESTAMP to define durations or periods. It tells the transformation engine that this timestamp represents a completion or end point.

Examples:

  • ORDER_DELIVERED_TS (When order was delivered to customer)
  • EVENT_END_TIME (When event concluded)
  • SUBSCRIPTION_END_DATE (When subscription expired)
  • CONTRACT_END_DATE (When contract expires)

⚠️ Important: END_TIMESTAMP is always used together with START_TIMESTAMP to define periods. For single timestamps, use START_TIMESTAMP instead.

Workflow Concepts

Init

Create a new project with template files. This sets up the recommended directory structure and creates starter files for your model, workflow, and connections.

Command:

daana-cli init my-project

What's created:

  • model.yaml - Your data model definition
  • workflow.yaml - Workflow orchestration
  • connections.yaml - Database connection profiles
  • mappings/ - Directory for mapping files

Install

Set up the Daana framework in your database. This creates the necessary schemas, tables, and stored procedures that power the transformation engine.

Command:

daana-cli install

What happens:

  • Focal framework schemas are created in internaldb
  • Stored procedures for transformations are installed
  • System tables for tracking workflows are set up

Check

Validate your configurations before deployment. This catches errors early and ensures your model, mappings, workflow, and connections are properly configured.

Commands:

daana-cli check model
daana-cli check mapping
daana-cli check workflow
daana-cli check connections

Deploy

Deploy your workflow to the database. This reads your YAML configurations, compiles them, and sets up the transformation infrastructure in your data warehouse.

What happens:

  • Model and mappings are validated and compiled
  • daana_dw schema is created in customerdb
  • Transformation procedures are generated
  • Target tables are set up for business entities

Command:

daana-cli deploy

Execute

Run data transformations. Execute reads source data from customerdb, applies transformation rules, and writes clean business entities to the daana_dw schema.

What happens:

  • Source data is read from raw tables
  • Transformations are applied
  • Business entities are written to daana_dw schema
  • BI-ready tables are created

Command:

daana-cli execute

Configuration

Connection Profile

A named database connection configuration that defines how to connect to your data warehouse. Connection profiles allow you to manage multiple environments (dev, staging, production) in a single YAML file.

Example:

connections:
  dev:
    type: "postgresql"
    host: "localhost"
    port: 5432
    user: "dev"
    password: "devpass"
    database: "customerdb"
    sslmode: "disable"
    target_schema: "daana_dw"
 
  production:
    type: "postgresql"
    host: "prod.example.com"
    port: 5432
    user: "${DW_USER}"
    password: "${DW_PASSWORD}"
    database: "analytics"
    sslmode: "require"
    target_schema: "daana_dw"

Use in commands:

daana-cli deploy --connection-profiles profiles.yaml --profile dev

Configuration File

The ~/.daana/config.yaml file that contains your internaldb connection settings and application preferences. This file is created during setup and eliminates the need to pass database flags with every command.

Example:

app:
  log_level: "info"
 
  db:
    host: "localhost"
    port: 5434         # internaldb
    user: "dev"
    password: "devpass"
    name: "internaldb"
    sslmode: "disable"

Configuration Priority

The order in which Daana CLI resolves configuration values (highest to lowest):

  1. Command-line flags (e.g., --db-host localhost)
  2. Environment variables (e.g., APP_DB_HOST=localhost)
  3. Configuration file (~/.daana/config.yaml)
  4. Default values

Data Transformation

Mapping

The definition of how source data fields connect to business model attributes. Mappings tell Daana which columns in your raw tables should populate which attributes in your business entities.

Example: Mapping the customer_id column from olist_customers table to the CUSTOMER_ID attribute in the CUSTOMER entity.

Workflow

An orchestration definition that links together:

  • A business model
  • One or more mappings
  • A connection profile
  • Execution parameters

Workflows define the complete end-to-end transformation from source data to business entities.

daana_dw Schema

The default schema name in your data warehouse (customerdb) where transformed business entities are written. This is where your BI tools should connect to access clean, analytics-ready data.

Contains:

  • Business entity tables (e.g., CUSTOMER, ORDER)
  • Transformation views
  • Helper functions

Advanced Concepts

Focal Framework

The underlying transformation engine that Daana CLI wraps. Focal provides the database schemas, stored procedures, and execution logic for data transformations. The daana-cli install command installs Focal into your databases.

Template Generation

The automatic creation of YAML scaffolds for mappings and workflows. Templates reduce boilerplate and ensure correct structure.

Commands:

daana-cli generate mapping --model model.json --entity CUSTOMER -o mapping.yaml
daana-cli generate workflow -o workflow.yaml

Common Terminology

BI (Business Intelligence)

Analytics tools and dashboards that query your data warehouse to provide business insights. Examples: Tableau, Power BI, Looker, Metabase.

ETL (Extract, Transform, Load)

The traditional process of moving data from source systems to a data warehouse. Daana CLI provides a modern, declarative approach to ETL.

Data Warehouse

A centralized repository of integrated data from multiple sources, optimized for analytics and reporting. In Daana CLI, this is your customerdb.

Source Data

Raw operational data from your business systems. In the Olist tutorial, this includes tables like olist_orders, olist_customers, olist_products, etc.

Business Entity

A clean, analytics-ready representation of a business concept. Business entities live in the daana_dw schema and are what BI tools query.

Quick Reference

Complete Workflow Sequence

1. daana-cli init
   ↓
2. daana-cli install
   ↓
3. (Edit model.yaml and connections.yaml to define your data)
   ↓
4. daana-cli generate
   ↓
5. (Edit mappings to connect to your data sources)
   ↓
6. daana-cli check
   ↓
7. daana-cli deploy
   ↓
8. daana-cli execute

File Types

  • .yaml or .yml: All definitions (models, mappings, workflows, connections)

Common Ports (Development)

  • 5432: customerdb (data warehouse)
  • 5434: internaldb (Daana's internal database)

Need more help? Check out:

Command Reference