Demonstrate HN: MetricFlow – delivery-offer metric framework

Welcome to MetricFlow

MetricFlow is a computational framework for building and declaring consistent metric good judgment. The name comes from the capability taken to generate metrics. The usage of the user-defined semantic model, a matter is first compiled into a metric dataflow belief. The belief is then converted to an abstract SQL object model, optimized, and rendered to engine-particular SQL.

MetricFlow presents a keep of abstractions that attend you invent sophisticated good judgment and dynamically generate queries to residence:

• Advanced metric kinds similar to ratio, expression, and cumulative
• Multi-hop joins between truth and dimension sources
• Metric aggregation to utterly different time granularities
• And so way more

As a developer, you can well perchance presumably moreover exercise MetricFlow’s interfaces to invent APIs for integrations to bring metrics into downstream instruments on your data stack.

MetricFlow itself acts as a semantic layer, compiling the semantic data described in the MetricFlow spec to SQL that would possibly perchance perchance well perchance moreover be carried out against the tips warehouse and served to downstream applications. It acts as a proxy, translating metric requests in the create of “metrics by dimensions” into SQL queries that traverse the tips warehouse and the underlying semantic construction to resolve every doubtless aggregate of metric and dimension.

Core Tenets

The framework depends on a keep of core tenets:

• DRY (Don’t Repeat Yourself): This precept is the core aim of the underlying MetricFlow spec. Duplication of good judgment results in incorrectly constructed metrics and desires to be averted thru thoughtfully-designed abstractions.
• SQL-centric compilation: Metric good judgment must always by no manner be constructed in a unlit-field. This SQL-centric capability to metric construction manner that metric good judgment remains broadly accessible and introspectable.
• Maximal Flexibility: Form any metric on any data model aggregated to any dimension. There are acquire away hatches, nonetheless we consistently work to invent them needless.

Beneficial properties

Key parts of MetricFlow contain:

• Metrics as Code: MetricFlow’s metric spec lets you outline a extensive-fluctuate of metrics thru a tidy keep of abstractions that attend DRY expression of good judgment in YAML and SQL.
• SQL Compilation: Generate SQL to construct metrics, with out the necessity to again and again explicit the identical joins, aggregations, filters and expressions from your data warehouse in picture to invent datasets for consumption.
• DW Connectors: Toughen for data warehouse (DW) connectors give the beginning-offer community the vitality to make contributions to DW-particular optimizations and toughen. DW connectors enable customers to invent metric good judgment to varied data warehouses.
• Show Line Interface (CLI): Pull data into a local context for testing and pattern workflows.
• Python Library: Pull metrics into local Python environments similar to Jupyter or utterly different analytical interfaces.
• Materializations: Elaborate a keep of metrics and a keep of dimensions that you actually are seeking to materialize to the tips warehouse. This enables fleet construction of denormalized datasets again to the warehouse.
• Accessible interfaces: Form APIs and SDKs so you can well perchance presumably pull metric datasets into downstream applications.
  • React Substances to construct embedded analytics
  • Airflow Operators to schedule API requests and pre-construction of metrics
  • GraphQL interface for live-customers to construct their very luxuriate in interfaces for metrics

Sources

Documentation

MetricFlow documentation would possibly perchance perchance well perchance moreover be came across on the Remodel Documentation standing.

Getting Started

Must you’re new to MetricFlow, we counsel that you analysis our Getting Started half and tutorial. To acquire began you can well perchance presumably rush the next instructions:

Install MetricFlow:

Must you create now now not hang postgres on your machine, first install Postgres:

• Postgres presents pre-constructed applications for find and installation
• Mac customers would possibly perchance perchance well perchance bewitch to make exercise of Homebrew: brew install postgresql

Then, proceed with fashioned install as follows:

pip install metricflow
mf setup
# Now you can well perchance presumably join your datawarehouse by editing the parameters in {$HOME}/.metricflow/config.yml
mf health-assessments # verify your datawarehouse connection is working

Speed the tutorial:

mf tutorial # optionally add `--skip-dw` while you are going to hang got already confirmed your datawarehouse connection works

For reference, the tutorial steps are under:

🤓 Please rush the next steps,

    1. In '{$HOME}/.metricflow/config.yml', `model_path` desires to be '{$HOME}/.metricflow/sample_models'.
    2. Strive validating your data model: `mf validate-configs`
    3. Take a look at out your metrics: `mf list-metrics`
    4. Predict your first metric: `mf question --metrics transactions --dimensions ds --picture ds`
    5. Demonstrate the SQL MetricFlow generates: `mf question --metrics transactions --dimensions ds --picture ds --label`
    6. Add one other dimension: `mf question --metrics transactions --dimensions ds,customer__country --picture ds`
    7. Add the next date granularity: `mf question --metrics transactions --dimensions ds__week --picture ds__week`
    8. Strive a more sophisticated question: `mf question --metrics transactions,transaction_usd_na,transaction_usd_na_l7d --dimensions ds,is_large --picture ds --where "ds between '2022-03-20' and '2022-04-01'"`
    9. For more techniques to work alongside with the pattern models, lunge to ‘https://doctors.remodel.co/doctors/metricflow/metricflow-tutorial’.
    10. Whilst you’re completed, rush `mf tutorial --skip-dw --tumble-tables` to tumble the pattern tables.

Additional Sources

• Website material
• Documentation
• Slack Community
• MetricFlow Git Repository
• CHANGELOG.md

Install MetricFlow

MetricFlow would possibly perchance perchance well perchance moreover be installed from PyPi to be used as a Python library with the next present:

pip install metricflow

Once installed, MetricFlow would possibly perchance perchance well perchance moreover be setup and connected to a data warehouse by following the directions after issuing the present:

mf setup

To appear what MetricFlow can create with out customized configurations, open the tutorial by working:

mf tutorial

To arise and working together with your luxuriate in metrics, you’ll want to always rely on MetricFlow’s documentation on hand at MetricFlow doctors.

Contributing and Code of Habits

This mission will seemingly be a standing where of us can with out distress make contributions top quality updates in a supportive atmosphere.

You can well perchance bewitch to read our code of conduct and engineering practices earlier than diving in.

To acquire began on bid contributions, head on over to our contributor guide

License

MetricFlow is delivery offer tool. The mission depends on a lot of licenses including AGPL-3.0-or-later and Apache (specified at folder stage).

MetricFlow is constructed by Remodel, the firm in the again of the main metrics store.