The Substitution Operator ($), Scoping and Naming, SQL Dialect, and SQL Block sections have moved to a new Incorporating SQL and Referring to LookML Objects page.
This page defines terms and concepts that appear repeatedly in LookML development. The following diagram shows relationships of which elements are contained within other elements. All terms shown here are defined in the following sections.
Relationships between LookML elements
Looks and user-defined dashboards are not part of this diagram as users create them without using any LookML. However, their queries rely on the underlying LookML elements in the chart above.
A LookML project is a collection of LookML files that describe a set of related models, explores, views, and LookML dashboards.
By convention, LookML code is segregated into three types of files: model files, view files, and dashboard files. In new LookML, these fields have the following extensions:
If you are using Git for project version control, one project constitutes a single Git repository.
Relationship between LookML projects, files, database connections, and Git repositories
Projects are managed in Looker under the Develop menu.
Accessing projects under the Develop menu
Where Do LookML Projects and Files Come From?
When you create a new project, Looker’s project generator creates a baseline set of files, which you use as a template for building out the project. Very rarely, if ever, will you write a LookML file from scratch.
When creating a project, you specify a database connection, and Looker’s project generator creates the following:
- Multiple view files, one file for every table in the database.
- One model file. The model file declares an explore for every view. Each explore declaration includes join logic to join any view that Looker can determine is related to the explore.
From here, you can refine the project by removing unwanted views and explores, and adding custom dimensions and measures.
Major Structural LookML Parameters
As shown in the diagram above, a project contains one or more model files which contain parameters defining a model and its explores and joins. The project also contains one or more view files, each containing parameters defining that view, its fields (including dimensions and measures), and sets of fields. This section describes those major structural parameters.
A model is a customized portal into the database, designed to provide intuitive data exploration for specific business users. Multiple models can exist for the same database connection in a single LookML project. Each model can expose different data to different users. For example, sales agents need different data than company executives, and so you would probably develop two models to offer views of the database appropriate for each user.
In the Looker app, queries are grouped by the model they belong to, and business users see models listed under the Explore menu.
Models listed under the Explore menu
A model file specifies the database to connect to and defines a collection of explores for that connection. By convention each file declares exactly one model and, in new LookML, model filenames end in
.model.lkml. The name of the model file determines the name that displays in the Looker app.
The general form of a model declaration in LookML is shown below. See the LookML Reference for details.
A view declaration defines a list of fields (dimensions or measures) and their linkage to an underlying table or derived table. In LookML a view typically references an underlying database table, but it can also represent a derived table.
A view may join to other views. The relationship between views is typically defined as part of a explore declaration in a model file.
In the Looker app, view names appear at the front of dimension and measure names to qualify what view the field belongs to.
View names listed as part of field names
By convention, in new LookML, a view is stored in a
.view.lkml file. The general form of a view declaration is shown below. See the LookML Reference for complete usage details.
An explore is a view that users can query. You can think of the explore as a starting point for a query, or in SQL terms, as the FROM in a SQL statement. Not all views are explores, because not all views describe an entity of interest. For example, a “States” view corresponding to a lookup table for state names doesn’t warrant an explore, because business users never need to query it directly. On the other hand, business users probably want a way to query an “Orders” view, and so defining an explore for Orders makes sense.
An explore declaration specifies the join relationships to other views. Continuing with the previous examples, the Orders view might join the States view, identifying the state in which a sale occurred. See Joins for more detail.
Explores listed under the Explore menu
By convention explores are declared in the model file. The example below demonstrates the declaration for an “Orders” explore for an ecommerce database. The views “orders” and “customers” are defined elsewhere, in their respective view files.
Example explore declaration
For details on
join declarations, see Joins. See the LookML Reference on join declarations for complete usage details.
Dimension and Measure Fields
Views contain fields, mostly dimensions and measures, which are the fundamental building blocks for Looker queries.
In Looker, a dimension is a groupable field and can be used to filter query results. It can be:
- an attribute, which has a direct association to a column in an underlying table
- a fact or numerical value
- a derived value, computed based on the values of other fields in a single row
For example, dimensions for a “Products” view might include: product name, product model, product color, product price, product created date, product end-of-life date.
A measure is a field that uses a SQL aggregate function, such as
MAX. Any field computed based on the values of other measure values is also a measure. Measures can be used to filter grouped values. For example, measures for a “Sales” view might include: total items sold (a count), total sale price (a sum), average sale price (an average).
The behavior and expected values for a field depend on its declared type, such as
time. For measures, types include aggregate functions, such as
percent_of_previous. For details, refer to dimension types and measure types.
In the Looker app, fields are listed on the Explore page when building and running queries.
Dimensions and measures are the building blocks for Looker queries
By convention fields are declared as part of the view they belong to, stored in a view file. The example below shows several dimension and measure declarations. Notice the use of the substitution operator (
$) to reference fields without using a fully-scoped SQL column name.
Example declarations of dimensions and measures
You can also define a dimension_group, which creates multiple time-related dimensions at once, and filter fields, which have a variety of advanced use cases such as templated filters.
See the LookML Reference for complete details on declaring fields and the various settings that can be applied to them.
As part of an
explore declaration, each
join declaration specifies a view that can be joined into the explore. When a user creates a query that includes fields from multiple views, Looker automatically generates SQL join logic to bring in all fields correctly.
join in an
For more details check out Working with Joins in LookML.
In Looker, a set is a list that defines a group of fields that are used together. Typically sets are used to specify which fields to display after a user drills down into data. Drill sets are specified on a field-by-field basis, so you get complete control over what data is displayed when a user clicks a value in a table or dashboard. Sets can also be used as a security feature to define groups of fields visible to specific users.
The following example shows a set declaration in a view
order_items, defining fields that list relevant details about a purchased item. Note that the set references fields from other views by specifying scope.
See the LookML Reference for complete usage details for sets.
In Looker, you can drill down on any fields that are setup that way when writing LookML. Drilling works in both query results tables and dashboards. Drilling starts a new query that is restricted by the value you clicked on.
Drill behavior is different for dimensions and measures:
- When drilling on a dimension, the new query filters on the drilled value. For example, if you click on a specific date in a query of customer orders by date, the new query will show only orders on the specific date.
- When drilling on a measure, the new query will show the data set that contributed to the measure. For example, when drilling on a count, the new query will show the rows to calculate that count. When drilling on max, min, and average measures, drilling still shows all the rows that contributed to that measure. This means that drilling on a max measure, for example, shows all the rows that were used to calculate the max value, not just a single row for the max value.
The fields to show for the new drill query are defined by a set.
Derived Tables and Facts Tables
A derived table is a table comprised of values from other tables, which is accessed as though it were a physical table with its own set of columns. A derived table is exposed as its own view using the
derived_table parameter, and defines dimensions and measures in the same manner as conventional views. The view for a derived table can be queried and joined into other views, just like any other view.
Derived tables are created by using the
derived_table parameter in a view declaration. For complete details, see Derived Tables Reference.
Using Derived Tables for Facts Tables
In Looker a common use for derived tables is to present a facts table, which computes facts about an entity, based on values derived from other views. For example, a common need is to analyze user traits based on past orders or actions, and then report, sort, and filter those traits like any other facet of a user.
Example: A Derived Table for User Order Facts
Consider an e-commerce data set with a
users table containing customer data and an
orders table containing details about customer orders. A derived table can be used to create a user-orders facts table, containing user-centric facts such as lifetime total revenue for a user, which doesn’t physically exist in the underlying tables. More example columns are: number of lifetime orders, latest order date, whether the user placed multiple orders, and so forth. See the diagram below.
Because the primary key for the fact table is
user_id, the view can be joined one-to-one with the
users explore, enabling rich query possibilities. An example is shown below:
Persistent Derived Tables
There are often cases where computing a derived table takes a significant amount of time. To avoid running an expensive derived-table computation more often than necessary, Looker can cache (or “persist”) the data in a derived table. A persistent derived table (PDT) is simply a derived table that is automatically regenerated. For more information on the ways to persist a derived table, see this page.
Persistent derived tables use a scratch table in the database to save results, which requires additional database configuration depending on the type of database.
Looker issues queries against a database, specified in the LookML model file. A Looker connection specifies a server hosting the database, and parameters defining how Looker should connect to the database. Database setup is typically done once (or infrequently) by the system administrator, and data modelers simply pick from the available connections.