Skip to content

Postgres by Example

A hands-on introduction to PostgreSQL through short, runnable lessons.

Dariush Abbasi
English edition · PostgreSQL 17 target

Fifty-seven short lessons take you from the first query through data modeling, querying, indexes, transactions, stored logic, search, and day-to-day operations. Each lesson includes a SQL file; the backup lesson documents shell commands, and the psql reference includes catalog-query alternatives.

Start reading · Print the whole book · Source repository

Prerequisites

The lessons target PostgreSQL 17 and use the psql command-line client. Later-version features are identified explicitly. The official documentation is a useful companion.

Download the complete SQL bundle and extract it into a new directory. Run commands from the directory containing source/. Use a disposable database: these lessons create, change, and drop their demo tables. The examples show the database name postgres; substitute your own practice database, such as pbe:

createdb pbe
psql -X -v ON_ERROR_STOP=1 -f source/first-query.sql pbe

Read in chapter order. Before chapter 6, create the sample table using the files introduced in chapters 16 and 24:

psql -X -v ON_ERROR_STOP=1 -f source/create-table.sql -f source/insert.sql pbe

Chapter 16 recreates the table; chapter 24 fills it again. Later join lessons depend on chapter 29, and the recursive example also uses chapter 31. When skipping ahead, follow the lesson’s prerequisites. Optional exercises can change later results. Dates, UUIDs, catalog listings, query plans, and timings vary; ellipses in console transcripts mark omitted output.

Two special cases

Chapter 46 deliberately raises a duplicate-key error: run it with ON_ERROR_STOP=off so the subsequent rollback and cleanup execute. For chapter 55, use a non-superuser database owner without BYPASSRLS; superusers bypass row security even with FORCE ROW LEVEL SECURITY.

Contents

Examples

  1. Example 1.1 — First Query
  2. Example 2.1 — psql Basics
  3. Example 3.1 — SELECT Basics
  4. Example 4.1 — WHERE
  5. Example 5.1 — ORDER BY
  6. Example 6.1 — SELECT from a Table
  7. Example 7.1 — LIMIT and OFFSET
  8. Example 8.1 — DISTINCT
  9. Example 9.1 — NULLs
  10. Example 10.1 — Expressions
  11. Example 11.1 — Numeric Types
  12. Example 12.1 — Text Types
  13. Example 13.1 — Boolean and Dates
  14. Example 14.1 — UUID and JSONB
  15. Example 15.1 — Arrays and ENUM
  16. Example 16.1 — CREATE TABLE
  17. Example 17.1 — Column Types and Constraints
  18. Example 18.1 — ALTER TABLE and DROP
  19. Example 19.1 — Primary Keys and UNIQUE
  20. Example 20.1 — Foreign Keys and REFERENCES
  21. Example 21.1 — NOT NULL and DEFAULT
  22. Example 22.1 — Identity Columns and Sequences
  23. Example 23.1 — Schemas and search_path
  24. Example 24.1 — INSERT
  25. Example 25.1 — UPSERT (ON CONFLICT)
  26. Example 26.1 — UPDATE
  27. Example 27.1 — DELETE
  28. Example 28.1 — RETURNING
  29. Example 29.1 — INNER and LEFT JOIN
  30. Example 30.1 — RIGHT and FULL JOIN
  31. Example 31.1 — Self-Join
  32. Example 32.1 — UNION, INTERSECT, EXCEPT
  33. Example 33.1 — COUNT, SUM, AVG
  34. Example 34.1 — GROUP BY
  35. Example 35.1 — HAVING
  36. Example 36.1 — Scalar and IN Subqueries
  37. Example 37.1 — EXISTS and Derived Tables
  38. Example 38.1 — Common Table Expressions (WITH)
  39. Example 39.1 — Window Functions
  40. Example 40.1 — String and Numeric Functions
  41. Example 41.1 — Date Functions and COALESCE
  42. Example 42.1 — CASE
  43. Example 43.1 — CREATE INDEX
  44. Example 44.1 — When to Index
  45. Example 45.1 — EXPLAIN and Query Plans
  46. Example 46.1 — BEGIN, COMMIT, ROLLBACK
  47. Example 47.1 — Savepoints
  48. Example 48.1 — CREATE VIEW
  49. Example 49.1 — Materialized Views
  50. Example 50.1 — Functions and PL/pgSQL
  51. Example 51.1 — Triggers
  52. Example 52.1 — Full-Text Search
  53. Example 53.1 — VACUUM and ANALYZE
  54. Example 54.1 — pg_dump and pg_restore
  55. Example 55.1 — Roles and GRANT
  56. Example 56.1 — psql Meta-commands
  57. Example 57.1 — COPY

About this edition

This English web edition follows the boringcollege/postgres-by-example repository at commit ffde0d358722498e37982b57bdbd711871a61c43. The lesson prose and SQL examples are kept together, with reviewed technical corrections documented in the editorial notes. The repository is licensed under Creative Commons Attribution 4.0 International.

Publication and license · Editorial corrections

1 First Query

Run your first query and check the PostgreSQL server and psql client versions.

The fastest way to confirm your PostgreSQL setup is working is to run a query that needs no tables. PostgreSQL is happy to evaluate expressions on their own — every SELECT does not need a FROM. We will use that property to verify three things in one go: the server is running, your client can reach it, and you have a recent enough version to follow along.

What you’ll learn

  • How to run a SQL file with psql -f
  • A minimal SELECT with no table
  • How to read the PostgreSQL version string
  • The difference between client and server versions

Example

Example 1.1 First Query
-- Simplest query: no tables, just a literal
SELECT 1;

-- Arithmetic and string expressions work without a FROM
SELECT 2 + 3 AS sum, 'hello, postgres' AS greeting;

-- Check your PostgreSQL server version
SELECT version();

-- A more readable version, when you only need the number
SHOW server_version;

SELECT 1 returns a single row with one column — the integer 1. The expression form (SELECT 2 + 3, 'hello, postgres') shows that any expression can be the body of a SELECT; PostgreSQL evaluates and returns it. version() is a built-in function that returns the full version string (operating system, compiler, etc.), while SHOW server_version returns just the version number. Both report the connected server’s version. psql --version reports the client version, which may differ.

Run the example

To run this file from the repository root:

$ psql -f source/first-query.sql postgres
 ?column?
----------
        1
(1 row)

 sum | greeting
-----+-----------------
   5 | hello, postgres
(1 row)

                                                    version
------------------------------------------------------------------------------------------------------------------------------
 PostgreSQL 17.x on x86_64-pc-linux-gnu, compiled by gcc ...
(1 row)

 server_version
----------------
 17.x
(1 row)

Your version string will vary. If you see output like above, everything is wired up.

Common pitfalls

  • “could not connect to server” — the daemon isn’t running. Start it with brew services start postgresql (macOS), sudo systemctl start postgresql (Linux), or whatever your install uses.
  • “role ‘you’ does not exist” — PostgreSQL doesn’t know about your OS user as a database user. Create one with createuser --superuser $(whoami) from a shell that has access to the cluster.
  • “database ‘postgres’ does not exist” — some installs (especially homebrew on macOS) don’t create a default postgres database. Run createdb postgres once, or use createdb pbe and pass pbe as the database name to every command.
Tip

If you omit the database name, psql uses your OS username as the default. To use the postgres database explicitly: psql -f source/first-query.sql postgres. To pass a connection string instead, use psql 'postgres://user:pass@host:5432/dbname'.

Try it

Run psql -f source/first-query.sql and confirm the output. Then connect interactively with psql postgres and type SELECT now(); followed by Enter. Exit with \q.

Downloads

SQL file: first-query.sql · All lesson files · Setup guide

2 psql Basics

Run SQL files, explore a database interactively, and get help from psql.

psql is the official command-line client for PostgreSQL, and it is excellent. Most professionals reach for it before any GUI — it is fast, scriptable, and ships with every PostgreSQL install. You use it two ways: in batch mode (psql -f file.sql) to run a script, or interactively (psql) to type SQL and meta-commands one at a time. This lesson focuses on the basics; a later lesson covers more meta-commands.

What you’ll learn

  • The two modes of psql: file and interactive
  • How to identify your session (current user, database, schema)
  • The difference between SQL statements and psql meta-commands
  • Where to get help: \? for meta-commands, \h for SQL

Example

Example 2.1 psql Basics
-- List user databases. Equivalent to \l in interactive psql.
SELECT datname
FROM pg_database
WHERE datistemplate = false
ORDER BY datname
LIMIT 5;

-- Identify the session: who am I, where am I, what's on my search_path?
SELECT current_user, current_database(), current_schema();

-- A few server settings worth knowing about
SHOW timezone;
SHOW server_encoding;

pg_database is one of many system catalogs — internal tables PostgreSQL exposes for inspection. The datistemplate = false filter hides template0 and template1, which are the templates new databases are cloned from. current_user, current_database(), and current_schema() are session functions; they answer questions you will ask yourself often when debugging connection or permission issues. SHOW displays runtime parameters — these can be changed at the server, database, role, or session level.

When you use psql interactively, you can also type meta-commands (also called backslash commands) that the client interprets locally:

\l         list databases
\c db      connect to database `db`
\dt        list visible tables in the current search_path
\d table   describe a table (columns, types, indexes)
\dn        list schemas
\df        list functions
\du        list roles (users/groups)
\timing    toggle showing query timing
\x         toggle expanded display (great for wide rows)
\?         help for meta-commands
\h SELECT  help for the SELECT command (or any SQL keyword)
\q         quit

Meta-commands are interpreted by the psql client, both interactively and in files read with psql -f. Commands such as \dt send catalog queries to the server; the backslash syntax itself is not server SQL.

Run the example

To run this file:

$ psql -f source/psql-basics.sql postgres
       datname
--------------------
 postgres
(1 row)

 current_user | current_database | current_schema
--------------+------------------+----------------
 your_user    | postgres         | public
(1 row)

 TimeZone
----------
 UTC
(1 row)

 server_encoding
-----------------
 UTF8
(1 row)

Common pitfalls

  • Meta-commands require the psql client. psql -f executes \dt in a file; a driver or GUI that sends the file directly as SQL will not interpret it.
  • Confusing current_user and session_user: when a SET ROLE is in effect, current_user reflects the effective role; session_user is the original login.
  • Encoding mismatches: if your terminal is not UTF-8 and your database is, you may see garbled text — set client_encoding to match your terminal.
Tip

Add \set PROMPT1 '%n@%/%x%R%# ' to ~/.psqlrc so your prompt shows user, database, and transaction state. Use \e to edit the previous query in your $EDITOR. Use \timing on when you are tuning queries.

Try it

Start an interactive session with psql postgres, run \l, then \dt, then \d pg_database, then \q to quit. Compare what \dt shows in different schemas using \dt pg_catalog.*.

Downloads

SQL file: psql-basics.sql · All lesson files · Setup guide

3 SELECT Basics

Select values and columns, name query results, and inspect system catalogs.

SELECT is the heart of SQL — it is how you read data. The simplest form is SELECT expression, with no FROM, which evaluates the expression and returns one row. The more common form, SELECT columns FROM table, returns one row per matching row in the table. Either way, the result is a relation: rows and columns, with column names you can rename using AS.

What you’ll learn

  • SELECT with literals and expressions (no table needed)
  • Naming result columns with AS
  • Selecting multiple columns in one query
  • Reading from a system catalog table
  • Column aliases vs. table aliases

Example

Example 3.1 SELECT Basics
-- Literals and expressions: no FROM required
SELECT 1 AS one;
SELECT 2 + 3 AS sum, 10 / 4 AS quotient, 10 / 4.0 AS real_quotient;
SELECT 'hello' AS greeting, length('hello') AS chars;

-- Multiple columns in one row
SELECT 1 AS a, 2 AS b, 'x' AS c;

-- Reading from a table (here, a system catalog)
SELECT relname, relkind
FROM pg_class
WHERE relname = 'pg_database'
LIMIT 1;

Each SELECT produces a result set. AS column_name renames the output column — without it, PostgreSQL invents a name like ?column? for literals or uses the source column name. The arithmetic line is worth a second look: 10 / 4 returns the integer 2 because both operands are integers, while 10 / 4.0 returns 2.5 because one operand is numeric and integer division does not apply. This trap shows up often when computing averages and ratios.

pg_class is the system catalog that lists relations — tables, indexes, views, sequences, materialized views, and more. The relkind column distinguishes them (r = ordinary table, i = index, v = view, m = materialized view, S = sequence). Looking at catalogs is a habit worth developing; everything about your schema is visible there.

Run the example

To run:

$ psql -f source/select-basics.sql postgres
 one
-----
   1
(1 row)

 sum | quotient | real_quotient
-----+----------+---------------
   5 |        2 |            2.5
(1 row)

 greeting | chars
----------+-------
 hello    |     5
(1 row)

 a | b | c
---+---+---
 1 | 2 | x
(1 row)

   relname   | relkind
-------------+---------
 pg_database | r
(1 row)

Common pitfalls

  • Forgetting AS on an expression: SELECT count(*) works, but SELECT count(*) FROM ... WHERE total > 0 GROUP BY id gets unreadable without AS order_count. Always alias derived columns.
  • Integer division: 1 / 2 returns 0. Cast one side: 1::numeric / 2, or 1.0 / 2.
  • Double-quotes vs. single-quotes: 'hello' is a string literal; "hello" is an identifier (a column or table name). Mixing them up is one of the most common SQL beginner errors.
Tip

You can mix kinds of columns freely: SELECT id, name, id * 2 AS doubled, current_timestamp AS fetched_at FROM fruits; is perfectly fine. The result columns appear in the order written and use the aliases as names; expression evaluation order is not guaranteed.

Try it

Run SELECT 10 * 2 AS twenty;. Then try SELECT relname FROM pg_class WHERE relkind = 'r' LIMIT 5; to see five table names in your database. Add , relkind after relname and observe.

Downloads

SQL file: select-basics.sql · All lesson files · Setup guide

4 WHERE

Filter rows with comparisons, patterns, ranges, and SQL’s three-valued logic.

WHERE is how you filter rows. The clause sits between FROM and ORDER BY, and PostgreSQL keeps only rows for which the condition evaluates to true. Anything that evaluates to false or NULL is dropped — this three-valued logic is one of SQL’s quirks and the cause of many beginner bugs. Get comfortable with it early.

What you’ll learn

  • Filtering with comparison operators (=, <>, <, >, <=, >=)
  • Pattern matching with LIKE (and ILIKE for case-insensitive)
  • Combining conditions with AND, OR, NOT
  • Membership and ranges: IN, BETWEEN
  • Why = NULL does not work and what to use instead

Example

Example 4.1 WHERE
-- Equality and ordering
SELECT schemaname, tablename
FROM pg_tables
WHERE schemaname = 'pg_catalog'
ORDER BY tablename
LIMIT 3;

-- Pattern matching: % matches any sequence, _ matches one character
SELECT tablename
FROM pg_tables
WHERE tablename LIKE 'pg_a%'
ORDER BY tablename
LIMIT 5;

-- Case-insensitive: ILIKE (PostgreSQL extension)
SELECT tablename FROM pg_tables WHERE tablename ILIKE 'PG_Class' LIMIT 1;

-- Multiple conditions
SELECT tablename
FROM pg_tables
WHERE schemaname = 'pg_catalog'
  AND tablename LIKE 'pg_%'
  AND tablename NOT LIKE 'pg_stat_%'
ORDER BY tablename
LIMIT 5;

-- IN: membership test
SELECT tablename
FROM pg_tables
WHERE tablename IN ('pg_class', 'pg_database', 'pg_namespace')
ORDER BY tablename;

-- BETWEEN: inclusive range
SELECT n FROM generate_series(1, 10) AS n WHERE n BETWEEN 3 AND 7;

LIKE is the SQL standard pattern operator: % matches any string of characters (including empty), _ matches exactly one character. ILIKE is a PostgreSQL extension for case-insensitive matching. Index use depends on the pattern, collation, and operator class. Alphabetic prefixes generally need a suitable expression or trigram index; a B-tree can support some patterns beginning with characters unaffected by case. For more powerful matching, PostgreSQL also supports ~ (regex) and ~* (case-insensitive regex), as well as full-text search (covered later).

AND binds tighter than OR, so A AND B OR C means (A AND B) OR C. When in doubt, add parentheses — your future self reading the query will thank you. IN (...) is shorthand for = ANY (ARRAY[...]) and is equivalent to a chain of ORs.

Run the example

To run:

$ psql -f source/where.sql postgres
 schemaname |  tablename
------------+--------------
 pg_catalog | pg_aggregate
 pg_catalog | pg_am
 pg_catalog | pg_amop
(3 rows)
 ...

Common pitfalls

  • = NULL and <> NULL are never true (they are NULL). Use IS NULL and IS NOT NULL. This trips up engineers from every other language.
  • NOT IN (subquery) becomes mysterious if the subquery contains a NULL — nonmatching values yield NULL while matching values yield false, so the filter keeps no rows. Prefer NOT EXISTS for nullable subqueries.
  • LIKE 'abc' does not match 'ABC'. Use ILIKE or apply LOWER() to both sides — but LOWER(col) = 'abc' defeats a plain index on col (add an expression index on LOWER(col) if you need it often).
  • Be careful with IN ('a', 'b', NULL) — the NULL in the list contributes nothing useful and silently makes NOT IN behave unexpectedly.
Tip

When filtering text by prefix, WHERE name LIKE 'foo%' can use a B-tree index with a suitable collation or text_pattern_ops operator class. WHERE name LIKE '%foo' (suffix) or '%foo%' (substring) cannot — consider a trigram index (pg_trgm extension) for those.

Try it

Change the condition to WHERE schemaname = 'public' — if you have no tables in public yet, you will get zero rows. Then try WHERE tablename ~ '^pg_a.*' to see regex matching.

Downloads

SQL file: where.sql · All lesson files · Setup guide

5 ORDER BY

Sort query results, break ties, and control where NULL values appear.

ORDER BY sorts the result set. Without it, the order of rows is undefined — PostgreSQL is free to return rows in whatever sequence the planner finds convenient, and that order can change between runs. If your application cares about order, say so explicitly.

What you’ll learn

  • Sorting with ORDER BY column
  • ASC (default) and DESC
  • Multi-column sort (tie-breakers)
  • Controlling where NULLs land (NULLS FIRST / NULLS LAST)
  • Sorting by expressions and by position

Example

Example 5.1 ORDER BY
-- Default ascending
SELECT tablename
FROM pg_tables
WHERE schemaname = 'pg_catalog'
ORDER BY tablename
LIMIT 4;

-- Descending
SELECT tablename
FROM pg_tables
WHERE schemaname = 'pg_catalog'
ORDER BY tablename DESC
LIMIT 4;

-- Multi-column: sort by schema, then by name within schema
SELECT schemaname, tablename
FROM pg_tables
ORDER BY schemaname, tablename
LIMIT 5;

-- NULLs FIRST / LAST: default for ASC is NULLS LAST, for DESC is NULLS FIRST
SELECT * FROM (VALUES (1), (NULL), (2)) AS t(n) ORDER BY n NULLS FIRST;

-- Sort by an expression
SELECT tablename, length(tablename) AS len
FROM pg_tables
WHERE schemaname = 'pg_catalog'
ORDER BY length(tablename) DESC
LIMIT 3;

ORDER BY tablename sorts in ascending order — ASC is the default. DESC reverses. When two rows have the same first sort key, the second is used as a tie-breaker, and so on. You can sort by any expression, including ones that don’t appear in the select list, and by output position (ORDER BY 1, 2) — though using column names is much more readable.

NULL handling is worth knowing: ASC puts NULLs last; DESC puts NULLs first. Override with NULLS FIRST or NULLS LAST. For applications that always want to see NULLs at the end regardless of direction, write it out — relying on the default is a source of bugs when someone later flips ASC to DESC.

Run the example

To run:

$ psql -f source/order-by.sql postgres
 tablename
--------------
 pg_aggregate
 pg_am
 pg_amop
 pg_amproc
(4 rows)

   tablename
-----------------
 pg_user_mapping
 pg_type
 pg_ts_template
 pg_ts_parser
(4 rows)
 ...

Common pitfalls

  • Sorting on text and getting surprising results: text sort respects the database’s collation (e.g. en_US.UTF-8), which may sort case-insensitively or treat accents specially. Use ORDER BY col COLLATE "C" for byte-order sorting.
  • Forgetting ORDER BY with LIMIT: LIMIT 10 without ORDER BY returns some 10 rows, not necessarily the first or smallest. Pagination is meaningless without a deterministic order.
  • Two rows with the same primary sort key may appear in any order — add the primary key as a final tie-breaker for fully deterministic results (ORDER BY created_at DESC, id DESC).
  • Sorting can be expensive on large result sets. PostgreSQL can use an index to avoid sorting if the index order matches the requested order.
Tip

When listing recent activity, sort by created_at DESC, id DESC. The id tie-breaker keeps pagination consistent if multiple rows share a timestamp.

Try it

Add a second sort column: ORDER BY schemaname, tablename to group by schema then sort within. Then sort by length(tablename) DESC and see which catalog table has the longest name.

Downloads

SQL file: order-by.sql · All lesson files · Setup guide

6 SELECT from a Table

Read the sample fruits table with explicit columns, aliases, and filters.

Now that fruits has rows, you can read them. SELECT * returns every column; listing column names returns just those, in the order you list them. The result is unordered unless you add ORDER BY — even if it looks ordered today, it may not be tomorrow.

What you’ll learn

  • SELECT * FROM table for all columns
  • Selecting specific columns and controlling their order in the result
  • Combining SELECT with WHERE and ORDER BY
  • Why SELECT * is fine for ad-hoc queries but risky in application code
  • Result column names and how to override them
Prepare the sample table

Before this lesson, run psql -f source/create-table.sql -f source/insert.sql postgres in your scratch database. These are the files introduced in chapters 16 and 24; the table must exist before the SELECT lessons.

Example

Example 6.1 SELECT from a Table
-- All columns. The order matches CREATE TABLE.
SELECT * FROM fruits ORDER BY id;

-- Specific columns, in the order you list them
SELECT name, id FROM fruits ORDER BY id;

-- Renaming output columns
SELECT id AS fruit_id, name AS fruit_name FROM fruits ORDER BY id LIMIT 3;

-- Combining a filter and an order
SELECT id, name
FROM fruits
WHERE id BETWEEN 2 AND 4
ORDER BY name;

-- Counting rows
SELECT count(*) AS total FROM fruits;

Run this after create-table.sql and insert.sql. SELECT * returns every column in the physical column order. Listing columns gives you total control over the result shape, which matters when the data flows into application code that maps columns by position. AS renames a column in the output without changing anything in the table.

The space character is optional around * and around operators; the SQL parser is generous with whitespace. Multi-line queries are normal and encouraged for anything beyond two columns.

Run the example

To run:

$ psql -f source/select-from-table.sql postgres
 id  |   name
-----+-----------
   1 | apple
   2 | banana
   3 | cherry
   4 | date
   5 | elderberry
 101 | fruit_1
 102 | fruit_2
 103 | fruit_3
(8 rows)

    name    | id
------------+-----
 apple      |   1
 banana     |   2
 ...

Common pitfalls

  • SELECT * FROM big_table; will fetch every column of every row across the network — slow and wasteful. List the columns you actually need.
  • Output column names are not guaranteed unique. SELECT id, id FROM fruits has two columns both named id — application drivers may not handle that gracefully.
  • In application code, prefer explicit columns over SELECT *. When the schema adds a column, the shape of SELECT * changes and may break code that assumes a particular column order or count.
Tip

When you’re exploring data, \x in psql toggles expanded display — each row prints one column per line. It is much easier to read a wide row that way.

Try it

Run SELECT * FROM fruits WHERE id > 100; to see only the generated rows. Then run SELECT name FROM fruits ORDER BY name; for alphabetical names. Then try SELECT id || ': ' || name AS label FROM fruits; to build a label column with the || string concatenation operator.

Downloads

SQL file: select-from-table.sql · All lesson files · Setup guide

7 LIMIT and OFFSET

Limit result size and compare offset pagination with keyset pagination.

LIMIT n returns at most n rows. OFFSET m skips the first m rows of the result. Combined, they implement classic page-based pagination: page k of size p is LIMIT p OFFSET (k - 1) * p. Without an ORDER BY, “first” is meaningless — always pair LIMIT/OFFSET with an explicit order.

What you’ll learn

  • Restricting result size with LIMIT
  • Skipping rows with OFFSET
  • Pagination patterns and their costs
  • Why offset-based pagination scales poorly on large tables
  • Keyset (cursor) pagination as the better alternative

Example

Example 7.1 LIMIT and OFFSET
-- First two rows, ordered by id
SELECT * FROM fruits ORDER BY id LIMIT 2;

-- Skip one, then take two: rows 2 and 3
SELECT * FROM fruits ORDER BY id LIMIT 2 OFFSET 1;

-- Page 2 of 3 per page
SELECT * FROM fruits ORDER BY id LIMIT 3 OFFSET 3;

-- Keyset pagination: rows with id > last_seen_id
SELECT * FROM fruits WHERE id > 2 ORDER BY id LIMIT 2;

LIMIT 2 OFFSET 1 reads “two rows, after skipping one.” Internally, PostgreSQL still has to compute all rows up to OFFSET + LIMIT and discard the offset — for page 10,000 of 50 rows per page, that means processing 500,000 rows for a one-based page number. Offset pagination is fine for the first few pages but terrible for deep pagination.

The keyset pattern (also called cursor-based or seek pagination) uses the last seen value as the filter: WHERE id > $last_id ORDER BY id LIMIT 50. The query touches only the next 50 rows regardless of how far you have paginated. It requires an indexed column (often the primary key or created_at, id) but is the standard approach in production systems.

Run the example

To run (requires fruits from earlier lessons):

$ psql -f source/limit-offset.sql postgres
 id |  name
----+--------
  1 | apple
  2 | banana
(2 rows)

 id |  name
----+--------
  2 | banana
  3 | cherry
(2 rows)
 ...

Common pitfalls

  • LIMIT 10 without ORDER BY returns some ten rows — and the choice can change between executions. Always sort.
  • Two rows with the same sort key may swap places on different pages. Add a tie-breaker (ORDER BY created_at DESC, id DESC) for stable pagination.
  • Deep offsets are slow. For page 1,000 of an “infinite scroll” feed, switch to keyset.
  • OFFSET is also used in SELECT ... FOR UPDATE patterns and can interact poorly with row locks; if you need to process batches, keyset over a primary key is again the safer pattern.
Tip

PostgreSQL also accepts FETCH FIRST n ROWS ONLY (SQL-standard), equivalent to LIMIT n. FETCH FIRST n ROWS WITH TIES returns more than n if there are ties at the boundary — handy for top-N queries.

Try it

Compute “the third-largest fruit id”: SELECT id FROM fruits ORDER BY id DESC LIMIT 1 OFFSET 2;. Then try OFFSET 0, OFFSET 1, etc. Then rewrite as keyset pagination: SELECT id FROM fruits WHERE id < (some_value) ORDER BY id DESC LIMIT 1.

Downloads

SQL file: limit-offset.sql · All lesson files · Setup guide

8 DISTINCT

Remove duplicate rows and choose one row per group with DISTINCT ON.

DISTINCT removes duplicate rows from the result set. The deduplication considers the entire row of the SELECT — DISTINCT a, b is “distinct combinations of (a, b),” not “distinct a, then any b.” PostgreSQL also offers a more powerful sibling, DISTINCT ON (expr), which keeps one row per distinct value of expr — useful for “the most recent X per Y” queries.

What you’ll learn

  • Eliminating duplicates with DISTINCT
  • How DISTINCT applies to all columns in the SELECT list
  • The PostgreSQL-specific DISTINCT ON (expr)
  • How NULL is treated by DISTINCT
  • When to use GROUP BY instead

Example

Example 8.1 DISTINCT
-- Unique values in a column
SELECT DISTINCT name FROM fruits ORDER BY name LIMIT 5;

-- Distinct combinations of multiple columns
SELECT DISTINCT id, name FROM fruits ORDER BY id LIMIT 3;

-- DISTINCT ON: keep one row per distinct expression value.
-- Here: one row per first letter of the name, picking the smallest id.
SELECT DISTINCT ON (substring(name, 1, 1)) substring(name, 1, 1) AS letter, id, name
FROM fruits
ORDER BY substring(name, 1, 1), id;

-- DISTINCT treats NULL as a single value
SELECT DISTINCT n FROM (VALUES (1), (NULL), (NULL), (2)) AS t(n) ORDER BY n;

In our fruits table every name is already unique, so DISTINCT name returns the same rows as SELECT name. With multiple columns, DISTINCT requires all listed values to be the same for two rows to count as duplicates.

DISTINCT ON (substring(name, 1, 1)) is PostgreSQL’s superpower for “first row per group” queries. The ON (...) expressions define the groups; the ORDER BY decides which row from each group is kept (the first one after sorting). The trick is that the ON expressions must appear at the start of the ORDER BY — otherwise PostgreSQL cannot guarantee which row you mean by “the first.”

For NULL handling, DISTINCT considers two NULLs equal — counterintuitive given that NULL = NULL is unknown, but it is what the standard requires.

Run the example

To run:

$ psql -f source/distinct.sql postgres
    name
------------
 apple
 banana
 cherry
 date
 elderberry
(5 rows)
 ...

Common pitfalls

  • SELECT DISTINCT a, b FROM t and SELECT DISTINCT a, (b) FROM t mean the same thing: deduplicate (a, b) pairs. Parentheses around b do not make the operation apply only to a.
  • DISTINCT and ORDER BY interact: any column in ORDER BY not in the DISTINCT list will cause an error (you cannot order by something that may have multiple values per distinct group).
  • DISTINCT ON is non-standard SQL. Code that needs to be portable should use a window function (ROW_NUMBER() OVER (PARTITION BY ...)) instead — covered in a later lesson.
  • SELECT DISTINCT on a large table forces a sort or hash; if you only need to know “are there duplicates?” use GROUP BY ... HAVING COUNT(*) > 1 or an aggregate.
Tip

count(DISTINCT col) counts distinct non-null values of col. It is slower than count(*) because it must deduplicate, but it answers “how many different X are there?” without needing a subquery.

Try it

Insert another row with an existing name (e.g. INSERT INTO fruits (id, name) VALUES (10, 'apple');) and run SELECT DISTINCT name FROM fruits ORDER BY name; — the count should not change. Then use DISTINCT ON (name) to keep the largest id per name (hint: ORDER BY name, id DESC).

Downloads

SQL file: distinct.sql · All lesson files · Setup guide

9 NULLs

Understand NULL, unknown comparisons, and NULL-aware expressions.

NULL is SQL’s marker for unknown or missing. It is not zero, not the empty string, and not equal to anything — not even to another NULL. This three-valued logic (true, false, unknown) is the source of more SQL bugs than any other feature, but the rules are simple once you know them.

What you’ll learn

  • What NULL represents
  • Testing for NULL with IS NULL / IS NOT NULL
  • How NULL propagates through expressions and comparisons
  • How NULL behaves in AND/OR (three-valued logic)
  • Replacing NULL with COALESCE and NULLIF

Example

Example 9.1 NULLs
-- NULL is missing/unknown
SELECT 1 AS a, NULL AS b;

-- = NULL is NULL (not true), so this returns 0 rows
SELECT * FROM (VALUES (1, NULL)) AS t(id, name) WHERE name = NULL;

-- IS NULL / IS NOT NULL are the correct tests
SELECT * FROM (VALUES (1, NULL)) AS t(id, name) WHERE name IS NULL;
SELECT * FROM (VALUES (1, 'x')) AS t(id, name) WHERE name IS NOT NULL;

-- Any arithmetic with NULL is NULL
SELECT 1 + NULL AS one_plus_null, 'a' || NULL AS concat_null;

-- COALESCE returns the first non-null argument
SELECT COALESCE(NULL, NULL, 'default') AS coalesced;

-- NULLIF returns NULL if the two args are equal, else the first
SELECT NULLIF(0, 0) AS hides_zero, NULLIF(5, 0) AS keeps_five;

-- Three-valued logic
SELECT (TRUE AND NULL) AS t_and_n, (TRUE OR NULL) AS t_or_n, (FALSE AND NULL) AS f_and_n;

= NULL does not work because the comparison evaluates to NULL, which is not TRUE, so the WHERE discards the row. The same applies to <>, <, >, IN, BETWEEN, and so on — almost any operator returns NULL when either operand is NULL. The exceptions are IS NULL, IS NOT NULL, IS DISTINCT FROM, and IS NOT DISTINCT FROM (the last two are NULL-aware versions of <> and =).

COALESCE(a, b, c) returns the first argument that is not NULL. It is the standard way to provide defaults: COALESCE(nickname, first_name, 'Anonymous'). NULLIF(a, b) returns NULL when a = b — handy when you want to turn sentinel values back into NULL (e.g. NULLIF(comment, '')).

The three-valued logic table is short and worth memorizing: TRUE OR NULL is TRUE (we know at least one side is true), but TRUE AND NULL is NULL (we don’t know the other side). FALSE AND NULL is FALSE (one false is enough). FALSE OR NULL is NULL.

Run the example

To run:

$ psql -f source/nulls.sql postgres
 a | b
---+---
 1 |
(1 row)
 ...

Common pitfalls

  • Forgetting that aggregates skip NULL: SUM, AVG, MIN, MAX, COUNT(col) ignore NULL inputs. COUNT(*) counts every row including those with NULLs.
  • NOT IN (subquery) becomes the empty result if the subquery returns a single NULL — because x <> NULL is NULL. Prefer NOT EXISTS.
  • NULL <> NULL is unknown (NULL), not true. DISTINCT groups NULLs together; IS DISTINCT FROM provides a comparison that always returns true or false.
  • Default ordering: ORDER BY col ASC puts NULLs last; DESC puts them first. Specify NULLS FIRST/NULLS LAST for predictable behavior.
Tip

When you really want NULL-aware equality, use IS NOT DISTINCT FROM. For example, WHERE a IS NOT DISTINCT FROM b is true when both are equal or both are NULL.

Try it

Run SELECT 1 = NULL;, then SELECT 1 IS NOT NULL;, then SELECT NULL IS NOT DISTINCT FROM NULL;. See how the answers differ. Then try SELECT count(*) AS rows, count(b) AS non_null_b FROM (VALUES (1, NULL), (2, 'x')) AS t(a, b);.

Downloads

SQL file: nulls.sql · All lesson files · Setup guide

10 Expressions

Combine arithmetic, strings, function calls, and type casts in SQL expressions.

An expression is anything PostgreSQL can evaluate to a single value: a literal, a column reference, an arithmetic operation, a function call, a CASE, even another SELECT that returns one value. Expressions can appear almost anywhere — in the select list, in WHERE, in ORDER BY, in GROUP BY, in index definitions. Mastering expressions is mastering SQL.

What you’ll learn

  • Arithmetic, string, and boolean expressions
  • Calling built-in functions in expressions
  • Using expressions in SELECT, WHERE, and ORDER BY
  • Casting between types with :: and CAST(... AS ...)
  • Naming expression results with AS

Example

Example 10.1 Expressions
-- Arithmetic and concatenation
SELECT 10 + 2 AS sum, 10 * 2 AS product, 10 / 4 AS int_div, 10 / 4.0 AS real_div;
SELECT 'Hello ' || 'world' AS greeting;
SELECT 'fruit_' || 1 AS labeled;  -- numbers are coerced to text for ||

-- Function calls in expressions
SELECT upper(name), length(name), name || ' (#' || id || ')' AS label
FROM fruits ORDER BY id LIMIT 3;

-- Expression in WHERE
SELECT * FROM fruits WHERE id * 2 > 4 ORDER BY id LIMIT 3;

-- Expression in ORDER BY
SELECT id, name FROM fruits ORDER BY length(name) DESC LIMIT 3;

-- Casting
SELECT '42'::integer AS to_int, 42::text AS to_text, '2024-01-15'::date AS to_date;
SELECT CAST(3.7 AS integer) AS rounded_value;  -- SQL-standard cast syntax

The || operator concatenates strings. In PostgreSQL, if either operand is not text, it is coerced to text (so 'id_' || 5 works). +, -, *, /, % are the usual arithmetic operators; remember that integer division truncates toward zero.

PostgreSQL casts come in two flavors. value::type is the PostgreSQL-specific shortcut and is the common style. CAST(value AS type) is SQL-standard and reads better when the expression is complex. Both compile to the same thing.

Expressions in WHERE and ORDER BY are evaluated for every candidate row, which means a function call there can prevent index use. WHERE lower(email) = 'a@b.com' cannot use a plain index on email; it can use an expression index defined as CREATE INDEX ON users (lower(email)).

Run the example

To run:

$ psql -f source/expressions.sql postgres
 sum | product | int_div | real_div
-----+---------+---------+----------
  12 |      20 |       2 |      2.5
(1 row)

   greeting
-------------
 Hello world
(1 row)
 ...

Common pitfalls

  • Integer division returning 0: 1 / 2 is 0. Cast: 1.0 / 2, 1::numeric / 2, or use numeric columns.
  • Casting strings that don’t parse — e.g. 'abc'::integer raises an error. PostgreSQL is strict about types compared to MySQL.
  • Expressions returning different types in different branches of CASE cause type-mismatch errors; cast to a common type.
  • Operators have precedence; 1 + 2 * 3 is 7. When in doubt, parenthesize.
Tip

When an expression is reused several times in one query, put it in a CTE or derived table once, instead of repeating it. PostgreSQL is smart enough to fold many duplicates, but the code stays cleaner.

Try it

Run SELECT id, name, id || ': ' || name AS label FROM fruits ORDER BY id LIMIT 3;. Then try SELECT name FROM fruits WHERE length(name) > 5;. Then cast: SELECT '3.14'::numeric * 2 AS doubled;.

Downloads

SQL file: expressions.sql · All lesson files · Setup guide

11 Numeric Types

Choose between integers, exact decimals, and floating-point numbers.

PostgreSQL offers three families of numeric types: integers, exact decimals, and floating-point. Each has its place. Integers are fast and compact for counters and IDs. numeric (also spelled decimal) gives you exact arithmetic — essential for money. Floats (real, double precision) are fast and small but can produce tiny rounding errors and should not be used for currency.

What you’ll learn

  • Integer family: smallint, integer, bigint
  • Exact decimals: numeric(p, s) and unconstrained numeric
  • Floating-point: real, double precision
  • How precision and scale work
  • When each type is the right choice

Example

Example 11.1 Numeric Types
-- Integer family
SELECT 1::smallint AS s, 1::integer AS i, 1000000::bigint AS b;

-- Range limits (smallint -32768..32767, integer -2.1B..2.1B, bigint very large)
SELECT pg_typeof(1) AS literal_int_type;  -- 'integer' by default

-- Exact decimal: numeric(precision, scale)
-- precision = total digits, scale = digits after the decimal point
SELECT 3.14::numeric(4, 2) AS pi, 12345.67::numeric(7, 2) AS money;

-- Unconstrained numeric is exact within PostgreSQL's implementation limits
SELECT '0.1'::numeric + '0.2'::numeric AS exact;  -- 0.3, exactly

-- Floats are approximate but fast
SELECT 0.1::double precision + 0.2::double precision AS approximate;  -- 0.30000000000000004

-- Useful generators
SELECT generate_series(1, 5) AS n;

integer is your default integer type and is what 1 parses as. Switch to bigint when you might exceed two billion (rows in a large table, milliseconds since epoch, etc.). smallint is rarely worth it — the storage savings are tiny and values outside its range raise an error.

numeric(p, s) enforces up to p total digits with s of them after the decimal point. numeric(10, 2) stores values from -99,999,999.99 to 99,999,999.99. Unconstrained numeric (no p, s) stores exact decimal values within PostgreSQL’s implementation limits — slower than fixed integers, but exact. Use it for currency, lab measurements, anything where rounding errors are unacceptable.

real (4 bytes, ~6 significant digits) and double precision (8 bytes, ~15 significant digits) are IEEE 754 floats. They are fast and small, but 0.1 + 0.2 is not exactly 0.3. Use them for physical measurements, ML features, statistical aggregations where you can tolerate the imprecision.

Run the example

To run:

$ psql -f source/numeric-types.sql postgres
 s | i |    b
---+---+---------
 1 | 1 | 1000000
(1 row)
 ...
 exact
-------
 0.3
(1 row)

   approximate
------------------
 0.30000000000000004
(1 row)

Common pitfalls

  • Storing money in double precision — eventually the cent-rounding catches you. Use numeric(p, 2) (or numeric without bounds) and let the database handle exact arithmetic.
  • Choosing integer for an ID column on a large table — you can hit the 2.1-billion limit. Use bigint by default for primary keys you expect to grow.
  • Casting strings with non-numeric characters: '$5'::numeric errors. Strip first, then cast.
  • Dividing integers and being surprised: 7 / 2 is 3, not 3.5. Cast: 7::numeric / 2.
Tip

pg_typeof(expression) returns the inferred type — useful when you are uncertain. For example, SELECT pg_typeof(1 / 2); returns integer, while SELECT pg_typeof(1.0 / 2); returns numeric.

Try it

Create a small table with an integer id and a numeric(10, 2) price column, then insert and inspect: CREATE TEMP TABLE p(id integer, price numeric(10,2)); INSERT INTO p VALUES (1, 9.99), (2, 1234.5); SELECT * FROM p;. Then watch what happens with SELECT 0.1::real + 0.2::real;.

Downloads

SQL file: numeric-types.sql · All lesson files · Setup guide

12 Text Types

Store and compare text, measure character lengths, and understand padding and collation.

PostgreSQL has three character types: text (unbounded), varchar(n) (variable up to n characters), and char(n) (fixed n characters, space-padded). For almost all use cases, text is the right choice. PostgreSQL’s text and varchar have identical performance characteristics — unlike MySQL, there is no benefit to capping the length unless you want the database to enforce a maximum.

What you’ll learn

  • The three text types and when each makes sense
  • Why text is the default in PostgreSQL
  • Length functions: length, char_length, octet_length
  • The basics of collation — how text sorts and compares
  • Encoding: PostgreSQL stores text in the database’s encoding (usually UTF-8)

Example

Example 12.1 Text Types
-- text: unbounded, recommended default
SELECT 'hello'::text AS t, length('hello'::text) AS chars;

-- varchar(n): same as text but enforces max length
SELECT 'hi'::varchar(10) AS v, length('hi'::varchar(10)) AS chars;

-- char(n): fixed length, padded with spaces
SELECT 'ab'::char(5) AS c, length('ab'::char(5)) AS chars, '|' || 'ab'::char(5) || '|' AS padded;

-- char_length vs octet_length on multibyte text
SELECT char_length('café') AS chars, octet_length('café') AS bytes;

-- A taste of collation: byte-order sort
SELECT * FROM (VALUES ('apple'), ('Banana'), ('cherry')) AS t(name)
ORDER BY name COLLATE "C";

text is the workhorse. It stores strings of any length and is what most application columns should use. Internally, PostgreSQL stores text and varchar the same way — using TOAST (the Oversized Attribute Storage Technique) for large values so the main table stays compact.

varchar(n) is text with a length check. Use it only when you have a real reason to cap length (e.g. integration with a system that requires it). varchar without a length is exactly text.

char(n) is the historical fixed-width type. It pads values with spaces to length n on storage and trims them in some operations — which leads to surprises. Avoid char(n) unless you have a very specific reason (e.g. interfacing with a system that requires fixed-length codes).

Run the example

To run:

$ psql -f source/text-types.sql postgres
   t   | chars
-------+-------
 hello |     5
(1 row)

 v  | chars
----+-------
 hi |     2
(1 row)

   c   | chars |  padded
-------+-------+----------
 ab    |     2 | |ab|
(1 row)
 ...

Common pitfalls

  • Choosing varchar(255) from habit: it does nothing useful in PostgreSQL. Use text. If the application needs a length cap, enforce it in the application or with a CHECK constraint.
  • char(n) padding silently changes your values. 'foo'::char(10) is stored as 'foo '.
  • Multibyte characters: length('café') returns 4 (characters), but octet_length returns 5 (bytes in UTF-8). Match the function to the question you are asking.
  • Sorting comparisons depend on the database’s LC_COLLATE. To get byte-order sorting (often faster, and predictable), use COLLATE "C".
Tip

For case-insensitive text columns, consider the citext extension (CREATE EXTENSION citext) — it lets = and LIKE behave case-insensitively at the column level, so your queries do not have to call LOWER() everywhere.

Try it

Compare lengths: SELECT length('x'::text), length('x'::char(3));. Both lengths are 1: length(character) ignores trailing padding. Use octet_length('x'::char(3)) to observe the three stored bytes. Then try SELECT 'foo '::char(6) = 'foo'::char(3); — the answer is true because char comparisons ignore trailing spaces.

Downloads

SQL file: text-types.sql · All lesson files · Setup guide

13 Boolean and Dates

Work with booleans, dates, timestamps, time zones, and intervals.

PostgreSQL’s boolean type stores true, false, or NULL. For dates and times you have a small family of types: date for a calendar day, time for a time of day, timestamp for date-and-time without a time zone, and timestamptz (timestamp with time zone) for an instant displayed in the session time zone (the original zone/offset is not retained). For almost all application use, timestamptz is what you want.

What you’ll learn

  • The boolean type and how it accepts many inputs (true, 't', 'yes', 1)
  • The four core date/time types and when each makes sense
  • Why timestamptz is preferred over timestamp
  • current_date, current_timestamp, now(), localtime
  • Date arithmetic with interval

Example

Example 13.1 Boolean and Dates
-- Boolean: true, false, NULL
SELECT true AS t, false AS f, (1 = 1) AS same;

-- Boolean accepts many input forms
SELECT 'yes'::boolean AS yes, 'n'::boolean AS n, '1'::boolean AS one;

-- Dates and times
SELECT current_date AS today,
       current_timestamp AS now_tz,
       localtimestamp AS now_local,
       current_time AS time_now;

-- Casting date/time literals
SELECT '2024-01-15'::date AS d,
       '2024-01-15 10:30:00'::timestamp AS ts,
       '2024-01-15 10:30:00+00'::timestamptz AS ts_tz;

-- Arithmetic with interval
SELECT current_date + 7 AS in_a_week,
       current_timestamp - interval '1 hour' AS an_hour_ago,
       date '2024-12-31' - date '2024-01-01' AS days_between;

-- Extracting parts
SELECT extract(year FROM current_date) AS year,
       extract(dow  FROM current_date) AS day_of_week;

The boolean type accepts a generous set of input strings: true, false, t, f, yes, no, y, n, 1, 0, on, off. The output is always shown as t or f in psql. Comparisons (=, <>, etc.) produce booleans; you can store the result of a comparison in a boolean column.

timestamp without time zone (just timestamp) stores a wall-clock value with no notion of where on Earth it was. If you insert 2024-01-15 10:00:00, that is exactly what you get back, regardless of who reads it. timestamp with time zone (timestamptz) stores the same instant in UTC internally and presents it in the session’s time zone. Two sessions in different time zones will see the same timestamptz as different wall-clock times, but the underlying instant is the same.

For nearly every “when did this event happen?” question — order placed, log written, user created — timestamptz is the right answer. Plain timestamp is for things like “the recipe says bake at 10am local time, in whatever local time the user is in” — rare.

Run the example

To run:

$ psql -f source/boolean-and-dates.sql postgres
 t | f | same
---+---+------
 t | f | t
(1 row)
 ...
   today
------------
 2024-xx-xx
(1 row)
 ...

Common pitfalls

  • Storing event times as timestamp instead of timestamptz: a year later, you cannot tell whether they were UTC, server-local, or user-local. Always timestamptz.
  • Comparing a date or timestamp without time zone to a timestamptz uses the session time zone to interpret the value without a zone. Be explicit when it matters.
  • now() returns the time of the transaction start, not the wall-clock now. Inside a long transaction, repeated calls return the same value. Use clock_timestamp() for true wall-clock time.
  • interval '1 month' plus a date gives a timestamp in the next month — sometimes with surprises (January 31 + 1 month = February 28/29).
Tip

Set your session’s time zone with SET TIME ZONE 'UTC';. In production, run the database in UTC and convert in the application layer when displaying to users.

Try it

Run SELECT current_timestamp, now(), clock_timestamp(); — current_timestamp and now() are identical; clock_timestamp() can already differ within the same statement. Then BEGIN; SELECT now(); SELECT pg_sleep(1); SELECT now(); COMMIT; and observe.

Downloads

SQL file: boolean-and-dates.sql · All lesson files · Setup guide

14 UUID and JSONB

Generate UUIDs and query structured JSON documents with jsonb operators.

Two of PostgreSQL’s most popular non-trivial types: uuid for opaque, globally unique identifiers, and jsonb for binary-encoded JSON that can be indexed and queried. UUIDs are the standard alternative to sequential integer IDs when you need IDs that are unguessable, generatable on the client, or globally unique across systems. jsonb is what makes PostgreSQL competitive with document stores for semi-structured data.

What you’ll learn

  • Generating UUIDs with gen_random_uuid()
  • Storing and querying JSON with jsonb
  • ->, ->>, #>, #>> operators and what they return
  • The @> containment operator (and how it uses GIN indexes)
  • The difference between json and jsonb

Example

Example 14.1 UUID and JSONB
-- UUID generation (built-in from PostgreSQL 13)
SELECT gen_random_uuid() AS id;

-- jsonb literal
SELECT '{"name": "Alice", "age": 30, "tags": ["admin", "beta"]}'::jsonb AS j;

-- -> returns jsonb (preserves type); ->> returns text
SELECT '{"name": "Alice"}'::jsonb -> 'name'  AS as_jsonb,
       '{"name": "Alice"}'::jsonb ->> 'name' AS as_text;

-- Array indexing (0-based)
SELECT '["a", "b", "c"]'::jsonb -> 1 AS second;

-- Path operators: #> returns jsonb, #>> returns text
SELECT '{"user": {"address": {"city": "Paris"}}}'::jsonb #>> '{user, address, city}' AS city;

-- Containment: @> checks if the right side is contained in the left
SELECT '{"name": "Alice", "age": 30}'::jsonb @> '{"name": "Alice"}'::jsonb AS contains_alice;

-- Querying jsonb columns in a real table
DROP TABLE IF EXISTS docs_example;
CREATE TABLE docs_example (id integer PRIMARY KEY, data jsonb);
INSERT INTO docs_example VALUES
  (1, '{"name": "Alice", "tags": ["admin"]}'),
  (2, '{"name": "Bob",   "tags": ["beta"]}');

SELECT id, data ->> 'name' AS name
FROM docs_example
WHERE data @> '{"tags": ["admin"]}';

gen_random_uuid() produces a random version-4 UUID. It is built in from PostgreSQL 13 onward — no pgcrypto extension needed. PostgreSQL 17 has no built-in UUIDv7 generator; use an application/library or extension if you need time-ordered UUIDs. PostgreSQL 18 adds uuidv7(). A bigint identity column is another option.

For JSON, prefer jsonb over json. json stores the text exactly as you give it (including whitespace and key order); jsonb parses, normalizes, and stores in a binary form. jsonb is what you index, what you query efficiently, and what almost every application uses. Reach for plain json only when you need to preserve the input bytes verbatim (rare).

The -> operator extracts and returns jsonb. ->> extracts and returns text (stripping JSON quoting). #> and #>> take a path (an array of keys/indexes). @> (“contains”) tests whether the left value contains the right — it is the foundation of JSON queries.

Run the example

To run:

$ psql -f source/uuid-jsonb.sql postgres
                  id
--------------------------------------
 xxxxxxxx-xxxx-4xxx-xxxx-xxxxxxxxxxxx
(1 row)
 ...
 contains_alice
----------------
 t
(1 row)
 ...

Common pitfalls

  • Comparing data -> 'name' to a string: data -> 'name' = '"Alice"' works (note the JSON-quoted string), but data ->> 'name' = 'Alice' is what you usually want.
  • Forgetting an index: filtering on a jsonb column without a GIN index does a full table scan. CREATE INDEX ON docs_example USING gin (data); enables @> and similar queries to use the index.
  • Mutating jsonb in place: UPDATE t SET data = data || '{"k": "v"}'::jsonb adds or replaces a top-level key; jsonb_set lets you change nested paths.
  • Schema drift inside jsonb: it is flexible, but you still need to know what is in there. Document the expected shape, and validate at the application layer or with a CHECK constraint.
Tip

Index just the parts of JSON you query. Instead of indexing the whole data column with GIN, index a single path: CREATE INDEX ON docs_example ((data ->> 'name'));. This is a B-tree expression index — small, fast, and exact.

Try it

Use ->> to get the name as text: SELECT '{"name": "Alice"}'::jsonb ->> 'name';. Insert a row with nested data and query a nested field with #>>. Then add a GIN index on data and use @> for containment lookups.

Downloads

SQL file: uuid-jsonb.sql · All lesson files · Setup guide

15 Arrays and ENUM

Store arrays, query their elements, and define ordered sets of ENUM values.

Two more PostgreSQL-specific types worth knowing early. Arrays let any column hold an ordered list of values of the same type — sometimes the right tool when a one-to-many relationship is so tightly scoped that a separate table is overkill (tags on a post, role names on a user). ENUM types give a column a closed set of named values, enforced by the database, and stored compactly. Both are non-standard but well-supported, and used widely in real-world Postgres apps.

What you’ll learn

  • Declaring and querying array columns
  • Array literals, indexing (1-based), and useful functions
  • The ANY / ALL operators on arrays
  • Defining and using ENUM types
  • When arrays/enums fit, and when a separate table is better

Example

Example 15.1 Arrays and ENUM
-- Arrays: any type can be turned into an array by appending []
DROP TABLE IF EXISTS posts_example;
CREATE TABLE posts_example (
  id   integer PRIMARY KEY,
  title text,
  tags  text[]                  -- array of text
);

INSERT INTO posts_example VALUES
  (1, 'Hello', ARRAY['intro', 'meta']),
  (2, 'SQL tips', '{tips, postgres}'),   -- shorthand literal
  (3, 'Untagged', ARRAY[]::text[]);

-- Indexing (1-based!) and length
SELECT id, title, tags[1] AS first_tag, array_length(tags, 1) AS tag_count
FROM posts_example
ORDER BY id;

-- ANY: row matches if ANY element of the array equals 'tips'
SELECT id, title FROM posts_example WHERE 'tips' = ANY(tags);

-- Array containment: @> "left contains all elements of right"
SELECT id, title FROM posts_example WHERE tags @> ARRAY['postgres'];

-- ENUM: defined type with a fixed set of allowed values
DROP TABLE IF EXISTS tickets_example;
DROP TYPE  IF EXISTS ticket_status;
CREATE TYPE ticket_status AS ENUM ('open', 'in_progress', 'closed');

CREATE TABLE tickets_example (
  id     integer PRIMARY KEY,
  title  text,
  status ticket_status NOT NULL DEFAULT 'open'
);
INSERT INTO tickets_example (id, title, status) VALUES (1, 'Bug 1', 'open'), (2, 'Bug 2', 'closed');

SELECT * FROM tickets_example ORDER BY id;

Arrays in PostgreSQL are first-class: you can index them, use them in WHERE clauses, build them with ARRAY[], expand them to rows with unnest(), and query for containment with @>, <@, && (overlap). They are stored compactly inline (or in TOAST for very large ones).

The most important quirk: array indexes default to 1-based. tags[1] is the first element of the arrays here, though PostgreSQL also supports explicitly chosen lower bounds. The literal form '{a, b, c}' is a shorthand string-style literal; ARRAY['a', 'b', 'c'] is the constructor — both produce the same value.

ENUM types are excellent when you have a known small set of allowed values that should be enforced by the database ('open' | 'in_progress' | 'closed', 'small' | 'medium' | 'large'). They store as 4 bytes regardless of label length, and sort by declaration order rather than alphabetically — useful for status workflows where 'open' < 'in_progress' < 'closed' is the natural order.

Run the example

To run:

$ psql -f source/arrays-and-enum.sql postgres
DROP TABLE
CREATE TABLE
INSERT 0 3
 id |  title   | first_tag | tag_count
----+----------+-----------+-----------
  1 | Hello    | intro     |         2
  2 | SQL tips | tips      |         2
  3 | Untagged |           |
(3 rows)
 ...

Common pitfalls

  • 1-based indexing trips up developers from every other language. tags[0] is NULL, not an error.
  • Modifying an ENUM (adding a value) requires ALTER TYPE; removing a value is essentially impossible without recreating the type. For sets that change often, prefer a small lookup table joined by foreign key.
  • Arrays look enticing for “I just need a list” — but if you ever need to query for items, join, sort, or constrain individually, a separate table with foreign keys is almost always cleaner. Use arrays when the elements are part of the row, not entities of their own.
  • tag = 'x' does not work on an array column — use 'x' = ANY(tag) or tag @> ARRAY['x'].
Tip

Index arrays with GIN: CREATE INDEX ON posts_example USING gin (tags); enables fast @>, <@, and && queries on the tags column. Without it, those queries scan the whole table.

Try it

Add a row with tags = ARRAY['intro','postgres','tips'] and run SELECT id, tag FROM posts_example, unnest(tags) AS tag; to expand each tag into its own row. Add a new ENUM value with ALTER TYPE ticket_status ADD VALUE 'reopened'; and insert a ticket with that status.

Downloads

SQL file: arrays-and-enum.sql · All lesson files · Setup guide

16 CREATE TABLE

Create a table, inspect its columns, and understand temporary and unlogged tables.

Tables are how PostgreSQL stores your data. CREATE TABLE defines a new table by listing its columns and their data types. Defaults, constraints, indexes, and foreign keys can all be added at create time, but a minimal table needs only a name and a column list. This lesson keeps things small; constraints get their own lessons.

What you’ll learn

  • Defining a table with CREATE TABLE
  • Picking a data type for each column (integer, text, etc.)
  • Making the create script idempotent with DROP TABLE IF EXISTS
  • Inspecting tables in the public schema
  • The difference between TEMP TABLE, regular tables, and UNLOGGED TABLE

Example

Example 16.1 CREATE TABLE
-- Drop the table if it exists so the script is idempotent
DROP TABLE IF EXISTS fruits;

-- A minimal table: id and name
CREATE TABLE fruits (
  id   integer,
  name text
);

-- Confirm it exists in the current schema
SELECT schemaname, tablename
FROM pg_tables
WHERE tablename = 'fruits';

-- See the columns and their types
SELECT column_name, data_type, is_nullable
FROM information_schema.columns
WHERE table_name = 'fruits'
ORDER BY ordinal_position;

fruits has two columns: id (a 32-bit signed integer) and name (variable-length text). The order of columns in CREATE TABLE is the physical order rows are stored in — for storage efficiency on wide tables, group fixed-width columns together. For small tables it doesn’t matter.

DROP TABLE IF EXISTS removes the table only if it exists; without IF EXISTS, the statement would error on a fresh database. Combined with CREATE TABLE, you get a script you can rerun safely while you iterate on the schema. In production, of course, do not blindly drop tables — use migrations.

There are several variants of CREATE TABLE worth knowing about:

  • CREATE TEMP TABLE (or CREATE TEMPORARY TABLE) creates a table that exists only for the current session and is dropped automatically.
  • CREATE UNLOGGED TABLE skips the write-ahead log — faster writes, but the contents are wiped if the server crashes. Useful for derived/staging data you can rebuild.
  • CREATE TABLE ... (LIKE other_table INCLUDING ALL) clones the schema of another table.
  • CREATE TABLE ... AS SELECT ... creates and populates a table from a query (called a CTAS).

Run the example

To run:

$ psql -f source/create-table.sql postgres
DROP TABLE
CREATE TABLE
 schemaname | tablename
------------+-----------
 public     | fruits
(1 row)

 column_name | data_type | is_nullable
-------------+-----------+-------------
 id          | integer   | YES
 name        | text      | YES
(2 rows)

Common pitfalls

  • Picking varchar(255) because that’s what you learned in MySQL: in PostgreSQL, text and varchar(n) have the same performance. Use text unless you specifically want a length cap.
  • Forgetting NOT NULL on columns that should never be null — you can add it later, but it is easier to set up correctly from the start.
  • Naming a column user, order, select — these are reserved words. Either rename (user_id, placed_at) or quote ("user"), but the second is friction forever.
Tip

Standard column types worth memorizing: integer (32-bit), bigint (64-bit), text, boolean, date, timestamp with time zone (almost always preferred over timestamp), numeric(p,s) for exact decimals, and uuid for opaque IDs. Each gets its own lesson.

Try it

Add a third column, e.g. price numeric(10,2), and run the script again. Then change integer to bigint and rerun. Inspect with \d fruits in an interactive psql session.

Downloads

SQL file: create-table.sql · All lesson files · Setup guide

17 Column Types and Constraints

Combine data types, defaults, and constraints in a practical table definition.

A real table picks a type for every column and often adds constraints — rules the database enforces on every row. The most common constraints are NOT NULL (the column must have a value), DEFAULT (use this value if none is given), CHECK (a boolean expression that must be true), UNIQUE, PRIMARY KEY, and REFERENCES (foreign key). Each gets its own lesson; here we combine several into one realistic table.

What you’ll learn

  • Combining multiple types and constraints in one CREATE TABLE
  • Inline DEFAULT and NOT NULL
  • CHECK constraints for domain rules
  • Using timestamptz with current_timestamp as a default
  • Inspecting constraints in the catalog

Example

Example 17.1 Column Types and Constraints
DROP TABLE IF EXISTS products;

CREATE TABLE products (
  id          integer PRIMARY KEY,
  name        text NOT NULL,
  price       numeric(10, 2) NOT NULL CHECK (price >= 0),
  in_stock    boolean NOT NULL DEFAULT true,
  category    text DEFAULT 'misc',
  created_at  timestamptz NOT NULL DEFAULT current_timestamp,
  CONSTRAINT name_not_blank CHECK (length(trim(name)) > 0)
);

INSERT INTO products (id, name, price)              VALUES (1, 'Widget',    19.99);
INSERT INTO products (id, name, price, in_stock)    VALUES (2, 'Gadget',    49.99, false);
INSERT INTO products (id, name, price, category)    VALUES (3, 'Sprocket',  4.95, 'parts');

SELECT * FROM products ORDER BY id;

-- Inspect the constraints on the table
SELECT conname, pg_get_constraintdef(oid) AS def
FROM pg_constraint
WHERE conrelid = 'products'::regclass
ORDER BY conname;

Each constraint is one promise the database makes about your data. NOT NULL says the value is always present. DEFAULT supplies a value when an INSERT omits the column. CHECK (expr) rejects any row where expr is false (NULLs pass). PRIMARY KEY implies NOT NULL and UNIQUE, and creates an index.

Constraints can be inline (after the column type, as above) or table-level (after all the columns). Table-level syntax is required for composite PRIMARY KEY, UNIQUE, and foreign-key constraints. A CHECK can reference several columns in either position, and both column and table constraints can be named with CONSTRAINT name for clearer errors.

timestamptz NOT NULL DEFAULT current_timestamp is one of the most common column patterns — the row’s creation time, stored in UTC, never null, set automatically.

Run the example

To run:

$ psql -f source/column-types-and-constraints.sql postgres
DROP TABLE
CREATE TABLE
INSERT 0 1
INSERT 0 1
INSERT 0 1
 id |   name   | price | in_stock | category |          created_at
----+----------+-------+----------+----------+-------------------------------
  1 | Widget   | 19.99 | t        | misc     | 2024-xx-xx xx:xx:xx.xxxxxx+00
  2 | Gadget   | 49.99 | f        | misc     | 2024-xx-xx xx:xx:xx.xxxxxx+00
  3 | Sprocket |  4.95 | t        | parts    | 2024-xx-xx xx:xx:xx.xxxxxx+00
(3 rows)
 ...

Common pitfalls

  • Forgetting NOT NULL on columns that should never be null — adding it later requires backfilling and may take a long lock on a large table.
  • CHECK accepts a row when the expression is NULL. To reject NULLs, combine NOT NULL and CHECK.
  • Using now() as a DEFAULT: it works, but inside a long transaction every default is the transaction-start time. Usually that is fine; for true per-row wall-clock use clock_timestamp().
  • Adding a DEFAULT and NOT NULL to a new column on a huge table used to rewrite the entire table; modern PostgreSQL (11+) avoids that rewrite when the default is a constant, but volatile defaults still rewrite.
Tip

Name your constraints. The default constraint names PostgreSQL generates are functional but cryptic. A name like products_price_nonneg is much more helpful when it appears in an error message in production.

Try it

Try INSERT INTO products (id, name, price) VALUES (4, ' ', 5); and observe the name_not_blank check rejection. Try INSERT INTO products (id, name, price) VALUES (5, 'Negative', -1); to see the price check trigger. Then add a constraint after the fact: ALTER TABLE products ADD CONSTRAINT category_valid CHECK (category IN ('misc','parts','tools'));.

Downloads

SQL file: column-types-and-constraints.sql · All lesson files · Setup guide

18 ALTER TABLE and DROP

Change tables and constraints while understanding rewrite and locking costs.

Tables evolve. ALTER TABLE changes an existing table: add or drop a column, rename a column or the table itself, add or drop a constraint, change a column’s type, change its default. DROP TABLE removes the table and every row it contains. Schema changes in PostgreSQL are transactional — a failed ALTER TABLE rolls back cleanly, and you can wrap several ALTERs in a BEGIN ... COMMIT to apply them atomically.

What you’ll learn

  • Adding, renaming, and dropping columns
  • Adding and dropping constraints
  • Changing a column’s type with USING
  • Renaming a table
  • The locking and rewrite implications of common alterations

Example

Example 18.1 ALTER TABLE and DROP
-- ADD COLUMN: new columns appear at the end
ALTER TABLE products ADD COLUMN description text;

-- ADD COLUMN with a default (cheap in PG 11+ for constant defaults)
ALTER TABLE products ADD COLUMN sku text NOT NULL DEFAULT '';

-- Rename a column
ALTER TABLE products RENAME COLUMN description TO short_desc;

-- Change a column type (use USING when a cast is needed)
ALTER TABLE products ALTER COLUMN sku TYPE varchar(40);

-- Add a constraint after the fact
ALTER TABLE products ADD CONSTRAINT sku_format CHECK (sku ~ '^[A-Z0-9-]*$');

-- Drop a column
ALTER TABLE products DROP COLUMN short_desc;

-- Rename the table itself
ALTER TABLE products RENAME TO products_old;
ALTER TABLE products_old RENAME TO products;

-- DROP TABLE removes the table and all data
-- (commented out so the rest of the lessons can still use products)
-- DROP TABLE products;

SELECT tablename FROM pg_tables WHERE tablename = 'products';

Each ALTER TABLE is a single transactional statement. You can also batch multiple operations: ALTER TABLE t ADD COLUMN a int, ADD COLUMN b text, DROP COLUMN c; — running them in one statement is faster because the table is touched once.

Two operations are worth understanding in terms of cost:

  • Adding a column with no default (or a constant default) is fast — PostgreSQL just records the new column metadata.
  • Many column type changes rewrite the table. ALTER COLUMN x TYPE bigint reads and rewrites every row. On a large table this can take hours and holds an exclusive lock the whole time. Some binary-compatible changes avoid a rewrite. For a rewrite-heavy change, plan a migration such as new column → backfill → switch; pg_repack does not perform arbitrary type conversions.

ALTER TABLE ... ALTER COLUMN col TYPE new_type USING expr lets you specify how to convert the existing value: ALTER COLUMN price TYPE numeric(12,2) USING price::numeric(12,2).

Run the example

To run:

$ psql -f source/alter-table-and-drop.sql postgres
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
ALTER TABLE
 tablename
-----------
 products
(1 row)

Common pitfalls

  • Adding NOT NULL to an existing column requires that the column already has no NULLs and acquires an ACCESS EXCLUSIVE lock long enough to verify. Backfill first, then SET NOT NULL. From PostgreSQL 12, you can ALTER COLUMN x SET NOT NULL faster if a matching CHECK constraint exists.
  • Dropping a column does not reclaim disk space immediately — the column is marked dead. Run VACUUM FULL (heavyweight) or recreate the table to reclaim it.
  • Renaming a column updates catalog-tracked dependencies such as views, but can break application SQL and function bodies stored as text. Review those uses before renaming.
  • ALTER TYPE on an ENUM cannot remove values; adding values is fine.
Tip

For zero-downtime schema changes, the standard pattern is: add the new column → backfill in batches → make the application write to both → switch reads → drop the old column. Tools like Liquibase, Flyway, or pg_repack help; the database does the locking, you do the choreography.

Try it

Add a column back: ALTER TABLE products ADD COLUMN notes text;. Then rename: ALTER TABLE products RENAME COLUMN notes TO remarks;. Then drop: ALTER TABLE products DROP COLUMN remarks;. Inspect with \d products in psql.

Downloads

SQL file: alter-table-and-drop.sql · All lesson files · Setup guide

19 Primary Keys and UNIQUE

Choose primary keys and unique constraints, including how they treat NULL.

A primary key uniquely identifies a row. Every table should have one — it gives you a stable handle into the row from indexes, joins, foreign keys, and application code. A table can have at most one primary key (which may span multiple columns). UNIQUE is the more general “no duplicates here” constraint; you can have several UNIQUEs per table, and they may include nullable columns.

What you’ll learn

  • The difference between PRIMARY KEY and UNIQUE
  • Single-column vs composite (multi-column) primary keys
  • Surrogate keys (a generated id) vs natural keys (e.g. an email)
  • How NULL is treated by UNIQUE (each NULL is distinct, unless NULLS NOT DISTINCT)
  • The index that a PK / UNIQUE automatically creates

Example

Example 19.1 Primary Keys and UNIQUE
DROP TABLE IF EXISTS users_example;

-- Inline primary key + unique
CREATE TABLE users_example (
  id    integer PRIMARY KEY,
  email text UNIQUE NOT NULL,
  name  text NOT NULL
);

INSERT INTO users_example (id, email, name) VALUES
  (1, 'alice@example.com', 'Alice'),
  (2, 'bob@example.com',   'Bob');

SELECT * FROM users_example ORDER BY id;

-- Composite primary key (across two columns)
DROP TABLE IF EXISTS enrollments_example;
CREATE TABLE enrollments_example (
  student_id integer,
  course_id  integer,
  enrolled_at timestamptz NOT NULL DEFAULT current_timestamp,
  PRIMARY KEY (student_id, course_id)
);

INSERT INTO enrollments_example (student_id, course_id) VALUES (1, 100), (1, 200), (2, 100);
SELECT * FROM enrollments_example ORDER BY student_id, course_id;

-- See indexes that the constraints created
SELECT indexname, indexdef
FROM pg_indexes
WHERE tablename IN ('users_example', 'enrollments_example')
ORDER BY indexname;

A primary key combines NOT NULL and UNIQUE and creates a B-tree index. PostgreSQL uses the primary key index for joins, for WHERE id = ? lookups, and as the natural row identifier in foreign keys. Every well-designed table has one.

UNIQUE is the more flexible cousin. You can have several uniques per table — UNIQUE (email), UNIQUE (handle). A UNIQUE constraint allows NULL values; under the standard, NULL is distinct from every other NULL, so multiple rows with NULL in the same unique column are allowed. From PostgreSQL 15, you can override this with UNIQUE NULLS NOT DISTINCT if you want NULL to count as a duplicate.

Composite keys (PRIMARY KEY (a, b)) are common for junction tables representing many-to-many relationships (a student is enrolled in many courses; a course has many students; the pair is unique).

Run the example

To run:

$ psql -f source/primary-keys-and-unique.sql postgres
DROP TABLE
CREATE TABLE
INSERT 0 2
 id |       email       | name
----+-------------------+-------
  1 | alice@example.com | Alice
  2 | bob@example.com   | Bob
(2 rows)
 ...

Common pitfalls

  • A UNIQUE column with NULLs allowed: two NULLs are not a duplicate by default. If your business rule says “the column is optional but if set, must be unique,” that is the default — but if your rule says “treat NULL as equal,” use UNIQUE NULLS NOT DISTINCT (PG 15+). A partial unique index with WHERE col IS NOT NULL still allows multiple NULLs and is not a substitute.
  • Choosing a natural key like email as the primary key seems clean — until the user changes their email. Surrogate keys (id + UNIQUE (email)) are usually safer.
  • Composite primary keys propagate into foreign keys; a child table that references a composite PK must duplicate both columns. Often a simpler design is to give the parent a surrogate id.
  • PRIMARY KEY and UNIQUE each create an index automatically; do not create another B-tree index on the same column.
Tip

Use bigint (or id bigint GENERATED ALWAYS AS IDENTITY) for the PK on any table you expect to grow past a few million rows. Going from integer to bigint later is painful — change the type up front when in doubt.

Try it

Try inserting a duplicate: INSERT INTO users_example (id, email, name) VALUES (1, 'x@example.com', 'X'); — the PK rejects it. Then INSERT ... VALUES (3, 'alice@example.com', 'Alice2'); — the UNIQUE on email rejects it. Then run \d users_example in psql to see the indexes and constraints.

Downloads

SQL file: primary-keys-and-unique.sql · All lesson files · Setup guide

20 Foreign Keys and REFERENCES

Connect tables with foreign keys and choose the appropriate update and delete actions.

A foreign key is a constraint that says “the value in this column must already exist as a primary key (or unique key) in some other table.” It is how relational databases enforce referential integrity — you cannot orphan a child row by pointing it at a non-existent parent, and (depending on the rules you choose) the database can cascade or restrict updates to the parent. Every relational model that involves multiple tables needs foreign keys.

What you’ll learn

  • Declaring a foreign key with REFERENCES
  • The ON DELETE and ON UPDATE actions: NO ACTION, RESTRICT, CASCADE, SET NULL, SET DEFAULT
  • Why an index on the FK column is almost always a good idea
  • How DEFERRABLE constraints let you batch changes inside a transaction
  • Inspecting foreign keys in the catalog

Example

Example 20.1 Foreign Keys and REFERENCES
-- Parent table
DROP TABLE IF EXISTS comments_example;
DROP TABLE IF EXISTS authors_example;

CREATE TABLE authors_example (
  id   integer PRIMARY KEY,
  name text NOT NULL
);

-- Child table referencing parent
CREATE TABLE comments_example (
  id        integer PRIMARY KEY,
  author_id integer NOT NULL REFERENCES authors_example(id) ON DELETE CASCADE,
  body      text NOT NULL,
  parent_id integer REFERENCES comments_example(id) ON DELETE SET NULL  -- self-reference!
);

-- Index the FK column so joins and cascading deletes don't seq-scan
CREATE INDEX ON comments_example (author_id);
CREATE INDEX ON comments_example (parent_id);

INSERT INTO authors_example VALUES (1, 'Alice'), (2, 'Bob');
INSERT INTO comments_example (id, author_id, body) VALUES
  (1, 1, 'First!'),
  (2, 2, 'Reply'),
  (3, 1, 'Nice');
UPDATE comments_example SET parent_id = 1 WHERE id = 2;

-- Refused: author 99 does not exist
-- INSERT INTO comments_example (id, author_id, body) VALUES (4, 99, 'orphan');

-- Cascade: deleting Alice removes her comments (and Bob's reply loses its parent_id)
DELETE FROM authors_example WHERE id = 1;
SELECT * FROM comments_example ORDER BY id;

-- Inspect foreign keys
SELECT conname, pg_get_constraintdef(oid) AS def
FROM pg_constraint
WHERE contype = 'f' AND conrelid = 'comments_example'::regclass;

REFERENCES other_table(column) declares the foreign key. The referenced column must be the primary key or a unique constraint of the other table. If you omit the column name, the referenced primary key is assumed: REFERENCES authors_example is equivalent to REFERENCES authors_example(id) here.

The ON DELETE and ON UPDATE actions decide what happens when the parent row’s key changes:

  • NO ACTION (default): error if any child still references the parent. Checked at the end of the statement.
  • RESTRICT: like NO ACTION but checked immediately (cannot be deferred).
  • CASCADE: delete or update the child rows too. Useful when the child rows are owned by the parent.
  • SET NULL: set the child column to NULL (the column must be nullable).
  • SET DEFAULT: set the child column to its default.

Foreign keys are checked at every INSERT/UPDATE/DELETE — they are not free. They are also one of the most important correctness features in your schema; the cost almost always pays for itself in avoided data corruption.

Run the example

To run:

$ psql -f source/foreign-keys.sql postgres
DROP TABLE
DROP TABLE
CREATE TABLE
CREATE TABLE
CREATE INDEX
CREATE INDEX
INSERT 0 2
INSERT 0 3
UPDATE 1
DELETE 1
 id | author_id | body  | parent_id
----+-----------+-------+-----------
  2 |         2 | Reply |
(1 row)
 ...

Common pitfalls

  • Forgetting an index on the FK column: when you delete a parent row, PostgreSQL must check (and possibly cascade to) child rows; without an index on the FK column this is a sequential scan. Always index the FK column.
  • Choosing CASCADE too eagerly: a deletion in production can cascade across millions of rows. Audit your ON DELETE CASCADEs, especially on user-facing tables.
  • Importing data with cross-table dependencies: load the parent first, then the child. Or use DEFERRABLE INITIALLY DEFERRED so the FK check happens at commit instead of per-row.
  • Cycles between FKs require deferrable constraints. PostgreSQL supports them, but design around the need when possible.
Tip

DEFERRABLE INITIALLY DEFERRED makes a constraint check at COMMIT rather than statement end. Combined with SET CONSTRAINTS ALL DEFERRED, this lets you make several inter-related changes inside a transaction without ordering them perfectly.

Try it

Try to insert a comment with a non-existent author: INSERT INTO comments_example (id, author_id, body) VALUES (10, 99, 'orphan'); and observe the FK violation. Then add a second author and a comment, and use DELETE on the author to see ON DELETE CASCADE in action.

Downloads

SQL file: foreign-keys.sql · All lesson files · Setup guide

21 NOT NULL and DEFAULT

Require values with NOT NULL and supply useful defaults for new rows.

NOT NULL and DEFAULT are the two most useful per-column rules. NOT NULL says the column always has a value — every INSERT and UPDATE must provide one (or rely on a default). DEFAULT expr provides the value when the column is omitted. Together, they make schemas easier to reason about: you never have to write IF column IS NULL in application code if the column is NOT NULL.

What you’ll learn

  • Enforcing required values with NOT NULL
  • Supplying defaults with DEFAULT expr
  • Volatile vs constant defaults and what that means at scale
  • Combining NOT NULL and DEFAULT for “always has a value”
  • Removing or changing defaults later

Example

Example 21.1 NOT NULL and DEFAULT
DROP TABLE IF EXISTS tasks_example;

CREATE TABLE tasks_example (
  id          integer PRIMARY KEY,
  title       text NOT NULL,
  done        boolean NOT NULL DEFAULT false,
  priority    integer NOT NULL DEFAULT 3 CHECK (priority BETWEEN 1 AND 5),
  due_at      timestamptz,                              -- nullable
  created_at  timestamptz NOT NULL DEFAULT current_timestamp
);

-- Defaults fill in done, priority, created_at
INSERT INTO tasks_example (id, title) VALUES (1, 'Learn SQL');

-- Explicit values override the defaults
INSERT INTO tasks_example (id, title, done, priority, due_at)
VALUES (2, 'Ship lesson', true, 1, current_timestamp + interval '1 day');

SELECT * FROM tasks_example ORDER BY id;

-- Remove a default later (or change it)
ALTER TABLE tasks_example ALTER COLUMN priority SET DEFAULT 1;
ALTER TABLE tasks_example ALTER COLUMN priority DROP DEFAULT;
ALTER TABLE tasks_example ALTER COLUMN priority SET DEFAULT 3;

-- Adding NOT NULL to a column requires no NULLs to exist
-- ALTER TABLE tasks_example ALTER COLUMN due_at SET NOT NULL;  -- would fail if any due_at is NULL

NOT NULL is one of the cheapest, highest-impact constraints you can add — it eliminates an entire class of bugs (forgotten null checks, broken aggregates, surprising joins). Mark columns NOT NULL whenever the application logic guarantees a value.

DEFAULT is evaluated at insert time, not at table creation time. DEFAULT current_timestamp runs current_timestamp once per inserted row, returning the transaction start time. DEFAULT gen_random_uuid() generates a fresh UUID for each row.

Defaults that are constants (DEFAULT 0, DEFAULT 'pending') are fast to add to existing tables — PostgreSQL 11+ stores the constant default in metadata without rewriting the table. Nonvolatile defaults, including the STABLE expression current_timestamp, can also use the fast path. Volatile defaults such as random() and clock_timestamp() require a value for each existing row and rewrite the table.

Run the example

To run:

$ psql -f source/not-null-and-default.sql postgres
DROP TABLE
CREATE TABLE
INSERT 0 1
INSERT 0 1
 id |    title    | done | priority | due_at | created_at
----+-------------+------+----------+--------+-------------
  1 | Learn SQL   | f    |        3 |        | 2024-xx-xx
  2 | Ship lesson | t    |        1 | 2024-xx-xx | 2024-xx-xx
(2 rows)
 ...

Common pitfalls

  • Treating '' (empty string) as “missing” — pick one and be consistent. Most teams use NULL for “no value” and reserve '' for “intentionally blank.” Mixing the two leads to half-broken queries forever.
  • Defaults are not automatically reapplied on UPDATE. To “reset to default,” use UPDATE t SET col = DEFAULT WHERE ....
  • Adding NOT NULL to an existing column that may have NULLs fails. Backfill the column first, then SET NOT NULL.
  • INSERT INTO t DEFAULT VALUES is your friend when every column has a default — it lets the database fill in everything.
Tip

A common pattern is created_at timestamptz NOT NULL DEFAULT current_timestamp paired with updated_at timestamptz NOT NULL DEFAULT current_timestamp. The created_at is set on INSERT and never changes; an updated_at needs a trigger (covered later) to bump on every UPDATE.

Try it

Try INSERT INTO tasks_example (id) VALUES (3); — the NOT NULL on title should reject it. Then INSERT INTO tasks_example (id, title, priority) VALUES (3, 'X', 9); to see the CHECK reject an out-of-range priority.

Downloads

SQL file: not-null-and-default.sql · All lesson files · Setup guide

22 Identity Columns and Sequences

Generate identifiers with identity columns and understand shared sequence behavior.

You rarely want to hand-pick primary key values. PostgreSQL gives you two ways to generate them automatically: the modern, SQL-standard GENERATED ... AS IDENTITY syntax, and the older serial / bigserial shortcuts. Both are powered by a sequence — an independent object that hands out incrementing numbers. IDENTITY is the recommended choice for new code; serial is still common in older codebases.

What you’ll learn

  • The GENERATED ALWAYS AS IDENTITY syntax
  • The differences from the older serial shortcut
  • What a sequence is and how to inspect it
  • Why generated IDs can have gaps
  • Resetting a sequence after a bulk import

Example

Example 22.1 Identity Columns and Sequences
-- Modern style: GENERATED AS IDENTITY (recommended)
DROP TABLE IF EXISTS messages_example;
CREATE TABLE messages_example (
  id   bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  body text NOT NULL,
  created_at timestamptz NOT NULL DEFAULT current_timestamp
);

INSERT INTO messages_example (body) VALUES ('first'), ('second'), ('third');
SELECT id, body FROM messages_example ORDER BY id;

-- Legacy serial shortcut (rewrites to a sequence + DEFAULT under the hood)
DROP TABLE IF EXISTS legacy_example;
CREATE TABLE legacy_example (
  id   serial PRIMARY KEY,
  name text NOT NULL
);

INSERT INTO legacy_example (name) VALUES ('a'), ('b');
SELECT id, name FROM legacy_example ORDER BY id;

-- Inspect sequences
SELECT sequencename, last_value, increment_by
FROM pg_sequences
WHERE sequencename IN ('messages_example_id_seq', 'legacy_example_id_seq');

-- Manually grab the next value (and consume it)
SELECT nextval('messages_example_id_seq') AS next_id;
SELECT currval('messages_example_id_seq') AS curr_id;

-- Reset a sequence (e.g. after a bulk import that supplied IDs)
SELECT setval('legacy_example_id_seq',
              (SELECT max(id) FROM legacy_example));

GENERATED ALWAYS AS IDENTITY tells PostgreSQL the column gets its value from the underlying sequence and cannot be overridden by INSERT (unless you use OVERRIDING SYSTEM VALUE). GENERATED BY DEFAULT AS IDENTITY is the more permissive variant — if the INSERT provides a value, it is used; otherwise the sequence supplies one. Use ALWAYS when you want to prevent accidental ID injection from application code; use BY DEFAULT when you want flexibility for migrations and tests.

The older serial (integer + sequence) and bigserial (bigint + sequence) are syntactic sugar for integer NOT NULL DEFAULT nextval('seq'). They still work, but IDENTITY is the SQL standard, plays better with pg_dump, and tracks ownership of the sequence more cleanly.

Sequences are independent of transactions: nextval() does not roll back. That means if a transaction inserts a row and rolls back, the ID is burned — the next row gets the value after it. IDs can have gaps, and your application should never assume they are contiguous.

Run the example

To run:

$ psql -f source/identity-and-sequences.sql postgres
DROP TABLE
CREATE TABLE
INSERT 0 3
 id | body
----+--------
  1 | first
  2 | second
  3 | third
(3 rows)
 ...

Common pitfalls

  • Reading “the last ID I just inserted” using currval() from a different connection — it won’t work. Sequences are shared objects; only the value remembered by currval() is per-session. Use RETURNING id instead.
  • After a bulk import that supplied explicit IDs, the sequence is still at its old value, so the next auto-generated ID collides. Reset it: SELECT setval('t_id_seq', (SELECT max(id) FROM t));.
  • Counting on IDs to be sequential or to reflect insertion order — they don’t, especially under concurrency. Sort by created_at, id if you need order.
  • Using integer (32-bit) for an IDENTITY column on a high-volume table — you can exhaust the range. Use bigint.
Tip

When migrating from MySQL’s AUTO_INCREMENT, the safest direct equivalent is bigint GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY. It accepts explicit IDs (good for migrations) and otherwise auto-generates.

Try it

Run two INSERTs into messages_example, then ROLLBACK one of them inside a transaction (BEGIN; INSERT ...; ROLLBACK;), then INSERT again — you should see a gap in the IDs. Then reset a sequence with setval() and observe the next INSERT resume from there.

Downloads

SQL file: identity-and-sequences.sql · All lesson files · Setup guide

23 Schemas and search_path

Organize database objects into schemas and control name resolution with search_path.

A schema in PostgreSQL is a namespace inside a database — a way to group tables, views, functions, sequences, and types under a name. Every database has at least one (public), and most production deployments use several. Schemas are how you organize a large project, host multiple tenants in one database, and isolate extensions from application code. Once you understand search_path, you understand how PostgreSQL resolves unqualified names.

What you’ll learn

  • Creating and dropping schemas
  • The schema.table qualified-name syntax
  • The search_path and how name resolution works
  • Per-role and per-database search_path defaults
  • Common patterns: an app schema, multi-tenant schemas, separating extensions

Example

Example 23.1 Schemas and search_path
-- Create a schema and a table inside it
CREATE SCHEMA IF NOT EXISTS app;

DROP TABLE IF EXISTS app.events_example;
CREATE TABLE app.events_example (
  id   bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  kind text NOT NULL,
  at   timestamptz NOT NULL DEFAULT current_timestamp
);

INSERT INTO app.events_example (kind) VALUES ('login'), ('logout');

-- Fully qualified vs. unqualified access
SELECT * FROM app.events_example ORDER BY id;

-- Make the schema findable without prefixing every name
SET search_path TO app, public;
SELECT * FROM events_example ORDER BY id;   -- resolved as app.events_example

-- Inspect the current search path
SHOW search_path;

-- List schemas and the tables in each
SELECT schemaname, tablename
FROM pg_tables
WHERE schemaname NOT IN ('pg_catalog', 'information_schema')
ORDER BY schemaname, tablename
LIMIT 10;

-- Reset to default
SET search_path TO public;

When you write SELECT * FROM events, PostgreSQL looks up the name in each schema listed in search_path, in order, and uses the first match. The default search_path is "$user", public — meaning “first look in a schema named after the connecting user, then public.” If a schema with the user’s name exists, it wins.

You can change the search_path per session (SET search_path TO ...), per role (ALTER ROLE alice SET search_path TO ...), or per database (ALTER DATABASE app_db SET search_path TO ...). The last two persist across reconnections.

Schema separation is the standard way to keep extension objects (postgis, pg_trgm) out of public, to give each tenant its own namespace, or to logically group audit.*, app.*, staging.* tables.

Run the example

To run:

$ psql -f source/schemas.sql postgres
CREATE SCHEMA
DROP TABLE
CREATE TABLE
INSERT 0 2
 id |  kind  |              at
----+--------+-------------------------------
  1 | login  | 2024-xx-xx xx:xx:xx.xxxxxx+00
  2 | logout | 2024-xx-xx xx:xx:xx.xxxxxx+00
(2 rows)
 ...
 search_path
-------------
 app, public
(1 row)
 ...

Common pitfalls

  • Forgetting search_path in a function: a SECURITY DEFINER function uses its owner’s privileges but, without a function-level setting, the caller’s search_path. Set a path containing only trusted schemas, with pg_temp last (for example, SET search_path = pg_catalog, app, pg_temp when app is trusted), and qualify sensitive object names.
  • Putting extensions in public: extensions install their objects in whatever schema is current. Many teams CREATE SCHEMA extensions; ALTER ROLE ... SET search_path = "$user", public, extensions; and create extensions there.
  • Dropping a schema with dependencies fails. Use DROP SCHEMA name CASCADE to drop the schema and its dependent objects. It can be rolled back inside a transaction; after commit, recovery requires a backup or recreation.
  • Forgetting that \dt only shows tables in the current search_path schemas. Use \dt *.* to see everything.
Tip

For multi-tenant systems, schema-per-tenant works at small scale (tens to hundreds of tenants) but becomes painful as the catalog grows. Row-level isolation (a tenant_id column with ROW SECURITY policies) is the more common large-scale pattern.

Try it

Run SHOW search_path; then SET search_path TO app, public; and \dt in interactive psql. Create a table in two schemas with the same name and see which one wins. Then drop the schema: DROP SCHEMA app CASCADE; (use only a disposable demo schema).

Downloads

SQL file: schemas.sql · All lesson files · Setup guide

24 INSERT

Insert single rows, batches, and query results while naming target columns explicitly.

INSERT adds rows to a table. The basic shape is INSERT INTO table (columns) VALUES (values). You can insert one row, many rows in a single statement, or the result of a SELECT. Bulk insertion through a single INSERT ... VALUES (...), (...), ... is dramatically faster than thousands of separate statements because each statement otherwise pays a network round trip and a fresh planning cost.

What you’ll learn

  • Single-row and multi-row INSERT
  • Why naming columns explicitly is the right default
  • Inserting from a query (INSERT INTO ... SELECT)
  • Using DEFAULT and omitting columns
  • Status messages: what INSERT 0 5 actually means

Example

Example 24.1 INSERT
-- One row at a time (most explicit, often fine for app code)
INSERT INTO fruits (id, name) VALUES (1, 'apple');
INSERT INTO fruits (id, name) VALUES (2, 'banana');
INSERT INTO fruits (id, name) VALUES (3, 'cherry');

-- Many rows in one statement (much faster for bulk loads)
INSERT INTO fruits (id, name) VALUES
  (4, 'date'),
  (5, 'elderberry');

-- INSERT ... SELECT: copy rows from another query
-- (here a generator, to avoid duplicating fruits we just inserted)
INSERT INTO fruits (id, name)
SELECT n + 100, 'fruit_' || n
FROM generate_series(1, 3) AS n;

-- Confirm
SELECT count(*) AS row_count FROM fruits;

Run this script after create-table.sql — the table must exist. The first three statements each insert one row; the fourth inserts two; the fifth inserts three rows generated by generate_series. The status message INSERT 0 5 from psql means inserted, with old OID 0, 5 rows affected — OIDs are legacy and you can ignore the first number; the second is the count.

Why list the columns? Two reasons. First, if the table later gains a column, your INSERT INTO fruits VALUES (...) (no column list) will break, whereas INSERT INTO fruits (id, name) VALUES (...) keeps working. Second, listing columns makes the intent obvious to anyone reading the code — values are not positional puzzles.

For bulk loading from a file (CSV, etc.), prefer COPY (covered later) — it is significantly faster than INSERT. Application code should batch rows into multi-row INSERT statements or use a driver-level prepared statement.

Run the example

To run:

$ psql -f source/insert.sql postgres
INSERT 0 1
INSERT 0 1
INSERT 0 1
INSERT 0 2
INSERT 0 3
 row_count
-----------
         8
(1 row)

Common pitfalls

  • Re-running the script when rows already exist causes primary key or unique violations (once you add a PK). The remedy depends on intent: drop and recreate the table, use TRUNCATE, or INSERT ... ON CONFLICT to upsert (next lesson).
  • Wrong column order in a positional INSERT quietly corrupts data — the database happily inserts the wrong value into the wrong column if the types are compatible. Always list columns.
  • Inserting timestamps with the wrong timezone is the usual cause of “the time is off by 8 hours” in production. Use timestamptz and let PostgreSQL handle the conversion.
Tip

INSERT INTO t DEFAULT VALUES; inserts a row using only column defaults — useful when every column has a DEFAULT (e.g. an id from a sequence and created_at from now()).

Try it

Add another row with (6, 'fig') and run the file again. (It will fail once id is unique — we will fix that in the upsert lesson.) Then try INSERT INTO fruits (id, name) SELECT n, 'gen_' || n FROM generate_series(200, 205) n; to see a query-driven insert.

Downloads

SQL file: insert.sql · All lesson files · Setup guide

25 UPSERT (ON CONFLICT)

Insert or update atomically with ON CONFLICT, EXCLUDED, and RETURNING.

“Upsert” is the operation that means “insert if it does not exist; otherwise update.” PostgreSQL spells it as INSERT ... ON CONFLICT ... DO UPDATE (or DO NOTHING). It is atomic, race-free, and one of the most-used patterns in production code. Before this clause existed, applications had to do a SELECT-then-INSERT-then-UPDATE dance that was either slow or wrong under concurrency.

What you’ll learn

  • INSERT ... ON CONFLICT DO NOTHING to skip duplicates
  • INSERT ... ON CONFLICT (col) DO UPDATE SET ... to merge new and old values
  • Referencing the proposed values via the EXCLUDED virtual table
  • The conflict target: column, constraint name, or expression
  • RETURNING with ON CONFLICT and when it gives you the row

Example

Example 25.1 UPSERT (ON CONFLICT)
-- Setup a table with a unique key
DROP TABLE IF EXISTS visitors_example;
CREATE TABLE visitors_example (
  email      text PRIMARY KEY,
  name       text NOT NULL,
  visits     integer NOT NULL DEFAULT 1,
  last_seen  timestamptz NOT NULL DEFAULT current_timestamp
);

INSERT INTO visitors_example (email, name) VALUES ('alice@example.com', 'Alice');

-- DO NOTHING: insert if absent, otherwise skip silently
INSERT INTO visitors_example (email, name) VALUES ('alice@example.com', 'Alice2')
ON CONFLICT (email) DO NOTHING;

-- DO UPDATE: on conflict, merge new values into the existing row
INSERT INTO visitors_example (email, name) VALUES ('alice@example.com', 'Alice')
ON CONFLICT (email) DO UPDATE
  SET visits    = visitors_example.visits + 1,
      last_seen = current_timestamp;

-- EXCLUDED refers to the row that was *proposed* by the INSERT
INSERT INTO visitors_example (email, name) VALUES ('bob@example.com', 'Bob')
ON CONFLICT (email) DO UPDATE
  SET name = EXCLUDED.name,             -- use the new name
      visits = visitors_example.visits + 1
RETURNING *;

-- Conflict on a partial / expression unique index works too
-- CREATE UNIQUE INDEX ON visitors_example (lower(email));
-- INSERT ... ON CONFLICT ((lower(email))) DO UPDATE ...

SELECT * FROM visitors_example ORDER BY email;

The conflict target — the parenthesized clause after ON CONFLICT — tells PostgreSQL which unique constraint or index defines the conflict. (email) matches the unique key on email. You can also name a constraint: ON CONFLICT ON CONSTRAINT visitors_example_pkey DO UPDATE .... For expression indexes, repeat the expression: ON CONFLICT ((lower(email))). A partial index can also require a matching predicate after the conflict target.

EXCLUDED is the virtual table representing the row your INSERT proposed. visitors_example (or whatever the actual table is called) refers to the existing row. Combining them lets you express increments (SET visits = visitors_example.visits + 1), merges (SET name = COALESCE(EXCLUDED.name, visitors_example.name)), and so on.

RETURNING with ON CONFLICT DO UPDATE returns every affected row (inserted or updated). DO NOTHING only returns truly inserted rows — if a conflict was skipped, you get nothing back.

Run the example

To run:

$ psql -f source/upsert.sql postgres
DROP TABLE
CREATE TABLE
INSERT 0 1
INSERT 0 0       -- DO NOTHING skipped the duplicate
INSERT 0 1       -- DO UPDATE merged
INSERT 0 1
       email       | name  | visits |           last_seen
-------------------+-------+--------+-------------------------------
 bob@example.com   | Bob   |      1 | 2024-xx-xx xx:xx:xx.xxxxxx+00
(1 row)

       email       | name  | visits |           last_seen
-------------------+-------+--------+-------------------------------
 alice@example.com | Alice |      2 | 2024-xx-xx xx:xx:xx.xxxxxx+00
 bob@example.com   | Bob   |      1 | 2024-xx-xx xx:xx:xx.xxxxxx+00
(2 rows)

Common pitfalls

  • ON CONFLICT requires a matching unique constraint or index to be defined. Without one, PostgreSQL has no way to know which row would conflict and raises an error.
  • The conflict target must exactly match a constraint or index. ON CONFLICT (lower(email)) only works if you have a unique expression index on lower(email).
  • DO NOTHING is fine when you really mean “best-effort insert.” It silently swallows real bugs if you actually expected the row to be inserted.
  • RETURNING after DO NOTHING does not return a row for the skipped insert. If you need “the row, whether you inserted it or it already existed,” use DO UPDATE with a no-op SET like SET email = EXCLUDED.email so a row is always returned.
Tip

From PostgreSQL 15 onwards, the SQL-standard MERGE statement is also available — more flexible (multiple matched / not-matched branches) but more verbose. INSERT ... ON CONFLICT is still the right tool for simple upserts.

Try it

Run the second insert (DO NOTHING) twice — Alice’s visit count should not increase. Then run the third (DO UPDATE) twice — it should each time. Add RETURNING * to each and observe which return rows.

Downloads

SQL file: upsert.sql · All lesson files · Setup guide

26 UPDATE

Update selected rows, join a data source, and return the values you changed.

UPDATE changes existing rows. The basic shape is UPDATE table SET column = value WHERE condition. Without a WHERE clause, every row in the table is updated — a mistake you only want to make in your own scratch database. Always check with a SELECT first, and consider wrapping risky updates in a BEGIN ... ROLLBACK until you are sure.

What you’ll learn

  • Updating one column, multiple columns, and many rows at once
  • Using WHERE to scope an update
  • UPDATE ... FROM to update one table using data from another
  • Returning the modified rows with RETURNING
  • The transactional safety of UPDATE (and how to abuse it)

Example

Example 26.1 UPDATE
-- Single row, single column
UPDATE fruits SET name = 'apricot' WHERE id = 1;

-- Update another row
UPDATE fruits SET name = 'blueberry' WHERE id = 2;

-- Expression-based update: bump every id by 1000
-- (commented out so the rest of the lessons can find the original ids)
-- UPDATE fruits SET id = id + 1000;

-- UPDATE ... FROM: use another table as the source of values
DROP TABLE IF EXISTS price_changes_example;
CREATE TABLE price_changes_example (id integer, new_name text);
INSERT INTO price_changes_example VALUES (3, 'chestnut'), (4, 'damson');

UPDATE fruits AS f
SET    name = pc.new_name
FROM   price_changes_example pc
WHERE  f.id = pc.id;

DROP TABLE price_changes_example;

-- Returning the rows that changed
UPDATE fruits
SET    name = name        -- no-op, just for the demo
WHERE  id = 5
RETURNING id, name;

SELECT * FROM fruits ORDER BY id LIMIT 5;

UPDATE is fully transactional: until you COMMIT, no other session sees your changes (depending on isolation level), and a ROLLBACK undoes them. The same WHERE rules from SELECT apply.

UPDATE ... FROM is the PostgreSQL extension that lets you update rows in one table using data from another. The FROM clause introduces additional tables; the WHERE joins them. This is often clearer (and faster) than a correlated subquery inside the SET list.

The status message UPDATE n from psql tells you how many rows were modified. UPDATE 0 (zero rows) is a common signal that your WHERE did not match — useful to detect typos or stale assumptions in application code.

Run the example

To run (requires fruits from earlier lessons):

$ psql -f source/update.sql postgres
UPDATE 1
UPDATE 1
CREATE TABLE
INSERT 0 2
UPDATE 2
DROP TABLE
 id |    name
----+------------
  5 | elderberry
(1 row)
UPDATE 1
 id |    name
----+------------
  1 | apricot
  2 | blueberry
  3 | chestnut
  4 | damson
  5 | elderberry
(5 rows)

Common pitfalls

  • Forgetting WHERE and updating the entire table — irreversible without a backup or a transaction. When in doubt, BEGIN; first, run the UPDATE, check the row count, COMMIT; or ROLLBACK;.
  • Updating a primary key that is referenced by foreign keys: the ON UPDATE action of the FK decides whether the change propagates, fails, or sets NULL.
  • Updating a column to the value it already has still rewrites the row, takes the lock, and triggers any BEFORE UPDATE trigger. Add WHERE col IS DISTINCT FROM new_value for a NULL-safe comparison if that matters.
  • Long-running UPDATE on a hot table holds row locks; concurrent updates wait. Batch big updates into smaller transactions.
Tip

To safely preview an update, use a CTE: WITH affected AS (SELECT id FROM fruits WHERE ...) SELECT * FROM affected; first. Then swap the outer query for the UPDATE when you are sure.

Try it

Change one row in fruits to a different name. Use UPDATE fruits SET name = name || '!' WHERE id BETWEEN 1 AND 2 RETURNING *; to see which rows changed. Then run BEGIN; UPDATE fruits SET name = 'X'; SELECT * FROM fruits; ROLLBACK; and confirm the rollback discards the change.

Downloads

SQL file: update.sql · All lesson files · Setup guide

27 DELETE

Delete selected rows and understand how DELETE differs from TRUNCATE.

DELETE FROM table removes rows. Like UPDATE, the WHERE clause is the difference between “remove these specific rows” and “wipe the whole table.” Like UPDATE, DELETE is transactional — a ROLLBACK undoes it. DELETE can filter a subset and fires row-level delete triggers. Both commands enforce foreign-key restrictions; TRUNCATE supports its own statement-level triggers.

What you’ll learn

  • Deleting rows with DELETE FROM ... WHERE
  • Why TRUNCATE exists alongside DELETE
  • DELETE ... USING for joining to a source table
  • Returning deleted rows with RETURNING
  • Transactional safety, foreign-key cascades, and triggers

Example

Example 27.1 DELETE
-- Demo on a temp table so we don't drain `fruits`
DROP TABLE IF EXISTS to_delete;
CREATE TABLE to_delete AS SELECT * FROM fruits;

-- Delete a single row
DELETE FROM to_delete WHERE id = 1;

-- Delete by membership
DELETE FROM to_delete WHERE name IN ('chestnut', 'damson');

-- Delete returning the removed rows
DELETE FROM to_delete WHERE id = 5 RETURNING id, name;

-- Delete using another table as a source (DELETE ... USING)
DROP TABLE IF EXISTS keep;
CREATE TABLE keep (id integer);
INSERT INTO keep SELECT id FROM to_delete LIMIT 1;

DELETE FROM to_delete AS d USING keep k WHERE d.id = k.id;

SELECT * FROM to_delete ORDER BY id;

-- Clean up
DROP TABLE to_delete;
DROP TABLE keep;

DELETE marks row versions as deleted and runs applicable triggers and foreign-key checks. Changes to logged tables generate WAL for crash recovery; MVCC preserves the old versions while transactions may still need them.

TRUNCATE table removes all rows without scanning them and is usually much faster for large tables. It is transactional: ROLLBACK restores the data. It takes an ACCESS EXCLUSIVE lock, enforces foreign-key restrictions, and fires ON TRUNCATE statement triggers, never row-level ON DELETE triggers. It is not MVCC-safe for older snapshots; use it deliberately on scratch or staging data.

DELETE ... USING is the analog of UPDATE ... FROM: lets you express “delete rows in t that match rows in s” by joining to s.

Run the example

To run (requires fruits from earlier lessons):

$ psql -f source/delete.sql postgres
DROP TABLE
SELECT 8
DELETE 1
DELETE 2
 id |    name
----+------------
  5 | elderberry
(1 row)

DELETE 1
DROP TABLE
CREATE TABLE
INSERT 0 1
DELETE 1
 id  |   name
-----+-----------
   2 | blueberry
 102 | fruit_2
 103 | fruit_3
(3 rows)

DROP TABLE

Common pitfalls

  • Forgetting WHERE — same warning as UPDATE, even more painful. Always preview with SELECT count(*) FROM t WHERE same_condition; first.
  • DELETE on a table with ON DELETE CASCADE foreign keys can fan out across many child tables. Audit before running.
  • DELETE does not reclaim disk space — the rows are marked dead and reclaimed by VACUUM later. After a huge delete, run VACUUM (VERBOSE) t; to reclaim and ANALYZE to refresh statistics.
  • Soft deletes (a deleted_at column instead of a real DELETE) are common in app design. Pick a strategy and stick to it.
Tip

For very large deletes, batch them: DELETE FROM logs WHERE created_at < now() - interval '30 days' AND id IN (SELECT id FROM logs WHERE created_at < ... LIMIT 10000);. This keeps each transaction small, autovacuum can keep up, and lock holds are short.

Try it

Add a row to a temp table and delete it. Then try BEGIN; DELETE FROM fruits; SELECT count(*) FROM fruits; ROLLBACK; SELECT count(*) FROM fruits; — the rollback should restore the rows. Then try TRUNCATE on a temp table.

Downloads

SQL file: delete.sql · All lesson files · Setup guide

28 RETURNING

Return generated identifiers and modified rows directly from data-changing statements.

RETURNING is a PostgreSQL extension that turns an INSERT, UPDATE, or DELETE into a query: the statement returns the rows it just touched. This eliminates the second round trip you’d otherwise need to fetch the auto-generated id, the updated timestamp, or the values you just deleted. Most professional codebases lean on RETURNING heavily.

What you’ll learn

  • RETURNING with INSERT, UPDATE, DELETE
  • Returning specific columns vs RETURNING *
  • Capturing generated values (identity, defaults) in one call
  • Using RETURNING inside a CTE (WITH ... AS (...)) to chain operations
  • Why it almost always replaces “INSERT then SELECT last_insert_id()”

Example

Example 28.1 RETURNING
-- Setup
DROP TABLE IF EXISTS logs_example;
CREATE TABLE logs_example (
  id   bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  msg  text NOT NULL,
  at   timestamptz NOT NULL DEFAULT current_timestamp
);

-- INSERT ... RETURNING: get the generated id and timestamp in one round trip
INSERT INTO logs_example (msg) VALUES ('first') RETURNING id, msg, at;
INSERT INTO logs_example (msg) VALUES ('second'), ('third') RETURNING id;

-- UPDATE ... RETURNING: see the new values of the modified rows
UPDATE logs_example
SET    msg = msg || ' (updated)'
WHERE  id = 1
RETURNING id, msg;

-- DELETE ... RETURNING: see what was removed
DELETE FROM logs_example
WHERE  id = 3
RETURNING *;

-- Chaining: insert into one table, then route the new ids into another
DROP TABLE IF EXISTS log_archive;
CREATE TABLE log_archive (id bigint PRIMARY KEY, msg text, archived_at timestamptz DEFAULT current_timestamp);

WITH moved AS (
  DELETE FROM logs_example
  WHERE id <= 2
  RETURNING id, msg
)
INSERT INTO log_archive (id, msg)
SELECT id, msg FROM moved
RETURNING id;

RETURNING accepts the same expressions you would put in a SELECT list — column references, expressions, function calls, even *. For an INSERT, the columns reflect the post-insert values, including defaults and generated identities. For an UPDATE, they reflect the new values; in PostgreSQL 17, OLD.col is available in trigger functions, not RETURNING. PostgreSQL 18 adds explicit old/new values in RETURNING. For a DELETE, they reflect the row as it was.

The chaining pattern (WITH del AS (DELETE ... RETURNING) INSERT INTO archive SELECT * FROM del) is one of PostgreSQL’s superpowers. The two statements run as a single, atomic operation in one round trip — much faster and simpler than doing it from application code.

Run the example

To run:

$ psql -f source/returning.sql postgres
DROP TABLE
CREATE TABLE
 id |  msg  |              at
----+-------+-------------------------------
  1 | first | 2024-xx-xx xx:xx:xx.xxxxxx+00
(1 row)
 id
----
  2
  3
(2 rows)

 id |       msg
----+-----------------
  1 | first (updated)
(1 row)
 ...

Common pitfalls

  • Forgetting RETURNING on an INSERT and then issuing a SELECT max(id) FROM t; — that is a race condition. Always use RETURNING id for the new row’s id.
  • Multi-row INSERT ... RETURNING returns rows in the order they were inserted, but only because of how the executor processes them. If you need a guaranteed mapping, include a stable key in the VALUES and the RETURNING.
  • RETURNING inside INSERT ... ON CONFLICT DO NOTHING does not return rows that were skipped due to conflict. Use DO UPDATE (covered in the next lesson) to make sure something is returned.
  • Drivers sometimes expose INSERT ... RETURNING as a query (with results) rather than as a statement (with row count). Check the language API.
Tip

INSERT INTO t (...) VALUES (...) RETURNING id is the canonical pattern for “create a row and tell me its id.” It works with bigint GENERATED ALWAYS AS IDENTITY, serial, UUID defaults, or anything else the database supplies. Use it everywhere.

Try it

Insert three rows with one statement and capture all three ids via RETURNING id. Then run UPDATE logs_example SET msg = msg WHERE id > 0 RETURNING id, msg; to see every row that was touched (a “no-op” update is still a row touch). Then use the CTE chaining pattern to move rows from logs_example to log_archive.

Downloads

SQL file: returning.sql · All lesson files · Setup guide

29 INNER and LEFT JOIN

Join matching rows and preserve unmatched rows with INNER JOIN and LEFT JOIN.

Joins combine rows from two (or more) tables based on a matching condition. They are how relational data is rebuilt into the shape your application needs. INNER JOIN keeps only rows where the condition matches on both sides; LEFT JOIN (also called LEFT OUTER JOIN) keeps every row from the left side and fills in NULL where the right side has no match. Mastering joins is mastering SQL.

What you’ll learn

  • INNER JOIN with ON: keep matched rows only
  • LEFT JOIN: keep everything on the left, NULL on the right when unmatched
  • Table aliases (c, o) for readability
  • Filtering on the JOIN target vs. in the WHERE clause
  • USING (col) as a shorthand for ON a.col = b.col

Example

Example 29.1 INNER and LEFT JOIN
-- Two small tables for joins
DROP TABLE IF EXISTS orders_example;
DROP TABLE IF EXISTS customers_example;

CREATE TABLE customers_example (
  id   integer PRIMARY KEY,
  name text NOT NULL
);

CREATE TABLE orders_example (
  id          integer PRIMARY KEY,
  customer_id integer,                      -- intentionally not a real FK so we can show unmatched rows
  total       numeric(10, 2) NOT NULL
);

INSERT INTO customers_example VALUES (1, 'Alice'), (2, 'Bob'), (3, 'Carol');
INSERT INTO orders_example   VALUES (1, 1, 50), (2, 1, 30), (3, 99, 10);  -- order 3 points at a non-existent customer

-- INNER JOIN: only rows that match on both sides
SELECT c.name, o.id AS order_id, o.total
FROM customers_example c
INNER JOIN orders_example o ON o.customer_id = c.id
ORDER BY o.id;

-- LEFT JOIN: every customer, even those without orders
SELECT c.name, o.id AS order_id, o.total
FROM customers_example c
LEFT JOIN orders_example o ON o.customer_id = c.id
ORDER BY c.id, o.id;

-- Filtering on the JOIN vs in WHERE matters for LEFT JOIN.
-- "Customers and their orders > 40, even if they have none":
SELECT c.name, o.id, o.total
FROM customers_example c
LEFT JOIN orders_example o ON o.customer_id = c.id AND o.total > 40
ORDER BY c.id;

-- vs "customers who have orders > 40" — the WHERE turns the LEFT into an INNER:
SELECT c.name, o.id, o.total
FROM customers_example c
LEFT JOIN orders_example o ON o.customer_id = c.id
WHERE o.total > 40
ORDER BY c.id;

The ON clause is the join condition. It can be any boolean expression, though usually it is parent.id = child.parent_id. Conditions in ON are part of the join itself — for a LEFT JOIN, a row from the left side survives even if the right side has no matching row that satisfies ON. Conditions in WHERE are applied after the join, so a WHERE child.col = 'x' on a LEFT JOIN filters out unmatched (NULL) rows — turning the join effectively into an INNER JOIN.

USING (col) is shorthand when the columns are named the same on both sides: USING (customer_id) is equivalent to ON a.customer_id = b.customer_id, and the resulting row has just one customer_id column instead of two.

Order 3 has customer_id = 99 (no customer with that id). INNER JOIN drops it; LEFT JOIN keeps customers without orders (Bob, Carol). The two together give the full picture.

Run the example

To run:

$ psql -f source/inner-and-left-join.sql postgres
...
 name  | order_id | total
-------+----------+-------
 Alice |        1 | 50.00
 Alice |        2 | 30.00
(2 rows)

 name  | order_id | total
-------+----------+-------
 Alice |        1 | 50.00
 Alice |        2 | 30.00
 Bob   |          |
 Carol |          |
(4 rows)
 ...

Common pitfalls

  • WHERE right_table.col = 'x' on a LEFT JOIN accidentally drops the unmatched rows you wanted to keep. Put the condition in ON if it should be part of the match, in WHERE if you want to filter after the join.
  • Joining on a column without an index: the planner falls back to a hash join or merge join that may scan the entire table. Index your foreign-key columns.
  • Cartesian products: CROSS JOIN or a comma-separated FROM list without a connecting predicate produces m × n rows. A plain INNER JOIN requires ON or USING; PostgreSQL does not warn about a valid cross join.
  • LEFT JOIN ... GROUP BY is the natural shape for “count of children per parent, including parents with zero.” Use count(child.id) (not count(*)) so unmatched rows yield 0.
Tip

Pick short, meaningful aliases. customers_example c and orders_example o reads better than customers_example cs1 and orders_example o1. Many style guides recommend a 1-3 character alias derived from the table name.

Try it

Change INNER JOIN to LEFT JOIN and see Carol appear with NULLs. Try the two filter variants above and confirm only one keeps unmatched customers. Then use USING: SELECT * FROM customers_example c JOIN orders_example o ON o.customer_id = c.id; becomes SELECT * FROM customers_example c JOIN orders_example o USING (id); — wait, the columns are named differently here, so you would need to rename one first. Try with two tables that share a column name.

Downloads

SQL file: inner-and-left-join.sql · All lesson files · Setup guide

30 RIGHT and FULL JOIN

Use outer joins to preserve either side and find unmatched records.

RIGHT JOIN mirrors LEFT JOIN: every row from the right side is kept, even if the left side has no match. FULL OUTER JOIN keeps everything from both sides, filling unmatched columns with NULL. In practice, LEFT JOIN is far more common than RIGHT JOIN — it usually reads more naturally to put the “primary” table first and use LEFT JOIN to bring in supplemental data. FULL JOIN shines when you need to see matches and both kinds of orphans at once (e.g. reconciliation reports).

What you’ll learn

  • RIGHT JOIN: keep everything on the right
  • FULL JOIN: keep everything on both sides
  • Why LEFT JOIN usually wins on readability
  • Detecting unmatched rows: WHERE other.id IS NULL
  • Ordering when both sides may be NULL

Example

Example 30.1 RIGHT and FULL JOIN
-- Uses customers_example and orders_example from the previous lesson
-- (run inner-and-left-join.sql first if you have not)

-- RIGHT JOIN: every order, even those without a matching customer
SELECT c.name, o.id AS order_id, o.total
FROM customers_example c
RIGHT JOIN orders_example o ON o.customer_id = c.id
ORDER BY o.id;

-- FULL OUTER JOIN: every customer and every order
SELECT c.name, o.id AS order_id, o.total
FROM customers_example c
FULL OUTER JOIN orders_example o ON o.customer_id = c.id
ORDER BY c.id NULLS LAST, o.id NULLS LAST;

-- "Orphans" — find unmatched rows on either side
-- Customers with no orders:
SELECT c.id, c.name
FROM customers_example c
LEFT JOIN orders_example o ON o.customer_id = c.id
WHERE o.id IS NULL;

-- Orders with no customer:
SELECT o.id, o.customer_id
FROM customers_example c
RIGHT JOIN orders_example o ON o.customer_id = c.id
WHERE c.id IS NULL;

RIGHT JOIN A ON ... is exactly LEFT JOIN A with the tables swapped — A LEFT JOIN B ON cond is the same as B RIGHT JOIN A ON cond. Most style guides recommend rewriting RIGHT JOIN to LEFT JOIN so the “primary” table appears first in the FROM clause.

FULL OUTER JOIN is the symmetric counterpart: every row from both sides appears, and unmatched columns are NULL. Order 3 (customer_id = 99) appears with NULL name; Bob and Carol appear with NULL order columns. The ORDER BY ... NULLS LAST keeps the result readable.

The classic anti-join pattern — “rows in A that have no match in B” — uses LEFT JOIN ... WHERE B.id IS NULL. PostgreSQL recognizes this idiom and often plans it as an efficient anti-join.

Run the example

To run (requires the tables from the previous lesson):

$ psql -f source/right-and-full-join.sql postgres
 name  | order_id | total
-------+----------+-------
 Alice |        1 | 50.00
 Alice |        2 | 30.00
       |        3 | 10.00
(3 rows)

 name  | order_id | total
-------+----------+-------
 Alice |        1 | 50.00
 Alice |        2 | 30.00
 Bob   |          |
 Carol |          |
       |        3 | 10.00
(5 rows)
 ...

Common pitfalls

  • Writing RIGHT JOIN and then needing to flip the columns mentally. Swap the table order and use LEFT JOIN for readability.
  • FULL JOIN results have NULLs on both sides; an ORDER BY without NULLS LAST/FIRST can scatter them confusingly.
  • WHERE other.id IS NULL after a LEFT JOIN is the anti-join idiom. NOT EXISTS (...) and NOT IN (...) are alternatives — NOT EXISTS is usually the safest with NULLs.
  • Joining multiple tables with FULL JOIN is rarely what you want — the cartesian-like behavior compounds. Stick to simple cases.
Tip

For “show me the symmetric difference of two sets” (rows in one but not the other), FULL OUTER JOIN ... WHERE a.id IS NULL OR b.id IS NULL is the idiom — handy for reconciling two snapshots of the same data.

Try it

Add a WHERE clause to the FULL JOIN to show only the unmatched rows: WHERE c.id IS NULL OR o.id IS NULL. Then rewrite the RIGHT JOIN above as a LEFT JOIN by swapping the table order — convince yourself the result is the same.

Downloads

SQL file: right-and-full-join.sql · All lesson files · Setup guide

31 Self-Join

Query relationships within one table and follow an employee hierarchy.

A self-join is when a table joins to itself. You give it two aliases so PostgreSQL can tell which copy you mean, and use a condition that links rows together. Self-joins show up whenever the data models a relationship between rows of the same kind — employees and their managers, comments and their parent comments, predecessors and successors. For arbitrary-depth hierarchies, the right tool is a recursive CTE (covered in a later lesson); for one level, a self-join is simple and fast.

What you’ll learn

  • Joining a table to itself with two aliases
  • Choosing INNER vs LEFT for self-joins
  • Modeling a one-level hierarchy (employee → manager)
  • A peek at recursive CTEs for arbitrary depth
  • Naming columns for clarity

Example

Example 31.1 Self-Join
DROP TABLE IF EXISTS employees_example;
CREATE TABLE employees_example (
  id         integer PRIMARY KEY,
  name       text NOT NULL,
  manager_id integer REFERENCES employees_example(id)
);

INSERT INTO employees_example VALUES
  (1, 'Alice',  NULL),    -- top of the hierarchy
  (2, 'Bob',    1),       -- reports to Alice
  (3, 'Carol',  1),       -- reports to Alice
  (4, 'Dave',   2),       -- reports to Bob
  (5, 'Eve',    2);       -- reports to Bob

-- One level: employee and their direct manager
SELECT e.name AS employee, m.name AS manager
FROM employees_example e
LEFT JOIN employees_example m ON e.manager_id = m.id
ORDER BY e.id;

-- Count direct reports per manager
SELECT m.name AS manager, count(e.id) AS direct_reports
FROM employees_example m
LEFT JOIN employees_example e ON e.manager_id = m.id
GROUP BY m.id, m.name
ORDER BY m.id;

-- A taste of recursion: full chain to the top (covered fully in the CTE lesson)
WITH RECURSIVE chain AS (
  SELECT id, name, manager_id, 1 AS depth
  FROM employees_example
  WHERE id = 4                                       -- start at Dave
  UNION ALL
  SELECT e.id, e.name, e.manager_id, c.depth + 1
  FROM employees_example e
  JOIN chain c ON e.id = c.manager_id
)
SELECT * FROM chain;

In the join FROM employees_example e LEFT JOIN employees_example m ON e.manager_id = m.id, e is the employee row and m is the manager row. Both refer to the same physical table — PostgreSQL treats the aliases as independent copies. Using LEFT JOIN means employees without a manager (Alice) still appear, with NULL in the manager columns.

The count(e.id) (not count(*)) in the second query is the standard pattern: count(*) would count NULLs from unmatched rows, giving every leaf manager at least 1 direct report incorrectly. count(e.id) only counts non-NULL ids — actual reports.

The recursive CTE at the end is a preview of the CTE lesson. WITH RECURSIVE lets you walk an arbitrary depth — the anchor row (Dave) is the starting point; the recursive step joins back to chain to climb one level at a time.

Run the example

To run:

$ psql -f source/self-join.sql postgres
DROP TABLE
CREATE TABLE
INSERT 0 5
 employee | manager
----------+---------
 Alice    |
 Bob      | Alice
 Carol    | Alice
 Dave     | Bob
 Eve      | Bob
(5 rows)
 ...

Common pitfalls

  • Using INNER JOIN instead of LEFT JOIN and silently losing the top of the hierarchy. The top row has no manager — INNER JOIN discards it.
  • Forgetting that both e and m are the same table — you can WHERE e.id <> m.id to exclude same-row matches if your condition might allow them.
  • Counting * instead of the joined-table id when looking for “groups with no children” — count(child.id) is the correct idiom.
  • For arbitrary depth, repeatedly self-joining (emp → mgr → mgr's mgr → ...) does not scale. Use WITH RECURSIVE instead.
Tip

Symmetric relationships (friend-of) usually use a separate junction table, not a self-join. Asymmetric (manager-of, parent-of) fit nicely as a self-referencing column.

Try it

Add a sixth employee under Carol. Re-run the direct-reports query — Carol should now have one. Then modify the recursive CTE to start at Eve instead of Dave and see how the chain changes.

Downloads

SQL file: self-join.sql · All lesson files · Setup guide

32 UNION, INTERSECT, EXCEPT

Combine query results with UNION, INTERSECT, and EXCEPT, with or without duplicates.

Set operations combine two result sets row-by-row. All three require the queries on either side to return the same number of columns with compatible types. UNION returns rows in either side (duplicates removed). INTERSECT returns rows in both. EXCEPT returns rows in the first that are not in the second. Each has an ALL variant. UNION ALL concatenates both inputs. INTERSECT ALL keeps the smaller occurrence count of each row; EXCEPT ALL subtracts the right count from the left, with a minimum of zero. The latter two still need to match and count rows.

What you’ll learn

  • UNION vs UNION ALL and when each is appropriate
  • INTERSECT and INTERSECT ALL
  • EXCEPT and EXCEPT ALL
  • Combining set operations and how precedence works
  • Performance: when UNION ALL is meaningfully faster

Example

Example 32.1 UNION, INTERSECT, EXCEPT
-- UNION removes duplicates across the combined result
SELECT 1 AS n UNION SELECT 2 UNION SELECT 1;

-- UNION ALL keeps duplicates (cheaper, no sort/hash)
SELECT 1 AS n UNION ALL SELECT 1 UNION ALL SELECT 2;

-- INTERSECT: rows present in both
SELECT id FROM fruits WHERE id <= 3
INTERSECT
SELECT id FROM fruits WHERE id >= 2;

-- EXCEPT: rows in the first query but not the second
SELECT id FROM fruits WHERE id <= 4
EXCEPT
SELECT id FROM fruits WHERE id >= 3;

-- Order of evaluation: INTERSECT binds tighter than UNION / EXCEPT.
-- Use parentheses to be explicit:
(SELECT id FROM fruits WHERE id < 5)
UNION
(SELECT id FROM fruits WHERE id > 100);

-- ORDER BY applies to the combined result; column refs by position or alias
SELECT id AS n FROM fruits WHERE id < 3
UNION
SELECT id     FROM fruits WHERE id > 100
ORDER BY n;

UNION deduplicates by sort or hash on the full row — that work is real and shows up in EXPLAIN. If you know the inputs cannot duplicate (because the predicates are disjoint, for example), prefer UNION ALL. On large queries the difference can be substantial.

INTERSECT and EXCEPT are less common in application code but handy for analysis: “users who clicked A and B,” “products in catalog A that are not in catalog B.”

Set operations are set-of-rows operations. The column types must align across the inputs — int and text cannot be unioned directly. Names come from the first query’s column names.

Run the example

To run (requires fruits):

$ psql -f source/union-intersect-except.sql postgres
 n
---
 1
 2
(2 rows)

 n
---
 1
 1
 2
(3 rows)
 ...

Common pitfalls

  • UNION instead of UNION ALL on queries that cannot produce duplicates wastes CPU on deduplication. Pick UNION ALL unless duplicates are possible and unwanted.
  • Misaligned column counts or types: SELECT id, name FROM t UNION SELECT id FROM t2 is an error. Pad with NULL::text or align columns.
  • ORDER BY after a set operation refers to the combined result; it cannot use table-qualified column names. Use positions (ORDER BY 1) or aliases from the first query.
  • EXCEPT is sensitive to NULL: EXCEPT treats two NULLs as equal (like DISTINCT), so a row with NULL on the left and NULL on the right is removed. This is usually what you want — but worth remembering.
Tip

When you find yourself writing SELECT ... WHERE x = 1 UNION ALL SELECT ... WHERE x = 2 UNION ALL ..., consider whether a single WHERE x IN (1, 2, 3) would do. Set operations are for genuinely different queries with results that need to be combined.

Try it

Write two SELECTs from fruits with different WHERE clauses and combine with UNION. Then change to UNION ALL and observe the duplicate count change (if any). Then try INTERSECT and EXCEPT between them.

Downloads

SQL file: union-intersect-except.sql · All lesson files · Setup guide

33 COUNT, SUM, AVG

Count, sum, average, and collect values while accounting for NULL and empty input.

Aggregate functions compute a single value across many rows. The five most common — count, sum, avg, min, max — are foundational. They behave a little differently with NULL: count(*) counts every row including those with NULLs; count(col) counts non-NULL values of col; sum/avg/min/max ignore NULLs entirely. There are dozens more (stddev, string_agg, array_agg, bool_and, bool_or, jsonb_agg, …), with behavior specific to each aggregate: array_agg and jsonb_agg, for example, retain NULL inputs.

What you’ll learn

  • count(*) vs count(col) vs count(DISTINCT col)
  • sum, avg, min, max on numeric and other types
  • How aggregates handle NULL
  • string_agg and array_agg for building lists
  • FILTER (WHERE ...) for per-aggregate conditions

Example

Example 33.1 COUNT, SUM, AVG
-- Basic counts
SELECT count(*) AS total_rows FROM fruits;
SELECT count(name) AS non_null_names FROM fruits;
SELECT count(DISTINCT name) AS unique_names FROM fruits;

-- Sum and average
SELECT sum(id) AS sum_id, avg(id) AS avg_id, round(avg(id), 2) AS avg_round
FROM fruits;

-- Min and max work on numbers, text, dates
SELECT min(name) AS min_name, max(name) AS max_name FROM fruits;

-- Build a string or array from a column
SELECT string_agg(name, ', ' ORDER BY id) AS names_csv,
       array_agg(name ORDER BY id)         AS names_arr
FROM fruits;

-- FILTER: per-aggregate WHERE; replaces messy CASE WHEN tricks
SELECT count(*) AS total,
       count(*) FILTER (WHERE id <= 3) AS low,
       count(*) FILTER (WHERE id > 3)  AS high,
       sum(id)  FILTER (WHERE id > 3)  AS high_sum
FROM fruits;

count(*) is the row count — fast and clear. count(col) skips NULL values of col and is the right call when you want “how many rows have a non-NULL email?” count(DISTINCT col) deduplicates first; it is slower because it must materialize the distinct set.

sum returns the sum, or NULL if there are no rows (not 0 — surprising, but standard). avg returns NULL on no rows; on integer inputs, avg returns a numeric because the average of integers is rarely an integer. min and max work on anything orderable, including text and timestamps.

string_agg and array_agg are PostgreSQL’s “concatenate the values” aggregates — exactly the right tool when you want to render a list. The optional ORDER BY inside controls the order of concatenation, which the default unordered behavior cannot guarantee.

FILTER (WHERE ...) is the SQL-standard way to apply a per-aggregate condition. It replaces the older sum(CASE WHEN cond THEN x END) idiom — same result, much clearer.

Run the example

To run:

$ psql -f source/count-sum-avg.sql postgres
 total_rows
------------
          8
(1 row)
 ...
   names_csv
-----------------------------------------------------------
 apricot, blueberry, chestnut, damson, elderberry, fruit_1, fruit_2, fruit_3
(1 row)

 total | low | high | high_sum
-------+-----+------+----------
     8 |   3 |    5 |      315
(1 row)

Common pitfalls

  • sum of zero rows returns NULL, not 0. Use COALESCE(sum(x), 0) if you need 0.
  • avg on an integer column produces a numeric result; if you want a rounded display, wrap in round.
  • count(col) skipping NULLs is a feature, but it surprises beginners. count(*) is the row count; everything else is “non-null count.”
  • Aggregates without GROUP BY collapse the entire result to one row. With GROUP BY you get one row per group (next lesson).
Tip

Combine aggregates and FILTER in a single pass to get many statistics at once. SELECT count(*), count(*) FILTER (WHERE is_active), avg(amount) FILTER (WHERE status = 'paid'), max(created_at) FROM ... is one scan, many answers.

Try it

Add a price column to fruits with ALTER TABLE fruits ADD COLUMN price numeric(10,2); (then UPDATE some prices), and run sum, avg, min, max on it. Then try SELECT string_agg(name, ', ' ORDER BY id DESC) FROM fruits; and see the order reverse.

Downloads

SQL file: count-sum-avg.sql · All lesson files · Setup guide

34 GROUP BY

Group rows for aggregation and build subtotals with ROLLUP and GROUPING SETS.

GROUP BY splits rows into groups by the value of one or more expressions and then applies aggregates to each group. Without GROUP BY, an aggregate collapses the whole result into one row. With it, you get one row per distinct group — count per customer, sum per month, avg per category. Every non-aggregated column in the SELECT list must appear in GROUP BY (or be functionally dependent on it).

What you’ll learn

  • Grouping rows by one or more columns
  • Applying aggregates per group
  • The rule that non-aggregated columns must be in GROUP BY
  • Grouping by expressions, not just columns
  • GROUPING SETS, ROLLUP, CUBE for multi-level summaries

Example

Example 34.1 GROUP BY
-- One row per customer with order count and total
SELECT customer_id,
       count(*)             AS order_count,
       sum(total)           AS total_spent,
       avg(total)::numeric(10,2) AS avg_order
FROM orders_example
GROUP BY customer_id
ORDER BY customer_id;

-- Group by an expression (first letter of fruit name)
SELECT substring(name, 1, 1) AS letter, count(*) AS cnt
FROM fruits
GROUP BY substring(name, 1, 1)
ORDER BY letter;

-- Group by multiple columns
SELECT schemaname, count(*) AS table_count
FROM pg_tables
GROUP BY schemaname
ORDER BY table_count DESC
LIMIT 5;

-- GROUPING SETS: several groupings in one query
SELECT customer_id, count(*) AS orders, sum(total) AS spent
FROM orders_example
GROUP BY GROUPING SETS ((customer_id), ())   -- per-customer rows AND a grand total
ORDER BY customer_id NULLS LAST;

-- ROLLUP: hierarchical subtotals
SELECT customer_id, sum(total) AS spent
FROM orders_example
GROUP BY ROLLUP (customer_id);

The rule is straightforward: in a query with GROUP BY, every column in the SELECT list must either be in the GROUP BY clause or be inside an aggregate. PostgreSQL does honor functional dependency — if id is the primary key, you can GROUP BY id and reference name directly because name is determined by id. But you cannot, say, write SELECT customer_id, total FROM orders GROUP BY customer_id; — total varies within the group.

GROUPING SETS and its shortcuts ROLLUP and CUBE produce multiple groupings in one query — the basis of OLAP-style reports. ROLLUP (a, b, c) produces groupings (a, b, c), (a, b), (a), () — successive subtotals up to a grand total. CUBE (a, b) produces every combination — (a, b), (a), (b), ().

Run the example

To run (requires orders_example from the joins lesson):

$ psql -f source/group-by.sql postgres
 customer_id | order_count | total_spent | avg_order
-------------+-------------+-------------+-----------
           1 |           2 |       80.00 |     40.00
          99 |           1 |       10.00 |     10.00
(2 rows)
 ...

Common pitfalls

  • “column ‘x’ must appear in the GROUP BY clause or be used in an aggregate function” — the most common GROUP BY error. Either group by x or wrap it in min(x), max(x), etc.
  • WHERE filters rows before grouping; HAVING filters groups after (next lesson). Mixing them up is a common confusion.
  • GROUP BY on a nullable column treats all NULLs as one group. If you want them out, add WHERE col IS NOT NULL.
  • Grouping by an expression (GROUP BY date_trunc('month', created_at)) is fine, but be aware that PostgreSQL must compute it for every row — index the expression if the query is hot.
Tip

GROUP BY 1, 2 groups by the first and second columns in the SELECT list. It is concise for ad-hoc queries; in committed code, name the columns for clarity.

Try it

Group fruits by the first letter of name: SELECT substring(name, 1, 1) AS letter, count(*) FROM fruits GROUP BY substring(name, 1, 1) ORDER BY letter;. Then try GROUP BY 1. Then add HAVING count(*) > 1 (preview of the next lesson) to see only letters with more than one match.

Downloads

SQL file: group-by.sql · All lesson files · Setup guide

35 HAVING

Filter aggregate groups with HAVING and distinguish group predicates from row filters.

WHERE filters individual rows before grouping; HAVING filters groups after the aggregates have been computed. If you want “customers with more than three orders,” that is a HAVING — you cannot test count(*) > 3 until the count exists. If you want “customers who placed an order in 2024,” that is a WHERE — the year is a per-row property.

What you’ll learn

  • Filtering groups with HAVING
  • The difference between WHERE (per row) and HAVING (per group)
  • Combining WHERE and HAVING in one query
  • Aggregates allowed inside HAVING
  • Why pushing predicates into WHERE is usually faster

Example

Example 35.1 HAVING
-- HAVING with one aggregate condition
SELECT customer_id, count(*) AS order_count, sum(total) AS total_spent
FROM orders_example
GROUP BY customer_id
HAVING sum(total) > 30
ORDER BY customer_id;

-- HAVING with multiple conditions
SELECT customer_id, count(*) AS order_count, sum(total) AS total_spent
FROM orders_example
GROUP BY customer_id
HAVING count(*) >= 1 AND sum(total) > 30;

-- WHERE then HAVING: filter rows first, group, then filter groups
SELECT customer_id, count(*) AS order_count
FROM orders_example
WHERE total > 20                      -- per-row filter
GROUP BY customer_id
HAVING count(*) >= 1;                 -- per-group filter

-- HAVING without aggregates is legal but pointless — same as WHERE
SELECT customer_id
FROM orders_example
GROUP BY customer_id
HAVING customer_id > 50;              -- could just be WHERE

The execution order helps remember the difference: FROM → WHERE → GROUP BY → HAVING → SELECT → ORDER BY. A row predicate written in WHERE makes its intent clear and can reduce the input to grouping; PostgreSQL can also push suitable HAVING predicates down. Push as much filtering as possible into WHERE.

HAVING lets you write aggregate predicates directly: HAVING avg(amount) > 100, HAVING count(DISTINCT product_id) > 5. These cannot appear in WHERE because the aggregates do not exist there.

The last query is technically valid but a smell: filtering on a non-aggregate in HAVING does the same as WHERE but later in the pipeline. Prefer WHERE for clarity; the planner can often push this grouped-column predicate down either way.

Run the example

To run (requires orders_example from the joins lesson):

$ psql -f source/having.sql postgres
 customer_id | order_count | total_spent
-------------+-------------+-------------
           1 |           2 |       80.00
(1 row)
 ...

Common pitfalls

  • Writing WHERE count(*) > 3 — aggregates are not allowed in WHERE. Use HAVING.
  • Writing HAVING amount > 100 is valid only when amount is grouped or functionally dependent on the grouping. Use WHERE amount > 100 to filter individual input rows. The planner usually rewrites this, but readers will be confused.
  • Forgetting that HAVING runs after GROUP BY, so any column referenced must be a GROUP BY column or inside an aggregate.
  • Using HAVING instead of a WHERE in count(*) FILTER (WHERE ...)-style logic. FILTER is per-aggregate and lives in SELECT; HAVING is per-group and decides which groups are kept.
Tip

When you want “the top N groups by some aggregate,” combine GROUP BY + ORDER BY agg DESC + LIMIT N — no HAVING needed. HAVING is for predicates (“at least 3”), not rankings (“top 3”).

Try it

Change the threshold: HAVING sum(total) < 50 and see how the result set shifts. Then move the filter into WHERE: WHERE total > 20 GROUP BY customer_id; and observe that the count drops because fewer rows reach the grouping.

Downloads

SQL file: having.sql · All lesson files · Setup guide

36 Scalar and IN Subqueries

Use scalar and membership subqueries and handle empty results and NULL correctly.

A subquery is a SELECT written inside another query. There are three common flavors. A scalar subquery returns at most one row and exactly one column and can be used wherever a single value is allowed. An IN subquery tests membership: WHERE col IN (subquery). An EXISTS subquery tests whether the subquery returns any rows (next lesson). Subqueries can be correlated (referencing the outer row) or uncorrelated (independent). PostgreSQL is very good at optimizing them — you can lean on them without much worry.

What you’ll learn

  • Scalar subqueries: one row, one column
  • IN (subquery) and how it tests membership
  • NOT IN and the NULL trap
  • ANY / ALL as more general comparators
  • When to use a subquery vs a join

Example

Example 36.1 Scalar and IN Subqueries
-- Scalar subquery: the fruit with the largest id
SELECT id, name
FROM fruits
WHERE id = (SELECT max(id) FROM fruits);

-- Scalar subquery in the SELECT list
SELECT name,
       id,
       (SELECT count(*) FROM fruits) AS total_fruits
FROM fruits
ORDER BY id
LIMIT 3;

-- IN: membership in a set
SELECT *
FROM fruits
WHERE id IN (SELECT id FROM fruits WHERE id <= 2)
ORDER BY id;

-- NOT IN with no NULLs is fine
SELECT id, name
FROM fruits
WHERE id NOT IN (SELECT id FROM fruits WHERE id > 100)
ORDER BY id;

-- ANY/ALL: generalize comparators with a subquery
SELECT id, name
FROM fruits
WHERE id > ALL (SELECT id FROM fruits WHERE id < 4)
ORDER BY id;

-- A correlated subquery references the outer row (here, c.id)
SELECT c.id, c.name,
       (SELECT count(*) FROM orders_example o WHERE o.customer_id = c.id) AS order_count
FROM customers_example c
ORDER BY c.id;

A scalar subquery must return at most one row. Zero rows is treated as NULL; more than one row raises an error. They are most useful when you want a value that depends on the table itself (max, count, latest).

IN (subquery) is equivalent to = ANY (subquery) — true when the outer value equals at least one row of the subquery. The planner usually turns this into a semi-join, which can be very efficient.

ANY and ALL generalize comparison to a set. x > ALL (s) is true when x is greater than every element of s (including the empty set, where it is vacuously true). x > ANY (s) is true when x is greater than at least one element.

Correlated subqueries reference columns from the outer query (e.g. o.customer_id = c.id above). They run once per outer row conceptually, though the planner often optimizes them into joins.

Run the example

To run (requires fruits and the join lesson tables):

$ psql -f source/scalar-and-in-subqueries.sql postgres
 id  |   name
-----+----------
 103 | fruit_3
(1 row)
 ...

Common pitfalls

  • A scalar subquery that returns more than one row raises “more than one row returned by a subquery used as an expression.” Add a LIMIT 1 and a clear ORDER BY if you genuinely want “the first.”
  • NOT IN (subquery) is the famous gotcha: if a NULL is present, nonmatching comparisons produce unknown while matching comparisons produce false; the filter keeps no rows. Use NOT EXISTS for safety.
  • Correlated subqueries inside SELECT lists run once per outer row. If the planner cannot optimize, the cost is O(outer * inner). Re-express as a LEFT JOIN + count() ... GROUP BY when the subquery is expensive.
  • IN on a long, hard-coded list (IN ('a', 'b', ..., thousands)) is fine but inelegant. For dynamic large lists, prefer a join against a temp table or unnest(array).
Tip

Many subqueries can be rewritten as joins or CTEs; the choice is mostly about readability. Use subqueries for single value, joins for multiple rows from another table, and CTEs (next major lesson) when you want to name intermediate results.

Try it

Find fruits whose id equals the average id (cast appropriately): SELECT * FROM fruits WHERE id = (SELECT avg(id)::integer FROM fruits);. Then rewrite the correlated subquery as a LEFT JOIN + GROUP BY and compare results.

Downloads

SQL file: scalar-and-in-subqueries.sql · All lesson files · Setup guide

37 EXISTS and Derived Tables

Test for related rows with EXISTS and name intermediate results with derived tables.

EXISTS (subquery) is true when the subquery returns at least one row. It is the workhorse of “does any related row satisfy X?” queries — usually written as a correlated subquery referencing the outer row. A derived table is a subquery in the FROM clause, given an alias and used like a table. The two together cover most of the SQL you write before you reach CTEs and window functions.

What you’ll learn

  • EXISTS and NOT EXISTS: tests for “any row exists” / “no row exists”
  • Why NOT EXISTS beats NOT IN for nullable subqueries
  • Derived tables: a subquery in FROM with an alias
  • When a derived table is clearer than a CTE
  • The relationship to IN and joins

Example

Example 37.1 EXISTS and Derived Tables
-- EXISTS: customers who have at least one order
SELECT c.id, c.name
FROM customers_example c
WHERE EXISTS (
  SELECT 1 FROM orders_example o WHERE o.customer_id = c.id
)
ORDER BY c.id;

-- NOT EXISTS: customers with no orders (safe even with NULL customer_id values)
SELECT c.id, c.name
FROM customers_example c
WHERE NOT EXISTS (
  SELECT 1 FROM orders_example o WHERE o.customer_id = c.id
)
ORDER BY c.id;

-- Derived table: a subquery used in FROM, given an alias
SELECT sub.customer_id, sub.total_spent
FROM (
  SELECT customer_id, sum(total) AS total_spent
  FROM orders_example
  GROUP BY customer_id
) AS sub
WHERE sub.total_spent > 30
ORDER BY sub.customer_id;

-- Derived table joined back to a base table
SELECT c.name, latest.total
FROM customers_example c
JOIN (
  SELECT customer_id, max(total) AS total
  FROM orders_example
  GROUP BY customer_id
) AS latest ON latest.customer_id = c.id
ORDER BY c.id;

The convention SELECT 1 inside EXISTS is not magic — PostgreSQL does not care what columns the subquery returns; it only checks whether any row would be produced. SELECT 1, SELECT *, SELECT NULL all work the same. SELECT 1 reads as “I do not care about the columns” and is the conventional choice.

A derived table — also called an inline view — is just a subquery in FROM. An explicit alias (after the closing parenthesis) gives the result a name you can refer to. Derived tables compose well; you can join them, filter them, group them, even nest them. The main downside is that a complex derived table buries the logic in the middle of the query. CTEs (next lesson) are usually a nicer way to name an intermediate result.

Run the example

To run (requires customers_example and orders_example):

$ psql -f source/exists-and-derived-tables.sql postgres
 id | name
----+------
  1 | Alice
(1 row)

 id | name
----+------
  2 | Bob
  3 | Carol
(2 rows)
 ...

Common pitfalls

  • IN (SELECT col FROM t) and EXISTS (SELECT 1 FROM t WHERE t.col = outer.col) are usually equivalent — but NOT IN and NOT EXISTS are not when NULLs are possible. Prefer NOT EXISTS whenever NULLs are on the table.
  • An alias such as AS x is useful for clarity and portability. PostgreSQL 16 and later allow subqueries in FROM without one; older PostgreSQL versions require an alias.
  • A derived table with ORDER BY inside is allowed but the order is not preserved when the outer query consumes it — sort outside.
  • Some old guides said EXISTS is “always slower than IN.” On modern PostgreSQL the planner handles both well; choose for correctness and readability first.
Tip

EXISTS short-circuits — once it finds the first matching row, it stops. So WHERE EXISTS (SELECT 1 FROM big_table WHERE ...) can be very fast if the matching row is found early.

Try it

Find fruits that have a name appearing exactly once: SELECT name FROM fruits f WHERE NOT EXISTS (SELECT 1 FROM fruits f2 WHERE f2.name = f.name AND f2.id <> f.id);. Then rewrite a derived table as a CTE in your head (preview of the next lesson): WITH spend AS (SELECT customer_id, sum(total) AS total_spent FROM orders_example GROUP BY customer_id) SELECT * FROM spend WHERE total_spent > 30;.

Downloads

SQL file: exists-and-derived-tables.sql · All lesson files · Setup guide

38 Common Table Expressions (WITH)

Structure queries with WITH, move rows atomically, and traverse trees recursively.

A common table expression (CTE) is a named subquery introduced with WITH name AS (SELECT ...). You can refer to it from the main query (and from later CTEs) as if it were a table. CTEs make complex queries readable: each CTE has a name and a job, and the main query reads top-down like a story. PostgreSQL also supports recursive CTEs (WITH RECURSIVE), which are how you walk trees, graphs, and chains in pure SQL.

What you’ll learn

  • Defining a CTE with WITH name AS (...)
  • Chaining multiple CTEs in one query
  • Data-modifying CTEs: INSERT, UPDATE, DELETE inside WITH
  • Recursive CTEs for hierarchies and graphs
  • The performance trade-offs (MATERIALIZED / NOT MATERIALIZED)

Example

Example 38.1 Common Table Expressions (WITH)
-- A single CTE
WITH spend AS (
  SELECT customer_id, sum(total) AS total_spent
  FROM orders_example
  GROUP BY customer_id
)
SELECT c.name, COALESCE(s.total_spent, 0) AS spent
FROM customers_example c
LEFT JOIN spend s ON s.customer_id = c.id
ORDER BY c.id;

-- Multiple CTEs, each named and easy to read
WITH per_customer AS (
  SELECT customer_id, sum(total) AS total_spent, count(*) AS order_count
  FROM orders_example
  GROUP BY customer_id
),
big_spenders AS (
  SELECT * FROM per_customer WHERE total_spent > 30
)
SELECT c.name, b.total_spent, b.order_count
FROM big_spenders b
JOIN customers_example c ON c.id = b.customer_id
ORDER BY b.total_spent DESC;

-- Data-modifying CTE: archive then return what was archived
DROP TABLE IF EXISTS orders_archive;
CREATE TABLE orders_archive (LIKE orders_example INCLUDING ALL);

WITH archived AS (
  DELETE FROM orders_example
  WHERE total < 20
  RETURNING *
)
INSERT INTO orders_archive
SELECT * FROM archived
RETURNING id;

-- RECURSIVE CTE: walk an employee hierarchy
WITH RECURSIVE reports AS (
  -- anchor: top-level employees (no manager)
  SELECT id, name, manager_id, 1 AS depth
  FROM employees_example
  WHERE manager_id IS NULL
  UNION ALL
  -- recursive step: rows whose manager appears in the previous level
  SELECT e.id, e.name, e.manager_id, r.depth + 1
  FROM employees_example e
  JOIN reports r ON e.manager_id = r.id
)
SELECT depth, name FROM reports ORDER BY depth, id;

A CTE is just a named subquery. You can have many, comma-separated, each able to reference earlier ones. Read top-to-bottom, each step a clear chunk. Compared to a deeply nested set of derived tables, CTEs are usually easier to follow and easier to change.

Data-modifying CTEs are a PostgreSQL specialty: an INSERT, UPDATE, or DELETE with RETURNING can be the body of a CTE, and its returned rows flow into the next statement — atomically, in one round trip. The “delete from one table, insert into another” archive pattern shown above is the canonical example.

Recursive CTEs have two halves combined with UNION ALL: an anchor query that produces the initial rows, and a recursive query that references the CTE by name to add more rows. PostgreSQL keeps iterating until no new rows are produced. Use them for hierarchies (employees → manager), graph traversal (find all transitive dependencies), and series generators (numbers, dates).

Run the example

To run (requires tables from the earlier lessons):

$ psql -f source/cte.sql postgres
 name  | spent
-------+-------
 Alice | 80.00
 Bob   |  0.00
 Carol |  0.00
(3 rows)
 ...
 depth | name
-------+-------
     1 | Alice
     2 | Bob
     2 | Carol
     3 | Dave
     3 | Eve
(5 rows)

Common pitfalls

  • A nonrecursive, side-effect-free SELECT CTE referenced once is, in PostgreSQL 12+, normally folded into its parent query — the planner treats it like a subquery. If you want the older “always materialize” behavior, write WITH x AS MATERIALIZED (...).
  • WITH RECURSIVE accepts both UNION ALL and UNION. UNION eliminates duplicate complete rows, including rows produced earlier; UNION ALL keeps them. Choose based on the recursion and its termination condition.
  • A recursive CTE that does not converge (the recursive step keeps adding rows) is an infinite loop. The query may run indefinitely, spill to disk, or fail; work_mem is not an execution limit. Add a termination condition (WHERE depth < 100 etc.) when in doubt.
  • Data-modifying CTEs inside WITH are evaluated in unspecified order relative to each other — do not rely on side effects between them. The main statement shares the same snapshot and has no guaranteed modification order relative to them. Pass data through RETURNING rather than relying on visible side effects.
Tip

When a query gets longer than ~20 lines or has more than two derived tables, refactoring into named CTEs almost always wins. You can even leave one CTE temporarily and inline another to see where the cost is.

Try it

Write a CTE that computes the average order total, then in the main query show every order with the difference from the average. Then try a recursive CTE that produces the numbers 1 through 10 (WITH RECURSIVE t(n) AS (SELECT 1 UNION ALL SELECT n+1 FROM t WHERE n < 10) SELECT * FROM t;).

Downloads

SQL file: cte.sql · All lesson files · Setup guide

39 Window Functions

Rank rows and compute running totals, neighboring values, and window frames.

Window functions compute a value for each row using a window — a set of related rows defined by an OVER (...) clause. Unlike GROUP BY, which collapses rows, window functions add a per-row result alongside the original columns. They are the right tool for rankings, running totals, moving averages, percentile bands, “previous row” comparisons — anything you used to do in your application code by sorting and iterating.

What you’ll learn

  • The shape of a window function: f() OVER (PARTITION BY ... ORDER BY ...)
  • The most useful functions: row_number, rank, dense_rank, lag, lead, sum, avg, first_value, last_value
  • Partitioning vs ordering inside OVER
  • Reusable windows with WINDOW name AS (...)
  • The frame clause: ROWS BETWEEN ... AND ...

Example

Example 39.1 Window Functions
-- Numbering rows within each partition
SELECT customer_id, id AS order_id, total,
       row_number() OVER (PARTITION BY customer_id ORDER BY total DESC) AS rn,
       rank()       OVER (PARTITION BY customer_id ORDER BY total DESC) AS rnk
FROM orders_example
ORDER BY customer_id, rn;

-- "Most expensive order per customer" — keep rn = 1
WITH ranked AS (
  SELECT *,
         row_number() OVER (PARTITION BY customer_id ORDER BY total DESC) AS rn
  FROM orders_example
)
SELECT customer_id, id, total FROM ranked WHERE rn = 1;

-- lag / lead: look at previous / next row in the partition
SELECT customer_id, id, total,
       lag(total)  OVER (PARTITION BY customer_id ORDER BY id) AS prev_total,
       lead(total) OVER (PARTITION BY customer_id ORDER BY id) AS next_total
FROM orders_example
ORDER BY customer_id, id;

-- Running total (a sum is also a window function)
SELECT customer_id, id, total,
       sum(total) OVER (PARTITION BY customer_id ORDER BY id) AS running_total
FROM orders_example
ORDER BY customer_id, id;

-- Reusable window
SELECT id, total,
       sum(total) OVER w AS running,
       avg(total) OVER w AS running_avg
FROM orders_example
WINDOW w AS (PARTITION BY customer_id ORDER BY id)
ORDER BY customer_id, id;

-- Frame clause: 3-row trailing average
SELECT id, total,
       avg(total) OVER (
         PARTITION BY customer_id
         ORDER BY id
         ROWS BETWEEN 2 PRECEDING AND CURRENT ROW
       ) AS trailing_avg
FROM orders_example
ORDER BY customer_id, id;

The window is defined by three pieces:

  1. PARTITION BY — splits the rows into independent groups. Like GROUP BY, but each group keeps its rows.
  2. ORDER BY — orders rows inside each partition. Determines row_number, lag, running totals, etc.
  3. Frame clause (optional) — ROWS BETWEEN ... AND ... or RANGE BETWEEN .... Defines which rows around the current row are in the window. Defaults differ by function family: ranking functions ignore the frame; with ORDER BY, the default frame includes all rows through the current row’s last peer; without ORDER BY, it covers the partition.

The differences between row_number, rank, and dense_rank matter when there are ties. row_number always assigns 1, 2, 3, … with ties broken arbitrarily. rank assigns 1, 2, 2, 4 (gap after a tie). dense_rank assigns 1, 2, 2, 3 (no gap).

The “top-N per group” pattern (PARTITION BY group_col ORDER BY rank_col DESC + filter rn = 1) is the canonical use of row_number. It replaces convoluted self-joins and subqueries.

Sample data

Chapter 38 archives order 3. The output here follows that chapter order and has two remaining orders. Rerunning chapter 29 restores all three original orders.

Run the example

To run (requires orders_example):

$ psql -f source/window-functions.sql postgres
 customer_id | order_id | total | rn | rnk
-------------+----------+-------+----+-----
           1 |        1 | 50.00 |  1 |   1
           1 |        2 | 30.00 |  2 |   2
(2 rows)
 ...

Common pitfalls

  • Window functions are not allowed in WHERE or HAVING (they are computed after those clauses). Wrap the query in a CTE or subquery and filter outside.
  • The default frame for sum() OVER (ORDER BY ...) is RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW — peer rows with equal ORDER BY values are summed into the same total. Use ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW to be strict about row-by-row running.
  • Without ORDER BY inside OVER, lag / lead use an unspecified row order and PostgreSQL does not warn. Specify the order and any tie-breaker.
  • Performance: window functions sort the partition. With a matching index on (partition_col, order_col), the sort can be skipped.
Tip

“Show the change vs. previous row” is total - lag(total) OVER (ORDER BY ...). “Show as percent of total” is 100.0 * total / sum(total) OVER (). Both replace a self-join or subquery with a single readable expression.

Try it

Add ntile(4) to put each order into quartiles: ntile(4) OVER (ORDER BY total). Then compute “today’s total vs. yesterday’s” on a time-series table by lag(daily_total) OVER (ORDER BY day). Then try first_value(total) OVER (PARTITION BY customer_id ORDER BY id) to see the customer’s first order amount on every row.

Downloads

SQL file: window-functions.sql · All lesson files · Setup guide

40 String and Numeric Functions

Transform strings and use common numeric, rounding, and random-number functions.

PostgreSQL ships with hundreds of built-in functions. The string and math families cover most day-to-day needs: case conversion, padding, trimming, splitting, replacing, hashing; rounding, absolute value, modulus, exponentiation, random. Get comfortable with the common ones and you will reach for them constantly.

What you’ll learn

  • Common string functions: upper, lower, length, substring, position, trim, replace, split_part
  • String concatenation with || and concat
  • Regex with ~, ~*, regexp_replace, regexp_match
  • Common numeric functions: abs, round, ceil, floor, mod, power, random
  • The \df meta-command to discover more

Example

Example 40.1 String and Numeric Functions
-- Case and length
SELECT upper('hello') AS u, lower('HELLO') AS l, length('hello') AS len;

-- Substring: SQL syntax and function syntax
SELECT substring('postgresql' FROM 1 FOR 8) AS sql_form,
       substring('postgresql', 1, 8)        AS fn_form;

-- Position: 1-based index, 0 if not found
SELECT position('gre' IN 'postgresql') AS pos;

-- Trim and replace
SELECT trim('  hi  ')                         AS trimmed,
       trim(both '/' FROM '/path/to/file/')   AS path_trim,
       replace('a-b-c', '-', '.')             AS replaced;

-- Splitting
SELECT split_part('a.b.c.d', '.', 2)                  AS second_part,
       string_to_array('a,b,c', ',')                  AS as_array;

-- Concatenation: || vs concat. concat tolerates NULL; || does not.
SELECT 'a' || NULL AS pipe_null,
       concat('a', NULL, 'b') AS concat_null;

-- Regex: ~ matches, ~* is case-insensitive
SELECT 'PostgreSQL' ~ 'gres' AS match,
       'PostgreSQL' ~* 'POSTGRE' AS imatch,
       regexp_replace('a1b2c3', '[0-9]', '#', 'g') AS no_digits;

-- Numeric: abs, round, ceil, floor, mod, power, random
SELECT abs(-5)                AS a,
       round(3.14159, 2)      AS r,
       ceil(2.1)              AS c,
       floor(2.9)             AS f,
       mod(10, 3)             AS m,
       power(2, 10)           AS p,
       round((random() * 100)::numeric, 2) AS rand;

substring has two forms: the SQL-standard substring(str FROM start FOR n) and the function-call style substring(str, start, n). Both are 1-based.

concat and || differ on NULL: 'a' || NULL is NULL (any operand makes the whole expression NULL), while concat('a', NULL, 'b') returns 'ab' — concat treats NULLs as empty strings. Pick the behavior you want explicitly.

The regex operators ~ and ~* are matching shortcuts. regexp_replace(text, pattern, replacement [, flags]) is the workhorse for substitutions; the 'g' flag means “replace all matches” (default is first only). regexp_match returns the captured groups.

Numeric functions are mostly self-explanatory. random() returns a double precision in [0, 1); use floor(random() * n)::integer for integers from 0 through n − 1. A direct cast rounds instead of truncating. For cryptographically secure random, use gen_random_bytes from pgcrypto.

Run the example

To run:

$ psql -f source/string-and-numeric-functions.sql postgres
   u   |   l   | len
-------+-------+-----
 HELLO | hello |   5
(1 row)
 ...

Common pitfalls

  • Confusing substring indexing with zero-based languages — PostgreSQL is 1-based.
  • replace is a literal-string replacement, not a regex. For regex, use regexp_replace.
  • random() is not cryptographically secure. For security-sensitive cases, use pgcrypto’s gen_random_bytes.
  • length counts characters; octet_length counts bytes. They differ for multibyte text (UTF-8).
  • round(numeric) rounds ties away from zero: round(0.5) is 1 and round(-0.5) is -1. round(double precision) commonly rounds ties to even, with platform-dependent behavior.
Tip

In interactive psql, \df *upper* lists every function with “upper” in the name. \df+ upper shows the source/owner. Excellent way to discover what is available.

Try it

Build a label: SELECT upper(name) || ' [' || length(name) || ']' FROM fruits;. Round all averages: SELECT customer_id, round(avg(total), 2) FROM orders_example GROUP BY customer_id;. Use regex to find names starting with a vowel: SELECT name FROM fruits WHERE name ~* '^[aeiou]';.

Downloads

SQL file: string-and-numeric-functions.sql · All lesson files · Setup guide

41 Date Functions and COALESCE

Group dates into time buckets, extract fields, and supply defaults with COALESCE.

PostgreSQL has rich date/time arithmetic. date_trunc rounds to a precision; extract pulls a single field; interval lets you add and subtract durations; date_part, age, to_char, to_date cover the rest. And COALESCE — not date-specific but worth pairing with date logic — is how you supply defaults for possibly-NULL values.

What you’ll learn

  • current_date, current_timestamp, now(), clock_timestamp()
  • date_trunc(unit, ts): truncate to year, month, day, hour, etc.
  • extract(field FROM ts): pull year, month, day-of-week, …
  • interval arithmetic and formatting
  • COALESCE for NULL fallbacks

Example

Example 41.1 Date Functions and COALESCE
-- "Now" functions
SELECT current_date AS today, current_timestamp AS now_tz, clock_timestamp() AS wall_clock;

-- date_trunc: truncate to a precision
SELECT date_trunc('month', current_timestamp) AS month_start,
       date_trunc('day',   current_timestamp) AS day_start,
       date_trunc('hour',  current_timestamp) AS hour_start;

-- Interval arithmetic
SELECT current_date + interval '7 days'    AS in_a_week,
       current_date - interval '1 month'   AS a_month_ago,
       current_timestamp + interval '90 minutes' AS in_90m;

-- Difference between two timestamps as an interval
SELECT age(timestamptz '2025-01-01', timestamptz '2024-03-15') AS gap;

-- extract: pull a field
SELECT extract(year      FROM current_date) AS year,
       extract(month     FROM current_date) AS month,
       extract(dow       FROM current_date) AS day_of_week,  -- 0=Sunday
       extract(isodow    FROM current_date) AS iso_dow,      -- 1=Monday
       extract(week      FROM current_date) AS iso_week;

-- Formatting and parsing
SELECT to_char(current_timestamp, 'YYYY-MM-DD HH24:MI:SS TZ') AS formatted,
       to_date('2024-01-15', 'YYYY-MM-DD')                    AS parsed;

-- COALESCE: first non-NULL
SELECT COALESCE(NULL, NULL, 'default') AS c1,
       COALESCE(NULL, 42)              AS c2,
       COALESCE(due_at, current_timestamp + interval '7 days') AS due_or_default
FROM tasks_example
ORDER BY id;

date_trunc('month', ts) is the standard way to “bin by month” in analytics. The result is a timestamp (or timestamptz) at midnight on the first of that month. Use extract(isodow FROM ts) to group by weekday, date_trunc('week', ts) for calendar weeks, or date_bin (PostgreSQL 14+) for fixed-width buckets.

extract returns a numeric (or double precision in older versions). The dow field uses Sunday=0; isodow uses Monday=1 (matching ISO 8601). When in doubt, isodow matches what most “Monday is the start of the week” assumptions expect.

age(end, start) returns an interval normalized into years/months/days, which is useful for displays (“3 years, 2 months”). For raw differences, subtract two timestamptz values to get an interval in days/hours/seconds.

COALESCE walks its arguments left-to-right and returns the first non-NULL. It is the canonical “default value” tool: COALESCE(user.display_name, user.email, 'Anonymous').

Run the example

To run:

$ psql -f source/date-functions-and-coalesce.sql postgres
    today    |          now_tz             |       wall_clock
-------------+-----------------------------+-----------------------------
 2026-xx-xx  | 2026-xx-xx xx:xx:xx.xxxxxx+00 | 2026-xx-xx xx:xx:xx.xxxxxx+00
(1 row)
 ...

Common pitfalls

  • now() returns the transaction start time. Inside a long transaction, every call returns the same value. Use clock_timestamp() for true wall-clock.
  • Mixing timestamp and timestamptz silently uses the session’s time zone. Be explicit: at time zone 'UTC' for cast/display, and store as timestamptz whenever possible.
  • date_trunc returns a value of the same type — date_trunc('day', timestamptz) returns a timestamptz, not a date. Cast if you need date.
  • extract(epoch FROM ts) returns numeric seconds since the Unix epoch in PostgreSQL 17; date_part('epoch', ts) returns double precision. Useful when interoperating with code that uses Unix time.
  • COALESCE short-circuits, so side-effecting expressions in later arguments are not evaluated when an earlier one is non-NULL.
Tip

For “the first day of next month,” date_trunc('month', current_date) + interval '1 month'. For “the last day of this month,” date_trunc('month', current_date) + interval '1 month' - interval '1 day'. These idioms are everywhere in reporting queries.

Try it

Compute the first day of the current year (date_trunc('year', current_date)). Format a timestamp as ISO 8601: to_char(current_timestamp, 'YYYY-MM-DD"T"HH24:MI:SSOF'). Use COALESCE in a SELECT against tasks_example to display due_at or ’no deadline'.

Downloads

SQL file: date-functions-and-coalesce.sql · All lesson files · Setup guide

42 CASE

Express conditional values with CASE and compare conditional aggregation with FILTER.

CASE is SQL’s conditional expression — the closest thing to an if/else. It has two forms. The searched CASE (CASE WHEN cond THEN ... END) evaluates arbitrary boolean conditions. The simple CASE (CASE expr WHEN value THEN ... END) compares an expression to specific values. CASE can be used anywhere an expression is allowed — in SELECT, WHERE, ORDER BY, GROUP BY, and in expressions inside aggregates.

What you’ll learn

  • Searched CASE for general conditions
  • Simple CASE for value equality
  • ELSE and what happens without it
  • Using CASE to bucket rows for grouping
  • The role of CASE in conditional aggregation (and when FILTER replaces it)

Example

Example 42.1 CASE
-- Searched CASE (most flexible)
SELECT id, name,
  CASE
    WHEN id <= 2  THEN 'low'
    WHEN id <= 4  THEN 'mid'
    ELSE              'high'
  END AS tier
FROM fruits
ORDER BY id
LIMIT 5;

-- Simple CASE (equality)
SELECT name,
  CASE name
    WHEN 'apricot' THEN 'A'
    WHEN 'blueberry' THEN 'B'
    ELSE '?'
  END AS code
FROM fruits
ORDER BY id
LIMIT 4;

-- CASE in WHERE
SELECT id, name
FROM fruits
WHERE CASE WHEN id < 5 THEN id > 1 ELSE false END
ORDER BY id;

-- CASE for bucketing and counting per bucket
SELECT
  CASE
    WHEN id <= 2 THEN 'low'
    WHEN id <= 4 THEN 'mid'
    ELSE              'high'
  END AS tier,
  count(*) AS cnt
FROM fruits
GROUP BY tier
ORDER BY tier;

-- Conditional aggregation: CASE inside an aggregate vs. FILTER (cleaner)
SELECT
  sum(CASE WHEN total > 30 THEN total ELSE 0 END) AS big_sum_old_style,
  sum(total) FILTER (WHERE total > 30)            AS big_sum_filter
FROM orders_example;

The searched form is the workhorse. PostgreSQL evaluates conditions top-to-bottom and returns the result of the first true one. If no condition matches and there is no ELSE, the result is NULL — sometimes useful, sometimes a bug.

The simple form is shorthand for equality: CASE name WHEN 'a' THEN ... END is the same as CASE WHEN name = 'a' THEN ... END. It is concise when you have many discrete values to map.

Conditional aggregation — counting or summing only matching rows — used to require sum(CASE WHEN cond THEN val ELSE 0 END). sum(val) FILTER (WHERE cond) expresses the condition directly. It can return NULL where sum(CASE WHEN cond THEN val ELSE 0 END) returns 0 (when input rows exist but none match). Use COALESCE if zero is the intended empty-result value.

Run the example

To run:

$ psql -f source/case.sql postgres
 id |    name    | tier
----+------------+------
  1 | apricot    | low
  2 | blueberry  | low
  3 | chestnut   | mid
  4 | damson     | mid
  5 | elderberry | high
(5 rows)
 ...

Common pitfalls

  • All THEN/ELSE branches must agree on a common type — CASE WHEN ... THEN 1 ELSE 'x' END is an error. Cast explicitly when mixing types: CASE WHEN ... THEN 1::text ELSE 'x' END.
  • Forgetting END is the most common syntax error.
  • Without ELSE, the result is NULL when no branch matches. If that is surprising, add an explicit ELSE ....
  • CASE in WHERE works but is usually unnecessary — most conditions are clearer as plain boolean expressions.
Tip

CASE and COALESCE overlap but are not the same. COALESCE(a, b, c) is shorter for “first non-null”; CASE is the right choice when conditions are not just “is null.”

Try it

Use CASE to map id to a label like ‘small’ / ‘medium’ / ’large’ and count per label. Use CASE in ORDER BY to sort priorities in a custom order: ORDER BY CASE priority WHEN 'high' THEN 1 WHEN 'mid' THEN 2 ELSE 3 END. Then rewrite a sum(CASE WHEN ...) as sum(...) FILTER (WHERE ...).

Downloads

SQL file: case.sql · All lesson files · Setup guide

43 CREATE INDEX

Create plain, multicolumn, partial, and expression indexes with appropriate locking.

An index is an auxiliary data structure that lets PostgreSQL find rows by a column value without scanning the whole table. A primary key automatically creates one. For other columns, you choose: CREATE INDEX idx ON table (col). The default and most useful index type is B-tree, which speeds up equality (=), range (<, >, BETWEEN), and prefix (LIKE 'abc%') lookups, plus ORDER BY. PostgreSQL also offers GIN (for full-text, arrays, jsonb), GiST (for ranges and geometry), BRIN (for huge, naturally-ordered tables), Hash, and SP-GiST.

What you’ll learn

  • Creating a B-tree index on a column or set of columns
  • Naming and listing indexes
  • Multi-column index order and what it implies
  • Partial and expression indexes
  • CREATE INDEX CONCURRENTLY for production

Example

Example 43.1 CREATE INDEX
-- Single-column B-tree index
CREATE INDEX IF NOT EXISTS fruits_name_idx ON fruits (name);

-- Multi-column index. Order matters!
-- Useful for queries that filter by customer_id, or by (customer_id, id).
CREATE INDEX IF NOT EXISTS orders_customer_idx
  ON orders_example (customer_id, id);

-- Partial index: only index rows matching a predicate. Tiny and fast.
CREATE INDEX IF NOT EXISTS tasks_open_idx
  ON tasks_example (id) WHERE done = false;

-- Expression index: lets queries that use LOWER(email) hit an index.
-- (using a temp table to avoid clutter)
DROP TABLE IF EXISTS people_example;
CREATE TABLE people_example (id integer PRIMARY KEY, email text);
INSERT INTO people_example VALUES (1, 'Alice@example.com'), (2, 'bob@example.com');
CREATE INDEX people_lower_email_idx ON people_example (lower(email));

-- Use it
SELECT * FROM people_example WHERE lower(email) = 'alice@example.com';

-- List indexes on a table
SELECT indexname, indexdef
FROM pg_indexes
WHERE tablename IN ('fruits', 'orders_example', 'tasks_example', 'people_example')
ORDER BY tablename, indexname;

A B-tree index is a sorted, balanced tree that supports =, <, >, BETWEEN, IS NULL, ORDER BY, and LIKE 'prefix%' (anchored prefix). It is the default and what 90% of indexes in a typical schema are.

Multi-column indexes follow the leftmost prefix rule: an index on (a, b, c) can serve queries that filter on a, on (a, b), on (a, b, c), and can also scan for predicates on b alone or (b, c), though without leading-column restrictions this may require much more of the index. Choose column order from the actual filters and ordering requirements. PostgreSQL 18 adds skip-scan optimization for some of these cases.

Partial indexes index only the rows where a predicate is true. Excellent for “is_active = true” style queries — the index is smaller, faster, and avoids covering most of the (uninteresting) rows. Expression indexes let you index a transformed value (lower(email), (data ->> 'name')). They are the only way to make WHERE lower(email) = 'x' index-fast.

CREATE INDEX CONCURRENTLY builds the index without taking a table-level write lock. It takes longer (two passes over the table) but does not block writes — required in production.

Run the example

To run (requires earlier tables):

$ psql -f source/create-index.sql postgres
CREATE INDEX
CREATE INDEX
CREATE INDEX
DROP TABLE
CREATE TABLE
INSERT 0 2
CREATE INDEX
 id |        email
----+---------------------
  1 | Alice@example.com
(1 row)
 ...

Common pitfalls

  • Over-indexing: inserts and many updates create index entries; HOT updates can avoid index work, while cleanup of dead index entries is usually deferred to vacuum. On write-heavy tables, indexes you do not actually use are pure overhead.
  • Indexing low-cardinality columns alone (e.g. boolean is_active) usually does not help — the planner will sequential-scan anyway. Combine with another column or use a partial index.
  • Naming indexes inconsistently makes debugging painful. Adopt a convention like tablename_column_idx.
  • CREATE INDEX without CONCURRENTLY takes a SHARE lock: ordinary reads continue, but writes wait. Use CONCURRENTLY when writes must continue; it cannot run inside a transaction block and has extra work and failure-handling requirements.
Tip

pg_stat_user_indexes shows how often each index is used (idx_scan). Periodically check it; an index with zero scans over weeks is a candidate for dropping.

Try it

Create an index on customer_id only and another on (customer_id, total). Look at EXPLAIN SELECT * FROM orders_example WHERE customer_id = 1 ORDER BY total; and see which one the planner picks. Drop the unused one.

Downloads

SQL file: create-index.sql · All lesson files · Setup guide

44 When to Index

Choose indexes from real queries and inspect whether PostgreSQL uses them.

Indexes help when you filter, sort, or join on the indexed column(s); they cost disk space and slow down INSERT/UPDATE/DELETE; and on small tables, the planner may sequential-scan anyway. So the question is not “should this column have an index?” — it is “does this query need an index, and which one would help?” The answer comes from EXPLAIN and pg_stat_user_indexes.

What you’ll learn

  • The fundamental trade-off: index = faster reads, slower writes, more disk
  • Which queries benefit (equality, range, join, ORDER BY)
  • Why low-cardinality columns rarely benefit from a plain B-tree
  • Detecting which indexes are unused with pg_stat_user_indexes
  • The role of ANALYZE in keeping statistics current

Example

Example 44.1 When to Index
-- Compare plans: without an index vs. with one
DROP INDEX IF EXISTS fruits_name_idx;
EXPLAIN (FORMAT TEXT) SELECT * FROM fruits WHERE name = 'apricot';

CREATE INDEX fruits_name_idx ON fruits (name);
ANALYZE fruits;   -- refresh planner statistics

EXPLAIN (FORMAT TEXT) SELECT * FROM fruits WHERE name = 'apricot';

-- Range queries also benefit
EXPLAIN (FORMAT TEXT) SELECT * FROM fruits WHERE id BETWEEN 2 AND 4;

-- Index usage stats — which indexes is the database actually using?
SELECT relname AS table_name,
       indexrelname AS index_name,
       idx_scan,
       idx_tup_read
FROM pg_stat_user_indexes
WHERE schemaname = 'public'
ORDER BY idx_scan DESC
LIMIT 10;

A B-tree index helps when:

  • Equality lookups on the indexed column (WHERE x = ?).
  • Range scans (WHERE x BETWEEN ? AND ?, WHERE x < ?).
  • Joins on the indexed column (JOIN on parent.id = child.parent_id). Always index foreign-key columns.
  • ORDER BY matching the index’s column(s) and direction.
  • Anchored prefix LIKE (LIKE 'abc%').

It does not help (or barely helps) when:

  • The query scans most of the table anyway (there is no universal percentage threshold; row width, correlation, cache state, and costs matter).
  • The column has very few distinct values (is_active boolean — index won’t help the common-case branch).
  • The condition wraps the column in a function (WHERE lower(email) = ...) — unless you have an expression index.
  • The table is tiny (sequential scan is faster than the index lookup overhead).

On small tables the planner often prefers Seq Scan even when an index exists — because reading the whole table from one or two pages is cheaper than the index lookup. That is a feature, not a bug.

ANALYZE table updates the statistics the planner uses to choose plans. Run it after large data changes; autovacuum runs it automatically based on row-churn thresholds, but a manual run is sometimes needed.

Run the example

To run (requires fruits):

$ psql -f source/when-to-index.sql postgres
DROP INDEX
                      QUERY PLAN
-------------------------------------------------------
 Seq Scan on fruits  (cost=0.00..1.10 rows=1 width=36)
   Filter: (name = 'apricot'::text)
(2 rows)

CREATE INDEX
ANALYZE
                      QUERY PLAN
-------------------------------------------------------
 Seq Scan on fruits  (cost=0.00..1.10 rows=1 width=12)
   Filter: (name = 'apricot'::text)
(2 rows)
 ...

Common pitfalls

  • Adding indexes for queries you “might” run someday. Each one slows every write. Index based on real workloads.
  • Indexes that are subsets of another. (a) is redundant if (a, b) exists for equality lookups on a. The same leading column also supports range scans. Check size, uniqueness, predicates, and workload before treating an index as redundant.
  • Functions on indexed columns block index use: WHERE upper(name) = 'X' ignores an index on name — use an expression index or rewrite.
  • Bloated indexes that grew over time: routine REINDEX CONCURRENTLY keeps them tight.
Tip

The textbook rule: every foreign-key column gets an index. Joining without one is slow, and cascading deletes scan-the-world. Postgres does not auto-index foreign-key columns the way primary keys are.

Try it

Run EXPLAIN ANALYZE (actually executes the query) before and after creating an index — the row counts and timing tell the truth. Then create a deliberately bad index (e.g. on a boolean) and observe that the planner ignores it.

Downloads

SQL file: when-to-index.sql · All lesson files · Setup guide

45 EXPLAIN and Query Plans

Read execution plans and compare estimated costs with actual rows, timings, and buffers.

EXPLAIN shows how PostgreSQL plans to execute a query — which indexes it would use, which join strategies, and how many rows it expects at each step. EXPLAIN ANALYZE actually runs the query and reports the actual row counts and timings alongside the estimates. The gap between estimate and actual tells you whether the planner’s statistics are good, and tracking down slow queries means becoming fluent in reading these plans.

What you’ll learn

  • Reading a query plan from the bottom up
  • The difference between estimated and actual rows / time
  • The most common node types: Seq Scan, Index Scan, Bitmap Heap Scan, Nested Loop, Hash Join, Merge Join, Sort, Aggregate
  • EXPLAIN ANALYZE, BUFFERS, and other options
  • When the estimate is “wildly off” and how to fix it

Example

Example 45.1 EXPLAIN and Query Plans
-- Plain EXPLAIN: estimates only
EXPLAIN SELECT * FROM orders_example WHERE customer_id = 1;

-- EXPLAIN ANALYZE: actually runs the query, shows real timing
EXPLAIN ANALYZE SELECT * FROM orders_example WHERE customer_id = 1;

-- With BUFFERS: how many pages came from cache vs. disk
EXPLAIN (ANALYZE, BUFFERS)
  SELECT c.name, count(o.id) AS orders
  FROM customers_example c
  LEFT JOIN orders_example o ON o.customer_id = c.id
  GROUP BY c.name
  ORDER BY orders DESC;

-- A more complex plan to study
EXPLAIN ANALYZE
  WITH spend AS (
    SELECT customer_id, sum(total) AS s FROM orders_example GROUP BY customer_id
  )
  SELECT c.name, s.s
  FROM customers_example c
  LEFT JOIN spend s ON s.customer_id = c.id
  ORDER BY s.s DESC NULLS LAST;

Read a plan from the innermost, deepest node upward. Each line is a step; child steps feed parents. The fields on each line tell you:

  • cost=startup..total — planner’s estimate, in arbitrary units. The first number is when the first row is ready; the second is when the last row is ready.
  • rows=N — estimated row count.
  • width=N — average row width in bytes.
  • actual time=startup..total ms (with ANALYZE) — real timings, per loop.
  • rows=N actual — real row counts.
  • loops=N — how many times this node was executed (in nested loop joins, the inner side runs once per outer row).

Common scan types:

  • Seq Scan: read the entire table. Fine on small tables; bad on large ones unless you really want every row.
  • Index Scan: walk the index, fetch matching rows from the table. Best when few rows match.
  • Index Only Scan: the index has the required values, but heap visits may still be needed to check visibility unless the visibility map marks the heap page all-visible.
  • Bitmap Heap Scan / Bitmap Index Scan: build a bitmap of matching pages, then read them. Good when many rows match.

Join strategies:

  • Nested Loop: for each outer row, look up matching inner rows. Fast for small outer + indexed inner.
  • Hash Join: build a hash of the smaller side, probe with the larger. Fast for moderate-size joins.
  • Merge Join: both sides sorted on the join key, walk in parallel. Best when both sides are already sorted (e.g. by indexes).

If rows=N (estimated) and rows=M actual are wildly different (orders of magnitude), the planner has stale or misleading statistics. Run ANALYZE table to refresh.

Run the example

To run:

$ psql -f source/explain.sql postgres
                                  QUERY PLAN
-------------------------------------------------------------------------------
 Seq Scan on orders_example  (cost=0.00..1.04 rows=2 width=10)
   Filter: (customer_id = 1)
(2 rows)
 ...

Common pitfalls

  • Reading a plan top-to-bottom instead of bottom-up. The leaves are the actual data sources; everything above is a transformation.
  • Trusting cost= literally. The numbers are unitless — they only matter relative to each other inside the same plan.
  • EXPLAIN ANALYZE actually runs the query, including any INSERT/UPDATE/DELETE. To preview a write, wrap in a transaction and ROLLBACK: BEGIN; EXPLAIN ANALYZE UPDATE ...; ROLLBACK;.
  • Tiny tables defeat optimization. Test on representative data; a Seq Scan over 10 rows means nothing.
Tip

Paste an EXPLAIN ANALYZE plan into https://explain.depesz.com/ or https://explain.dalibo.com/ for a color-coded visualization that highlights the slowest steps. Indispensable for production debugging.

Try it

Drop an index, run EXPLAIN ANALYZE and note the plan. Recreate it and re-run — observe the plan change. Then run a query with a join: EXPLAIN ANALYZE SELECT ... FROM customers_example c JOIN orders_example o ON o.customer_id = c.id; and identify the join strategy chosen.

Downloads

SQL file: explain.sql · All lesson files · Setup guide

46 BEGIN, COMMIT, ROLLBACK

Group changes into transactions, commit or roll back, and observe an expected failure.

A transaction groups statements into an all-or-nothing unit. BEGIN (or START TRANSACTION) starts one; COMMIT makes the changes permanent; ROLLBACK discards them. Outside a transaction, each statement runs in its own implicit one — PostgreSQL is always transactional, you just choose whether to name the boundaries. Transactions are how you keep your data consistent through multi-step changes.

What you’ll learn

  • Starting a transaction with BEGIN and finishing with COMMIT or ROLLBACK
  • The ACID properties (especially Atomicity and Isolation)
  • Implicit per-statement transactions
  • Isolation levels: READ COMMITTED (default), REPEATABLE READ, SERIALIZABLE
  • The ABORT / “transaction is aborted” state

Example

Example 46.1 BEGIN, COMMIT, ROLLBACK
-- A transaction that we throw away
BEGIN;
DROP TABLE IF EXISTS tx_demo;
CREATE TABLE tx_demo (id integer PRIMARY KEY, val text);
INSERT INTO tx_demo VALUES (1, 'a');
ROLLBACK;

-- The table does not exist after rollback
SELECT count(*) FROM pg_tables WHERE tablename = 'tx_demo';

-- A transaction that we keep
BEGIN;
CREATE TABLE tx_demo (id integer PRIMARY KEY, val text);
INSERT INTO tx_demo VALUES (1, 'a'), (2, 'b');
COMMIT;

SELECT * FROM tx_demo ORDER BY id;

-- Set isolation level for the next transaction
BEGIN ISOLATION LEVEL REPEATABLE READ;
SELECT count(*) FROM tx_demo;       -- sees a consistent snapshot
COMMIT;

-- When a statement errors inside a transaction, the whole transaction is aborted.
-- Subsequent statements fail until you ROLLBACK.
BEGIN;
INSERT INTO tx_demo VALUES (1, 'duplicate');  -- PK violation
-- Any further statements here would fail with "current transaction is aborted".
ROLLBACK;

-- Cleanup
DROP TABLE tx_demo;

PostgreSQL provides full ACID guarantees:

  • Atomic: either every change in the transaction takes effect or none does.
  • Consistent: constraints are checked; the database moves from one valid state to another.
  • Isolated: concurrent transactions do not see each other’s uncommitted work (degrees of strictness vary by isolation level).
  • Durable: with the normal durability settings, a successful COMMIT makes logged changes survive a crash; settings such as synchronous_commit = off change that guarantee.

The default isolation is READ COMMITTED: each statement sees a snapshot of the database at the moment it starts. REPEATABLE READ gives the whole transaction a single snapshot — repeated reads see the same data. SERIALIZABLE adds checks for serialization anomalies and can roll back transactions that would produce a non-serial schedule.

PostgreSQL’s transaction error model is strict: any error inside a transaction aborts the transaction. Subsequent statements fail with “current transaction is aborted, commands ignored until end of transaction block” until you ROLLBACK. To handle errors gracefully mid-transaction, use savepoints (next lesson).

Expected error

This lesson deliberately inserts a duplicate primary key to demonstrate transaction failure. Run it with psql’s default ON_ERROR_STOP=off, so the following ROLLBACK and cleanup execute. Other lessons can use -v ON_ERROR_STOP=1.

Run the example

To run:

$ psql -f source/begin-commit-rollback.sql postgres
BEGIN
DROP TABLE
CREATE TABLE
INSERT 0 1
ROLLBACK
 count
-------
     0
(1 row)
BEGIN
CREATE TABLE
INSERT 0 2
COMMIT
 id | val
----+-----
  1 | a
  2 | b
(2 rows)
 ...

Common pitfalls

  • Forgetting to COMMIT in interactive psql means everything you did is rolled back when you disconnect. Set \set AUTOCOMMIT off if you want explicit control; the default AUTOCOMMIT on runs each statement in its own implicit transaction.
  • Long-running transactions hold row locks and prevent autovacuum from reclaiming dead tuples. Keep transactions short.
  • Mixing schema changes (DDL) and data changes (DML) in one transaction is fine in PostgreSQL — unlike MySQL, DDL is transactional too.
  • In READ COMMITTED, two statements in the same transaction can see different values of the same row (if a concurrent transaction committed between them). For consistent multi-statement reads, switch to REPEATABLE READ.
Tip

BEGIN; ... ROLLBACK; can undo transactional table changes during a rehearsal, but it still takes locks and runs triggers. Sequence increments and external side effects are not rolled back. Use a disposable database for these exercises.

Try it

The script drops tx_demo during cleanup. First recreate it with CREATE TABLE tx_demo (id integer PRIMARY KEY, val text); INSERT INTO tx_demo VALUES (1, 'a');. Then open two psql sessions. In session A: BEGIN; UPDATE tx_demo SET val = 'x' WHERE id = 1; (do not commit yet). In session B: SELECT * FROM tx_demo; — you should see the old value. Then session A COMMIT; and session B’s next SELECT will see the new value.

Downloads

SQL file: begin-commit-rollback.sql · All lesson files · Setup guide

47 Savepoints

Roll back part of a transaction and recover from errors with savepoints.

A savepoint is a named point inside a transaction. You can ROLLBACK TO SAVEPOINT name to undo work done after that point while keeping everything before it. Savepoints are how PostgreSQL implements nested-transaction-like semantics — useful when you want to try a risky step, recover from an error, and continue, or when application code emulates nested transactions.

What you’ll learn

  • Setting a savepoint with SAVEPOINT name
  • Rolling back to a savepoint
  • Releasing a savepoint
  • Recovering from an error inside a transaction
  • The relationship to BEGIN/COMMIT

Example

Example 47.1 Savepoints
-- Setup
BEGIN;
DROP TABLE IF EXISTS sp_demo;
CREATE TABLE sp_demo (id integer PRIMARY KEY);

INSERT INTO sp_demo VALUES (1);
SAVEPOINT after_one;

INSERT INTO sp_demo VALUES (2);
SAVEPOINT after_two;

INSERT INTO sp_demo VALUES (3);

-- Undo only the last insert
ROLLBACK TO SAVEPOINT after_two;

-- After rolling back to after_two, the table has 1 and 2 (not 3)
SELECT * FROM sp_demo ORDER BY id;

-- Release a savepoint when we are sure we won't need it
RELEASE SAVEPOINT after_one;

-- Recovering from an error mid-transaction
SAVEPOINT before_risky;
-- This insert will fail (duplicate)
DO $$ BEGIN
  INSERT INTO sp_demo VALUES (2);
EXCEPTION WHEN unique_violation THEN
  RAISE NOTICE 'duplicate caught — continuing';
END $$;
-- Or, without the PL/pgSQL: after a failing statement, you can ROLLBACK TO before_risky and keep going.

SELECT * FROM sp_demo ORDER BY id;

COMMIT;

DROP TABLE IF EXISTS sp_demo;

SAVEPOINT name marks a point. ROLLBACK TO SAVEPOINT name undoes everything between the savepoint and now, but the transaction continues — you can keep doing work and eventually COMMIT. RELEASE SAVEPOINT name forgets the savepoint (you cannot roll back to it anymore) but does not undo any work.

Savepoints are how application frameworks emulate “nested transactions”: when application code calls begin_transaction while already inside one, the driver sets a savepoint instead of starting a new top-level transaction. A “rollback” inside the inner block rolls back to that savepoint.

The error-recovery use case is important. Inside a transaction, any error puts the transaction in an “aborted” state where every subsequent statement fails until you ROLLBACK. With savepoints, you can ROLLBACK TO SAVEPOINT before_risky and continue — the transaction is back to a usable state.

Run the example

To run:

$ psql -f source/savepoints.sql postgres
BEGIN
DROP TABLE
CREATE TABLE
INSERT 0 1
SAVEPOINT
INSERT 0 1
SAVEPOINT
INSERT 0 1
ROLLBACK
 id
----
  1
  2
(2 rows)
RELEASE
 ...
COMMIT

Common pitfalls

  • Savepoints have overhead. A transaction with thousands of savepoints (a common ORM pattern) can be expensive.
  • After ROLLBACK TO SAVEPOINT s, the savepoint s itself still exists — you can roll back to it again. To remove it, use RELEASE.
  • RELEASE SAVEPOINT s does not “commit” the work — only COMMIT (of the enclosing transaction) does. RELEASE just removes the marker.
  • Savepoints only exist inside an explicit transaction. SAVEPOINT s; outside of one is an error.
Tip

ORMs often use savepoints to implement “transactional” tests where each test runs inside a savepoint that gets rolled back, leaving the database untouched. Look up pytest-postgresql, RSpec’s transactional fixtures, etc.

Try it

Add a second savepoint, insert more rows, then ROLLBACK TO the first savepoint and see all the work after it gone. Try the error-recovery pattern: BEGIN; INSERT ... SAVEPOINT s; INSERT (bad row); ROLLBACK TO SAVEPOINT s; INSERT (good row); COMMIT;.

Downloads

SQL file: savepoints.sql · All lesson files · Setup guide

48 CREATE VIEW

Expose reusable queries through views and understand when views accept updates.

A view is a named, saved SELECT. You query it like a table, but the rows are computed on the fly every time — nothing is stored. Views are excellent for hiding complexity (a big join lives behind a short name), enforcing column-level access (grant SELECT on the view but not the underlying table), and giving applications a stable interface as the schema underneath evolves. PostgreSQL also supports materialized views, which do store rows; that is the next lesson.

What you’ll learn

  • Creating and replacing views with CREATE OR REPLACE VIEW
  • Updatable views and the rules they must follow
  • WITH CHECK OPTION to enforce that updates stay within the view
  • Granting view-level access without exposing the underlying table
  • Listing and dropping views

Example

Example 48.1 CREATE VIEW
-- A simple view: customer + their order count
CREATE OR REPLACE VIEW customer_orders AS
SELECT c.id, c.name, count(o.id) AS order_count, COALESCE(sum(o.total), 0) AS total_spent
FROM customers_example c
LEFT JOIN orders_example o ON o.customer_id = c.id
GROUP BY c.id, c.name;

-- Use it like a table
SELECT * FROM customer_orders WHERE total_spent > 0 ORDER BY total_spent DESC;

-- Simple updatable view (no joins, no aggregates, no DISTINCT)
CREATE OR REPLACE VIEW alice_orders AS
SELECT * FROM orders_example WHERE customer_id = 1;

-- You can INSERT through an updatable view
INSERT INTO alice_orders (id, customer_id, total) VALUES (10, 1, 99.00);

-- WITH CHECK OPTION: refuse INSERTs/UPDATEs that wouldn't satisfy the view's WHERE
CREATE OR REPLACE VIEW alice_orders_strict AS
SELECT * FROM orders_example WHERE customer_id = 1
WITH CHECK OPTION;

-- This would fail because customer_id = 2 is not in the view's scope
-- INSERT INTO alice_orders_strict (id, customer_id, total) VALUES (11, 2, 1.00);

-- List views
SELECT viewname, definition
FROM pg_views
WHERE schemaname = 'public'
ORDER BY viewname;

-- Drop a view
DROP VIEW IF EXISTS alice_orders;
DROP VIEW IF EXISTS alice_orders_strict;
DROP VIEW IF EXISTS customer_orders;

Views are virtual: every SELECT against the view re-runs the underlying query. There is no caching — the view is essentially a macro the planner inlines. That makes views free of staleness but also means they offer no performance benefit on their own; they are about abstraction, not speed.

Updatable views — those that map cleanly to a single underlying table without aggregates, joins, DISTINCT, or set operations — accept INSERT, UPDATE, and DELETE. The action affects the underlying table. WITH CHECK OPTION is the safety belt that prevents an UPDATE from moving a row out of the view’s predicate.

CREATE OR REPLACE VIEW updates the definition if the view already exists. The column list and types must remain compatible — you cannot remove columns or change their types this way. Use ALTER VIEW ... RENAME COLUMN to rename a column; other incompatible changes may require drop and recreate.

Run the example

To run (requires customers_example and orders_example):

$ psql -f source/create-view.sql postgres
CREATE VIEW
 id | name  | order_count | total_spent
----+-------+-------------+-------------
  1 | Alice |           2 |       80.00
(1 row)
 ...

Common pitfalls

  • Views are not magic performance — every query inlines them. For repeated heavy work, consider a materialized view.
  • Updatable view rules are strict: any aggregate, DISTINCT, GROUP BY, set operation, or multiple base tables typically makes the view non-updatable. You can write INSTEAD OF triggers to allow writes anyway, but that is advanced.
  • CREATE OR REPLACE VIEW must preserve existing column names, order, and types, even when there are no dependents. It can append columns; use ALTER VIEW for a rename.
  • Views can hide expensive joins, so a one-liner against a view may execute as a multi-second query. Inspect with EXPLAIN.
Tip

Views are an underused tool for application security. Grant the application role SELECT only on a view that returns just the columns it should see (e.g. excluding password_hash). The underlying table remains protected.

Try it

Create a view that lists customers without orders (HAVING count(o.id) = 0). Use \d customer_orders in psql to inspect a view. Then create an UPDATE-able view scoped to one customer and update through it — observe the underlying table change.

Downloads

SQL file: create-view.sql · All lesson files · Setup guide

49 Materialized Views

Store query results in materialized views and refresh them while managing reader access.

A materialized view is a view whose result is stored on disk. Unlike a regular view, it does not re-run the query on every read — it returns the saved rows. The trade-off is staleness: until you REFRESH it, you see the result as of the last refresh. Materialized views are the standard way to cache expensive aggregations, search indexes, and pre-computed reports inside PostgreSQL.

What you’ll learn

  • Creating and refreshing a materialized view
  • REFRESH MATERIALIZED VIEW vs REFRESH MATERIALIZED VIEW CONCURRENTLY
  • Indexing a materialized view
  • Detecting how stale a matview is
  • When to use a matview vs. a regular view vs. a cache table

Example

Example 49.1 Materialized Views
-- A materialized view: aggregate per customer
DROP MATERIALIZED VIEW IF EXISTS customer_spend_mv;
CREATE MATERIALIZED VIEW customer_spend_mv AS
SELECT c.id              AS customer_id,
       c.name            AS customer_name,
       count(o.id)       AS order_count,
       COALESCE(sum(o.total), 0) AS total_spent,
       max(o.id)         AS last_order_id
FROM customers_example c
LEFT JOIN orders_example o ON o.customer_id = c.id
GROUP BY c.id, c.name;

-- Query it like a table
SELECT * FROM customer_spend_mv ORDER BY total_spent DESC;

-- Index it (a matview is a real table, so it supports indexes)
CREATE UNIQUE INDEX customer_spend_mv_pk ON customer_spend_mv (customer_id);
CREATE INDEX customer_spend_mv_total_idx ON customer_spend_mv (total_spent);

-- Add another order, then refresh
INSERT INTO orders_example VALUES (200, 1, 25.00);

-- Plain REFRESH locks the matview against reads for the duration
REFRESH MATERIALIZED VIEW customer_spend_mv;

-- CONCURRENTLY lets readers continue. Requires a UNIQUE index on the matview.
REFRESH MATERIALIZED VIEW CONCURRENTLY customer_spend_mv;

SELECT * FROM customer_spend_mv ORDER BY total_spent DESC;

-- Find matviews and when they were last refreshed (PostgreSQL keeps the relation modification time)
SELECT schemaname, matviewname, hasindexes, ispopulated
FROM pg_matviews
WHERE schemaname = 'public';

DROP MATERIALIZED VIEW IF EXISTS customer_spend_mv;

A materialized view is a table that PostgreSQL keeps in sync with a query — but only when you ask. REFRESH MATERIALIZED VIEW name re-runs the underlying SELECT and replaces the stored rows. By default, the refresh takes an ACCESS EXCLUSIVE lock — readers cannot query during the refresh.

REFRESH MATERIALIZED VIEW CONCURRENTLY updates the matview without blocking readers. The price is that it requires an already populated matview and a UNIQUE index covering all rows, using column names rather than expressions. It reruns the full query and reconciles the results; it is not incremental view maintenance. For matviews that need to stay queryable, this is the right choice.

A matview is a real table on disk — you can create indexes on it, ANALYZE it, even VACUUM it. That makes it ideal for caching expensive query results that many other queries then read.

Run the example

To run (requires customers_example and orders_example):

$ psql -f source/materialized-views.sql postgres
DROP MATERIALIZED VIEW
SELECT 3
 customer_id | customer_name | order_count | total_spent | last_order_id
-------------+---------------+-------------+-------------+---------------
           1 | Alice         |           3 |      179.00 |            10
           2 | Bob           |           0 |           0 |
           3 | Carol         |           0 |           0 |
(3 rows)

CREATE INDEX
CREATE INDEX
INSERT 0 1
REFRESH MATERIALIZED VIEW
REFRESH MATERIALIZED VIEW
 customer_id | customer_name | order_count | total_spent | last_order_id
-------------+---------------+-------------+-------------+---------------
           1 | Alice         |           4 |      204.00 |           200
           2 | Bob           |           0 |           0 |
           3 | Carol         |           0 |           0 |
(3 rows)

 ...

Common pitfalls

  • A matview is stale until refreshed. If you forget to refresh, reports lag silently. Set up a cron or scheduler.
  • REFRESH ... CONCURRENTLY requires at least one UNIQUE index. Without one, PostgreSQL errors. Add a unique index on whatever you treat as the matview’s “primary key” (often a column from the source table’s PK).
  • Foreign keys cannot reference a matview, and triggers do not fire on refresh — these are real differences from ordinary tables.
  • REFRESH runs the original SELECT from scratch. If that query is slow, the refresh is slow. For incremental refreshes, you currently need to roll your own or use the pg_ivm extension.
Tip

Use matviews for dashboards: pre-aggregate “yesterday’s metrics” once per night, then dashboards read instantly. For sub-minute freshness, a real-time write-path update or a regular view is usually a better fit.

Try it

Run this experiment before the script’s final DROP MATERIALIZED VIEW, or recreate the view and its indexes first. Add several rows to orders_example. Query the matview — it still shows the old totals. Run REFRESH MATERIALIZED VIEW customer_spend_mv; and re-query. Then add more rows and use the CONCURRENTLY form while another session is reading.

Downloads

SQL file: materialized-views.sql · All lesson files · Setup guide

50 Functions and PL/pgSQL

Define SQL and PL/pgSQL functions with appropriate volatility and security settings.

PostgreSQL lets you define your own functions. The simplest are SQL functions — a function body that is one or more SQL statements; PostgreSQL can often inline them. The more powerful are PL/pgSQL functions — written in a procedural language built into the server, with variables, loops, conditions, and exceptions. PL/pgSQL is what you use for triggers, complex business logic, and one-off maintenance routines. There are also PL/Python, PL/Perl, and others, but PL/pgSQL is always available and is what you should learn first.

What you’ll learn

  • The CREATE FUNCTION syntax
  • The difference between LANGUAGE sql and LANGUAGE plpgsql
  • Volatility categories: IMMUTABLE, STABLE, VOLATILE
  • Variables, IF, LOOP, and RAISE NOTICE in PL/pgSQL
  • SECURITY DEFINER and the search_path warning

Example

Example 50.1 Functions and PL/pgSQL
-- A simple SQL function
CREATE OR REPLACE FUNCTION square(x integer)
RETURNS integer
LANGUAGE sql
IMMUTABLE
AS $$
  SELECT x * x;
$$;

SELECT square(7) AS result;

-- A SQL function that returns a set (multiple rows)
CREATE OR REPLACE FUNCTION recent_orders(min_total numeric)
RETURNS TABLE (id integer, customer_id integer, total numeric)
LANGUAGE sql
STABLE
AS $$
  SELECT id, customer_id, total
  FROM orders_example
  WHERE total >= min_total
  ORDER BY id;
$$;

SELECT * FROM recent_orders(20);

-- A PL/pgSQL function with control flow
CREATE OR REPLACE FUNCTION classify_total(total numeric)
RETURNS text
LANGUAGE plpgsql
IMMUTABLE
AS $$
DECLARE
  result text;
BEGIN
  IF total IS NULL THEN
    result := 'unknown';
  ELSIF total < 20 THEN
    result := 'small';
  ELSIF total < 100 THEN
    result := 'medium';
  ELSE
    result := 'large';
  END IF;
  RETURN result;
END;
$$;

SELECT id, total, classify_total(total) AS bucket
FROM orders_example
ORDER BY id;

-- A function that loops and uses RAISE NOTICE (a print statement)
CREATE OR REPLACE FUNCTION ping(n integer)
RETURNS integer
LANGUAGE plpgsql
AS $$
DECLARE
  i integer;
BEGIN
  FOR i IN 1..n LOOP
    RAISE NOTICE 'ping %', i;
  END LOOP;
  RETURN n;
END;
$$;

SELECT ping(3);

-- Cleanup
DROP FUNCTION IF EXISTS square(integer);
DROP FUNCTION IF EXISTS recent_orders(numeric);
DROP FUNCTION IF EXISTS classify_total(numeric);
DROP FUNCTION IF EXISTS ping(integer);

A function declaration includes a name, argument list, return type, language, optional volatility, and body. The body is bracketed by dollar quotes ($$ ... $$) — these avoid the need to escape quotes inside the body. You can use $tag$ ... $tag$ if you ever need nested dollar quotes.

The volatility categories are critical for performance:

  • IMMUTABLE: same inputs always produce the same output, no database state involved. The planner can constant-fold these into expressions and even pre-evaluate them at planning time. Use for math, string formatting, deterministic transforms.
  • STABLE: same inputs produce the same output within a single statement, but may differ across statements (e.g. functions that read tables). This allows optimizations such as use in index scan conditions; it does not promise one evaluation per statement.
  • VOLATILE (default): may return different results even within one statement (e.g. random(), clock_timestamp(), anything that modifies data); now() is STABLE. The planner re-evaluates every time.

Wrong volatility leads to wrong query plans — most beginners leave everything VOLATILE and pay the cost. Mark functions as IMMUTABLE / STABLE whenever true.

Run the example

To run:

$ psql -f source/functions-and-plpgsql.sql postgres
CREATE FUNCTION
 result
--------
     49
(1 row)
 ...
NOTICE:  ping 1
NOTICE:  ping 2
NOTICE:  ping 3
 ping
------
    3
(1 row)

Common pitfalls

  • Defining a SECURITY DEFINER function (runs as the function owner) without setting search_path is a known privilege-escalation pattern. Use only trusted schemas, explicitly put pg_temp last, and qualify sensitive names; for example, SET search_path = pg_catalog, app, pg_temp if app is trusted.
  • Marking a function IMMUTABLE when it actually reads tables — the planner caches a wrong value. Reserve IMMUTABLE for true math/string functions.
  • Overloading by argument types is allowed but easy to abuse. recent_orders(numeric) and recent_orders(integer) are different functions; small calls may unexpectedly target the wrong one.
  • PL/pgSQL uses semicolon-terminated statements. Keywords and unquoted identifiers are case-insensitive; quoted identifiers and string contents are case-sensitive. Be careful with reserved words inside the function body.
Tip

Stored procedures (CREATE PROCEDURE, PostgreSQL 11+) are like functions but can manage transactions (COMMIT, ROLLBACK inside the body). Use functions for things that return values; procedures for batch operations that need transaction control.

Try it

Write an IMMUTABLE SQL function discount(price, pct) that returns the discounted price. Apply it in a query: SELECT id, total, discount(total, 10) FROM orders_example;. Then write a PL/pgSQL function that returns the largest order amount per customer using a loop or a single SQL query (the SQL form is much faster — try both to see).

Downloads

SQL file: functions-and-plpgsql.sql · All lesson files · Setup guide

51 Triggers

Maintain timestamps and audit changes with row-level and statement-level triggers.

A trigger is a function that PostgreSQL calls automatically when a specific event happens on a table — INSERT, UPDATE, DELETE, or TRUNCATE, fired BEFORE or AFTER the row change, for each row or for the whole statement. Triggers are the standard tool for enforcing invariants the type system cannot, auditing changes, denormalizing for performance, and keeping updated_at timestamps current.

What you’ll learn

  • The BEFORE vs AFTER distinction
  • Row-level (FOR EACH ROW) vs statement-level (FOR EACH STATEMENT)
  • The NEW and OLD row variables
  • A canonical example: auto-updating updated_at
  • An audit-log trigger that writes to a side table

Example

Example 51.1 Triggers
-- Setup
DROP TABLE IF EXISTS articles_example;
DROP TABLE IF EXISTS articles_audit;
CREATE TABLE articles_example (
  id          bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  title       text NOT NULL,
  body        text NOT NULL,
  created_at  timestamptz NOT NULL DEFAULT current_timestamp,
  updated_at  timestamptz NOT NULL DEFAULT current_timestamp
);

CREATE TABLE articles_audit (
  id          bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  article_id  bigint NOT NULL,
  action      text   NOT NULL,
  by_user     text   NOT NULL DEFAULT current_user,
  at          timestamptz NOT NULL DEFAULT current_timestamp,
  old_row     jsonb,
  new_row     jsonb
);

-- Trigger function: bump updated_at before every UPDATE
CREATE OR REPLACE FUNCTION set_updated_at()
RETURNS trigger
LANGUAGE plpgsql
AS $$
BEGIN
  NEW.updated_at := current_timestamp;
  RETURN NEW;
END;
$$;

CREATE TRIGGER articles_updated_at
BEFORE UPDATE ON articles_example
FOR EACH ROW
EXECUTE FUNCTION set_updated_at();

-- Trigger function: write to the audit log after every change
CREATE OR REPLACE FUNCTION audit_article_change()
RETURNS trigger
LANGUAGE plpgsql
AS $$
BEGIN
  INSERT INTO articles_audit (article_id, action, old_row, new_row)
  VALUES (
    COALESCE(NEW.id, OLD.id),
    TG_OP,
    CASE WHEN TG_OP IN ('UPDATE', 'DELETE') THEN to_jsonb(OLD) END,
    CASE WHEN TG_OP IN ('INSERT', 'UPDATE') THEN to_jsonb(NEW) END
  );
  RETURN NULL;  -- AFTER trigger return value is ignored
END;
$$;

CREATE TRIGGER articles_audit
AFTER INSERT OR UPDATE OR DELETE ON articles_example
FOR EACH ROW
EXECUTE FUNCTION audit_article_change();

-- Exercise the triggers
INSERT INTO articles_example (title, body) VALUES ('Hello', 'world');
UPDATE articles_example SET body = 'edited world' WHERE title = 'Hello';
DELETE FROM articles_example;

SELECT * FROM articles_audit ORDER BY id;

-- Cleanup
DROP TRIGGER IF EXISTS articles_updated_at ON articles_example;
DROP TRIGGER IF EXISTS articles_audit ON articles_example;
DROP FUNCTION IF EXISTS set_updated_at();
DROP FUNCTION IF EXISTS audit_article_change();
DROP TABLE IF EXISTS articles_example;
DROP TABLE IF EXISTS articles_audit;

A trigger is two pieces: a trigger function (returns trigger) and a trigger that ties the function to a table and an event. Inside the function:

  • NEW is the new row (for INSERT and UPDATE).
  • OLD is the old row (for UPDATE and DELETE).
  • TG_OP is the operation ('INSERT', 'UPDATE', 'DELETE', 'TRUNCATE').
  • TG_TABLE_NAME etc. are also available for reusable triggers.

For BEFORE ROW INSERT/UPDATE triggers, return NEW (possibly modified); for DELETE, return OLD. Returning NULL skips that row’s operation. For AFTER ROW triggers, the return value is ignored — the operation already happened. Statement-level triggers (FOR EACH STATEMENT) fire once per statement regardless of row count and do not have NEW/OLD.

The updated_at bumper is the most common trigger in the world. Define one trigger function, attach it to every table with an updated_at column.

Run the example

To run:

$ psql -f source/triggers.sql postgres
DROP TABLE
DROP TABLE
CREATE TABLE
CREATE TABLE
CREATE FUNCTION
CREATE TRIGGER
CREATE FUNCTION
CREATE TRIGGER
INSERT 0 1
UPDATE 1
DELETE 1
 id | article_id | action  | by_user  |              at               | old_row | new_row
----+------------+---------+----------+-------------------------------+---------+---------
  1 |          1 | INSERT  | your_user | 2026-xx-xx ...                |         | {"id":1,...}
  2 |          1 | UPDATE  | your_user | 2026-xx-xx ...                | {...}   | {...}
  3 |          1 | DELETE  | your_user | 2026-xx-xx ...                | {...}   |
(3 rows)
 ...

Common pitfalls

  • A BEFORE trigger that returns NULL silently swallows the row — the INSERT/UPDATE is skipped without an error. Return NEW for INSERT/UPDATE and OLD for DELETE unless you mean to skip the row.
  • Triggers fire inside the same transaction as the triggering statement. Heavy logic in a trigger slows every modification of the table.
  • Triggers do not fire on COPY ... FROM by default? They do, actually — but TRUNCATE does not fire row-level triggers (only statement-level). Plan accordingly.
  • Putting business logic in triggers makes it hard to discover from application code. Many teams limit triggers to infrastructure concerns (updated_at, audit) and keep business logic in the application.
Tip

Use pg_trigger_depth() inside a trigger to detect recursion. A trigger that updates the same table that fired it can loop — guard with IF pg_trigger_depth() > 1 THEN RETURN NEW; END IF;.

Try it

The script drops its demo tables and trigger functions at the end. Recreate the timestamp trigger function from the example before continuing. Add an updated_at column to fruits and attach the trigger from above. Update a row and confirm updated_at changes. Then add a BEFORE INSERT trigger that uppercases name automatically, and see it in action.

Downloads

SQL file: triggers.sql · All lesson files · Setup guide

52 Full-Text Search

Search and rank text with tsvector, tsquery, language configurations, and GIN indexes.

PostgreSQL has a competent, built-in full-text search engine — no Elasticsearch, no Solr needed for most apps. The core abstractions are tsvector (a parsed-and-normalized representation of a document) and tsquery (a parsed search expression). With a GIN index on a tsvector, you get fast text search with stemming, ranking, and multi-language support, all in plain SQL.

What you’ll learn

  • to_tsvector and to_tsquery
  • The @@ operator that tests a vector against a query
  • Configuring language (English, simple, etc.)
  • Indexing with GIN for fast search
  • Ranking results with ts_rank

Example

Example 52.1 Full-Text Search
-- Setup
DROP TABLE IF EXISTS docs_fts;
CREATE TABLE docs_fts (
  id    bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY,
  title text NOT NULL,
  body  text NOT NULL
);

INSERT INTO docs_fts (title, body) VALUES
  ('Postgres tips',    'PostgreSQL is a powerful relational database with full-text search built in.'),
  ('SQL joins',        'Inner joins keep matching rows; left joins keep all left rows.'),
  ('Indexing',         'B-tree indexes speed up equality and range queries. GIN indexes are great for arrays and text search.'),
  ('Cats',             'Cats are not databases. This sentence exists to add noise.');

-- to_tsvector parses, normalizes, and removes stop words
SELECT to_tsvector('english', 'PostgreSQL has powerful full-text search') AS v;

-- to_tsquery turns a query string into a tsquery
SELECT to_tsquery('english', 'powerful & search') AS q;

-- @@ tests "does this document match this query?"
SELECT title
FROM docs_fts
WHERE to_tsvector('english', title || ' ' || body) @@ to_tsquery('english', 'powerful & search');

-- websearch_to_tsquery: parses Google-style input (quotes, OR, -minus)
SELECT title
FROM docs_fts
WHERE to_tsvector('english', title || ' ' || body) @@ websearch_to_tsquery('english', 'index OR join -cats');

-- Add a generated tsvector column + GIN index for fast search at scale
ALTER TABLE docs_fts ADD COLUMN search_tsv tsvector
  GENERATED ALWAYS AS (to_tsvector('english', title || ' ' || body)) STORED;

CREATE INDEX docs_fts_search_idx ON docs_fts USING gin (search_tsv);

-- Rank results
SELECT title, ts_rank(search_tsv, websearch_to_tsquery('english', 'database search')) AS rank
FROM docs_fts
WHERE search_tsv @@ websearch_to_tsquery('english', 'database search')
ORDER BY rank DESC;

DROP TABLE IF EXISTS docs_fts;

to_tsvector('english', text) tokenizes the input, lowercases it, applies the English stemmer (so running and runs map to the same root), removes stop words (the, a, is), and returns a tsvector — a sorted list of normalized tokens with positions. Different configurations (english, simple, german, russian, …) implement different stemming and stop-word rules.

to_tsquery parses the expression: & (and), | (or), ! (not), <-> (followed by). websearch_to_tsquery accepts a friendlier syntax — double-quoted phrases, OR keyword, - to exclude. The latter is what you want behind a user-facing search box.

The @@ operator tests whether a tsvector matches a tsquery. With a GIN index on the tsvector, this is the fast path PostgreSQL uses for text search. The pattern above uses a generated column that always reflects title || ' ' || body — no triggers needed, no risk of forgetting to update.

Run the example

To run:

$ psql -f source/full-text-search.sql postgres
DROP TABLE
CREATE TABLE
INSERT 0 4
                                  v
---------------------------------------------------------------------
 'full':5 'full-text':4 'postgresql':1 'power':3 'search':7 'text':6
(1 row)
 ...

Common pitfalls

  • Calling to_tsvector('english', ...) in the WHERE clause without an expression index defeats indexing. Either index the expression or use a generated column.
  • Choosing the wrong language: to_tsvector('english', russian_text) produces meaningless stems. Match the configuration to the data.
  • LIKE '%foo%' is not full-text search. It is substring matching, slow without trigram indexes, and does not understand stemming or stop words. Use full-text search for natural-language queries.
  • For multi-language data, you may need to store the language alongside the text and parameterize the configuration.
Tip

For substring or fuzzy matching that does not need stemming, the pg_trgm extension provides trigram-based similarity (similarity(a, b)) and an index that supports LIKE '%foo%' efficiently. Often used alongside full-text search.

Try it

Run this before the final DROP TABLE docs_fts, or recreate the demo table first. Add more documents and search for them. Use <-> for phrase queries: to_tsquery('english', 'full <-> text') matches “full text” specifically. Add ts_headline('english', body, query) to highlight the matching snippet of each result.

Downloads

SQL file: full-text-search.sql · All lesson files · Setup guide

53 VACUUM and ANALYZE

Maintain row visibility and planner statistics with VACUUM, ANALYZE, and autovacuum.

PostgreSQL uses multi-version concurrency control (MVCC): an UPDATE does not overwrite a row, it writes a new version and marks the old one dead. A DELETE only marks rows dead — they stay on disk. VACUUM reclaims that space and prevents transaction-ID wraparound. ANALYZE updates the planner’s statistics so it makes good decisions. PostgreSQL runs both automatically via autovacuum, but understanding what they do — and when to nudge them — is part of being a competent operator.

What you’ll learn

  • Why MVCC produces dead rows and what VACUUM does about them
  • The difference between VACUUM, VACUUM FULL, and VACUUM ANALYZE
  • What ANALYZE actually computes
  • Reading pg_stat_user_tables to see autovacuum activity
  • The transaction-ID-wraparound horizon and why you cannot ignore it

Example

Example 53.1 VACUUM and ANALYZE
-- Inspect the per-table autovacuum stats
SELECT relname, n_live_tup, n_dead_tup,
       last_vacuum, last_autovacuum,
       last_analyze, last_autoanalyze
FROM pg_stat_user_tables
WHERE schemaname = 'public'
ORDER BY n_dead_tup DESC
LIMIT 5;

-- Manually vacuum and analyze a table
VACUUM (VERBOSE) fruits;
ANALYZE VERBOSE fruits;

-- Combined: VACUUM ANALYZE (most common manual invocation)
VACUUM ANALYZE fruits;

-- See the planner's stored statistics for a column
SELECT attname, n_distinct, most_common_vals, most_common_freqs
FROM pg_stats
WHERE schemaname = 'public' AND tablename = 'fruits';

-- Bloat estimate (approximation): pg_stat_user_tables gives live vs dead row counts
-- pgstattuple extension gives exact numbers if you need them.

-- VACUUM FULL: rewrites the whole table, fully reclaims space, takes an exclusive lock.
-- Use sparingly.
-- VACUUM FULL fruits;

MVCC is what lets PostgreSQL achieve high concurrency: writers do not block readers and vice versa, because each transaction sees a consistent snapshot. The cost is dead rows. VACUUM walks the table, marks dead rows’ space as free for reuse, and updates the visibility map (a per-page bitmap that lets index-only scans avoid heap visibility checks on all-visible pages).

Plain VACUUM normally makes space reusable within the table. It can also truncate empty pages at the end of the table and return that space to the operating system. VACUUM FULL rewrites the entire table in a new file, releasing the old one — it reclaims space but takes an ACCESS EXCLUSIVE lock and is slow. Use it rarely; the routine maintenance is autovacuum.

ANALYZE samples the table to update pg_statistic. The planner uses these statistics — number of distinct values, most common values, histograms — to estimate row counts and pick plans. Stale statistics produce bad plans.

PostgreSQL’s autovacuum daemon runs VACUUM and ANALYZE automatically based on per-table thresholds (autovacuum_vacuum_scale_factor, autovacuum_analyze_scale_factor). Autovacuum handles routine maintenance, but its progress and thresholds need monitoring, especially after bulk changes and on large or frequently updated tables.

Run the example

To run:

$ psql -f source/vacuum-and-analyze.sql postgres
 relname | n_live_tup | n_dead_tup | last_vacuum | ...
---------+------------+------------+-------------+----
 ...
INFO:  vacuuming "public.fruits"
INFO:  ...
VACUUM
 ...

Common pitfalls

  • Disabling autovacuum to “save resources” — almost always a mistake. The dead rows pile up, queries slow down, and the eventual catch-up is much worse than steady-state autovacuum overhead.
  • Running VACUUM FULL on a busy table — it blocks all reads and writes for the duration. Use pg_repack (extension) for online compaction.
  • Forgetting to ANALYZE after a bulk import — the planner has no statistics for the new rows and may pick terrible plans until autovacuum catches up. Run ANALYZE schema.table; manually after big imports.
  • Long-running transactions block autovacuum from cleaning up rows visible to them. A forgotten BEGIN; in psql can prevent vacuuming across the whole database — watch pg_stat_activity for state = 'idle in transaction'.
Tip

pg_stat_user_tables.n_dead_tup is the canonical “do I need to vacuum?” metric. A high ratio of n_dead_tup / n_live_tup on a hot table is a sign autovacuum cannot keep up — tune the per-table thresholds with ALTER TABLE t SET (autovacuum_vacuum_scale_factor = 0.02);.

Try it

Run a bunch of UPDATEs on a table, then look at n_dead_tup in pg_stat_user_tables. Run VACUUM and re-check. Then run EXPLAIN ANALYZE on a query before and after ANALYZE to see if the estimates changed.

Downloads

SQL file: vacuum-and-analyze.sql · All lesson files · Setup guide

54 pg_dump and pg_restore

Back up and restore scratch databases using plain and custom-format logical dumps.

pg_dump is the standard PostgreSQL backup tool. It produces a consistent snapshot of a single database, either as a plain SQL script or as a custom binary archive. Its companion pg_restore reads the binary archives back. For full-cluster backups (pg_dumpall), per-tablespace data, and point-in-time recovery, PostgreSQL offers separate tools — but pg_dump covers the great majority of “back up my database” needs. It is what you reach for first.

What you’ll learn

  • The two output formats (plain .sql and custom .dump)
  • pg_dump and pg_restore flags worth memorizing
  • Dumping schema only vs data only vs both
  • Including/excluding specific tables or schemas
  • Restoring in parallel for speed

Setup. These are shell commands, not SQL. Run them from your terminal. The source file for this lesson is a thin wrapper that documents the commands.

Example

Example 54.1 pg_dump and pg_restore
# Plain-text SQL dump (human-readable, runnable with psql)
pg_dump --dbname=postgres --format=plain --file=backup.sql

# Custom-format dump (compressed binary, restorable with pg_restore)
pg_dump --dbname=postgres --format=custom --file=backup.dump

# Schema only (DDL, no data) — useful for reproducing the shape of a database
pg_dump --schema-only --dbname=postgres --file=schema.sql

# Data only (no DDL)
pg_dump --data-only --dbname=postgres --file=data.sql

# Dump only specific tables (or use --exclude-table)
pg_dump --table=fruits --table=orders_example --dbname=postgres --file=subset.sql

# Dump only one schema
pg_dump --schema=app --dbname=postgres --file=app_schema.dump --format=custom

# Restore from a custom-format dump
pg_restore --dbname=postgres --jobs=4 --clean --if-exists backup.dump

# Restore a plain SQL dump
psql --dbname=postgres --file=backup.sql

# Dump every database in the cluster, plus roles and globals
pg_dumpall --file=cluster.sql

The custom format (--format=custom or -Fc) is what most teams use. It is compressed by default, supports parallel restore (--jobs=N), and lets you selectively restore individual objects (pg_restore --list backup.dump to inspect, then --use-list to pick what to restore). Plain format is easier to read and grep, but less flexible.

pg_dump uses a consistent snapshot and takes ACCESS SHARE locks on the tables it dumps. Concurrent data changes do not invalidate that snapshot and ordinary writers can continue. Conflicting DDL may wait or cause a dump to fail; parallel dumps use synchronized snapshots.

--clean --if-exists drops objects included in the archive before recreating them; it does not remove every unrelated object or guarantee an error-free restore. Restore into a fresh scratch database for a predictable result. --jobs=4 can speed up a custom- or directory-format restore.

For production backups, pg_dump is part of the strategy but not the whole story. Continuous archiving (archive_mode = on) plus base backups (pg_basebackup) give you point-in-time recovery. Cloud-managed PostgreSQL handles this transparently.

Run the example

To run (this is a shell example — see the SQL file for the documented commands):

$ pg_dump --dbname=postgres --format=custom --file=backup.dump
$ pg_restore --dbname=postgres --clean --if-exists --jobs=4 backup.dump

Common pitfalls

  • Use pg_dumpall --globals-only for roles and tablespaces. Neither it nor pg_dump backs up replication slots; record and recreate those separately as needed.
  • Restoring into a database that already has the schema produces conflicts. Use --clean --if-exists, drop and recreate the database, or restore into an empty one.
  • A plain SQL dump restored with psql does not stop on the first error by default — pass -v ON_ERROR_STOP=1 to make it fail loud.
  • Version compatibility: pg_dump cannot dump a server newer than its own major version. Dumps are intended for the same or newer PostgreSQL major version; restoring to an older major is not guaranteed. For upgrades, use the target major’s pg_dump if it supports the source server.
Tip

pg_dump -t my_table -Fc db | pg_restore -t my_table -d other_db copies one table from one database to another in a single pipeline — no intermediate file. Handy for moving snapshots around.

Try it

Dump your scratch database, drop a table, then restore from the dump. Run pg_restore --list backup.dump to see the manifest of objects. Try pg_restore --schema-only to recreate just the structure on a fresh database.

Downloads

SQL file: pg-dump-and-restore.sql · All lesson files · Setup guide

55 Roles and GRANT

Grant object privileges and demonstrate row-level security with an ordinary database role.

PostgreSQL has a single concept — the role — for both users and groups. A role can log in (with LOGIN), be granted into other roles (for group-like behavior), own objects, and have privileges granted to it. GRANT adds privileges; REVOKE removes them. Every database object has an owner with full rights; everyone else needs explicit grants. The principle is least privilege: give each role only what it needs.

What you’ll learn

  • Creating roles with and without login
  • Granting and revoking privileges on schemas, tables, and columns
  • The USAGE privilege on schemas (often forgotten)
  • DEFAULT PRIVILEGES for objects created in the future
  • Row-level security (RLS) — a brief introduction

Example

Example 55.1 Roles and GRANT
-- Inspect the current session
SELECT current_user, session_user;

-- Create ordinary roles (requires CREATEROLE or superuser privileges)
-- CREATE ROLE app_read NOLOGIN;                                 -- a "group" role
-- CREATE ROLE alice LOGIN PASSWORD 'change-me' IN ROLE app_read;
-- CREATE ROLE app_write NOLOGIN;

-- Grant access to a schema and read on tables
-- GRANT USAGE  ON SCHEMA public TO app_read;
-- GRANT SELECT ON ALL TABLES IN SCHEMA public TO app_read;

-- Future tables in this schema also get SELECT for app_read
-- ALTER DEFAULT PRIVILEGES IN SCHEMA public
--   GRANT SELECT ON TABLES TO app_read;

-- Write-level
-- GRANT app_read TO app_write;                          -- inheritance: app_write also reads
-- GRANT INSERT, UPDATE, DELETE ON fruits TO app_write;

-- Revoke
-- REVOKE INSERT ON fruits FROM app_write;

-- Inspect your privileges on a table
SELECT grantee, privilege_type
FROM information_schema.role_table_grants
WHERE table_name = 'fruits' AND grantee = current_user;

-- Row-level security: per-row visibility decided by a policy
-- (demo on a separate table so we don't change `fruits` permissions)
DROP TABLE IF EXISTS tenant_docs;
CREATE TABLE tenant_docs (
  id       integer PRIMARY KEY,
  tenant   text NOT NULL,
  content  text NOT NULL
);
INSERT INTO tenant_docs VALUES (1, 'acme', 'hello'), (2, 'globex', 'world');

ALTER TABLE tenant_docs ENABLE ROW LEVEL SECURITY;
-- Apply the policy to the table owner as well (superusers/BYPASSRLS still bypass it).
ALTER TABLE tenant_docs FORCE ROW LEVEL SECURITY;
CREATE POLICY tenant_isolation ON tenant_docs
  USING (tenant = current_setting('app.tenant', true));

-- Set the tenant for the session and try a select
SET app.tenant TO 'acme';
SELECT * FROM tenant_docs;        -- only sees the 'acme' row

SET app.tenant TO 'globex';
SELECT * FROM tenant_docs;        -- only sees the 'globex' row

-- Cleanup
DROP POLICY IF EXISTS tenant_isolation ON tenant_docs;
DROP TABLE IF EXISTS tenant_docs;

Table privileges include SELECT, INSERT, UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER, and, in PostgreSQL 17, MAINTAIN. On a schema, the most important are USAGE (lets a role refer to objects inside) and CREATE (lets the role create new objects). On a database, CONNECT, CREATE, and TEMPORARY matter.

Privileges on tables are not inherited from schema-level grants. To use a table in schema public, a role needs USAGE on public and a privilege (e.g. SELECT) on the table. Missing the schema-level USAGE produces the common “permission denied for schema public” error.

DEFAULT PRIVILEGES solves the “new tables get no grants” problem: any object created later by a role inherits the configured grants automatically. They apply to objects created by the configured creating role, not automatically to objects created by every role. Configure each migration/owner role that creates tables.

Row-level security (RLS) is a more advanced feature: you ENABLE ROW LEVEL SECURITY on a table and write POLICYs that filter rows by predicate. With RLS, a query like SELECT * FROM tenant_docs returns only the rows visible to the current role/setting when the role is subject to the policy. A freely changeable app.tenant setting is only a demonstration: a real application must prevent untrusted clients from choosing another tenant.

Run the example

To run, use a non-superuser role without BYPASSRLS that can create tables in your scratch database. FORCE ROW LEVEL SECURITY makes this table owner obey the policy. Superusers still see both tenants. The commented role-management commands need appropriate role-administration privileges.

$ psql -f source/roles-and-grant.sql postgres
 current_user | session_user
--------------+--------------
 your_user    | your_user
(1 row)
 ...

Common pitfalls

  • Forgetting GRANT USAGE ON SCHEMA public. A role can have SELECT on every table and still fail to query because it lacks schema-level usage.
  • Granting privileges on existing tables without configuring DEFAULT PRIVILEGES — every new table the app creates is unreachable by the read-only role.
  • Mixing SECURITY DEFINER functions with a permissive search_path opens a privilege-escalation door. Always set search_path on such functions.
  • Storing passwords in CREATE ROLE ... PASSWORD 'plain' is acceptable for personal databases; for production use SCRAM authentication and never hardcode passwords in SQL scripts.
Tip

A common pattern is three roles: app_owner (owns objects, used for migrations), app_read (read-only role for analytics and replicas), app_write (read+write role for the application). Grant app_owner ownership; grant rights to the others.

Try it

With superuser access, create a role and grant it SELECT only on fruits. Reconnect as that role (psql -U newrole) and confirm reads work but writes do not. Then experiment with RLS: enable on a table, add a policy, and watch how SELECT results change with SET.

Downloads

SQL file: roles-and-grant.sql · All lesson files · Setup guide

56 psql Meta-commands

Explore objects, format results, run scripts, and customize sessions with psql commands.

psql’s meta-commands (also called backslash commands) are client-side shortcuts that make interactive exploration of a database fast and pleasant. They are interpreted by the client, not the server — and work both interactively and in files read with psql -f. Once they become muscle memory, you stop reaching for GUI tools for most day-to-day work.

What you’ll learn

  • The essential exploration commands (\l, \dt, \d, \df, \dn)
  • Help shortcuts (\?, \h)
  • Output and formatting commands (\x, \timing, \pset)
  • Running scripts and saving output (\i, \o, \copy)
  • Customizing your session with ~/.psqlrc

Example

Example 56.1 psql Meta-commands
psql command reference (backslash commands also work in psql -f scripts):

Exploration
  \l                    list databases
  \c dbname             connect to a database
  \dn                   list schemas
  \dt                   list tables in the current search_path
  \dt schema.*          list tables in a specific schema
  \dt *.*               list every table everywhere
  \d table              describe a table (columns, types, indexes, constraints)
  \d+ table             describe a table verbosely (storage, comments)
  \di                   list indexes
  \dv                   list views
  \dm                   list materialized views
  \df pattern           list functions whose name matches
  \du                   list roles (users/groups)
  \dx                   list installed extensions

Help
  \?                    list all meta-commands
  \h                    list SQL commands
  \h SELECT             help on the SELECT statement

Session/output
  \x                    toggle expanded display (column-per-line)
  \timing               toggle showing query timing
  \pset pager off       disable the less-style pager
  \pset format aligned  default table format; alternatives include csv, html, unaligned
  \a                    toggle "unaligned" mode (use format csv for CSV escaping)
  \o file               redirect query output to a file (\o without arg disables)

Files and scripts
  \i file.sql           run a SQL script from inside psql
  \ir file.sql          run relative to the current script
  \copy table FROM 'file.csv' WITH (FORMAT csv, HEADER true)  client-side import
  \copy (SELECT ...) TO 'file.csv' WITH (FORMAT csv, HEADER true)  client-side export

Editing
  \e                    open the last query in $EDITOR (or write a new one)
  \ef function_name     open a function definition in $EDITOR
  \ev view_name         open a view definition in $EDITOR
  \q                    quit

psql -f script.sql processes both SQL and backslash commands. The SQL file linked below shows catalog-query alternatives, useful when the caller is a database driver rather than psql. Do not paste the descriptive labels in this command reference into a script.

The single most useful command is \d table (and its verbose cousin \d+ table). It shows columns, types, defaults, indexes, constraints, foreign keys, and triggers — everything you need to understand a table’s shape.

The single most useful toggle is \x (expanded display). On a row with 30 columns, the default tabular output is unreadable; expanded display prints column | value one per line. Use \x auto to switch only when the row is too wide for the terminal.

\copy is the client-side equivalent of COPY. It reads/writes files on the client machine; COPY reads/writes files on the server. For loading a CSV into a remote PostgreSQL, \copy is what you want.

Run the example

To run the SQL part of this lesson:

$ psql -f source/psql-meta-commands.sql postgres
 ...
 current_database | current_user | current_schema
------------------+--------------+----------------
 postgres         | your_user    | public
(1 row)
 ...

Common pitfalls

  • Sending \dt directly through a database driver fails because it is psql syntax. It works interactively and in a file executed by psql -f.
  • \d (no arguments) lists everything in the current search_path — long output. Filter: \dt, \di, \dv, etc.
  • \copy and COPY: \copy runs on the client (so relative paths are client paths and CSV files come from your machine); COPY runs on the server. Pick the right one based on where the file lives.
  • Your psqlrc runs at every session start. A broken statement there can make psql unusable until you fix the file.
Tip

~/.psqlrc is where to keep your customizations. Common entries:

\set PROMPT1 '%n@%/%x%R%# '
\set HISTSIZE 5000
\timing on
\pset null '∅'
\set COMP_KEYWORD_CASE upper
Try it

From an interactive psql postgres session, run \dt, \d fruits, \df *agg*, then \h SELECT. Toggle \x and run SELECT * FROM pg_indexes LIMIT 1; to see expanded output. Run a query and pipe to a file: \o out.txt, SELECT * FROM fruits;, \o (to stop redirecting).

Downloads

SQL file: psql-meta-commands.sql · All lesson files · Setup guide

57 COPY

Import and export CSV data using server-side COPY and client-side psql copy commands.

COPY is PostgreSQL’s bulk-data import/export command. It moves rows between a table and a file (or standard input/output) in CSV, text, or binary format. Compared to INSERT, COPY is dramatically faster — sometimes 10–100x — because it skips much of the per-row overhead. For loading large amounts of data, COPY is what you reach for.

What you’ll learn

  • COPY ... TO for export and COPY ... FROM for import
  • The difference between COPY (server-side) and \copy (client-side)
  • CSV format and the HEADER, QUOTE, DELIMITER options
  • Streaming via STDIN / STDOUT
  • Why COPY beats INSERT for bulk loads

Example

Example 57.1 COPY
-- Export to STDOUT in CSV (the rows go to your terminal when run via `psql -f`)
COPY (SELECT id, name FROM fruits ORDER BY id) TO STDOUT WITH (FORMAT csv, HEADER true);

-- Export a whole table
-- (commented because most workflows use the CTAS / pg_dump path instead)
-- COPY fruits TO STDOUT WITH (FORMAT csv, HEADER true);

-- Server-side import: the SERVER must read the path.
-- This is rarely what you want unless you control the server filesystem.
-- COPY fruits FROM '/srv/data/fruits.csv' WITH (FORMAT csv, HEADER true);

-- Client-side import: the psql CLIENT reads the file from your machine.
-- In interactive psql:
-- \copy fruits FROM '/Users/you/Desktop/fruits.csv' WITH (FORMAT csv, HEADER true)

-- COPY can also stream from STDIN — used by drivers for bulk loading.
-- Example shows the protocol; the actual rows would follow the command in a stream.
-- COPY fruits FROM STDIN WITH (FORMAT csv, HEADER true);

COPY ... TO STDOUT writes to the connection’s output stream. When you run a SQL file with psql -f, that stream is your terminal — handy for ad-hoc exports. Redirect from the shell to save to a file: psql -f source/copy.sql postgres > out.csv.

COPY ... FROM <path> reads from a file on the server. The PostgreSQL process needs read access; on a managed cloud database, you usually do not. The client-side \copy (a psql meta-command) reads the file on your machine, streams it through the connection, and inserts it on the server — exactly the same behavior as COPY FROM STDIN, just wrapped for convenience.

For application code doing bulk loads, almost every driver exposes a “COPY FROM STREAM” API (psycopg’s copy_from, pgx’s CopyFrom, etc.). For a few thousand rows it is much faster than batched INSERTs; it is often the best starting point for millions of rows, though the benefit depends on the workload.

COPY FROM invokes triggers and enforces check constraints and foreign keys on the destination table. Trigger work therefore contributes to load cost. It does not invoke rules. A staging table is useful for validating and transforming a load before inserting into the final table.

Run the example

To run:

$ psql -f source/copy.sql postgres
id,name
1,apricot
2,blueberry
3,chestnut
4,damson
5,elderberry
101,fruit_1
102,fruit_2
103,fruit_3

Common pitfalls

  • Mixing up COPY and \copy: the first runs on the server (paths are server paths, permissions are server permissions); the second runs on the client (paths are local). A common mistake is writing COPY ... FROM '~/file.csv' and getting “permission denied” because the server cannot read your home directory.
  • Encoding mismatches: The file uses the ENCODING option, or the current client encoding when that option is absent. Mis-encoded CSVs produce garbled text or “invalid byte sequence” errors. Use WITH (ENCODING 'utf8') or convert beforehand.
  • CSVs with embedded newlines need QUOTE and ESCAPE options. The defaults (" for both) usually work.
  • COPY with constraints on a huge load is slow. Trick: load into a staging table, deduplicate / validate, then INSERT ... ON CONFLICT into the real table.
Tip

For very large imports, drop indexes (except the PK) before COPY, then recreate. The COPY is faster, and a single index build at the end is faster than maintaining indexes per-row across millions of rows. Wrap in a transaction so failure rolls back.

Try it

Export fruits to a file: psql -f source/copy.sql postgres > out.csv. Then import it back into a staging table: create fruits_stage in one psql session, then run \copy on its own line:

CREATE TEMP TABLE fruits_stage (id integer, name text);
\copy fruits_stage FROM 'out.csv' WITH (FORMAT csv, HEADER true)
SELECT count(*) FROM fruits_stage;

Compare the counts.

Downloads

SQL file: copy.sql · All lesson files · Setup guide

Back to Table of Contents

Editorial Corrections

Corrections to the pinned English source edition, with PostgreSQL references.

This web edition corrects technical errors in the pinned source, and marks the adaptation in each page’s attribution. The original repository and its license remain available from Publication and License.

Chapter order is unchanged. The three comma-containing titles are recovered from their chapter headings, and OINK provides the sequential chapter navigation.

Displayed SQL and downloadable SQL use the same corrected example. Chapters 54 and 56 instead pair a shell example or command reference with a SQL companion.

Console output is illustrative: dates, UUIDs, users, catalog listings, statistics, and query plans depend on the environment. Ellipses mark omitted output.

1. First Query

Read the corrected lesson.

The client/server version explanation now distinguishes the version reported by the connected server from the local psql binary.

PostgreSQL documentation: reference 1.

2. psql Basics

Read the corrected lesson.

The psql section now explains that -f processes backslash commands and that these commands are client syntax which may issue catalog queries.

PostgreSQL documentation: reference 1.

3. SELECT Basics

Read the corrected lesson.

The SELECT wording no longer promises an evaluation order that SQL does not guarantee.

PostgreSQL documentation: reference 1.

4. WHERE

Read the corrected lesson.

The runnable series query is fixed, and the ILIKE and NULL predicate guidance now reflects PostgreSQL’s actual planner and three-valued-logic behavior.

PostgreSQL documentation: reference 1, reference 2, reference 3.

7. LIMIT and OFFSET

Read the corrected lesson.

The OFFSET cost example now describes the number of rows processed without an off-by-one claim.

PostgreSQL documentation: reference 1.

8. DISTINCT

Read the corrected lesson.

The DISTINCT example now correctly explains that parentheses around a selected expression do not change the distinct key.

PostgreSQL documentation: reference 1.

9. NULLs

Read the corrected lesson.

The NULL comparison explanation now separates unknown comparisons, DISTINCT grouping, and IS DISTINCT FROM.

PostgreSQL documentation: reference 1.

10. Expressions

Read the corrected lesson.

The cast example uses an unambiguous output name.

PostgreSQL documentation: reference 1.

11. Numeric Types

Read the corrected lesson.

Numeric ranges, overflow behavior, and unconstrained numeric limits are stated more precisely.

PostgreSQL documentation: reference 1.

12. Text Types

Read the corrected lesson.

The character padding example now distinguishes character length from stored byte length and labels the collation example accurately.

PostgreSQL documentation: reference 1.

13. Boolean and Dates

Read the corrected lesson.

The date/time section now describes timestamptz display semantics and current_timestamp versus clock_timestamp correctly.

PostgreSQL documentation: reference 1.

14. UUID and JSONB

Read the corrected lesson.

UUIDv7 availability is version-scoped: PostgreSQL 17 has no built-in generator, while PostgreSQL 18 provides uuidv7().

PostgreSQL documentation: reference 1.

15. Arrays and ENUM

Read the corrected lesson.

The array indexing text acknowledges PostgreSQL’s default lower bound without overstating the rule.

PostgreSQL documentation: reference 1.

17. Column Types and Constraints

Read the corrected lesson.

Constraint placement is described accurately for composite constraints and multi-column CHECK expressions.

PostgreSQL documentation: reference 1.

18. ALTER TABLE and DROP

Read the corrected lesson.

ALTER TABLE guidance now distinguishes changes that require a table rewrite from binary-compatible changes and describes rename dependencies more carefully.

PostgreSQL documentation: reference 1.

19. Primary Keys and UNIQUE

Read the corrected lesson.

The NULLS NOT DISTINCT recommendation no longer presents a partial unique index as an equivalent solution.

PostgreSQL documentation: reference 1, reference 2.

21. NOT NULL and DEFAULT

Read the corrected lesson.

The table-default discussion distinguishes nonvolatile current_timestamp from volatile defaults such as random().

PostgreSQL documentation: reference 1, reference 2.

22. Identity Columns and Sequences

Read the corrected lesson.

Sequence objects are shared across sessions; currval() state is the session-local part.

PostgreSQL documentation: reference 1.

23. Schemas and search_path

Read the corrected lesson.

SECURITY DEFINER and search_path guidance now uses a trusted path with pg_temp last, and schema drops are described as transactionally reversible before commit.

PostgreSQL documentation: reference 1, reference 2.

25. UPSERT (ON CONFLICT)

Read the corrected lesson.

The ON CONFLICT examples now use the actual constraint name and qualify the partial-index caveat.

PostgreSQL documentation: reference 1.

26. UPDATE

Read the corrected lesson.

The update lesson now uses IS DISTINCT FROM for a NULL-safe change test.

PostgreSQL documentation: reference 1.

27. DELETE

Read the corrected lesson.

DELETE and TRUNCATE behavior now distinguishes row-level triggers, foreign-key checks, locking, WAL, and rollback semantics.

PostgreSQL documentation: reference 1, reference 2, reference 3.

28. RETURNING

Read the corrected lesson.

RETURNING syntax is scoped to PostgreSQL 17, with the PostgreSQL 18 old/new extension identified separately.

PostgreSQL documentation: reference 1, reference 2.

29. INNER and LEFT JOIN

Read the corrected lesson.

The join warning no longer says PostgreSQL warns about valid cross joins or treats a missing INNER JOIN condition as valid syntax.

PostgreSQL documentation: reference 1.

32. UNION, INTERSECT, EXCEPT

Read the corrected lesson.

The ALL variants of UNION, INTERSECT, and EXCEPT are described with their different duplicate rules.

PostgreSQL documentation: reference 1.

33. COUNT, SUM, AVG

Read the corrected lesson.

Aggregate NULL behavior is no longer generalized to aggregates such as array_agg and jsonb_agg that retain NULL inputs.

PostgreSQL documentation: reference 1.

35. HAVING

Read the corrected lesson.

The WHERE/HAVING performance guidance now leaves room for PostgreSQL to push suitable predicates down.

PostgreSQL documentation: reference 1.

36. Scalar and IN Subqueries

Read the corrected lesson.

A scalar subquery is correctly described as returning at most one row, with zero rows producing NULL.

PostgreSQL documentation: reference 1, reference 2.

37. EXISTS and Derived Tables

Read the corrected lesson.

Derived-table alias guidance reflects PostgreSQL 16’s optional alias behavior while retaining the portability recommendation.

PostgreSQL documentation: reference 1.

38. Common Table Expressions (WITH)

Read the corrected lesson.

CTE folding, recursive UNION choices, nontermination, and data-modifying CTE ordering are described without unsupported guarantees.

PostgreSQL documentation: reference 1.

39. Window Functions

Read the corrected lesson.

Window-frame defaults and the need for an explicit lag/lead order are stated precisely.

PostgreSQL documentation: reference 1, reference 2.

40. String and Numeric Functions

Read the corrected lesson.

The examples distinguish random integer generation from a direct floating-point cast and correct the rounding description.

PostgreSQL documentation: reference 1.

41. Date Functions and COALESCE

Read the corrected lesson.

Date bucketing and extract(epoch) are aligned with PostgreSQL 17’s return types and date functions.

PostgreSQL documentation: reference 1.

42. CASE

Read the corrected lesson.

FILTER is compared with CASE without claiming identical empty-result behavior.

PostgreSQL documentation: reference 1.

43. CREATE INDEX

Read the corrected lesson.

Multicolumn index guidance reflects modern planner options, lock modes, and HOT-update behavior.

PostgreSQL documentation: reference 1, reference 2, reference 3.

44. When to Index

Read the corrected lesson.

The index-selection guidance removes a universal selectivity percentage and an overbroad redundancy rule.

PostgreSQL documentation: reference 1, reference 2, reference 3.

45. EXPLAIN and Query Plans

Read the corrected lesson.

The EXPLAIN aggregate counts actual child rows, and index-only scans now mention visibility-map heap checks.

PostgreSQL documentation: reference 1, reference 2.

46. BEGIN, COMMIT, ROLLBACK

Read the corrected lesson.

Transaction durability and dry-run limitations now mention normal durability settings, sequence increments, locks, and external effects; the two-session exercise recreates its demo table first.

PostgreSQL documentation: reference 1, reference 2, reference 3.

48. CREATE VIEW

Read the corrected lesson.

CREATE OR REPLACE VIEW restrictions are separated from ALTER VIEW column renames.

PostgreSQL documentation: reference 1.

49. Materialized Views

Read the corrected lesson.

REFRESH MATERIALIZED VIEW CONCURRENTLY is described as a full-query refresh requiring a populated matview and a suitable unique index.

PostgreSQL documentation: reference 1, reference 2.

50. Functions and PL/pgSQL

Read the corrected lesson.

Function volatility, SECURITY DEFINER search paths, and PL/pgSQL statement syntax are corrected.

PostgreSQL documentation: reference 1, reference 2.

51. Triggers

Read the corrected lesson.

Trigger return values now distinguish INSERT/UPDATE from DELETE, and the exercise accounts for the script’s cleanup.

PostgreSQL documentation: reference 1.

52. Full-Text Search

Read the corrected lesson.

The full-text-search exercise tells readers to run it before the script drops its demonstration table.

PostgreSQL documentation: reference 1, reference 2.

53. VACUUM and ANALYZE

Read the corrected lesson.

VACUUM’s visibility-map role, tail-page truncation, and autovacuum guidance are more exact.

PostgreSQL documentation: reference 1.

54. pg_dump and pg_restore

Read the corrected lesson.

pg_dump snapshot, globals, version compatibility, and restore behavior are corrected and qualified.

PostgreSQL documentation: reference 1, reference 2.

55. Roles and GRANT

Read the corrected lesson.

The role and RLS example now forces row security for the table owner and documents the limits of a freely changeable tenant setting.

PostgreSQL documentation: reference 1, reference 2, reference 3, reference 4.

56. psql Meta-commands

Read the corrected lesson.

The psql command reference now accurately covers files executed with -f, SQL alternatives, and CSV output mode.

PostgreSQL documentation: reference 1.

57. COPY

Read the corrected lesson.

COPY FROM trigger behavior, encoding selection, bulk-load performance, and the executable multiline \copy exercise are corrected.

PostgreSQL documentation: reference 1, reference 2.

Publication and License

Authorship, source revision, license, and editorial changes for Postgres by Example.

Postgres by Example is by Dariush Abbasi (dariubs). The source edition is maintained at github.com/boringcollege/postgres-by-example.

The book and its runnable examples are available under the Creative Commons Attribution 4.0 International License. See the complete license text and the license file included with this web edition.

This English web edition follows source revision ffde0d358722498e37982b57bdbd711871a61c43. It adds OINK book navigation, stable chapter URLs, numbered examples, a whole-book print view, Markdown exports, and a downloadable SQL bundle.

The edition includes technical corrections to the prose, SQL, and illustrative output. Displayed SQL and the corresponding downloads are kept in sync. These editorial changes are marked in the page attribution; they do not represent changes made by the original author.

The upstream license file begins with the title “Python by Example”, apparently a copied heading. The file is preserved verbatim for provenance. The source repository’s README and book metadata identify Postgres by Example and the CC BY 4.0 license.