From fa96ca764338346be759bdbca22f3f2ede19e24e Mon Sep 17 00:00:00 2001 From: developerjamiu Date: Mon, 20 Apr 2026 12:30:12 +0100 Subject: [PATCH 1/4] docs: Add JSONB column support and GIN index documentation --- docs/06-concepts/06-database/02-models.md | 67 ++++++++++++++++++-- docs/06-concepts/06-database/04-indexing.md | 69 ++++++++++++++++++--- docs/06-concepts/07-configuration.md | 23 +++---- 3 files changed, 136 insertions(+), 23 deletions(-) diff --git a/docs/06-concepts/06-database/02-models.md b/docs/06-concepts/06-database/02-models.md index cd2e3d6a..6cb6a18c 100644 --- a/docs/06-concepts/06-database/02-models.md +++ b/docs/06-concepts/06-database/02-models.md @@ -17,7 +17,7 @@ When you add a `table` to a serializable class, Serverpod will automatically add ::: -### Non persistent fields +## Non persistent fields You can opt out of creating a column in the database for a specific field by using the `!persist` keyword. @@ -25,14 +25,14 @@ You can opt out of creating a column in the database for a specific field by usi class: Company table: company fields: - name: String, !persist + name: String, !persist ``` All fields are persisted by default and have an implicit `persist` set on each field. -### Data representation +## Data representation -Storing a field with a primitive / core dart type will be handled as its respective type. However, if you use a complex type, such as another model, a `List`, or a `Map`, these will be stored as a `json` object in the database. +Storing a field with a primitive / core dart type will be handled as its respective type. However, if you use a complex type, such as another model, a `List`, or a `Map`, these will be stored as a `json` column in the database by default. ```yaml class: Company @@ -54,6 +54,65 @@ fields: For a complete guide on how to work with relations see the [relation section](relations/one-to-one). +### Storing fields as JSONB + +By default, complex types are stored as `json` in PostgreSQL. You can opt into `jsonb` storage instead using the `serializationDataType` keyword. JSONB is a binary format that supports efficient querying and [GIN indexing](indexing#gin-indexes). + +You can set `serializationDataType` at three levels, each overriding the one above it: + +#### Field level + +Applies to a single field: + +```yaml +class: Product +table: product +fields: + tags: List, serializationDataType=jsonb + metadata: Map, serializationDataType=jsonb +``` + +#### Class level + +Applies to all serializable fields in the class: + +```yaml +class: Product +table: product +serializationDataType: jsonb +fields: + tags: List # Stored as jsonb + metadata: Map # Stored as jsonb + name: String # Not affected — primitive types are not serialized +``` + +Individual fields can still override the class-level setting: + +```yaml +class: Product +table: product +serializationDataType: jsonb +fields: + tags: List # jsonb (from class) + metadata: Map, serializationDataType=json # json (field override) +``` + +#### Project level + +Applies to all models in the project. Add this to your `config/generator.yaml`: + +```yaml +serialize_as_jsonb_by_default: true +``` + +When enabled, all serializable fields across all models default to `jsonb` unless overridden at the class or field level. + +:::info +The `serializationDataType` keyword is only valid on serializable field types (models, Lists, Maps). Primitive types like `String` and `int` have their own native database column types and are not affected by this setting. +::: + +The three levels follow a **field > class > project** precedence: a field-level setting always wins over class-level, and class-level always wins over project-level. If no level specifies a value, the default is `json`. + ## Change ID type Changing the type of the `id` field allows you to customize the identifier type for your database tables. This is done by declaring the `id` field on table models with one of the supported types. If the field is omitted, the id field will still be created with type `int`, as have always been. diff --git a/docs/06-concepts/06-database/04-indexing.md b/docs/06-concepts/06-database/04-indexing.md index b2162e2d..8c58d485 100644 --- a/docs/06-concepts/06-database/04-indexing.md +++ b/docs/06-concepts/06-database/04-indexing.md @@ -2,7 +2,7 @@ For performance reasons, you may want to add indexes to your database tables. These are added in the YAML-files defining the serializable objects. -### Add an index +## Add an index To add an index, add an `indexes` section to the YAML-file. The `indexes` section is a map where the key is the name of the index and the value is a map with the index details. @@ -29,7 +29,7 @@ indexes: fields: name, foundedAt ``` -### Making fields unique +## Making fields unique Adding a unique index ensures that the value or combination of values stored in the fields are unique for the table. This can be useful for example if you want to make sure that no two companies have the same name. @@ -46,7 +46,7 @@ indexes: The `unique` keyword is a bool that can toggle the index to be unique, the default is set to false. If the `unique` keyword is applied to a multi-column index, the index will be unique for the combination of the fields. -### Specifying index type +## Specifying index type It is possible to add a type key to specify the index type. @@ -63,7 +63,60 @@ indexes: If no type is specified the default is `btree`. All [PostgreSQL index types](https://www.postgresql.org/docs/current/indexes-types.html) are supported, `btree`, `hash`, `gist`, `spgist`, `gin`, `brin`. -### Vector indexes +## GIN indexes + +GIN (Generalized Inverted Index) indexes are designed for efficiently querying composite values such as JSONB data. When all fields in an index are stored as `jsonb`, Serverpod automatically defaults the index type to `gin`: + +```yaml +class: Product +table: product +fields: + tags: List, serializationDataType=jsonb +indexes: + product_tags_idx: + fields: tags + # type defaults to gin since all indexed fields are jsonb +``` + +You can also set the type explicitly: + +```yaml +indexes: + product_tags_idx: + fields: tags + type: gin +``` + +### Operator classes + +GIN indexes support different operator classes that control which query operators the index can accelerate. Use the `operatorClass` keyword to specify one: + +```yaml +indexes: + product_tags_idx: + fields: tags + type: gin + operatorClass: jsonbPathOps +``` + +| Operator Class | Description | Use Case | +| -------------- | -------------------------------------------------- | --------------------------------------------------------------- | +| `jsonbOps` | Default. Supports `@>`, `?`, `?\|`, `?&` operators | General-purpose JSONB querying | +| `jsonbPathOps` | Supports only `@>` (containment) | Faster and smaller index when you only need containment queries | +| `arrayOps` | For array containment queries | Array-typed columns | +| `tsvectorOps` | For full-text search | Text search with `tsvector` columns | + +:::tip +If you only need containment queries (`@>`), use `jsonbPathOps` — it produces a smaller and faster index than the default `jsonbOps`. +::: + +:::info +GIN indexes are only supported on PostgreSQL. If your project targets SQLite, a validation error will be reported at generation time. +::: + +For details on configuring JSONB storage on your model fields, see [Storing fields as JSONB](models#storing-fields-as-jsonb). + +## Vector indexes To enhance the performance of vector similarity search, it is possible to create specialized vector indexes on vector fields (`Vector`, `HalfVector`, `SparseVector`, `Bit`). Serverpod supports both `hnsw` and `ivfflat` index types with full parameter specification. @@ -71,7 +124,7 @@ To enhance the performance of vector similarity search, it is possible to create Each vector index can only be created on a single vector field. It is not possible to create a vector index on multiple fields of any kind. ::: -#### HNSW indexes +### HNSW indexes Hierarchical Navigable Small World (HNSW) indexes provide fast approximate nearest neighbor search: @@ -112,7 +165,7 @@ Available HNSW parameters: - `m`: Maximum number of bidirectional links for each node (default: 16) - `ef_construction`: Size of the dynamic candidate list (default: 64) -#### IVFFLAT indexes +### IVFFLAT indexes Inverted File with Flat compression (IVFFLAT) indexes are suitable for large datasets: @@ -135,12 +188,12 @@ Available IVFFLAT parameters: - `lists`: Number of inverted lists (default: 100) -#### Distance functions +### Distance functions Supported distance functions for vector indexes (`distanceFunction` parameter): | Distance Function | Description | Use Case | -|-------------------|-------------------------------|------------------------------| +| ----------------- | ----------------------------- | ---------------------------- | | `l2` | Euclidean distance | Default for most embeddings | | `innerProduct` | Inner product | When vectors are normalized | | `cosine` | Cosine distance | Text embeddings | diff --git a/docs/06-concepts/07-configuration.md b/docs/06-concepts/07-configuration.md index 3ae4cae3..a1caca6a 100644 --- a/docs/06-concepts/07-configuration.md +++ b/docs/06-concepts/07-configuration.md @@ -288,17 +288,18 @@ While the above configurations control how your server runs, Serverpod also uses ### Generator configuration options -| Option | Type | Default | Description | -| ---------------------- | ------ | --------------------------- | ---------------------------------------------------------------------------------- | -| type | string | server | The package type. Valid options are `server`, `module`, or `internal`. | -| nickname | string | - | For modules only. Defines how the module is referenced in code. | -| client_package_path | string | ../[name]\_client | Path to the client package relative to the server. | -| server_test_tools_path | string | test/integration/test_tools | Path where test tools are generated. Remove this to disable test tools generation. | -| shared_packages | list | - | Paths to shared packages containing models usable by both server and client. | -| modules | map | - | Module dependencies with optional nicknames. | -| extraClasses | list | - | List of custom serializable classes to include in code generation. | -| features | map | \{database: true\} | Feature flags. Currently only `database` is supported. | -| experimental_features | map | - | Experimental features. Available keys: `all`, `inheritance`. | +| Option | Type | Default | Description | +| ----------------------------- | ------ | --------------------------- | ---------------------------------------------------------------------------------- | +| type | string | server | The package type. Valid options are `server`, `module`, or `internal`. | +| nickname | string | - | For modules only. Defines how the module is referenced in code. | +| client_package_path | string | ../[name]\_client | Path to the client package relative to the server. | +| server_test_tools_path | string | test/integration/test_tools | Path where test tools are generated. Remove this to disable test tools generation. | +| shared_packages | list | - | Paths to shared packages containing models usable by both server and client. | +| modules | map | - | Module dependencies with optional nicknames. | +| extraClasses | list | - | List of custom serializable classes to include in code generation. | +| serialize_as_jsonb_by_default | bool | false | When true, all serializable fields default to `jsonb` storage instead of `json`. | +| features | map | \{database: true\} | Feature flags. Currently only `database` is supported. | +| experimental_features | map | - | Experimental features. Available keys: `all`, `inheritance`. | ### Package types From 846a62ce6233546a2fd23d6ac2a12d1bcafec513 Mon Sep 17 00:00:00 2001 From: developerjamiu Date: Thu, 23 Apr 2026 17:29:05 +0100 Subject: [PATCH 2/4] fix: Address docs review comments --- docs/06-concepts/06-database/02-models.md | 28 ++++++++------------- docs/06-concepts/06-database/04-indexing.md | 2 +- docs/06-concepts/07-configuration.md | 2 +- 3 files changed, 12 insertions(+), 20 deletions(-) diff --git a/docs/06-concepts/06-database/02-models.md b/docs/06-concepts/06-database/02-models.md index 6cb6a18c..4283cf84 100644 --- a/docs/06-concepts/06-database/02-models.md +++ b/docs/06-concepts/06-database/02-models.md @@ -54,10 +54,14 @@ fields: For a complete guide on how to work with relations see the [relation section](relations/one-to-one). -### Storing fields as JSONB +### Storing serializable fields as JSONB By default, complex types are stored as `json` in PostgreSQL. You can opt into `jsonb` storage instead using the `serializationDataType` keyword. JSONB is a binary format that supports efficient querying and [GIN indexing](indexing#gin-indexes). +:::info +The `serializationDataType` keyword is only valid on serializable field types (models, Lists, Maps). Primitive types like `String` and `int` have their own native database column types and are not affected by this setting. +::: + You can set `serializationDataType` at three levels, each overriding the one above it: #### Field level @@ -74,7 +78,7 @@ fields: #### Class level -Applies to all serializable fields in the class: +Applies to all serializable fields in the class. Can be overridden by field-level setting. ```yaml class: Product @@ -83,18 +87,8 @@ serializationDataType: jsonb fields: tags: List # Stored as jsonb metadata: Map # Stored as jsonb - name: String # Not affected — primitive types are not serialized -``` - -Individual fields can still override the class-level setting: - -```yaml -class: Product -table: product -serializationDataType: jsonb -fields: - tags: List # jsonb (from class) - metadata: Map, serializationDataType=json # json (field override) + name: String + history: List, serializationDataType=json # Stored as json (override) ``` #### Project level @@ -107,11 +101,9 @@ serialize_as_jsonb_by_default: true When enabled, all serializable fields across all models default to `jsonb` unless overridden at the class or field level. -:::info -The `serializationDataType` keyword is only valid on serializable field types (models, Lists, Maps). Primitive types like `String` and `int` have their own native database column types and are not affected by this setting. -::: +#### Migrating between json and jsonb -The three levels follow a **field > class > project** precedence: a field-level setting always wins over class-level, and class-level always wins over project-level. If no level specifies a value, the default is `json`. +If you change the `serializationDataType` between `json` and `jsonb` at any level, the migration system will convert existing columns automatically with no data loss. ## Change ID type diff --git a/docs/06-concepts/06-database/04-indexing.md b/docs/06-concepts/06-database/04-indexing.md index 8c58d485..f40d08f7 100644 --- a/docs/06-concepts/06-database/04-indexing.md +++ b/docs/06-concepts/06-database/04-indexing.md @@ -111,7 +111,7 @@ If you only need containment queries (`@>`), use `jsonbPathOps` — it produces ::: :::info -GIN indexes are only supported on PostgreSQL. If your project targets SQLite, a validation error will be reported at generation time. +GIN indexes are a PostgreSQL feature. On SQLite, GIN index definitions are silently skipped during migration generation. ::: For details on configuring JSONB storage on your model fields, see [Storing fields as JSONB](models#storing-fields-as-jsonb). diff --git a/docs/06-concepts/07-configuration.md b/docs/06-concepts/07-configuration.md index a1caca6a..7f0a5d87 100644 --- a/docs/06-concepts/07-configuration.md +++ b/docs/06-concepts/07-configuration.md @@ -299,7 +299,7 @@ While the above configurations control how your server runs, Serverpod also uses | extraClasses | list | - | List of custom serializable classes to include in code generation. | | serialize_as_jsonb_by_default | bool | false | When true, all serializable fields default to `jsonb` storage instead of `json`. | | features | map | \{database: true\} | Feature flags. Currently only `database` is supported. | -| experimental_features | map | - | Experimental features. Available keys: `all`, `inheritance`. | +| experimental_features | map | - | Experimental features. No experimental features are currently available. | ### Package types From 7ea2c80e9e020db4935a45bb57e19e4672f37fe0 Mon Sep 17 00:00:00 2001 From: developerjamiu Date: Thu, 23 Apr 2026 17:43:00 +0100 Subject: [PATCH 3/4] fix: Update broken anchor after heading rename --- docs/06-concepts/06-database/04-indexing.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/06-concepts/06-database/04-indexing.md b/docs/06-concepts/06-database/04-indexing.md index f40d08f7..9b2fe594 100644 --- a/docs/06-concepts/06-database/04-indexing.md +++ b/docs/06-concepts/06-database/04-indexing.md @@ -114,7 +114,7 @@ If you only need containment queries (`@>`), use `jsonbPathOps` — it produces GIN indexes are a PostgreSQL feature. On SQLite, GIN index definitions are silently skipped during migration generation. ::: -For details on configuring JSONB storage on your model fields, see [Storing fields as JSONB](models#storing-fields-as-jsonb). +For details on configuring JSONB storage on your model fields, see [Storing serializable fields as JSONB](models#storing-serializable-fields-as-jsonb). ## Vector indexes From 9957658aa9acc8ab31903012a318063ff80611b0 Mon Sep 17 00:00:00 2001 From: developerjamiu Date: Fri, 24 Apr 2026 04:21:45 +0100 Subject: [PATCH 4/4] fix: Update experimental features description per review --- docs/06-concepts/07-configuration.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/06-concepts/07-configuration.md b/docs/06-concepts/07-configuration.md index 7f0a5d87..f4999c87 100644 --- a/docs/06-concepts/07-configuration.md +++ b/docs/06-concepts/07-configuration.md @@ -288,18 +288,18 @@ While the above configurations control how your server runs, Serverpod also uses ### Generator configuration options -| Option | Type | Default | Description | -| ----------------------------- | ------ | --------------------------- | ---------------------------------------------------------------------------------- | -| type | string | server | The package type. Valid options are `server`, `module`, or `internal`. | -| nickname | string | - | For modules only. Defines how the module is referenced in code. | -| client_package_path | string | ../[name]\_client | Path to the client package relative to the server. | -| server_test_tools_path | string | test/integration/test_tools | Path where test tools are generated. Remove this to disable test tools generation. | -| shared_packages | list | - | Paths to shared packages containing models usable by both server and client. | -| modules | map | - | Module dependencies with optional nicknames. | -| extraClasses | list | - | List of custom serializable classes to include in code generation. | -| serialize_as_jsonb_by_default | bool | false | When true, all serializable fields default to `jsonb` storage instead of `json`. | -| features | map | \{database: true\} | Feature flags. Currently only `database` is supported. | -| experimental_features | map | - | Experimental features. No experimental features are currently available. | +| Option | Type | Default | Description | +| ----------------------------- | ------ | --------------------------- | ------------------------------------------------------------------------------------------- | +| type | string | server | The package type. Valid options are `server`, `module`, or `internal`. | +| nickname | string | - | For modules only. Defines how the module is referenced in code. | +| client_package_path | string | ../[name]\_client | Path to the client package relative to the server. | +| server_test_tools_path | string | test/integration/test_tools | Path where test tools are generated. Remove this to disable test tools generation. | +| shared_packages | list | - | Paths to shared packages containing models usable by both server and client. | +| modules | map | - | Module dependencies with optional nicknames. | +| extraClasses | list | - | List of custom serializable classes to include in code generation. | +| serialize_as_jsonb_by_default | bool | false | When true, all serializable fields default to `jsonb` storage instead of `json`. | +| features | map | \{database: true\} | Feature flags. Currently only `database` is supported. | +| experimental_features | map | - | Experimental features. Available keys: `all` (no experimental feature currently available). | ### Package types