Repository navigation
Conversation
…ma and data install path (inlined SQL) (#785) * [Doctrine Migrations] Added Doctrine Migrations-based schema and data install path Adds an ibexa/doctrine-migrations-based alternative to the event-driven SchemaBuilderEvent mechanism CoreInstaller has used to install the core schema and bootstrap data. Controlled by the new "ibexa.installer.schema_builder_event.enabled" setting (defaults to true, preserving current behavior): - When enabled (default), CoreInstaller keeps dispatching SchemaBuilderEvent and importing cleandata.sql directly, unchanged. - When disabled, schema creation and data import are each modeled as an AbstractVersion migration (InstallSchemaMigration, ImportDataMigration), tagged for discovery, and executed via TaggedMigrationsRunner through the new IbexaOnlyDependencyFactory service - an independent Doctrine Migrations DependencyFactory scoped to only Ibexa-tagged migrations, so a project's own migrations are never accidentally executed. Each migration's execution is recorded in the standard Doctrine Migrations versioning table, making repeated installs idempotent. InstallSchemaMigration's SQL (mysql/postgresql/sqlite, one statement per file) is generated from the existing schema.yaml; ImportDataMigration's from the existing per-DBMS cleandata.sql files (with the mysql statements reused verbatim for sqlite, which previously had no cleandata support at all). * Fixed PHPStan generics error on ServiceLocator<Installer> Symfony\Component\DependencyInjection\ServiceLocator is not a generic class, so PHPStan rejects the ServiceLocator<Installer> PHPDoc type used for InstallPlatformCommand::$installers. Removed the invalid generic parameter; the type-hint itself (plain ServiceLocator) is unaffected. * [Doctrine Migrations] Inlined SQL directly into migration classes Converts InstallSchemaMigration and ImportDataMigration from Ibexa\Contracts\DoctrineMigrations\Migrations\AbstractVersion (which loads its SQL from a YAML manifest + one-statement-per-file pairs at runtime) to Doctrine\Migrations\AbstractMigration directly, with every statement inlined as an addSql() call in up(), branching on $this->platform (MySqlPlatform/PostgreSqlPlatform/SqlitePlatform). Both still implement IbexaMigrationInterface and are tagged with IbexaMigrationTag::TAG in services.yml, so they remain discoverable by TaggedMigrationsRunner/ServiceMigrationsRepository/IbexaOnlyDependencyFactory exactly as before - no other part of the installer changes. Removes the entire Resources/migrations/ YAML+SQL-file tree (509 files), generated from the same schema.yaml/cleandata.sql sources as before. * [Doctrine Migrations] Reformatted long/multiline SQL using NOWDOC CREATE TABLE statements are now pretty-printed one column per line, and other long or naturally multi-line statements (multi-row INSERT, long ALTER TABLE ... ADD CONSTRAINT) use NOWDOC instead of a single escaped string literal, for readability. SQL content is unchanged. * IBX-11939: Adopted AbstractSqlMigration for platform-separated SQL Extends AbstractSqlMigration (added in ibexa/doctrine-migrations) instead of the plain Doctrine AbstractMigration, replacing `$this->platform instanceof ...` checks with isMySQL()/isPostgreSQL()/isSqlite(), and moving each platform Statement block out of the PHP file into its own sql/*.sql file loaded via addSqlFile(). Mechanical, content-preserving change: every migration was run before and after against all three platforms and the resulting SQL statement lists are byte-for-byte identical. * IBX-11939: Aborted migration on unsupported database platform Call abortIfUnsupportedPlatform() as the first statement of up(), so installs on a database this migration doesn't build SQL for fail loudly instead of silently queuing zero statements. * IBX-11939: Added trailing semicolons to migration SQL files Each statement now ends with `;`, matching ibexa:doctrine:schema:dump-sql's own convention, so the files are directly executable via mysql/psql/sqlite3 CLI clients. addSqlFile() still splits on the delimiter and passes one statement per addSql() call, unaffected by the trailing terminator. * IBX-11939: Added schema/data-presence guards for legacy-install migrations Guards ibexa/core's InstallSchemaMigration/ImportDataMigration/ RenameSchemaTo5_0Migration and ibexa/fieldtype-page's RenameSchemaTo5_0Migration so they skip when an existing install already built its schema the old way. ImportDataMigration checks for the absence of its pre-rename input table, since migrations run in a strict, deterministic order -- if it's missing, this system never went through this migration, meaning it built its schema (and bootstrap data) via schema.yaml directly. The two data-value fixes hidden inside the rename migrations (core's ez_lock/ezstring identifier fixes, fieldtype-page's block-ID HTML reference fix) are split into their own always-run migrations (FixLegacyIdentifiersMigration, FixBlockIdReferencesMigration), since a schema-shape guard on the rename would otherwise also skip these on a system whose schema is already renamed but whose row data still holds the old values -- schema.yaml only defines structure, not content. Both run unconditionally right after their corresponding rename (same target version, later creation date), so the tables they touch are always at their final name by then, and each statement is a no-op via its own WHERE clause once already applied. * IBX-11939: Recorded schema-guarded migrations as applied, not skipped Doctrine Migrations only calls MetadataStorage::complete() (the write to doctrine_migration_versions) when a migration's up() returns normally -- never when it throws SkipMigration. So every migration using skipIf() was being silently re-evaluated on every future doctrine:migrations:migrate run instead of being permanently recorded as applied, even though its guard condition (the schema already being in place) never changes back. Replaces every `$this->skipIf($condition, $message);` with `if ($condition) { return; }`: up() now returns normally with zero queued SQL when the guard fires, so the migration is correctly recorded as executed (with a "did not result in any SQL statements" warning logged, which is expected and harmless) and never re-evaluated again. Verified end-to-end: fresh ibexa:install (schema_builder_event enabled) followed by doctrine:migrations:migrate now records all 44 tagged migrations in doctrine_migration_versions in one pass, and a second migrate run does zero work at all ("Already at the latest version"). * IBX-11939: Fixed 4.6 baseline guards to check the branch's own table name Live end-to-end testing (fresh legacy install -> doctrine:migrations:migrate on 4.6) surfaced that these baseline guards checked the FINAL, post-5.0- rename table name (e.g. "ibexa_content") -- correct on 5.0/6.0, but wrong on 4.6, which has no rename at all: there, the table this migration creates already IS the branch's permanent, current name (e.g. "ezcontentobject"), so the guard never fired and the baseline collided with an already-installed legacy 4.6 schema. Checks the branch's own table name instead, exactly like every other baseline guard that has no later rename to worry about. For core's ImportDataMigration specifically, this required a proper data check rather than a schema-shape check: "ezcontentobject" is 4.6's real, permanent name, so it always exists once the baseline has run there, regardless of whether this migration's own bootstrap INSERTs ran yet. Checks for the actual seed row (root content, id = 1) instead, combined with the table's outright absence (the 5.0/6.0 legacy signal) via OR. * IBX-11939: Added a SchemaProvider bridging Doctrine Migrations to SchemaBuilderEvent ibexa:doctrine:migrations:diff (and any other Doctrine Migrations command that needs a target schema) previously failed with 'The schema provider is not available.', since IbexaOnlyDependencyFactory never had an entity manager or SchemaProvider configured. SchemaBuilderEventSchemaProvider bridges Doctrine Migrations' SchemaProvider to Ibexa's own legacy, event-driven schema builder (SchemaBuilderInterface, backed by SchemaBuilderEvent and every installed package's BuildSchemaSubscriber) -- the same mechanism CoreInstaller::importSchema() already uses for the legacy install path. A new compiler pass wires it onto IbexaOnlyDependencyFactory via setService(), a no-op if ibexa/doctrine-schema or ibexa/doctrine-migrations isn't installed. * IBX-11939: Fixed PHP 7.4 compatibility in SchemaBuilderEventSchemaProvider * IBX-11939: Addressed review feedback (arrow-fn style, removed redundant importData() call) - CoreInstaller::getQueriesFromSchemaBuilderEvent() now builds its Query list via an arrow function instead of a static closure, per review suggestion. - CoreInstaller::importData() no longer re-invokes TaggedMigrationsRunner::run() in the migrations-runner branch; importSchema() already runs every tagged migration (including ImportDataMigration) to the latest version, so the second call was a pure no-op with no real caller relying on it standalone. - getDropSqlStatementsForExistingSchema()'s docblock tightened to list<string>, which it always genuinely returns. - TaggedMigrationsRunner's constructor documents why $dependencyFactory is nullable (optional service, wired via "@?..." in services.yml; run() throws a clear exception if called while null). * IBX-11939: Removed ibexa/doctrine-migrations VCS repository entry ibexa/doctrine-migrations is now published on Packagist (which mirrors all of its branches, not just tags), so the explicit VCS repository pointing composer directly at GitHub is no longer needed to resolve the dev-branch require constraint. * IBX-11939: Made TaggedMigrationsRunner's DependencyFactory dependency mandatory Instead of TaggedMigrationsRunner tolerating a null DependencyFactory at runtime, RemoveTaggedMigrationsRunnerPass now removes its service definition entirely when "ibexa/doctrine-migrations" isn't installed/enabled, and CoreInstaller depends on it as an optional (nullable) service instead, throwing a clear configuration exception if "ibexa.installer.schema_builder_event.enabled" is disabled without it. * IBX-11939: Enforced list<Query> return from getQueriesFromSchemaBuilderEvent() Passing an empty array as array_map()'s third argument re-indexes its result, so PHPStan can type it as list<Query> instead of Query[]. * Added guarded migrations for core content/URL performance indexes Converts installer's upgrade/db/ibexa-4.5.1-to-4.5.2.sql (indexes on ezcontentobject_link, ezcontentclass_attribute, ezurl_object_link, ezcontentobject_attribute) and upgrade/db/ibexa-4.6.20-to-4.6.21.sql (ezurlalias_ml "link" index) into guarded Doctrine migrations, for installs whose schema already predates these indexes. * Fixed guard crash on installs that never had the pre-rename table $schema->getTable() throws TableDoesNotExist rather than returning null, so a guard that calls it directly (without checking hasTable() first) crashes -- rather than skipping -- on any install that never had the table under this name at all (e.g. a database built directly at the current schema, which never passed through this pre-rename/pre-delta state). Caught via live testing against a real database. Guard now checks hasTable() first. * Removed pre-4.6-origin AddContentPerformanceIndexesMigration, folded into baseline Same rationale as ibexa/product-catalog: doctrine migrations only exist starting at 4.6, so InstallSchemaMigration's baseline already includes these indexes (originally shipped via installer's upgrade/db/ibexa-4.5.0-to-4.5.1.sql). A standalone migration for pre-4.6 content can never have real work to do on a genuine 4.6+ install. * Fix ibexa/doctrine-migrations constraint to track the 4.6 branch The feat/doctrine-migrations branch it previously required has been merged into and deleted from the primary 4.6 branch upstream, so the old constraint no longer resolves. * Fix TaggedMigrationsRunner passing an empty Schema to every migration $migration->up(new Schema()) gave every migration's hasTable()/ getTable() guard checks an always-empty schema to inspect, regardless of what earlier migrations in the same run (or a prior run) actually created against the real database -- guards written the correct way (checking hasTable() before trusting a table exists) would incorrectly conclude the table is missing and either skip work that still needs doing, or crash outright on getTable() for a table that does exist. Introspect the live database instead, the same way the standalone 'doctrine:migrations:migrate' command already does via DBALSchemaDiffProvider::createFromSchema(). * Move core's own Doctrine Migrations into IbexaCoreBundle InstallSchemaMigration, AddUrlAliasMlLinkIndexMigration, and ImportDataMigration only depend on the persistence connection -- they don't need anything from IbexaRepositoryInstallerBundle (CoreInstaller, BuildSchemaSubscriber), which is scoped to installer-time concerns and additionally requires DoctrineSchemaBundle to boot at all. Declaring them in IbexaCoreBundle instead matches how every other package in this effort declares its own InstallSchemaMigration in its own bundle, and lets any consumer discover core's schema migrations via ibexa:doctrine:migrations:migrate without needing the installer bundle registered -- useful for integration test kernels that don't otherwise need CoreInstaller. * Fix SQLite-dialect bugs in InstallSchemaMigration/ImportDataMigration SQL install-schema-sqlite.sql and import-data-sqlite.sql were generated from the MySQL source without adapting to SQLite's stricter SQL semantics: - import-data-sqlite.sql used MySQL-style backslash escapes (\" and \n) inside string literals. SQLite (unlike MySQL) does not interpret backslash as an escape character at all, so these were stored as literal two-byte sequences, corrupting PHP-serialized content-type names/descriptions (unserialize() failures) and embedded XML field data (DOMDocument parse failures, e.g. the admin user's own ezimage field). - install-schema-sqlite.sql collapsed the composite primary keys on ezcontentclass, ezcontentclass_attribute and ezcontentobject_attribute (id, version) down to a single-column `id INTEGER PRIMARY KEY AUTOINCREMENT`, dropping the `version` component entirely -- unlike the mysql/postgresql variants, which both correctly declare `PRIMARY KEY(id, version)`. This made SQLite reject the second row (e.g. publishing a content-type draft, which legitimately reuses the same id with a different version/status) as a duplicate primary key. Both only affect the SQLite platform; MySQL and PostgreSQL were already correct. * IBX-11939: Switched the install SQL to utf8mb4 The MySQL install SQL hardcoded "DEFAULT CHARACTER SET utf8 COLLATE utf8_unicode_ci". On MySQL 8 "utf8" is an alias for utf8mb3, so a Doctrine Migrations install produced 3-byte columns while a SchemaBuilderEvent install produced utf8mb4 - the latter reads the configured database_charset/database_collation, which default to utf8mb4 and utf8mb4_unicode_520_ci. The practical effect was that a migrations-based install could not store 4-byte characters (emoji, CJK extensions) at all, and sorted/compared text differently. Verified against a real MySQL 8.0 server: the schema a migrations install produces now matches a SchemaBuilderEvent one. * IBX-11939: Added a Bootstrapper hook installing the schema via Doctrine Migrations DatabaseSchemaHook builds an integration suite's database from the SchemaBuilderEvent path. This adds its counterpart for the other path: DoctrineMigrationsSchemaHook runs every Ibexa-tagged Doctrine migration through TaggedMigrationsRunner - the same service CoreInstaller uses - so a suite can be booted either way and the two results compared. Disabled by default, unlike every other hook: enabling it by default would stack a second schema install on top of DatabaseSchemaHook's in every existing suite. Registered only in the "test" environment, and removed alongside TaggedMigrationsRunner when ibexa/doctrine-migrations isn't installed. * IBX-11939: Added append-only fixtures, which do not truncate before inserting FixtureImporter truncates every table a fixture touches before inserting, on the assumption that the fixture is the sole author of those tables. That assumption breaks once the baseline content can arrive another way - the Doctrine Migrations install path inserts it via ImportDataMigration - because truncating then destroys it. A fixture marked AppendOnlyFixture is inserted without truncating, so a package can contribute extra rows to tables something else has already populated. Plain fixtures are unchanged and still truncate, which is what keeps repeated runs deterministic. * IBX-11939: Ran tagged migrations through Doctrine's Migrator TaggedMigrationsRunner now hands each migration to DependencyFactory::getMigrator() - the Migrator "ibexa:doctrine:migrations:migrate" uses - instead of executing its SQL itself. Each migration runs in its own transaction where isTransactional() allows it, so on PostgreSQL and SQLite a failed migration no longer leaves half-created tables behind, and a failure now names the migration it came from. On MySQL this relies on ibexa/doctrine-migrations#13, which makes AbstractSqlMigration non-transactional there. Without it, Doctrine's executor leaves the connection's transaction nesting level raised after an implicit commit. * IBX-11939: Marked DoctrineMigrationsSchemaHook as internal * IBX-11939: Renamed bootstrapper.yml to bootstrapper.yaml * IBX-11939: Added a dedicated exception for failing tagged migrations TaggedMigrationsRunner threw a plain RuntimeException, which SonarCloud flags (php:S112) and callers can't tell apart from other runtime errors. MigrationFailedException carries the failing migration's version on top of the same message, and still extends RuntimeException, so existing catch blocks keep working. * IBX-11939: Replaced SqlPlatform with doctrine-schema's DatabasePlatformName * IBX-11939: Wrapped long lines in the Doctrine Migrations SQL files Each CREATE TABLE now lists one column, index or constraint per line, and ALTER TABLE and CREATE INDEX statements longer than 120 characters are wrapped. Only whitespace changes, apart from the comma moving in front of SQLite's --(DC2Type:...) comments, which keeps each comment with its column. * IBX-11939: Added MariaDB to the platforms the migrations support ibexa/doctrine-migrations#19 makes MariaDB a platform of its own, so each migration now lists SqlPlatform::MARIADB and runs its MySQL SQL there too. * IBX-11939: Created the tables with JSON columns the way Doctrine DBAL does on MariaDB On MariaDB, DBAL creates a JSON column as LONGTEXT with a (DC2Type:json) comment, which is what the legacy SchemaBuilderEvent path gets too. MariaDB turns MySQL's native JSON into LONGTEXT with utf8mb4_bin and a json_valid() check instead, which DBAL 2.13 then reads back as TEXT. Those tables now come from install-schema-json-tables.mariadb.sql on MariaDB and install-schema-json-tables.mysql.sql on MySQL, ahead of the rest of install-schema-mysql.sql. * IBX-11939: Targeted AddUrlAliasMlLinkIndexMigration at 4.6.21 It replaces installer's upgrade/db/ibexa-4.6.20-to-4.6.21.sql, so 4.6.21 is the release it is for. The target version only orders Ibexa migrations: it now runs after the 4.6.0 baselines instead of right after core's own, which is fine since it only needs core's tables.
…ibute_ml's SQLite foreign key (#856) schema.yaml declares ezcontentclass_attribute_ml_lang_fk with onUpdate: CASCADE, and the MySQL and PostgreSQL baselines have it, but the SQLite one only had ON DELETE CASCADE. The schema parity test reports it on SQLite's fresh-install scenario.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Caution
This is a merge pull request
Related PRs: