Benchmark demo content

[thibaudcolas]

The site should support for a separate “benchmark” content set, with the following metrics.

• Scale of content complexity
  • Number of fields: 100
  • StreamField
    • 100 blocks
      • 10 levels of nesting
  • 100 items in InlinePanel
  • 100 paragraphs in rich text
• Scale of implementation complexity
  • 10 levels of multi-table inheritance
• Scale of site history
  • 100,000 revisions for a page
  • 1M revisions across the whole
• Scale of page tree
  • Amount: 100,000 pages
  • Breadth of page tree
    • 100,000 pages - 1M: blog section, online marketplace
  • Depth of page tree (depending on default treebeard configuration)
    • 10? Intranet use case could have more?
    • Attempt to retrieve this number for real-world sites / SQL query or QS snippet
  • 100 language translations
• Scale of images
  • Hundreds - 10,000
  • Usage references - 1000 (one image referenced 1000 times)
    • And-or maybe 1000 references across multiple images
    • Average: images are used once
• Scale of snippets
  • 1M instances

A lot of those are very feasible for one person as a one-off. We need this done in a way that is more readily accessible. This might not be realistic but ideally as simple as initializing the site with a load_benchmark_data management command or similar, and that’s it.

[awais786]

@thibaudcolas I am interested in this.

[thibaudcolas]

@awais786 great! Please take a look at the list and think of how to approach this gradually? I imagine we can set up a "seed" command that generates data for 1-2 content types first, scale it up over time?

[awais786]

We can divide this into small phases. We can start with the following plan:

Add a new benchmark app with models

• BenchmarkIndexPage (root page).
• BenchmarkPage (content page with 20 fields, StreamField, InlinePanel etc).

Add a load_benchmark_data management command:

• Creates benchmark index page, pages, items, and snippets.
• Uses get_or_create so it is idempotent and can be run multiple times to add new data.
• Publishes pages automatically for browser view.

Next phases:
Will expand content complexity, add more fields, nested StreamFields, larger numbers of pages/snippets/items, multi-table inheritance, and revision history.

Please let me know your thoughts.

[thibaudcolas]

@awais786 I’d rather reuse existing content types unless there was big need to introduce new ones? From my list I think "number of fields" and "levels of multi-table inheritance" are the only items that would require new content types. We can set those requirements aside for now.