[MDEV-31414] Implement optional lengths for string types

[OsamaNabih]

Jira: https://jira.mariadb.org/browse/MDEV-31414
Adding support for SQL standard 2023's T081: Optional string types maximum length

This PR allows the user to not specify a length for a VARCHAR field. We default to the maximum possible value, which is 65,535 (0xFFFF) according to https://mariadb.com/docs/server/reference/data-types/string-data-types/varchar
After trying 65,535, I got the error that size is too big, maximum is 16383

Questions:

1. Is 16383 reported as the maximum due to collation choice? I am guessing the max bytes is 65,535 but maybe due to collation every char requires 4 bytes and that's how we end up with 16383? Is there a way to get the current char size for the server?
2. Should the m_has_explicit_length property in Lex_length_and_dec_st in this case be set to true? (default is false here as no value was provided to Lex during parsing of the VARCHAR clause.

[CLAassistant]

All committers have signed the CLA.

[FooBarrior]

Hi,

Is 16383 reported as the maximum due to collation choice?

That is a nice guess!

Our documentation claims it's 21,844 characters, though I think it's a bit outdated, since we support 4-byte encodings for some time. So yes, it's got to be UINT16_MAX/4 - header_length, where header_length is something like 3.

[FooBarrior]

We follow the TDD paradigm - every feature should be covered with tests (and every bugfix, if possible). Include both .test and .result files in the commit.

I'm fine with a hard constant of 16383 in scope of this small work, we can always extend it and make a function of charset any time we want, since 16383 is expected to be the minimum of the maximums.

[FooBarrior]

Declare a constant in the appropriate place, like MAX_VARCHAR_SIZE, or maybe better MAX_VARCHAR_ESTIMATED_SIZE, signifying the imprecise nature.

I tried to look for the existing one and turns out it's pretty widely used in a raw literal form in the sql directory. Let's be the first good guys who'll add it there.

There is a constant in innodb -- we can't refer to it from sql, because this will reverse the dependencies. (innodb depends from sql, not vice-versa)

[FooBarrior]

I'm pretty surprised this change would work on its own. Fine by me, if it does, though. Please add the test with a new syntax.

[FooBarrior]

You can try to make tha max value variable if you want. Though it may require some research.

The problem is that we have (at least) three layers of charset settings:

• system-wide
• per-table
• per-field

One overrides another. At some point, Filed would get a deduced value of charset, and perhaps Create_field would get it, too. Though it's hard to say, when exactly. Create_field has charset field in its parent Column_definition_attributes, you may check if it's set to the right encoding at different points in time.

You'd have to test all three cases though:

• When a system-wide encoding is set
• When it's overriden with table
• When the latter is overriden with field charset.

Again, I'm NOT putting this as a requirement for this submission!