Add draft of static require RFC

[HaroldCindy]

Motivating statement from Canny:

I'd argue for a very limited form of require() , with roughly the same capabilities and limitations as the #include feature of Firestorm's pre-processor. Please keep it simple and get something useable into users' hands.
[...]
Is it possible to remove dead code when compiling? Firestorm does that, and it means you can have libraries of useful functions without worrying about pulling in too much. That's easier in LSL, because you can't alias a function name and because #include is done in a prepass. Trickier to do in Luau, but desirable.

Absolutely, that's exactly what we're aiming for, along with support for dead code elimination for unused imports. To optimize cross-module effectively with Luau, you really have to pull everything into a single translation unit, giving you something close to #include-like semantics, but with a separate globals table for modules to force encapsulation.

We're not going to overly-complicate things such that it takes ages for people to get access to something useful, there'll be no 5-year multi-stakeholder working groups for this :)

As you mention, the aliasing problem does complicate things (DCE in LSL itself is pretty trivial,) as Luau doesn't currently do any tree-shaking to get rid of unused function references. We're in a pretty fortunate position as we don't have any existing modules we need to support, so we can impose additional restrictions on modules that Luau proper would not be able to impose to allow effective tree-shaking.

We're looking at a number of options, including forcing modules to only use pure expressions at the top level, as well as forcing explicit declaration of exports that can be resolved statically if you want that tree-shaking / dead code elimination behavior. Ideally we end up with something that's compatible with Luau proper, but allows us to do extra optimizations that are only generally relevant to SL as a compile target.

We expect that those with a background in programming language theory will have thoughts on what makes sense here, so we're going to put together a proper an RFC once we've collected our thoughts and done a review of what others have done in similar spaces. Existing tree-shakers and bundlers for CommonJS are the most obvious sources of inspiration.

graph of the proposed editing workflow:

[WolfGangS]

What is libs dir in this instance?

If its a user defined dir, I'd personally prefer keeping that functionality strictly to the user set aliases.

[HaroldCindy]

I was thinking it could implicitly search through the @libs alias, so that you don't necessarily have to set up a specific alias for every single package you install. In other languages terms, where @libs is equivalent to your node_modules or dist-packages or what have you. I imagine the same would work for in-world editing where you could either

a) have a "modules" subfolder with symbolic links to what you want to use for that particular folder of scripts, but this gets weird in script-inside-object cases since you can't have subfolders there
b) just load from some user-defined global "modules" folder in user inventory

whichever turns out to be less annoying for people's usecases, will need to do some investigation there. Option B seems the most obvious and frictionless, to me at the expense of things getting weird if you have two projects that both want different, incompatible versions of the same library and you want to manage your dependencies in inventory. Anything more than that and you end up effectively having to implement a package manager in the viewer for the developer experience to be acceptable.

[WolfGangS]

Scripters currently are used to specifying a system folder to the fs_preprocessor I think having them provide one explicitly and using @libs/.. instead of magically including something would be preferable.

Automagic choosing of a source feels like it will end up eventually conflicting with something.

For instance the viewer needs its own built in resolver for some aliases to do bundling into the package for users that don't user external tools. that would need to be disabled if the user wants to use the external vscode plugin, unless the plugin can provide a bundle where the dependencies are included in it, without aliases, so the viewer wont resolve those to anywhere but things already explicitly in the bundle.

So a save in vscode would produce this something like this.

--[[!!SLUA:BUNDLE!!]]
local a = require("thing")
print(a())
--[[!!SLUA:MODULE:thing!!]]
return function()
    return "test"
end

The viewer would do no further bundling there, as "thing" was provided, if it wasn't it should error imo, rather than behaviour being different if it is and isn't provided.

Essentially the viewer / server bundling steps only to pull in defined code with @ aliases, not "magic" that could get something not well defined by the user.

So a require("test") would error unknown module if that is not explicitly included by the user/plugin in the bundle.

[WolfGangS]

Should this have a namespace reserved in advance?

• @my-inv for user inventory?
• @inv for current object inventory?

It may be prudent to declare @sl-* as reserved to give LL room to extend for more than just a direct library of scripts, without scripters hitting conflicts in the future.

For instance if inventory based requires don't receive an alias ahead of time, reserving @sl-* would allow avoiding conflicts with user made aliases by using @sl-inv to add inventory support.

[Rider-Linden]

We're reserving @sl-*, so @sl-inv might be the way to go here.

[WolfGangS]

That should probably be added to the RFC then.
At the moment it only states @sl for possible LL provided libraries.

[HaroldCindy]

I'm concerned about specifying where to pull things from instead of what to pull inside the actual code. What would @inv mean when running a test script against your code locally, or using a local bundler (as in the VSCode plugin)?

If the answer is "@inv/foo is equivalent to foo when executing locally", does that mean the only difference with @inv would be that it would never try to resolve to a local file if compiling an in-world script?

[WolfGangS]

That's sort of why I was thinking @sl-inv or @sl-object, local bundler like vscode, would basically ignore it, and pass it to the viewer, the viewers bundler should see it and go, Ahh, I need to go add that from the object/avatar inventory (depending on whats implemented for that), anything with @sl/ or @sl-*/ would be fairly obvious to any local tooling to leave to the sl ecosystem.

Like your bit about Platform libs (@sl/...) not included - provided by runtime.

Basically the way i picture it on save the viewers built in bundler code will try and resolve any @ aliases with its own list of rules, be that for local system folders, or avatar inventory folders or so. And the viewer have a config area on script development or so to specify other non @sl-* aliases

• @sl-inv would look in the avatars inventory scripts folder.
• @sl-obj would look inside the current object the script is being saved in. (would throw an error on save when saved a script in inventory with unknown alias or something)
• @libs would check viewer config for what local system folder has been defined for libs
• @my-http-lib same as above but for what was defined against my-http-lib

[WolfGangS]

Which default file semantics will be used for the luau file if a directory is specified instead of a file directly?

At the moment it seems luau rfcs state either module.luau or init.luau I can't find a clear answer on which.

[HaroldCindy]

I'm open to either, may have to look at Lute to see if they have different resolution strategies there.

[WolfGangS]

Is the meta data just any comments after the bundle comment?
Or explicit with some sort of --[[!!SLUA:COMMENT: <multiline comment text> ]] style comment?
Will they be removed for the purpose of compile and errors?

Alternatively as the bundle comment is a multi line one, anything after the first newline inside the comment could be ignored by the bundle system e.g.

--[[!!SLUA:BUNDLE!!
  <meta info about bundle>
]]

--[[!!SLUA:MODULE:@libs/lib!!
  <meta info about module>
]]

[HaroldCindy]

Yep, the metadata would be part of the slua bundle directive itself.

[WolfGangS]

Could change the line in the rfc to this then to show it explicitly?

--[[!!SLUA:BUNDLE!!
   NOTE: May have some metadata in the header too
]]