defaultExtensions are way too fat, resulting in long installation times

Running eg. nix develop github:loophp/nix-shell#php74 in folder with composer.json that has no ext- sections takes still quite long time. I can see it compiling things like gd, mysql, iconv and so on which are not needed at all. Is it possible to either improve the speed of installing these (binaries instead of having to compile it? Installing multiple extensions at the same time, or at least each with -J flag?), or skip them? Otherwise amazing feature of this shell, which is auto-installation of extensions from composer.json is quite under-utilized.

Hey hi!

Have you added nix-shell in Cachix ? This will actually download binaries instead of compiling stuff.

Let me know if you need help to enable it!

Cheers

Unfortunately Cachix doesn't seem to do anything on my machine, because I have m1 arm64 and there are no build artifacts for this machine published. What's worse, Github doesn't seem to have such runners at the moment actions/runner-images#5631 (although it's possible to use self-hosted ones, or buildjet ones but that's not free)

This could be important:

➜  ~ uname -a
Darwin PL00562-3.local 22.3.0 Darwin Kernel Version 22.3.0: Thu Jan  5 20:48:54 PST 2023; root:xnu-8792.81.2~2/RELEASE_ARM64_T6000 arm64

@ostrolucky

Dear @ostrolucky ,

I aksed on the Nix PHP Matrix channel.

The issue comes from the fact that we merged the latest update of PHP without making sure that it was successfully built on that architecture.

If cannot be built (due to test or otherwise) it cannot be cached

@LeSuisse partially fixed the issue in NixOS/nixpkgs#215004 but it seems that there is another issue here NixOS/nixpkgs#215220.

I believe we need to wait the next release of PHP to make sure it builds correctly, unless there is an alternative, maybe @LeSuisse might know more than me on this?

Yes there is one remaining flaky test on darwin, I opened NixOS/nixpkgs#215539 so we can work around it and have everything in cache.nixos.org again.

Note that is note the last PHP upgrade that caused the issue, it seems it is the case for quite a while (I did not bother further than that in time).

@LeSuisse

Thanks @LeSuisse, let's bury that issue in your new PR :)

@LeSuisse

Thanks patch has been merged 🎉 !

We just need to wait for the next hydra iteration (https://hydra.nixos.org/job/nixos/trunk-combined/tested#tabs-constituents) and you'll be able to use the binaries from the cache!

Thanks @LeSuisse and @etu !

@ostrolucky

@ostrolucky I think it should be fixed by now. Can you try ?

@drupol

@drupol Another thing to consider is that PHP 7.4 isn't cached by hydra since it's not a supported version in nixpkgs. Do you have a separate cache somewhere?

Yes, we do have fossar :) but it's not building for M1 architecture since it's not available on Github actions.

I had to do nix channel --update first, right?

Following extensions are still being built
php-amqp, php-dom php-mysqlnd php-xdebug php-mysqli php-pdo_mysql php-xmlreader

but rest seems not, so I would say that's a big improvement :)

No, we don't use channels, we just use the flake feature, I never liked the idea of using channels.

To try correctly, do:

rm -rf ~/.cache/nix
nix run nixpkgs#php82 -- -v
That said, it is possible that extensions that are not enabled by default in PHP are being built locally on your machine.

And that should be quite fast now :)

Still same thing with above listed 7 extensions, but yeah it's good enough now I would say. But I think we can still remove
these php-dom php-mysqlnd php-mysqli php-pdo_mysql php-xmlreader from here

nix-shell/src/phps.nix

Lines 5 to 43 in 24b0670

    
           defaultExtensions = [ 
        
             "bcmath" 
        
             "calendar" 
        
             "ctype" 
        
             "curl" 
        
             "dom" 
        
             "exif" 
        
             "fileinfo" 
        
             "filter" 
        
             "gd" 
        
             "gettext" 
        
             "gmp" 
        
             "iconv" 
        
             "intl" 
        
             "mbstring" 
        
             "mysqli" 
        
             "mysqlnd" 
        
             "opcache" 
        
             "openssl" 
        
             "pdo" 
        
             "pdo_mysql" 
        
             "pdo_odbc" 
        
             "pdo_pgsql" 
        
             "pdo_sqlite" 
        
             "pgsql" 
        
             "posix" 
        
             "readline" 
        
             "session" 
        
             "simplexml" 
        
             "sockets" 
        
             "soap" 
        
             "sodium" 
        
             "sqlite3" 
        
             "sysvsem" 
        
             "tokenizer" 
        
             "xmlreader" 
        
             "xmlwriter" 
        
             "zip" 
        
             "zlib"

right? Or how can we convince upstream php pkg to include these packages by default?

@jtojnar

I will think about it but if we want to convince upstream to have those enabled by default, then we have to motivate the request in an issue on NixOS/nixpkgs.

This is also something that we should discuss with the PHP Maintainers team (@jtojnar, @etu, @aanderse,
etc etc... )

What you could also do to avoid compilation time is to not use loophp/nix-shell and use php from nixpkgs only in the meantime.

It's not a very hard change to implement, however, it's a breaking change.

Here's the steps of what we need to do to do that breaking change:

Decide which the new defaults should be (and a good motivation for this, the current defaults haven't been decided recently, they where decided many years ago before PHP in Nix was modular so to enable a module you needed to compile the whole thing.
Then we need to alter the three files (one for each supported version) 8.0 8.1 8.2 with the new defaults, however, this shouldn't just remove a bunch of things. Instead we should look at the stateVersion and make sure to not change the defaults if one is on an older version.
Document it in the release notes.

@etu

@etu are you sure this would be a breaking change? Here, we are suggesting adding extensions, not removing any. Why would this be considered a BC ?

Oh, yeah, I misread that then... I'm however questioning if we have sensible defaults to begin with 😃

I don't think we have indeed... We should really discuss that thing :)

Oh crap,... the page I used to take inspiration from is no more existing.

At https://symfony.com/doc/current/setup.html#technical-requirements only these extensions are listed:

ctype
iconv
PCRE
session
simplexml
tokenizer

Navigation Menu

Repository navigation

defaultExtensions are way too fat, resulting in long installation times #128

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

	defaultExtensions = [
	"bcmath"
	"calendar"
	"ctype"
	"curl"
	"dom"
	"exif"
	"fileinfo"
	"filter"
	"gd"
	"gettext"
	"gmp"
	"iconv"
	"intl"
	"mbstring"
	"mysqli"
	"mysqlnd"
	"opcache"
	"openssl"
	"pdo"
	"pdo_mysql"
	"pdo_odbc"
	"pdo_pgsql"
	"pdo_sqlite"
	"pgsql"
	"posix"
	"readline"
	"session"
	"simplexml"
	"sockets"
	"soap"
	"sodium"
	"sqlite3"
	"sysvsem"
	"tokenizer"
	"xmlreader"
	"xmlwriter"
	"zip"
	"zlib"

defaultExtensions are way too fat, resulting in long installation times #128

Description

Activity

drupol commented on Feb 8, 2023

ostrolucky commented on Feb 9, 2023

drupol commented on Feb 9, 2023

LeSuisse commented on Feb 9, 2023

drupol commented on Feb 9, 2023

drupol commented on Feb 10, 2023

drupol commented on Feb 13, 2023

etu commented on Feb 13, 2023

drupol commented on Feb 13, 2023

ostrolucky commented on Feb 13, 2023

drupol commented on Feb 13, 2023

ostrolucky commented on Feb 13, 2023

drupol commented on Feb 13, 2023

etu commented on Feb 13, 2023

drupol commented on Feb 13, 2023

etu commented on Feb 13, 2023

drupol commented on Feb 13, 2023

drupol commented on Feb 13, 2023

ostrolucky commented on Mar 12, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions