Support development with Podman

[rgaiacs]

I'm trying to use Podman Desktop more and move all my software development from Docker to Podman.

I'm following Develop Kubernetes integration.

The setup of kind with Podman Desktop was possible with a few clicks. 🚀

The BinderHub server, i.e.

python3 -m binderhub -f testing/local-binder-k8s-hub/binderhub_config.py

failed due

    Traceback (most recent call last):
      File "/home/raniere/github.com/jupyterhub/binderhub/.pixi/envs/default/lib/python3.13/site-packages/docker/api/client.py", line 223, in _retrieve_server_version
        return self.version(api_version=False)["ApiVersion"]
               ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
      File "/home/raniere/github.com/jupyterhub/binderhub/.pixi/envs/default/lib/python3.13/site-packages/docker/api/daemon.py", line 181, in version
        return self._result(self._get(url), json=True)
                            ~~~~~~~~~^^^^^
      File "/home/raniere/github.com/jupyterhub/binderhub/.pixi/envs/default/lib/python3.13/site-packages/docker/utils/decorators.py", line 44, in inner
        return f(self, *args, **kwargs)
      File "/home/raniere/github.com/jupyterhub/binderhub/.pixi/envs/default/lib/python3.13/site-packages/docker/api/client.py", line 246, in _get
        return self.get(url, **self._set_request_timeout(kwargs))
               ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      File "/home/raniere/github.com/jupyterhub/binderhub/.pixi/envs/default/lib/python3.13/site-packages/requests/sessions.py", line 602, in get
        return self.request("GET", url, **kwargs)
               ~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^
      File "/home/raniere/github.com/jupyterhub/binderhub/.pixi/envs/default/lib/python3.13/site-packages/requests/sessions.py", line 589, in request
        resp = self.send(prep, **send_kwargs)
      File "/home/raniere/github.com/jupyterhub/binderhub/.pixi/envs/default/lib/python3.13/site-packages/requests/sessions.py", line 703, in send
        r = adapter.send(request, **kwargs)
      File "/home/raniere/github.com/jupyterhub/binderhub/.pixi/envs/default/lib/python3.13/site-packages/requests/adapters.py", line 682, in send
        raise ConnectionError(err, request=request)
    requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

but worked after setting

export DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock

The build pod in the Kubernetes cluster failed due

MountVolume.SetUp failed for volume "docker-socket" : hostPath type check failed: /var/run/docker.sock is not a socket file

I will look into Podman-inside-Kubernetes as mentioned in the documentation. I hope that a small change in ./testing/local-binder-k8s-hub/install-jupyterhub-chart will be enough.

[consideRatio]

Thank you soo much @rgaiacs for working through these hurdles and making others avoid them!!

[rgaiacs]

The documentation says to add

imageBuilderType: pink

to config.yaml. This is for people using the BinderHub helm chart. Based on

binderhub/helm-chart/binderhub/files/binderhub_config.py

Lines 50 to 55 in b39a9d7

	imageBuilderType = get_value("imageBuilderType")
	if imageBuilderType in ["dind", "pink"]:
	hostSocketDir = get_value(f"{imageBuilderType}.hostSocketDir")
	if hostSocketDir:
	socketname = "docker" if imageBuilderType == "dind" else "podman"
	c.BinderHub.build_docker_host = f"unix://{hostSocketDir}/{socketname}.sock"

what we want to change is build_docker_host. I will try to copy the above snippet to ./testing/local-binder-k8s-hub/install-jupyterhub-chart.

[rgaiacs]

BinderHub is sending to Kubernetes the following:

apiVersion: v1
kind: Pod
metadata:
  annotations:
    binder-repo: https://github.com/binder-examples/requirements
  creationTimestamp: 2025-07-28T11:17:27.000Z
  generation: 1
  labels:
    component: binderhub-build
    name: build-binder-2dexamples-2drequirements-55ab5c-339-f4
  name: build-binder-2dexamples-2drequirements-55ab5c-339-f4
  namespace: default
  resourceVersion: "9902"
  uid: adb96929-8e9c-4fbc-9ce4-2a823fe3db0b
spec:
  affinity:
    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
        - podAffinityTerm:
            labelSelector:
              matchLabels:
                component: binderhub-build
            topologyKey: kubernetes.io/hostname
          weight: 100
  containers:
    - args:
        - jupyter-repo2docker
        - --ref=339cda9f64f2827946dadfbec6431a47e7705711
        - --image=binder-2dexamples-2drequirements-55ab5c:339cda9f64f2827946dadfbec6431a47e7705711
        - --no-clean
        - --no-run
        - --json-logs
        - --user-name=jovyan
        - --user-id=1000
        - --push
        - https://github.com/binder-examples/requirements
      image: quay.io/jupyterhub/repo2docker:2024.07.0
      imagePullPolicy: IfNotPresent
      name: builder
      resources:
        limits:
          memory: "0"
        requests:
          memory: "0"
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: File
      volumeMounts:
        - mountPath: /var/run/docker.sock
          name: docker-socket
        - mountPath: /root/.docker/config.json
          name: docker-config
          subPath: config.json
        - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
          name: kube-api-access-rhzf9
          readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  nodeName: binderhub-control-plane
  preemptionPolicy: PreemptLowerPriority
  priority: 0
  restartPolicy: Never
  schedulerName: default-scheduler
  securityContext: {}
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  tolerations:
    - effect: NoSchedule
      key: hub.jupyter.org/dedicated
      operator: Equal
      value: user
    - effect: NoSchedule
      key: hub.jupyter.org_dedicated
      operator: Equal
      value: user
    - effect: NoExecute
      key: node.kubernetes.io/not-ready
      operator: Exists
      tolerationSeconds: 300
    - effect: NoExecute
      key: node.kubernetes.io/unreachable
      operator: Exists
      tolerationSeconds: 300
  volumes:
    - hostPath:
        path: /var/run/docker.sock
        type: Socket
      name: docker-socket
    - name: docker-config
      secret:
        defaultMode: 420
        secretName: binder-build-docker-config
    - name: kube-api-access-rhzf9
      projected:
        defaultMode: 420
        sources:
          - serviceAccountToken:
              expirationSeconds: 3607
              path: token
          - configMap:
              items:
                - key: ca.crt
                  path: ca.crt
              name: kube-root-ca.crt
          - downwardAPI:
              items:
                - fieldRef:
                    apiVersion: v1
                    fieldPath: metadata.namespace
                  path: namespace
status:
  conditions:
    - lastProbeTime: null
      lastTransitionTime: 2025-07-28T11:17:27.000Z
      status: "False"
      type: PodReadyToStartContainers
    - lastProbeTime: null
      lastTransitionTime: 2025-07-28T11:17:27.000Z
      status: "True"
      type: Initialized
    - lastProbeTime: null
      lastTransitionTime: 2025-07-28T11:17:27.000Z
      message: "containers with unready status: [builder]"
      reason: ContainersNotReady
      status: "False"
      type: Ready
    - lastProbeTime: null
      lastTransitionTime: 2025-07-28T11:17:27.000Z
      message: "containers with unready status: [builder]"
      reason: ContainersNotReady
      status: "False"
      type: ContainersReady
    - lastProbeTime: null
      lastTransitionTime: 2025-07-28T11:17:27.000Z
      status: "True"
      type: PodScheduled
  containerStatuses:
    - image: quay.io/jupyterhub/repo2docker:2024.07.0
      imageID: ""
      lastState: {}
      name: builder
      ready: false
      restartCount: 0
      started: false
      state:
        waiting:
          reason: ContainerCreating
      volumeMounts:
        - mountPath: /var/run/docker.sock
          name: docker-socket
        - mountPath: /root/.docker/config.json
          name: docker-config
        - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
          name: kube-api-access-rhzf9
          readOnly: true
          recursiveReadOnly: Disabled
  hostIP: 10.89.0.2
  hostIPs:
    - ip: 10.89.0.2
  phase: Pending
  qosClass: BestEffort
  startTime: 2025-07-28T11:17:27.000Z

Notice that the volumeMounts is only for Docker. We need a way to also mount the Podman socket and define DOCKER_HOST in the container.