Fix local vs. DevContainer Poetry package conflicts

[hkeeler]

Per @lchen-2101 on #63 (comment)

Also making a note that devcontainer and native poetry doesn't seem to play nice together. When I ran the code in my local terminal outside of devcontainer, poetry shell wasn't working properly until I did a poetry install to get all the dependencies sync'ed up again; but that seems to break the dependencies for the devcontainer which then requires a poetry install in the vs code shell running in devcontainer.

I suspect this is related to a COPY in the Dockerfile and/or a volume mount where the Mac installed Python packages are not compatible with Linux Python...and vise-versa. Let's get to the root of the problem, and see how to make the two setups play nice. While we're at it, let's check in with each of our devs to make sure we're still getting value out of the DevContainer setup.

[michaeljwood]

I was messing with this a little bit, and it does seem like an incompatibility between MacOS/Linux due to mounting the workspace folder into the container rather than cloning the repository into the container.

Also, seems like something changed with the dependencies since the devcontainer setup was created, as I am no longer able to create a new devcontainer as is. I get errors like the following:

[2025-02-04T20:24:29.638Z] 26.40     | error: command 'cmake' failed: No such file or directory
26.40 
26.40 Note: This error originates from the build backend, and is likely not a problem with poetry but one of the following issues with pyarrow (19.0.0)
26.40 
26.40   - not supporting PEP 517 builds
26.40   - not specifying PEP 517 build requirements correctly
26.40   - the build requirements are incompatible with your operating system or Python version
26.40   - the build requirements are missing system dependencies (eg: compilers, libraries, headers).
26.40 
26.40 You can verify this by running pip wheel --no-cache-dir --use-pep517 "pyarrow (==19.0.0)".
26.40 

Even after adding in a couple extra apps in the initial apk add command (make, cmake and g++) the install will fail with some other errors while trying to build pyarrow. From what I was reading there is no precompiled binary for pyarrow on Alpine that can be installed by pip/poetry. It should be possible to install a corresponding package with apk, but the version would have to be specified/kept in sync with the pyproject.toml file.

I switched the devcontainer base image to python:3.12-bookworm and was able to start the devcontainer (with other appropriate changes to the Dockerfile). If I used the clone option rather than mounting the host folder in the container, that was enough to be able to run poetry commands in both a VSCode window on the host, and one running in a devcontainer at the same time.

I do question how often it would be required to be able to work in a devcontainer and the local workspace. However, I did find a solution there as well with mounting the .venv folder as a volume in the devcontainer.json file from this comment

So I guess long story short I would propose moving to a Debian bookworm base for the devcontainer, exclude the .venv folder by mounting a volume to that folder, and move the poetry install into a postCreateCommand, assuming of course that the devcontainer stuff is worth keeping around. I created draft PR #336 to show these changes.