> Beyond the product itself, pyx is also an instantiation of our strategy: our tools remain free, open source, and permissively licensed — forever. Nothing changes there. Instead, we'll offer paid, hosted services that represent the "natural next thing you need" when you're already using our tools: the Astral platform.

Basically, we're hoping to address this concern by building a separate sustainable commercial product rather than monetizing our open source tools.

I was surprised to see the community here on HN responding so cautiously. Been developing in python for about a decade now- whenever astral does something I get excited!

I'm so glad all the Python packaging challenges are "solved"

This comes across as uninformed at best and ignorant at worst. Python still doesn't have a reliable way to handle native dependencies across different platforms. pip and setuptools cannot be the end all be all of this packaging ecosystem nor should they be.

And also if you mean that pyx might be hosted on uv, well I think the discussion can go towards that pyx should be made open source but honestly, I am pretty sure that someone might look at pyx and create a pyx api compliant hosted server or I am still curious as to how pyx works and what it actually truly does.

Hopefully at the point the community is centralized enough to move in one direction.

Debian decided you should use venv for everything. But when packages are installed in a venv, random cmake nonsense does not find them. There are apt-get level packages, some things find those, others do not. Names are not consistent. There's a thing called pipx which my console recommended for much the same experience. Also the vestiges of 2 vs 3 are still kicking around in the forms of refusing to find a package based on the number being present or absent.

Whatever c++headerparser might be, I'm left very sure that hacking python out of the build tree and leaving it on the trash heap of history is the proper thing to do.

I went with Python because I never had this issue. Now with any AI / CUDA stuff its a bit of a nightmare to the point where you use someone's setup shell script instead of trying to use pip at all.

At the same time, even on Windows, libc has been stable since Win10 - that's 10 years now. Which is to say, any binary compiled by VC++ 2015 or later is C-ABI-compatible with any other such binary. The only reasons why someone might need a specific compiler version is if they are relying on some language features not supported by older ones, or because they're trying to pass C++ types across the ABI boundary, which is a fairly rare case.

This itself is IMO a product of Python having a relatively healthy consensus-driven standardization process for packaging, rather than an authoritative one. If Python had more of an authoritative approach, I don't think the language would have done as well as it has.

(Source: I've written at least 5 PEPs.)

> "An example of what this might look like (we may not do this, but it's helpful to have a concrete example of the strategy) would be something like an enterprise-focused private package registry."

https://hachyderm.io/@charliermarsh/113103605702842937

Astral has been very transparent about their business model.

https://www.buzzfeed.com/kristenharris1/famous-people-same-n...

and yet no one bemoans this (hospitals don't consult name registries before filling out birth certificates). that's because it's almost always extremely clear from context.

> The real pyx

what makes that pyx any more "real" than this pyx? it's the extension of the language py plus a single letter. there are probably a thousand projects that could rightfully use that combination of letters as a name.

We actually support this basic idea today, even without pyx. You can run (e.g.) `uv pip install --torch-backend=auto torch` to automatically install a version of PyTorch based on your machine's GPU from the PyTorch index.

pyx takes that idea and pushes it further. Instead of "just" supporting PyTorch, the registry has a curated index for each supported hardware accelerator, and we populate that index with pre-built artifacts across a wide range of packages, versions, Python versions, PyTorch versions, etc., all with consistent and coherent metadata.

So there are two parts to it: (1) when you point to pyx, it becomes much easier to get the right, pre-built, mutually compatible versions of these things (and faster to install them); and (2) the uv client can point you to the "right" pyx index automatically (that part works regardless of whether you're using pyx, it's just more limited).

> Since this is a private, paid-for registry aimed at corporate clients, will there be an option to expose those registries externally as a public instance, but paid for by the company? That is, can I as a vendor pay for a Pyx registry for my own set of packages, and then provide that registry as an entrypoint for my customers?

We don't support this yet but it's come up a few times with users. If you're interested in it concretely feel free to email me (charlie@).

I think based on the quality of their work, there's also an important component which is trust. I'd trust and pay for a product from them much more readily than an open source solution with flaky maintainers.

They did say they want their thing to have understanding of the code, so maybe they’ll sell semgrep-like features and SBOM/compliance on top. Semgrep is ok popular, but if it maybe bundled into something else (like the package registry itself) that might get enough people over the line to buy it.

Private registries and “supply chain security” tools individually aren’t the hottest market, but maybe together the bundle could provide enough value. Let’s see how it goes.

Go and look up why you can't run scripts with `python3 foo.py` on Windows. It's like a one-line fix and they've come up with all sorts of naysaying reasons rather than just adding python3.exe (which Microsoft did years ago in their Python packages).

Some of those are package tools, some are dependency managers, some are runtime environments, some are package formats...

Some are obsolete at this point, and others by necessity cover different portions of programming language technologies.

I guess what I'm saying is, for the average software engineer, there's not too many more choices in Python for programming facilities than in Javascript.

uv addressed these flaws with speed, solid dependency resolution, and a simple interface that builds on what people are already used to. It unifies virtual environment and package management, supports reproducible builds, and integrates easily with modern workflows.

You just need to understand how to setup a venv, and for each thing you want to depend on how to install it in the venv. Put those bits in a shell script and you can clone the project, run the script and then have a working env

It will break sometimes because python and a lot of python packages have zero respect for backward compatibility, but that's python for you.

It's ugly and horrible, but you don't have to relearn the latest python packaging "standard" every other year

I don't know what guides you're reading but I haven't touched easy_install in at least a decade. It's successor, pip, had effectively replaced all use cases for it by around 2010.

Its a pain in the ass to work with professionally.

They have: uv

> Don't understand how a private company like Astral is leading here

They wrote the single tool to rule them all.

Design by committee is hard. Especially building consensus around reconsidering what at this point should be considered bad design. Python has a few decades of that. It took decades for the to even consider starting the process of removing the GIL, which is a great example of "it's clearly broken but lets not fix it". Packaging is the same.

These guys are laser focused on just fixing things the right way and that's what it takes,

There seems to be a lot of movement around some of these topics. The Gil is being removed, python's performance issues are meling away with recent releases, and things like uv are happening. I like it.

Anyway, if you're familiar with Node then I think you can view pip and venv as the npm of Python. Things like Poetry are Yarn, made as replacements because pip sort of sucks. UV on the other hand is a drop-in replacement for pip and venv (and other things) similar to how pnpm is basically npm. I can't answer your question on why there isn't a "good" tool to rule them all, but the single tool has been pip since 2014, and since UV is a drop-in, it's very easy to use UV in development and pip in production.

I think it's reasonable to worry about what happens when Astral needs to make money for their investors, but that's the beauty of UV compared to a lot of other Python tools. It's extremely easy to replace because it's essentially just smarter pip. I do hope Astral succeeds with their pyx, private registries and so on by the way.

It’s entirely volunteer based so I don’t blame them, but the reality is that it’s holding back the ecosystem.

I suspect it’s also a misalignment of interests. No one there really invests in improving UX.

because they're obsessed with fixing non-issues (switching out pgp signing for something you can only get from microsoft, sorry, "trusted providers", arguing about mission statements, etc.)

whilst ignoring the multiple elephants in the room (namespacing, crap slow packaging tool that has to download everything because the index sucks, mix of 4 badly documented tools to build anything, index that operates on filenames, etc.)

What happens in Go/TS when you want to use a C library underneath that's dynamically linked? Oh wait, there's no mechanism in it's package manager to support this, so it's assumed it's installed already, or in Go, C code is bundled instead. This is the problem all the Python packaging stuff above has tried to solve.

Now we have pyproject.toml and uv to learn. This is another hour or so of learning, but well worth it.

Astral is stepping up because no one else did. Guido never cared about packaging and that's why it has been the wild west until now.

Whatever I would say at this point about PyPA would be so uncharitable that dang would descend on me with the holy hammer of banishment, but you can get my drift. I just don't trust them to come out with good tooling. The plethora they have produced so far is quite telling.

That said, pip covers 99% of my needs when I need to do anything with Python. There are ecosystems that have it way worse, so I count my blessings. But apparently, since Poetry and uv exist, my 99% are not many other people's 99%.

If I wanted to package my Python stuff, though, I'm getting confused. Is it now setup.py or pyproject.toml? Or maybe both? What if I need to support an older Python version as seen in some old-but-still-supported Linux distributions?

> They should remove the "There should be one-- and preferably only one --obvious way to do it." from the Python Zen.

Granted, tooling is different from the language itself. Although PyPA could benefit from a decade having a BDFL.

0. A lot of those "tools and concepts" are actually completely irrelevant or redundant. easy_install has for all practical purposes been dead for many years. virtualenv was the original third party project that formed the basis for the standard library venv, which has been separately maintained for people who want particular additional features; it doesn't count as a separate concept. The setup.py file is a configuration file for Setuptools that also happens to be Python code. You only need to understand it if you use Setuptools, and the entire point is that you can use other things now (specifically because configuring metadata with Python code is a terrible idea that we tolerated for far too long). Wheels are just the distribution format and you don't need to know anything about how they're structured as an end user or as a developer of ordinary Python code — only as someone who makes packaging tools. And "pypy" is an alternate implementation of Python — maybe you meant PyPI? But that's just the place that hosts your packages; no relevant "concept" there.

Imagine if I wanted to make the same argument about JavaScript and I said that it's complicated because you have to understand ".tar.gz (I think, from previous discussion here? I can't even find documentation for how the package is actually stored as a package on disk), Node.js, NPM, TypeScript, www.npmjs.com, package.json..." That's basically what you're doing here.

But even besides that, you don't have to know about all the competing alternatives. If you know how to use pip, and your only goal is to install packages, you can completely ignore all the other tools that install packages (including poetry and uv). You only have any reason to care about pipenv if you want to use pip and care about the specific things that pipenv does and haven't chosen a different way to address the problem. Many pip users won't have to care about it.

1. A lot of people actively do not want it that way. The Unix philosophy actually does have some upsides, and there are tons of Python users out there who have zero interest in participating in an "ecosystem" where they share their code publicly even on GitHub, never mind PyPI — so no matter what you say should be the files that give project metadata or what they should contain or how they should be formatted, you aren't going to get any buy-in. But beyond that, different people have different needs and a tool that tries to make everyone happy is going to require tons of irrelevant cruft for almost everyone.

2. Reverse compatibility. The Python world — both the packaging system and the language itself — has been trying to get people to do things in better, saner ways for many years now; but people will scream bloody murder if their ancient stuff breaks in any way, even when they are advised years in advance of future plans to drop support. Keep in mind here that Python is more than twice as old as Go.

3. Things are simple for Go/JS/TS users because they normally only have to worry about that one programming language. Python packages (especially the best-known, "serious" ones used for heavyweight tasks) very commonly must interface with code written in many other programming languages (C and C++ are very common, but you can also find Rust, Fortran and many more; and Numpy must work with both C and Fortran), and there are many different ways to interface (and nothing that Python could possibly do at a language level to prevent that): by using an explicit FFI, by dlopen() etc. hooks, by shelling out to a subprocess, and more. And users expect that they can just install the Python package and have all of that stuff just work. Often that means that compiled-language code has to be rebuilt locally; and the install tools are expected to be able to download and set up a build system, build code in an isolated environment, etc. etc. All of this is way beyond the expectations placed on something like NPM.

4. The competition is deliberate. Standards — the clearest example being the PEPs 517/518/621 that define the pyproject.toml schema — were created specifically to enable both competition and interoperation. Uv is gaining market share because a lot of people like its paradigms. Imagine if, when people in the Python community first started thinking about the problems and limitations of tools from the early days, decided to try to come up with all the paradigms themselves. Imagine if they got them wrong, and then set it in stone for everyone else. When you imagine this, keep in mind that projects like pip and setuptools date to the 2000s. People were simply not thinking about open-source ecosystems in the same way in that era.

> They should remove the "There should be one-- and preferably only one --obvious way to do it." from the Python Zen.

First, define "it". The task is orders of magnitude greater than you might naively imagine. I know, because I've been an active participant in the surrounding discussion for a couple of years, aside from working on developing my own tooling.

Second, see https://news.ycombinator.com/item?id=44763692 . It doesn't mean what you appear to think it does.

Python packaging is (largely) solving problems that Go and Node packaging are not even trying to address.

- there's no way to do an installation dry run without pre-downloading all the packages (to get their dep info)

- there's no way to get hashes of the archives

- there's no way to do things like reverse-search (show me everything that depends on x)

I'm assuming that a big part of pyx is introducing a dynamically served (or maybe even queryable) endpoint that can return package metadata and let uv plan ahead better, identify problems and conflicts before they happen, install packages in parallel, etc.

Astral has an excellent track record on the engineering and design side, so I expect that whatever they do in this space will basically make sense, it will eventually be codified in a PEP, and PyPI will implement the same endpoint so that other tools like pip and poetry can adopt it.

[1]: Top-level: https://pypi.org/simple/ Individual package: https://pypi.org/simple/pyyaml/

Python packaging for for Python only modules has never been a problem. When people say they hate python packaging they are usually talking about being able to successfully install dependencies without much thinking. But, the biggest reason that doesn't work is because of the dependencies that have to be compiled. Which brings it's own problems. Have you ever had a c dependency on node or ruby on a system that wasn't the same system they built it with? Turns out it sucks in all the languages. It's just that the amount of c-level packages in python is quite larger than say ruby. The likelihood of a problem is significantly larger.

Especially in the GPU accelerated space.

Python is used heavily for DS/ML/AI, which is exactly the area where native code packages are necessary and prevalent. Worse yet is that those packages often involve GPU code, and things like CUDA bring their own complications.

If you're writing web apps in Python, dependencies haven't really been a problem for a long time now.

(As the user in question: three times.)

> Leverage progress in the systems programming ecosystem to create repeatable builds. Turn prebuilt binaries from “sources” into cacheable artifacts that can be deleted and reconstructed at will. Institute a way of creating secondary caches that can start shouldering some of the workload.

This doesn't avoid the need for the wheels to exist and be publicly available. People running CI systems should figure out local caching that actually works, sure. But if you delete that cacheable artifact on the public PyPI website for something like, I don't know, numpy-2.3.2-cp312-cp312-win_arm64.whl, you're going to be re-creating it (and having it downloaded again) constantly. Windows users are just not going to be able to build that locally.

And you know, space usage isn't the problem here — we're talking about a few hard drives' worth of space. The number of downloads is the problem. Outside of CI, I guess that's mostly driven by end users defaulting to the latest version of everything, every time they make a new virtual environment, rather than using whatever's in their package installer's cache. I do know that uv makes the latter a lot easier.

To be precise: pyx isn't intended to be a public registry or a free service; it's something Astral will be selling. It'll support private packages and corporate use cases that are (reasonably IMO) beyond PyPI's scope.

(FD: I work on pyx.)

> pyx is also an instantiation of our strategy: our tools remain free, open source, and permissively licensed — forever.

From the guidelines [1]

> Please don't post shallow dismissals, especially of other people's work.

[1] https://news.ycombinator.com/newsguidelines.html

PYX is a package registry, and therefore an alternative to PyPI (like how JSR is an alternative to NPM).

The alternative to `pip` that Astral has built is called `uv`. Feel free to not use that either, but personally I'd consider it if I were you. It has full pip compatibility (with significantly more speed), and many other nice features besides.

https://en.wikipedia.org/wiki/Pyx

the two ways of spacing the em dash on that quote is a joke about how it's not actually possible to do that? (And there's a third way in another line of the zen)

Do I buy it? Not sure. But apparently there’s more to this line than what it suggests.

The dependencies in question are compiled C code that Python interfaces with. Handling dependencies for a graph of packages that are all implemented in pure Python, is trivial.

C never really solved all the ABI issues and especially for GPU stuff you end up having to link against very specific details of the local architecture. Not all of these can be adequately expressed in the current Python package metadata system.

Aside from that, a lot of people would like to have packages that use pre-installed dependencies that came with the system, but the package metadata isn't designed to express those dependencies, and you're also on your own for actually figuring out where they are at runtime, even if you take it on blind faith that the user separately installed them.

This is not really my wheelhouse, though; you'll find a much better write-up at https://pypackaging-native.github.io/ .

uv becomes a client that can install those packages from your private registry.

I imagine pip will be able to install them from a PYX registry too.

I don't understand why people who don't do weird AI stuff would use any of that instead of sticking to distribution packages and having the occasional 1 or 2 external modules that aren't packaged.

It's the default installation path for aider, for instance.

Still don’t get how they are solving what they claim to solve.

  deno run http://uri.of.the/dang.program.ts