Oh, that’s quite fancy hardware.

Hmm… Unless exllama is explicitly recommended by NVIDIA for that particular GPU and setup, it seems “risky”. vLLM seems to be the popular choice for most “production” systems. I’m switching from llama.cpp to vLLM because of better performance and its the engine recommended by most model providers. I don’t really have the time to benchmark, so I’ll just do what the documentation says. And it’s really hard to do good benchmarks. Especially when “qualitative language performance” can vary for the same weights on different hardware/software.

With that kind of hardware, I would do exactly what NVIDIA and your model provider(s) say. Otherwise you might waste a lot of GPU power.

TLDR: Yes, it matters. Especially when it comes to inference and “new” features and hacks it relies on.

What GPU and what inference engine are you using?

On Debian I would use the stable version (not old stable) and I would enable nonfree firmware and also the backports version of the kernel and nonfree firmware. Then you’re probably set for a year or two.

An old kernel with only free firmware likely performs much worse. Look at the release logs of the Linux kernel and any GPU driver.

If your hardware is very old, it probably doesn’t matter super much. But sometimes it does (like when a manufacturer decides to unlock some sleeping feature in an old forgotten device).

Git

Firefox (or some derivative). But I guess the real solution is to make that Linux switch.

I wish Ununtu Touch switched name, since its neither Ubuntu nor Canonical any longer.

Debian and then maybe Guix on top of that. Rootless Podman for services.

There is no good reason to choose Arch in 2025. If you want to feel special, NixOS or Guix System is the way to go.

I think Guix is way more coherent than Nix. It also has better documentation and a more friendly community. And you use Scheme instead of Nix lang.

New non-copyleft Rust implementation. While we’re at it, let’s throw in some blockchain and AI as well. The eccentric South African billionaire CEO will be pleased.

This is what it’s all about. We all know this.

I’m pretty sure bugs were accepted into the Linux kernel well before the existence of LLMs.

Readest. But I’m looking for a free app with good TTS and (maybe this breaks the free) which is able to handle DRM content from Adobe digital editions (Or is able to remove the DRM). Unfortunately, my library only provide books through ADE.

How dare you!

Wow! There’s one with a brain over here!

How dare you! Can’t you see there’s a circle jerk in progress?

deleted by creator

A fully open-source LLM

As a fully open language model, Apertus allows researchers, professionals and enthusiasts to build upon the model and adapt it to their specific needs, as well as to inspect any part of the training process. This distinguishes Apertus from models that make only selected components accessible.

“With this release, we aim to provide a blueprint for how a trustworthy, sovereign, and inclusive AI model can be developed,” says Martin Jaggi, Professor of Machine Learning at EPFL and member of the Steering Committee of the Swiss AI Initiative. The model will be regularly updated by the development team which includes specialized engineers and a large number of researchers from CSCS, ETH Zurich and EPFL.

Its good for legacy MATLAB projects. Use Python for new projects.

And weird, since the model is licensed under Apache 2.0.

What did we expect?

deleted by creator

If one wants rolling, I would suggest NixOS, Guix System or Tumbleweed. Or something container based like Silverblue or openSUSE micro.

But rolling doesn’t really make sense. Just go with Debian/Leap and then use Flatpak, podman, Nix and/or Guix on top of that. For Desktop.