musicmatzes blog

nix

I am happy to announce that I am officially a (part-time/side-job) #freelancer for #rust #rustlang and #nixos #nix now!

You noticed the recent spike of Rust in your timelines? That's because Rust is the best thing since sliced bread! But Rust is not easy to learn!

That's where I come in! I am the guy you want for the time after the training of your developers! Because no training makes you a rockstar #rustlang programmer!

#hireme if you want to try out Rust in one of your projects to help your developers succeed! Whether it is #codereview, #consulting for your Rust experiments or for actual #softwaredevelopment, I can help your team get up to speed with #rustlang!

You also heard about #nix or #nixos and want to try it out? Let me help you exploring the ecosystem of functional package management to speed up your CI, development workflow and making your deployments reproducible!


For the readers of my blog: Above is clearly an advertisement. I won't post more ads on my blog, so be assured that this blog won't transform into an advertising machine!

This is mostly a reminder for myself how to do it, because the documentation non the nix side is pretty sparse.

What I want to do is to build imag on a remote server using nix 2.0 features - nix remote builds.

Here are the steps I did for achieving that goal:

  1. build the Cargo.lock file with cargo generate-lock for the imag workspace
  2. generate a build.nix file using carnix --standalone --src . --output build.nix Cargo.lock
  3. monkey-patching the generated nix file because carnix generates wrong pathes for the build.rs file in the nix expressions. I did that with a simple substitude pattern in vim: %s/"\.\.\/\.\.\/\.\.\/build\.rs"/.\/build.rs/
  4. ensure the remote machine has nix 2.0
  5. use NIX_REMOTE=ssh-ng://user@host nix build -f ./build.nix imag_0_7_0 to build imag on the remote machine.

Unfortunately, because the generated expressions do not use cargo but execute the rustc compiler directly, the build fails because I need the CARGO_BUILD_VERSION and CARGO_PKG_VERSION environment variables which are set by cargo during the build.

I hope carnix gets support for automatically adding those variables to the build soonish, so I can build imag completely remote.

tags: #imag #nix #rust

When working with Rust on NixOS, one always had the problem that the stable compiler was updated very sparsely. One time I had to wait six weeks after the rustc release until I finally got the compiler update for NixOS.

This is kind of annoying, especially because rustc is a fast-moving target and each new compiler release brings more awesomeness included. As an enthusiastic Rust programmer, I really want to be able to use the latest stable compiler all the time. I don't want to wait several days or even weeks until I can use it.

Use the overlay, Luke!

Finally, with NixOS 17.03 and unstable as of now, we have overlays for nixpkgs. This means that one can “inject” custom nix expressions into the nixpkgs set.

Meet the full awesomeness of nixpkgs!

Soon after overlays were introduced, the Mozilla nixpkgs overlay was announced on the nixos-dev mailinglist.

Now we can install rustc in stable, beta and nightly directly with pre-build binaries from Mozilla. So we do not need to compile rustc nightly from source on our local machines if we want the latest great rust feature.

This is so awesome. Everybody should use the rust overlay now, in my opinion. It's really convenient to use and gives you more flexability with your rust installation on NixOS. So why not?

tags: #nixos #nix #rust #software

When the last semester came to an end, I noticed that my Thinkpad behaved weird. I couldn't nix-store --optimize it, and some other things began to fail silently. I suspected the SSD was dying, a Crucial C400 with 256GB. So I ran the smart tools with a short test – But it told me everything was alright. Then I ran the extended self-test on the drive and after 40% of the check (60% remaining) it told me about dead sectors, nonrecoverable.

So I got a new SSD and installed NixOS from my old installation. Here's how.

So I got a nice new Samsung EVO 850 PRO with 256 GB. I was really amazed how light these things are today. No heavy metal in there like in a HDD!

Preparation

First of, you need to prepare your current installation. Make some backups, be sure everything is fine with them.

Then, verify that your configuration.nix and your hardware-configuration.nix file list your partitions not by UUID, but by /dev/sda1 and so on. That could be really helpful later.

If you have some crypto keys you need to keep, maybe make another backup of them.

The installation

First of, we need to format the new drive. Use gdisk for this if you have a UEFI setup like with an Thinkpad X220. Format your partitions after that. Make sure that your boot partition is formatted as vfat (fat16). I don't know why, but it is only possible to boot from vfat, according to the nixos documentation. Also, do your cryptsetup.

For simplicity, I refer to the boot partition by /dev/sda1 and to the root/home partition as /dev/sda2 – you can, of course, have more partitions, maybe for a seperate /home. But I saw no need for it. With only one partition I do not have to take care of the size of the /nix/store and if I have few things in the store I can grow my music collection a bit – so I'm really flexible. And yes, I know about LVM, but I really don't need these things, do I?

Now, mount the partitions as follows:

  • /dev/sda2 in /mnt
  • /dev/sda1 in /mnt/boot (you might need to mkdir this directory first)

Ensure things are properly mounted. This broke my neck twice during my installation, as /mnt/boot wasn't mounted properly and I failed to rebuild the system. Took me some time to see this, actually.

Now you can nixos-generate-config --root /mnt. After that you might want to modify your configuration.nix file in the newly generated setup under /mnt/etc/nixos/configuration.nix – I did not! I nixos-install --root /mnted to get a minimal bootable system.

Then I rsync -aed my /home/$USER to /mnt/home/ and symlinked the configuration.nix (which lives in /home/$USER/config/nixos/$HOSTNAME.nix on my machines) to /etc/nixos/configuration.nix. I renamed the host as well, to avoid confusion.

Then, I shutdowned, removed the old SSD, assembled the new one and booted. I had some problems with failing mounts during boot (because I had mount operations specified by UUID rather than via /dev/sdaX). I got a rescue shell and was able to fix things up. After several reboots I was able to get my system up and running.

When I was able to boot my minimal installation, I just followed the manual and created my user and so on. Then, I nixos-rebuild switched. And because I copied my whole configuration.nix setup from my old drive, everything got build for me.

After some more nix-env -iA calls (because some things only live in my user environment), I fully restored my system. Awesome!

Conclusion

Installing NixOS from NixOS works really nice. You have to be careful with some things, UUIDs and so on, but overall it is rather simple.

Anyways, you benefit if you really know your system. I wouldn't necessarily recommend this to an inexperienced NixOS user – hacking things into the TTYs and getting a rescue shell for fixing the installation is no thing that a newbie really wants to do – except for learing and if backups are at hand!

Because of the awesomeness of NixOS and the configuration.nix file, I was able to rebuild my complete system within a few minutes. Despite my extensive adaptions in my configuration.nix file – speaking of container setup, custom compile flags for packages, custom vim setup with plugins compiled into the vim derivation (and the same again for neovim), hundreds of packages and stuff – I was able to rebuild my system without much effort.

Overall, leaving out the UUID fail, I think I am able to redo a complete setup (including syncing /home, which was ~100GB data, and reinstalling everything) in maybe 90 minutes, depending on how fast the internet connection is for downloading binaries.

One could even mount the old /nix/store from the old installation and copy over derivations, which would be a hell lot faster and would result in a reinstallation without the need for internet access. But I don't know how to do it, so I leave it as exercise to the reader.

tags: #desktop #linux #nix #nixos

This article is about the annoyance of programming language tutorials. First, I will argue why learning new programming languages always feels a bit like a pain to me. And then I will tell you why learning Rust was different.

I know a few programming languages. I've done things in C, Ruby, Java, Python, PHP, C++, Bash, Lisp (Racket, actually), Haskell, LaTeX, Nix, Make, Lua* and JavaScript*.

*: Only a bit, but still enough to say “I've done things in this language”

When I started learning programming, I learned Java and shortly after that Ruby. Both languages have their oddities, but besides their overall concept (imperative programming and object orientated programming) they are both rather easy to learn. After that I learned C. Learning C was a new world for me back then, as I only knew object orientation. I cannot remember which resources I've been using learning C, though I remember it was a pain. I had a good friend helping me, which was awesome, though I struggled a lot with the new concept.

After learning C, I did a fair amount of PHP. Learning PHP was simple (I actually did it in one weekend and a few thousand lines), though it again was a pain. It was a pain because of the tutorials and resources. And that's the point I want to discuss here.

All other programming languages I've been learning so far were painful to learn. Not because of their concepts (see Haskell, functional programming at all), but because of the resources. Haskell is a really good example here. There is exactly one resource I enjoyed while learning Haskell, and that's “Learn you a Haskell for great good”. And why? Because it takes your hand and shows you the new world. It is good because the concept of functional programming is new for most people reading this resource. Learning JavaScript, Bash, Lua or Python sucks because you already know what an object is. You know what a function or method is and you know that the basic syntax for arithmetic is the very same as in the languages you've been using by now. But all tutorials expect you start from zero – which is fine if you're actually starting from zero, but most of the time you're not. After learning Java and Ruby I understood what OOP is (actually not after learning Java, but after learning Ruby, because Java isn't a fully object oriented language in my opinion).

So, learning Haskell was kinda amazing. Learn you a Haskell is a really great book and if you want to learn Haskell, you really should read it. But what was even a better experience was learning Rust.

So why is that? It is because Rust is a new concept and the old concept (imperative programming). Learning Rust is not like learning Python after learning Ruby. It is like Learning Haskell after learning Racket. The concept behind the language is kinda the same, though it is different in most details.

Plus, the Rust tutorials and resources are freakin' awesome. Let me repeat that. The tutorials and documentation of the tools, libraries and everything, the resources on how to do things, are awesome!

Every documentation for each library looks exactly the same, which is a huge advantage over other languages library documentations. cargo, the Rust build-tool builds the documentation locally for you, if you want that. So working offline after you fetched the dependencies or your project (also via cargo) is no hassle. Building a new project with cargo is absolutely painless, its literally just executing a command, editing a textfile and executing another command (setting up the project, writing down dependencies and meta information about the project and building the project, which fetches the dependencies).

So really, if you want to learn a new programming language, check out Rust. I haven't even talked about its language features, and I will not, as this is not the goal of this article. This article is about resources and tutorials. And my conclusion is:

All programming language tutorials suck. Except the Rust language tutorial. tags: #programming #open source #bash #c #c++ #haskell #nix #racket #rust #shell

From time to time it can happen that nixos-rebuild fails due to a package build error.

Here is how to fix this.

So your nixos-rebuild tells you something like this:

(hashes removed for readability)

building Nix...
building the system configuration...
these derivations will be built:
  /nix/store/<hash>-libvterm-v2015-02-23-src.drv
  /nix/store/<hash>-neovim-libvterm-2015-02-23.drv
  /nix/store/<hash>-neovim-0.1.0.drv
  /nix/store/<hash>-neovim-0.1.0-configured.drv
  /nix/store/<hash>-system-path.drv
  /nix/store/<hash>-dbus-conf.drv
  /nix/store/<hash>-unit-dbus.service.drv
  /nix/store/<hash>-unit-polkit.service.drv
  /nix/store/<hash>-system-units.drv
  /nix/store/<hash>-etc.drv
  /nix/store/<hash>-nixos-system-yuu-16.03pre71289.7ae05ed.drv
building path(s) ‘/nix/store/<hash>-libvterm-v2015-02-23-src’
builder for ‘/nix/store/<hash>-libvterm-v2015-02-23-src.drv’ failed with exit code 1
cannot build derivation ‘/nix/store/<hash>-neovim-libvterm-2015-02-23.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/<hash>-neovim-0.1.0.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/<hash>-neovim-0.1.0-configured.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/<hash>-system-path.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/<hash>-dbus-conf.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/<hash>-unit-polkit.service.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/<hash>-unit-dbus.service.drv’: 1 dependencies couldn't be built
cannot build derivation ‘/nix/store/<hash>-system-units.drv’: 2 dependencies couldn't be built
cannot build derivation ‘/nix/store/<hash>-etc.drv’: 2 dependencies couldn't be built
cannot build derivation ‘/nix/store/<hash>-nixos-system-yuu-16.03pre71289.7ae05ed.drv’: 2 dependencies couldn't be built
error: build of ‘/nix/store/<hash>-nixos-system-yuu-16.03pre71289.7ae05ed.drv’ failed

And you don't know what to do? Well, there's a solution to that, and the solution is rather simple, actually.

Step 1: Find the broken package

If the break happens due to an error in your configuration.nix, you should fix this error. Then you don't need to go on reading here.

If the build fails due to an error in the nixpkgs, as above, you should continue reading.

So we have to find the broken package. From the backtrace nixos-rebuild shows us, we can find out that the break happens due to an error in the libvterm package. This is clearly an error in the nixpkgs and we can fix this error only with the nixpkgs available, so...

Step 2: Get nixpkgs

If you havn't already cloned the nixpkgs repository, you need to do this first. You also need to fetch the unstable channel commit, which is provided by the nixpkgs-channels repository:

git clone https://github.com/nixos/nixpkgs && cd nixpkgs
git remote add channels https://github.com/nixos/nixpkgs-channels

Step 3: Check out the commit of your system

Now you should have a commit which shows the current unstable channel. You can check this like so:

nixos-version # 16.03.git.ab0a7e9 (Emu)
git show nixos-unstable # commit ab0a7e9

If the commit is the same as the hash which is shown from calling nixos-version you have found the channel commit your system is build from. You can now check this commit out and create a new branch in the nixpkgs from it.

git checkout ab0a7e9
git checkout -b nixos-unstable-fixes

Step 4: Search for fixes.

Sometimes there are already fixes available in the master branch of nixpkgs, but are not in unstable yet. You can simply do something like this:

  1. Find out where the broken package lives
  2. Get the log from the current unstable channel commit to master
  3. Apply these fixes in your -fixes branch
  4. Rebuild the system with the local nixpkgs

If there are no fixes available, you have to fix the build yourself and I cannot help you with that, because the build problems are often very package-specific.

We already know that libvterm is the problem here.

grep -rnil libvterm pkgs/
# or 'ag -l libvterm pkgs', which is much faster

This gives us four possible locations:

pkgs/top-level/all-packages.nix
pkgs/applications/window-managers/vwm/default.nix
pkgs/applications/editors/neovim/default.nix
pkgs/development/libraries/libvterm/default.nix

We also know (from the backtrace) that it has something to do with neovim. So we check the changes of this one first:

git log --oneline nixos-unstable..master -- pkgs/applications/editors/neovim/default.nix
2617919 neovim: fix wrong ad30764d68a

Well, we found a fix. Let's apply it!

git checkout nixos-unstable-fixes # just to be sure
git cherry-pick 2617919

Now we can rebuild our system with this patch applied:

sudo nixos-rebuild switch -I nixpkgs=/home/user/nixpkgs/

... and our system is updated. Now you can try to update packages the same way using the diff from nixos-unstable to master and apply the appropriate patches onto your -fixes branch. After a channel update you can simply rebase the -fixes branch onto the new channel commit. If rebase fails due to conflicts (which shouldn't happen that often) you can just hard-reset your -fixes branch onto the unstable channel commit and re-apply patches as needed.

tags: #nix #nixos #software

This post is about how to setup vim (vim_configurable) on NixOS with reusable parts and the possibility to replace vim with neovim without rewriting the whole setup.

configuration.nix

What we have in the configuration.nix file is simple:

environment.systemPackages = let
    vimPackages = import ./some/directory/vimPackages.nix pkgs;
    # neovimPackages = import ./some/directory/neovimPackages.nix pkgs;
in
    vimPackages ++ with pkgs; [ some other packages ];

You can see that we simply import some nix expressions and pass them the pkgs of from our configuration.nix file. Everything else is done in this file and some sub-files.

vimPackages.nix

So here starts the fun. We define a function which gets the pkgs variable and the lib variable from the outside scope (we don't pass the lib variable, though nix is smart enough to figure out what we mean). We already know that we need to return an array of packages, so we can return vim and some things we use in our vim scripts, like so:

# ...
[ vim pkgs.ctags ]

vim is not prefixed with a pkgs. here because we define it in the file itself. But first, we need to know what we want to override. vim_customize allows us to compile vim without GUI support, for example (there are a lot more options). We also want to build our vimrc and we want to put plugins into our vim setup, because this is the NixOS way of life.

customization = {
    vimrcConfig = (import ./customization.nix { pkgs = pkgs; });
} // { name = "vim"; };

Here we import our customization (vimrc, plugins, etc) and ensure that the executable is named "vim". We can now use this customization to customize vim_configurable:

custom_vim = pkgs.vim_configurable.customize customization;

and afterwards we override this derivation and set variables, what features should be compiled into vim:

vim = lib.overrideDerivation custom_vim (o: {
    ftNixSupport = true;
    gui          = false;
    luaSupport   = true;
    # some more
});

Now we put these parts together and into a function, so the whole file looks like this:

{ pkgs, lib, ... }:

let
  customization = {
    vimrcConfig = (import ./vim/customization.nix { pkgs = pkgs; });
  } // { name = "vim"; };

  custom_vim = pkgs.vim_configurable.customize customization;

  vim = lib.overrideDerivation custom_vim (o: {
    aclSupport              = false;
    cscopeSupport           = true;
    darwinSupport           = false;
    fontsetSupport          = true;
    ftNixSupport            = true;
    gpmSupport              = true;
    gui                     = false;
    hangulinputSupport      = false;
    luaSupport              = true;
    multibyteSupport        = true;
    mzschemeSupport         = true;
    netbeansSupport         = false;
    nlsSupport              = false;
    perlSupport             = false;
    pythonSupport           = true;
    rubySupport             = true;
    sniffSupport            = false;
    tclSupport              = false;
    ximSupport              = false;
    xsmpSupport             = false;
    xsmp_interactSupport    = false;
  });

in [
  vim
  pkgs.python
  pkgs.ctags
]

But that's not all to it. The customization.nix is where the real fun begins.

customization.nix

So this is the file where we define which plugins we want to use. There are some plugins which are not yet included in the nixpkgs repository, so we have to build these packages ourselves to be able to install them. Luckily, the nixpkgs provide some helper functions to build vim plugin packages rather easily.

Also, we build our vimrc file here, which consists of another bunch of script files.

vimrc.nix

The vimrc itself is written in viml of course. If you have a lot of vimrc configuration, you can split this file into several smaller ones. For example, I have split these files into the following structure:

./
├── colorschemes
│   ├── mm_neverness_share.vim
│   └── razorlight.vim
├── filetypes.vim
├── folding.vim
├── general.vim
├── gui.vim
├── insertmode.vim
├── mappings.vim
├── menuconf.vim
├── plugins
│   ├── # ... some things
│   └── ycm.vim
├── pluginconfigurations.nix
├── scripts.vim
├── searchnreplace.vim
├── statusline.vim
└── textediting.vim

So what we do in our vimrc.nix file (which later gets included in our customization.nix file) is: We include all these .vim files and put them into one giant string, which is then our vimrc file:

{ stdenv, writeText }:

let
    generic     = builtins.readFile ./vimrc/general.vim;
    textediting = builtins.readFile ./vimrc/textediting.vim;
    # ... more here

    plug = import ./vimrc/pluginconfigurations.nix;
in

''
    ${generic}

    ${textediting}

    # ... more here

    ${plug}
''

You might noticed the vimrc/pluginconfigurations.nix file. This one is just another layer of abstraction here and is basically the same as the vimrc.nix file:

let
    # some files
    ycm = builtins.readFile ./plugins/ycm.vim;
in

''
    # some variables
    ${ycm}
''

plugins.nix

Another file we need is the plugins.nix file. As said, not all vim plugins are in nixpkgs (yet). So we have to build some vim plugins ourselves (don't get scared here. This does not mean you have to run compilejobs yourself, most plugins don't contain any binaries so rebuilding the system with vim plugins installed is normally a matter of seconds).

So this one is also rather simple in the end. In the example below you can see how I build the vim-trailing-whitespace plugin. The version might be outdated, though.

{ pkgs, fetchgit }:

let
  buildVimPlugin = pkgs.vimUtils.buildVimPluginFrom2Nix;
in {

  "vim-trailing-whitespace" = buildVimPlugin {
    name = "vim-trailing-whitespace";
    src = fetchgit {
      url = "https://github.com/bronson/vim-trailing-whitespace";
      rev = "d4ad27de051848e544e360482bdf076b154de0c1";
      sha256 = "594769a6f901407609b635a5041966456bfd91b13437169a4562857544e1dca3";
    };
    dependencies = [];
  };

  # more?
}

Finally... put the parts together.

Now we can write down our customization.nix file, which imports the plugins.nix and vimrc.nix files and puts them in the right context, so the nixpkgs infrastructure understands how we want to have our vim package build:

{ pkgs }:

let
  # this is the vimrc.nix from above
  vimrc   = pkgs.callPackage ./vimrc.nix {};

  # and the plugins.nix from above
  plugins = pkgs.callPackage ./plugins.nix {};
in
{
  customRC = vimrc;
  vam = {
    knownPlugins = pkgs.vimPlugins // plugins;

    pluginDictionaries = [
      # from pkgs.vimPlugins
      { name = "youcompleteme"; }

      # from our own plugin package set
      { name = "vim-trailing-whitespace"; }
    ];
  };
}

Building

After putting these things together you can rebuild your system. The vim binary might be compiled from source, though this isn't that much of a problem because there are only few vim updates, so rebuilding vim will happen every four weeks or something, not on every system rebuild.

nixos-rebuild switch

neovim

As said, you can now use the infrastructure we just wrote to compile the very same configuration into neovim, including your vimrc, plugins and everything.

{ pkgs, lib, ... }:

let
  nvim = pkgs.neovim.override {
    # don't alias neovim to vim, yet.
    vimAlias = false;

    configure = (import ./vim/customization.nix { pkgs = pkgs; });
  };

in [
  nvim
  pkgs.python
  pkgs.ctags
]

Just make sure your vimrc settings don't mess with neovim. Some things are not supported by neovim, so you have to put them into if-else conditions like so:

if has("nvim")
  " we don't need this in nvim
else
  set clipboard=exclude:.*
endif

And that's it. tags: #nix #nixos #vim #neovim #programming

I just noticed that I have no blog article about nixos-scripts version 0.3 yet, though the tool was released exactly one month ago.

So I catch up on this now.

Changelog

The third release contains some neat things for the “switch” and the “update-package-def” command, lets start with the latter.

The “update-package-def” command is now able to translate arguments for --cores and --max-jobs now, so you can pass the number of cores or jobs to use for the build. This is extremely helpful for updating large packages. I myself happen to maintain a set of rather small packages, but from time to time I want to update packages I do not maintain, for example i3 or similar packages. Building these packages with more jobs results in much better build time, especially on my OctaCore processor.

The “update-package-def” command is now able to push to a pre-configured remote after successful build now. So your update-workflow is a bit smoother now:

  1. Get the URL for the patch from monitor.nixos.org
  2. Run “update-package-def”
  3. Create pull request on github

The “switch” command got some neat updates as well. First of, the command is now quiet by default. This means you don't get this noisy output anymore when updating your system, as nixos-rebuild is called with -Q now. The command has changed some background behaviour: The nixpkgs repository (if known) gets updated (via git fetch) before a tag is created. This way we can be sure the commit exists before creating a tag on it. Tagging is now disabled for build as command type. Also the codebase was refactored, so its maintainability was improved a bit.

Another big point is the “channel” command. We have commands for channel updates now, they also create tags in your configuration repository. My workflow for updating my system is now:

nixos-script -v channel update
nixos-script -v switch

Which updates the channel with the first command and rebuilds the system with the second command. I don't use the nix-* commands anymore for my system, really!

We have another command (“container”) for container management. This tool has three subcommands:

  1. “setup” for building a new container from a template. You can specify a configuration.nix file, which will be copied to the container after it is created. It will call your editor to customize the configuration file right afterwards and rebuild the container then.
  2. “stats” which prints a simple table with statistics about your containers (name, IP, status and host key).
  3. “kill” kills a container and gives you the option to destroy it as well.

More options will be added to the container subcommand as soon as I need them. We'll see...

The “show-generation” command is now able to show generations from a custom profile.

A tool was added for downloading sources of packages into the store. So you can download all the sources you need for building a package and then disconnect from the Internet and rebuild your system while beeing offline.

A tool I'm rather proud of is the REPL I've added. Its capabilities are limited, but I'm still a bit proud of it. So you can have a simple “shell” where you don't need to type nixos-scripts all the time. Normal bash commands are allowed as well, though functionality is limited here. It also lacks features like autocomplete and “one up for last command” or these things...

The help texts were polished.

What's up next

Well, there are some releases planned in the github issue tracker by now... Lets iterate:

0.4

This release will contain auto-complete for all the things. Nothing more, nothing less. It is something like an one-feature-release. I hope to get this done soon, but at the moment I really have no time to work on nixos-scripts, so ... feel free to send me pull requests to help me there.

0.5

The 0.5 release will introduce user-defined functions. So the user is allowed to put custom code in the configuration file and these functions will then be used for generating the tag names for example.

I'm really looking forward to this, as I do not want to restrict the user in matter of tag names my scripts generate in the users configuration repository. Backward compatibility will be provided, though.

Another feature I want to include is a (sort of) meta-command, which updates and cleans the system in litterally one step:

  1. Update channels
  2. Rebuild the system
  3. Remove old generations
  4. run the GC
  5. Offer a reboot of the system

0.6

As the 0.4 and 0.5 releases are sort of one-feature-releases, this release will introduce all the small fixes and cleanups, new functionality for commands and so on. As this milestone gets rather big on github, I may split it into 0.6 and 0.7, I don't know yet.


So that's it. I apologyze for writing this post now, not one month ago. Though I'm not committing to the repository at the moment, the project is not dead, really! Feel free to send pull requests and issues!

Stay tuned! tags: #linux #nix #nixos #software #tools

Everyone starts using shell environments, where custom variables are available, custom bash functions are enabled, special commands are available, maybe even some additional programs are installed.

Projects on github pop up for handling these things, installing bash “environments” and so on. And I have to ask: Why?

Why would you do that?

I tell you, there's a solution to all of your problems. It is called “nix”. It is a package manager which provides you opportunities, you've never dreamed of!

Yes, you could hack the functionality of these project-environment-manager things yourself (that's what people did, then they put it on github and now you see it and decide to write your own becaues you can do it better). You could use this piece of clusterbash and suffer from impurities you'd never expect. Or, you could just use the purity of the nix package manager to deploy the world in a shell.

With the nix package manager, you can use custom project-based environments to install packages neither globally nor in your user package environment:

nix-shell -p haskellPackages.pandoc

for example installs “Pandoc” and starts a shell where it is available. The executable is not available anymore after you type exit. It is not installed in your system, it is not even installed in you user package set. It was available in this shell and only will be available again after you entered this shell via nix-shell.

I have a lot of packages...

... so have I. Normally, I install one or two packages via commandline in my nix-shell environment. If there are more packages or I need the packages on multiple systems (when developing something, for example), I add a default.nix file to the repository:

{ stdenv, pkgs ? (import <nixpkgs> {}) }:

let
  env = with pkgs; [ racket ];
in

stdenv.mkDerivation rec {
    name        = "blog";
    src         = ./.;
    version     = "0.0.0";
    buildInputs = [ env ];
}

The I only have to type nix-shell to enter the environment.

Another idea would be to install the packages “persistent”, so after a call to nix-collect-garbage won't remove the packages from the ominous store where the packages live:

nix-instantiate . --indirect --add-root $PWD/shell.drv
nix-shell $PWD/shell.drv

(copy-pasted from the nixos-wiki)

tags: #bash #nix #nixos #programming #software

I started a project some time ago, where I develop a template for a wiki which is statically compiled into a html page. As the template is more mature now, I want to introduce you wiki.template and explain why I wrote it.

Once upon a time...

If you read this blog you know that I'm a member of the NixOS community. The NixOS community has, of course, a wiki. And the quality of this wiki is bad. No really, it is just one large page (the landing page) linking to sub-pages which are more or less maintained and contain only snippets of information. Sure – there's a lot of content in there, but (especially beginners) one has to search for it. You can't simply find it.

That's why I wanted to have a new wiki for the NixOS community. And because I like git, I wanted the content to be stored in a git repository.

So I came up with the idea to recreate the NixOS wiki and start it from scratch. Of course, the content of the original wiki has to be migrated to the new wiki then, but this has to be done carefully and by hand to ensure that things are ordered and easy to find afterwards.

The requirements

The requirements where rather simple, but nevertheless important to me. As already said, I wanted to have the content version controlled with git. The templates and markups for other data should be version controlled as well, if possible. This way, the content (or whole wiki) can be distributed with git, hosted on github or similar hosting platforms and so on.

I wanted to be able to customize the style of the wiki completely and I wanted to be able to write snippets of markup which can be reused, such as a warning-alert template where you can pass custom text and get a red alert box in your wiki page. Pretty normal stuff for a wiki.

Syntax highlighting, TOC-generating, all these things should be integrated or available through plugins (or there should be the possibility to write such plugins).

So I searched for wiki software with git backends...

The state of git-backend-driven wiki software

The state of wiki software with git backend is bad. No, it is even worse. There are actually two projects I had a look at and one which is written in Perl (and I didn't have a look on this one, because Perl).

gitit

First of all, there is gitit. Gitit is a wonderful piece of Haskell software. It works great and is fast. But it has its issues:

  1. No Templates. Period.
  2. Content is checked in, but neither templates nor style information
  3. Haskell. So writing plugins – nah, not really.
  4. Just few plugins available

So it was pretty clear that gitit is not an option for me.

gollum

Gollum was the other alternative. Developed by github, ruby thing, sounds well, right?

But. There's a big but.

  1. No templates
  2. No plugins
  3. No custom styling or at least no documentation about it

It was clear to me: No option!

The solution

Well, the solution? Just write it yourself! And that's what I did. I started to work on a template for a wiki which is statically compiled with nanoc. Later I switched to jekyll.

Compiling the wiki to a static site has several advantages. First of all, hosting the site is cheap. A common web server can host thousands of pages without even noticing. Of course, you are also protected against hacking and all this stuff, as you do not have the problems dynamic websites have.

But these are common advantages of static compiled sites. An even bigger advantage is, that contributions to the wiki must pass a review-process of some kind. Of course, I will not grant push access to the repository to anyone and of course nobody gets access to merge pull requests, besides myself. I will protect myself against pushes to master as well, so each change to the wiki has to be reviewed. And that's what matters when it comes to quality: Changes must be reviewed.

Besides this big advantage, I do not have to care about updates (of the compilersoftware). I can do them locally and roll back if they fail. I do not have to keep track of spam bots, security updates, user account data, etc. etc. etc.

So here's the story how I wrote the template:

wiki.template

I started the project with nanoc, the static site compiler I love. The project is at version 4 these days and the compiler is really really good. It is flexible, scales well, fast and a joy to work with.

I just failed integrating foundation, the CSS framework I wanted to use. I played around with Jekyll a bit, but wasn't able to get Haml working. But at some point, foundation seemd to be more important than Haml, so I switched to Jekyll.

I also thought of the option to build the site with Jekyll directly from the github repository and build a github page with it, just because this way I can save money for hosting. I still think of that option, but as I use Jekyll with plugins I cannot build it on github directly. So I searched for tutorials on how to build it with travis and deploy my github page from travis. This is documented and a known as working.

Current state of wiki.template

To get my stuff working, I had to use some plugins. And these plugins basically work for me, but the code was either a mess or lacked features I wanted to have. So what did I do? Right, I forked them and send pull requests to the maintainers.

At this very point in time, I have three pending pull requests to these plugins, two of them just normal “cleanup codebase” PRs, one feature. I also opened issues for other features I do not necessarily need but would like to see implemented. I would implement them myself, but well... not enough time.

So what is the current state? Well, wiki.template is basically usable. I want to integrate some more features and it completely lacks of documentation on how to use it.

But these things will be done soon and I hope I can then create a fork of it to start my NixOS Wiki.

The NixOS Wiki

I already announced the NixOS Wiki fork (semi) officially on the NixOS mailinglist. Of course, as always in such communities, a discussion started on whether to do such a thing or not, which software to use and whatever. I don't care about this, I will simply do this and we will see whether it succeeds or not. Discussion all over the place helps nobody.

tags: #git #linux #nix #nixos #open source #programming #software #wiki