Dear Iceland,

this is not only a love letter, but also a letter which should give you moments of thought.

I love you. Your nature is most beautiful, most astonishing, most cruel and also soft nature I have ever seen and I will likely ever see in my life. Your rivers, mountains, waterfalls and highlands are incredible. Your harsh winds keep one cold and shivering even if the sun burns. The sky looks more blue than anywhere in the world and the waters look more clean than air itself.

I'm by no means a poet, so I really hope anyone who reads this understands what I want to say: I'm in love with Iceland.

Although the land can be harsh and cruel, the winds can freeze you and I bet the snow in the winter can kill you easily, I have to say that Iceland is now my favourite country.

Sadly, there are bad things about Iceland, too. But these are human made and therefore they could be fixed.

First of all, please Iceland, fix your Roads! It is unbelievable how bad they are. We always laughed when the road signs said “80” or “60” (Km/H) because one could hardly drive 40 on those roads because of all the bumps and holes in it. We had holes that where about 10cm deep into the road.

Secondly, please fix your camping sides. With two toilets for two hundred people, that's not gonna work for any longer. Especially not for 30€ (3000 IK) for two people per night. For this price, one can have a nice camping side in France, Spain, Italy or Germany (and other countries in central Europe) with tens of toilets, ehashing places, a common room and incl. electricity. So if you want the hikers to stay at camping sides, you either have to lower the prices or provide more luxery – and I'd suggest lowering the prices because not-so-wealthy people will still camp in the wild if the prices are too high. The mini-campers will continue to shit into your landscape if camping prices are too high.

And then, of course, the many people. We talked to a ranger during our trip which said the same: There are too many tourists coming to the country. I guess the problem is not that there are too many people, that's just annoying for everyone. The real problem is that there are too many people who do notbcare about nature, but about their pictures and how they can brag about their great vacation and about the big car they drove and rivers they crossed. They don't care about your nature, running through protected area, not caring about the breeding birds they disturb and such. You, Iceland, should put fines on that. And by fines I do not mean “fines” but FINES. Let them pay. Let them help with their money fix your roads. Let each of them play 1000 € or $ if they run trough protected area. And don't tell me it is hard to do so if nobody watches: We saw a ranger who clearly saw what was happening and who did not care about it.

Despite these issues, which clearly can be solved. It just needs someone to step in in your politics and do it.

So, dear Iceland, please think about your issues. It will help to keep you as beautiful as you are. And you're clearly the most beautiful county on this planet. I really want to come back some time and see more of your nature.

Sincerely yours.

tags: #life #traveling #iceland

This was written on 9th August, during my trip through Iceland.

On this day we witnessed an accident of three young guys from Spain. They wanted to cross a river with their car, but got stuck and the car broke down. The complete car was in the river and drowned up to the steering wheel. After we pulled them out with our car (which thankfully has the right tools to pull 9 tons of weight) we noticed that the engine was broken. The three guys, luckily, got out soon enough and were not hurt (but in shock, especially the driver).

I can assure you, their vacation was over at that point. And you know what: This was the first day after they have rented the car.

What drives me to write this article is not their miserable situation or how lucky they were coming out of this alive (five meters further down the river they would be all dead now, probably). Also not because I want to shed some light on us because we are the saviours in this situation (we were just at the right place at the time, lucky for them).

What I really want to talk about is the number of strangers that cared more for the good pictures they got out of this as for helping. Overall it took us about 2.5 hours to get the car out of the river, the guys to calm down, drink some tea and call the car rental company. There were about 50 people passing by in this time, only three groups asked whether help was needed. All others stopped, took a few pictures and continued their journey.

Of course we also documented every step we made with my camera, but we did it while we helped, not while we stood there laughing our asses off (I don't actually think anyone laughed their asses of because it was clearly visible that these guys were in a really dangerous situation down there).

After we got them calmed a bit (one was literally shaking, not only because of the cold water he fell into, but also of fear of having to pay for the whole mess), we brought them to the next camping side from where they could call they police and car rental company.

tags: #life #traveling #iceland

Here I want to describe how I plan to refactor the logging back end implementation for imag.

This post was published on imag-pim.org as well as on my personal blog.

What we have

Right now, the logging implementation is ridiculously simple. What we do is: On every call to one of the logging macros, the log crate gives us an object with a few informations (line number, file, log message,...) – we apply our format, some color and write it to stderr.

This is of course rather simple and not really flexible.

What we want to have

I want to rewrite the logging backend to give the user more power about the logging. As we only have to rewrite the backend, and the log crate handles everything else, the actual logging looks non different and “client” code does not change.

+----------------------------+
| imag code, libs, bins, ... |
+----------------------------+
              |
              | calls
              |
              v
+----------------------------+
| crate: "log"               |
+----------------------------+
              |
              | calls
              |
              v
+----------------------------+
| imag logging backend impl. |
+----------------------------+

So what features do we want? First of all, the imag user must be able to configure the logging. Not only with the configuration file but also via environment variables and of course command line parameters. The former will be overridden by the latter, respectively. This gives the user nice control, as she can configure imag to log to stderr with only warnings being logged but when calling a script of imag commands or calling imag directly from the command line, these settings can be temporarily (for the script or one command) be overridden.

The configuration options I have in mind are best described by an example:

# The logging section of the configuration
[logging]

# the default logging level
# Valid values are "trace", "debug", "info", "warn", "error"
level = "debug"

# the destinations of the logging output.
# "-" is for stderr, multiple destinations are possible
default-destinations = [ "-", "/tmp/imag.log" ]

# The format for the logging output
#
# The format supports variables which are inserted for each logging call:
#
#  "%no%"       - The of the logging call
#  "%thread%"   - The thread id from the thread calling the log
#  "%level%"    - The logging level
#  "%module%"   - The module name
#  "%file%"     - The file path where the logging call appeared
#  "%line%"     - The line No of the logging call
#  "%message%"" - The logging message
#
# Functions can be applied to the variables to change the color of
# the substitutions.
#
# A format _must_ contain "%message%, else imag fails because no logging should
# be forbidden
#
[logging.formats]
trace = "cyan([imag][%no%][%thread%][%level%][%module%][%file%][%line%]): %message%"
debug = "cyan([imag][%no%][%thread%][%level%][%module%][%file%][%line%]): %message%"
info  = "[imag]: %message%"
warn  = "red([imag]:) %message%"
error = "red(blinking([imag][uppercase(%level%)]): %message%)"

# Example entry for one imag module
# If a module is not configured or keys are missing
# the default values from above are applied
[logging.modules.libimagstore]
enabled = true
level = "trace"
destinations = [ "-" ]
# A format is only globally configurable, not per-module

One of the most complex things in here would be the format parsing, as variable expansion and functions to apply are some kind of DSL I have to implement. I hope I can do this – maybe there's even a crate for helping me with this? Maybe the shellexpand library will do?

These things and configuration options give the user great power over the logging.

The approach

Because imag already logs a lot, I think about an approach where one thread is used for the actual logging. Because each logging call involves a lot of complexity, I want to move that to a dedicated thread where other threads speak to the logging thread via a MPSC queue.

Of course, this should be opt-in.

The idea is that the logging starts a thread upon construction (which is really early in the imag process, nearly one of the first operations done). This happens when the Runtime object is build and hence no “client code” has to be changed, all changes remain in libimagrt.

This thread is bound to the Runtime object, logging calls (via the logging backend which is implemented for the log crate) talk to it via a channel. The thread then does the heavy lifting. Of course, configuration can be aggregated on construction of the logging thread.

The logging thread is killed when the Runtime object is dropped (one of the last operations in each imag process). Of course, the queue has to be emptied before the logging is closed.

I am also thinking about converting the code base to use the slog crate, which offers structured logging. But I'm not yet sure whether we will benefit from that, because I don't know whether we would need to pass a state-object around. If that is the case, I cannot do this as this would introduce a lot of complexity which I don't want to have. If no such object needs to be passed around, I still have to evaluate whether the slog crate is a nice idea and of course this would also increase the number of (complex) dependencies by one... and I'm not sure whether the benefits outrule the inconveniences.

tags: #linux #open source #programming #rust #software #tools #imag

Today I released version 0.3.0 of my imag project, which contains over 30 sub-crates in the project repository.

This was pain in the ass (compared to how awesome the rust tooling is normally). Here I'll explain why, hopefully triggering someone to make this whole process more enjoyable.

There is no cargo publish --all

Yep, you've read that right. I had to manually cargo publish each crate individually. This is even worse as it sounds! One might expect that, the hacker I am, I wrote a short script a la each crate do cargo publish – but that's not possible – because they depend on each other. I had to find the “lowest” in the stack and build from there.

And as cargo-graph does not yet support workspace projects, I could not even find out which one was the “lowest” crate in the stack (but I know my project, so that was not that much of an issue, actually).

Still, I had to run cargo publish on each crate by hand.

Dependency-specs have to be re-written

As I depend on my own crates by path, I had to re-write all dependency specifications in my Cargo.toml files from something like

[dependencies.libimagstore]
path = "../libimagstore"

to something like

libimagstore = "0.3.0"

which is easily doable with vim and some macros, but still inconvenient.

How it should be

How I'd like it to be: The cargo publish command should have a --all flag, which:

1. verifies that all metadata is present (before building the code)
2. checks all path = "..." dependencies and whether they are there
3. expects the user to provide some data on how to re-resolve the path dependencies, so dependencies are fetched from crates.io after publishing

and then automatically finds the lowest crate in the stack and starts building everything from there up to the most top-level crates and publishes them all in one go and only if all verification and building succeeded.

tags: #rust

I will fly to iceland tomorrow. I'm really excited about this and I really hope I will have the opportunity to take great pictures while there.

But vacation also means no activity on my open source projects. This post is mainly for your notice that there won't be any updates on imag or other projects in the next weeks, until I'm back.

I guess I have to set a photo gallery up when I'm back, so I can share the pictures I take in iceland.

Have a nice summer, you all!

tags: #blog #life

A major problem with open source projects is, that, most of the time, it is not clearly visible in which state the project is. At least not for a casual user of the software, for example a guy like me wanting to try the software.

A version number can be a way to express a certain stability, especially if the project uses semver, though even a 1.0 in semver does not express the state a project is in, only that you won't see breaking changes between (minor) version upgrades.

This is my attempt for defining a scale which can be used to define the maturity of a open source project.

The Levels

In the following, I'm defining Levels for the stability of an open source project.

I define them based on my experiences and views. This means that you might disagree with their definition or maybe even with the need of such a scale. This is fine. I wrote this article because I think we need a discussion about this and I'm more than happy to hear your opinion! Maybe though, we should not do this in a closed manner but on a platform where everyone can participate, for example on a community board like reddit.

Level 0

Computer Scientists start with zero. Projects also start at zero. So am I.

Level 0 is the fundamental idea of a software. You think you, your companion or maybe the whole world needs a tool for a thing. You did not think about this idea that hard, you have no plan how to implement it or even thought about which language to use.

Then your project is in Level 0.

Level 1

You wrote some code. That's awesome! But anyways, the code might not even compile / the interpreter might complain about invalid syntax. The code is just a basic concepts – an expression of thoughts in a textual manner.

Maybe it does compile / the syntax is valid, but still the tool does not yet do what you're planning to do.

Level 2

Wow, now the tool works. At least for you. You know how to build and use the tool.

This is fine. You might be the only developer by now and you don't care that much about other developers, because you're still playing around. You reached Level 2.

Level 3

If you did not share your code with others you might want to do that now, in Level 3. Because now, your code works for you and maybe even your neighbor programmer who thinks you did something cool and wants to try it out.

Still, your code is hacky, you might not even have documentation how to use it or how to build it, but you're confident that you could at least show the world that you're working on something which does something.

(I would say this is the level where most small projects on github are when they're published).

Level 4

Now you care about contributions. You include a basic guide how you did your build, how others might be able to reproduce the build and get a working software package.

You might respond to issues and pull requests and you start interacting with other people when they ask questions. Maybe you even posted your tool on reddit, hackernews or another site of your choice.

Level 5

Community is important. You try to help everybody getting their first contribution merged. Your tool has grown a lot since Level 0 (or Level 3). You think about coding style, development flow and all these things which have to do with your project, but not with solving the problem you're actually working on.

Welcome to Level 5!

This also means that at least some people with a technical background are now able to understand what you're doing and why you're doing it. They might even be able to build your tool to try it out. Maybe even on another platform than you initially started developing.

Level 6

The community around your project is more than just your friend or fellow student hacking on the thing, but people from the internet start watching your project and from time to time, an anonymous person opens a pull request to help you implementing a feature or bugfix.

This is awesome and it really feels good getting other people involved in your codebase!

Level 7

A major event in your project is the first issue/bug report/change request/feature request from a person who is not involved in development of the tool itself and does not even want to – but still wants to use or at least try the tool.

This is when you entered Level 7 – you should now start thinking about how to talk to users of your software!

Level 8

You have a small but dedicated group of users. That means that there are people on this planet using your software. And these people might not even know the programming language you're programming your tool in!

You have people answering questions on IRC which are also developers of your software and you don't have to answer them yourself.

Welcome to Level 8.

Now you have a certain responsibility to these users to keep your software running, because they are interested in using your software!

Level 9

You commit to stability, good documentation and bug fixes as soon as possible.

This is because your community has grown. There are people on your IRC asking questions you can't remember whether you've seen them before in the IRC. You're maybe even getting mails about how to use your software.

Everyone in your community knows that this is still a hobby for you, so you might be unresponsive for a certain amount of time, but anyways they are hoping to get bugs fixed, functionality implemented and questions answered - better sooner than later.

At this point, the development of your software is not the main point anymore. The users are. The support truly is. Development slowed down a lot – not only for but also because your commitment to API stability.

Level 10

At this point, your software (hopefully) is not in pre-1.0 version anymore. This is because you have a really large user base by now. You have IRC channels, maybe a forum, a message board, maybe even a subreddit, google groups or facebook groups for supporting users.

Other developers are heavily involved in your software. There are people around who can do magic with your software. Maybe even people who help developing the software, but never contributed a single line of code because they only contributed translation, style and design, documentation or tutorials and guides. These things might even pop up in the internet without you noticing.

Welcome to Level 10 – where our journey ends. Your software is now used by a lot people and hopefully your bus factor has grown to at least 2.

Where the journey ends

After writing the 10 Levels of open source project maturity, I could rewrite the title of this article to “Open source project community size definitions” or something like that. But I won't do that, because it would not express what I was thinking about before writing these words down.

Overall, one might disagree strongly with these points. As said above, this is okay. The scale might be extended as well. Maybe there are Level 11, Level 12 and Level 42 as well? I don't know, I did not think further than Level 10.

Examples for the Levels

The levels from above do not have any examples stated. That's because I want the reader to make their own thoughts about this.

Either way, I'm providing some examples that I'd define for these Levels. I hope they match the readers view:

• Level 0 – No example necessary I guess.
• Level 1 – No example necessary I guess.
• Level 2 – No example necessary I guess.
• Level 3 – No example necessary I guess.
• Level 4 – Maybe my tool nixos-scripts is a good example for this Level.
• Level 5 – This is where I actually think my own project is: imag. Besides that I'd include khal here.
• Level 6 – I guess bjoern is a nice example for this.
• Level 7 – I failed finding a good example for this. Feel free to suggest one.
• Level 8 – tig might be a good example here
• Level 9 – Tools like dwm, surf and other tools from the [suckless] community would be a nice example here, I guess.
• Level 10 – This is where tools like git, libreoffice, KDE and other “big” pieces of software should be placed.

If anyone feels offended because I categorized their tool in a level which is inaccurate for their tool, feel free to send me an email and I'll remove the link to your project.

You always can argue about “size” of a community as well, so these points are all from my perspective and might not be that accurate from your point of view.

The end

Thanks for joining me on this journey. I hope we can have an open discussion about this.

As stated above, discussion these things via email might not be beneficial, as discussion is not a two-way communication in my opinion, but a multi-directional one.

So, if you care at all or maybe even want to write about these things, feel free to submit a link to your blog post about this and I'll link them here - maybe we can have a discussion on reddit or somewhere, but I'm not sure where to post a thread for this topic.

tags: #open source #software

The imag-pim.org website just got a new face.

I was really eager to do this becaues the old style was... not that optimal (I'm not a web dev, let alone a web designer).

Because the site is now generated using hugo, I also copied the “What's coming up in imag” blog posts over there (I keeping the old ones in this blog for not breaking any links). New articles will be published on the imag-pim.org website.

This very blog article will be published on both sites, of course.

tags: #linux #open source #rust #software #tools #imag

Right now, github shows you this:

And these things will happen more frequently in future, I assure you!

In the first half of 2017, we already had 3 major service outages, 3 minor service outages and 21 other problems. Yes, indeed, this is very good service quality. It really is. Anyways, it is not optimal. Github advertises itself with 22M developers, 59M repositories and 117k businesses world-wide, which is a freakin' lot.

I really like github, it is a great platform for the open-source community and individual projects and developers.

But. There's a but.

Github will not scale endlessly. It will vanish at some point. Maybe not in 2 years, maybe not in 5 or even 10 years. But at some point, a competitor will step up and get bigger and better than github. Or it will be taken down by someone. Who knows.

But when that happens we, as a community, have to be prepared. And that's why we need distributed issue tracking like we implemented with git-dit.

Yes, it is unfortunate that git-dit itself is hosted on github. And we do not have the issues from github in the repository, yet, as there is no mapper available. But we will get there.

I won't go into detail how git-dit works here, there's a talk from GPN17 on youtube about that (in german), where you can learn about these things.

With git-dit, we won't be tied to github and if github vanishes from today to tomorrow, we would be able to continue development seamlessly because the issues are mirrored into the repository itself. In fact, we won't even need github in the first place, because the repository itself would contain everything which is needed for development.

But we are not there yet.

If you're feeling brave, you're more than welcome to try out git-dit or contribute to the codebase.

tags: #git #github #open-source #software #tools

This is the 25th iteration on what happened in the last four weeks in the imag project, the text based personal information management suite for the commandline.

imag is a personal information management suite for the commandline. Its target audience are commandline- and power-users. It does not reimplement personal information management (PIM) aspects, but re-uses existing tools and standards to be an addition to an existing workflow, so one does not have to learn a new tool before beeing productive again. Some simple PIM aspects are implemented as imag modules, though. It gives the user the power to connect data from different existing tools and add meta-information to these connections, so one can do data-mining on PIM data.

What happenend?

Luckily I can write this iteration on imag. After we had no blog post about the progress in imag in April this year, due to no time on my side, I'm not very lucky to be able to report: We had progress in the last 4 (8) weeks!

Lets have a look at the merged PRs (I'm now starting to link to git.imag-pim.org here):

• #915 merged a libruby dependency for travis.
• #918 removed some compiler warnings.
• #917 merged some travis enhancements/fixes.
• #916 superceeded PR #898, which simplified the implementation of the FoldResult extension.
• #895 started a re-do of the ruby build setup.
• #911 changed the interface of the StoreId::exists() function to return a Result now.
• #904 added initial support for annotations in the libimagentrylink library, which gives us the posibility to add annotations to links. There are no tests yet and also no remove functionality.
• #921 was a cleanup PR for #911 which broke master unexpectedly.
• #914 fixed a compiler warning.
• #929 removed libimagruby entirely because we couldn't merge to master because a dependency on master started to fail. The whole ruby thing is a complete mess right now, dependencies are not found, tests fail because of this... it is a mess.
• #927 removed unused imports.
• #924 updated links in the readme file.
• #926 added tests for the StoreId type.
• #919 merged preparings for the 0.3.0 release, which is overdue for one month right now, because the ruby scripting interface does not work.
• #930 updated the toml-rs dependency to 0.4, which gives us even more superpowers.
• #932 added some tests for the configuration parsing functionality.
• #933 Adds a new dependency: is-match, a library I extracted from the imag source code into a new crate.

The libimagruby mess

Well, this is unfortunate.

libimagruby should be ready for one month by now and usable – and it is (the basic things, few things tested also)! But as the CI does not work (fuck you travis!) I cannot merge it. I also don't know how to properly package a Ruby gem, so there's that.

I really hope @malept can help me.

I'm already thinking about adding another scripting interface, so I can continue and start implementing frontends for imag, for example I'm thinking about a lua or ketos interface, still. Lua might be the better idea, as there are libraries around for certain things, while there are no libraries for ketos (I assume).

What will happen

I honestly don't know. I will continue working on imag, of course, but right now, the libimagruby is stalled. I'm not sure where to start working besides libimagruby – a Ruby scripting interface is what I need right now, but it won't work ... so there's that.

As soon as the Ruby interface is ready, we can have nice things. But right now, it is really hard to continue.

tags: #linux #open source #programming #rust #software #tools #imag

When working with Rust on NixOS, one always had the problem that the stable compiler was updated very sparsely. One time I had to wait six weeks after the rustc release until I finally got the compiler update for NixOS.

This is kind of annoying, especially because rustc is a fast-moving target and each new compiler release brings more awesomeness included. As an enthusiastic Rust programmer, I really want to be able to use the latest stable compiler all the time. I don't want to wait several days or even weeks until I can use it.

Use the overlay, Luke!

Finally, with NixOS 17.03 and unstable as of now, we have overlays for nixpkgs. This means that one can “inject” custom nix expressions into the nixpkgs set.

Meet the full awesomeness of nixpkgs!

Soon after overlays were introduced, the Mozilla nixpkgs overlay was announced on the nixos-dev mailinglist.

Now we can install rustc in stable, beta and nightly directly with pre-build binaries from Mozilla. So we do not need to compile rustc nightly from source on our local machines if we want the latest great rust feature.

This is so awesome. Everybody should use the rust overlay now, in my opinion. It's really convenient to use and gives you more flexability with your rust installation on NixOS. So why not?

tags: #nixos #nix #rust #software