As imag gets more traction, I get asked more often “What is its purpose?” or “What does it do?” – here's an explanation!

After I posted my 8th iteration on “What's coming up in imag” on reddit I noticed a huge increase of the number of stars of the project. It went from less than 20 stars up to over 50 within one week. Amazing!

But I also got questions: What is this “imag” thing you are writing – what does it do? So here's an attempt to explain.

A CLI PIM suite?

The README file in the imag source repository states that imag is a personal information management suite for the commandline. Its goal is

Create a fast, reliable, forwards/backwards compatible commandline personal information management suite which covers all aspects of personal information management, consists of reusable parts and integrates well with known commandline tools.

Which is exactly what it is.

Yes, but...

imag is a CLI PIM suite but tries to implement as little functionality as possible – how does it do that? Well, we try to re-use existing tools. We do not implement the PIM tools ourselves, but we re-use known tools, so a user does not have to learn a new tool when switching to imag – she can simply add imag to her setup!

(Bad visualization skills ahead!)

As the picture above shows, all these fancy tools have some sort of database where they store their data. This is awesome and works – what it doesn't provide is to connect this data and add annotations to the connections – that's where imag steps in:

As the picture above shows, imag tracks the data other tools create (it does not duplicate it). It provides ways to link this data and therefor creating a network of data you can query later.

Ah, and what does work by now?

Not that much, as it is an really early stage right now. Some of the modules are already implemented, namingly the

• counter module
• notes module
• diary module

as well as some utility modules for more plumbing-like commands:

• imag-link for linking entries together
• imag-tag for adding, removing and altering tags of an entry
• imag-view for viewing entries. It only supports printing to commandline as of this writing, but will support more in the future, including showing dependency graphs of entries, compiling markdown and showing it in the browser and so on.
• imag-store A commandline-interface to the store itself. This shouldn't be used by the end-user and is for debugging purposes and development work. You can destroy data with this tool rather easily, so if you want to play around, make sure you have a backup of your data!

All tools mentioned above are implemented without using third-party software, which means that there are no tools other than imag which need to be installed to use the advertised functionality (of course imag itself depends on libraries). Future work will be based on other software, so the todo module will be based on taskwarrior, the ledger module will be based on beancount and so on.

The plan

Currently, I'm having in mind to add these tools to the imag suite:

• Todo module: taskwarrior
• Ledger module: beancount
• lend-tracker module: [no backend tool found yet] – I guess I will implement this myself as I cannot think of any complex functionality here.
• bibliography module: [no backend tool found yet] – I doubt that there are commandline tools available for organizing bibtex files. You are welcome to proof me wrong!
• bookmark module: [self-implementation] – I will implement bookmark functionality myself if I do not find a nice commandline tool which fits my needs.
• shopping list module: [not decided yet] – I want to create a module which helps me doing my shopping lists. I'm not sure whether there is a CLI tool for this available – I doubt it actually.
• calendar module: [self-implementation] – I will implement this functionality myself using the icalendar data format if I do not find some nice tool. Maybe khal is good enough, but I'm not sure it offers an API.
• contact module: [self-implementation] – Same as for the calendar module: I will implement this myself using the vcard data format if I cannot find a nice commandline tool. Maybe khard fits my needs, but I'm also pretty sure that this one does not offer an API either
• cue cards module: anki (not sure here, as anki is not a commandline tool or does not have a CLI interface (yet?))
• mail module: [self-implementation] – I'm pretty sure I have to implement this myself, because beeing able to reference emails means that I have to be able to read each of the mails anyways. Maybe this will be a complex module offering functionality to open mutt for mails/mailboxes and query sup or notmuch for more information about mails... - The possibilities are rather endless in this case and I think this will be the most complex tool in the imag suite.
• image module: [self-implementation] – The image module needs to be able to read the meta information of images and give me the possibility to query them for meta-information. I guess I can implement this myself.
• movie module: [self-implementation] – Here goes the same as for the image module – a module to query meta-information of movie files.
• music module: [self-implementation] – and another one equal to the image module – a module to query meta-information of music files.
• news module: newsbeuter – I'm a bit insecure because newsbeuter does not offer an API and uses a sqlite database as data storage. I would need to read this database and try to get my information from there. This is do-able in rust, but I'm not sure whether I like it or not. There should be a non-curses CLI interface for newsbeuter, really! This not only includes RSS and ATOM, but maybe also usenet and podcasts.
• password module: password-store – This is a home-run, as I won't do much besides creating pointers inside the store to each password file – so absolutely no complexity here!
• weather module: [not decided yet] – I'm not sure what commandline tools are out there that fit my needs here. For starters, I just want to be able to put weather information into the store and refer to it from other files. I've not read into the topic at all by now, so I won't promise things here.
• weight tracker module: [no backend tool found yet] – Are there commandline tools for weight tracking available? I would love to have one!
• workout module: [not decided yet] – This just came up while writing this blog post. Pretty much the same as for the weight tracker module... this could even include the weight tracker module!
• wiki module: [no backend tool found yet/not decided yet] – There are some wiki CLI tools available, but I'm not that happy with either of them. I would use vim-wiki personally, but it is rather hard to learn all the commands and I'm pretty sure I would screw up pretty bad when starting to use it actively. I wouldn't include vim-wiki in imag either, because I want to be able to be editor-agnostic.
• project management module: This is nothing more than a brain-fart by now, but maybe I can add a module to do issue tracking and project notes and so on. This shouldn't replace a bug tracker or something, just a personal-project-helper tool or something like that.

Infrastructure needed

I already wrote a lot of infrastructure libraries for imag, including but not limited to functionality for listing, viewing, tagging, linking filtering entries.

There are also libraries for user interaction, selecting entries on the commandline, markdown parsing or time specifying on the commandline to enhance the user experience.

Some more libraries for error handling, store hooks and runtime functionality are in the imag codebase for simplifying the process of writing imag modules.

And, of course, there is the very heart of imag, the store library. This is the library which ties everything together and the univeral interface to the filesystem for all the functionality in imag.

There is also a library in progress for referencing files which do not live in the imag store which can be used to “link” files into the imag store by creating pointers to them. It doesn't use symlinks, as symlinks can be killed by moving the original file. It also doesn't use hardlinks, as the user would have to remove two files instead of one when she removes data from her system – and this is clearly not feasible. So this library simply implements some kind of pointer to files. It also hashes the content of files to be able to re-find a file if it is moved (as long as the content does not change, of course).

The most important part which is almost not covered yet is testing. There is so much code to test.

My wishes for the future.

I have a varity of hopes and wishes for the future. The most important one is

Contributors!

I really really hope for contributors. I mean, I really hope some people start contributing. imag has grown to be my never-ending-project, as there are so many things to do, but I also think such a tool was not created before – so there is a lot of potential to discover and things to discuss or try out!

I think, though, I have layed out the groundwork on this. I created a list of issues in the github repository which are marked with the complexity/easy label so there's a list of easy-to-do issues which a contributor could check and try out.

I also write a two-weekly blog series on what is going on in the project (1, 2, 3, 4, 5, 6, 7, 8, 9) – I also post these on reddit in the /r/rust subreddit – so I really try to spread the word.

Flexible backends

This is another big thing. I wrote down a list of tools I want to integrate into imag above. All functionalities should re-use existing tools to cover there needs, I do not want to reimplement things others have implement very well before.

But, maybe someone does not like my choises and wants to use some other tool than I have used to integrate into imag. Or the maintainer of the external tool stops working on the tool – what then?

I want to create an abstraction layer over the backends, so one can simply switch from (for example) taskwarrior to imaginary-other-todo-tool.

That would be a really complex library, as it has to be completely tool and use-case agnostic. But I think it is do-able, just not yet, as we do not have the experience of binding several tools to imag.

Moving away from github (actually not...)

I'm also tempted to move away from github. The only thing that holds me there is the urge to find contributors and github is the easiest way to do so in my opinion. Also, if I move away from github, I really want to use text-only tools for developing imag – just as in the spirit of imag itself. The problem is, that there is no plain-text bug tracking software available that fits my needs (I wrote about that before). I could use a mailing list and ask contributors to send patches by mail. But I don't like that approach that much.

Also, some of the conveniences that github offers are rather nice. If I would do the move, I would setup a website, CI system, issue tracker, git repositories including a web-frontend (gitweb or similar, not a full-blown gogs or something), wiki, mailinglist, ... and import the old data from github, of course.

But as said, the tooling is not available for doing bug tracking in plain text and distributed, so I won't move.

To end this wall of text...

I didn't expect this article to get this long. But now it is. I hope I could explain what the purpose of imag is.

Feel free to ask me questions (via mail, you will find it) or suggest improvements to this article.

tags: #linux #open source #programming #rust #software #tools #imag